Much of the testing that goes into load development is aimed at understanding what the variation in pressure, velocity, and other performance characteristics that a round has. In order for the end user to have confidence in the ammunition he or she has to trust that it will do everything it is advertised to do, again and again, without fail. It only takes a surprisingly few defects in order for a consumers confidence to be shaken. Given that bad word travels faster and further then good word, the impact of a few bad rounds can forever taint a products release.
When Remington Launched the R51 in January 2014, Remington had high expectations having invested millions into the modernized pistol design. It was an updated design based off John Pedersen’s original Model 51 carried by historical figures such as General George S. Patton. With the conceal carry market booming, a compact, low bore axis, 9x19mm pistol was certainly in demand. There was no doubt that this pistol design had the potential to be a winner. However by July 2014, Remington suspended production, it took took two years for Remington to “Re-release” the R51. By then the reputation of the gun was destroyed, and it never took off as expected.
Remington filed for bankruptcy in 2018. This is not to say that the failure of the R51 was directly responsible for Remington’s failing as a company. However it certainly was one nail in the coffin of an already struggling company, and was no help to the ill repute among firearm owners for quality issues.
So what happened? Remington indicated that there was an issue transferring the design from prototype to production. Having spoken to many of the engineers who were either directly or indirectly involved, there were multiple problems. The majority of which centered around failing to account for variation in production, and the affect that has on something called “tolerance stack up”.
Natural variations in the parts that were being source were causing reliability issues. Often times it was not one single part, but multiple parts that work together were all too large or all too small but individually within the tolerance allowed. This created a “stacking” effect, which lead to problems with the guns reliability. This lead to almost a complete redesign of the firearm as each individual part had to be “retolerance” to ensure that no matter where an individual part fell in it’s tolerance window that it would not add to a cumulative defect. This is known as “Tolerance Stack up”.
This is the power of variation, small variation, on its own may not be consequential. However in the presence of other parts and pieces it can add to a cumulative effect which can produce undesirable results. This is why it is important to study variation and its effects for a given design, or product. The majority of Load development is centered around understanding variation, and ensuring that natural variation does not lead to defective product, or a potentially dangerous product.
So what is variation and how to do we study it?
For the purposes of this discussion we will define variation as the difference between two or more measurements or observations
There are many different ways we can study Variation and its effects on quality. For load testing and development we are typically looking for “Natural” Variation, or variation that is the result of elements that may be beyond our control, and separating them from “Special Cause” variation which is something we want to control.
For example, when powder burns, it doesn’t always produce the same pressure or yield the same velocity. Minute variation that is inherent in the chemistry of the powder will cause different results, usually these differences are small. This is natural variation, it variation in herit in the process, and it is not something we can control.
When a powder thrower throws a charge, it measures it out volumetrically, and is usually pretty consistent, within .2 or .3gr of the target. However due to a loose screw, it will sometimes throw a charge that is 1.0gr off. This is special cause variation, and we seek to identify its source and control/eliminate this type of variation.
Delineating between Natural Variation and Special Cause Variation is one of the purposes of Load Development. Is the Load performing consistently, or does it spike unexpectedly? If it spikes, why does it spike and can this behavior be eliminated?
A sample is defined as an individual observation or measurement, made by someone conducting a study, test, or inspection.
In order to study variation is becomes necessary to collect samples. Usually the more samples, referred to as “Sample Size” the high confidence we can have in the results of the study. A test that only samples 10 rounds will typically only find variations that occur 1 out of 10 times. Variations that occur once in 1000 times or once in a 1,000,000 times are much harder to discover.
Since testing and sampling can be expensive, and in the ammunition industry, destructive, it is impractical to sample 10,000’s of rounds per lot looking for the rare event. For this reason studying variation rely heavily on statistical methods to look at the potential for defects without needing to actually observe one.
We will get into the details on how we use statistics to evaluate the gathered samples to determine the likelihood of a defect in a future write up. Right now we just want to introduce a few terms that are commonly used, and you will need to be familiar with.
Min or Minimum – This is the smallest measurement or observation in the data set.
For example if we measure the COL of three different rounds to be 1.312, 1.316, 1.311. The minimum is 1.311.
Max or Maximum – This is the opposite of Min, in the above example the Max would be 1.316
Mean or Average – This is the sum of the samples, divided by the number of samples. The above example mean would be 1.313.
Extreme Spread – This is the Maximum sample subtracted by the Minimum sample. The extreme spread of a sample set is a quick way to see how much variation there may be within a sample set. The extreme spread in the above example is .005.
Standard Deviation – To put it simply Standard Deviation is the measurement of variation within a sample. It is sometimes abbreviated as Std. Dev. or by the Greek Letter Sigma “σ ” This is a very important metric, and is what will be used to take the results from a small sample size and infer them to a much larger scale.
A good rule of thumb to know is the 68%-95%-99.7% rule.
This states that ~68% of individual samples will fall within 1 Standard Deviation of the Mean, ~ 95% are within 2 Standard Deviations, and 99.7% within 3 Standard Deviations. In the above example the Standard Deviation is 0.0026.
Rarely will you ever calculate the Standard Deviation of a data set by hand, as it can be cumbersome to do with more than a few samples.
Lucky for us, programs such as Excel makes it easy to take a data set and get all of the information we need.
In the next few write ups we will show you how to use these basic tools to study the variation in your own loads, or products. There are whole books and fields of study dedicated to this. However learning a few simple tools will help you solve that majority of the problems you will come across, and improve the consistency of your loaded ammunition.