Microbiology and failure rates

Of principal concern to many growers is the risk of failure in the microbiology test required by the legal marijuana market. Products are required to be tested for certain microbiological contaminants, and values above a certain threshold result in failure of the material to meet the “fit for consumption” criteria. Failures can lead to dramatic loss of revenue and can put a business out of production. Rather than look for ways around the microbial test, we at Confidence look to our data for clues as to why the product has failed, and we work closely with our clients to help them develop plans to avoid more failures in future lots.

When samples are failing, it’s usually caused by a systemic problem in the facility of production or processing. In our database, microbial values are highly correlated with producer name, indicating that different marijuana production and processing facilities have different average levels of contamination and therefore different probabilities of failure. Some growers never fail… they never even come close to failing. Others struggle with it repeatedly.

In our experience, the contamination usually takes place at the time of trimming and so that’s where we tend to start our investigations. When crops are failing, we send our scientists out to grow operations all around the state to work collaboratively with the marijuana manufacturing staff to identify possible sources of contamination. Sometimes it’s easy to find and fix; sometimes it’s not. We’ve observed that by thinking critically about the issue, most producer/processors have been able to make adjustments to their facility or process that lead to reductions in failure rates over time, sometimes drastic reductions.

output_9rDXSg

Failure rates for the microbial test have fallen dramatically over time. There are certainly numerous factors that contribute to the improvement in Pass rates, but probably the biggest contributor is simply that producer/processors have been made aware of their microbial bioburdens and they’ve taken steps to reduce their risk of contamination. That’s Quality Assurance at work.

The first thing that jumps out about these scatter plots is that the distribution of microbial values is very wide. Some flower samples have microbial burdens so high that we can’t even count them, and we just record those values as “999,999” or “too numerous to count.” You can see those samples clustered up at the very top of the scatter plots. Other samples have so few microbes on them that we aren’t able to detect any. That’s not a surprise.  It’s just the nature of the beast.

Because the distribution is so wide compared to the failure threshold, it might look like a lot of the samples are failing. In reality, more than 85% of the samples are below the failure threshold and are a PASS in the State treaceability system. In fact, most of the samples are clustered near the zero mark because most samples are pretty clean.

Visualizing the trend in the scatter plots might take a little imagination at first, but you’ll notice that the left side of the graph has more high values than does the right side. To make this visualization easier, we’ve included a moving 60-day failure rate to give you a rough idea of the trend over time, and we’ve also fit our data to a logistic regression model which uses test date to predict probability of failure and estimates that the failure rate decays by 0.271% of its existing value per day (p-value = 1.35e-46; 95% confidence interval = 0.234 – 0.308). We’ve gone from a failure rate of roughly 12% to a failure rate of roughly 3.5% in just over 15 months, and we predict it’ll get better still as the industry improves its practices.