10 Critical Validation Parameters For Microbiological Experiments

In studying a microbiological method, different validation parameters require assessment. These are variables or factors that can be controlled, changed, or measured in the experiment. This article outlines the key parameters to consider.

In reviewing these, it is important to note that not every parameter will be appropriate, either due to the nature of the method (qualitative or quantitative)¹ or due to its capabilities (a modern, rapid microbiological method, for example, will have been designed to meet more parameters than an older cultural method). In addition, several of the categories have been revisited by microbiologists in light of the emergence of rapid and alternative microbiological methods.² However, considering what is presented below ensures that a logical thought process is applied to the executed study.

Outlined below are those elements of the experiment that are both useful, and often critical, when evaluating the performance, status, or condition of a microbiological method.³ Identifying the model parameters requires good study design and a clear pre-planned protocol. In approaching these, it is important to bear in mind that mathematical models do not always reflect the real microbiological process, especially when assessing microbial growth and the variables that influence the growth of microorganisms in culture.

Specificity

Specificity is the capability of the method to resolve or measure a range of microorganisms in the presence of other compounds or microorganisms. This may be a single microorganism (such as a test for coliforms) or a range (such as a general bioburden test). Freedom from interference from excipients or active pharmaceutical ingredients, degradation products, or impurities must be noted as part of a recovery (accuracy) study. When selecting an appropriate culture medium is part of the study, the properties of the medium against selective, non-selective, and mixed cultures must also to be considered.

The microbial challenge should be set above the limit of detection or quantification, while also being at a level that provides a measure of the efficacy of the method. Here:

• For a growth-based method, a low number of <100 CFU (colony forming units) is appropriate. All challenge microorganisms should be recovered. Where atypical colony morphology is observed, supporting identification should be considered.
• For a non-growth-based method, suitable positive and negative controls should be used to show that any extraneous matter does not interfere with the detection of the microorganisms.

Appropriateness

A new method or test must demonstrate that it is appropriate for its intended use. If a method or test is intended to replace an established method, parallel testing of both methods must take place and the collected data compared, if possible, ideally by statistical tests of significance. In some cases, a direct comparison is not possible (for example, two different models of particle counters cannot be directly compared because they will not be sampling the same volume of air).

Accuracy

Accuracy is the closeness of agreement between the measured value and the “true” or expected measure or reaction across the range of the test. This can be assessed by determining the recovery of known quantities of a microorganism that has been added to a sample.

For quantitative tests, this is predicted from:

• The dilution of a microbial suspension or,
• By examining for presence/absence or,
• From a taxonomic identification or,
• By comparing the new method to an established test method (here the new method must give equivalent or better results than the established method).

For enumeration methods, the level of recovery should reflect the test method. This is normally determined by the percentage of microorganisms recovered by the method.

1. If “good” recovery is considered to be achievable, then a recovery level of 50% should be the minimum (this can also be expressed as a productivity ratio). Where an upper level of recovery is required, this is normally set at 200% (with a range of 50 to 200% quoted).
2. When comparing two methods, 70% is sometimes set, although other acceptance criteria may be appropriate.

Comparison of accuracy can be further examined by significance testing, such as Student’s t-test or an alternative method. For example, accuracy can be expressed as:

Accuracy % = (Number of Correct Results in Agreement/Total Number of Results) x 100

For qualitative methods, it is recognized that many hundreds of comparisons may be required if a negative result is the expected outcome, such as with the sterility test. A limitation must be established in relation to the number of samples, because testing samples until a positive result is obtained will be impractical. When comparing two methods, the relative rates of positive and negative results should be compared.

Precision

Precision is the closeness of agreement between a series of test results or the variation in a series of test results, when a method is applied repeatedly to multiple samples.

Precision can be subdivided into:

1. Repeatability (within test variation)

This is the variation in results obtained on the same sample when assayed repeatedly with one test or within a short period of time, by the same technician using the same reagents and equipment.

The key to acceptability is the amount of variation. It may be expressed as:

1. Standard deviation
2. Coefficient of variation (relative standard deviation)
3. Confidence interval of the mean
4. With specific tests, such as microbial identification systems, other criteria will be used to determine the similarity of the recovered organisms.
5. Other statistical techniques, like Chi-squared, maybe more appropriate (Sutton, 2005).

Normally at least three replicates are required. Depending on the type of sample, more than one determination may be required (for example, different dilutions or a range of microorganisms). Because the testing technician and consumables are the same, this approach shows variation in sample as assessed against the method.

2. Intermediate Precision

This is the variation in results obtained on the sample when assayed on several separate occasions by different technicians using different reagents and equipment, etc. This shows reproducibility and may be expressed as:

1. Standard deviation
2. Coefficient of variation (relative standard deviation)
3. Confidence interval of the mean
4. With specific tests, such as microbial identification systems, other criteria will be used to determine the similarity of the recovered organisms.
5. Other statistical techniques, like Chi-squared, maybe more appropriate.

A minimum of three determinations should be carried out. Appropriate acceptance criteria (such as ≥95%) should be set for repeatability. This approach shows variation across people and reagents, and variability within the method.

Range

Range is the interval between the upper and lower levels of microbial count, for which the procedure has been established as suitable with accuracy, linearity (if appropriate, where there is requirement to construct a curve), and precision.

For example, the commonly used range for microorganism recovery is less than 100 CFU for total count techniques. Where a range is required to assess the range of the test, this is covered by diluting a microbial population to, for example, 10⁰ to 10⁶ cells.  With some methods, a regression analysis can be considered to compare two methods.

Robustness and Ruggedness

Robustness is the reliability of a method or test to withstand small (but deliberate) variations due to external influence. These can include different technicians, instruments, incubation times, ambient temperatures, and reagents. With rapid methods, robustness can be undertaken by the method supplier. However, once in the laboratory, long-term performance must be considered and, if possible, checked by internal control samples. For example, the distribution of microorganisms on a membrane can affect robustness.

Ruggedness is the degree of reproducibility by testing samples using different testers and equipment; this is assessed by coefficient of variation.

Limit of Detection

This is the lowest number of microorganisms that can be detected but not necessarily quantified (such as a low-level challenge) under the stated experimental conditions. This test is generally used for rapid and alternative methods.

Often, the amount of sample tested of the initial dilution of the sample and any subsequent dilution of the sample may determine the limit of detection. The challenge microorganisms selected should be of an appropriate range, as indicated above. The challenge can consist of taking each microorganism and making a serial dilution range.

The outcome can be expressed as presence/absence or enumeration. Presence/absence is normally qualitative. An attempt can be made for a semi-quantitative analysis by varying the microbial challenge (to develop a limit test, i.e., to detect <100 CFU, <10 CFU, etc.).

It is recognized that the act of dilution may result in a greater loss due to lack of homogeneity and the typical Poisson distribution of microorganisms in liquid, such as in the evaluation of a raw material by using the pour plate method and where the limit of detection is <10 cfu/g. This is further complicated by the impossibility of obtaining reliable samples containing a single microorganism. Therefore, microbiological limits of detection must sometimes be considered as theoretical rather than practically demonstrable. For this reason, many pharmacopeial tests require the use of a low-level challenge (<100 CFU), and this is normally considered sufficient. For comparing methods, Chi-squared is often the statistic of choice.

Limit of Determination

This is the lowest level of the sample where the microbial content can be quantitatively determined with defined precision and accuracy. Again, further complication arises because of the impossibility of obtaining reliable samples containing a set number of microorganisms.

Therefore, microbiological limits of determination must sometimes be considered as theoretical rather than practically demonstrable. Many pharmacopeial tests require the use of a low-level challenge (<100 CFU), and this is normally considered sufficient. Limits of quantification are normally determined by three or more replicates across the range.

Linearity

This is the ability to elicit results that are proportional to the concentration of microorganisms within a given range. It is measured by correlation coefficient or a goodness of fit test (such as Chi-squared). This will only be applicable to enumeration methods using an analytical system.

When comparing two methods, non-linearity will occur if one method is superior to the other (in terms of microbial recovery). Here, Spearman’s rank may be appropriate as a statistical tool to use (a non-parametric measure of statistical dependence between two variables).

Predictive Value

For qualitative tests such as the sterility test or growth of selective media, the use of positive or negative predictive values may be appropriate. This can be expressed as a percentage of the observed test results against the total or expected test results.

Summary

Microbiological techniques play a major role in pharmaceuticals and healthcare, in relation to ensuring that medicines are contamination-free and efficacious. The evaluation of products and environments is achieved through established and novel methods. For the methods to be meaningful, they need to be developed and assessed for specific microbiological parameters, as described in this article.

This article has been adapted from chapter 4 of the book Digital Transformation and Regulatory Considerations for Biopharmaceutical and Healthcare Manufacturers, Volume 2, written by Tim Sandle and co-published by PDA and DHI. Copyright 2021. All rights reserved.

References

1. Sandle, T. (2014) Examination of the Order of Incubation for the Recovery of Bacteria and Fungi from Pharmaceutical Cleanrooms, International Journal of Pharmaceutical Compounding, 18 (3): 242 – 247
2. Sandle, T. (2015) FDA Signals a New Approach for Analytical Method Validation, Journal of Validation Technology, 21 (2): 1-5
3. Sutton, S. (2005) Validation of Alternative Microbiology Methods for Product Testing: Quantitative and Qualitative Assays. Pharmaceutical Technology. 29(4): 118-122

About The Author:

Tim Sandle, Ph.D., is a pharmaceutical professional with wide experience in microbiology and quality assurance. He is the author of more than 30 books relating to pharmaceuticals, healthcare, and life sciences, as well as over 170 peer-reviewed papers and some 500 technical articles. Sandle has presented at over 200 events and he currently works at Bio Products Laboratory Ltd. (BPL), and he is a visiting professor at the University of Manchester and University College London, as well as a consultant to the pharmaceutical industry. Visit his microbiology website at https://www.pharmamicroresources.com.