Purpose: The foundation for any vaccine immunogenicity trial is the measurement of binding and neutralizing antibodies with well-designed bioassays. It is also crucial to recognize that the methods used to quantify assay endpoints (e.g., antibody concentration) require robust statistical approaches. The statistical methods used during method validation can have a significant effect on their accuracy and precision.
This poster discusses a number of statistical methodological issues related to bioassay design and analysis that would ultimately lead to highly accurate assay endpoint determinations with a minimum variance that would be comparable across laboratories. In addition, it also demonstrates how these statistical methodologies can be implemented in a simple and efficient manner using a commercially available software- Aegris™.
Methods: The intended purpose of vaccine immunogenicity trials is to evaluate vaccines is the recognition that one or more biological endpoints exist that may be used as surrogates for immunity. For most vaccine trials, the most accepted endpoints are related to antibody levels expressed in concentration units (e.g., µg/ml) or titers (1/serum dilution) for functional assays.
The key step in the implementation of such methods is the validation of the method to ensure that the most accurate estimate for antibody concentrations can be derived with minimal variability. This requires sophisticated statistical approaches to ensure that the assay is validated for the intended purpose. The statistical approaches include standard curve modeling, parallelism test, trending to identify variability, drift, degradation, etc. We have utilized a robust fitting technique that identifies outliers in an iteratively reweighted least squares algorithm to improve assay accuracy and precision. In addition, a robust statistical analysis should be performed to define following in regards to in-study validation i.e. sample analysis.
• Define assay acceptance criteria based on standard curve and measurement of known quality control samples
• Define assay range of the assay where the antibody concentration measurement is most reliable based on variability exceeding a pre-defined limit
• Define a decision tree approach to on calculating antibody concentrations when samples are run in multiple dilutions
• Verifying parallelism between the standard and sample dilution curves.
We have also developed a statistical approach to compare data between laboratories using series of serum specimens with known or assigned antibody concentrations. The results are then evaluated to determine the extent of agreement or disagreement.
All statistical analysis was performed using web-based software platform Aegyris.
Results: There are a wide variety of mathematical functions and curve-fitting methods are available to model serially diluted standard reference serum. We have evaluated seven different models using an evaluation index calculated from the deviations of the predicted concentrations of the standards from their known values to determine the model that provides the least deviation. While 4 or 5 parameter logistics curve is most widely used in the industry as “default” we demonstrate that no one model will be optimal for all experimental situations and as statistical measure should be used to identify right model on a case by case basis. We have also demonstrated that the selection of the mathematical function used to model standard curves can have significant impact within-assay and between assay variability. In some cases the differences in data processing techniques account for a significant portion of between-assay variability.
It is important to establish parallelism between the serially diluted standard reference serum and serum sample curves to support the assumption that the antibody-binding characteristics are similar enough to allow the determination of antibody levels in the diluted serum sample. We have utilized most commonly used “recovery” method and ANOVA a method. We demonstrate that the ANOVA method is more robust when used in combination with linearizing transformation such as log-log, logit-log etc. We have also utilized ANOVA procedures to test for the equality of one or more parameters in nonlinear functions, such as the four-parameter logistic-log function.
We have utilized box plots of percent error calculations where the percent error measures the degree of error between a laboratory’s calculated value for antibody concentration and the assigned value for the serum to determine if a laboratory is able to estimate the concentrations within an acceptable degree of tolerance.
Conclusion: The successful validation of vaccine immunogenicity assays requires robust statistical approaches. Due to lack of easy-to-use software tools, analytical scientists often revert back to “routine” statistical methods. We demonstrate that the selection of statistical techniques used to calculate antibody levels have a significant impact on their accuracy and precision. While there are numerous software systems available for bioanalytical laboratories, they mostly focus on supporting “routine” analysis with minimal statistical procedures and not suitable for vaccine immunogenicity analysis. Aegrys software can be a valuable tool for bioanalytical scientists looking to develop statistically robust immunogenicity method to support vaccine clinical trials.