Pharma's Quest For Consistent Quality

By Ken Congdon

“A maximally efficient, agile, flexible, pharma manufacturing sector that reliably produces high quality drug products with limited regulatory oversight.” Dr. Janet Woodcock, director of the FDA’s Center for Drug Evaluation and Research (CDER)

This visionary statement made by Dr. Woodcock way back in 2005 serves as the unofficial rallying cry behind the International Society for Pharmaceutical Engineering’s (ISPE’s) recent Quality Metrics Initiative. Delivering a high-quality product has always been important to most pharmaceutical manufacturers, and several metrics — including deviation rate, and lots attempted/rejected — are typically evaluated by drug makers to assess overall production excellence. However, the problem is, each company tends to collect data in its own way, using a range of definitions and sampling plans. For example, the way a “batch,” “lot,” or “rejection” are defined and measured can differ depending on the sector or company. Heck, these definitions and measurements can even differ between different sites within the same organization.

Standardization is clearly needed to ensure that metrics reported to the FDA are defined and measured the same way by all. The ISPE has been leading the charge in this regard. The first conversations on quality metrics took place at workshops held during ISPE’s 2nd Annual Current Good Manufacturing Practices (CGMP) Conference in June 2013 and at the ISPE Annual Meeting in November 2013. From those discussions, ISPE developed a whitepaper proposing an initial list of quality metrics linked to six systems inspected by the FDA. The association also launched a pilot program in June 2014 to refine the set of metrics and definitions, data submission process, and evaluation. The preliminary results of this first pilot effort were unveiled at the ISPE Quality Metrics Summit on April 21-22, 2015 in Baltimore. This just so happened to be the first event I attended as the new Chief Editor of Pharmaceutical Online.

Quality Metrics Pilot Overview

Eighteen pharmaceutical companies and 44 individual sites participated in Wave 1 of the Quality Metrics Program. These participants were a mix of generic, innovator, over-the counter, and contract manufacturing organizations that leverage a variety of manufacturing technologies including solid dosage forms, sterile drug products, active pharmaceutical ingredients, and analytical testing laboratories spanning both small and large molecule platforms.

The primary objectives of Wave 1 of the Quality Metrics Pilot were as follows:

• Test, in actual manufacturing settings, the set of quality performance metrics proposed by ISPE based on industry and FDA input to date, and harmonize a set of metric definitions. A complete description of each metric and its associated definitions can be found here.
• Test the feasibility of data collection across different companies
• Assess the reporting burden this data collection effort places on the pharmaceutical sector
• Explore industry practices in quality culture and use of process capability
• Detect any patterns in the data, in particular testing whether certain metrics are predictive
• Provide participating companies with a view of their performance versus peers
• Provide input on the relevance of metric data

Quality Metrics Pilot Reveals Compelling Data Patterns

The guiding principle behind the Quality Metrics Initiative is the idea that if you can’t measure something effectively, you can’t manage it effectively. Wave 1 of the pilot did reveal some compelling, albeit not entirely unexpected, data patterns. These key findings include:

• Trends in critical complaints are consistent with the number of recalls, while total complaint trends are not
• Sites with lower lot acceptance rates have higher critical complaints rates
• Sites with higher deviation rates have higher critical complaints rates
• Higher deviations recurrence is correlated with lower lot acceptance rate
• Sites with U.S. recalls have higher deviations recurrence
• Stronger culture is correlated with fewer action limit excursions in steriles
• Stronger culture scores are associated with higher lot acceptance
• Sites with higher quality culture scores have lower deviations recurrence
• Sites with higher quality culture scores have higher Corrective Actions and Preventative Actions (CAPA) effectiveness
• CAPA effectiveness is correlated with Annual Product Quality Reviews (APQRs) completed on time

Lessons Learned & Next Actions

Wave 1 of the Quality Metrics pilot revealed some interesting data, but it also alerted ISPE to potential problem areas to address in Wave 2 of the initiative. For example, while standardizing metrics definitions across companies and sites was deemed feasible by Wave 1 pilot participants, it was also clear that these definitions must be exact to avoid multiple interpretations. Going forward, more effort will need to be devoted to removing ambiguity from metric definitions to ensure consistent standards and clear expectations are established.

Furthermore, the Wave 1 pilot revealed that few pharmaceutical companies currently aggregate metrics across their supply chain to be able to report on the differences in product quality based on the site in which it is produced. This trend was validated at the ISPE Quality Metrics Summit when Russ Wesdyk, Scientific Coordinator for the FDA’s Office of Pharmaceutical Science, conducted an informal, on-the-spot survey of attendees to identify if companies performed APQRs at a site level or across all sites. Only an estimated 30 percent of the audience claimed to conduct an APQR across all sites. This result caused Wesdyk to challenge attendees saying, “Everyone in your family — your spouse, children, and parents — take the drugs that you make, yet more than 60 percent of you do not aggregate an APQR to understand the quality of your products across the supply chain. Think about that. Is this something you’re comfortable with?” Based on this revelation, I’d expect increased focused to be placed in standardizing this practice in Wave 2 of the pilot as well.

Finally, Wave 1 of pilot showed that the quest for standardized quality metrics won’t be an insignificant endeavor for the pharmaceutical industry from a labor and cost perspective. On average, pilot sites spent 90 hours engrossed in pure data collection and the annual cost of this effort to the industry is conservatively estimated at $35 million.