FDA And Quality Metrics: Where Do Things (Currently) Stand?
By Robert Matje, PE, CPIP, RemTech LLC
Are you ready to give the FDA more? “What?” you ask, as a manufacturer of API and finished goods. Are you ready to give them more data and information about your manufacturing process? You think, “Don’t they get all the necessary information from me during their inspections and from my filings?” Well, the answer is yes and no. Quality metrics was a topic first introduced by Janet Woodcock, director of the Center for Drug Evaluation and Research (CDER) at the FDA over a decade ago. The program at that time was (and still is) established in line with the FDA’s vision for 21st century manufacturing and product quality (see Pharmaceutical cGMPs for the 21st Century: A Risk-Based Approach), but in a recently published Federal Register Notice (FRN) (Docket No. FDA-2018-N-1903), the FDA states “we have not fully realized our 21st century vision for manufacturing and quality, and indicators of serious product quality defects persist.” Further, there are several reports that show that product quality is a leading cause of drug shortages (Report on the ISPE Drug Shortages Survey, 2013 and related documents). The FDA believes that since “what gets measured gets done,” measuring and reporting proactive quality indicators is the next logical step in improving manufacturing and product quality, and it is right, to a point.
The FDA quality metrics program got its first meaningful shot in the arm when the FDA published its Request for Quality Metrics Draft Guidance in July 2015. The guidance essentially called for industry to report on lot acceptance rate, product quality complaint rate, invalidated out of specification rate, and annual product review (APR) or product quality review (PQR) ‘on time rate,’ with complementing definitions of the data and discussions around how the FDA intends to use the data. There was significant feedback from industry in the responses to the July 2015 draft guidance, fundamentally (among other issues) due to the high burden of capturing these metrics compared to the low value of the data that would be received. In particular, implying that correlation of the metrics mandated by the FDA influenced or was a predictive indicator of the cause of drug shortages was not readily accepted by the pharmaceutical industry. The FDA, to its credit, listened to the many voices of industry and has since reevaluated its position on the matter.
So where does that leave us?
Several initiatives are underway to tackle this difficult and multifaceted issue; manufacturers have a high level of anticipation, and the FDA is anxious about making progress. But first, we should address the perspective of the manufacturing community. A primary concern stems from, among a handful of others, the question of what is going to be done with the metrics that are collected (how will the data be managed, how will confidentiality be maintained, etc.) and how the data will be used (will it be normalized, will it be compared to other operations [akin to a McKinsey POBOS assessment], will it be aggregated with other data for an overall risk-based assessment, etc.). The pharmaceutical industry’s compliance status has long been informed by the small “c” in front of the cGMP, or current good manufacturing practice regulations, and regulatory observations in cases of failure to follow current practice can result in increased costs and negative business impacts. Conversely, the industry should be continually improving its quality, and patient safety should be paramount. More than once I have heard senior individuals in the FDA ask industry Q&A panels why it takes regulatory action for pharmaceutical manufacturers to invest appropriately in their operations. The question is very valid.
With regard to the current initiatives underway, the FDA has really stepped up and taken heed of the industry recommendations since the publication of the July 2015 Draft Guidance, as evidenced by the recently issued FRN. The International Society of Pharmaceutical Engineering (ISPE) has been instrumental in providing both data-based recommendations and pharmaceutical cross-industry (pharma, biotech, OTC, API, etc.) commentary, which has been reflected very well in the recent FRN. ISPE has been advocating for some time the need to thoughtfully proceed with implementing a quality metrics program. Specifically, the guidance was given to the FDA to start small, make the program voluntary initially, and maintain confidentiality of the data, which we plainly see as elements of the current FRN. In addition to the recent FRN detailing the intent to follow industry’s recommendations, the FDA has also issued a second FRN (Docket No. FDA-2018-N-1896) that highlights the desire to perform site visits to facilitate CDER and CBER staff experiential learning. This opportunity gives industry the ability to work with the FDA in defining “the advantages and challenges associated with implementing and managing a robust quality metrics program,” of which there are several.
Pharmaceutical manufacturing operations can be very simple (from a geographic, management, and control perspective) but can also be very complex. A simple manufacturing operation may involve a straightforward noncomplex formulation that includes in-house material receipt, testing, manufacturing, packaging, and, in some cases, distribution. Complex manufacturing operations can have multiple testing, manufacturing, and packaging locations causing significant logistical, supply chain coordination, and management challenges and involving a high degree of competing priorities. And while NDA and ANDA holders desire to subcontract requirements to the greatest extent, separate profit centers within companies, outsourced laboratories, contract manufacturers, contract packagers, and the like are trying to minimize their liability and, of course, make a profit, which can cause competing interests. Collecting metrics across these supply chains can be very difficult and requires careful coordination. Again, credit should be given to the FDA for taking the time to analyze these differing scenarios and going to the field to experience them.
Of course, one cannot discuss quality without emphasizing the cultural element within organizations and linking the results with robust operational excellence and looking at the data in a manner that allows an understanding of risk to the patient. Again, the FDA is taking the right steps here. The FDA has engaged the University of St. Gallen to review the correlation between quality metrics and operational excellence. According to the Pink Sheet (published by Informa, 15 June 2018), St. Gallen has identified 10 St. Gallen maturity attributes, and the study “found that 91 percent of the variability of quality behavior can be explained by the top 10 quality maturity attributes.” Powerful, to say the least. Here also, ISPE has played a leading role in performing a Wave 1 Pilot and then a Wave 2 Pilot that provided database correlations between proposed metrics and outcomes, which have been referenced by the FDA consistently along the quality metrics journey. The Parenteral Drug Association (PDA) has been instrumental in developing a culture assessment audit approach that can be used by companies to define their maturation level within their own organization and begin to develop plans to implement programs to strengthen gaps in their approach to product quality while continuing the practices that drive a strong quality culture. The FDA has been meeting with PDA to discuss this program.
But much like manufacturing a product, a robust quality management system is the sum total of many measures, and these are not all available in one form, can be reduced to a measurable metric, or compared across the industry. More is required. A batch record defines the story of a product manufacturing process, but it is supplemented with laboratory procedures, investigations, and observations, manufacturing procedures, investigations, gemba walk discussions, manufacturing team and management meetings, leader decisions, material management planning, and sales and marketing, among many other aspects indigenous to how the pharmaceutical factory is governed and managed. So, too, the manufacturing and product quality processes have their own stories and are managed through multiple processes including procedures, quality councils, shop floor huddles, deviation management, and the like. While the FDA accesses many of these documents and outcomes during its inspections, it does not access data and deliverables from many of these systems we have in place to govern our manufacturing and product quality. Over time, as we further develop universal definitions, understand linkages between the predictive data and quality outcomes, and understand quality cultural aspects that influence positive outcomes, we can expect that some of the required inputs will be from sources not routinely required by inspectors and perhaps proactively reported as part of an industry developed and FDA-governed reporting system. This may then be validated by the FDA through multiple channels, risk assessed, and then acted on depending on the what the aggregated data and information shows. The FDA has made provision in the recently published FRNs to perform voluntary site visits to help drive further collaboration with and industry and ultimately drive a better quality metrics program.
At any rate, quality metrics remains a key focal area for the FDA (see the Statement from FDA Commissioner Scott Gottlieb, M.D., on FDA’s work to mitigate shortages of intravenous drugs, shorten supply disruptions and better predict vulnerabilities, May 31, 2018). While industry and the FDA are in large part aligned on the overarching objectives of a quality metrics program, there is still a way to go to get to common ground between what can and should be achieved to be able to measure and report data from our manufacturing and product quality operation. It is reassuring to see that a collaborative, listening, and learning path is being established in a responsible and pragmatic manner.
About The Author:
Robert "Bob" Matje, PE, CPIP, is a principal of RemTech LLC and has held increasing levels of responsibility in several branded and generic pharmaceutical companies including general management of two pharmaceutical manufacturing facilities, and recently serving as the vice president of technical operations with Endo Pharmaceuticals. Bob has also lead reliability and maintenance, environmental, health and safety, automation, capital and qualification teams and served as the Serialization Project Management Office (PMO) lead for both Endo and Pfizer. This included responsibility for establishing serialization governance, scope development, schedule, and cost. Bob is a member of the International Society of Pharmaceutical Engineering (ISPE) and has served two terms on ISPE’s International Board of Directors. Bob earned a BS in engineering at Lafayette College and an MS in engineering at Villanova University. He is a Registered Professional Engineer in Pennsylvania and was awarded his Certified Pharmaceutical Industry Professional certification in 2012. You can reach him at rmatje@remtechllc.com or connect with him on LinkedIn.