What Is The Link Between Quality Metrics, Data Integrity, And Quality Culture?

By Sue Schniepp, Regulatory Compliance Associates Inc.

Among the current and emerging topics of interest to the pharmaceutical industry, quality metrics, quality culture, and data integrity are of particular concern to both the industry and regulatory authorities. To understand how these three topics relate to each other, it is important to understand some of the history that led up to today’s environment, where quality metrics, quality culture, and data integrity come together in establishing current global regulatory expectations.

FDA’s Growing Interest In Quality Metrics

Let’s start by looking at the quality metrics initiative. Establishing, maintaining, and interpreting meaningful metrics rose to prominence based on an article¹ and a Federal Register notice² that explored the question of what types of metrics should be applied to pharmaceutical operations, giving meaningful insight to their overall quality and compliance.

In 2012 the U.S. Congress passed the Food Drug Administration Safety and Innovation Act (FDASIA), which enhanced the FDA’s capability to proactively react to, prevent, and alleviate drug shortages. This direction is embedded in the language contained in Title VII, Drug Supply Chain, and Title X, Drug Shortages. Specifically, Title VII Section 705 of the Act states that the FDA “shall inspect establishments described in paragraph [1] that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs (referred to in this subsection as ‘drug establishments’) in accordance with a risk-based schedule established by the Secretary.”³

This section also describes the risk factors considered in establishing the inspection schedule. One of the risk factors listed in this section is “Any other criteria deemed necessary and appropriate by the Secretary for purposes of allocating inspection resources.” Section 706 of the same act authorizes the FDA to request certain company information in advance of or in lieu of inspections by stating, “Any records or other information that the Secretary may inspect under this section from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection…”.

Following the adoption of FDASIA, the FDA posted notice in the Federal Register on Feb. 12, 2013,² asking the industry to “assist the Food and Drug Administration in drafting a strategic plan on drug shortages as required by the Food and Drug Administration Safety and Innovation Act…”. This notice asked a series of thought-provoking questions, including “What metrics do manufacturers currently use to monitor production quality?” and “How frequently would such metrics need to be updated to be meaningful?”

Industry reaction to this information was varied. Many trade organizations responded to the questions in the Federal Register. Some prepared white papers, while others held meetings to discuss the issue with their members. The general consensus was that industry needed to seriously engage with the FDA to define which quality metrics would provide information to the agency to support efforts to eliminate drug shortages and to be used in establishing a risk-based approach to inspections.

In July 2015 the FDA posted a draft guidance titled Request for Quality Metrics.⁴ The metrics that were chosen for the 2015 guidance were Lot Acceptance Rate, Product Quality Complaint Rate, Invalidated OOS Rate, and Annual Product Review or Product Quality Review on Time Rate. The guidance also contained three optional metrics intended to measure quality culture: Measuring Senior Management Engagement, CAPA Effectiveness, and Process Capability/Performance. This version of the quality metrics guidance generated 83 comments from industry, with a number of these focusing on the practicality of submitting the requested metrics.

In response to this concern the agency issued the Quality Metrics Technical Conformance Guide: Technical Specifications Document⁵ in June 2016 that explained how companies would be expected to submit the data to the agency. A revised version of the original guideline was issued in November 2016 titled Submission of Quality Metrics Data Guidance for Industry.⁶ This version of the guidance document clearly stated the “…metrics can be also useful to FDA: to help develop compliance and inspection policies and practices…. improve the agency’s ability to predict, and therefore, possibly mitigate future drug shortages; and to encourage the pharmaceutical industry to implement state-of-the-art, innovative quality management systems for pharmaceutical manufacturing.” The FDA also clarified the goal of the metrics program by stating “As described in this guidance, FDA is initiating a voluntary reporting phase of the FDA quality metrics reporting program.”  This current version of the document requires companies to report on only three metrics: Lot Acceptance Rate, Product Quality Complaint Rate, and Invalidated OOS Rate. The optional metrics intended to measure quality culture were removed.

Global Regulators Respond To Rise In Data Integrity Issues

At the same time the issue of quality metrics was being discussed, the issue of data integrity was emerging as a concern. Integrity of the data supporting product quality throughout the product life cycle has always been of concern to global regulatory authorities. Requirements specifying that data records need to be attributable, legible, contemporaneous, original, and accurate (ALCOA) have been included in regulations around the world for a number of decades.

Recently, there has been a resurgence of data integrity problems in the industry, evidenced by the number of citations that reference this issue. Between 2005 and 2016 there were approximately 225 warning letters issued with observations for data integrity. These observations included repeat human error deviations, insufficient training, system failures, inappropriate qualification or configuration of systems, poor procedures or not following procedures, and intentional acts of falsification.

The increase in data integrity observations has prompted the regulatory authorities to address the issue by releasing a series of guidelines that reemphasize the importance of data integrity. The FDA, MHRA, WHO, and PIC/S have all released documents to reeducate the industry on data integrity concepts and expectations.^7-11 In addition to the regulatory guidelines, the Parenteral Drug Association (PDA) released a free document titled Elements of a Code of Conduct for Data Integrity to help address the problem.¹²

Quality Culture — The Key To Achieving Data Integrity

One common theme permeating the recent data integrity documents is that of quality culture. The regulators have linked the reliability of the data to the existence of a quality culture, as exemplified by statements taken directly from the guidances.

• Section 6.3 of the PIC/S guidance Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments states “Management should aim to create a work environment (i.e., quality culture) that is transparent and open, one in which personnel are encouraged to freely communicate failures and mistakes. Organisational reporting structure should permit the information flow between personnel at all levels.”

• The MHRA guidance titled ‘GXP’ Data Integrity Guidance and Definitions discusses organizational culture in the document’s Section 3: The principles of data integrity, by stating “The organisation needs to take responsibility for the systems used and the data they generate. The organisational culture should ensure data is complete, consistent and accurate in all its forms, i.e. paper and electronic.” and “The impact of organisational culture, the behaviour driven by performance indicators, objectives and senior management behaviour on the success of data governance measures should not be underestimated. The data governance policy (or equivalent) should be endorsed at the highest levels of the organisation.”

• The World Health Organization deals with the concept of quality culture in the introduction and the section on Principles in its document titled Guidance on good data and record management practices. The introduction links quality culture to an example of “controls that may require development and strengthening to ensure good data management strategies” by indicating “adoption of a quality culture within the company that encourages personnel to be transparent about failures so that management has an accurate understanding of risks and can then provide the necessary resources to achieve expectations and meet data quality standards” as one of the controls to be implemented in a company to ensure adherence to data integrity expectations.

  This same document goes on to describe quality culture in section 4.7 as: “Management, with the support of the quality unit, should establish and maintain a working environment that minimizes the risk of non-compliant records and erroneous records and data. An essential element of the quality culture is the transparent and open reporting of deviations, errors, omissions and aberrant results at all levels of the organization, irrespective of hierarchy. Steps should be taken to prevent, and to detect and correct weaknesses in systems and procedures that may lead to data errors so as to continually improve the robustness of scientific decision-making within the organization. Senior management should actively discourage any management practices that might reasonably be expected to inhibit the active and complete reporting of such issues, for example, hierarchical constraints and blame cultures.”

Based on the data integrity guidances, it is clear that regulatory authorities consider quality culture an important element in establishing the veracity and integrity of the data being generated by companies that support the products they manufacture. Achieving a quality culture requires management and employees to establish an environment where responsibility, accountability, and reliability are paramount, and to understand the role each person performs in delivering a high-quality product to the customer and sustaining that performance on a continual basis. Management must educate employees and provide the tools and environment where they can perform their functions in an atmosphere that encourages excellence and continuous improvement.

An unhealthy quality culture is easy to identify. People in a poor culture do not understand their job and its importance to the business. They often appear stressed, and they hide their mistakes or blame others for their errors. There is no evidence of teamwork. People work in silos and rarely, if ever, seek input or advice from others.

In contrast, a robust, healthy quality culture can be evidenced by alignment of goals between quality and operations, self-sustained work teams that focus on continual improvement and employees who incorporate quality into their jobs on a daily basis. They are not afraid to speak up and offer suggestions for improvement to their colleagues. People understand the importance of their jobs and respect each other and their management. This culture welcomes inspections and views these inspections as another tool to use in their continual improvement initiatives.

The trouble with quality culture is determining how to measure it. The PDA has developed a culture assessment tool that links organizational attributes to specific behaviors.¹³ The effort started with a membership survey in 2015 that established a relationship between behaviors and attributes. Attributes were defined as elements of a quality system such as, but not limited to, deviations reporting, change control, CAPA, complaints, and environmental monitoring programs or systems. Behaviors were defined as intangibles such as, but not limited to, robust communication and transparency, rewards and recognition, employee engagement, and cross functional vision. The theory was if quality attributes equaled quality behaviors equaled quality culture, then if the quality attributes of a company could be measured, that would reflect the maturity of the quality culture of an organization. The PDA tool involves several steps, which include training employees on the use of the tool, an on-site assessment, an all-staff survey, and, finally, analysis and action on the results. Several companies and regulators from the MHRA and FDA have been trained on PDA’s culture assessment tool.

There are, of course, other tools available to measure the culture of an organization. The real point is whatever tool your company uses to measure its culture will be important in determining your data integrity risks and remediating them before an inspection. Regulators have clearly delineated through the guidance documents that there is a link between the culture of an organization and the veracity of the metrics data being generated. Auditing a company to determine if its culture is conducive to generating data that is attributable, legible, contemporaneous, original, and accurate is on the horizon and may become part of routine audits performed by regulators or industry auditors when evaluating the suitability of a potential partner or service provider.

References:

1. J. Woodcock and M. Wosinska, Clinical Pharmacology & Therapeutics, 93 (2) (February 2013), http://www.nature.com/cpt/
2. Docket No. FDA-2013-N-0124, Food and Drug Administration Drug Shortages Task Force and Strategic Plan; Request for Comments, Federal Register, 78 (29) (February 12, 2013).
3. The Food and Drug Administration Safety and Innovation Act, Pub.L. 112-144, 126 Stat. 933 (2012)
4. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Request for Quality Metrics: Guidance for Industry, July 2015.
5. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER),Quality Metrics Technical Conformance Guide: Technical Specifications Document, June 2016.
6. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Submission of Quality Metrics Data: Guidance for Industry, November 2016
7. MHRA GMP Data Integrity Definitions and Guidance for Industry (March 2015)
8. MHRA: A GxP Data Integrity Definitions and Guidance for Industry Draft version for consultation (July 2016)
9. FDA Data Integrity and Compliance With CGMP Guidance for Industry Draft Guidance (April 2016)
10. WHO: Annex 5, Guidance on good data and record management practices (June 2016)
11. Draft PIC/S Guidance: Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments (August 2016)
12. Parenteral Drug Association (PDA): Elements of a Code of Conduct for Data Integrity (2015 – Free on PDA Website)
13. 3^rd PDA Europe Annual Meeting, Presentation by Susan Schniepp: Assessing and Improving Quality Culture: Tool & Resources, June 27, 2018

About The Author:

Sue Schniepp, Executive VP – Pharma and Distinguished Fellow, has more than 35 years’ experience and service in a leadership role for Regulatory Compliance Associates, Inc. Her background includes cGMPs, audit readiness, change control, investigations, supplier quality, and batch record review. She has served in leadership roles at Allergy Laboratories, Inc.; Oso BioPharmaceuticals LLC; Searle; Abbott; and Hospira. She has served the Parenteral Drug Association (PDA) as a member of the Board of Directors, PDA / FDA Joint Regulatory Affairs Conference Chair, conference presenter, and Chair of PDA’s Regulatory Affairs / Quality Advisory Board. She was also awarded PDA’s Distinguished Service Award in 2008. You can reach her at s.schniepp@rcainc.com.