Guest Column | January 17, 2018

Cheating In The Lab: 3 Data Integrity Pitfalls To Avoid In Laboratory Operations

By Kip Wolf, X-Vax Technology, @KipWolf

FDA inspectors continue to observe CGMP violations involving data integrity during CGMP inspections of laboratory operations. At least 20 percent of all warning letters issued by the CDER Office of Manufacturing Quality in 2017 included explicit observations by inspectors of blatant data integrity violations in laboratory operations. While there has been an increase in warning letters issued to firms in China and India for flagrant data integrity violations, firms in the United States, Europe, and Japan have also been cited with similar violations.

These observations should give pause to all of us who work in the life sciences industry as business models move toward outsourcing and virtualization. Our industry is facing a serious problem with a lack of understanding and maturity of basic CGMP practices and quality culture related to laboratory operations, as evidenced by these violations and reinforced by the FDA now explicitly citing “CGMP Consultant Recommended” in the majority of the warning letters issued since August 2017. A foundation of basic CGMP understanding is found in good documentation practices (GDocP). Guidance from WHO suggests that “the basic building blocks of good GXP data are to follow GDocP and then to manage risks to the accuracy, completeness, consistency, and reliability of the data throughout their entire period of usefulness – that is, throughout the data life cycle.”1

It is incumbent upon us to enforce and reinforce basic CGMP and GDocP practices in our own laboratories and those of our suppliers through technical quality agreements, contractual obligations, and periodic audit and assessment of laboratory operations. While much can be accomplished through the legal and administrative diligence during contract negotiations, laboratory operations remains an area that must be observed to ensure compliance (i.e., audit and assessment). By auditing for these basic CGMP and GDocP practices, we may improve our probability of avoiding negative impact to product, patients, and the business continuity.

Do Not Delete Or Manipulate Laboratory Data

A basic tenet of data integrity in laboratory operations is expressed in CGMP regulation, namely, that “laboratory records shall include complete data derived from all tests necessary to assure compliance with established specifications and standards”.2 Recent observations from FDA warning letters indicate that this basic tenet is still not being observed.  

  • “Our investigators observed […] deletion of at least six (b)(4) and (b)(4) tests in the audit trails for two instruments used to test sterile (b)(4). Your systems allowed operators to delete files.” And “you identified a total of 25 deleted (b)(4) test results.”3
  • “The audit trail for these tests included the message, ‘deleted result set,’ but neither of these two incidents were recorded in the analytical packages for these batches of drug products, nor were they reviewed or investigated by the quality unit.”4
  • “Your analyst was unable to retrieve requested data, and explained that he deletes older data to make space for newly acquired data.”5

Performing in-person audits of laboratory operations is the most effective method of ensuring compliance in this area. Auditing for basic GDocP is of paramount importance, as this is a fundamental tenet of CGMP.

  • “Our investigators found that you had deleted entire chromatographic sequences and individual injections from your stand-alone computers.”6
  • “You stored original data in an ‘unofficial’ and uncontrolled electronic spreadsheet on a shared computer network drive. Your analyst stated that original data was first recorded in this ‘unofficial’ spreadsheet and transcribed later to an ‘official’ form. This spreadsheet showed failing results above the limits you established in your procedure, PCH 035 Visible Particle Determination in use prior to September 1, 2014.”7
  • “Our investigator observed many copies of uncontrolled blank and partially-completed CGMP forms (e.g., environmental monitoring recordings, OOS forms, water testing sheets, and cleanroom entry and exit logs) without any accountability or oversight of your quality unit.”8
  • “Our investigators found a large number of trash bags behind a building on your property. The trash bags contained torn original laboratory and production records, such as analytical test reports, (b)(4) water testing reports, and sample notebooks.”9
  • “Our investigator documented that your employees used paper shredders to destroy critical laboratory and production records without the appropriate controls and procedures. Shredded documents included high performance liquid chromatography (HPLC) chromatograms and a partially-completed OOS form.”10
  • “Investigators observed torn, partially complete QA-signed calibration records in the trash and observed QA staff shredding documents without recording the identity or the reason for shredding the documents.”11

Do Not Conceal Undesirable Test Results

Manipulation of tests and/or test results to achieve desirable outcomes must be prevented by sufficient quality systems and control strategies, to include (but not limited to) validation, qualification, and training of personnel. Recent observations from FDA warning letters indicate that this basic tenet is still not being observed. 

  • “Our investigators found a recurring practice of retesting samples until acceptable results were obtained.”12
  • “Your original data showed failing results, but data you reported showed passing results.”13
  • “The chromatogram was then manually rescaled, which hid the presence of this peak. Your laboratory set the integration parameters to omit this peak from integration.”14
  • “You permanently deleted the first five sample injections. You then renamed the last two injections and reported that they met specifications.”15
  • “Your analyst told our investigator that it is laboratory practice to perform more injections than are required by the procedure, and then delete any undesirable result to ensure passing system suitability results.”16
  • “Your firm reported only the passing results from repeat analyses.”17
  • “None of the 19 chromatograms generated in the first sequence were maintained and available for review. Only the second set of chromatograms was maintained and relied upon in releasing lots.”18
  • “Without providing scientific justification, you repeated analyses until you obtained acceptable results.” And, “You relied on these manipulated test results and incomplete records to support batch release decisions.”19

Understand And Review The Audit Trail

While the FDA is said to be reexamining 21 CFR “Part 11 as it applies to all FDA-regulated products,”20 the compliant use of audit trails for computer-connected equipment in the laboratory remains an area of great importance, as it provides one means of evidence of compliance with GDocP and CGMP. Surprisingly, the audit trail remains an area of confusion; however, it may be simply understood as “a chronology of the ‘who, what, when, and why’ of a record.”21 For computer-connected laboratory equipment, it “means a secure, computer-generated, time-stamped electronic record that allows for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record”.22 Recent observations from FDA warning letters indicate that this basic tenet is still not being observed. 

  • “Analysts manipulated and deleted audit trails.”23
  • “An analyst deleted audit trails in your gas chromatography equipment #YQ-07-10 from September 15, 2015, through April 24, 2016, and permanently deleted audit trails from November 6 to 13, 2015. In addition, our investigator observed that your quality control manager and quality control deputy manager had full administrative rights on all of your computerized systems, which allows them to manipulate data and turn off audit trails.”24
  • “The audit trail showed that you performed this testing in duplicate.”25
  • “Our review of audit trail data revealed that your analysts manipulated the date/time settings on your high performance liquid chromatography (HPLC) systems. During the inspection your analysts admitted to setting the clock back and repeating analyses for undocumented reasons.”26
  • “Our investigator observed that your Empower 3 system audit trail displayed many instances of a ‘Project Integrity Failed’ message, which indicates that injections were missing from the results of analytical testing. For example […] no chromatogram was rendered for the initial run of testing. The data package for this testing clearly shows that the initial run is missing, but your quality unit did not investigate the incident.”27

Audit trail functionality is common among computer-connected laboratory equipment but must be enabled, configured, and managed appropriately. Sufficient qualification/validation must be performed to include procedures and operator training to ensure compliance with CGMP and GDocP in laboratory operations.

Reinforce Basic CGMP And GDocP In The Laboratory

Whether in our own laboratories or those of our suppliers, we must demand a higher standard in establishing, enforcing, and reinforcing basic CGMP and GDocP. Auditing of laboratory operations remains a critical success factor.

  • Do not delete or manipulate laboratory data.
  • Do not conceal undesirable test results.
  • Understand and review the audit trail.

Only by ensuring that these basic tenets are a component of our auditing and supplier management processes will we begin to see positive improvement in data integrity in laboratory operations and stop the cheating in the laboratory.

This article is part of a series that aims to discuss specific data integrity improvement opportunities within individual quality systems (e.g., document control, records management, materials management) with specific operational examples (e.g., master batch record design, efficient review of executed batch records, improved schedule adherence).

References:

  1. WHO Technical Report Series No. 996, 2016, Annex 5, p. 182
  2. 21 CFR 211.194(a)
  3. Warning Letter 320-17-29, USV Private Limited, India, March 10, 2017
  4. Warning Letter 320-17-32, Mylan Pharmaceuticals, Inc., PA, April 3, 2017
  5. Warning Letter 320-17-39, Shandong Analysis and Test Center, China, June 22, 2017
  6. Warning Letter 320-17-24, Chongqing Pharma Research Institute Co., Ltd., China, February 14, 2017
  7. Warning Letter 320-17-17, FACTA Farmaceutici S.p.A, Italy, January 13, 2017
  8. Warning Letter 320-17-17, FACTA Farmaceutici S.p.A, Italy, January 13, 2017
  9. Warning Letter 320-17-28, Badrivishal Chemicals & Pharmaceuticals, India, March 2, 2017
  10. Warning Letter 320-17-17, FACTA Farmaceutici S.p.A, Italy, January 13, 2017
  11. Warning Letter 320-17-26, Megafine Pharma (P) Ltd., India, February 24, 2017
  12. Warning Letter 320-17-36, China Resources Zizhu Pharmaceutical Co., Ltd., China, April 24, 2017
  13. Warning Letter 320-17-17, FACTA Farmaceutici S.p.A, Italy, January 13, 2017
  14. Warning Letter 320-17-25, Jinan Jinda Pharmaceutical Chemistry Co., Ltd., China, February 24, 2017
  15. Warning Letter 320-17-25, Jinan Jinda Pharmaceutical Chemistry Co., Ltd., China, February 24, 2017
  16. Warning Letter 320-17-24, Chongqing Pharma Research Institute Co., Ltd., China, February 14, 2017
  17. Warning Letter 320-17-36, China Resources Zizhu Pharmaceutical Co., Ltd., China, April 24, 2017
  18. Warning Letter 320-17-15, Sato Yakuhin Kogyo Co., Ltd., Japan, January 6, 2017
  19. Warning Letter 320-17-24, Chongqing Pharma Research Institute Co., Ltd., China, February 14, 2017
  20. Data Integrity and Compliance with CGMP, FDA Draft Guidance for Industry, April 2016
  21. Data Integrity and Compliance with CGMP, FDA Draft Guidance for Industry, April 2016
  22. Data Integrity and Compliance with CGMP, FDA Draft Guidance for Industry, April 2016
  23. Warning Letter 320-17-25, Jinan Jinda Pharmaceutical Chemistry Co., Ltd., China, February 24, 2017
  24. Warning Letter 320-17-25, Jinan Jinda Pharmaceutical Chemistry Co., Ltd., China, February 24, 2017
  25. Warning Letter 320-17-15, Sato Yakuhin Kogyo Co., Ltd., Japan, January 6, 2017
  26. Warning Letter 320-17-36, China Resources Zizhu Pharmaceutical Co., Ltd., China, April 24, 2017
  27. Warning Letter 320-17-32, Mylan Pharmaceuticals, Inc., PA, April 3, 2017

About The Author:

Kip Wolf is a senior managing consultant at Tunnell Consulting, where he leads the data integrity practice. Wolf has more than 25 years of experience in quality assurance and regulatory affairs, GMP and IT compliance, technical operations, and product supply. His areas of expertise include business transformation, new business development, organizational change leadership, and program/project management. He has held various management positions at some of the world’s top life sciences companies. Wolf can be reached at Kip.Wolf@tunnellconsulting.com.