Data Integrity, Deviations, And Shop Floor Quality

Continuous improvement in data integrity can advance a firm on the journey toward a mature culture of quality, particularly through the implementation of QA on the shop floor. Batch record review (BRR) and product disposition are often complicated by data integrity issues and poor data quality. The process of BRR and product disposition is made further complex by production deviations and investigation of quality events, adding to the volume of data that must be reviewed and resolved in order to facilitate product disposition. The ISPE GAMP® Guide for Records and Data Integrity (p. 34) states that “data creation should provide accurate data which is needed throughout the business process for decisions based on that data,” one example of which is where production data is necessary to inform BRR and product disposition.  

Improved Data Integrity

Having staff from the quality assurance (QA) function present or easily accessible to all shifts and operations in a production facility may improve data integrity by providing consultative QA support on the shop floor, even in facilities with mature and robust quality management systems and data integrity practices. The consultative approach is a critical success factor when implementing QA staff on the shop floor. The understanding must be that the QA staff is there to provide guidance and coaching during operations, as well as to assess and audit manufacturing processes in real time, giving immediate review and constructive (perhaps instructive) feedback. Here are some examples related to the ALCOA+ attributes that may be reinforced by having QA staff on the shop floor.

Table 1: Examples Of Shop Floor Activities Related To The Individual ALCOA+ Attributes

By leveraging real-time teaching moments from QA staff while they are observing operations, feedback is not delayed nor perceived as criticism later when batch record review is performed well after operations are complete. Real improvements in data integrity at the shop floor may be realized where QA staff are eager to contribute value to the operations and where manufacturing staff are able to accept coaching and contemporaneous constructive feedback from QA staff.

Improved Data Quality

Data quality may also be improved at the shop floor level by consistently classifying production deviations as a result of quality events from shift-to-shift and operator-to-operator. Quality events and production deviations may be prevented or their impact minimized by having QA staff on the shop floor or nearby and readily available to production operators. QA staff may offer consultative direction in real time where production deviations occur, so that initial documentation of quality events is more robust, descriptive, and accurate.

This consultative approach may prevent over-classifying the production deviation, until it can be investigated and understood (i.e., classifying an event as critical until it is declassified to a lesser impact later). We have often seen that shop floor deviations are classified as critical, as operators are accustomed to erring on the side of caution. While this practice is common and the intent is admirable, the outcome is very often (at least) an inefficient use of resources At worst, over-classifying events results in escalating events and notifying management unnecessarily,  leading to  the desensitization of management to event escalation.

Figure 1 shows real data from the first three months of production operations for a firm that implemented QA on the shop floor during all shifts of production operation. By improving the quality of data related to quality events and production deviations on the shop floor, improvements in overall numbers of deviations (quantitative results) and reduction in the escalation of critical deviations (qualitative results) were achieved.

Figure 1: Results in reduction in quality events after implementing shop floor QA

Real results can be seen after implementing a shop floor QA role during each manufacturing shift. In this case, the result was a noticeable decrease in serious and critical deviations and an overall decrease in total deviations, with no critical deviations indicated by month three.

Specific metrics and performance indicators related to QA on the shop floor are listed below. We have found that tracking/trending of and responding accordingly to these metrics and performance indicators may have a substantial positive impact on data integrity and data quality as a component of fully implementing QA on the shop floor.

• Number of spot checks by QA on the shop floor: Total number of spot checks (i.e., full or partial record review by QA staff on the shop floor) per period (e.g., weekly, monthly, etc.) by area (e.g., manufacturing line, operating area, or manufacturing step).
• Near misses and early-identified investigations: These are defined as a measurement of any identified errors during spot checks that, if gone undetected and uncorrected, would have likely resulted in a formal investigation and measurement of any investigations that were the result of early-identification or errors during spot checks.
• Good documentation practice errors on the shop floor: Tracking and trending of errors in good documentation practices as identified from the spot checks performed by QA on the shop floor. Examples for trending include, but are not limited to:
  • Calculation error
  • Illegible entry
  • Incomplete entry
  • Incorrect entry
  • Missing entry
  • Missing signature and/or date
  • System (e.g., MES) error
  • Time/date entry error
  • Transcription error
  • Write-over

Other metrics and performance indicators related to deviations may also provide insights into the value of such improvements. Consider these when implementing QA on the shop floor to look for indicators of success and areas for continuous improvement in the methods and results of the consultative QA approach. These include, but are not limited to:

• Deviations % overdue
• Deviations % on time (by due date)
• Deviations # per batch
• Deviations # (%) recurring
• Rate of Deviations # open
• Rate of Deviations # closed

The results shown in Figure 1 also support an improvement in quality culture realized by having QA staff readily available during operations, as the operators better understood how to identify and classify manufacturing events and deviations. Rather than erring on the side of caution and classifying manufacturing deviations as critical until later determination would appropriately classify them as less serious, the operators were more accurately classifying and resolving deviations more promptly. The results are more accurate data, more efficient use of human resources, and more effective resolution of manufacturing events.

One Step On The Journey To Improved Data Integrity And Culture Of Quality

While these data are limited (only three months) and a total reduction of critical deviations long-term would present an entirely different set of concerns (i.e., misclassifying deviations and insufficient escalation of deviations that would otherwise require management notification), this anecdotal evidence demonstrates that real cultural change may occur with the maturation of data incident and problem management through improvements to production records as a result of QA staff on the shop floor. A fully mature data incident and problem management may be indicated by an “established data incident and problem management process, supported by tools and appropriate metrics, leading to process improvement” (ISPE GAMP® Guide for Records and Data Integrity, p. 63).

Real improvements to data integrity are possible through consultative approaches that reinforce a culture of quality that leads to overall improvement of our business and industry. There is no “quick fix” for improving data integrity and data quality, as it remains a journey of continuous improvement; we find that QA staff on the shop floor using a consultative approach provides a real and sustainable value on that journey.

About The Author:

Kip Wolf is a principal at Tunnell Consulting, where he leads the data integrity practice. Wolf has more than 25 years of experience in quality assurance and regulatory affairs, GMP and IT compliance, technical operations, and product supply. His areas of expertise include business transformation, new business development, organizational change leadership, and program/project management. He has held various management positions at some of the world’s top life sciences companies. Wolf can be reached at Kip.Wolf@tunnellconsulting.com.