Auditor Evaluations: A GMP Auditing Best Practice

The success and value of an audit rests largely on the shoulders of the lead auditor and the audit team. And while technical expertise is important, it alone does not make for a good audit. Of course, if the auditor is not familiar with the technology, he will be handicapped. But equally important are audit preparation, how the audit is handled from the first day until its close, and, ultimately, how the results are communicated. Given the importance of auditor performance, do managers responsible for audit programs solicit and provide feedback on auditors? Very rarely.

Auditing programs perhaps implicitly provide feedback at the reporting stage of an audit. Most companies and third-party auditing programs conduct a technical review of completed audit reports. These reviews offer an opportunity to comment on the auditor’s work. But what about other elements, such as audit preparation and professional conduct during the audit? And what about the strength of the auditing findings? Are the findings credible, are they sufficiently substantiated with evidence, and do they reference a regulatory standard?

Well-managed audit programs should include robust auditor evaluation procedures. For example, an accredited excipient certification program must include auditor evaluations as part of the audit program.  A certification audit may require three days onsite thoroughly examining an excipient manufacturing facility against each element of the NSF/IPEC/ANSI 363 GMP standard. The outcome of this audit will guide a decision on whether a site is compliant with the GMP standard, and it will also guide company management in terms of the next steps to improve their operations.

Consider the number of personnel hours required to support an audit. Time is a precious commodity in all plant operations; therefore, the time dedicated to supporting an audit should be justified. The audit closing meeting and audit report should convey that a rational, evidenced-based approach was used. And audit findings must carry the weight of the audit evidence. While the auditee may not entirely like the outcome, credible findings will stand on their own and warrant little discussion during a closing meeting. If a finding is not credible or not sufficiently substantiated, it behooves the host site to question the auditor in a respectful manner. In some cases, auditees end up addressing the most trivial issues and risk adding complexity to their operation. This is a vicious circle we need to guard against. We must ensure our actions add value, not complexity.

Auditor evaluation criteria should be developed by company or third-party audit programs and include these principles: 

• Professional and ethical conduct is the underpinning of the auditor’s work and requires good judgment in handling audit information and confidentiality.

ISO 19011 Principles That Guide Auditor Conduct

• Independence ensures the auditor is free of any bias and conflict of interest in conducting an audit. Programs will not permit an auditor to audit the same site multiple times, since the risk that familiarity and bias could cloud judgment is too great.
• Fair presentation requires factual and truthful reporting of findings. In addition, fair presentation means reporting any obstacles encountered during the audit. For instance, if an auditor is restricted from accessing certain manufacturing areas, it is important to report this obstacle. There may be a good reason why access is restricted, but to ensure fair presentation it is important the auditor report such constraints.
• An evidence-based approach requires a logical and rational approach to managing audit findings. For example, an auditor may suspect an issue related to data integrity; however, it is the auditor’s responsibility to present evidence that substantiates this suspicion. Repeated problems with good documentation practice should not rise to the threshold of a data integrity issue unless there is evidence, for example, that data is not recorded contemporaneously or is being manipulated in some manner. Exaggerating or escalating findings without good reason risks auditor credibility.

The closing meeting is one of the most important stages of an audit. Picture a room or an auditorium full of company personnel armed with pens and notepads. The auditor will have a rank-ordered list of findings ranging from major to minor and, in some cases, critical findings. Often a theme or thread runs through the audit findings. It may have to do with cleaning procedures, poor investigation reports, lapses in tech transfer, or weaknesses in the company training system. A strong closing meeting will highlight any themes. And, not to be ignored are the best practices and positive aspects identified during the audit. Recognizing the hard work completed by the host-site audit team is always appreciated. By prioritizing audit findings, ensuring findings are evidence-based, taking care to avoid bias or opinion, and focusing on what will take the plant forward, the auditor will ensure the closing meeting is useful to all personnel present.

A good audit program and/or third-party auditing organization should provide strategic value to company operations. A robust auditor evaluation program will keep key audit principles top of mind and help ensure the audit is ultimately a value-added exercise that will guide management decisions.

About The Author:

Jim Morris has over 25 years of pharmaceutical management experience in both plant operations and corporate offices in the U.S. and Europe. He has held positions as deputy director of QA/QC and regulatory affairs at Mass Biologics, director of QA/QC for the Biologics business unit of Cilag AG, and a number of quality assurance and manufacturing roles with Pfizer over a 16-year timeframe, culminating as the head of quality assurance in Latina, Italy. His areas of recognized expertise include: quality leadership development, supply chain auditing and managing audit programs; quality management systems; parenteral product manufacture and compliance; and OTC product manufacture and compliance.