The Vendor Audit - How to Get the Most Out of Your Selected Software Supplier
Who Should Carry Out the Audit?
What Type of Audit Should Be Undertaken?
Carrying Out the Audit
What to Ask, What to Look For
After the Audit—What Happens?
Proof of Experience Is Vital
The cost of a software project failure to an organization can be immense—especially for a laboratory, where its future continuity can be at risk if a large and sizeable investment in IT goes wrong. In the case of LIMS, a large amount of money is usually invested in the system, and its implementation can take up to two years. The wrong decision can be very costly for the organization, and involvement in a failed project can be career-limiting for individuals.
For decision-makers contemplating an IT system, therefore, caution is the watchword. The first step should be an in-depth analysis of the market to select a potential software supplier. The next step should be a thorough evaluation of this company. In the same way that you would not buy a house without first checking whether claims made about the property can be substantiated, you should not rush into a business relationship without finding out as much as you can about a potential key supplier. The best and most widely accepted procedure for such assessment is the vendor audit.
The vendor audit is a valuable pre-contract tool in the procurement process. It should produce answers to these questions:
- Do I have a high level of confidence that this vendor can meet or exceed my organization's technical, commercial, and any regulatory requirements?
- Can we develop a mutually beneficial relationship with this vendor?
The following steps are some guidelines on how to undertake a vendor audit. They are representative of the type of audit that we at Thermo LabSystems—as a vendor/supplier—are often subjected to.
Who Should Carry Out the Audit? (Back to Top)
Audits should be carried out by a team. A lead auditor should be experienced and qualified, and supported by representatives who can include technical specialists, users, project leaders, purchasing employees, and quality assurance personnel. If the appropriate team is not available in-house, a consultant can be called in.
What Type of Audit Should Be Undertaken? (Back to Top)
The audit typically consists of two parts: a company and product audit. The company audit covers examination of the organization and any Quality Management System, whereas the product audit focuses on specific products or services only.
Typically, there are four stages in the vendor auditing process:
- Initial evaluation
- Detailed audit
- Follow-up audit
- Surveillance audit
The objective of the initial evaluation is to obtain enough information to take a broad view on the suitability of prospective vendors. A questionnaire is often used, and the initial evaluation does not usually involve visiting the vendors, although an initial evaluation may certainly precede a detailed audit. It is a useful method for producing a short-list of potential vendors.
The detailed audit precedes any contractual commitment, and is both in-depth and full-quality. It examines in detail all the business and development activities of the vendor. This type of audit should be conducted prior to placing the contract and be an intrinsic part of the procurement process.
The follow-up audit is the monitoring opportunity. It is used to check on issues generally raised during a detailed audit. It can also be used to provide evidence on any agreed corrective and preventive actions.
The surveillance audit is periodic (every twelve months is best) to verify that the vendor is maintaining the required standards, as per contract or as seen on previous audits.
Detailed audits should be conducted by at least two people, one of which must be the lead auditor. Follow-up or surveillance audits can be performed by a single lead auditor. The lead auditor should have overall responsibility for the entire audit process, and should be the main interface in coming to terms with the vendor.
Carrying Out the Audit (Back to Top)
To undertake a successful audit, it is useful to use checklists. A typical checklist may contain broad headings such as:
- Company Overview,
- Organization and Quality Management System,
- Software Development Life-Cycle,
- Planning and Product/Project Management,
- Operation and Maintenance and Supporting Activities.
What to Ask, What to Look For (Back to Top)
In practice, all checklist topics should be covered to a minimum level, with key activity areas selected for closer examination. In all cases the audit team should adopt a "show me" approach following interviews and discussions with vendor personnel. If an auditor is told, for example, that all test plans are reviewed and approved, the auditor must ask to see objective evidence of this. If the vendor states that all software development engineers are trained in C++ programming, training certificates, course attendance, and/or CVs must be examined to substantiate this claim.
Following an audit trail is a good method of determining that sound practices are in place and implemented effectively throughout a number of functional areas.
It goes without saying that the best client/vendor relationships are based upon honesty and openness. So, during and prior to the audit, the vendor should be maintaining a "business-as-usual" attitude. One telltale sign that this is not the case is with documentation dates. Alarm bells should sound with auditors when they notice that substantial levels of documentation have been generated the week before the audit.
Auditors should also try to resist being guided by the hosting vendor toward specific individuals. Ask for an organization chart and select project team members at random. It is not unheard of for vendors to even ensure that key, less-than quality-conscious individuals remain offsite for the audit's duration.
Auditors should also beware of delays that can limit the scope and objectives of a typical one- or two-day audit. A vendor can, in effect, take some control over an audit's course. Auditors should beware of long welcoming presentations and protracted retrieval times for requested documents, which should be timely and not reduce the time for more sampling by the auditor. It is also worth inquiring further with the vendor if access is denied to some significant parts of the office building.
Finally, on a more light-hearted note, auditors should be suspicious of offers for business lunches off-site during the audit itself. What type of influence might the vendor be trying to exercise, and furthermore, what frantic preparations for the afternoon's audit schedule are being made as you're enjoying your second glass of Chardonnay?!
After the Audit—What Happens? (Back to Top)
The lead auditor produces an audit report, which serves as a formal record of the audit and its findings; it is the key input for determining corrective actions. On audit completion there are a number of options:
- Use the vendor unconditionally, because you are happy with the result of the audit and you think the vendor is wonderful,
- Use the vendor for only certain products or services,
- Use the vendor subject to submission of a corrective action plan,
- Mutually agree a set of standards for the vendor to use for the purposes of the contract,
- Reject the vendor.
Proof of Experience Is Vital (Back to Top)
The vendor should have demonstrable experience of customer audits, and certification to standards such as to ISO 9001 and the TickIT scheme for Software Development. The audit procedure offers a potential purchaser of IT the confidence that the vendor is a suitable supplier with whom to enter a partnership and a successful audit allows both parties to proceed in comfort.
Auditing is not a cheap option—a recent PDA (Parental Drug Association) Supplier Management Task Force survey sets the average cost of a single vendor audit at $8,000 to $10,000! While this is a joint cost borne by both the client and the vendor, it can pale into insignificance alongside the cost to a research organization of selecting an unsuitable supplier!
Ian Herd is Quality Assurance Manager at N/A.
For more information, contact the author at 800-719-1853, or by email at ian.herd@labsystems.com.