This content is current only at the time of printing. This document was printed on 25 June 2021. A current copy is located at http://www.plural-consulting.com/node/19676
You are here
Audit ratings and scores
The Australian Pesticides and Veterinary Medicines Authority (APVMA) audit review and closure process considers the:
- overall number and severity of all non-conformances (NCs) reported
- corrective actions undertaken or planned
- types of products being manufactured
- staffing and process changes within the facility
- number of product recalls undertaken.
1. Non-conformance classifications
Non-conformances (NCs) are classified by the APVMA as ‘critical’, ‘major’ or ‘minor’.
Classifications are based on the extent to which the NC represents a failure to satisfy a key requirement and/or poses a risk to product quality. While the risks posed by one minor NC may be small, the risks posed by a number of such NCs may be significant.
It is not always feasible to describe the overall outcome of an audit in terms of the number of separate minor, major and critical NCs. As a consequence, the outcome of each audit is described in terms of a:
- quantitative weighted non-conformance score – taking into account the nature and number of all NCs reported
- qualitative audit rating level – reflecting the relative capability of the facility’s quality management system to maintain compliance.
APVMA staff assign NC scores and audit ratings as part of their audit review process for both domestic and foreign audits by APVMA authorised auditors. In most cases, this does not apply to audits undertaken by other recognised domestic or overseas conformity assessment bodies, such as the Therapeutic Goods Administration (TGA)?or US?Food and Drug Administration (USFDA), where responsibility for scheduling audits rests with that body.
|Minor non-conformance||A minor or less serious non-conformance which is unlikely to pose a risk to product quality.|
|Major non-conformance||A failure to satisfy a key or mandatory requirement and/or one which may pose a risk to product quality.|
|Critical non-conformance||A major non-conformance which poses a risk to treated animals or users and must be corrected immediately.|
2. Critical non-conformances
While less frequently observed, a critical NC is a major NC which poses a risk to treated animals or users and must be corrected immediately.
In cases where there are critical NCs reported, manufacturers need to implement effective corrective actions quickly. The audit schedule proposed by the APVMA anticipates that other compliance and enforcement tools may be used to support auditing where appropriate.
3. Non-conformance scores
In order to determine a quantitative non-conformance score, from 1 March 2016, each minor NC will be assigned a nominal value of one non-conformance unit and each major NC will be assigned a value of 4 non-conformance units. The non-conformance score represents the total number of NC units arising from an audit.
|Level of non-conformance (NC)||Non-conformance unit value|
4. Audit ratings
The APVMA assigns an ‘audit rating’ – also referred to as an ‘audit level’ – after each audit. The audit rating contributes to determining the relevant audit interval for the manufacturing site.
When it is apparent key information was not made readily accessible to the auditor, the APVMA reserves the right to assign an audit rating as it sees fit. This can include circumstances where an overseas manufacturing site has not provided the APVMA-authorised auditor with English translations of key documents.
4.1. Audit level 1
Audit level 1 is assigned when the audit has reported no major NCs and the non-conformance score is 4 or less.
When a manufacturing site is assigned an audit level 1, the high levels of compliance with the manufacturing standards provides a basis for allocating longer audit intervals.
4.2. Audit level 2
Manufacturing sites are assigned an audit level 2 when an audit has reported no more than 5 major NCs and their non-conformance score is between 5 and 20.
This means the APVMA considers manufacturing sites with an audit level 2 rating to have a functioning quality management system, although with some identifiable weaknesses to address.
4.3. Audit level 3
Audit level 3 is assigned to manufacturing sites with non-conformance scores of 21 to 40, with no more than 10 major NCs.
For manufacturers assigned an audit level 3 rating, the audit intervals will remain similar to those applied prior to 1 March 2016.
4.4. Audit level 4
Manufacturing sites are assigned an audit level 4 when the audit reports more than 10 major NCs and/or a non-conformance score greater than 40.
For manufacturers assigned an audit level 4 rating, the audit intervals will remain similar to those applied prior to 1 March 2016.
In addition to closer regulatory supervision through more frequent audits, other compliance and enforcement options may be considered on a case-by-case basis.
|Audit result||Audit rating|
|Number of major NCs||NC score|
|0||0 to 4||Audit level 1|
|No more than 5||5 to 20||Audit level 2|
|No more than 10||21 to 40||Audit level 3|
|More than 10||Greater than 40||Audit level 4|
Altering audit ratings and scores
The APVMA does not normally alter the NC rating provided by an auditor, however it may de-aggregate distinct non-conformances when determining a non-conformance score.
For example, under ‘cleaning’ an auditor may identify NCs related to the adequacy of the cleaning procedure (NC 1a), cleaning schedules and records (NC 1b) and validation of cleaning (NC 1c). In these cases, separate corrective actions are still required to address each reported NC, regardless of whether they are aggregated or not. In other cases, the APVMA may aggregate a number of minor NC's where the same corrective action is required.
If a reported NC is not clearly expressed or it is challenged by the manufacturer, the APVMA may ask the auditor to provide further information to justify the score and rating assigned.
5. Breaches of licence conditions or manufacturing standards
Where the audit report describes intentional or management sanctioned breaches of licence conditions, manufacturing standards, a loss of process control or a critical non-conformance is identified, the APVMA reserves the right to rate, audit and manage the facilities as audit level 4, regardless of the number of NCs reported.
Examples of such breaches include:
- the release of raw materials or finished product that was out-of-specification (OOS)
- repeat non-conformances
- breaches of licence conditions
- critical non-conformances
- reports demonstrating audit level 3 (or below) compliance at two successive audits.