As stated in the FDA November 2016 draft Guidance, Submission of Quality Metrics Data – Guidance for Industry (here), one of the Quality Metrics that the FDA intends to monitor is the Invalidated Out-of-Specification (OOS) Rate (IOOSR). The IOOSR quality metric is the number of OOS test results for lot release and long-term stability invalidated by the covered establishment due to an aberration of the measurement process divided by the total number of lot release and long-term stability OOS test results. The target timeframe for the voluntary reporting of Quality Metrics to the FDA is in early 2018. So why the rush?
The IOOSR metric is intended to be an indicator of a laboratory’s ability to accurately perform testing. The International Society for Pharmaceutical Engineering (ISPE) has commented to the FDA on this proposed metric and has suggested significant changes to the metric (here). One concern is whether the IOOSR metric (calculated as currently proposed) is an appropriate measure of the laboratory performance. ISPE has proposed an alternate metric intended to be more meaningful based on the number of invalidated OOSs divided by the total number of tests performed. The merit of an OOS metric is to help identify if the laboratory needs to improve training, methods, equipment, or the thoroughness of laboratory investigations to ensure the root cause of each OOS is identified and a suitable CAPA is created to address the root cause.
Although Data Integrity issues have been monopolizing the recent news, from the beginning of 2016 to date, the number of Warning Letters with OOS-related Observations (28 Observations) have nearly matched those for Data Integrity (31 Observations). Furthermore, it has become apparent from recently issued FDA Form 483s that the FDA, during inspections, are inquiring more about the rate of “Invalidations”, “Disregards”, etc. for Laboratories’ OOS investigations. An Invalidation or Disregard is a laboratory investigation for which the OOS data is invalidated as an aberration of the measurement process (i.e., laboratory error). A high rate of Invalidations/Disregards may be interpreted as resulting from investigations that are not thorough, leading to unjustified invalidation of data. Unfortunately, for many laboratories, their metrics on laboratories’ investigations are confounded by the practices chosen for categorization and tallying of investigations. During regulatory inspections, agencies are often presented with laboratory investigation metrics that are not well defined and, therefore, subject to interpretation and these metrics can then lead to regulatory agency observations. With laboratory quality metrics, like with good food, the presentation can really matter.
The “take home” message is, laboratories should start tracking the proposed IOOSR metric and, also, consider alternate presentations for the data (such as the ISPE proposal) that they feel might better represent their laboratory performance. Independent of what metrics are chosen, laboratories should strive through thorough investigations and suitable CAPAs to continuously improve their performance.
For further information and assistance on the topic of laboratory quality metrics and the Invalidated Out-of-Specification (OOS) Rate (IOOSR), please contact either Ron George, Ph.D. at r.george@LachmanConsultants.com or James Davidson, Ph.D. at j.davidson@LachmanConsultants.com.
