Deviation Causes as KPIs – Collection vs. Assessment

7 min. reading time | by Felix Kern, PhD and Liwa Schneider
Published in LOGFILE 11/2022

Deviation management plays an increasingly important role in the collection and review of KPIs (Key Performance Indicators) for pharmaceutical production and quality processes. Tracking the cause of deviations in particular provides information on how stable the respective processes are, where there are gaps and where countermeasures need to be taken with corrective and preventive actions (CAPAs).

However, difficulties can arise depending on how these indicators are collected.

In general, there is a wide variety of different causes for deviations. Classic examples are:

• Environmental conditions: A storm damages the ventilation system for a production plant.
• Machine: A tabletting stamp breaks due to motor damage.
• Human Error: An employee forgets to carry out an in-process control due to carelessness.
• Material: During the incoming goods control, glass particles are found in a raw material.
• Method: A manufacturing instruction is formulated too imprecisely and left room for interpretation or was not understood by the staff.
  The manufacturing instruction specifies work steps in an illogical order.
• Data Integrity: A system crash with loss of raw data occurs during measurement in a laboratory.

The KPI ‘human error‘ in particular is gaining in importance and is increasingly being assigned upper limits in the performance targets. A high human error rate can indicate many things. This could be, for example:

1. A superficial deviation handling. In many cases it is easier to attribute the cause of a deviation to an employee error than to spend time researching what is really behind the error. This could be, for example, training deficits, specification deficits or system deficits.
2. A systematic staff problem. This can be multi-tasking, for example, which means that staff members perform too many tasks in parallel: A staff member takes care of the operation of two devices at the same time: the visual inspection of units at defined time periods and the cleaning of surfaces.
3. There is an actual human error. This could be, for example, the deliberate contravention of an instruction or a lack of concentration.

However, attention must also be paid to how the human error rate is collected. Absolute or relative values are possible here. For absolute values, this means that out of 100 deviations, 20 deviations are caused by ‘human error‘. The relative value would be 20%. For tracking, the absolute value is more suitable than the relative value. Why is this so? The following is an example:

Let's assume that out of 100 deviations, we have 50 with the cause ‘human error‘ and 50 with the cause ‘machine‘. Now the target (relative value) for the KPI human error is set at 20% and a project is launched to reduce the human error rate. At the same time, a project is started to stabilise the manufacturing processes.

Through the human error reduction project, the absolute value of deviations with this cause is reduced from 50 to 30. This is huge progress. At the same time, by establishing modern machines and modern processes to protect these machines, the number of deviations with the cause ‘machine‘ is reduced from 50 to 10.

Of 40 remaining deviations, 30 fall into the cause category ‘human error‘, which corresponds to an increased relative value of 75%. This means that despite a respectable reduction in the number of deviations with the cause ‘human error‘ from 50 to 30 deviations (absolute value), the relative value has increased from 50% to 75% (cf. Table 1).

Table 1

However, this only considers the case of a constant production volume. If the production volume increases, for example from 100 to 200 batches, it can be assumed that the absolute value of deviations will also increase. This must then also be taken into account when specifying the KPIs as absolute values.

Summary:

Tracking the cause categories of deviations is becoming increasingly important in the KPIs. The focus here is particularly on the cause ‘human error‘. It makes more sense to track this KPI according to the absolute value than according to the relative value, as we have illustrated in the example. In order to interpret these figures correctly and derive meaningful measures from them, one must consider in advance which factors need to be taken into account for correct collection. This is how the KPIs can be properly evaluated.

Do you have any questions or suggestions? Please contact us at: redaktion@gmp-verlag.de