Facts or Intuition – How Reliable is Your Risk Analysis?

Excerpt from the GMP Compliance Adviser, Chapter 19, Quality Risk Management

4 min. reading time | by Martin Mayer
Published in LOGFILE 18/2025 

How does subjectivity impact quality risk management? Martin Mayer discusses the dos and don'ts for an objective quality risk management. A more comprehensive overview on the topic of subjectivity in pharmaceutical industry can be found in the GMP Compliance Adviser under the heading ‘Quality Risk Management’.

What does it depend on if a risk analysis is reliable? 

The success of a risk analysis and associated risk evaluation depends strongly on the quality, validity, and reliability of the underlying information ("garbage in = garbage out") and the objectivity of the persons involved. All those involved should be aware of potential subjectivity and its possible sources. This applies especially if quantitative methods are used prospectively. 

Let´s consider the following example: 

You carry out a risk assessment to support the planning process for a new process machine. Here it is determined which critical process parameters have what (possible) influence on which critical product quality attributes of a medicinal product to be produced in the future and how these are to be controlled, monitored and recorded. 

If experience of the product and/or manufacturing equipment is in place, historical data may be used as a basis for the risk analysis and evaluation. 

However, if there is no process or product experience or the technology is unknown, it is difficult to evaluate the probability of a failure occurring using quantitative methods. If a quantification is nevertheless carried out, the basis of the assessment is at best a well-founded presumption and in the worst case pure speculation. 

If the evaluation of the probability of occurrence / detection of a failure or the severity of damage is based on an assumption that is then expressed by a numeric value, a precision of statement is implied that has not actually been achieved. It is therefore a feigned precision. As a result, the probability of occurrence is often significantly overestimated or underestimated. For this reason, it is important to identify assumptions, presumptions, and other sources of uncertainty in order to clearly state the reliability of the statements and subsequent risk evaluation, or to outline the limited significance of the conclusions. 

How to get a handle on subjectivity and feigned precision? 

Different experiences, perspectives, or interests of those involved in quality risk management (QRM) – everyone is human! – can result in distortions, assumptions, one-sidedness, bias, and intolerance – in short, subjectivity. These human factors can influence every phase of a QRM process, especially where many participants come together. 

The identification of hazards and the estimation of probability of occurrence / severity of health damage are particularly susceptible to subjectivity. However, the definition of risk-reducing measures (hazard prevention) and the evaluation of the effectiveness of measures can also be negatively influenced – risks are misjudged, over- or underestimated. As a result, incorrect or inappropriate decisions are made. 

Subjectivity can also be the result of uncertainty, ambiguity and a lack of product and process knowledge, or if the intention or objective of a QRM process is insufficiently defined. Other causes of subjectivity can be the selection of an unsuitable method, poorly designed risk scoring scales and a lack of methodological expertise. When QRM methods and tools are ‘worked through’ mechanically, without common sense, only because it is required, when quantitative methods are applied based on qualitative knowledge or assumptions, the results are usually very subjective. 

Just as craftsmen have to learn fundamentally and through repetition how the tools available to them can be used most effectively in a situation, those involved in a QRM process have to familiarise themselves with the topic. They must want to act and learn based on knowledge, they must be familiar with the available QRM methods and know how they can be used and how not to use them. 

It would not be possible to avoid or eliminate subjectivity completely, but being aware of the sources and manifestations of subjectivity will help to minimize it. To this end, all participants involved with QRM should be willing to acknowledge, anticipate, address, and minimize the potential for subjectivity. 

The lesson is clear: 

Risk-based decisions require not only product and process knowledge, but also knowledge of QRM methods and experience in their application. 

Conversely, a lack of product and process knowledge and a lack of methodological expertise can lead to subjective (mis)assessments. QRM can only be used effectively if all those involved are aware of this problem and work together to minimize subjectivity.