Topic 7.9: Quantitative analysis
Qualitative analysis, which we have described to this point, is usually subjective and ultimately relies on the judgement of individuals.
It is the examination, analysis and interpretation of observations for the purpose of discovering underlying meanings and patterns of relationships.
This is why experts and their experience might give us more confidence in any qualitative findings.
Even so, two or more experts may look at the same information and reach two different conclusions.
In the narrow context of project risk analysis, this means that different stakeholders may have unique perceptions of what ‘high probability’ and ‘high impact’ are, for example.
You could argue that this is a risk in itself, which is why one response to that is to develop, through the risk management plan, a standard set of risk definitions.
Quantitative analysis, on the other hand, is research using information dealing with numbers and anything that is precisely measurable.
The most common method is statistical analysis.
You would think that this is more reliable; however, the garbage-in-garbage-out rule applies.
Quantitative findings are only as good as the reliability and – in the context of risk management – relevance of the data.
Subjectivity, right up to the choices of statistical methods used, can still significantly colour the outcomes.
What is it they say about lies, damn lies and statistics?
The essential difference between qualitative and quantitative risk analysis is that qualitative analysis allows you to compare risks relative to each other (this is our risk ranking, the main output of the process); whereas quantitative analysis assigns hard, empirical values to assets, expected losses, and the cost of controls.
A three-point estimate, for example, is actually a quantitative method to account for the best and worst case risk scenarios when making an estimate.
Similarly, we used the expected monetary value technique in the project initiation stage to analyse the aggregate risk of different scenarios.
We will revisit this in the next topic on contingency planning.
A related simulation technique, Monte Carlo analysis, uses a model that translates the specified uncertainties of the project into their potential impact on project objectives.
Monte Carlo problem-solving is used to approximate the probability of certain outcomes by running thousands, or hundreds of thousands, of trial runs, called iterations, using random variables.
It lets you see, in a probability distribution, all the possible outcomes of your decisions and assess the impact of risk, allowing for better decision-making under uncertainty.
For a cost risk analysis, a Monte Carlo simulation would use cost estimates.
The output from a cost–risk simulation is shown here.
Similar curves can be developed for schedule outcomes, using the schedule network diagram and duration estimates.
You are not expected to be able to perform Monte Carlo simulations to pass this course; it is shown here as an example of one of the more sophisticated quantitative methods that are available to you.
This is especially true in projects, given that many projects are inherently unique or unusual undertakings for an organisation.
Similarly, the more complex our quantitative model, the more opaque our analysis becomes.
In other words, the more difficult it is for the average stakeholder to understand how we arrived at our conclusions.
And quite often, some or all of the model inputs are subject to sources of uncertainty, including errors of measurement, absence of information and poor or partial understanding of the driving forces and mechanisms.
This uncertainty imposes a limit on our confidence in the response or output of the model – an uncertainty that is itself a risk we should analyse and respond to!
For that reason, we express no preference nor make any recommendation as to which method is best.
Nonetheless, if the intention of project risk management is to anticipate and control uncertainty, then one of the challenges is to find the right mix – in the context of the work to be performed – of diagnostic and analytic tools.