Reprint – Risk Analysis for Tough Issues

2000 words. 8 minutes. Updated 30 January 2024

Back in December 2006 I published an article with Risk Management Magazine, demonstrating that Enterprise Risk Management goes beyond the conventional categories of hazard and insurable risk. I discuss the application of what I call High Quality Risk Assessment to administrative plans and policy. 

I wouldn’t change a word of this article today, because it is relevant to the question of how to bring risk assessment not only to operations, but to corporate strategy – still a source of confusion. 

Here is a reprint:

=======

Navigating the public policy labyrinth: using risk analysis for tough issues.

By Edward Robertson

Copyright Risk Management Magazine, Dec, 2006, Volume: 53, Issue: 12

All organizations have intractable policy issues that lie outside the realm of conventional

risk management. If risk managers wish to promote greater interest in risk techniques

among all enterprise disciplines--they must ask themselves: How can complex issues be

subjected to structured risk analysis?

Studies on program implementation show that people will take up a new practice, such as

enterprise risk management, only if it helps them do their existing jobs more effectively.

There is a continuing need for practitioners to share stories about how to apply risk

analysis in unfamiliar contexts, where purely quantitative methods cannot be applied.

Some useful insights come from the inner workings of the government of British

Columbia. This Canadian province is already well on its way to establishing risk

management as part of its core policy. Under the guidance of its risk management branch,

the government adopted the Australia/ New Zealand 4360 standard and produced a 29-

page ERM guideline to adapt the Australian standard to government business.

In particular, two British Columbia government executives demonstrate how risk analysis

can inform policy and program design. Operating in the challenging areas of land use

planning and safe drinking water systems (as with similar public and private sector issues

such as infrastructure planning, contaminated sites and nuclear facilities), these

individuals found that unclear goals, a diversity of stakeholders with competing aims,

conflicting practices, high public expectations and political sensitivity make solutions in

these areas quite elusive.

Land Use Planning

Kathy Chopik is an executive director who leads policy development at the highest

echelon in the provincial public service, reporting to deputy ministers. She had to write

policy to reconcile the planning processes for crown (government-owned) land. Forest

certification, watershed management, First Nations treaty negotiations and environmental

assessments were just the start of a long list of independent activities that were producing

unclear and unpredictable outcomes.

The government's approach to administering crown land lacked "horizontal" or crossgovernment

coordination. It was already clear to Chopik that a comprehensive framework

was required to ensure that the government's priorities would be met. She decided the

best solution would be based on a risk profile developed by a multi-disciplinary group in

a roundtable session.

The first step in risk analysis is to establish the context. This means carefully defining the

scope and assumptions of the exercise in a few pages, for the benefit of the session

participants. If this is not done, the discussion of risk tends to range so far and wide that

the facilitator despairs, and participants lose sight of what the original aim was. In this

case, it was useful to keep strategic and operational contexts distinct, and to consider

emergency and business continuity planning as a specialized discipline and separate

exercise.

Chopik established the context for crown land planning under the following five

headings:

1. Subject of the analysis. What is the policy, plan, project or program under scrutiny?

Chopik did not yet have an overall policy framework. She therefore specified the subject

of the risk analysis as: "the government's current approach to land use planning."

2. Goals and objectives of the policy, plan or program in question. Chopik and the

ministry staff listed the government's typical land use planning objectives, such as:

"finalize the definition of parks and protected areas" and "enable the tenuring or fee

simple sale of crown land."

3. Value Criteria. These are the guiding principles of the organization, such as a

professional ethical code, business practices, political priorities or operating principles.

Team members refer to them to identify or assess risks. Chopik passed over pre-existing

general value statements and instead used her notes taken at senior executive meetings.

Participants had actually expressed the guiding principles that they wanted to see inform

land use planning.

4. Stakeholder analysis. The context paper should identify stakeholders and provide some

analysis of their roles and interests. Stakeholder views must be represented, directly or

indirectly, in a balance determined by the project sponsor. Stakeholders can be both

bearers and sources of risk. In this case, Chopik drew on background materials to put

together a synopsis and listed prior consultations for reference.

5. Statement of deliverables for the session. What is the practical result that the group will

achieve? Chopik’s answer tied in to her intent to produce a new framework: “The

deliverable is a comprehensive list of risks to the government’s approach to land use

planning, ranked by consensus, with mitigations to inform the new Land Use Planning

Framework.”

The next step was to identify risks in facilitated sessions. Chopik invited representatives

from the ministry of finance and the integrated land management bureau. She avoided

presenting lists of pre-conceived risks. Rather, participants in facilitated sessions

identified their own risks associated with the goals, program activities and values set out

in the context paper. They also reviewed generic risk categories such as financial, legal,

technical, etc., in order to ensure that their efforts would be comprehensive.

When Chopik's group turned their attention to value criteria, they hit on the most

profitable angle of the analysis. They quickly listed many risks that might hinder the way

the government wanted to do business--values expressed as: "certainty for strategic

priorities" (the horizontal coordination problem), "transparency," "timeliness,"

"consistency and defensibility regarding scientific standards," and asserting "the

government's role as final decision maker." The group listed all risks before proceeding

to scoring and mitigation.

Chopik's group used the five-point scales of likelihood (probability) and consequence

(degree of severity) from the government's ERM guideline to score each risk. Multiplying

likelihood by consequence gives the rankings of low, medium, high and extreme. (See

Figure 1 for an example of what scoring a single risk might look like.)

Practitioners can develop scales and descriptors suited to their business. They can refine

the analysis by having the team review existing controls, collective tolerance and possible

mitigation strategies for each risk.

It is crucial that statements of mitigation inform revisions to the policy in question, or

even determine priorities in setting objectives (as in the safe drinking water case

discussed below). Mitigation does not simply mean implementing that which has already

been planned. It means innovative strategies--subject to cost-benefit review--to reduce

the identified risks.

Results can be captured in a spreadsheet with headings denoting risk description,

likelihood, consequence and ranking, as well as controls, tolerance and mitigation. The

result is a risk profile, which can be sorted by score, category or even business unit.

Before considering specialized software, practitioners should experiment by introducing

risk review as an agenda item in regular business meetings and build the risk culture by

demonstrating value.

Writing the context paper helped Chopik to challenge and clarify the government's goals

and objectives, and differentiate them from values. She was pleased with the way risk

analysis allowed her to deconstruct a complex issue more effectively than would have

been done by listing options in a standard briefing note.

Stakeholders--in this case, various ministries, private sector delivery partners, technical

experts and interest groups--had diverse interests, but their specific impacts on policy

were better understood when seen through the risk lens.

Moreover, expert opinion was elicited much more effectively. "Subject matter experts,

when engrossed in detail, are not able to speak to the core elements of a decision at a

strategic level," says Chopik. "Risk analysis lets policy makers ask the hard questions

that go around entrenched positions--and get answers that are practical."

Mitigation proposals, and the resulting new Land Use Planning Framework, now had

clear priorities. These included strategic coordination through regional plans,

comprehensive terms of reference for negotiation and creative reforms to resolve

confrontation.

Safe Drinking Water Systems

Ron Duffel is the executive director of ActNow BC, a major health promotion initiative,

and was recently in charge of ministry of health special projects. There, he led

government through the policy maze for both meat inspection and safe drinking water--

both high priorities on the public agenda.

When the drinking water file came to Duffel, it consisted of high-level statements of

principle and proposed legislative changes, created largely in response to the Walkerton,

Ontario tragedy in 2000 where contaminated drinking water led to seven deaths. Duffel

took the lead to develop priorities for operations. To do so, he launched not only risk

analysis, but also research and province-wide consultation with water systems operators.

These all fed into his recommendations in a 2004 report to assistant deputy ministers.

Participants in Duffel's facilitated sessions included the provincial drinking water officer,

the water use planning manager (environment ministry) and representatives from the

British Columbia water and waste association. The analysis took into consideration

program design and implementation risk.

The highest risks included the confusion among various agencies over definitions of

water systems, inconsistency in interpreting standards and possible resistance to poorly

targeted operator training--all of which would directly impede program goals. The risk of

under-funding to local authorities for implementation also ranked high.

Duffel reached conclusions similar to Chopik's about integrating expert opinion. "The

value [of the risk session] is that it gives audience to different views around a common

issue," he says. "People must hear about the effects of their rules and actions upon

others." Experts begin to understand other views of risk in the context of a common aim

and, once consensus is achieved, the risk profile and mitigations represent their collective

blueprint for change.

The mitigation strategies conceived during these sessions determined the priorities for

Duffel's operational program. They included amendments to regulations, operator

certification improvements, links among previously isolated professional associations,

contributions from a university research group to find technical solutions and a risk pool

organized among water systems operators with the assistance of a commercial broker.

Duffel had always used consultation--this was not new. What risk analysis offered was

"to put more rigor into the process." He explained that policy makers have tended to

discuss risk only informally. Following an unwritten rule, they often make policy based

on unexamined beliefs or institutional group-think.

"There is a movement now towards evidence-based policy development," he says. "When

you have all the critical issues framed and quantified, you can point back and say, yes, we

covered that. This makes your recommendations much harder to refute, especially when

it comes to funding."

Lessons

Complex issues are often not susceptible to statistical analysis or modeling. Instead, they

demand a semi-quantitative method, where diverse viewpoints can be expressed as

discrete risks and assigned a numerical ranking. The resulting risk profile and associated

priorities for mitigation will only be as valid as the:

• accuracy of the context paper, including clarity of goals and values;
• fairness in the balance of views represented in facilitated sessions;
• comprehensiveness in identifying risks;
• degree of consistency in ranking those risks, by consensus, according to common criteria.

If your critics accept your assumptions with regard to goals and values, and the other

criteria are reasonably met, then the results--and your policy proposal--are defensible. Of

course, we all operate in political environments where the executive order will carry the

day, possibly against our best advice. But our job as risk managers is to provide sound

analysis. A structured approach to tough issues will always constitute proactive risk

management.

_________________________________

Edward Robertson, MPA, CRM, is the manager of enterprise risk management for the

risk management branch of the government of British Columbia, Canada.