e-Interview with Dr. Edmund H. Conrow,
Management and Technology Consultant, and Author of
Effective
Risk Management: Some Keys to Success,
Second Edition
Conducted by Dr. Lawrence D. Pohlmann, Strategics Consulting
for the International Council on Systems Engineering
Copyright © by INCOSE 2003, 2005. All Rights Reserved
Posted with Permission of INCOSE
This e-Interview was originally conducted in the fall of 2003. It was developed in concert with a review of
the book Effective Risk Management: Some Keys to Success, Second Edition by Dr. Conrow. The book review was published in the journal of the International
Council on Systems Engineering (INCOSE) Insight, Volume 6, Issue 2, January
2004, p. 44.
The review is also posted here with special permission from INCOSE: Conrow_Review_Incose_Insight_January_2004_Distribute.pdf.
(Minor updates and additions were made in January 2005.)
1) Dr. Conrow, why did you write this
book? What specific objectives did you
have in mind?
Basically, I was not
satisfied with the project risk management guidance that was available. My overall goal was to provide a resource
that was not only easier to use, but that provided a more thorough and advanced
treatment of risk management technology and processes.
Most project risk management
books published in the last 5 to 10 years that I've read include a limited
amount of original and useful information.
They also contain a fair amount of material that is either erroneous
from a mathematical and probabilistic perspective, overly simplistic, inconsistent
with risk management best practices, or speculative. For example, there is often little guidance provided for
implementing risk management on actual programs. (By and large, tools and techniques are the easy part of project
risk management--imHplementation considerations, including organizational and
behavioral factors are the hard part.)
The material is typically written at an academic level, difficult for
the reader to understand and apply, and assertions are often made without the
author providing any justification. In
addition, discussions of risk management associated with technical/performance
are often very weak or absent. Yet it
can be shown that many systems are developed and procured with three primary
variables specified--cost, performance, and schedule.
In my case I've written a
book that is easy to understand, builds upon introductory material, provides
examples and/or theoretical considerations to substantiate key points, and is
easy to apply to either a new or existing program.
2) Who is your intended audience? Who do you think should use your book?
The book was written for
project managers as well as systems engineers, risk managers, and other
practitioners across a wide variety of industries. I've tried to strike a balance between too much versus too little
detail, and between information that is too technical\ versus
non-technical. Although a lot of the
material is applicable to DoD and NASA programs, I made special efforts to make
sure that the guidance provided was also directly applicable to a wide variety
of commercial programs across industries.
For example, risk management process steps, risk analysis methods, a
structured approach to developing risk handling strategies, and barriers to
successfully implementing risk management and methods to overcome them are almost
universally applicable. They can be
used across a wide range of programs and industries, and across a wide range of
program sizes. While many of the
lessons learned apply to "high tech" programs, I've also used the
material in an industry that has absolutely zero technology and technical risk,
and on very small programs.
3) When people ask you “what is your book
about?,” what do you say?
The book provides an
introduction to project risk management then moves to implementation topics for
each risk management process step as well as across process steps. It includes more than 700 tips to succeed
and traps to avoid for implementing project risk management. Many of these tips and traps have been
derived from work on actual programs.
The book is formulated in such a way that a one or two sentence summary
is provided after many tips and traps to help managers and others with limited
time to encapsulate the key message of a topic and get to the "bottom
line." At the same time
substantial levels of are routinely provided.
These discussions help the systems engineer, risk manager, and other
practitioners who either want to or need to, because of their position, achieve
a deeper understanding of the material.
4) Essentially all good books on systems or
software engineering include discussions to various degrees on risk
management. What do you consider unique
or different or important about your book?
First, the tips to succeed
and traps to avoid are organized by risk management process step rather than
being intermixed. [In cases where the
item spans more than a single process step it is included in a separate risk
management process-level chapter (Chapter 3).]
Second, the items are
written in such a way to enable managers to extract key information as well as
systems engineers, risk managers, and other practitioners to focus on more
extensive details.
Third, a number of
implementation examples are given, and key organizational and behavioral
considerations are discussed. These
topics are often weakly covered or absent from a number of project risk
management books.
Fourth, many of the tips and
traps are gleaned from actual programs. I've also attempted to write each tip
and trap presented in the book in such a way that it could be considered by the
reader for their specific program.
Fifth, the concepts
presented are mathematically and probabilistically sound, yet approachable to
the reader to permit easy implementation.
Sixth, an extensive survey
and statistical analysis associated with words of estimative probability (e.g.,
high) is included. Often times authors
or analysts discuss such words and ascribe probability levels to them (e.g.,
high probability = 0.8) without having any
basis for the specified probability levels associated with the words. This can lead to substantial errors when
applied to a risk analysis.
Seventh, there is a large
body of literature that incorrectly performs mathematical operations on results
from ordinal "probability" and consequence of occurrence scales. I have included an extensive discussion of
why such practices are flawed and lead to erroneous results, and also provided
sound methods for estimating risk levels when risk scales are used.
Eighth, while many in the
project management community intermix opportunity and risk, my book includes a
carefully written Appendix by Dr. Robert N. Charette on why opportunity should
not be merged into the definition of risk, and why opportunity management
should not be pursued as a mirror to risk management.
5) I was pleased to see the expanded table of
contents and index in the second edition.
What else do you view as the most significant changes in this second
addition?
First, I've provided
top-level guidance for implementing project risk management in Chapter 2, Sec.
III, "Some Risk Management Implementation Guidelines." This section also serves as a roadmap to the
remainder of the book, as it points to specific sections containing key
information.
Second, an important
discussion authored by Dr. Robert N. Charette on whether/not to include
opportunity as part of risk management is contained in a new Appendix to the
book (Appendix E). Dr. Charette points
out numerous flaws in the arguments advanced by proponents of such positions,
and argues convincingly that including opportunity in the definition of risk is
unwarranted as is developing "opportunity management" as a parallel
to risk management.
Third, I've added more than
450 additional tips to succeed and traps to avoid for implementing risk
management on an actual project.
6) How would you suggest people in the systems
engineering community use your book?
My goal was to structure the
book so that it can serve three major types of use. First, it contains introductory materials that are designed to be
useful for those who are just learning about risk management. Second, the book contains detailed and
advanced treatment of all phases of the risk management process. These are intended to be used by those who
have direct responsibilities in the area of project risk management – especially
on complex programs. Thirdly, I’ve
included a variety of navigation aids to make it a good reference book – that
is a book in which it is easy to locate the risk related topic of current
interest.
The book can be used in the
traditional arenas of systems analysis and control (e.g., establishing,
implementing, and verifying the risk management process) and requirements
flowdown (e.g., risk identification followed by risk analysis, handling, and
monitoring as warranted). The
guidelines for risk planning can also be used to write a risk management plan
and/or a risk management section in the corresponding systems engineering plans
(e.g., SEMP). The material is also
suited for function, matrix and project implementations for systems
engineering. Here, the risk management
process fundamentals (e.g., process steps, their order, and functions per step)
are often the same, but what organization owns the process and how it is
implemented may vary widely.
Because there are a large
number of tips and traps (more than 700), I would expect users to benefit from
repeatedly browsing sections of the book that may be relevant to their task at
hand.
7) Your background includes extensive
experience in the aerospace projects sector.
How do you see project risk management as having evolved in this sector
over the last five to seven years?
Let me first state that
project risk management processes and techniques and tools have been evolving
for decades. More recently on the DoD
side there was a major evaluation of risk management performed in
1996-1997. This was an outgrowth of Dr.
Paul Kaminski's (Under Secretary of Defense for Acquisition and Technology) 4
December 1995 memo that established Cost As an Independent Variable
(CAIV). As part of that memo Dr. Kaminski
encouraged the DoD to examine and enhance its risk management process as
warranted. A major effort involving
roughly 20 to 30 individuals from the Office of the Secretary of Defense, Air
Force, Army, Navy, Defense Acquisition University (DAU), Defense Systems
Management College (DSMC) and other organizations led to an enhanced risk
management process, most readily available to the public as the "Risk
Management Guide for DoD Acquisition," through the DAU web site. I served as an unpaid technical advisor and
consultant to both the Air Force and OSD on this study, and subsequently to DAU
and DSMC to assist in updating the "Risk Management Guide for DoD
Acquisition," now in the Fifth Edition, Version 2 (2003). The resulting document can be downloaded
free of charge by anyone at:
Risk Management Guide
for DoD Acquisition, Fifth Edition, Version 2
This document is widely used
in government and industry, and often far outside of DoD programs. My book can be viewed as an adjunct to this
DoD guide, in terms of tips and traps for each process step, implementation
guidelines, and especially advanced topics.
In terms of individual
programs, the average implementation effectiveness has likely increased (at
least somewhat). However, there is
still a substantial standard deviation of implementation effectiveness across
programs. It is not uncommon that some
programs exposed to the same reference material will embrace risk management
much more so than others even though they are in the same organization.
8)
How is your
crystal ball working? What kinds of
evolution in project risk management processes do you see in the next five to
seven years?
On the positive side,
enlightened organizations will do three things:
·
First,
they will use a refined project risk management process for most if not all
programs (albeit the specific details may be different on a case by case
basis).
·
Secondly,
they will place increased emphasis (as warranted) on explicitly evaluating and
managing integration risks (e.g., hardware/hardware, hardware/software, and
software/software) and systems level risks (both for a single system and across
systems).
·
Thirdly,
they will more consciously focus on implementation considerations throughout
the development process.
On the negative side, there
is the distinct possibility that, as Dr. Robert N. Charette has observed, risk
management may go the way of disco.
This is because risk management is too frequently overhyped and oversold
by consultants and trainers with little or no experience and accountability on
making risk management work on an actual program. This can lead to havoc on actual programs where the opportunity
cost in terms of un-recoverable schedule and dollars may be very high. For example, in one case I began working on
a program that was in substantial trouble.
The very first day I was confronted by program management and told that
the last project risk management consultant primarily focused on developing a
schedule risk analysis rather than risk management. Within a short period of time I had documented more than 25 risk
management-related deficiencies with recommended changes—many of which existed
when the prior consultant was engaged.
Here the opportunity cost was huge—the wasted time and ineffective
treatment of both risks and problems could never be recovered.
Also, it is commonplace that
project risk management trainers have never held a long-term risk manager
position on an actual program, and the courses they offer have little/no
implementation insights. Caveat
Emperor!
9) Briefly, how would you compare and contrast
project risk management challenges and processes in the DoD, other Government
agency, and commercial sectors?
In some respects the project
risk management process and implementation has been stronger in DoD programs
than many other government agencies and commercial programs. This may seem counter-intuitive to some
given that $500 toilet seats may come to their minds. But the perception that anything done by DoD can’t be “cutting
edge” or is overly bloated is incorrect.
The DoD focus on risk management is the result of both requirements
(e.g., specification in Directives, Instructions, and source selection), good best
practices (e.g., “Risk Management Guide for DoD Acquisition”), and desired
advances in the state of the art sought for many development activities.
While you may "get
away" with a relatively simple, unstructured risk management process on a
program far below the state of the art, doing so on a program that stretches
the state of the art will likely lead to substantial cost, performance, and
schedule issues, if not program cancellation without good risk management. Some government agencies have either
developed their own risk management processes or borrowed from existing
processes. In some cases, I have found
these risk management processes are relatively weak compared to the DoD
process, both in terms of process characteristics and implementation considerations. In addition, with one agency the "not
invented here" syndrome is routinely practiced despite the fact that the
organization that developed their risk management process was disbanded several
years ago. I've also looked at project risk
management material offered by large and reputable commercial firms, ranging
from industry leaders to traditional consulting firms. By and large, this material is overly
simplistic, and far less comprehensive than the readily available DoD material
that is not copyrighted and available free of charge. There is also typically little/no information provided that
assists the reader in implementing risk management on their program--this often
appears to be the "hook" from those peddling their services.
10) During the last several years, we have
experienced continued and increased emphasis on process maturity models in
software engineering, systems engineering, acquisition, and other areas. How has this helped risk management
technology? How has it hurt?
I don't believe that process
maturity models have had much impact on project risk management to date. Risk management is subject to the same
concerns as with process maturity concerns for other disciplines. On the plus
side they may cause some organizations to take a hard look at different risk
management processes implemented on various programs (e.g., are the process
variations across programs reasonable) and standardize on a given process even
if implementation differences may exist.
On the minus side the criteria for assessing maturity, in the worst
case: 1) may be subjective,
counterintuitive, or erroneous; 2) may be oriented or tied to a specific risk
management process; 3) may be biased towards the experience of those that
developed the criteria (and not how risk management is broadly practiced on
actual programs); 4) may not suitably capture organizational and behavioral
characteristics; 5) are not reflective of potential contractual requirements
external to the organization that may dictate a specific implementation or
methodology for a given program; 6) may not scale with the scope of a program,
etc. For example, in one risk
management model at the highest level of maturity there is mention of project
risk management being top-down from upper management. I agree that this is highly desirable in a mature risk management
process. But it is insufficient, thus
the model is incomplete (or incorrect). There is no mention that project risk management also needs to be bottom-up
from the workers. Without this the
process will be ineffective.
I view risk management
maturity models at the present time as a work in progress--worth observing but
not accurate or mature enough to mandate or apply on actual programs else the
results may be erroneous.
11) You have provided a large number of
heuristics in your book, literally hundreds.
This will be intimidating to many.
How can people start to use your guidance without being overwhelmed?
My goal was to provide a
thorough treatment of project risk management.
I clearly recognize that good risk management can be complex in that
lots of things may need to be considered.
Thus, I provided a large number of heuristics – or tips and traps. Naturally, I do not expect readers to try to
understand and practice these all at once.
Rather, I suggest the reader
first examine the guidance for implementing risk management in Chapter 2, Sec.
III, "Some Risk Management Implementation Guidelines." This section also serves as a roadmap to the
remainder of the book, as it points to specific sections containing key
information. Also see Chapter 3, Secs.
IX ("Some Risk Management Implementation Considerations), XII
("Organizational Considerations and Risk Management Process
Implementation"), XIII ("Support for Risk Management Within the
Program"), and XIV ("Some Behavioral Issues Associated with
Implementation"). Additional, and
more detailed information can then be found by consulting the Table of Contents
and Index.
12) What do you see as the most common “errors”
in project risk management? And why?
First, there is often blind
faith that the existing risk management process is "good enough,"
when it may, at least in some cases, be substantially flawed. Continued use of a risk management process
does not verify or validate it. These
same organizations are often opposed to performing an independent review of
their process or incorporating enhancements that will improve the process
(balanced against resources available).
Second, there is generally a
much greater focus on tools and techniques (including advanced tools and
techniques) than implementation considerations. Yet, tools and techniques are often not valuable if there are
top-level issues associated with process implementation (e.g., limited upper
management participation). And
attempting to implement advanced tools and techniques when the process is
missing a step or if you have fundamental organizational and behavioral issues
is generally problematic and may lead to substantial opportunity cost. Similarly, there is often a much greater
emphasis on risk analysis than all the remaining process steps combined, yet it
is only one of five project risk management process steps (risk planning,
identification, analysis, handling, and monitoring).
Third, it is common that
mathematical operations are performed on values from ordinal probability and
consequence of occurrence scales. This
will almost always lead to erroneous results.
This flawed practice was promulgated by a major contractor through DoD
and grew considerably for almost 10 years (1980s) before I challenged it in the
early 1990s. Another 10 years have
since passed and only now has this erroneous practice been curtailed. (Unfortunately, once it appeared in DoD documents
it began to take on a life of its own.)
I've included both a proof of why such mathematical operations are
erroneous and an irrefutable example that has an average error of more than 600% in my book (Chapter 6,
Section IV.B)!
Fourth, there are
mathematical forms of risk that are inherently wrong, yet widely used without
recognizing the errors present. Simply
stated, if you must perform mathematical operations, use Risk = Probability *
Consequence, and not another representation.
13) If I tell you that I have been recently
assigned to be the risk manager in a new project, what kinds of advice would
you give me?
I'd first recommend
examining contractual and other requirements for project risk management,
organizational best practices, applicable "command media" (if any)
and the (organizational and behavioral) implementation environment.
As mentioned in question 12
(above), use the top-level guidance for implementing project risk management
contained in Chapter 2, Sec. III, "Some Risk Management Implementation
Guidelines," as well as the references to other sections of the book (as
warranted) to outline the desired process.
Also examine Chapter 3, Secs. IX ("Some Risk Management
Implementation Considerations), XII ("Organizational Considerations and
Risk Management Process Implementation"), XIII ("Support for Risk
Management Within the Program"), and XIV ("Some Behavioral Issues
Associated with Implementation") to assist with roles and
responsibilities, and organizational and behavioral considerations.
Beyond the references I'd
say that it's important to realize that risk management is not a "silver
bullet" to save a program and the process itself is not "rocket
science." However, it must be
logical; structured; repeatable; well integrated with higher, similar, and lower-level
processes; and continuously practiced.
A substantial key to success
is for the project manager, deputy project manager and other key managers to
use risk management principles in his decision making; else working level
personnel will question the commitment to risk management and not truly support
it. However, even with upper management
support it is important to realize that "progress will take time" –
effective risk management is something that doesn't happen overnight. Fortunately, and effective risk management
process can be built up over time. Continuing success leads to increased
interest and practice on the part of other project personnel.
14)
A term we are
seeing more of is “systems of systems.”
What kinds of risk management challenges do you see in this context?
One key consideration is the
integration of often a diverse set of systems of varying types and levels of
maturity. This often points to a series
of integration risks and system (or “top level”) risks not typically seen on a
single program, or if present much smaller in magnitude. It may also present the challenge of
integrating legacy systems with those that are presently under development and
in some cases entirely new risk categories than what existed on individual
programs. From the implementation
perspective there will also likely be more than one risk management process in
place and/or under consideration, and widely varying views of risk management
across key managers and stakeholders.
This is a potentially challenging environment for implementing risk
management, but the risk management principles discussed in my book also can
also be tailored and applied to “systems of systems.” I specifically discuss systems of systems considerations in
greater length in my February 2005 article contained in CrossTalk, The Journal
of Defense Software Engineering (“Risk Management for Systems of
Systems”). The reader can view and
download this article beginning in February 2005 from the CrossTalk web site:
CrossTalk Web Site
Conrow Article
or directly
from this site:
Conrow_CrossTalk_Article_February_2005.pdf
15) Do you
foresee a third edition or writing another book?
I'm on a long vacation from
book writing and not contemplating another edition of this book or writing
another book beyond updates to the project risk management chapter I’ve written
in Harold Kerzner’s best selling project reference book, “Project Management: A Systems Approach to Planning,
Scheduling, and Controlling,” Wiley, and related collaborations.
16) In closing, what final comments would you
like to add about your book?
The book serves as both a
guide for developing and implementing a risk management process, as well as a
comprehensive reference source to use during the course of the program. I encourage the reader to apply the material
to their programs, and leave a legacy that builds upon the information provided
for current and future project managers, systems engineers, risk managers, and
other practitioners.
Lastly, I would like to
thank INCOSE both for publishing a review of my book and for sponsoring this
e-Interview.
And thank you Dr. Conrow. As our
systems and processes continue to increase in complexity and interdependence,
we can view good project risk management as more important than ever. Lastly, let me reiterate that I found your
book to be, in my opinion, the most thorough and advanced treatment of risk
management that I am aware of. It is a
journeyman’s reference book. I highly
recommend it to anyone with direct responsibilities in that area of risk
management. Thank you again!
|
Links to Project Risk Management Services |
||
|
Second Edition of Dr. Conrow's Risk Management Book with Reviews |
||
|
Risk Management e-Interview by International Council on Systems
Engineering |
||
|
Links to Other Services |
||
This web site owned and operated by:
|
Dr. Edmund H. Conrow, CMC, CPCM, CRM, PMP |
|
Reproduction or any other use of this material is prohibited without written permission. |
This
material is Copyright © 2000-2007 by Edmund H. Conrow, All Rights Reserved
|