10% \/ P = 0.3 O Labor cost = 0.8 x lognormal (600,25) Material cost = 0.5 x Triangular(2000,3000,4200) Equipment cost = 0.75 x lognormal(4000. 650) High Winds \/ p = 0.2 Ice Cover < 10% p = 0.7 o Labor cost = 1.3 x lognormal (600,25) Material cost = 1.4 x Triangular(2000,3000,4200) Equipment cost = 1.3 x lognormal (4000,650) Fig. 5.9. Assignment of Multiplicative Factors to Primary Variables Each of the stages identified above are described in detail in the following section. 5.5.2 Derivation of Influence Diagram for the Problem The first step of the analysis of Type B risks is the preparation of an influence diagram that shows the relationship between the external factors and the target variable. In some cases there might be uncertain events which influence the external factors themselves leading to an influence diagram having three or more levels of nodes as shown in figure 5.10. The process of deriving the influence diagram for the problem using information obtained from experts is described in chapter 6. Thereafter, the corresponding probability tree is drawn up and probabilities assigned to each branch except to the final level of nodes which depict the decision variable. In the case of independent factors, only the prior probabilities corresponding to each state of the individual factors need to be assessed. In the event of dependency between factors, conditional assessments will be required. 146 Fig. 5.10. Influence Diagram with a Hierarchy of Chance Nodes The branch which corresponds to all external factors being inactive is considered as the base case. However, any scenario can theoretically be considered as the base case. Therefore, in the absence of a scenario in which all the external factors are inactive, another scenario may be chosen as the base case. In the example of a fuel spill clean up, the scenario in which a minor spill occurs when thick ice and high winds are not present may be considered as the base case. 147 5.5.3 Assignment of Probability Values to the Primary Variables A suitable decomposition method is chosen to model the decision variable as in Step 1 of the analysis of Type A risks. This leads to an identification of the primary variables that need to be considered in the analysis. Probability estimates for the 5 t h, 25 t h, 50 t h, 75 t h and 95 t h percentiles are elicited from experts for each of the primary variables considering only the base case scenario of the problem. The uncertainty of each primary variable is then quantified adopting the method described for risk type A. The correlation coefficients are assessed as described in section 4.3.3 of the previous chapter. 5.5.4 Assignment of Adjustment Coefficients In order to utilize the benefits offered by decomposition, individual factors which account for the percentage change in the value of the primary variable relative to the base case due to the effect of externalities will be defined for each primary variable. These will be termed as adjustment coefficients instead of Cost Factors as used by Diekmann and Featherman (1998), in order to avoid confusion with the term, external factor. It will be assumed that the change in the distribution of a variable due to an external factor can be modeled by multiplying it with a suitable coefficient. As in the probability factor model introduced by Diekmann and Featherman (1998) the multiplication of the distribution by a coefficient will result in the expected value and the standard deviation of the base case distribution being increased by a factor equal to the coefficient. However, the shape of the distribution would remain unchanged. The inability to model any changes in the statistical shape of the distribution may be considered as a disadvantage of using a single point estimate to model the effect of 148 external factors. A more detailed analysis would require the adjustment coefficients to be expressed as a distribution function instead of a single point estimate. This would be impractical in terms of the detailed information that would be required. Additionally, this approach also cannot model the transformation of a deterministic variable into a probabilistic variable due to the effect of external factors, or vice versa. However, the correlation matrix between the primary variables will remain unchanged subsequent to the application of coefficients since the value of the rank correlation coefficient is invariant under transformations that produce strictly increasing distributions. The probability distributions that are obtained by multiplying the base distribution by a coefficient is simply a linear transformation of the original distribution. Therefore, the same correlation matrix can be used in all scenarios pertaining to the problem. This can be considered as an advantage of the approach. The example of the fuel oil spill clean up can be used to illustrate the advantages of using primary variables instead of derived variables in the analysis. If the decomposition of the clean up cost is carried out on a general level as given in equation (4.13), the clean up cost can be expressed as Clean up cost (C) = [ Mobilization cost + Labor cost + Material cost + Equipment cost + Subcontractor cost ] (5.14) In this case overheads, profit margins and market conditions have been neglected for simplicity. The subcontractor cost in the event of a fuel spill would be the cost of reimbursing the Coastguard and other agencies for their efforts in controlling the spill. 149 If the probability factor model was used, the distribution of the clean up cost would first be evaluated using Monte Carlo simulation for the base case corresponding to a minor spill, when the effects of wind and ice conditions are non-existent. In obtaining a distribution for the case of a minor spill with ice cover being more than 10%, the cost distribution for the base case would merely be adjusted by a factor, Fj. The benefit of improved accuracy brought on by decomposition is not utilized in making such an assessment. The expert providing the factor Fj would have to collectively assess the impact of the ice cover being more than 10% on the individual primary cost components and provide an estimate of a factor which accounts for this collective impact. An approach whereby the expert could assess the individual impact of the ice condition on the individual cost component is likely to be more accurate than a collective estimate as the complexity of the mental computation the expert would be required to carry out is reduced. Therefore, the use of individual coefficients that take into account the change in the primary variables due to the external uncertainties will be obtained and used in the proposed analysis method. However, the need for a function that takes into account the combination of several external uncertainties remains. A method of incorporating the effect of several external uncertainties acting in combination on a variable is necessary in order to assess the percentage change in the values of the variable due to two or more external uncertainties which act upon it at the same time. For the case when the incremental changes caused by two or more external uncertainties are mutually exclusive, the combined effect is given simply by the addition of the percentage changes. This can be illustrated as follows. 150 If (FA - 1) is the percentage change expected in the number of personnel needed for an activity S due to an external uncertainty A, and if (FB \u2014 1) is the percentage change expected in the number of personnel needed for S due to an external uncertainty B, and if s was the original number of personnel when A and B were inactive then, The change in personnel due to A only = s x (FA - 1) (5.15) The change in personnel due to B only = s x (FB - 1) (5.16) If the increases in usage of the personnel due to A and B are mutually exclusive, then Personnel needed when both A and B occur = s x (FA - 1) + s x (FB - 1) (5.17) Therefore, Percentage change in S due to both A and B = (s x (FA -1) + s x (FB -1)) \/ s = (F A -1) + (F B -1) (5.18) However, in general the incremental effects might not be mutually exclusive. A new modeling paradigm is introduced in this thesis which enables the analysis of the combined effects of multiple external uncertainties. This modeling paradigm enables 151 the combined effect of the external uncertainties to be quantified irrespective of their degree of mutual exclusiveness. Modeling paradigm for the analysis of the combined effects of multiple external uncertainties Consider a case when thick ice conditions are present during the clean up of an accidental oil spill. Assume that the base case scenario constitutes of low wind conditions and an ice cover of less than 10 percent. Due to the thick ice conditions, the value of a variable such as labor productivity would decrease relative to the base case as clean up crews based on boats would find it difficult to maneuver around the ice. If high wind conditions were present instead of thick ice, the productivity would also decrease relative to the base case due to difficulties in navigation that arise due to high winds. Therefore, it can be seen that both high winds and thick ice conditions decrease productivity when considered individually. However, in a situation when both winds and ice are present the decrease in productivity might not be equal to the sum of the individual decrease amounts. This is due to the fact that in some cases the ice would have a moderating effect on the waves induced by high winds thus making navigation easier than in a case when high winds are present in isolation. Therefore, while the difficulty of navigating through ice infested waters remains, the additional difficulty caused by high winds could be of a reduced magnitude. This physical phenomenon can be thought to mirror the behavior of sets in mathematics. Consider two sets, A and B. The union of A and B is not simply (A + B) but {A + B - (A n B)}. This is analogous to the behavior of the effect on productivity caused by ice and wind. If the 152 effect of ice on the productivity is considered as a set of minute effects instead of a single effect and if the effect due to the wind is considered in a similar manner, their union or their combined effect can be thought as the union of the two sets of minute effects. The difference between the sum of their individual effects and actual combined effect can be thought of as the intersection between the two sets of minute effects. This mathematical analogy is used in this thesis to model the combined effect of several external factors upon a variable. The analogy of sets is applied to two additional scenarios in order to test the robustness of the analogy. The first scenario consists of an example in which two external factors bring about opposite effects in a variable. Consider the case when the increase or reduction in the value of a variable when two external factors are active, is not equal to the difference between the individual effects of the two factors. The behavior of the variable Material Cost, in the example of an oil spill clean up is used as an illustration. Should high winds (W) be present in isolation the material cost would be increased as more sorbents would need to be employed to collect the spilt oil. Should thick ice (I) be present in isolation, the material cost would be reduced significantly as clean up consists of simply burning the spilt fuel. However, should winds and ice both be present in combination, the fuel would still be set on fire thus reducing the material cost sizably in this case as well. In this case the reduction might be greater than the value obtained as the difference between the two individual effects. Applying the analogy of sets to the effects of the external factors yields ( W u I ) = W + I - ( W n I ) (5.19) 153 If however, the value (W u I) is in reality not equal to W +1, then (W n I) should have a value other than zero. However, such an explanation seems unintuitive in this instance since wind and ice cause effects in the opposite direction and thus should have no common effects. Positive effects increase the value of the variable from its base case value by adding onto its value, while negative effects decrease the value of the variable. Therefore, negative and positive effects should be distinctly different from each other, thus leading to the statement that they share no common effects. However, this scenario is easily explained by looking at the effect of the external factors upon the material cost in detail. In the event of thick ice being present, the need for material such as sorbents and chemical dispersants in combating the oil spill is nullified. However, a need for igniters to bum the fuel now arises. Therefore, the material cost has actually decreased and then increased with respect to its base case value. The overall effect is a decrease of the material cost. Recognition of the fact that the overall effect is considered of a positive as well as a negative component provides an explanation of the interaction between the two external factors in this case. Let the negative component of the effect of ice on the material cost be IN and the positive component be IP. Then the overall effect of ice denoted by I would be equal to (IP + IN). In this example L would consist of the percentage increase in cost caused by the need for igniters while IN would be equal to the percentage decrease in material cost caused by foregoing the use of sorbents and chemicals. Similarly the total effect of wind (W) can be written as W = W P + W N . The combined effect of wind and ice can now be written as 154 (W u I) = (WP + WN) u (IP + IN) (5.20) As the intersection of a positive effect and a negative effect is zero, equation (5.20) effectively reduces to Equation (5.22) explains the scenario under consideration in which the combined effect of two externalities that create opposite effects is not equal to the difference between the two effects. (W u I) in this case need not be equal to (W +1) as {(WP n IP) + (WN n IN)} can have a value that is not equal to zero. The second scenario considered reflects an instance in which the change in the value of a variable due to a combination of two factors having an effect in the same direction is more than the sum of the individual changes caused by the two factors. An example of such a scenario would be the impact of design changes and regulatory changes upon the productivity of an excavation operation at a hazardous waste remediation site. Assume that design changes require that a large part of the excavation be carried out in an area with a steep gradient as opposed to the fairly level area associated with the base case. In this case the productivity of the equipment WuI = (WP + Ip + WN + IN)-{(WPnIp) + (WNnIN)} (5.21) which can be written as (W u I) = (W +1) - {(WP n IP) + (WN n IN)} (5.22) 155 operators would decrease due to the increased difficulty of the excavation. Let the regulatory changes that may be forthcoming require that equipment operators wear cumbersome safety clothing. This would also decrease the productivity of the operators by a certain amount. In considering the case when design changes and regulatory changes occur together, the operators would have to wear cumbersome safety clothing while carrying out a complex excavation. A possibility exists that the productivity loss in such a case could actually be greater than the sum of the two individual reductions. Such a scenario could also be explained by considering the set analogy at a level in which the positive and negative components of the effect of an external factor are considered separately. Let D denote the set of effects caused by design changes and R denote the set of effects caused by regulatory changes. Adopting notation similar to the previous scenario, it can be written (D u R) = (D + R) - {(Dp n R P) + (DN n RN)} (5.23) The term (D u R) which is the combined effect of the external factors would be greater than (D + R) when {(DP n R P) + (DN n RN)} is negative. Therefore, the scenario can also be successfully modeled using the analogy of sets. The model in this case suggests that the intersection between the negative components is greater than the intersection between the positive components of the effects of the two external factors. It is noted that the adjusted product function developed by Diekmann and Featherman (1998) does not accommodate the analysis of scenarios of this type. 156 Therefore, it is suggested that the use of sets can be employed in drawing a robust analogy to the interaction between the effects of various external factors in a variety of different situations as illustrated by the scenarios considered above. A calculation procedure based on the use of sets is therefore used in obtaining values for the adjustment coefficients in the case of multiple external factors acting in combination upon a variable. The following premises are introduced in order to provide a robust framework for the development of the calculation procedure. Premise 1 The effect of an external uncertainty on a variable can be expressed as a set of a finite number of mini effects. Two mutually exclusive types of these mini effects are assumed to exist, mini effects that cause positive changes in the variable and mini effects that cause negative changes in the variable. The two types are defined to account for increases and reductions in the value of the variable. Each of these mini effects is assumed to cause the variable to increase or decrease its value by a single percentage point from its base case value. For example, i f ice conditions decrease the material cost by 25% it is assumed that this is the overall effect of several separate effects, the net number of which is 25 negative mini effects. The number of effects has been tied to the percentage change for simplicity. Alternatively, it could have been assumed that the 25% change is caused by the sum of 250 different effects each of which causes a 0.1% change in the variable. Since each of these mini effects is assumed to increase or decrease the value by a single percentage point, the individual mini effects are similar in magnitude. In the above example, the value of a 25% decrease could be the net value of 45 negative mini effects and 20 positive mini 157 effects. The concept of a mini effect has been introduced to tie the physical phenomenon of a change in a variable, with the concept of sets that exists in mathematics. The mini effects are thought of as the elements of the set. Premise 2 The impact of each mini effect is considered to be independent of the value of the variable. This premise allows the percentage change in the variable for a particular scenario to stay the same whatever the value of the variable may be. As an example, application of this premise to the effect of high winds on the material cost would imply that the material cost would increase by 80% due to high winds irrespective of whether the material cost for the base case condition was $ 25,000 or $ 75,000. This premise plays an important in role in dealing with probability distributions of variables. Since a probability distribution encompasses a range of values of a variable, all the values can be multiplied by the same adjustment coefficient based on this premise. However, several disadvantages of using a single point coefficient exist as has been discussed previously in section 5.4.4. The calculation procedure for adjustment coefficients that is based on the premises introduced above consists of several steps. In step (I), the percentage changes in a variable due to the external factors are evaluated by considering the scenarios in which the external factors act individually. An example would be the percentage change in productivity due to high winds, when all the other external factors are at their base case values. The value of this percentage change is obtained from experts through a suitable elicitation process as described in chapter 6. The percentage change 158 values obtained in this step will be termed as individual adjustment coefficients to reflect the fact that they are due to individual external factors. Let A be the effect brought on by an external factor A. A will consist of a set of mini effects that cause the change in the variable under consideration. An example would be an instance where winds cause a 25% decrease in productivity. In this case A would consist of a set of mini effects whose net impact is a 25% reduction in the value of the productivity. In step II of the process, the factors are considered in a pair-wise manner. The percentage change due to each pair of the external factors will be elicited from experts for all pairs of external factors that can feasibly exist together. Since the expert has to consider the combined effect of only two uncertainties at the same time it is assumed that the assignment of an accurate coefficient is within the capability of an expert. An example would be the percentage change in productivity in conditions of high winds and thick ice, when all the other external factors are at the base case values. The percentage change values obtained in this step will be termed as binary adjustment coefficients to reflect the fact that they are due to pairs of external factors. Let A and B, be two effects brought on by two external factors A and B. Invoking the analogy of sets, the combined effect of A and B is given by (A u B), Therefore, it can be written, {AvB) = (A) + (B)-{Ac\\B) (5.24) The percentage value obtained from an expert in this step is therefore equal to (A u B). In step III, adjustment coefficients are obtained for scenarios in which 3 or more external factors are acting in unison. While theoretically such values can also be 159 elicited from experts, the accuracy of the estimates would decrease as the number of external factors increase. This is due to the fact that the mental computation an expert would have to carry out in order to provide a value for the adjustment coefficient while accounting for the interrelationship between factors would become very complex as the number of factor increase. The likelihood of an estimate provided by an expert being erroneous could be considered as being high in such a scenario. Therefore, the values of adjustment coefficients that relate to scenarios involving 3 or more external factors are calculated using the information obtained in step I and II. If A , B and C are 3 external factors then their combined effect given by (AKJBUQ can be expressed as, (A u B u Q = (A) + (B) + (O - (A n B) - (A n Q - (B n Q + (A n B n Q (5.25) by invoking the analogy of sets. However, the term (Ar\\Br\\C) contains only the positive and negative mini effects that are common to A, B and C. Therefore, this term will tend to have only a very small number of mini effects. Therefore, it is assumed that the term (Ar\\Br\\C) can be neglected without a significant loss in accuracy. Equation (5.25) can now be written as 04ufluQ~ (A) + (B) + (Q-(AnB)-(AnQ-(BnQ (5.26) 160 The assumption made above is used to simplify the calculation process and the data requirements in obtaining the value of the adjustment coefficients for scenarios that involve 3 or more external factors. The assumption can be expressed in general terms as (Ai n A2c\\ A3 Ak.inAk)~0 (5.27) where k> 3. The value of (A) and other individual coefficients in equation (5.26) can be obtained from step I, while values for (A n B), (B n Q and (Ar\\C) can be obtained using the binary adjustment coefficients obtained in step II. For example (A n B) can be obtained as (AC\\B) = (A) + (B)-(AKJB) (5.28) Substituting for values such as (A n B) with values obtained from equation (5.28) in equation (5.26), it can be written 0 4 u 5 u Q ~ (AVB) + (AKJQ + (BKJC)-(A)-(B)-(C) (5.29) A value for (A u B u Q is obtained in this manner using equation (5.29). 161 Similarly, in the case of four externalities A, B, C and D the combined effect is given by (A{JB(JC{JD) = A+B + C + D-(Af]B)~(Bf]C)-(Af]Q-(Af]D) -(Bf]D)-(Cf]D) + (AnBf]Q + (Bf]Cf]D) + (CnDf]A) + (Df]AnB)-(Af]BC\\Cf]D) (5.30) Neglecting terms that involve the intersection of the effect of 3 or more variables (A U B U C U D) can be expressed as (A\\JB[JC\\JD) r A + B + C + D-(Af]B)-(Bf]Q-(Af]Q -(Af]D)-(Bf]D)-(Cr\\D) (5.31) Substituting values obtained from step II, for (A f] B), (B f) Q, (A f] Q, (A f] D), (B D D) and ( C f l - D ) , it can be written that (A[jB[jC[jD) ~ (A[jB) + (B[jQ + (A[jC) + (A[jD) + (B[jD) + ( C U D) - 2A -IB - 2C - 2D (5.32) Similar results can be obtained for even larger numbers of external factors. The adjustment coefficients obtained in this manner model the percentage change the value of a variable undergoes as a result of the external factors. Therefore, in order to obtain the value of a variable under the changed circumstances the original value is multiplied by' 1 + adjustment coefficient'. 162 5 . 5 . 5 The Probability Distribution of the Decision Variable The adjusted distributions of the primary variables will serve as input to a Monte Carlo simulation carried out for each combination specified by the probability tree of the problem. An equal number of iterations are carried out for each scenario represented by the branches of the probability tree. Let the number of iterations be N . Then a sample value generated in carrying out a Monte Carlo simulation for a particular branch of the probability tree, with a joint probability of P A will have a probability of occurrence of P A x (1\/N) with regard to the final distribution of the decision variable. The moments of the distribution of the decision variable are calculated using the values generated from each branch along with their associated probabilities. Since this distribution is an aggregate of several individual distributions with considerable differences in spread, the result would be a multi-modal probability distribution in most instances. Thus, it would not be possible to characterize the decision variable with the aid of a standard uni-modal statistical distribution. Therefore, the uncertainty surrounding the decision variable is represented by the moments of its probability distribution. 5.6 Application of the Quantification Methodology The use of the methodology described above is illustrated by considering an example of assessing the duration of excavation in an environmental remediation project. The primary step in the analysis methodology is the derivation of the influence diagram for the problem utilizing expert opinion. Let figure 5.11 be the outcome of this step, considering a hypothetical project in which the excavation duration is affected by the external factors of (1) regulatory changes; (2) design changes requested 163 by stakeholders; and (3) the outcome of a lawsuit filed by a member of the public. Regulatory changes and design changes would bring about a change in the volume of excavation thus affecting the duration. The change in the difficulty and complexity of the excavation caused by these changes would also affect the duration through its primary variable of equipment productivity. In this example it has been assumed that both the quantity as well as the difficulty of excavation would increase as a result of design and regulatory changes. It is also assumed that the lawsuit filed in this case is aimed at limiting the amount of excavation that is to be carried out. Therefore, an unsuccessful defense against the lawsuit would result in a reduction in the amount of excavation that can be carried out as part of the environmental restoration project. The probability tree corresponding to the influence diagram is prepared in the next step of Fig. 5.11. Influence Diagram for the Example of Evaluating Excavation Duration 164 the quantification methodology. The probability tree for this example is shown in figure 5.12. Two states each of regulatory changes and design changes, i.e. their existence and non-existence, as well as two states of the outcome of the lawsuit, i.e. successful and unsuccessful, have been considered in the analysis. As is implied by the influence diagram, the three external factors have been assumed to be independent of each other. The prior probabilities of the various states are elicited from experts. The branch probabilities of the probability tree are calculated based on the prior probability values obtained from experts. The prior probability values used in the analysis as well as the calculated values of the branch probabilities are shown in figure 5.12. The base case scenario of the problem is considered in detail in the next step of the analysis. The base case corresponds to a scenario in which regulatory changes and design changes do not occur while the outcome of the lawsuit is successful. The duration (T) of the excavation work package which is the decision variable in this case is decomposed into its primary variable as, T = \u2014 (5.33) PL where, Q = quantity of the work L = resource usage level of a resource, R P = productivity of resource R (Output \/ Input) 165 Fig. 5.12. Probability Tree for the Example of Evaluating Excavation Duration Probability distributions are obtained for the primary variables considering the base case scenario. In obtaining the probability distributions for the variables, the 5 t h, 25 t h, 50 t h, 75 t h and 95 t h percentile estimates are elicited from experts. Suitable standard probability distributions are selected using the distribution selection procedure described in the previous section. The values used in the analysis and the outputs obtained are shown in Table 5.1. The primary variables have been assumed to be uncorrelated for simplicity. 166 Individual and Binary adjustment coefficients are elicited from experts considering the three external factors and their pair-wise combinations, in the next step of the analysis. Adjustment coefficients are obtained for all three primary variables. The hypothetical values used in the analysis are shown in Table 5.2. Table 5.1. Percentile Values of the Primary Variables used in the Example of Excavation Duration Estimation 5th percentile 25m percentile 50th percentile 75th percentile 95m percentile Probability distribution Q(m 3) 14,000 14,600 15,200 15,900 16,800 Triangular (13,200, 15,200, 17,400) L (Excavator-hours \/ hour) 3 3 3 3 3 3 P (m3 \/ Excavator -hour) 82 86 90 94 98 Beta(1.43, 1.14)* 24.97 + 75.15 Table 5.2. Adjustment Coefficients* used in the Example of Excavation Duration Estimation Design changes (Occur) Regulatory changes (Occur) Lawsuit (Lose) Des. change + Reg. change Des. Change + Lawsuit Reg. Change + lawsuit Q 100 60 -20 110 70 50 L 0 0 0 0 0 0 P -20 -10 5 -20 -18 -8 Values are provided as percentages of the base-case values. The adjustment coefficients for the combination of the 3 external factors are calculated using the values given in Table 5.2. An example calculation is shown considering the 167 quantity of work (Q). From equation (5.29), the combined effect, (D u R u L) of the design changes (D), regulatory changes (R) and the lawsuit (L), is given as (DvRvL) = (DvR) + (DvL) + (RvL)-D-R-L (5.34) Therefore, ( D u i ? u Z ) = 1 1 0 + 70 + 50-100-60-(-20) ( D u \/ ? u l ) = 90% Similarly, the adjustment coefficient for the productivity, when all external factors are active is obtained as -21%. In the next step of the analysis, the probability distribution for each scenario is obtained by multiplying the base case distribution by the value ' 1 + adjustment coefficient'. The scenario when all three factors are active is used as an illustration. In this case, the adjustment factors obtained for the quantity, resource usage and the productivity are 0.9, 0 and -0.21 respectively as shown in the previous calculation. The base case probability distributions for the variables are multiplied by \"1 + adjustment factor\" to obtain the distribution for the scenario when all external factors are active. Therefore, the probability distributions for the scenario when all external factors are active would be 1.9 x {Triangular (13,200,15,200,17,400)}, 3 (deterministic) and 0.79 x {Beta(1.43,1.14) * 24.97 + 75.15}. 168 A Monte Carlo simulation is then carried out using the probability distributions given above and the functional relationship T = Q \/PL. Let the number of iterations be 100. Therefore, each sample value generated for the duration would have a probability of 1 \/100 within the scenario considered. However, the probability of the scenario occurring is 0.09. Therefore, from a global perspective each sample value of the duration would have a probability of occurrence of 0.09 x 0.01. Monte Carlo simulation is carried out for all scenarios of the problem. Simulation software such as @Risk\u00ae allow the simultaneous simulation of several scenarios which reduces the input effort. The output of the simulation process is a set of values for the duration, each with an associated probability of occurrence as described above. The set of values and their associated probabilities can be used to calculate the first four moments of the duration using the standard statistical definitions for the first four moments. An example would be the expected value, E(T) which is defined as where t is the sample value and p is its probability. The results obtained for the statistical parameters of the duration using the procedure described above are shown in table 5.3. The results correspond to a case where 100 iterations were used in carrying out Monte Carlo simulation. (5.35) 169 Table 5.3. Statistical Outputs obtained for the Excavation Duration Stat is t ica l P a r a m e t e r V a l u e Expected value (mean) 97.134 2 n d moment about the mean 1297.152 3 r d moment about the mean 3830.895 4 t h moment about the mean 3614231.912 Standard deviation 36.016 Skewness 0.082 Kurtosis 2.148 5.7 S u m m a r y The steps in the quantification method for Type B environmental risks is summarized in this section. Step 1 - The relationship between the decision variable and external uncertainty factors which are likely to have an impact are identified by preparing an influence diagram which depicts the relationship of the decision variable to external factors. Step 2 - The probability tree corresponding to the influence diagram drawn up in Step 1 is prepared considering the possible states of the external factors. Prior probabilities in the event of the external uncertainties being independent, or conditional probabilities in the event of the externalities being dependent are assigned to the branches of the probability tree except to the final level of nodes which depict the target variable. 170 Step 3 - The scenario which corresponds to all the external factors being inactive is considered as the base case for the analysis. Steps 1 to 9 in the analysis of risk type A are performed considering the base case. Step 4 - Adjustment coefficients are obtained from experts for each primary variable considering each individual factor separately. Step 5 - Adjustment coefficients are obtained for each primary variable considering the effect of two externalities being active at the same time. A l l the external uncertainties are considered in a pairwise fashion in this step of the analysis. Step 6 - Adjustment coefficients corresponding to situations when multiple factors are active are obtained by applying the values obtained in Steps 5 and 6 to equations (5.29), (5.30), or similar equations corresponding to an even larger aggregate of externalities. Step 7 - The probability distributions of the primary variables obtained in Step 3 are multiplied by the value ' 1 + adjustment coefficient' to obtain the probability distribution for the scenario under consideration. This procedure is repeated for each branch of the tree, which correspond to all the possible scenarios. Step 8 - Monte Carlo simulation is carried out for each collection of the adjusted distributions as well as for the base case scenario. An equal number of iterations is carried out at each branch in order to simplify the analysis. Step 9 - The joint probability of each value generated by the simulation is calculated by dividing the joint probability of each branch by the number of iterations. This yields the probability of each value generated as part of the simulation process. Step 10 - The first four moments of the decision variable are calculated using the values generated from the Monte Carlo simulation and their associated probabilities. 171 The uncertainty of the decision variable is represented by the first four moments obtained in this manner. 5.8 Conclusion Environmental risks of Type B can create a need for additional work packages to be executed during the life cycle of the project or cause extraordinary changes in the scope of the existing work packages of a project in other instances. Due to their strong relationship to external events and uncertainties, the scope of such work packages are difficult to express in a form suitable for analysis using Monte Carlo simulation or moment based analytical methods. Influence diagrams, which are capable of representing and analyzing the uncertainties of such work packages, have disadvantages of their own. These include the large number of probability assignments that need to be made in carrying out the analysis and difficulties in assessing conditional values required for the analysis. The risk quantification method described in this chapter makes use of multiplicative factors in order to model the effect of external uncertainties. The use of such factors is not novel and has been used previously in analyzing economic risks. However, the quantification methodology described herein enables consideration of the effects of external uncertainties at a primary variable level which is unique to this approach. This allows the benefits offered by decomposition to be utilized in carrying out the analysis. In addition, a modeling paradigm which enables the representation and quantification of the effect of multiple external uncertainties is also introduced. The disadvantages of methods previously suggested in literature to model multiple uncertainties are overcome by this approach. 172 The treatment of external uncertainties at the primary variable level as well as the introduction of a novel approach towards representing and quantifying the effect of multiple external uncertainties are considered as the major contributions of this chapter. 173 Chapter 6 Elicitation of Information from Experts 6.1 Introduction An expert is identified as an individual who has superior knowledge in a particular field brought on by training, skill, and experience, and who is capable of imparting that knowledge in a coherent and accurate manner. The use of experts in obtaining subjective information is particularly useful in economic risk analysis of projects when actuarial or relative frequency based data are unavailable. The unavailability of such data within a project proponent's or contractor's organization can be due to several reasons: (1) Data from past projects of a similar nature might be unavailable or unreliable due to the absence of proper record keeping; (2) Difficulties in getting suitable and reliable data from other organizations due to the reluctance of organizations to share information (Russell and Froese 1997); and, (3) Due to the unique nature of civil engineering projects, the attributes of a project such as location, methods of construction and characteristics of the end product differ widely between projects. Therefore, it might not be possible to use data obtained from one project in evaluating the feasibility of another project. In addition to providing estimates for parameters which cannot be quantified using frequency data, expert judgement can also be used in the development of 174 scenarios and in making value judgements (Hora 1992). In analyzing environmental risks as set out in the previous chapters, expert assessment of various parameters, and events will be required. The term \"information elicitation\" will be used throughout this thesis as a wide range of information, not limited to probability estimates, needs to be obtained from experts in order to carry out environmental risk quantification. The use of strategies such as decomposition of decision variables into their primary variables can contribute to the overall success of the information elicitation process. This chapter presents a discussion of such strategies as well as of the factors which impedes the elicitation process. An information elicitation process for the analysis of environmental risks that incorporates decomposition, the use of influence diagrams, and the use of directed questions based on argument types is also presented in this chapter. The final section of the chapter illustrates the use of the environmental risk analysis framework by considering the hypothetical example of a dam removal project. 6.2 Evocation of Information from Experts The information generation process within an expert's mind consists of two distinct phases (Curley and Benson 1994). In the belief assessment phase, relevant information is evoked by the expert and applied towards the formation of a belief. In the response assessment phase, a numerical qualifier is attached to the belief, which serves as an output of the elicitation process. Humans make use of several argument types in the processes of belief formation and numerically qualifying such a belief (Brockriede and Ehninger 1960, Curley and Benson 1994). However, the limitations of the information processing capability of the human mind acts as an impediment to 175 the evocation of knowledge from experts. External aids such as directed questions and prompts based upon the argument types used by humans in reasoning can play an important role in overcoming impediments and assisting the evocation of relevant information (Browne, Curly and Benson 1997). A study carried out by Browne, Curly and Benson has shown that prompts and directed questions based upon argument types can aid an expert in evoking a much larger body of information than in a case when such assistance is not given. A significant loss in the quality of the information has not been observed during the process of generating additional information. A taxonomy of such argument types identified by Brockriede and Ehninger (1960) as well as examples of directed questions based upon such arguments, which are relevant to civil engineering projects is shown in table 6.1. Influence diagrams have also been found useful in evoking knowledge and capturing disparate knowledge in a coherent form (Howard 1989). In addition to serving as a direct quantification tool, influence diagrams can also serve as an evocative tool which evokes proper considerations in making numerical assignments of probability. An influence diagram, which shows several levels of dependency as in the example shown in figure 5.10, is useful in ensuring that all the necessary information is considered prior to making numerical assessments. The numerical assessment itself can be simplified by assessing only the nodes that are immediate predecessors of the decision variable. The other nodes, whose primary function would be to evoke knowledge and provide consistency to the problem in the case of multiple experts being utilized, can be considered as being evocative (Howard 1989). Evocative nodes are linked to other nodes using dashed line arrows in a diagram. 176 Table 6.1. Directed Questions and Prompts based upon Argument Types Argument Type Description Example Cause Non-intentional causal link between data and claim. What do you think were the causes of the cost overruns in the last highway project the company undertook? Motivation Intentions of human agent used as a cause of some result. Have you thought about whether the labor force has sufficient incentive to follow these environmental protection guidelines drawn up to reduce environmental accidents? Sign Covariational, but not directly causal. So you think the cool summer we've had is a sign that the timing of the salmon run would change and throw the construction schedule off target? Generalization Induction from a sample to its population. Do you think the equipment operators the company hires on a contract basis will act in the same environmentally responsible manner as the ones on the permanent work force? Classification Concluding from a general population to a specific instance. In general, is there a high turnover rate in the casual labor force during the summer months, which could cause a loss of productivity on this project during that period? Parallel case Concluding based on a similarity between instances. Can you think of an instance of a hazardous material spill occurring on a similar project that you've been involved in? Analogy Concluding based on a similarity of relationship with something outside of the current domain. Are you saying that productivity is like a train, starting off slowly and then building up speed? Would this mean that larger quantities result in higher average productivity values? Authority Appeal to an external source as having relevant information. Have you read or heard of any specialists endorsing the new fuel spill containment booms the company intends to use on this project? 177 6.3 Decomposition of Variables The use of decomposition in the elicitation process is based on the principle of divide and conquer. The assumption that experts find it easier to analyze individual events rather than a complex combination of such events provides the platform for the decomposition concept. In economic risk analysis of engineering projects, decomposing a decision variable such as work package cost into its primary variables is often regarded as a useful technique for reducing the complexity of the estimation problem. The necessity of developing a link between time and cost in engineering economic analysis is considered to be the main reason for decomposing decision variables by Ranasinghe and Russell (1993). In addition, decomposition has also been found to be useful in reducing biases which affect the accuracy of judgmental forecasts (Merkhofer 1987). Andradottir and Bier (1998) have considered a form of decomposition in which a number of mutually exclusive, collectively exhaustive conditioning events are identified, unknown quantities of interest are estimated conditional upon such events, and finally the resulting estimates are aggregated. This type of decomposition mirrors the quantifying methodology adopted for Type B environmental risks. The study carried out by Andradottir and Bier has shown that decomposition is useful when the disaggregate estimates are likely to be accurate but prone to errors when disaggregate estimates have large errors. However, decomposition has been found to be useful even in cases where the errors in the disaggregate estimates are larger than those in the direct estimation process, provided that the estimated marginal probabilities are sufficiently precise. Ravinder, Kleinmuntz and Dyer (1988) have carried out analysis of the same type of decomposition. They concluded that decomposition could 178 contribute towards the reduction of the random errors associated with the probability encoding process. However, the marginal increase in accuracy has been shown to decrease as the number of conditioning events increase. However, Ravinder, Kleinmuntz and Dyer have also noted that dependencies among variables could adversely effect the accuracy brought on by decomposition. A more common method of using decomposition in risk analysis is to express a decision variable as a known function of several component variables (Hora, Dodd and Hora 1993, Ranasinghe and Russell 1993). The estimate for the decision variable is obtained by the recombination of the estimates for the primary variables. This type of decomposition is common to the analysis methodology adopted for all the environmental risk types. An empirical study carried out by Hora, Dodd and Hora has shown that probability distributions obtained for variables using decomposition were much better calibrated than those obtained holistically. A disadvantage of carrying out decomposition which has been highlighted by Ranasinghe and Russell (1993) is the danger of the experts losing the mental awareness of interdependencies between the primary variables while providing judgmental forecasts. Several authors have also questioned the validity of the belief that decomposition leads to more accurate estimates. Henrion, Fischer and Mullin (1993) have carried out an empirical study, the results of which show that decomposition does not significantly affect either the accuracy of the assessed median or the calibration of the subjective probability distributions. The results of the study have indicated that decomposition changes the underestimation bias in the direct estimation method to an overestimation bias. However, the majority of literature suggests that decomposition is beneficial in carrying out information elicitation. 179 6.4 Biases in Information Elicitation A bias in the knowledge encoding process can be defined as a conscious or subconscious discrepancy between the elicited information from the expert and an accurate description of the expert's underlying knowledge (Spetzler and Stael Von Holstein 1975). Biases may be introduced into the output knowledge due to several reasons. Adjustments in the responses caused by a perceived set of rewards are termed as motivational biases, while biases introduced by the mental information processing technique of the expert are termed as cognitive biases (Spetzler and Stael Von Holstein 1975). Motivational bias may be introduced when an expert views a variable as a goal rather than an uncertain quantity for which an estimate should be made. Bias due to an inaccurate perception of the role of an expert can encourage the expert to provide estimates which underestimate the uncertainty of the target variable. A belief that experts are supposed to be certain in their estimates would give rise to this type of a motivational bias (Merkhofer 1987). Cognitive bias can arise in experts due to several reasons. These include giving too much weight to events in recent memory, aligning estimates around a convenient value, and due to stereotyping. Different modes of judgement employed by experts and their contribution towards producing biases have been identified by Spetzler and Stael von Holstein (1975). These basic modes of judgement are described below. (a) Availability - Due to the comparative ease with which recent events can be recalled compared to ones more distant in the past, more recent events are given too much weight by experts. Therefore, outcomes which resemble recent events are assigned an inappropriately high probability. 180 (b) Adjustment and Anchoring - In some cases, the estimates are anchored around a convenient value instead of being assessed independently. Central bias, the most common form of bias caused by anchoring results when estimates are anchored around the median. (c) Representativeness - The estimation of the probability of an event based on its representativeness to the process from which it originates leads to a mode of judgement in which probability judgements are reduced to judgements of similarity. (d) Unstated Assumptions - As a result of unstated assumptions being made, the probability distribution will not reflect the total uncertainty of the variable. (e) Coherence - The assignment of probability to an event based on the ease with which a plausible scenario that would lead to its occurrence can be fabricated is termed as coherence. Several steps can be taken in order to prevent bias being active in expert judgment. These steps have been constructed taking into account the reasons that are likely to trigger biases in expert judgement. 1. Analysis of the relationship between the expert and the information which is being elicited can lead to the identification of motivational bias. For example, a manager who is responsible for the sales volume of a department might provide a motivationally biased estimate when asked for a forecast of future sales in order to ensure that actual sales have a higher probability of exceeding the predicted value (Spetzler and Stael Von Holstein 1975). In such cases the estimate can be 181 restructured, for example by using decomposition, in such a manner that the expert would not be aware of the link between his or her interests and the estimate. 2. Appraising the expert of the purpose of the elicitation process would also contribute towards reducing motivational bias. The expert should be made to understand that the purpose of the elicitation process is to obtain the best judgement of the expert, including an accurate assessment of the variability involved with the estimate. This would discourage the expert from attempting to make a prediction of the value of the target variable which encompasses minimum uncertainty (Merkhofer 1987). 3. Experts can be prompted to recollect relevant events which occurred a significant period into the past in order to overcome bias caused by the comparative availability of recent events. This will prevent experts from giving more weight than necessary to recent events. 4. In most instances, the median serves as a convenient value around which other values of the estimate can be aligned. Central bias can be overcome by encouraging experts to consider scenarios in which extreme outcomes can occur (Spetzler and Stael Von Holstein 1975). In carrying out the elicitation of several percentile values of a variable, eliciting the extreme values initially is also considered as being useful in reducing central bias (Ranasinghe and Russell 1993). 5. Discussion of the possibility of the scenarios which correspond to the elicited estimates can be carried out with the expert. Experts can be made aware that the ease or difficulty with which scenarios can be constructed does not necessarily correspond to the possibility of the actual event occurring. 182 In addition to eliciting the estimates, the analyst should also strive to obtain the explicit and implicit assumptions the expert may have made during the estimation process. This would ensure that all experts and analysts share a common set of assumptions. It would also perform as a consistency check as to whether the expert has been provided with all the available information necessary for the estimation process. 6.5 Information Requirements for Analyzing Environmental Risks Several types of information need to be elicited from experts in order to analyze environmental risks as described in the previous chapters. The assessments which would be required from experts are described below. Initial estimates for the statistical parameters of the decision variables As described in section 3.4.3, the identified risks need to be screened in order to identify the most important risks. Estimates for the expected value, and the minimum and maximum possible values of the cost and duration of the work packages needs to be obtained from experts in order to prioritize risks. Identification of the external uncertainties which affect primary variables In analyzing Type B risks, it is necessary to identify the external uncertainty factors which are capable of making a significant impact on the decision variables of cost and duration. In some instances it would be necessary to consider other uncertain events which in turn are likely to affect the outcome of the external uncertainties which directly affect the target variable. The identification of external uncertainty factors which would be carried out as part of the information elicitation process would lead to 183 the development of an influence diagram which would serve as the foundation for the analysis of a Type B risk. Estimation of the probability distributions which characterize uncertain variables and events. Subjective estimates for the 5 t h, 25 t h, 50 t h, 75 t h and 95 t h percentile values would be required for each primary variable. In the case of Type B risks, the percentile values are obtained for the base case. In addition, event probabilities would have to be assessed in order to assign probability values to the probability tree derived for the analysis of Type B risks. Estimation of adjustment coefficients The adjustment coefficients which characterize the change in the primary variable due to external uncertainties should be obtained from experts. Two types of assessments need to be made in this regard. 1. Individual adjustment coefficients should be obtained for each state of all the externalities. 2. Binary adjustment coefficients should be obtained considering the external factors in a pairwise manner. Estimation of the interdependency between primary variables The percentage reduction in uncertainty of a primary variable due to the knowledge of the exact value of a second primary variable is required in the analysis in order to calculate the value of the rank correlation coefficient between the two variables. A l l 184 primary variables are considered in a pair-wise manner in order to obtain the rank correlation matrix. 6.6 The Information Encoding Process The encoding process involves the generation of accurate information regarding the parameters and events described in section 6.5. This process also involves the adoption of measures to reduce biases, carrying out consistency checks, and adopting steps to generate a complete body of information using information evocation methods. The information encoding process consists of 2 phases. Phase I involves eliciting information required to carry out risk screening, while Phase II involves obtaining information required to carry out in-depth analysis of selected risks. The encoding process is described in detail in the following sections. 6.6.1 Phase I - Eliciting Information for Risk Screening The elicitation of information for risk screening involves obtaining initial estimates for the statistical parameters of all the environmental risks that were identified as being relevant to the project under consideration during the risk identification process. The statistical parameters that need to be elicited are the expected value, and the minimum and maximum possible values of the economic variable. Information elicitation for risk screening should be carried out using a limited number of experts such as project managers and estimators who preferably have experience in projects of a similar nature. Information regarding the project should be presented to these experts in advance of the actual encoding process. 185 The values for the statistical parameters maybe elicited from experts by asking direct questions related to the parameters. The minimum and the maximum values can be obtained by asking questions of the form \"What is the minimum \/ maximum possible value you would expect the parameter P of work package Wto be, assuming that everything associated with this work package goes as planned \/ goes wrong?\". An example of such a query would be \"What is the maximum possible value you would expect the duration of the re-vegetation program to go up to assuming that everything associated with this program that could go wrong, goes wrong?\". An estimate for the expected value of the variable can be obtained by asking questions of the form \"What is the value of the parameter P of the work package W, you would expect on average given the circumstances of the project?\". 6.6.2 Phase II - Eliciting Information for In-depth Analysis of Selected Risks Phase II of the information encoding process consists of 11 distinct steps, some of which are however applicable only to Type B environmental risks. Step 1 - Selection of experts The primary step in the elicitation process involves the identification of suitable experts who will be able to provide the required information. The comprehensive analysis of environmental risks would involve dealing with issues in a wide range of specializations such as ecology, wildlife conservation, meteorology and hydrology. Therefore, experts external to the project proponent's organization would have to be approached in most cases to obtain the required information when carrying out in-depth analysis of important risks. In instances where the environmental issues under 186 consideration are under heavy public scrutiny it might be necessary to ensure that the experts who are used in the elicitation process have a reputation for being impartial and are well qualified in their field. Experts external to the organization can be selected using registries of professional organizations and consulting firms, listings of universities and government agencies as well as through literature searches (Hora 1992). Experts within the organization can be selected by considering previous experience in similar projects, overall work experience and their involvement with the project under scrutiny. Experts who are directly involved with the project and whose personal performance could possibly be tied to the performance of the project should not be used unless an alternative expert is not available, in order to reduce the possibility of motivational biases. Step 2 - Definition of the problem Information regarding the project in general and specific information pertaining to the experts' field of specialization should be presented to the expert well in advance to the actual encoding process. The information provided to experts should include a description of the decomposition method which will be used, the parameters for which the expert is required to make estimates as well the assumptions which are required to be made. Step 3 - Identification of external uncertainty factors (Type B risks only) This step involves the identification of the external factors that need to be taken into account while making estimates for the decision variable or its decomposed primary variables, initially, the analyst should prepare a basic influence diagram which shows 187 the relationship between an event occurring, and the time or cost uncertainty associated with it. Figure 5,2 is an example of such a diagram. The analyst and the expert should then proceed to add additional nodes which represent chance events that need to be taken into account in order for an accurate estimate of the variable to be made. The analyst should make use of directed questions and prompts based upon argument types to evoke information from the expert. Subsequent to identifying the external factors, the analysts should repeat Step 1 of the encoding process and identify suitable experts who could estimate the uncertainty of the external factors. However, in some cases the analyst might be able to make use of existing frequency data in order to obtain such estimates. In the example of a marine oil spill the analyst might be able to use frequency data to calculate the probability that the water will be ice infested during the period marine construction is taking place. Steps 2 and 3 should be repeated with any new experts who are selected. This will lead to further development of the influence diagram. For Type B risks, the iteration among the first three steps should be carried out until the influence diagram is developed to a level sufficient enough to calculate the probability distribution of the variable under consideration. The number of distinct states of the external factors that need to be considered should be obtained from the expert. The analyst should use his or her judgement in selecting the nodes for which numerical assessments are required. Limiting the quantitative assessment to nodes that are immediate predecessors of the decision variable node is suggested. A l l the other nodes should be considered as evocative. In some cases even nodes that are immediate predecessor to the decision variable need 188 not be assessed numerically. As an example consider the case when an expert affirms that the presence of marine wildlife would affect the cost of a marine fuel spill cleanup as indicated in the influence diagram shown in figure 5.10. However, i f further discussion indicates that the change in cost due to the presence or absence of wildlife at the time of the spill is minimal, the chance node pertaining to the presence of marine wildlife can be considered as being evocative in order to simplify the analysis. However, it is important to portray such nodes in the influence diagrams in order to ensure all relevant knowledge is considered in the analysis as well as to encourage further generation of information. Step 4 - Receive feedback from the expert regarding the problem definition The analysts should discuss and receive feedback from the expert regarding gaps in the information which are necessary to be filled in order to make an educated estimate of the parameters, the validity of the assumptions that have been made, additional assumption which need to be made, and the level of comfort of the expert regarding the decomposition method that has been adopted. Step 5 - Redefinition of the problem Redefinition of the problem is only necessary should any expert request a major change regarding the assumptions that have been made or the type of decomposition which has been carried out. In such an event the problem will have to be restructured and all the experts who are affected by the changes will have to be consulted. This necessitates for steps 4 and 5 to be repeated until all the experts are comfortable with the problem definition. 189 Step 6 - Training in probability judgement The training process is designed to assist the experts in expressing their knowledge as probability estimates and to make them aware of the biases that might affect their judgement. The training session should include a brief introduction to the basics of probability theory. Topics that should be explained are the probability axioms, the concept of correlation and the application of Bayes' rule. These topics should be explained in a very general manner. Training in the basics of probability is necessary as experts in substantive fields such as engineering may not be effective in expressing their beliefs in the form of probability distributions (Hora 1992). Research carried out by Gebotys and Claxton-Oldfield (1989) has indicated that subjects who received training in probability theory provided much better probability assignments than subjects who did not receive such training. The effect of such training has been shown to be evident even several weeks after the training. Subsequent to training in the basics of probability, the experts should be given a brief introduction to the biases that could affect their judgement as well as to the modes of judgement that leads to such biases. The training session should also include a practice session designed to allow experts to gain confidence in the process. The use of questions from almanacs is suggested as parameters to be estimated during the practice session. Step 7 - Encoding of probability estimates for primary variables The 5 t h, 25 t h, 50 t h, 75 t h and 95 t h percentile values of primary variables need to be obtained as subjective estimates from experts. Questions of the form \"What is the value of variable V, which you feel has only a P % chance of being \/ not being 190 exceeded?\". An example query would be \"What is the value of the excavation quantity that you feel has only a 5% chance of being exceeded?\". In order to reduce central bias the extreme values should be elicited initially (Ranasinghe and Russell 1993, Spetzler and Stael Von Holstein 1975). The analyst should ask directed questions based upon argument types in order to encourage the expert to take a wider range of issues into account in order to reduce biases such as availability bias. Step 8 - Consistency checking Once the required probability values of the primary variables are available, a probability density function should be generated for each primary variable using computer graphics and displayed to the experts. A probability density function is a more preferred mode of presentation than cumulative distribution plots as characteristics such as skewness are more clearly presented by a probability distribution plot. Percentile values, or extreme values obtained from the probability distribution can be read out to the expert in order to check the validity of the results. The expert should be asked to comment on the final result. Should grave concerns be voiced regarding the values, the analyst should consider repeating steps 7 and 8 of the elicitation process (AbouRizk and Sawhney 1993). Step 9 - Encoding of event probabilities (Type B risks only) In executing this step the analyst should derive the probability tree corresponding to the influence diagram obtained in step 3. Prior probabilities for events which are not dependent upon any other event in the influence diagram should be elicited initially 191 from the relevant experts. Direct questions such as \"What is the probability of the seas around the project location being ice infested during the construction period from April to December?\" can be used to elicit the probability values associated with each state of the event. Once prior probabilities are assigned to independent events, conditional probabilities are elicited for dependent events. Direct questions such as \"What is the probability that a large number of seabirds will be present in the area, given that the ice cover is more than 10 percent?\" can be used in such instances. Step 10 - Encoding of adjustment coefficients (Type B risks only) Adjustment coefficients, defined in Chapter 4, need to be elicited from experts in the case of Type B environmental risks. Individual adjustment coefficients can be obtained by asking questions of the form \"By what percentage would the average value of parameter P change if state A of external uncertainty Z is present?\" An example of such a query related to the marine fuel spill work package would be \"By what percentage would the labor force need to be increased to combat the spill, on average, if high winds are present during the time of the fuel spill?\" Adjustment coefficients for two simultaneous uncertainties can be assessed by asking questions of the form \"By what percentage would the average value of parameter P change if state A of external uncertainty Z and state L of external uncertainty Y are present at the same time?\" An example of such a query would be \"By what percentage would the labor force need to be increased to combat the spill, on average, if high winds are present during the time of the fuel spill and the waters are ice infested at the same time?\" 192 Step 11 - Elicitation of interdependency parameters The process of eliciting interdependency parameters between variables involves identification of the existence of interdependence, and in the event the variables are correlated, elicitation of the rank correlation coefficient from an expert. The percentage reduction in the uncertainty of one variable due to the knowledge of the exact value of the other variable is utilized in order to identify the existence of interdependence as well as to obtain an indirect value for the rank correlation coefficient (van Dorp and Duffey 1999). Subsequent to selecting a pair of variables X , and X 2 , the expert is asked if knowing the exact value of X , would help in reducing the uncertainty associated with X 2 . The choice between X , and X 2 should be made taking into account the relative ease with which an expert can visualize the conditional values of one variable given another. For example, an expert would find it intuitively easier to think of the values of productivity conditional upon the knowledge of the exact value of the size of the work force, rather than about the values of the size of the work force based upon the knowledge of the value of the productivity. In the event of the expert answering 'No' to the above question, the two variables would be deemed to be independent. In the event of an 'Yes' answer the two variables are interdependent, necessitating the elicitation of a value for the rank correlation coefficient. In order to obtain the rank correlation coefficient, the expert is asked for the percentage reduction in the uncertainty of one variable due to the knowledge of the exact value of the other variable. Questions of the form \"To what values or range could you restrict the value of X 2 if you knew thatthe value of X ! is exactly A ? \" should be asked from the expert. If the expert answers that the value of X 2 will now 193 fall within M and N , the corresponding cumulative probabilities can be obtained from the marginal distribution of X 2 as described in section 4.3.3. If the corresponding cumulative probability values are P M and P N then the corresponding percentage reduction in uncertainty would be equal to {1 - (PN - PM)} x 100 %. The percentage reduction value should be obtained for several values of X , that divide the axis of the diagonal band distribution corresponding to X , into equal parts. The uncertainty reduction of the mid-point of the sub-range is used to represent the sub-range. For example in figure 6.1, \"(1 - b)\" is used to represent the average percentage reduction in uncertainty in the 1st quartile. The reduction in uncertainty is obtained by multiplying \"(1 - b)\" by the interval width. Summation of the values obtained for the four intervals would give the overall percentage reduction in uncertainty. Division into four equal parts as shown in figure 6.1 is recommended. This would involve obtaining the conditional range of X 2 at points corresponding to the 12.5th, 37.5th, 62.5 th and 87.5th percentile values of X , . The value of the rank correlation coefficient is obtained using equation (4.36) utilizing the overall percentage reduction in uncertainty as input. 0,0 0.125 0.25 Fig. 6.1. Representation of the Percentage Reduction in Uncertainty 194 Once the interdependence between X , and X 2 has been characterized, another primary variable, X 3 is introduced into the analysis by considering the pairs X ! and X 3 and X 2 and X 3 . The correlation coefficient between these pairs are calculated using the procedure set out above. Subsequently, the correlation matrix between the three variables X , , X 2 and X 3 is evaluated in order to satisfy the theoretical requirement of the correlation matrix being positive definite (Ranasinghe 1990). In the event of the matrix not being positive definite the correlation coefficient between X 2 and X 3 is re-evaluated. In the event of the correlation matrix not being positive definite subsequent to the re-evaluation, the correlation between the other pairs of variables are considered for re-evaluation until this theoretical requirement is met. The information encoding process for the analysis of the two risk types is shown graphically in figures 6.2 and 6.3. 6.7 The Use of Information Technology and Multimedia in Information Elicitation This section briefly identifies the usefulness of Information Technology (IT) and multimedia in information encoding. Advocacy for the use of computers and related tools in knowledge elicitation is limited in the literature due to the established belief that a better quality elicitation results from a direct interaction between an analyst and the expert. This reflects a philosophy that an attempt at self-elicitation without the aid of an analyst would result in inaccurate values (Merkhofer 1987). Spetzler and Stael Von Holstein (1975) recommend against the use of interactive computer programs as the lack of the balancing effect of the analyst can adversely affect the results. However, recent research has started to challenge these beliefs (Walther 1996). 195 Identification of experts Definition of the problem to experts . . \u2014 < r Feedback from experts Yes No Training in probability judgement Encoding of percentile estimates U Generate PDF for primary variables Fig. 6.2. Flowchart of the Information Encoding Process for Type A Risks 196 Identification of experts Definition of the problem to experts Identification of external factors Yes No r Training in probability judgement Encoding of percentile estimates Generate PDF for primary variables No Elicit event probabilities, dependency parameters and adjustment coefficients j -W End . 6.3. Flowchart of the Information Encoding Process for Type B Risks 197 The use of computer software and graphics in calculating and generating visual representations of probability distribution functions for the values elicited from experts in order to carry out consistency checks at the end of an encoding process is recommended by Ranasinghe (1990). A tool developed by AbouRizk and Sawheny (1993) for the purpose of fitting beta distributions to subjective data utilizes visual representation of the fitted distributions. The ability of such a representation to immediately reveal the quality of the fit is cited as a significant advantage. It is suggested that further use of new developments in information technology and multimedia can contribute towards the successful elicitation of information while reducing the time and costs involved with the process. Multimedia can be classified as an application that uses two or more media to convey a message or achieve a given purpose (Vanegas and Smith 1994). Information technologies that can be used in conjunction with multimedia in information encoding include Group Decision Support Systems (GDSS), Desktop videoconferencing, Electronic mail (E-mail) and the Internet. Multimedia can assist in the description and definition of the project and the natural environment, while IT tools can provide versatility in helping maintain personal interaction between the analyst and the experts while reducing associated costs simultaneously. A set of instances in which IT tools and multimedia can be used in the information elicitation process, and the tools that can be used in order to facilitate the process are presented in table 6.2. 198 Table 6.2. Use of Information Technology and Multimedia in Information Elicitation Steps in The Elicitation Process Available Tools Definition of the problem - Providing description of the work site, the natural environment of the project area, and proposed construction and operation methods to experts Video clips, Interactive CD-ROMs Definition of the project - Visualization of the finished product and its processes Graphics based computer applications Definition of the project - Distribution of project information among experts at the start of the elicitation process. E-mail, Internet or Intranet, Bulletin boards (GDSS) Receiving feedback from experts - Clarification of information and provision of additional information which may be required by individual experts E-mail, Videoconferencing, Internet chat Training in probability judgement Videoconferencing, Multimedia educational environments (Interactive CD-ROMs, Interactive H T M L documents on Intranet), video clips Information encoding Videoconferencing Consistency checking - Providing feedback to experts and carrying out consistency checks on the information obtained Interactive distributed computer applications The advantages of the use of IT tools and multimedia in information elicitation can be summarized as follows: 1. Multimedia based descriptions of the project site, the surrounding natural environment, the construction, operation and demolition methods, and the finished project components can assist experts in gaining a better understanding of the 199 project characteristics thus enabling them to apply their knowledge to suit such conditions. 2. The use of IT and multimedia-based tools for the training of experts in probability encoding allows self-paced learning. This is an important factor with regard to experts with tight time schedules. The use of hypertext linkages in CD-ROM or web-based documents allow experts to easily navigate though the project documents and effectively focus attention on topics of relevance. 3. The use of videoconferencing during the actual elicitation process eliminates the need for the experts and the analyst to be in the same location. Thus, videoconferencing can provide the personal interaction required for a successful elicitation while reducing associated costs. 4. The use of distributed computer applications to display outputs of the elicitation process in a graphical form allows consistency checks to be carried out in real-time while the participants are at several different locations. 6.8 Application of the Environmental Risk Analysis Framework This section illustrates the use of the environmental risk analysis framework. Screen snapshots of the prototype computer system developed to implement the framework are used to aid the illustration. The prototype system uses a combination of commercially available software, using Microsoft Visual Basic\u00ae as the main programming environment. The example deals with the removal of a dam which is part of a large hydropower project. The application of the risk analysis framework towards studying the duration uncertainty associated with this operation is considered in this hypothetical example. 200 Background Smaller dams may be constructed in conjunction with large hydropower dams for reasons such as to dampen the effects of water fluctuations in the downstream reaches. However, instances can arise when such dams need to be removed. The reasons for such an action could be the prohibitive cost of rehabilitation which makes removal the only viable option in the case of damaged or degraded dams, and due to concerns over negative environmental effects that might have arisen unexpectedly. The hypothetical example introduced in this section concerns the removal of a rock and earthfill dam located on the Ecstall river in British Columbia. The dam is assumed to be 9.3 m in height and 78 m in length. The degraded dam is to be removed as its further use could pose a danger to the public. The planned schedule calls for the removal of the dam during the summer of 2001. The dam is located downstream of a Uranium mine. There exists a possibility that the sediment trapped behind the dam may be highly contaminated with toxic waste discharged from the mine several years ago. While the removal of dams with a height less than 10 m is generally not required to undergo an EIA assessment (British Columbia 1995) the possible presence of contaminants has prompted the Environmental Assessment Office to reconsider the exemption. At the time of this analysis, assumed to be early December 2000, tests are being conducted to determine the concentrations of toxic contaminants within the trapped sediment. The results are expected by the end of the month. The requirement for an EIA would depend largely upon the test results. Therefore, the EIA process regarding this project can be considered as a project component that may or may not arise. 201 The uncertainty associated with the EIA process as well as the uncertainty associated with other project components that may be affected by environmental issues contribute to the overall uncertainty of the dam removal project duration. The environmental risk analysis framework is applied to this problem in order to assist in the determination of the likelihood of being able to complete the dam removal process by the end of September 2001. The risk analysis process is described below. Stage 1 - The Define Stage The define stage involves obtaining a definition of the project and the environment. Definition of the project The project is defined in terms of its scope and component work packages. Generally, the scope of small dams is provided in terms of dam height measured from the lowest point of the foundation to the crest of the dam. The component work packages considered include only the work packages that are required to achieve the basic objectives of the project. Additional work packages that may arise due to environmental issues need not be considered at this stage. The project definition document is shown in figure 6.4. Definition of the natural environment The natural environment is described in terms of the valued ecosystem components that can be found v^thin the project area (see figure 6.5). 202 Pr oj ect - Rjemo val of dam No.3 Location - Ecstall river (Northern BC) Scope- dam height = 9.3 m W o r k Package Details Breach dam Quantity of rock \/ earth to be removed-Approx. 10,000 m 3 Work duration-May 2001 to September 2001 U sing B ackho e 1 o ader(s) Dispose dam fill material Quantity-Approx. 10,000 m 3 Work duration-May 2001 to September 2001 Using Dump truck(s) Fig. 6.4. Dam Removal Project on Ecstall River - Definition of the Project Project \u2014 Removal of dam No.3 Location \u2014Ecstall river (Northern BC) Important Environmental Components E nviro nm ent al C omp onent Importance Description Chinook Salmon High Has high economic value. River harbors juveniles. Extensive spawning migration occurs from May to mid June. Moose Low Infrequently encountered in the project area. Listed as Threatened. Grizzly Bear Low Infrequently encountered in the project area. Listed as Threatened. Ecstall river waters Medium Downstream reaches are used for recreational purposes Fig. 6.5. Dam Removal Project on Ecstall River - Definition of the Environment 203 Stage 2 - The Risk Identification Stage The next stage in the analysis framework is the risk identification stage. A l l work packages that arise or are affected by environmental issues are identified during this stage. Public perceptions are analyzed prior to the actual identification of risks. Analysis of public perceptions Analysis of public perceptions on this project would involve carrying out discussions with first nations people, outdoor recreational groups including anglers and hunters, and representatives of nearby communities with the aim of identifying concerns regarding the project. In this hypothetical project, it is assumed that the only major concern that has been raised is by a downstream community regarding the degradation of water quality of the river during the dam removal operation. Concerns have been heightened due to the possible presence of toxic material trapped behind the dam. The river is used by the community for recreational purposes. Identification of risks A risk register serves as the starting point of the risk identification process. The risk register provided in Appendix A - l that has been incorporated into the prototype system as a Microsoft Access\u00ae database is utilized for this purpose (see figure 6.6). The entries in the register are considered individually to determine their applicability to the project under consideration. An example would be the determination of the applicability of the work package \"Construction of a sediment trap downstream of the dam to trap sediments that will be eroded\". This work package is relevant to the project in view of the presence of juvenile Chinook salmon in the river waters. High 204 ^\u2022Environmental Risk Analysis - [Select Risks] EJ\u00bb File,. Window Construction of a sediment trap downstream of , Risk Type A1 the dam ', H i iFi:k \u2022 !\u2022! i Details In carrying out dam removal; sediment tiaps would need to reconstructed to prevent sediment that , were ortgirtally trapped beninci ihejeiam being washed downstream and causing sedimentation in the OK Analyse Fig. 6.6. Dam Removal Project on Ecstall River - Use of the Risk Register loads of sediments introduced into the waters can smother the juveniles. The work package is also relevant in view of the concerns raised by the downstream community. Review of the register as well as review of government regulations leads to the identification of the following project components that contribute to the project duration and its uncertainty: (i) Carrying out Environmental Impact Assessment Studies and obtaining a Project Approval Certificate; (ii) Removal of 9, 000 m 3 (approximate estimate) of possibly toxic sediment from behind the dam prior to breaching; (iii) Disposal of 9,000 m 3 (approximate estimate) of possibly toxic sediment that is removed from behind the dam; 205 (iv) Construction of a sediment trap downstream of the dam to trap sediments that will be eroded; Additionally, the work packages \"Breach dam\", \"Dispose dam fill material\", \"Remove sediment\" and \"Dispose sediment\" match the characteristics of generic work packages ' A ' , and 'D ' (see Appendix A) as they involve the use of heavy machinery and may have schedules that overlap with the migration period offish. Therefore, these work packages would have to be halted during the migration period of salmon. As the migration occurs during the period late spring - early summer, this effectively means that these work packages cannot commence until the migration period ends. The uncertainty regarding the culmination of the migration would affect the duration uncertainty of the work packages that are scheduled to commence immediately after the migration ends. While these work packages would also need to be halted during wet weather as is characteristic of a generic work package ' A ' , the installation of a sediment trap would nullify the need for such stoppages. The \"installation of sediment trap\" itself matches the characteristics of generic work package \"D\". Therefore, the installation cannot be carried out during the salmon migrating season. An attempt is also made to identify additional environmental risks that may not be included in the risk register using expert interviews. In this project it has been identified that the dam material may also be contaminated with toxic mine waste. Therefore, additional precautions will need to be taken during dam breaching. The dam fill material will also need to be treated prior to disposal. These measures will contribute to the duration uncertainty of the \"Breach dam\" and \"Dispose dam fill material\" work packages. However, this addition to the scope of work is contingent 206 upon the results of the toxicity determination tests. As the original risk register does not include these risks, it is updated using the register update form of the prototype system (see figure 6.7). ^ Environmental Risk Analysis > [Update Risk Registei] 5 File V'r.dcu\" Kelp Update Risk Register i , n,.-., .1,1,, .,i Disposal of dam fill material Please input the work package sut>|ect to environmental* I;>M Please mpjl a de:criphon lor fu>ue rolcrcme Is the existence of thework , component an uncertain'^ event? \u2022. The dam fill material may be _> contaminated with toxic substances ,r; that are introduced in the upstream i|i reaches and are trapped by the dam. f~ Is the work component a new work \u00ab ^ package or an addition to an existing , f? Yes r NO \u2022 ... work package? r NewWP (\u2022 Addit'on Update Register Quit Fig. 6.7. Dam Removal Project on Ecstall River - Updating the Risk Register Risk categorization The risks are categorized in accordance with the definitions set out in Chapter 4. The classifications given to the duration uncertainty caused by the project components are described below. 207 (a) The uncertainty surrounding the \"EIA duration\" is classified as a Type B l risk in view of the fact that its occurrence is not certain at the time of the analysis, while its magnitude is also of an uncertain nature; (b) The need to remove sediment from behind the dam is known with certainty. However, as the results of the toxicity determination tests are unavailable, it is not known with certainty whether special removal procedures need to be adopted. As the addition to the scope due to special removal procedures is uncertain, the duration uncertainty of the \"sediment removal\" process is therefore a Type B2 risk. (c) The duration uncertainty of the disposal of sediment can be considered in the same manner as (b) above, and be classified as a Type B21 risk. (d) The provision of a sediment trap is work package that will need to be carried out with certainty. Therefore, the duration risk associated with this work package is classified as a Type A l risk. (e) The scope of the \"breach dam\" work package would be affected by the discovery of toxic material behind the dam in a manner similar to (b) and (c) above. While the breaching of the dam is a certain event, the addition to its scope is uncertain. Therefore, its duration uncertainty is a Type B2 risk. (f) The duration uncertainty of the \"dispose dam fill material\" work package can also be classified as a Type B2 risk in a manner similar to (b), (c) and (e) above. The dam removal process can therefore be characterized by the network of work packages shown in figure 6.8. 208 o CO __>T3 fi > o fi o 2 X ! P 2 i\u2014H -4\u2014\u2022 Cd Kl cd \u00a3 \u00a3 o \u00ab M I l l CD O fi O o o a .5 e-i\u2014i o .a a o a \u00a3 \u202255 S CD oo Q 2 fi 2P OO CD fi O CD O \u00bb ^ 4\u2014* a^ DC g S w c3 O H a ^ CD > i .2 OH oo oo rt fi cd JS --fi DH fi CD oo CD - f i __