Transportation Infrastructure Project Cost Overrun Risk Analysis Risk Factor Analysis Models b y Qing W u B.Sc, Shanghai University of Finance & Economics, 1999 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES (Business Administration) THE UNIVERSITY OF BRITISH COLUMBIA July 2006 © Qing Wu, 2006 Abstract The main purpose of this thesis is to analyze some common risk factors and to propose several useful analytical models for cost overrun risk analysis in transportation infrastructure investments. Probability models and regression models are proposed and, partially (due to data insufficiency) applied using the VIHP (Vancouver Island Highway Project) data. The VIHP case study shows that cost overrun ratio increases as project sizes increase for small road/highway projects (budget < $0.25 million) and bridge/tunnel projects (budget >$0.85 million). However for road/highway projects with budgets over $0.25 million, cost overrun ratio decreases as project size increases. Using the VIHP database, results of a distribution fitting model and a Monte Carlo simulation model are compared. Compared with the distribution fitting model, the Monte Carlo simulation model is shown to underestimate both the upper bound value of project cost overrun ratio and slightly the probabilities of cost overrun. The distribution fitting model and regression model are shown to have close estimates of project costs and cost ranges at each confidence level. Table of Contents Abstract i Table of Contents ii : v List of Tables vi List of Figures 1. Introduction and Background 1 2. Literature Review • • • 2 2.1. Introduction 2 2.2. Definition of Risk and Risk Analysis 2 2.2.1. Risk and Uncertainty 2 2.2.2. Risk Components and Definition 2 2.3. Necessity and Importance of Risk Analysis 3 2.4. Sources and Types of Risks 4 2.5. Methods of Risk Analysis 9 .2.5.1. Qualitative versus Quantitative Risk Analysis 9 2.5.2. Regression Models for Risk Analysis 10 2.5.3. The Decision Tree •• 12 2.5.4. Sensitivity Analysis 13 2.5.5: Probability Distributions Used in Project Risk Analysis 2.6. Empirical Studies on Magnitude of Cost Overrun Risk for Transportation Projects : 2.7. Conclusions on Literature Review ; 3. Risk Analysis Models and Discussions : 3.1. Introduction 13 14 15 16 16 3.2. Probability Distributions for Project Cost Overrun Risks .-. 17 3.2.1 Distribution Fitting Method vs. Monte Carlo Method 18 3.2.2 Probability Distribution for Project Cost Overrun Ratio 19 ii 3.2.2 Conditional Probability Distribution Model for Project Cost Overrun Ratio ;20 3.3. The PPP Models .22 3.3.1. Public-Private Partnership (PPP or P3) 22 3.3;2 Risk Allocation in PPP 24 , 3.3.3 Conditions for PPP Success , 24 3.3.4 Comparisons between Public, PPP and Private Projects 25 3.3.5 PPP Cost Overrun Risk Analysis Models 26 3.4. Regression Model 31 3.4.1. Objective and Importance 31 3.4.2 General Introduction of Risk Variables for the Model 31 3.4.3 Variable Specification and Analysis..... :. 33 3.4.4 Incorporation of Cost Overrun Risk into the Project Evaluation Process 38 3.5. Vancouver Island Highway Project Case Study : 40 3.5.1. The Database 40 3.5.2 Cost Overrun Ratio Distribution Fitting : 41 3.5.3 Monte Carlo Simulation Model Results vs. Beta Fitting Model Results. 44 3.5.4 Project Cost Overrun vs. Project Size in the VIHP 46 3.5.5 A Numerical Example of Predicting Project Cost for a New Project Using a Regression Model and a Beta Fitting Model 55 3.5.6. Multiple Regression Model Analysis : 58 4. How the Project Managers Can Use these Models 60 5. Summary and Conclusions : ....... 61 References...: Appendix 62 ;. ; ; '. Appendix 1. Beta Distribution Appendix 2 Maximum Likelihood Estimation for Beta Distribution Shape Parameters in •••••• 64 64 65 Appendix 3 Regression Analysis Reports for the VIHP 65 Appendix4. Distribution Summary in Arena 69 Appendix5. Multiple Regression Report 69 iv List of Tables Table 1. Akingtoye's Survey Conclusions 6 Table 2. Main Project Risk Sources and Their Impacts.. 7 Table 3. Main Findings in Four Project Cost Overrun Studies 14 Table 4. TRRL Report Conclusions 14 Table 5. Aalborg University Report Conclusions for Different Project Type..... 15 Table 6. Probabilities for Project Cost to Exceed Certain Values at Period n+1, with Beta Distribution (4, 6, 2, 15) and Project Budget (G) of $10 million 22 Table 7. Scenarios for PPP Projects Cost Overrun Risk and Revenue Shortage Risk Allocation in Private Sector's Perspective 27 Table 8. Cost Overrun Risk Bearer Dummy Variable Values under Different Scenarios 33 Table 9. Cost and Cost Overrun Ratio Ranges of VIHP data 41 Table 10. Cost Overrun Ratio Descriptive Statistic Report 42 Table 11. Beta Distribution Parameter Setting and Estimates 43 Table 12. Probabilities for Project Cost over Certain Values 44 Table 13. Beta Distribution Parameter Estimates Comparison for Cost Overrun Ratio Variable between Monte Carlo Simulation Method and Beta Fitting Method 45 Table 14. Cost Overrun Probability Estimation Comparison between Monte Carlo Method and Beta Fitting Method.......... 45 Table 15. Expected Cost and Confidence Interval Estimation for a Project with Budget of $1,200,000 -. .' 56 Table 16. Project Cost Prediction Intervals at Different Confidence Levels for a Project with Budget of $1,200,000 56 Table 17. Mean, Mode and Standard deviation Estimates 57 Table 18. Project Type Variable Values for Each Project Type 59 Table 19. Cost Overrun Ratio Means and 95% Confidence Intervals at Each Project Stage : v 60 List of Figures Figure 1. Risk, Uncertainty and Information Availability for Risk Events 2 Figure 2. Main Risks for Transportation Project . 9 Figure 3. A Simple Decision Tree Example 12 Figure 4. Beta Distribution (4, 6, 2, 15) PDF for Project Cost at period n+1 21 Figure 5. Cost Overrun Ratio Histograms for the VIHP data 42 Figure 6. Beta Distribution PDF Plot for Cost Overrun Ratio in the VIHP 43 Figure 7. Beta Probability Plot for Cost Overrun Ratio Variable .. 43 Figure 8. Probability Density Function Comparison between Monte Carlo Simulation Method and Beta Fitting Method 45 Figure 9. Scatter Plot of Cost vs. Budget for Road/Highway Projects 47 Figure 10. Scatter Plot of Log (Cost) vs. Log (Budget) for Road/Highway Projects 47 Figure 11. Project Cost vs. Budget Scatter Plot for Small R/H Projects 48 Figure 12. Estimated Cost vs. Budget Model for Small Road/Highway Projects 49 Figure 13. Cost Increase for Each $10,000 Project Budget Increase for Small Road/Highway Projects 49 Figure 14. Log (Cost) vs. Log (Budget) Scatter Plot for Medium Sized R/H Projects .... 50 Figure 15. Cost vs. Budget for Medium Sized Road/Highway Projects 50 Figure 16. Cost Increase for Each $50,000 Project Budget Increase for Medium Sized Road/Highway Proj ects 51 Figure 17. Project Cost vs. Budget Scatter Plot for Large Road/Highway Projects 51 Figure 18. Cost vs. Budget for Large R/H Projects , 52 Figure 19. Cost Increase for Each $500,000 Project Budget Increase for Large Road/Highway Projects 52 Figure 20. Cost vs. Budget for Bridge and Tunnel Projects 53 Figure 21. Log (Cost) vs. Log (Budget) for Bridge and Tunnel Projects 53 Figure 22. Log (Cost) vs. Log (Budget) for Large Bridge and Tunnel Projects 54 Figure 23. Cost vs. Budget for Large Bridge and Tunnel Projects (Budgets over $850,000) 54 Figure 24. Cost Increase for Each $500,000 Project Budget Increase for Bridge/Tunnel Projects with Budget over $850,000 55 Figure 25. Prediction Intervals at Each Prediction Confidence Level for a Project with Budget of $1,200,000 56 Figure 26. Comparisons on Project Cost Prediction and Confidence Intervalsfromthe Regression and Beta Fitting Models respectively 58 vi 1. Introduction and Background Today, with globalization and urbanization, transportation infrastructure construction projects are carried out all over the world. According to Transportation Canada's provincial transportation investment report, transportation investment in each province (including purchases of machinery and equipment) constitutes more than a quarter of total social investment in Newfoundland, Prince Edward Island and Nova Scotia between 1992 and 1995. In Quebec, Ontario and the western provinces, the corresponding ratio is about one fifth. Transportation infrastructure construction projects are stated as a major contributor to economic growth and community well-being. This point is emphasized in the reports of Transportation Canada. Yet, we frequently hear reports of project failures. The abandonment of Mirabel Airport in Montreal, Canada, is such an example. Transportation project failures often cause millions or even billions of dollars in losses to the society, the costs of which are generally transferred to investors and taxpayers. Although Mirabel Airport is an extreme example, most transportation infrastructure projects are vulnerable to another widely existed risk which imposes magnificent costs on the society as well: cost overrun. Cost overrun is the excess of actual project costs over budgeted project costs and may be caused by underestimation of costs at the planning stages or by the escalation of the original scope of the project. Cost overrun may not necessarily lead to project failure if the project can obtain sufficient revenue and benefits to cover its costs. However, the viability of the project, initially based on the underestimated costs, would be in question when cost overrun risk materializes. Projects with underestimated costs are chosen for their higher estimated NPV compared with other alternative projects. With cost underestimation, the choice becomes questionable. The main objective of transportation infrastructure projects is to improve net social benefit. Net social benefit can be calculated by deducting a project's social costs from its social benefits. However, with the large and irreversible investments required for transportation projects, neglecting or underestimating project risks creates large costs for society - exactly opposite to projects' general objective and viability. Risk analysis is therefore crucial to transportation projects. Underestimation of project risk may cause large social costs. On the other hand, overestimation of risk, leading to over-conservative designing, planning and budgeting, may lose potential social benefits. Reliable risk analysis is important to transportation project cost benefit analysis and management. The main tasks of this thesis are to analyze cost overrun risk factors for transportation infrastructure projects, to propose models to quantify their influences on project cost overrun, and to fit proper probability distributions for project cost overrun risk variables. This work focuses on the development of models for transportation project cost overrun risk analysis, including probability distribution models, Public-Private-Partnership models, and risk factor regression analysis models. Vancouver Island Highway Project 1 case study will be an application example for probability distribution and regression analysis models. Project cost estimates using different analysis models are compared. Hopefully our analysis, models and conclusions will help project evaluators make better risk-adjusted cost estimation for transportation infrastructure projects. . 2. Literature Review 2.1. Introduction This literature review provides a general picture for the application of risk analysis in transportation project assessment and builds a theoretical base for our project cost overrun risk analysis models. It covers topics of risk definition, project risk sources, and risk evaluation methods. Section 2.2 will review definitions of risk and risk analysis; Section 2.3 explains the importance and necessity of risk analysis for transportation project assessment; Section 2.4 identifies the main risk sources for transportation projects; Section 2.5 discusses some widely used methods in risk analysis; Section 2.6 shows some interesting results and conclusionsfromseveral empirical studies of project risk. 2.2. Definition of Risk and Risk Analysis 2.2.1. Risk and Uncertainty As Pouliquen (1970) states, risk analysis is a method for dealing with uncertainty. However, risk and uncertainty are not synonymous. Frame (2003) explains the difference is that when making decisions under conditions of risk, we know or can estimate the probability distribution of this risk event, whereas under conditions of uncertainty, we are unable to estimate the probability distribution. So Frame distinguishes risk and uncertainty according to information availability. Figure 1 roughly depicts this difference. Figure 1. Risk, Uncertainty and Information Availability for Risk Events . Total Ignorance• Perfect Information " Uncertainty Risk (Sources: Frame (2003)) . ^ : y.- f- •;, Referring to project risk and risk analysis instead of project uncertainty, we actually assume we know or can estimate the probability distributions for project risks. Project risk analysis should be based on these probability distributions. 2.2.2. Risk Components and Definition In some cases, such as in financial markets, the risk often means the variability of return. Two stocks with the same expected return might be categorized differently with respect to risk, because of different degrees of variability of their returns. The riskier stock (with greater variability of return) may bring a higher return than the less risky stock in favorable scenarios. It may also bring a lower return than the less risky stock in unfavorable scenarios. Investors with different level ofrisktolerance will have different preferences toward these two stocks. Therisk-aversepeople would prefer the stock with 2 less return variability and the risk-takers would prefer the one with greater variability. Normally, investors behave to be risk averse. Similarly i n project management, two projects may have equivalent expected net present values, but reasonable project sponsors and investors are assumed to prefer the one with less variability in expected N P V . Although i n some cases risks are simply interpreted as return variability, risk likelihood and risk impact are the two important faces o f risk. Neglecting either o f these two elements leads to an incomplete view o f risk. A s Frame (2003) points out, some risk events may have high impact but have such low likelihood o f happening that we normally do not take them into concern. H e gives the example o f a comet hitting the earth. Cooper and Chapman (1987) define risk as the "exposure to the possibility o f economic or financial loss or gain, physical damage or injury, or delay, as a consequence o f the uncertainty associated with pursuing a particular course o f action". A n d risk analysis is said to involve various approaches for dealing with the problems caused b y uncertainty including the identification, evaluation, control and management o f risks. Sturk et al. (1996) defines risk as "expected consequence" or "expected loss o f utility". Based on this definition, they evaluate risks as: Risk = Probability x Consequence Similar to Sturk et al, Jaafari (1999) defines risk as "the exposure to loss/gain, or the probability o f occurrence o f loss/gain multiplied b y its respective magnitude". F o r example, project cost overrun risk can be interpreted as the product o f project cost overrun probability and the cost overrun magnitude. These formula and definition are rather for risk definition than for risk measurement. They are used to articulate the common view that risk has two components: likelihood and impact. Every project faces the risk o f failure. Project failure can be caused b y cost overrun, benefit shortage, construction delay, and bad quality. A n d these risks may be traced further to more specific risk factors. Our interest here is what these risk factors are and how they affect project consequences. 2.3. Necessity and Importance of Risk Analysis In cost benefit analysis o f projects, the most common methods used today are based on net present value estimation! However, Pindyck (1991) argues that the N P V rule, "Invest in a project when the present value o f its expected cash flows is at least as large as its cost" would not be reliable any more i n scenarios with large irreversible investments and the option o f postponing investments. Investment irreversibility, accompanied b y large sunk costs, is common to transportation infrastructure projects. Pindyck finds that irreversibility makes a project much more sensitive to project risks . H i s study illustrates the unreliability o f traditional cost benefit analysis, which simply relies on the N P V rule, and the necessity o f analysis o f project risks. 1 Project risks refer to the uncertain events that may cause project failure. These risks normally can be depicted by some random variables. 1 3 Flyvbjerg (2003) also argues that risk analysis is very important for project decision makers in order to obtain a clearer and more realistic view of possible outcomes. Risk analysis is fundamental for risk management. It provides project managers with a basis for identifying strategies to reduce risk, including: allocating risk among parties, transferring risks to professional risk management institutions, hedging positions in financial markets, and so on. As Cooper and Chapman (1987) point out, risk analysis is crucially important when lowprobability, high-consequence events are critical for decision making. These risk events are very likely to be neglected in the calculation of the net present value of projects. For examples, terrorist attack has very low probability of happening. However, without any preparation for its happening, the outcome would be an even larger disaster. As an example, New York City's preparation for disaster system helps to prevent the terrorists' attack on World Trade Center from thousands of additional life loss. Risk analysis is important for better understanding project risks, risk effects and risk interaction, according to Cooper and Chapman's discussion. Risk analysis requires a conscious and systematic method to analyze the risks. During this process, project investors and evaluators can gain more complete and detailed knowledge about risk events, interactions of these risk events, and their relationships with project consequences. Furthermore, based on knowledge of these risk events and relationships, project investors and designers can make better plans and designs to control potential loss, in case risk events are materialized. Thus risk analysis can help with better planning and responses for risk events; help and provide feedbacks to project design and planning; and help with better project construction and operation risks handling. Cooper and Chapman conclude that risk analysis is especially important when projects require large capital investments; have unbalanced cash flows; require a large proportion of total investment before any returns are obtained; are subject to significant new technology; have unusual legal, insurance or contractual arrangements; are subject to important political, economic or financial parameters; face sensitive environmental or safety issues; or have stringent regulatory or licensing requirements. Transportation infrastructure projects obviously fit this description. Although risk analysis can neither prevent nor accurately predict risks, it can facilitate superior preparedness and responses to risks. One example is a transportation project designed to meet the emergency safety requirement for minimizing life loss in case of fire. 2.4. Sources and Types of Risks Ayyub (2003) discusses potential risk events for projects, including technological risks, economic climate risks, political risks, large project risks, and contractual or legal risks. Technological risk refers to the fact that rapidly improving technology may make a project out-dated and abandoned. Technology planned for a project can require replacement by newer technology during the project construction period, adding to 4 project costs. The decision to replace the old technology or not should be based on whether the.benefit of the new technology will at least compensate for the cost of replacing the old one. In some situations, there may be no option: the new technology will be necessary due to technological improvement throughout the industry or technological improvements of complementary equipment. Project outcomes are influenced by social and domestic economic trends, called economic climate. The economic growth of the society has high impacts on traffic volume, financial market stability, and project performance. Political risk refers to the circumstances under which projects are subjective to political factors. For example, foreign investors in some developing countries may face unfriendly local governments, or potential expropriation. Since large transportation projects often require approval and financial support from local and federal governments, conflicts between and within governments may bring about project failure. 1 Contract and legal risks arise from inappropriate responsibility division among contractors. During the whole project period, land using, payment dispute, and other legal issues might be raised unexpectedly. These contractual and legal issues may cause delay, suspension or even abandonment of the projects. Unfortunately, Ayyub (2003) does not give theoretical analysis or quantified estimation for these risks' impacts on project outcomes. For transportation infrastructure projects specifically, Flyvbjerg (2003) emphasizes the financial risks, including: cost overrun, revenue shortage (lower than expected revenues, caused by insufficient traffic volume) and increased financing costs (exchange rate for cross-nation projects and interest rate). Inefficient project management, contractor conflicts and accidents may cause construction cost overrun. Benefit shortage is caused directly by insufficient users, probably due to poor project design or social economy changes. Given the huge amount of debt required for financing transportation projects, even a small change in interest rates can increase financing costs dramatically and affect the viability of the projects. Investors finance their investments in capital markets via bonds, stocks and other borrowing tools. Changes in interest rates influence the values of these financing tools and influence project financing costs. For those projects financed in international capital markets, exchange rates are another concern. With regard to large projects, Jaafari (2001) suggests project risks include: market, political, technical,financing,environmental, cost estimate, operating, etc. Market risks refer to the adverse changes in labor, material or any other supply markets, which may increase costs or decrease benefits. Political risks are caused by policy or regulation changes from political sectors. Technical risks refer to situations in which new technologies create the possibility of not achieving budget, schedule or other targets. Financing risk is correlated with cost overrun, and benefit shortage. Environmental risks refer to the possibility of natural hazards. Cost estimate risk arises from estimated costs 5 ) that are insufficient compared with actual costs. Operating risks refer to the unexpected events which adversely affect project operation. < Based on their surveys on the Kuwait construction industry, Kartam et al. (2001) present their findings about Kuwait contractors' perceptions of sources of risk. Financial factors are regarded as the most significant risks, followed by contractual and labor, material and equipment availability. Kartam et al. also find that sub-contracting is a powerful tool for minimizing project risks, as long as risks are properly allocated between the investors and contractors. According to Akintoye (1997), construction project risks can be defined as variables causing variability in construction project costs, duration and quality. They identify environmental, design, financial, legal, political, construction and operation risks. According to their survey on UK project contractors and managers, there exist some differences between these two groups' perceptions for individual risk source premium ranking. However, the two groups draw similar conclusions regarding the relative importance of these risk sources. They both regard financial and contractual risks as the most important types of risks to projects, consistent with Kuwait contractors' views according to Kartam. Their views are summarized in Table 1. Table 1. Akingtoye's Survey Conclusions Risk Sources Perception of Risk Premium Contractor Low (e.g. Moderate Environmental (e.g. weather) Political, Social & Economic inflation) Contractual agreement (e.g. responsibilities) Financial Construction (productivity, injury, safety) Market/industry (availability of workload) Company (corporate) Development in IT Project (design information) (Sources from Akintoye (1997)) Manager Moderate High High High High Moderate High Moderate Low High High High High Moderate Low High According to this survey, both contractors and managers depend mainly on their intuition and subjective judgment to manage risks. About half of managers claim to be familiar with sensitivity analysis , yet few managers use this technique in practice. Based on this survey, contractors and managers are said to doubt on the usage of quantitative risk analysis techniques in practice. 2 Shapira (1994) also finds that managers are quite 'insensitive' to the probability estimation for project outcomes. They seem to emphasize outcome values more than Sensitivity analysis is a method of risk analysis that examines the influence of possible changes to project parameters on project NPV. We will discuss this method in detail in Section 5. 2 6 probabilities. Shapira reasons that this is due to managers' beliefs that risks can be controlled. Unfortunately, this belief may not necessarily be true. The backup insurance companies and other financial instruments may create the sense to project managers that risks are controllable. However, project managers neglect the fact that there always exist some risks that are impossible to be controlled, such as social economic trends, which could put insurance companies andfinancialmarkets into difficulty as well. Shapira finds that under unique, non-repeated decision conditions, managers neglect statistical analysis easily. This kind of managerial behavior has significant influence on the reliability of project cost benefit analysis. Public-Private-Partnerships are now widely used in transportation infrastructure projects, as well as other projects requiring large investment which exceeds the means of private investors or governments alone. One example of a Public-Private-Partnership is "BOT" (build-operate-transfer), where private companies build and operate the infrastructure for a certain period then transfer its ownership to the public sector. The key risks of design, construction, cost overrun and financing are believed to be transferred to private companies. The popularity of PPP depends on the assumption that private sectors can manage certain project risk better than the public sector can. According to Grimsey and Louis' (2002), risks of PPP projects differ little from those of other projects and can be evaluated and managed similarly. The main risks for PPP projects are said to be caused by the complexity of the PPP arrangements themselves, in terms of complicated documentation, financing, sub-contracting and so on. In PPP projects, the success of documentation depends on interactions between different contractors and investors. It requires additional effort and time to reach agreements on the issues raised by different contractors. The assignment of responsibilities and rights among the parties and the complicated contract issues become key sources of risk to the projects, in addition to the risk factors present in any single project sponsor scenarios. We discuss PPP project risks in details in the PPP model section. The above studies provide a general portrait of major risk factors in transportation projects. They help identify risk variables for our models. Table 2 provides a summary of them. Table 2. Main Project Risk Sources and Their Impacts Risk Factors Major impacts on Project Management Perception Financial Risk Economics Risk Benefit deficit Benefit deficit and Cost overrun Benefit deficit and Cost overrun Cost overrun Cost overrun Project termination Benefit deficit, Cost overrun and Project abandonment High High Construction Risk Contractual Risk Environment Risk Funding Risk Project Design Risk . 7 High High Moderate High High One important issue here is whether.these risks are measurable. Due to the uniqueness of each project, exact project risk probability distributions for individual projects are almost impossible to obtain. Previous project data could be used to obtain approximate probability distributions for these risk factors. Based on the probability distributions, we can estimate the likelihoods of project risks for individual projects. The risk analysis method section in the Literature Review provides a more detailed discussion of risk measurement. The risk factors discussed above are mostly in project investors and contractors' perceptions. Since project cost benefit analysis is made from society's perspective, externalities may be another significant concern for project risk sources. For example, the rapid transit line connecting Vancouver and Richmond (RAV) may expose residents along its route to noise and safety issues, which may lead to conflicts or even law suits between the project operators and the residents. Externality risks should be included in project risk analysis for RAV line project evaluation. - Identification of project risk factors and their relationships are presented in Figure 2. This structure will be used as the basis for our regression analysis. As reflected in Figure 2, transportation project failure can be caused by negative project 3 NPV, funding risk, externality risk and design risk. Funding risk refers to the possibility that investors may fail to provide further funding. Externality risk materializes when large negative externality of the project causes legal suits from the affected community, which may lead to construction delay, suspension or even project abandonment. Poor project designs not only influence costs and benefits, but only affect quality adversely, making projects vulnerable to other risks such as natural hazards. Negative project NPV is directly caused by cost overrun and benefit shortage. Cost overrun can be caused by construction risk, project design risk, environmental risk, contractual risk and economic risk. Design, financial, economic and construction risks are the major contributors to project benefit shortage. These risk factors are somehow correlated. For example, economic risk may influence financial and construction risk, due to their sensitivities to social economic changes. Environmental and contractual risks may increase construction risk. Changes in weather and natural hazards may cause construction delays, safety issues and so on. Contractual disputes may cease project construction, causing project construction delay and productivity or quality issues. 3 The words in italics are corresponding to the terms in Figure 2 . 8 Figure 2. Main Risks for Transportation Project Funding Risk Negative NPV Interest . Rate; Inflation Rate • Revenue Deficit'or, Demand: Risk J Construction Delay; Quality Issue;.Safety; Productivity Externality Risk; Weather; Natural > Hazard K. Design Risk Responsibility • Issues; PPP Issues J K. (Sources from studies in section 2.4 in this Literature Review) 2.5. Methods of Risk Analysis 2.5.1. Qualitative versus Quantitative Risk Analysis According to the methods used for risk assessment, risk analysis can be categorized as either qualitative or quantitative. Qualitative analysis assesses variable probabilities and consequences using subjective judgments, ^expertise opinions and experiences. Quantitative analysis is based on probabilistic and statistical methods. To contrast these two: When one declares some risk events are very likely to occur, he/she is doing qualitative risk analysis. When one demonstrates that the probability for a specific risk event to happen is about 60%, which is based on statistical analysis of historical data, he is doing quantitative risk analysis. As Ayyub (2003) concludes, qualitative analysis may be sufficient to identify the risks for a system, while quantitative risk analysis examines the system in greater details for risk assessment. Frame (2003) argues that narrative statements in qualitative risk analysis are normally unclear and subject to varied interpretations by different people. Furthermore, they are often not testable. Therefore, it is difficult to judge whether their conclusions are wrong or not. Quantitative risk analysis, on the other hand, generally gives, clear and testable conclusions. According to Ayyub and Frame's arguments, quantitative risk analysis is important for giving project evaluators clear and detailed views of project risks, which cannot be replaced by qualitative risk analysis. 9 Based on a survey of project managers, Lyons (2004). finds that the most commonly used tools for risk identification in practice are brainstorming, the case-based approach and the checklist. His finding is consistent with several other researchers' findings, as discussed in this Literature Review, such as Akintoye's. Intuition, judgment and experience are the most frequently used risk assessment techniques in practice. Qualitative risk assessments are said to dominate quantitative ones. However, according to this survey, there is no significant factor to limit implementation of quantitative risk analysis in practice. It is said that all these factors, including: cost concerns, difficulty in seeing benefits, human/organizational resistance, lack of accepted industry model for risk analysis, lack of dedicated resources, lack of expertise in the techniques, lack of familiarity with the techniques, lack of information, and lack of time, were shown to be low or moderately relevant to managers' reluctance to do quantitative risk analysis. One agreement from the above papers and survey results is that project managers normally do not make quantitative risk analysis in practice. They seem to rely on qualitative risk analysis only. This is not because they lack the techniques of quantitative risk analysis, but probably due to their lack of belief in its value. Despite the importance to use expertise experiences and judgments in projects, absolute dependence on subjective judgments may bring severe problems. First, it is questionable about the qualification and definition for the expertise. Who can be titled as an expertise? What are the criteria? Even for a person spending his lifetime on project assessment, his experience is still quite limited, compared with the huge project database used to achieve statistical analysis. And his judgments and estimates may not be accurate, even if they are not mixed with his personal bias. Second, absolute dependence on subjective analysis may easily raise moral hazard problems in project evaluation. One example is the overestimation for future project user numbers and project benefits, which can be found now and then in project cost benefit analysis to balance the project budgets. The project evaluators are normally employed by the project sponsors. They are very likely to be influenced for the project sponsors' interest concerns. And this challenges the objectiveness of their analysis and evaluation. Although quantitative risk analysis does not eliminate this problem, it can help control it. Reliable and easily interpretable quantitative risk analysis is necessary for improving project risk analysis. 2.5.2. Regression Models for Risk Analysis Regression models are widely used to analyze the influences of risk factors. Not only are they effective for explaining risk events and quantifying the impacts of risk factors, they are also easily understood by people. The Center for Operation Excellence at the University of British Columbia undertook a project to analyze the risk of oil leakage for an oil shipping company. Using historical data, Poisson regression model is successfully used to analyze the relationships between potential risk factors and oil leaking risk. Some interesting and unexpected conclusions about the risk factors' influences on oil leaking risk are concluded from that regression model, which shows the existence of subjective misunderstanding or bias in previous qualitative risk analysis. 10 Flyvbjerg (2004) uses simple linear regression models to analyze the relationships between transportation project cost overrun and three potential risk factors respectively: length of project implementation phase, project size and project ownership. The length of implementation phase is defined as the period from the project construction decision to completion of construction. Flyvbjerg treats projects as outliers if their implementation phases are 13 years long or even longer. The estimated regression model for project cost overrun and project length is AC = 0.4 + 4.64*7, where AC is the cost escalation (in %) and T is the length (years) of the implementation phase. The 95% confidence interval for the slope is (2.10, 7.17). According to this estimated model, for every additional year of project implementation, the project cost escalation is expected to increase by 4.64%. The influences of implementation phase length on cost escalation are similar for all modes: rail, fixed-link (bridge and tunnel) and road projects. Using project budgets to represent project sizes, Flyvbjerg.looks into the impacts of project size on project cost. Flyvbjerg finds that for bridge and tunnel projects, larger projects tend to have larger cost escalations; however, for rail and road projects, this relationship is not significant. For all project types, the data do not support the assumption that abigger project has a larger risk of cost escalation.. In Flyvbjerg's model, project ownership is categorized into private, state-owned enterprise, and other public ownership. The data do not support the assumption that public ownership is problematic and private ownership is a main source of efficiency in controlling cost escalation. State-owned enterprises show the poorest performance with an average cost overrun of 110%. Privately owned fixed links have an average cost overrun of 34%. Other public ownership shows the best performance with an average cost overrun of "only" 23%. Flyvbjerg argues that the main problem in relation to cost overrun may not be public versus private ownership but a certain kind of public ownership, namely state-owned enterprises, which lack not only the transparency and public control that public sectors would implement but also the competence the private sector would bring. Flyvbjerg's models test the assumed impacts of these three potential risk factors on project cost Overrun. Regression models help to provide intuitive and accurate understandings for these impacts. For example, he concludes that the average cost escalation would be about 4.64% for each additional project year, instead of the vague and subjective assumption that project delay may cause cost overrun. However, Flyvbjerg investigates only the individual impact of each risk factor on project cost escalation and neglects the combined impact of several risk factors on project cost. He does not discuss the relationships between the risk factors. However, the relationships between these risk factors may influence the estimates of their individual impacts on project cost. He uses simple linear regression models for his study on the impacts of risk factors. He does not explain the viability of using simple linear regression models. 11 An advantage of using a regression model for risk analysis is its intuitive appeal. The uncertainties projects face can be related to risk factors with economical meanings. For example, uncertainty about interest rate changes can be represented by the sensitivity of project performance on interest rate changes, using the regression coefficient estimate for the interest rate change variable estimated in the regression model. 2.5.3. The Decision Tree A decision tree is another risk analysis tool widely used in practice. Cooper and Chapman (1987) give a simple example for this method. For each single cost risk factor, we can give the probability of its happening. For its happening case, the risk event can be grouped, for example, Minor, Modest, Major or Maximum, with their respective probability. For each group of risk events, its impact on the project can be further grouped into sub-groups, such as None, Negligible, Significant, and Catastrophic, based on probability estimation. The outcome under each condition can be calculated based on this structure. Figure 3. A Simple Decision Tree Example Project Risk • Project NPV- Boardman and Greenberg (1996) carry out decision tree analysis in two basic stages. First, they use a diagram, called a decision tree, to specify the logical structure of the decision problem with sequences of decisions and their contingencies. Second, work' backward from final outcomes to the initial decision, calculating expected net benefit values under each condition and finding out the branch with largest net benefit value. As an example, for the decision tree in Figure 3, the expected NPV of "Taking Project" would beP x NPV + P x NPV . The expected NPV of "Not Taking Project" would be the l ] 2 2 result of Do-Nothing strategy . If P x NPV^ + P x NPV is greater than the quantified value of Do-Nothing strategy, we would take the project, otherwise, we would suspend or abandon the project. 4 x 2 2 Let P = 0.6, P = 0.4, NPVi= -2, NPV = 2, and quantified Do-Nothing result is -0.2, the expected NPV of "Taking Project" is -0.4, less than the quantified result of Do-Nothing strategy, -0.2. Therefore, we should not carry on with this project. t 2 2 Quantified result of Do-Nothing Strategy is often a negative value and not necessarily equal to 0. For example, without necessary road expansion, traffic congestion would lead to extra costs to society making the N P V of "not taking" the road expansion project would be negative. 12 The decision tree method is very intuitive for risk analysis. However, it is generally used under conditions of discrete probability distributions and may not applicable to analyze risk factors with continuous and correlated probability distributions. 2.5.4. Sensitivity Analysis As shown in the surveys on contractors and managers in section 2.4, sensitivity analysis may be the most frequently used risk analysis method in project evaluations. In many cases, it is the only risk analysis tool used in project cost benefit analysis. Partial sensitivity analysis takes uncertainties into cost benefit analysis by varying cost and benefit estimation within a certain presumed ranges of variability. Range of expected project net present value can be estimated. Boardman and Greenberg (1996) mention worst-and-best-case analysis, checking whether combinations of reasonable assumptions reverse the sign of net present value. Worst-and-best analysis is used to obtain the most conservative and most optimistic estimates for project net present values. For example, project evaluator may have conservative estimates for three independent risk factors. Using these conservative estimates for risk variables, which is the worst-case scenario, the worst-case estimate can be obtained. Similarly, the best-case estimate can be calculated by using optimistic estimates of risk variables. Worst-and-best case analysis gives a range of project performance estimation. Neither partial sensitivity analysis nor worst-and-best case analysis takes into account the probability distributions of risk factors and project outcomes. They provide only the range of net present values for changes in risk factors. 2.5.5. Probability Distributions Used in Project Risk Analysis For choosing proper probability distribution, Pouliquen (1970) argues that the target of risk analysis is not to find the real distribution but rather the distribution best representing "the judgment of an appraisal team". If this statement is true, a probability distribution would be valid as long as it reflects the project team's experience and understanding of project. For example, the project appraisal team may think one project risk factor follows a normal distribution based on their knowledge and experience. Then in risk analysis, this risk factor can be assumed to follow the normal distribution. This argument is obviously based on the strong assumption on the appraisal team's reliability and credibility. Two well-known probability distributions are used widely in project risk analysis: the normal distribution and the beta distribution . 5 Frame (2003) concludes that the normal distribution is generally used when routine processes are involved. For example, the duration of each bus trip is likely to follow the normal distribution for its repeated and routine schedules. However, the normal distribution is not generally useful when unique events are involved, for example, a 5 See Appendix 1 for introduction of Beta distribution. 13 transportation project using new technology. The normal distribution is not feasible to those variables with upper or lower bounds, or has a PDF (probability density function) shape different from the normal distribution's bell shape. Regnier (2005) states that the probability distribution of project activity time is widely represented by the beta distribution due to desirable properties of Beta distribution. • The Beta distribution has finite limits. Many real-world random variables have lower and upper bounds. For example, project duration has a lower bound that is greater than zero. The model should reflect this property of these random variables. • The Beta distribution can be asymmetric. Many variables in reality are right or left skewed to some unlikely but significant result. • The Beta distribution is flexible and can reflect different shapes, including U and inverted-U shapes. 2.6. Empirical Studies on Magnitude of Cost Overrun Risk for Transportation Projects Flyvbjerg (2003) describes four studies comparing budgeted and actual costs for transportation projects. His main findings are listed in Tables 3, 4 and 5. Table 3. Main Findings in Four Project Cost Overrun Studies Study Sample Size ' __. Average Cost Overrun Range Lower Bound Upper Bound Auditor-General of Sweden 15 86% 2% 182% US Department of Transportation 61% -10% 106% TRRLUK 13 Aalborg University, Denmark 258 28% 1. The study by Auditor-General of Sweden study covers rail and road projects. For rail projects, the cost overrun average was 17%, rangingfrom-14%.to 74%. This study might underestimate the real cost overrun, since two-thirds of its sample projects were still carried on. 2. Rail projects only 1 2 3 4 3. TRRL: Transportation and Road Research Laboratory Table 4. TRRL Report Conclusions Cost Overrun # of Metro Projects -10%-20% 4 (Sources: Flyvbjerg (2003)) 20%-50% 3 50%-100% 4 100%-500% 2 4. Covers bridge, tunnel, road and rail projects, located in 20 countries, and completed between 1927 and 1998. Its project cost overrun mean and standard deviation estimates for each project type are listed in Table 5. 14 T a b l e 5. A a l b o r g U n i v e r s i t y R e p o r t C o n c l u s i o n s for D i f f e r e n t P r o j e c t T y p e Project Type Average Cost Overrun Standard Deviation Rail 45% 38 Tunnels & Bridges 34% 62 Road 20% 30 (Sources: F l y v b j e r g (2003)) T h e s e f o u r studies have a c o m m o n c o n c l u s i o n : cost o v e r r u n o c c u r s c o m m o n l y a m o n g transportation projects. A c c o r d i n g to the A a l b o r g U n i v e r s i t y report, standard deviations o f cost o v e r r u n for their sample projects are large. A s s u m i n g project cost o v e r r u n f o l l o w s the n o r m a l distribution, the 9 5 % c o n f i d e n c e i n t e r v a l for cost o v e r r u n e s t i m a t i o n for r a i l projects w o u l d be - 3 0 % to 120%. F o r tunnel and b r i d g e projects, the c o n f i d e n c e i n t e r v a l w o u l d be w i d e r , due to their larger standard deviations. T h e h i g h l i k e l i h o o d o f cost o v e r r u n i n transportation projects increases the i m p o r t a n c e o f cost o v e r r u n risk analysis i n transportation project evaluation. A n o t h e r interesting finding i n the A a l b o r g u n i v e r s i t y report is that project cost o v e r r u n has not i m p r o v e d over the past seventy years. T h e r e seems to be n o element o f learning i n transportation project cost e s t i m a t i o n and management, despite i m p r o v e d k n o w l e d g e , e x p e r i e n c e and technology. H o w e v e r , due to the v a r i e d locations o f the s a m p l e projects, it is not clear whether this c o n c l u s i o n is v a l i d i n a l l the states or j u s t s o m e o f them. T h e Aalborg university report finds that cost underestimation and overrun are more p r o n o u n c e d i n d e v e l o p i n g countries than i n d e v e l o p e d countries. T h e p o s s i b i l i t y exists therefore that this n o - l e a r n i n g characteristic c o m e s from including more developing c o u n t r y observations i n recent years. Further analysis is necessary to d r a w a c o n c l u s i o n . 2.7. Conclusions on Literature Review R i s k analysis is important to transportation infrastructure projects because o f their large i r r e v e r s i b l e investments and h i g h s e n s i t i v i t y to risks. A l t h o u g h managers and contractors a l m o s t r e l y o n l y o n qualitative r i s k analysis i n practice, quantitative risk analysis is essential for better project r i s k assessment and management. T r a n s p o r t a t i o n projects encounter a l o n g list o f risks i n c l u d i n g e c o n o m i c , f i n a n c i a l , safety, contractual, regulatory risks and natural hazard. H e r e w e f o c u s o n the r i s k o f cost overrun. D e s p i t e o u r focus o n project cost o v e r r u n risk, cost o v e r r u n is not n e c e s s a r i l y a p r o b l e m i t s e l f for transportation projects. It is a deficit i n project net present v a l u e caused b y cost, o v e r r u n that w i l l lead to failure o f transportation projects. U n f o r t u n a t e l y , current literature does not p r o v i d e p r o b a b i l i t y d i s t r i b u t i o n analysis for transportation project risk factors either f r o m e m p i r i c a l studies or i n theoretical analysis. T h i s i m p e d e s the a p p l i c a t i o n o f quantitative risk analysis to project cost benefit analysis. F u r t h e r m o r e , although w o r k s o n project risk factor i d e n t i f i c a t i o n h a v e b e e n done, f e w theoretical studies have been c o n d u c t e d to b u i l d m o d e l s to q u a n t i f y the i m p a c t s o f these r i s k factors o n project consequences. In this thesis, w e attempt to m a k e up for these m i s s i n g p o i n t s i n project cost o v e r r u n risk analysis. 15 3. R i s k A n a l y s i s M o d e l s a n d Discussions 3.1. Introduction F r o m the L i t e r a t u r e R e v i e w , w e k n o w that r i s k is c o m p o s e d o f l i k e l i h o o d a n d i m p a c t s . N e g l e c t i n g either one w i l l lead to an i n c o m p l e t e u n d e r s t a n d i n g o f project r i s k s . T h e L i t e r a t u r e R e v i e w also discusses some methods for project r i s k analysis, i n c l u d i n g r e g r e s s i o n m o d e l s , d e c i s i o n trees, a n d sensitivity analyses. E a c h m e t h o d has advantages and disadvantages. B o t h the d e c i s i o n tree a n d the s e n s i t i v i t y analysis methods are case-based. F o r each i n d i v i d u a l project, w e use a d e c i s i o n tree a n d s e n s i t i v i t y a n a l y s i s to a n a l y z e project r i s k s . T h e i r dependence o n i n f o r m a t i o n f r o m p r e v i o u s projects is m o s t l y brought b y project e v a l u a t o r s ' subjective experiences a n d v i e w s . Quantitative a n a l y s i s o n r i s k p r o b a b i l i t y and i m p a c t s are necessary to support these k i n d s o f case-based m e t h o d s , not o n l y f o r better u s i n g the i n f o r m a t i o n f r o m h i s t o r i c a l data, but also for a l l e v i a t i o n o f subjective bias i n r i s k analysis. It w o u l d be n o use to take h i s t o r i c a l data into account i f a project is absolutely u n i q u e , w h i c h means it is b e i n g c a r r i e d out w i t h absolutely n e w t e c h n o l o g y , d e s i g n , a n d materials i n a n a b s o l u t e l y n e w e n v i r o n m e n t , under n e w p o l i c y a n d so o n . H o w e v e r , project uniqueness does not m e a n projects are absolutely u n i q u e , rather that each project has s o m e u n i q u e features. That i s , projects m a y share s o m e c o m m o n r i s k factors, w h i l e each is also affected b y some u n i q u e r i s k factors. In section 2.4 o f L i t e r a t u r e R e v i e w , w e d i s c u s s s o m e c o m m o n project r i s k factors. U n d e r s t a n d i n g the i m p a c t s o f these c o m m o n r i s k factors o n project outcomes w i l l h e l p to better m a n a g e these r i s k s a n d to better estimate project outcomes. R e g r e s s i o n m o d e l s u s i n g h i s t o r i c a l data are useful to test a n d q u a n t i f y the i m p a c t s o f these c o m m o n r i s k factors o n project outcomes. F o r e x a m p l e , u s i n g r e g r e s s i o n analysis, project evaluators c a n c o n c l u d e that project cost is expected to increase 4 % for each oneyear increase i n the length o f the project c o n s t r u c t i o n phase, instead o f the subjective a n d obscure statement "project delay m a y increase project cost". I n a d d i t i o n , this i m p a c t estimate o f r i s k w i l l help project evaluators obtain m o r e accurate project cost benefit a n a l y s i s , assist project d e c i s i o n - m a k e r s m a k e better d e c i s i o n s a n d strategy a n d i m p r o v e project m a n a g e r s ' risk management strategy. I n c o r p o r a t i n g project r i s k estimates into project e v a l u a t i o n c a n m a k e cost and benefit e s t i m a t i o n m o r e realistic. Project r i s k estimates also p r o v i d e a quantitative reference to project d e c i s i o n m a k e r s a n d managers. T h e s e estimates c a n be different from project evaluators' subjective expectations o n project r i s k s . T h i s difference a c t u a l l y y i e l d s some useful i n f o r m a t i o n from h i s t o r i c a l data for project evaluators' reference. I n S e c t i o n 3.4.4, w e w i l l discuss i n c o r p o r a t i n g project r i s k estimates into project e v a l u a t i o n i n details. A s d i s c u s s e d i n the Literature R e v i e w , w h e n w e analyze r i s k s , w e cannot neglect either the p r o b a b i l i t y o f the risk event o r the i m p a c t o f the r i s k event. A c c o r d i n g l y w e w i l l p r o p o s e t w o data-based m o d e l s to analyze project cost o v e r r u n r i s k . T h e p r o b a b i l i t y 16 distribution model can be used to estimate the probability of project cost overrun risk. The regression model would be used to estimate impacts of project risk factors on project cost. • ' We will introduce probability distribution models to analyze project cost overrun risk variables in Section 3.2. In Section 3.3, we discuss Private Public Partnership (PPP) in transportation projects and propose a PPP cost overrun risk analysis model. In Section 3.4, we propose a multiple linear regression model to analyze the impacts of risk factors on transportation project cost overrun risk. The Vancouver Island Highway Project (VIHP) case study in Section 3.5 provides examples of applications of probability distribution analysis and regression analysis to analyze transportation project cost overrun risk. 3.2. Probability Distributions for Project Cost Overrun Risks Project evaluators may be interested in the expected cost overrun ratio for a new project as well as the probability of the project cost overrun ratio exceeding a specific value. Probability distribution analysis can provide estimates of mean and standard deviation and confidence intervals for project cost overrun risk variables. 6 Project evaluators normally estimate project cost and set project budgets using their knowledge and experience from similar past projects. However, knowledge and experience are subjective and difficult to be quantified accurately. Moreover, individual project evaluator's experience and knowledge may be biased and may not reflect the reality of the whole industry. Well-founded statistical models are needed to analyze valid historical data and provide reliable information and estimates for project costs. Both of the two probability distribution models in, this section are based on historical data. An important requirement for properly using these two models is that the database for fitting these distributions should include only those projects with similar construction, economic and political conditions. To estimate the costs for a new project, project evaluators should use probability distribution models fitted to data from projects with similar characteristics, such as: • construction and management technologies • size • economic and political conditions • private investment structures (for example, PPP) and public-private risk sharing/arrangements We treat cost overrun risk variables as random variables, which are influenced by some random risk factor and build probability distribution models for them. The project cost overrun ratio is calculated as project cost/project budget. Project cost is what project investors actually pay for the project; project budget is the estimated project cost before projects start. Projects have cost overrun problem i f project cost overrun ratio is greater than one; otherwise project costs are under control. 6 17 3.2.1 Distribution Fitting Method vs. Monte Carlo Method Distribution fitting is an analytical method which uses empirical data to find the 'best' fit probability distribution for variables. In many statistical analysis tools, distribution fitting functions are available for various distribution types, including beta, gamma, exponential, and others. The general steps of distribution fitting are: 1. Using valid data, plot the histogram of the target variable. 2. Based on this histogram and experience, determine the distribution type most clearly reflected. 3. Use distribution fitting functions to estimate distribution parameters. Some software, such as the Input Analyzer in Arena, may automatically choose the 'best' fitting distribution model, based on the sample data pattern. However, project evaluators should test the validity of this 'best' fit distribution model with their own judgment. These automatically fitted distributions may not be realistic. The Monte Carlo method is based on simulation techniques. The general steps of Monte Carlo simulation are the following: 1. Make assumptions on the probability distributions of the factor variables 2. Use computer to generate a large number of 'random draws' from the assumed probability distribution of each factor variable 3. Use these 'randomly' drawn values of the factor variables to estimate the target variable 4. Use these target variable estimates to calculate mean, standard deviation, etc. for the target variable ^ The Monte Carlo simulation method is especially useful when the target variable is influenced by many complicated and correlated factor variables or/and when data are not available. However, there are some potential problems in the implementation of Monte Carlo simulation in transportation project evaluation. First, Monte Carlo simulation results can be biased due to project evaluators' subjective views on project risk factors. Since the whole Monte Carlo simulation model is based on project evaluators' prior assumptions on project risk factors, the results of Monte Carlo simulation are possibly biased. Similarly, Monte Carlo simulation results can be intentionally made up, by adjusting the prior assumptions. Second, the 'random draws' from a computer are not really random. To date, computers have not been able to generate real random numbers. These 'random draws' may lose some information from the original data. In the VIHP case study, we provide comparisons between results from Beta Fitting model and Monte Carlo Simulation model. The simulated probability distribution of cost overrun ratio in the Monte Carlo Simulation model has a smaller range than that of the Beta Fitting model. The discrepancy between these two model results is due to the inability of the computer to provide truly random draws. This may not be a severe problem in some cases. However, for project cost overrun ratio variable, the bounds of the values, especially the maximum value of the project cost overrun ratio, are what interest project sponsors most. 18 Underestimation of the value of the maximum cost overrun would cause problems. In VIHP case study, we will show that cost overrun probability estimates in Monte Carlo simulation model are slightly lower than those in Beta Fitting model. Third, when a system becomes more complicated, Monte Carlo simulation may be difficult to be implemented. It is common that when the system gets complicated in real cases, simulation techniques may give results, which cannot be validated by real data. Often project evaluators/analysts' countermeasure to this situation is to 'adjust' model assumptions to 'make' the model validated. Due to the large time and expertise efforts required to examine a complicated simulation model, the validity of a simulation model would hardly be challenged. However, the validity and reliability of a Monte Carlo Simulation model is a big question. However, we are not arguing that Monte Carlo Simulation would not be applicable in transportation project evaluation. The Distribution Fitting method cannot be a full replacement for Monte Carlo Simulation, especially when valid data are not available. The advantage of the Distributing Fitting method is that it is based on real data and therefore its results are expected to be more objective and realistic. We recommend using Distribution Fitting method to estimate the distributions of factor variables to be used in a Monte Carlo method. It would help alleviate the bias problem in Monte Carlo Simulation Models. And we would recommend using Distribution Fitting method in project evaluation when valid data are available, due to its better fitting the reality, its relative simplicity in application, and the possibility for its results to be audited and examined. In this thesis, we focus on Beta Distribution Fitting (Beta Fitting), since our target variables, transportation project cost and project cost overrun ratio, are expected to follow Beta distributions. The models in this thesis can be extended to analyze other variables following various distributions. 3.2.2 Probability Distribution for Project Cost Overrun Ratio We use the Beta distribution to fit project cost overrun ratio data, due to its nice features, such as its finite upper and lower bounds and flexible shape. A project cost overrun ratio has finite minimum and maximum values as do Beta distributions. When project construction and operation progress as scheduled, actual project cost should be reasonably controlled; project cost overrun ratio should be reasonably close to the industry norm. However, if unexpected situations arise and cause, for example, project delays or safety or quality issues, the actual project costs may increase and the cost overrun ratio may deviate unfavorablyfromthe expected. 7 So transportation project cost overrun ratio histogram is expected to have a tail to the right, an upper bound, and a lower bound greater than 0, due to the fact that project real cost is always larger than 0. Probability distributions with infinite bounds (for example, normal distribution) are incapable to depict these features for project cost overrun ratio variable. Nor would those distributions with inflexible shapes different from the cost overrun ratio data pattern be applicable, for example, the exponential distribution. 7 See Appendix 1 for Beta distribution introduction 19 The histogram pattern of the project cost overrun ratio provides the basis for our assumption that we can use the Beta distribution to reflect the transportation project cost overrun ratio variable. With valid and adequate data, we can fit a beta distribution to transportation project cost overrun ratio. Based on the fitted distribution, we can estimate probabilities of project costs exceeding a certain level and the confidence intervals for these probability estimations. In VIHP case study, we have a numerical example using this model. 3.2.2 Conditional Probability Distribution Model for Project Cost Overrun Ratio Transportation projects have several phases: evaluation phase, design, construction, operation, etc. Moreover, during the construction phase, project managers may evaluate project cost control at several sub-stages. The conditional probability distribution can help the project evaluator use information from earlier stages to better estimate project cost overrun probability. Based on the estimated cost for the next stage, the project manager can adjust and schedule construction to control costs. Model Assumptions • The project can be divided into several periods or stages! Project costs at these periods or stages are independent of one another. Absolute independence between costs at different project stages may not be exactly true. However this does not make this assumption unrealistic. Independent costs of different project stages mean that project cost at one stage would not be quite different no matter whether project cost at other stages high or low. Take RAV line project as an example, RAV line is composed with several sections of high speed rail, some of which are underground while others are on the ground or aboveground. In contrast to construction costs of on the ground sections and above ground sections, construction costs for the underground section are likely to have different structure and sources ofrisk.We assume project evaluators are able to divide projects into some evaluation phases with reasonably independent costs. Project costs at each period or stage follow a Beta distribution. Since we made the assumption from the earlier model that project cost overran ratio can be fitted with a Beta distribution and since the cost overran ratio is calculated as actual project cost divided by project budgeted cost, which is constant, project cost can be assumed to follow Beta distribution as well. 8 • The Model Suppose we are at the end of project evaluation period n, and we know the accumulated n project cost from earlier n periods is£c]. We are interested in the probability for project 1=1 cost exceeding project budget at period n+l. Assume cost at period n+l follows a Beta distribution (P, Q, A, B) , probability for project cost to exceed budget at period n+l is: 9 Absolute independence may not exist at all in some philosophers' view. See Appendix 1 for Beta Distribution introduction. P and Q are the shape parameters; A and B are the value parameters. 9 ) .. 20 : 1 q >(G-XQlIq +1 1=1 ifJC^G i=i 1=1 l^f{x\P,QAB)dx ifYf,<G d = project cost at each period i G = project budget = *~ -~} ', ,A < x < B;P,Q > 0 B(P,Q)(B - A) ~ ^ = { V ^ ' O - x) -'dx f(x\P,Q,A,B) ( A ) P l( B X Q n p+a B{P,Q) l Q This model can also be used to calculate the probability of the project cost being between specific value bounds. As an example, the probability of a project cost being between 1.2 times and 1.5 times of budget is: n+l n Pr(1.2G<£q <1.5G|£q)=Pr (i.2G-Xq)<q<(i.5G-Xq)iXq 1=1 i=i i=i i=i i=i =J r -\2G-Yp f(x\P,QAB)dx Project evaluators can use this method to estimate probabilities of project costs exceeding various special values, for example, 2 times project budget. A Numerical Example Say a project has a budget (G) of $10 million and we know accumulated project costs up to the end of project period n are about 40% of project budget. We assume project costs in period n+l follow a Beta Distribution (4, 6, 2, 15) . Figure 4 shows the probability density function for this Beta distribution. 10 Figure 4. Beta Distribution (4, 6, 2, 15) PDF for Project Cost at period n+l 0. 25 X 0. 2 <tH w CQ 0. 15 C 0. 1 c •rH .o 0. 05 O CU 10 'roject 12 16 Cost With G = 10, P = 4, Q = 6, A = 2 and B = 15, the probability for project cost to exceed budget at the end of period n+l can be estimated as: 1 P=4, 0 = 6 , A = 2 , B = 1 5 21 AS n Hjf /=i >G\ Yfi =40%G)=Pr(q >60%G) = | f(x\4,6,2,15>fc=71.2% +1 1=1 The probability of project cost exceeding 1.5 times budget at the end of period n+l is: M+l n P n £ q > L 5 G I Yfi =40%C7)=Pr(q >1.1G) = +1 1=1 i=] AS f f(x\4,6,2,15)^=2.9% J l The probability of the project cost exceeding budget, but not exceeding 1.5 times budget at the end of period n+1 is: n+1 n , j =40%G)=Pr(0.6G<q <1.1G) = f f(x\4,6,2,15)^=68.3% HG<Yfi \Yfi <l5G +1 /=i i=i The following shows probabilities of project costs exceeding certain values at the end of period n+1, given various project cumulative cost values for the previous n periods. i Cum Projc at < nerir 3 0.1G 0.2G ~ ^ J l 0.4G 0.5G 0.8G Prist 1 of tive Table 6. Probabilities for Project Cost to Exceed Certain Values at Period n+1, with Beta Distribution (4, 6, 2, 15) and Project Budget (G) of $10 million For project cost at the end of period n+1 >0.5G >G >1.2G 96.3% 18.4% 2.9% 99.7% 33.5% 8.3% 100% 71.2% 33.5% 100% 86.8% 52.1% 100% 100% 96.3% >1.5G 0 0.7% 2.9% 8.3% 52.1% The advantage of this conditional probability model is that it makes use of cost information from previous periods or stages of the project. This method would give more reliable project cost overrun probability estimates than our first model, in which project costs are not divided and analyzed at stages. The key requirements for this conditional probability model are project stage division, validity of model assumptions and reliability of the project database. Regulated standards and rules are necessary for project managers to properly define project stages; Cost records and report systems should be set in each project; Project cost information at each stage should be collected and built into a database at transportation research institutions, for example, at risk management departments of provincial Transport Canada offices. 3.3. The PPP Models 3.3.1. Public-Private Partnership (PPP or P3) Grimsey and Lewis (2004) define PPPs as "arrangements whereby private parties participate in, or provide support for, the provision of infrastructure". A PPP project results in a contract, based on which a private entity provides public infrastructure-based services. 22 PPP means more than the statement that the private sector finances public infrastructure. Grimsey and Lewis view the essence of PPP as the public sector buying "a stream of services under specified terms and conditions". Compared with simple contracting-out, PPP requires close partnerships between the parties, as well as more complicated and specific contracts identifying each party's risk, responsibilities and rights. PPP is expected to improve project efficiency by involving the private sector entity into project management, revenue or demand management and cost control. Compared to typical public project management, the private sector can manage projects better by setting clear targets or goals at each stage of the project, clarifying the responsibilities of managers, and making managers accountable for their responsibilities. The private entity is expected to be more proficient at revenue or demand analysis and other marketing research work. Marketing information will improve project design, construction and operation to better meet user demands. The private sector is expected to bring "market disciplines into public service" (Grimsey and Lewis, 2004). PPP is expected to bring cost effectiveness. Instead of contracting out project design, building, operation and maintenance to different contractors, PPP often uses one contractor. As Grimsey and Lewis point out, "bundling" of a long-term partnership compels the project contractor to manage projects more thoughtfully and carefully starting at the design stage, which leads to project cost effectiveness. There are many types of PPPs. They can be BOT (Build Operate Transfer), BOO (Build Own Operate), joint ventures of public and private sectors, and others. Different types of PPPs may reflect different risk-sharing relationships between public and private partners. Simply grouping projects into "private" and "public" may be misleading, since what really bring about project cost effectiveness are the risk and responsibility shared by private sector, which drive the private sector to manage project in a more efficient way. There is another fact increasing the importance to investigate project risk sharing structures between partners. A reasonable private entity would try to avoid risks that are not fully compensated for. Although one PPP project may be classified as BOT, in which case project risks are considered to be primarily on private partner's side, the public sector may nevertheless take the majority of project revenue or demand risk. The RAV (Richmond-Airport-Vancouver) project may be one example. The private entity will build and operate this rapid transit line and transfer it to public sector after a certain period. However according to the contract, TransLink will collect the fares and pay the private entity a certain amount of fees every period, which are based on the pre-assumed RAV line ridership numbers. This term almost exempts the private sector from revenue shortage risk. The public sector will face an even worse situation if RAV Line ridership forecasts turn out to be over estimated. 23 3.3.2 Risk Allocation in PPP Should all projectrisksbe transferred to private sector to induce its cost saving incentive? This depends on the ability of the private sector to manage project risks. As Grimsey and Lewis argue, if the private sector is not the better manager of project risks, the risk premium it demands would outweigh the benefits to the public sector. So, they conclude, "optimum, rather than maximum, risk transfer is the objective of the PPP arrangement". Here the optimum refers to the scenario when risks are assigned to the parties who can best manage them. Optimal risk allocation of transportation projects requires minimizing both probability and impacts of risks at least cost. One typical case is when the public sector transfers the majority of design, construction, and operation risk to the private sector, including cost overrun risk, construction delay risk, and so on. The public sector retains the risk of land use, policy, environment, and so on. On the government's side, the failure of a transportation infrastructure project would be unlikely to influence the "credibility" of the government. Governments would still be regarded as with little default risk. The project risks are actually born by the taxpayers and other members of the society. The private sector normally sets a subsidiary limited company for the project and project risks are transferred to this limited company. If the project fails, this subsidiary limited company bears all the financial loss and may declare bankruptcy, while the private sector may still survive. In this way, the project risks, namely transferred to the private project sector, are actually transferred to the investors, financiers and other creditors of that subsidiary limited company. Although the private sector can transfer majority of project risks to a limited company, they do bear certain "credibility" risk. Failure of a project would adversely influence credibility of companies responsible for it. And diminished company credibility would further influence the future borrowing rates and financing costs of the private sector. The private sector will therefore have incentives to manage project risks efficiently unless it can always be a monopoly, which is unlikely in developed and open markets today. So we can assume proper involvement of private sector would improve project risk management. 3.3.3 Conditions for PPP Success Grimsey and Lewis (2004) quote six key 'drivers' for successful projects in one report from Arthur Andersen: • Risk transfer • Long-term contracting • Output-based project specification • Competition • Performance measurement 11 " Arthur Andersen (2000), Value for Money Drivers in the Private Finance Initiative, London: Arthur Andersen and Enterprise LSE. 24 • Private sector management skills In addition, they recommend centralized monitoring of project performance, communication among government departments, use of ongoing benchmarks, and careful assessment of the risk transferred to the private sector. Grimsey and Lewis list another four main aspects of successful projects from one study of the National Audit Office: clear definition of objectives, proper procurement processes, high-quality bids, and ensuring final deal make sense. 12 From these studies, we can find some key conditions for PPP project success: 1. Clear identification of risk and responsibility sharing among each partner 2. Output-based performance measurements at each project stage 3. Competitive bids 4. Long-term partnership between public and private sectors 5. Cost-effective risk allocation so that risks are allocated to the party who can best manage them These conditions can be instructive to all transportation infrastructure projects. The key issue for these conditions is to make sure allocating public and private sector the tasks they are best with and obliging both of them to be responsible for their tasks. 3.3.4 Comparisons between Public, PPP and Private Projects The conditions for PPP project success show that private involvement alone cannot ensure project success. The key for PPP project success is that the parties in projects are taking what they can best cope with and are responsible for their tasks indeed. In an ideal world, these conditions may be achieved no matter whether private sector would be a partner or not. However in real world, competitiveness and other market contributors must be implemented into this system to achieve these conditions. Incorporation of private capital is expected to bring up better project management and other market contributors to transportation infrastructure projects. Should all transportation projects be purely privatized then? In other words, should government leave transportation infrastructure projects to the market completely? The answer should be "no". First, transportation infrastructure projects often require huge amounts of investments, which can hardly be funded by single private companies. Even if private companies can raise such large amounts of money, the capital costs and the project financing risk may be high. Second, private companies make investment decisions for the sake of their own profitability, not for the society's well-being. However, major benefits of transportation infrastructure projects, such as economic improvement, alleviation of congestion, and so on, are in society's interest. For their own profits concern, the private companies might 12 National Audit Office (2001), Managing the Relationship to Secure a Successful Partnership in PFI projects, JC 375 Session 2001/2002, London: H M S O 25 either abandon some projects or collect high tolls from users, both of which would depress project social benefits and worsen transportation problems such as congestion and safety. Third, absolute privatization may cause land-use problems. As an example, in some developing countries, the profitability of toll ways has attracted the building of additional toll ways in the same area and connecting the same cities. Redundancy of highways causes not only profit loss to private companies but also land waste. The "invisible hand" of the market might bring balance finally. However transportation construction is quite different from many other industries such as pencil production. One redundant highway may cause millions of loss to the society, or even more. Project assessment and approval, auditing, monitoring and regulation should be important tasks of governments. Fourth, because of their high requirements of investments, land-use and environmental impact, transportation infrastructure projects bear high political and environment risks, both in risk possibility and impacts. Once these risks materialize, the project is likely to fail, since individual private sector can hardly manage these risks. For example, delay of government approval for land use or changes to environmental standards may cause transportation projects to be postponed, delayed, or even abandoned. Involvement of public sector will at least minimize or alleviate these risks. However, it does not mean privatization is not feasible for transportation projects ever. Private projects can be successful but should not be out of the monitor of governments. As a cooperative relationship between governments and the private sector, is PPP the best choice for transportation projects? It depends. The complicated contracts and legal issues of PPP lead to extra costs, such as negotiation costs and contracting costs. For a small transportation project, a PPP would be too costly. Simple contracting-out would usually be a better choice. For large transportation projects which require high management skills and cooperation between the sectors, PPP may be a good choice. PPP should be used only when benefits of PPP to the project outweigh its costs. 3.3.5 PPP Cost Overrun Risk Analysis Models 3.3.5.1 PPP Cost Overrun Risk and Revenue Shortage Risk Allocation Scenarios The success of PPP project asks for optimal risk allocations between public and private sectors. For example, the private sector takes design and constructionrisksand the public sector takes environmental, political and land-use risks. Even under this optimal risk allocation, different cost overrun risk and revenue shortage risk allocation between public and private sectors may cause different results to projects. Here we define the cost overrun risk taker as the one who will reinvest for increased project cost in addition to project budget; The revenue shortage risk taker is the one exposed to revenue or demand variation. For PPP projects, the private sector may bear cost overrun risk and revenue shortage risk or leave them to the public sector. There are four scenarios in general for PPP project cost 26 overrun risk and revenue shortage risk allocation, in perspective of whether private sector takes these two risks. Cost efficiency in PPP projects is generally assumed to be ensured by the private sector's cost control management. So we focus on discussing and comparing the incentives of the private sector to control project cost in these scenarios. Project benefits to the private sector can be more than its project revenue share. As we have discussed, the private sector may want to improve its reputation or credibility in the industry for future profits. Unfortunately this aspect of benefit depends on the viewpoint and evaluations of individual private company, and is too difficult to be included in the following scenario comparison, which discusses private sector in general. So we assume private sectors in these scenarios are pursuing profit maximization. Benefits other than revenues are not included in the following discussion. Table 7. Scenarios for PPP Projects Cost Overrun Risk and Revenue Shortage Risk Allocation in Private Sector's Perspective Scenario Index I II III IV Cost Overrun Risk Not Responsible Not Responsible Responsible Responsible Revenue Shortage Risk Not Responsible Responsible Not Responsible Responsible First, we compare the effects of scenarios I and II on project costs. In these two scenarios, the public sector takes the cost overrun risk and will invest the additional capital if project costs exceed budgets. • In scenario I, the public sector also takes revenue shortage risk, which means the private sector does not pay the increased costs and receives the fixed revenue. With fixed revenues, the private sector will try to lower costs so that its profit is maximized. As long as the private sector expects possibility of controlling project costs under budget to pay less for the project, it will have an incentive to lower project costs. • In scenario II, the revenue of the private sector depends on future demands arising from this project. In this case, project cost increases may improve project design and quality and, therefore, future demand and revenue. To maximize its profit, the private sector may increase project cost to improve project quality as long as the expected marginal revenue is greater than the marginal cost. There is no guarantee that the private sector would keep project costs as low as possible in this scenario. For project cost control, the risk allocation in scenario I may be better at driving the private sector to reduce project cost than in scenario II. However, it would be too cursory to conclude that scenario I would be a better choice for project risk allocation. The cost efficiency in scenario I is likely to be obtained in the prices of project quality decline and may cause the public sector, or even the whole project, to incur losses of benefits. In scenarios III and IV, the private sector is responsible for project cost overrun risk, which means when project costs exceed budget, private sector will be asked to pay at least part of the additional. 27 \ • • Compared with scenario I, private sector in scenario III would not change its cost control behavior much. As long as the private sector faces fixed revenue, it will always have an incentive to lower its costs so that its profit would be maximized. In scenario IV, the private sector will still try to find the optimal cost input, with which the marginal benefit of the private sector equals its marginal cost and its profit is maximized, as in scenario II. However, scenarios II and IV are likely to result in different optimal cost input amounts, since the revenue and cost functions of the private section under these two scenarios are not the same. According to the analysis of these four scenarios, it seems that when private sector is offered fixed benefits, its cost control behavior is irrelevant to the project cost overrun risk allocation. However, if the project budget in scenario I is 'delusively' underestimated at the project assessment and evaluation stage and private sector knows it, scenarios I and III may induce different cost management behavior in the private sector. If the private sector knows or expects the project budget to be underestimated, in scenario I, since private sector is not required to re-invest the increased costs and expects little possibility to control the costs under budget, private sector may not have as much incentive to control project cost as in scenario III. Compared with revenue conditional on real project demand, fixed revenue may be better at inducing private sector to control costs. However, this scenario analysis is from the perspective of the private sector's profit and helps to understand private sector's cost control behaviors under these scenarios. The better choice for cost control in this scenario analysis may not be the better choice from the perspective of the social benefit. Corresponding regulations are necessary to make sure project cost reductions do not damage public or social interests. For example, more detailed and clearly defined project quality standards may prevent the private sector decreasing project quality to lower costs. This PPP risk allocation scenario analysis provides a general picture of how the involvement of the private sector in project cost overrun risk and revenue shortage risk can influence project cost control. The scenarios simply classify private sector involvement in these two project risks as "Responsible" (involved) and "Not Responsible" (not involved). In the following model, we use the proportion of private investment to represent private sector involvement in transportation projects and analyze its impacts on project cost. 3.3.5.2. PPP Cost Overrun Risk Analysis Model: A Necessary Condition for CostEfficient Private Involvement in Transportation Project As we discuss in earlier sections, when properly monitored and used, private sector involvement in transportation projects can achieve better revenue and cost management. In this model, we use the proportion of private investment to represent private sector involvement under the assumption that the more the private sector invests, the greater the private sector involvement. Our objective is to find some necessary conditions for making sure private involvement will help control project costs in the scenario under which private sector benefit is not fixed. In this model, project benefits to private sector include revenue and other benefits, such as raising company reputation and future opportunities. 28 Model Assumptions 1. For a project with $/ (in current dollar value) investment budget, the private sector invests a (in %) and the public sector invests 1- a of total investment, including project budget and increased cost if there is project cost overrun. 2. To private sector, the value of expected project cost overrun ratio is Ey . 3. y is assumed to be a function of a, which means the project cost overrun ratio would be influenced by how much the private sector is involved in the project. This is based on our assumption that more investment may strengthen the power of the private sector management in projects and make private sector more responsible, which will induce the private sector to better control project costs. As we have discussed, private sector's involvement in transportation projects may improve project cost control, as long as some conditions, such as optimal project risk allocation, are satisfied. We have concluded that neither purely privatization nor no private involvement of transportation projects should not be the optimal way to control project cost. So, we assume that with an increased private investment proportion, expected project cost 13 d Ey(a) 2 overrun ratio will first decrease, and then increase, i ^dap > . 4. The private sector will make a return of R (in current $ value). As we have discussed, involvement of private sector may bring market advantages to projects and improve project demand/revenue management. So, we assume R could be expressed as a function of a. Since reasonable private companies will not invest without being paid, we assume that the larger the private investment proportion, the greater the return to w 2 h c h i s 0 dR(cc) the private sector, which means > 0. However, it is unlikely for private sector da to obtain an increasing marginal benefit for its investment in transportation projects. We assume that when private investment proportion increases, marginal private d R(a\ da 2 benefit dR(a) to investment proportion will decrease, J 1 — < 0 . When a increases, da - decreases until it reaches zero. According to assumptions 3 and 4, for increasing a, private sector benefit would increase at a decreasing rate, while private sector cost would first decrease and then increase. So, we expect project profit to the private sector would first increase and then decrease as a increases, potentially subjective to a maximum. For the private company, its target is to maximize its profits. MaxR(a)-aIxEy(a), 0<a<\,y(a)>0 a The First Order Condition for private company profit maximization function would be: — da 13 IxEy(a)-aI—t—=0, da Y = Project Cost/ Project Budget; both cost and budget are in current $ value. 29 ; '- E;VA da al Since al>0, When ^>IxE[r(a)}, >()... da (4.2) da When ^<UE[r(a)],^^<0 da L (4.3) 1 da dR(a} Intuitively, refers to the marginal benefit of the private investment proportion. da IxEy(a) is the expected project cost for a certain proportion of private investment. dEv( a i — is the expected marginal cost overrun ratio of the proportion of private da investment. According to Equation 4.2, for every percentage that the proportion of private sector investment increases, if the private company can achieve marginal benefits greater than the value of expected project costs, more involvement of private company will increase expected cost overrun ratio. One possible intuitive explanation for equation 4.2 is that when the marginal benefit of private investment proportion exceeds expected project cost, the private company may not be induced to control project costs, since increasing project costs may bring more profit to the private sector. In this case, more involvement of private sector will increase expected project cost overrun ratio. Once provided with marginal benefit greater than expected project cost, private sector may be over-paid for the risks they actually take. Private sector may have an incentive to increase its investment proportion to gain more profit, as long as its marginal benefit is larger than expected project cost. Moreover, private sector may take risks that they may not be able to manage. So, project cost overrun risk, in terms of the expected project cost overrun ratio, may increase as private sector investment proportion increases. Now we have one necessary condition for cost-efficient private sector involvement in transportation projects, which is that the marginal benefit of private investment proportion to the private sector should not exceed the expected project cost. One important point is that the marginal benefit to the private sector could be much larger than the marginal revenue. For a newly launched company, for example, an important benefit may be the building of its reputation in the industry and future opportunities. Such project benefits are hidden from the contract. Yet they are important considerations for investment and project management strategies including project cost control strategies. The public sector should investigate the marginal benefits the private sector expects from projects and pay special attention to those associated with large potential marginal benefits. Terms related to this condition should be included in PPP contracts to ensure that the involvement of private sector indeed exerts positive influence on project cost control. 30 3.4. Regression Model 3.4.1. Objective and Importance Despite their uniqueness, transportation projects are influenced by common risk factors such as social interest rate changes and project delays. We propose this regression model to test and quantify the impacts of these transportation risk factors on the project cost overrun ratio. Quantifying the sensitivities of the project cost to these risk factors helps project managers to obtain a better view of the impacts of these risk factors, to improve their ability to estimate the likelihood of cost overrun risk and to implement proper risk management strategies at early stages. Furthermore, quantification of the sensitivity of project costs to these risk factors is meaningful to governments. Governments are generally the major investors in transportation infrastructure projects. Improved understanding on how risk factors, such as interest rate changes affect project costs help governments improve their economic development and fiscal strategies. For example, provided with significant project cost sensitivity estimate to the interest rate fluctuation, the governments can adjust interest rates to favorable levels to control transportation project costs. Since transportation projects are often financed using local, provincial or federal taxes, improved estimation and management of project cost overrun risk are beneficial to the society in general. Probability distribution modeling methods, such as Distribution Fitting, cannot provide quantitative analysis of the impacts.of risk factors on project cost. A regression model, designed to analyze the impacts of factor variables on the target variable, can be used as a supplementary tool for project cost overrun risk analysis! 3.4.2 General Introduction of Risk Variables for the Model This regression model is based on risk source analysis in the literature review. The model uses historical data from transportation projects. In addition to their expected effects on project cost, the following risk variables are chosen in concerns of mathematical modeling possibility and data collection convenience. In this section, we discuss these risk variables in general. Detailed risk variable specification and analysis are discussed in the next section. The dependent variable is the project cost overrun ratio, calculated as project cost divided by project budget . 14 The independent variables are: 1. Average annual interest rate change ratio over construction period Both project cost and budget are in current $ values. 31 The interest rate change ratio is calculated by dividing next-year interest rate by current-year interest rate. This variable is used to test the impact of annual interest rate changes on project costs. Changes in the interest rate may increase project costs, especially if project financing is subjective to floating interest rates. 2. Annual interest rate variability Standard deviations of annual interest rates during the project period are used to quantify the impact of annual interest rate fluctuations on project cost overrunrisk.In contrast to the annual interest rate change ratio, whose log values show negative and positive interest rate changes, the standard deviation of interest rate during the project period is always positive and depicts interest rate volatility only. 3. Project construction duration 15 Project construction duration is the periodfromthe time the project decision is made to the time construction is finished and the project is ready to use. Project construction duration influences other project risk factors. For example, construction delay may increase the chance that input material prices will increase. 4. Percentage Investment of total project funding by private sector We assume that more participation by the private sector in projects may improve project cost control at first, and that this effect will diminish after private investment proportion reaches a certain level. Here, the assumptions are the same as they were in the PPP cost overrun risk analysis model. 5. Percentage Investment of total project funding by federal government The federal governments generally use tax revenue for project funding and are assumed to be less sensitive to individual project results than private companies and local governments. This variable is used to estimate project cost sensitivity to investments by federal governments in transportation projects. 6. 15 Transportation construction projects may have both public and private investors. The public investors may include local, provincial and/or federal governments. Here we assume a federal government would not fund the project without local government investment, which means the investment ratio of local government would be greater than zero: So, the sum of investment proportions for private sector and federal government would always be less than one. Cost overrun risk bearer Private sector is generally regarded as better at risk management. Project capital costs are expected to be better controlled when the private sector is responsible for them. Cost overrun risk bearer variable is used to examine whether this assumption is true and to quantify its impact on project cost overrun risk. We assume that it is the investors responsible for project increased cost who bear project cost overrun risk. There are three scenarios for cost overrun risk bearer: cost overrun risk is born by private sector alone, public sector alone or both public and private sector. Two dummy variables are used to analyze these three scenarios. In Flyvbjerg's model, this variable is called project implementation phase. 32 Table 8. Cost Overrun Risk Bearer Dummy Variable Values under Different Scenarios Cost Overrun Risk Bearer Dummy Variable Values Scenarios Private risk Bearer Public and Private Risk Bearer Private 1 0 Public 0 0 Public and Private 1 0 Transportation project cost overruns are often caused by project cost underestimation. One reason for cost underestimation may be project uncertainties such as delays. This kind of cost underestimation is caused by variations of project risk factors including the risk variables in our regression model. Another reason for cost underestimation is "biased" or "delusive" behavior of project evaluators whose activities are influenced by project sponsors and their interests. As discussed in Section 2.4, Literature Review, investors may regard their own investment, instead of those from all investors, as project costs. So, investors may benefit from projects even though the project net present value is negative. Employed by project sponsors to carry out project evaluation, project evaluators are likely to consider the interests of project sponsors to be their first concern and project costs may be underestimated. This kind of cost underestimation is due to human behavior and is difficult to analyze in a regression model. Objective numerical estimates from probability distribution and regression analyses can be helpful to minimizing this problem. For example, if every one year delay in project construction is expected to increase project costs by 4%, the project evaluators would have difficulty assuming that project costs will not change when projects are delayed. 3.4.3 Variable Specification and Analysis When we discuss project cost overrun risk, we mean both likelihood and magnitude of project cost overrun. However, what most interest project evaluators and investors may not be whether cost overrun will occur, but how large the overrun ratio will be. Because of their large investment requirements, transportation infrastructure projects can incur millions of dollars in extra costs with just moderate cost overrun ratios. So, our regression model considers the size of the cost overrun in transportation infrastructure industry. The projects of transportation modes (road, highway, rail, bridge, tunnel, etc.) may not have the same cost sensitivities to risk factors. For example, for road projects, one-year construction delay may not have the same effect on project cost overrun ratio as the same delay in a bridge or tunnel project. Bridge and tunnel projects generally face higher construction risks and technical requirements. The cost sensitivity to construction delay is likely to be larger for bridge or tunnel projects than for road projects. Different project size may lead to different estimated regression models due to their differences in technologies, cost structures and investment structures. For example, a 1% increase in the private investment ratio for a project with a budget of $10,000 will have different implications to the private investors from the implications of a similar increase 33 to a project with a budget of $1 billion. It affects the incentives of private investors to control project costs. So, a regression model should use datafromprojects which are similar in mode, type and size. Project cost overrun ratio variable The project cost overrun ratio should always be positive. If we treat this variable as the dependent variable in the regression model and use the estimated model for the prediction of cost overrun ratio, we may get negative predicted values of the project cost overrun ratio. To avoid this problem, we use log values of project cost overrun ratio variable as the dependent variable for our regression model. When project costs exceed project budget, the log value of the cost overrun ratio will be positive; otherwise, it will be negative. Project construction duration variable According to Flyvbjerg's model and data, the project duration variable is shown to be linearly correlated with cost escalation variable. However, the present value of project cost is calculated as PV(Q=^C (1+$ it '=0 / Tj = Number of project construction years for project i t = Project construction year indexfromconstruction starting time (year 0) to finish (year T) r = Project discount rate i = Project index assigning a number to each project in database According to this project cost present value calculation formula, project duration is likely to have a non-linear relationship with project cost. As discussed in the conditional cost overrun probability model, projects can be divided into several evaluation stages. For each stage, the project may have different cost structures and marginal cost increases per year. For example, the cost structure and marginal cost increase per year in the construction period of a road bed may not necessarily be the same as those of a road facility (for example, street plate, road painting, traffic light, etc.). So, we assume a positive, non-linear relationship between project cost overrun ratio and project duration. Interest Rate Variables The average annual interest rate change over the construction period is t=\ / = Project index assigning a numerical number to each project in database Tj = Number of project construction years for project i t = Project construction year indexfromconstruction starting time (year 0) to finish (year T) r = The interest rate for project i at year t lt 34 This formula shows not only the magnitude of interest rate changes over the construction year but also the direction (increase or decrease).- A value which is greater than one indicates an increase. As we have seen in section 5.4, Literature Review, unfavorable changes in interest rates will cause project financing costs increase. The annual interest rate variability variable can be estimated by the standard deviation of annual interest rates during project periods. Two main factors affecting interest rates are inflation and government monetary policy change. Inflation increases goods prices over the economy, including the capital goods, which are priced in interest rates; Governments may use interest rates as a tool to achieve goals of monetary or fiscal policy. For example, government may lower interest rate to encourage investments. According to the project cost present value calculation formula above, changes in interest rates will cause non-linear changes in the project cost present value, no matter whether such changes are caused by inflation or by government monetary policy. Nominal project costs at each time period are discounted by project discount rate powered with the number of time periods. A one-percent increase in the discount rate would have a different effect on the PV of project cost in period 1 from that in period 5. If the interest rate change is caused by inflation, both interest rates and project costs will change, since inflation will change the prices of project inputs. In our models, interest rate change variables are assumed to be non-linearly correlated with the project cost overrun variable. Log values of the variables are used to test these relationships. The interest rate change variables are expected to be positively correlated with the project cost overrun ratio. Private investment percentage variable Instead of using ownership dummy variables, which are used in Flyvbjerg's model, our regression model is interested in the effects of respective proportions of private sector and public sector investment on project cost overrun. As discussed, classifying project investment structure simply as private or as public may lead to misleading conclusions, since what really influence project cost control are the risk sharing structure and obligations of each party. It is reasonable to expect that a larger percentage of private investment would increase the influence of the private sector on transportation projects and so improve project efficiency. A larger percentage of investment entitles private companies to have more control over project decisions and management. With larger investments, private companies have more incentive to actively participate in projects. So greater involvement of private sectors would help control cost overrun risk. However, as we have assumed in the PPP cost overrun risk analysis model, when the private investment proportion reaches a certain level, further increase in private investment may not bring further cost efficiency. The private sector may take more risks than it can handle well, for * which it will seek large risk premiums. Moreover, improper risk allocation will increase project cost 35 overrun risk. So, we assume that when private investment reaches a particular level, further private investment would increase project cost overrun ratio. We assume a nonlinear relationship between private investment proportion and project cost overrun variable. Federal government investment percentage variable The involvement of a federal government in transportation projects is an interesting topic. The key question is how local governments and other project sponsors view the role of federal government. Although federal government funding would be considered as a part of project total cost in project cost benefit analysis, local governments or private sponsors may regard this funding as a subsidy from the federal government for local transportation improvement. To obtain this subsidy, the local government may be induced to overoptimistically estimate project benefits to get it approved and funded. Normally, investors are regarded as risk averse, which means facing a game with a half chance to double his property and a half chance to lose his property, an investor would choose not to play the game. However, in the case with federal government funding, local governments seem to be no more risk-averse. Suppose there are two projects with the same total investment amounts and same expected net present values, and project A has a higher probability of project failure than project B. The present values of investments and benefits (in Millions of dollars) by each investor for each project are: =3 J T 1 A,L IB,L A,L R > =4 / ' A,P J 1 ~^' IB,P = B,L R =3 A,F ~^ = 7 A = Project index for project A B = Project index for project B L = Investor index for local government P = Investor index for private sector F = Investor index for federal government I = Investment R = Project Benefit Which project will the local government prefer? The answer would highly probably be project A despite its higher failure probability. Does this mean local government is risk seeking, instead of risk averse? We would suggest that the expected net present values of these two projects in society's view are not what they are in the local government's view. With $3 million of federal government investment in project A, the "real" cost to local government is $3 million. Using this "real" cost as its project cost, project A has a higher expected net present value to the local government. NPV = 7 - 3 = 4 NPV =l-6 =\ AtL BL . If project A's net benefit is large enough to compensate for its higher risk, local government would prefer project A to project B. As long as the risk premium that local 16 16 Risk premium is the compensation investor would ask for taking more risk. 36 governrnent obtains for project A is less than NPV local government will choose project A. AL -NPV B L = $4-$l - $3 million, the Furthermore, the federal government often does not have a reliable and complete inspection and management system for the projects, which makes financial abuses and poor management possible. We suggest that the greater involvement of the federal government would increase cost overrun risk. Similar to the relationship between the proportion of private investment and project cost overrun, federal government participation in the project may not be linearly related to project cost overrun. It is unlikely that increasing federal government investment from 10% to 11% will increase project cost overrun at the same proportion as increasing it from 80%) to 81%. We assume a non-linear relationship between the proportion of federal government investment and project cost overrun. Project cost overrun risk bearer variable Two dummy variables are used for the project cost overrun risk bearer. One is private cost overrun risk bearer, which equals one when the private sector bears capital cost overrun risk completely and zero otherwise. Similarly, the public-private cost overrun risk bearer variable equals one when both the public and private sectors bear capital cost overrun risk and zero otherwise. See Table 8 for the three cost overrun risk bearer scenarios. The key issue here is to define the risk bearer. The cost overrun risk taker is the party who is responsible for making additional investment when project costs exceed budget. These two dummy variables are helpful for examining the assumption that project costs can be better controlled when cost overrun risk is properly allocated. The PPP contracts provide information on private and public investment ratios in the budget and increased cost when project cost exceeds budget. So, the regression model for analyzing the impacts of these risk factors would be: LogY=/3 +P LogX +/3 LogX +&LogX +&LogX + B LogX +/J X +#X 0 x x 2 2 3 4 5 5 6 6 7 17 Y = AciualCostj Cost Budget X = Average annual interest rate change over construction period X = Annual interest rate variability X = Project construction duration x 2 3 X = Investment ratio of private sector in total project funding 4 X = Investment ratio of federal government in total project funding X = Private cost overrun risk bearer X = Public Private cost overrun risk bearer 5 6 1 /? = Constant term 0 We have discussed on the non-linear relationships between the dependent variables and the numerical independent variables. To test and estimate these non-linear relationships, we use Log values of these numerical independent values in this regression model. 17 37 /?,. = Regression coefficients of each independent variable We do not have the data to test and estimate this regression model. However, data used in this regression model are not difficult for project evaluators to collect. This model can be used in practice to test and quantify the impacts of these risk factors on project cost overrun. Furthermore, it would help the project evaluator predict the cost overrun ratio for a new project. 3.4.4 Incorporation of Cost Overrun Risk into the Project Evaluation Process First of all, our question is: Can we use the conclusions from our regression model for the prediction of a new transportation project cost overrun risk? The regression model we propose tries to test and estimate effects of some common risk factors on transportation project cost overrun. This is based on our understanding that despite the uniqueness of each transportation project, transportation project costs are influenced by common risk factors. This regression model is very likely to have an Rsquare value less than 0.5, which means the regression model explains less than half of the variance of the project cost overrun ratio. However, this does not mean the models are meaningless for project cost overrun risk analysis. Low R-square values show that project cost overrun is influenced by some other unknown factors or is due to the uniqueness of projects. The seven independent variables in our regression model definitely cannot represent all risk factors for project cost overrun. We use them for modeling because we assume they have influences on project cost overrun and their data are relatively easy to gather and analyze. We are interested in estimating the effects of these common risk factors on project costs. We call them common risk factors not only because they are risk factors shared by all transportation projects but also because their data are relatively easy to collect. So, estimations of their influence have empirical meaning to a new project sharing these common risk factors. Projects of different types use different technologies and have different cost structures. Even different sized projects of the same type may use different technologies. So, we suggest using data from similar sizes and types of projects. If data are available, it would be interesting to "add some dummy variables to the regression model to test and estimate the influences of project technologies and cost structures on project cost. For example, to test and estimate the influence of technology A on project cost overrun ratio, we can set a dummy variable T , where T = 0 if the project does not use this technology 1 if the project uses this technology We can use the regression model to analyze technology A's influence on project cost overrun ratio. • j A A Another factor that may make these regression models infeasible is the macroeconorhic and policy environment. Flyvbjerg's study shows that the average transportation cost overrun ratio seems to be different among areas that have different macroeconomic and policy environments. So, data for these regression models should be specific to location. In Canada, provincial governments control most transportation projects and have project 38 cost tracking systems. We assume the economic and political environment is reasonably stable between successive transportation projects in the same province. Using data from projects of similar size and type done in the same specific province may exclude the majority of project cost overrun variation caused by technology, cost structure and economic and political factors when regression models use data from similar sized projects, of the same project type, and in the same province. The residuals of these regression models can be assumed to represent some random variation attributable to project uniqueness. It would be reasonable for us to assume that the influences of these common risk factors on project cost overrun are not very different from each other on projects with the same technological, financial, economic, political and geographic characteristics. This is the basic assumption for using the estimated regression model to predict the cost overrun ratio for a new project. Actually, this assumption is used in practice when project evaluators use their experiences from similar projects to estimate project costs and benefits for a new project. However, there is no guarantee that the predicted value of the project cost overrun ratio will be exactly equal to the real project cost overrun ratio. As discussed, the uniqueness of project may cause large variation in the project cost overrun ratio. So, we need some statistical estimation in a probability range for it. Prediction Interval Based on the regression model, we estimate the prediction interval for the dependent variable. The prediction interval gives the range of dependent variable for a particular set of independent variable values, within which there is a certain probability (95%, for example) that the dependent variable value will be, based on the fit of current data . As Wooldridge (2000) states, the prediction interval is a confidence interval for a predicted value. 18 A prediction interval for cost overrun for a new project can be computed as: Y new +1 aj2,n-p : sL.=MSE •s Ynew Mln + iX^-Xfl^iX^Xf 1=1 Y = Predicted value of dependent variable for a new project MSE = Mean Squared Error of the regression model Sy = Variance of the prediction error X = The independent variable values of a new project X = Mean values of independent variables for regression model observations new new new 18 In contrast, the 95% confidence interval is the variable range that has a 95 % chance of containing the true regression line, which provides a useful way to assess the quality of prediction. Prediction interval is generally wider than confidence interval at the same probability level. This is because prediction interval not only includes the uncertainties or noises from current data, but also those from the new data. 39 n = Number of observations for the regression model p = Number of independent variables for the regression model In project evaluation, the net present value of the project is the key measurement. k k WV=PV(B)-PV(Q=^B /(l+r)'-^C /(l+r)' t r t=0 1=0 Instead of using a single value estimate of PV(C), we use the prediction interval for project cost overrun from our model to calculate a range for PV(C) with a certain confidence level. So, the N P V formula would become: For example, from the regression model, we may get the 95% prediction interval for the project cost overrun to be 20%-100%. Let the present value of the project budget be $100 million, we'can say with 95% confidence that the expected present value of project costs will be between $120 million and $200 million. In words, the likelihood that project costs to be beyond this range is 5%. We can calculate a 95% prediction interval for project net present value using this project cost range. Similarly, we can obtain 95% prediction interval for project net benefit, say $140 million to $220 million. If we can assume project benefits and project costs are independent of each other, we can find a 95% prediction interval for this project N P V , which is -$60 million to $100 million. The question is whether we can make the assumption that benefit and cost are independent. Cost and benefit estimation in project evaluation are based mainly on the project design, including quality and safety issues. As long as the project design is not changed and there are no large fluctuations in macroeconomic environment, changes in project costs would have little impact on project benefits. So, it can be assumed that benefit and cost estimates in project N P V estimation are independent of each other as long as the social economy is reasonably stable and project design is not changed. Similar estimations can be applied to other project evaluation parameter such as cost-benefit ratio. 3.5. Vancouver Island Highway Project Case Study 3.5.1. The Database The Vancouver Island Highway Project (VIHP) involved upgrading the existing highways between Swartz Bay and Nanaimo, building a new parkway in Nanaimo and constructing a new inland highway from Parksville to Campbell River. The project included improvements to approximately 174 kilometers of highway and the construction of 146 kilometers of new highway. The VIHP was announced in 1993 and was substantially complete by 2003. This database is provided by Transport Canada office in British Columbia. 40 In this database, the VIHP is broken down into 163 independent projects, excluding the projects not in initial VIHP plan and two other projects with extremely high cost overrun ratios . The projects consist of road and highway construction projects in different areas and individual bridge and tunnel projects. We assume that the project costs are independent of one another. 19 20 There are 127 road and highway construction projects and 36 bridge and tunnel projects in VIHP database, including their budget and cost information. Unfortunately construction beginning and ending times for each project are not available. Nor we have the investment and private involvement information. Due to inefficient data, our proposed multiple regression model can not be applied to the VIHP data. The cost overrun ratio probability distribution model and the simple regression model will be used to analyze cost overrun risk for the VIHP projects. The costs and budgets are in 1993-dollar values. Cost overrun ratios are calculated as dividing project cost by project budget. There are 104 road/highway projects and 29 bridge/tunnel projects, each costing over $1 million. Table 9 gives aggregate cost data ranges for road/highway projects and for bridge/tunnel projects separately. Table 9. Cost and Cost Overrun Ratio Ranges oi~VTHP data Number of Number of Cost Projects Projects with with Cost Cost Overrun Min. Max. Overrun Ratio > 1.5 Road & 104 3 $7,559 $88,076,460 Highway Projects Bridge & Tunnel 29 2 $7,592 $18,349,840 Projects Cost Ratio Min. Overrun 0.48 2.34 0.80 1.90 Max. 3.5.2 Cost O v e r r u n R a t i o Distribution Fitting We discuss the probability distribution of the cost overrun ratio variable in the probability distribution model section. According to our analysis, the Beta Distribution fits our cost overrun ratio data. Figure 5 shows histograms of the cost overrun ratio for road/highway construction projects and for bridge/tunnel projects using the VIHP data. These projects had zero budgets. One is a road construction project, whose cost was about 14 times its budget; the other is a highway construction project, whose cost was about 42 times its budget. 19 2 0 41 Figure 5. Cost Overrun Ratio Histograms for the VIHP data 0.0 Cost Overrun Ratio Histogram C o s t O v e r r u n Ratio H i s t o g r a m for R o a d & H i g h w a y Projects for B r i d g e & T u n n e l Projects 0.8 1.7 2.5 0.6 Cost_Overrun_Ratio_for_Road_and_Highway_Projects 1.1 1.5 2.0 Cost_Overrun_Ratio_for_Bridge_Tunnel_Projects According to the histograms, cost overrun ratio data are centered just higher than one, where project costs equal project budget, with a few observations reflecting high cost overrun ratios and skewing the histogram to right. These histograms confirm our expectation that most projects control their real costs reasonably well; however, in some cases, real costs can be much higher than budgets. 21 As shown in Table 10, the respective descriptive statistics for road/highway projects and for bridge/tunnel projects are close to one another. Table 10. Cost Overrun Ratio Descriptive Statistic Report Road/Highway Projects Bridge/Tunnel Projects Mean 1.059 1.052 Median 1.055 1.055 Mode 1.045 1.059 Standard Deviation 0.27 0.23 Bridge and tunnel projects are generally expected to face more complicated technical and construction conditions, which make their costs more difficult to control. However, in the VIHP, the average cost overrun ratio and standard deviation of bridge and tunnel projects are even lower than those of road and highway projects. One possible reason may be that the bridge and tunnel projects in VIHP are relatively small, with costs mostly below $8 million. These bridge and tunnel projects are mostly supplementary projects connecting road and highway in VIHP. In our distribution fitting model, we do not separate road and highway projects from bridge and tunnel projects because of their close descriptive statistics and insufficient data for bridge and tunnel projects. However, if data were sufficient, we would recommend setting different probability models for each transportation project type separately. We set zero as the Beta distribution lower bound and 2.35 as the upper 21 See Table 10 for number of projects with cost overrun ratio larger than 1.5. 42 22 bound to fit the VIHP data range, whose maximum cost overrun ratio data point is just lower than 2.35. Distribution shape parameters, P and Q, are calculated using maximum likelihood estimation (MLE) method. MLE is an analytical maximization procedure. Likelihood refers to the probability of obtaining sample data, given a particular distribution model. The values of the model parameters that maximize the likelihood of certain sample data are the Maximum Likelihood Estimates. Table 11. Beta Distribution Parameter Setting and Estimates Minimum (A) 0 Maximum (B) 2.35 P MLE 9.02 Q MLE 11.05 Note: A and B values are set manually, based on cost overrun ratio data range of VIHP; P and Q values are estimated using MLE method in NCSS. 23 Figure 6. Beta Distribution PDF Plot for Cost Overrun Ratio in the VIHP 1.5 1 0.5 2 P r o j e c t Cost Overrun R a t i o Figure 7. Beta Probability Plot for Cost Overrun Ratio Variable 0.3 0.5 0.6 Beta Ouantile For statistical analysis in this case study, we use NCSS, which requires setting Beta distribution value parameters manually. The upper bound should be not much larger than the maximum value in the sample. Otherwise it may result in a beta distribution not fitting the sample data well. The Beta distribution value parameters can also be calculated using M L E . See Appendix 2 23 43 According to the beta probability plot in Figure 7, most data fit this beta distribution well, while some observations deviated from model estimates. The model would be good at estimating probability for project cost overrun ratio to exceed a certain value, which is less than 1.25. However when estimating the probability for project cost overrun ratio exceeding some value over 1.25, the model is very likely to give a conservative estimate. It means model probability estimate for a project to have a cost overrun ratio exceeding, for example, 1.5, may be an underestimation and should be used as a conservative estimate for project cost overrun risk. This conservative project cost overrun ratio estimate is caused by the statistical methods used for fitting distribution, instead of data pattern. In Beta distribution fitting, project data are divided into quartiles according to their values. The statistical Beta fitting model focuses on maximizing the likelihood of obtaining data from Beta quartile 25% to 75%. For VIHP database, 75% Beta quartile is at the point with cost overrun ratio equal tol .25. The cost overrun probability estimations over this point are likely to be underestimated. Table 12. Probabilities for Project Cost over Certain Values For Cost Overrun Ratio Probability >0.2 99.99% > 0.4 99.79% >0.6 96.71% >0.8 83.68% >1.0 57.82% > 1.2 28.84% 9.34% 1.68% 0.12% > 1.4. > 1.6 > 1.8 According to this fitted beta distribution, the probability of road and highway construction project costs exceeding their budgets is 57.82%; the probability is 0.12% that a road or highway project cost exceeding 1.8 times of budget. However according to the Beta probability plot in Figure 7, this estimate may be underestimated. This model would be better to be used as providing conservative estimates of cost overrun for road, highway, and small bridge or tunnel construction projects. 3.5.3 Monte Carlo Simulation Model Results vs. Beta Fitting Model Results Using Monte Carlo simulation , we can get an estimated beta distribution (6.99, 8.91, 0.15, 2.22) for the project cost overrun ratio variable in the VIHP. Table 13 presents comparisons on the beta distribution parameter estimates by both a simulation tool (Arena) and a statistical tool (NCSS). Figure 8 provides graphical comparison on PDF patterns between Monte Carlo simulation and Beta Fitting method. 24 Using R, 5000 data points are generated for the estimated Beta distribution (9.02, 11.05, 0, 2.35) fitted with VIHP data. Then we use this generated new database to fit a beta distribution in Arena. Please read Appendix 4 for analysis summary report. 44 Table 13. Beta Distribution Parameter Estimates Comparison for Cost Overrun Ratio Variable between Monte Carlo Simulation Method and Beta Fitting Method Beta Fitting Monte Carlo Simulation Minimum (A) 0.04 0 M a x i m u m (B) 2.00 2.35 Shape Parameterl (P) 9.02 7.18 Shape Parameter2 (Q) 6.73 11.05 Figure 8. Probability Density Function Comparison between Monte Carlo Simulation Method and Beta Fitting Method 1.6 1.4 ! £ in 1.2 1 j ? 0.8 n n 0.6 I 0.4 02 0 ) I 0.5 1 1.5 2 2.5 Cost Overrun Ratio Beta Fitting Monte Carlo Simulation In Figure 8, the distribution P D F from the Monte Carlo simulation model has a smaller range (higher minimum value A and lower maximum value B ) , a higher mode, as compared with the Beta Fitted distribution P D F . A s discussed i n the Probability Distribution M o d e l section, this may be due to that the computer cannot generate truly random values. Despite the slight differences i n the shapes o f their PDFs, the probability distribution simulated by Monte Carlo method gives similar estimates on project cost overrun probabilities. Table 14 gives the numerical comparison. Table 14. Cost Overrun Probability Estimation Comparison between Monte Carlo Method and Beta Fitting Method Beta Fitting M o d e l Monte Carlo Simulation For Project Cost Overrun Ratio > 99.0% 98.7% 0.5 87.6% 88.2% 0.75 57.9% 57.9% 1 22.8% 22.8% 1.25 3.7% 4.3% 1.5 0.1% 0.3% 1.75 0.0% 0.0% 2 0.0% 0.0% 2.25 45 According to Table 14 and as compared to the Beta Fitting model, the Monte Carlo simulation model gives equal or slightly lower estimates of probabilities of project cost overrun. We have discussed that the Beta Fitting Model provides understated cost overrun probability estimates for cost overrun ratios greater than 1.25 so that its estimation should be regarded as a lower bound. Unfortunately, Monte Carlo Simulation seems not to be a tool for providing upper-adjusted estimation on project cost overrun probabilities. On the contrary, it seems to further underestimate project Cost overrun probabilities as shown in Table 14. Project cost overrun probability estimation using Monte Carlo simulation is likely to give underestimated estimates. The Beta Fitting Method and the Monte Carlo Method provide project cost overrun probability estimates. However, they can not explain why overrun may occur and how those cost overrun causes would influence project costs. They are not feasible tools if project managers want to examine the exact factors for project cost overrun in order to control project costs. For this purpose, we use regression models to test and analyze the impacts of risk factors on project cost. Unfortunately, we have only cost and budget data for each project in the VIHP database. In the following sections, we use project budget to represent project size and analyze its impact on project cost. 3.5.4 Project Cost Overrun vs. Project Size in the V I H P Large transportation projects attract attention for their huge funding requirements. If the costs of large projects are more likely to be out of control, project sponsors should put extra efforts on controlling the associated risks. Project size can be represented by project construction time, budget and cost. Unfortunately, the VIHP database provides only project budget and cost information. In Flyvbjerg's study, project budget is used as the variable to represent project size. Project budget alone may not be a good indicator of project size, when underestimation of cost is purposely made at the project evaluation stage. Even without 'delusive' human behavior, project evaluators may overestimate or underestimate real project size without full information about construction conditions, such as the conditions of rock layers in tunnel projects. On the other hand, project cost alone may also provide misleading information about project size, if legal, environmental or political risks materialize. For example, new environmental regulations requiring design or construction changes may cause project cost overrun. In this regression analysis, we assume there is no 'delusive' human behavior and project evaluators are able to reasonably estimate project cost. Based on these assumptions, we use project budget to represent project size. Road/Highway (R/H) Projects First of all, we look at the scatter plot showing how project costs change as project budget increases. In Figure 9, project cost equals to project budget on the straight blue line; the green dots are the data points from the VIHP database. According to this figure, costs for larger projects deviate more from the Line. Although not obvious, the data show a nonlinear relationship between project cost and budget. As discussed earlier, we use log values of project cost and budget and test for a linear relationship between them. 46 Figure 9. Scatter Plot of Cost vs. Budget for Road/Highway Projects Cost vs Budget f o r R/H P r o j e c t s Cost 100,000, 000 90,000,000 80, 000, 000 70, 000, 000 60, 000, 000 50, 000, 000 40,000, 000 30, 000, 000 20, 000, 000 10, 000, 000 Budget 0 20,000,0 40,000,0 60,000,0 80,000,0 100,000, 00 00 00 00 000 Using 1.05 as the log base , we test the linear relationship between log values of project cost and budget. The model is: LogY^ft+fiLogX+e Y = Project cost X = Project budget /? = Constant term /?, = Regression coefficient s = Error term 0 Figure 10. Scatter Plot of Log (Cost) vs. Log (Budget) for Road/Highway Projects Log_R_H_Project_Cost vs Log_R_H_Project_Budget 350.0-, 1 50.0-I 150.0 , , , , , 200.0 1 , , 1 , 250.0 , 1 , , , , 300.0 Log_R_H_Project_Budget , , 1 , 350.0 Y Axis: Log values of Cost for Road/Highway Projects X Axis: Log values of Budget for Road/Highway Projects Figure 10 shows non-constant cost variance for different project size. For log (Budget) values below 250, the VIHP data fit the model well. For log(Budget) values between 250 and 290, VIHP data show high variances; For log(Budget) values above 290, VIHP data Log bases from 1.01 to 1.5 do not cause much difference in data pattern for project cost and budget. Log bases above 1.5 generate too few data points for the regression model. So we choose 1.05 as the log base. 2 5 47 show reasonably constant variances. Based on this data pattern, we divide VIHP road and highway into three groups: small (project budget lower than $250,000), medium (project budget between $250,000 and $1 million), large (project budget over $1 million). Unfortunately, we do not have engineering and technical backgrounds for these projects and cannot explain whether this division is due to project technology or other differences related to engineering. Small Road/Highway Projects Figure 11. Project Cost vs. Budget Scatter Plot for Small R/H Projects Y Axis: Log values of Cost for Small Road/Highway Projects X Axis: Log values of Budget for Small Road/Highway Projects According to linear regression assumption tests , the model assumptions, such as linearity, homoskedasticity and so on, are satisfied. The ANOVA table shows that the model is statistically significant (P value = 0.0000). We estimate the model for small road and highway projects to be: Log(Cost) = -0.9$2 + 1.006Log(Budget) Therefore approximately, Cost = 0.953Budget , as shown in Figurel2. h006 See regression analysis report in Appendix 3 48 F i g u r e 12. E s t i m a t e d C o s t vs. B u d g e t M o d e l f o r S m a l l R o a d / H i g h w a y P r o j e c t s C o s t v s . B u d g e t f o r S m a l l R/H P r o j e c t s i n thousand $ C o s t 300 I — ; T h e cost increase associated w i t h each $ 1 0 , 0 0 0 budget increase increases as budget increases. T h i s i s s h o w n i n F i g u r e 13. T h e costs o f larger project costs are m o r e l i k e l y to g o out o f c o n t r o l than the costs o f s m a l l e r r o a d and h i g h w a y projects. F i g u r e 13. C o s t Increase f o r E a c h $10,000 Project B u d g e t Increase f o r S m a l l R o a d / H i g h w a y Projects Cost Increase for Each $10,000 Budget Increase Budget I n c r e a s e i n thousand $ — ill - .# Ill s\> ^ ^s? ^S?^ , - V ^V" <S> <§> .£> .«? . # A <§> 0 '<V '<V '<V 'V <$> -?> <$> 49 ^q, Cost I n c r e a s e ^ " V i n thousand $ Medium Sized Road/Highway Project igure 14. Log (Cost) vs. Log (Budget) Scatter Plot for Medium Sized R/H Projects 255.0 262.5 270.0 277.5 285.0 Log_Medium_R_H_Project_Budget Y Axis: Log values of Cost for Medium Sized Road/Highway Projects X Axis: Log values of Budget for Medium Sized Road/Highway Projects The linear regression assumption tests of this model show that the assumptions are satisfied. The ANOVA table shows that the model is statistical significant (P value= 0.04). We estimate the model for medium sized road and highway projects to be: Log(Cost) = 105.068 + 0.62Log (Budget) And therefore Cost = 168.39Budget Figure 15 shows how project costs increase with project budget for medium-sized road and highway projects. 062 Figure 15. Cost vs. Budget for Medium Sized Road/Highway Projects C o s t v s . B u d g e t f o r Medium R/H C Projects i n thousand $ o s t 1000 900 800 700 600 500 400 300 200 100 <v* ^ # * fo ^ ^ % # % # # c # As shown in Figure 16, the cost increase associated with each $50,000 budget increase decreases as budget increases. The larger projects are subject to less cost overrun risk than are smaller road and highway projects. 50 Figure 16. Cost Increase for Each $50,000 Project Budget Increase for Medium Sized Road/Highway Projects Cost Increase for Each $50,000 Budget Increase Budget Increase i n thousand $ 50 45 40 35 30 25 2015 10 <,\>, <o v # /o <o nfe A A # ^^^ < .,% ,c (g, * Cost Q I ncrease thousand °5 <o Large Road/Highway Project igure 17. Project Cost vs. Budget Scatter Plot for Large Road/Highway Projects 280.0 280.0 300.0 320.0 340.0 360.0 Log_Large_R_H_Project_Budget Y Axis: Log values of Cost for Large Road/Hig:hway Projects X Axis: Log values of Budget for Large Road/Highway Projects Model assumption reports show constant variance assumption and linearity assumption are satisfied; however the residuals do not follow a normal distribution. The residual histogram shows a lightly skewed pattern. The violation of the normality assumption may influence the accuracy of the F-ratio of the model. However, the normality assumption "plays no role in the unbiasedness of O L S " (Wooldridge, 2000). Therefore it is not necessary to discard the tstatistics or the F ratio. In the ANOVA table, we find extremely low P-values. Therefore we would assume the violation of the normality assumption does not invalidate this model. The estimated model is: 27 Ordinary Least Square method used in linear regression analysis 51 Log(Cost) = 1.657 + 0.995Log(Budget) And therefore Cost = 1.0S4Budget 0995 . , as shown in Figure 18 Figure 18. Cost vs. Budget for Large R/H Projects Cost vs. Budget for Large R/H Projects in million $ Cost 12 10 8 6 4 2 0 \ % V <V ^ V " 3 < 3 <V V Budget « V 3 V <b- V %• <V X sjs- X Figure 19 shows that the increase in cost associated with each $500,000 increase in project budget decreases as budget increases. Similar to medium-sized road and highway projects, large road and highway projects face decreasing cost overrun risk as project size increases. Figure 19. Cost Increase for Each $500,000 Project Budget Increase for Large Road/Highway Projects Cost Budget I n c r e a s e 0. 51 r. u. 0 Increase in b i l l i o n f o r Each $500,000 Budget Increase $ 1 0. 50 0. 50 0. 49 0. 49 0. 48 Cost Increase J? ,,Nln b i l l i o n O) 1 $ o o- These regression models show that project costs are non-linearly correlated with project budget. According to this analysis, when project budget increases, marginal project cost increases for small projects, and decreases for medium and large projects. For small projects with budgets less than $250,000, project evaluators are more likely to lose control of costs the larger the project budget. However, for projects with budgets over $250,000, project costs are better controlled. One possible explanation for this project cost control pattern is that project evaluators may tend to be more careful when 52 estimating the costs of large projects. Another possible reason may be that when project costs are out of control on large projects, project managers may separate some components of the projects and title them as new projects. In the VIHP database, we see that some projects with zero initial budgets were added to the project list during the VIHP construction periods. Although we exclude these projects from our regression analysis, they may deliver some hidden information about the costs of large projects. Bridge and Tunnel Project Regression Models As we did in Road/Highway regression analysis, we look at the cost budget scatter plot for bridge and tunnel projects first. Although not obviously, the data reflect a non-linear relationship between project cost and budget for bridge and tunnel projects, as we found for road and highway projects. We take log values of project cost and budget and test for a linear relationship between these two new variables. Figure 20. Cost vs. Budget for Bridge and Tunnel Projects Cost vs Budget f o r B/T Projects Cost 20, 000, 000 18, 000, 000 16, 000, 000 14, 000, 000 12, 000, 000 10, 000, 000 8, 000, 000 ' 6, 000,000 4, 000, 000 2, 000, 000 Budget 0 5,000,000 10,000,000 15,000,000 20,000,000 igure 21. Log (Cost) vs. Log (Budget) for Bridge and Tunnel Projects Y Axis: Log values of Cost for Bridge/Tunnel Projects X Axis: Log values of Budget for Bridge/Tunnel Projects This scatter plot in Figure 21 shows that four data points with log (Budget) values below 280 deviate more from the predicted than the other observations do. When we remove these four 53 small bridge and tunnel projects (with budget below $850,000) out and run a linear regression, we get a new scatter plot for log values of project cost over budget, as follows. Figure 22. Log (Cost) vs. Log (Budget) for Large Bridge and Tunnel Projects 300.0 Log 320.0 ' 340.0 large_B_T_Project_Budget The model assumption reports show that the constant variance and linearity assumptions are valid; but again the residuals seem not to have a normal distribution. The histogram of the residuals shows a tail to right. As was true for large road and highway projects, the P-value for this model is negligible. We assume that the violation of the normality assumption does not invalidate this model. The model is estimated to be: Log(Cost) = -0.983 +1.003Log(Budget) And therefore Cost = 0.953Budget , as shown in Figure 23. 1003 Figure 23. Cost vs. Budget for Large Bridge and Tunnel Projects (Budgets over $850,000) Cost vs. Budget f o r B / T P r o j e c t s in Cost million.$ 12. 000 10. 000 8. 000 6. 000 4. 000 2. 000 0. 000 V <£> <t? <&> V <& V <6- V <£> <b' • <& B u d g e t °>- ^s- Figure 24 shows that the marginal project cost increases when budget increases for bridge and tunnel project with budget greater than $850,000. Project costs are more difficult to control or more easily underestimated for large bridge and tunnel projects. 54 Figure 24. Cost Increase for Each $500,000 Project Budget Increase for Bridge/Tunnel Projects with Budget over $850,000 Cost Increase f o r Each $500,000 Budget 0. 502 Budget Increase. Increase in m i l l i o n $ 0. 501 0. 500 0:499 0. 498 0. 497 0. 496 Cost ,,-v , v A ' A ' A ' A ' A ' A Ci' v v <v -V <V <V v Increase A' A' A' A' A' A' A' A' A' A' <o' <o' fc' fc' ^.' ^ ' <b' <b" o>' Using the VIHP database, our regression models test the linear relationships between the log values of project cost and project budget for each project type. The models give different conclusions from Flyvbjerg's, in which project budget is shown to have a linear relationship with the project cost overrun ratio for bridge and tunnel projects. Since project cost overrun can be caused by various risk factors, as we discuss in the risk source analysis in the Literature Review and in the multiple regression model section, our models that are based on the VIHP database may not be applicable for all projects worldwide. We expect that they could be applied to projects with similar political, environmental, technology and financial characteristics. However, the analytical methods and techniques used in modeling the probability distributions and regression models can be used to analyze transportation project cost overrun risk for any project type in any part of the world. Furthermore, these models can be applied to other transportation project risk analyses such as revenue shortage risk analysis. 3.5.5 A Numerical Example of Predicting Project Cost for a New Project Using a Regression Model and a Beta Fitting Model Using the estimated regression models, we can predict expected cost for a new project. However, project evaluators may find a prediction interval to be more useful for project analysis than the single estimated value of expected cost. What follows is an example of a prediction interval in project cost benefit analysis. According to the estimated model for medium sized road and highway projects, a project with $1,200,000 budget (logi.osBudget = 286.8986) has an expected Log(cost) value that is estimated to be 287.1212 (log 5Cost = 1.657+0.995Logi. osBudget). The mean squared error (MSE) for this model is 3.57, the average log project budget value is 319.86, number of observations (n) is 100 and the number of the independent variable (p) is 2. ^(X.-X) = 43643.51. So 10 2 55 s 2 Ynew = MSE [l +1 / n + (X new -Xfl^ (X,. - X) ] 2 = 3.57 [l +1 /100 + (286.8986 - 319.86) / 43643.54] 2 = 3.6946 The 95% prediction interval for the log project cost value is (282.7457, 291.4967) , corresponding to the project cost interval ($979,905, $1,501,793). With 95% confidence, we can conclude that the project costs for a road or highway project with a budget of $1,200,000 would be between $979,905 and $1,501,793. 28 Table 15. Expected Cost and Confidence Interval Estimation for a Project with Budget of $1,200,000 Project Budget Model Estimated 95% Prediction Interva Project Cost Lower Upper $1,200,000 $1,213,098 $979,905 $1,501,793 We also present Table 16, project cost prediction intervals for various confidence levels. Figure 25 provides a graphic view of how the prediction interval changes for different confidence levels for this project. When the confidence level decreases, the prediction interval shrinks. Table 16. Project Cost Prediction Intervals at Different Confidence Levels for a Project with Budget of $1,200,000 Confidence Level Lower Prediction Bound Upper Prediction Bound 95% $979,905 $1,501,793 80% $1,038,162 $1,417,519 60% $1,074,847 $1,369,138 50% $1,088,342 $1,352,161 30% $1,110,835 $1,324,782 10% $1,129,812 $1,302,530 Figure 25. Prediction Intervals at Each Prediction Confidence Level for a Project with Budget of $1,200,000 [Project Cost Confidence ^ cn cr: ^ LO a: ^ o cp ^ * o LO ae oo ?r LO r- o t- if? LO ID . o ua :~? o LO LQ__-LO. * ?r. io c st. _xC LO res. £ o ca # LO M j!? o M LO ^ o -——-Lower Bound — — « U p p e r Bound ««™««««*Model E s t i m a t e d te* ± W , • me» = 287.1212 ± 2.276x V3.6946 s 56 Level ? # m o Cost We have shown how to use regression models to test and estimate the impact of a risk factor on project costs; how to use the estimated regression model to predict the costs of a new project; and how to determine the prediction interval for project cost at a certain confidence level. In the following section, we compare the estimates from this regression model with those from the Beta Fitting model. Regression Analysis vs. Beta Fitting Method To make a fair comparison between the project cost overrun risk estimates from these two methods, we must ensure that the databases used are identical. The regression model in this numerical example uses the data for large road and highway projects whose budgets are greater than $1 million. We use these data to fit a Beta distribution model also. Using the techniques shown in Section 3.5.2, we obtain a Beta distribution (67.44, 88.28, 0, 2.35). Table 17 gives the estimated.mean, mode and standard deviation for the cost overrun ratio variable of this Beta distribution model. Table 17. Mean, Mode and Standard deviation Estimates Mean Mode Standard Deviation Beta Distribution (67.44, 88.28, 0, 2.35) 1.018 1.016 0.093 In this Beta fitted model, the project cost for a project with budget of $1.2 million is expected to be $1,221,600 , $8,502 higher than the regression model estimate. The difference ratio is about 0.71 % . 29 30 In regression models, we use prediction intervals to estimate with certain confidence levels the range within which the real project cost will fall. In probability models, we use confidence intervals to estimate with certain confidence levels the range within which the real project cost will fall. It would be interesting to examine the difference between these two. Figure 26 gives a graphical comparison between the estimated prediction interval from the regression model and the estimated confidence interval from the Beta fitted model for cost estimation of this new project. Expected Cost = Budget*Mean Project Cost Overrun Ratio = $1.2* 1.018=$ 1.2216 million Difference ratio = Difference in Cost Estimates/ Budget = $8,502/ $1,200,000 = 0.71% 57 Figure 26. Comparisons on Project Cost Prediction and Confidence Intervals from the Regression and Beta Fitting Models respectively Project C o s t 900000 800000 FTediction hterval Low er Bound (Regression Model) FTediction Interval Upper Bound (Regression Model) Confidence Interval Low er Bound (Beta Fitting Model) Confidence Interval Upper Bound (Beta Fitting Model) According to Figure 26, the prediction and confidence intervals for project cost using Regression and Beta Fitting models, respectively, give estimate ranges close to one another, especially at lower confidence levels. At confidence levels above 75%, the confidence intervals from the Beta Fitting model are lower than the prediction intervals from the regression model. While at confidence levels below 40%, the confidence intervals from the Beta Fitting model are higher than those from the regression model. These results show that, everything else the same, the regression model and the beta fitting model yield similar cost interval estimates for road and highway projects with budgets over $ 1 million. The close estimates for these two intervals in this case might be due to that the cost overrun ratios of VIHP projects are centered at about 1.02. Therefore both the regression model and the beta fitting model have close estimated values for project cost overrun ratio. 3.5.6. Multiple Regression Model Analysis In the previous sections, we analyze the relationship between project budget and cost overrun ratio for each project type. It would be interesting to see the effect of using more information from the data in the regression model. First, we can add a project type variable as an independent variable in addition to project budget. In the previous sections, projects are grouped into small road/highway, mediumsized road/highway, large road/highway, and bridge/tunnel projects. In the VIHP, the cost overrun ratios for small road/highway projects have larger variations than those for other project types. So, we categorize the VIHP projects into two groups : small road/highway projects and other projects. We add one dummy variable to this multiple regression model as an independent variable: small road/ highway projects. Table 18 gives the dummy variable values for the previous four project types: 31 31 We tested other dummy variables in different grouping. However none of those is statistically significant. 58 Table 18. Project Type Variable Values for Each Project Type Project Types Small R/H Proj ect ^Variable Small R/H Projects 1. Other Projects •0 * 'Other Projects' include medium and large road and highway projects and all bridge and tunnel projects. Second, we add another independent variable, the cost overrun ratio frequency. For each observation, we calculate its cost overrun ratio and then the frequency of this cost overrun ratio value happens among cost overrun ratios of all the observations. For example, there are 163 projects in VIHP database, 10 of which have cost overrun ratios of 1.03. So the cost overrun ratio frequency for each of these 10 projects is 10/163. So, this multiple regression model can be written as: Y = f3 +j3 X +/3 X 0 l l 2 2 + i3 X 3 3 Y = Log (Cost) Xi= Log (Budget) X2= Cost Overrun Ratio Frequency X 3 = 1 for small road and highway projects = 0 for other projects According to the statistical tests , these three independent variables are all significantly correlated to the independent variable, project cost. The estimated model is: 32 Log(Cost) = 9.66 + 0.97Log(Budget) - 4.1 IFrequency - 2.15SmallR IH The estimated regression coefficient for Log (Budget) is 0.97, which indicates that project costs increase with larger budget, everything else the same. However, marginal project cost decreases as budget increases (/?,=0.97 < 1), which means project costs are better controlled in larger projects, everything else the same. The estimated regression coefficient for the Small Road/Highway project variable is 2.75, indicating that small road and highway projects have lower costs than other projects. This conclusion also implies that small bridge and tunnel projects may have larger project cost overrun than small road and highway projects, since small bridge and tunnel projects are included in the "other projects" group. The negative estimate for the regression coefficient for cost overrun ratio frequency indicates that for projects of a certain type and with a certain budget, an increase in project cost overrun ratio frequency is associated with a decrease in project costs. The increase in project cost overrun ratio frequency means that the project cost overrun ratio approaches its mode in the cost overrun ratio histogram. The negative sign of this regression coefficient indicates that projects with lower costs are more likely to have cost overrun ratios closer to the mode. One possible reason for this is most of VIHP projects are in small sizes. See Appendix 5 for statistical analysis reports 59 This model cannot be used for project cost prediction, since the project cost overrun ratio frequency data are not available at the project evaluation stage. However, it provides some interesting conclusions on the relationships among the cost, budget, type and cost overrun ratio frequency of a project. For example, all else the same, project costs are better controlled in larger projects. 4. How the Project Managers Can Use these Models For a new project, project managers can use the estimated probability or regression models that are based on historical project data with similar conditions (such as similar political, economical, environmental, cost structure and investment structure) to estimate project costs and to analyze the impacts of risk factors. In some cases, the new project might use new technologies or have some new features (for example, new construction designs), which makes it incapable to use historical data fitted models directly. However, uniqueness of the new projects is unlikely to be absolutely unique. These new projects should incorporate some technologies or features used in previous projects. In these cases, we can split the new project into two parts: stages that can be analyzed using historical data and stages that are new. For example, RAV line is a new project launched in BC and we may not be able to find enough historical data of similar projects in similar sizes. One alternative can be splitting RAV line construction into stages as: underground rail construction, on-the-ground rail construction, and station construction (managers may set different project stages based on practical situations and needs). When splitting other urban rail projects, such as the Sky Train projects, into similar stages, project managers can use probability models or regression models to estimate costs and to analyze cost overrun risks for RAV line at each stage. Suppose the probability distribution fitting model or the regression model shows the cost overrun ratio means and confidence intervals as listed in Table 19. Table 19. Cost Overrun Ratio Means and 95% Confidence Intervals at Each Project Stage Project Stages Cost Overrun Ratio Mean Cost Overrun Ratio CI (95%) Over ground stage 1.05 0.9-1.5 Under ground stage 1.5 1.0-2.5 Station Building stage 1.07 1.0-1.5 For a new urban rail project with $10 million budget for over ground stage, $ 50'million budget for under ground stage and $10 million budget for station building stage. The expected costs would be $96.2 million ($10*1.05 + $50*1.5 + $10*1.07) in total for this new project. And project cost confidence interval at 95% level is between $69 million ($10*0.9 + $50*1 + $10*1) and $155 million ($10*1.5 + $50*2.5 + $10*1.5). Another possible solution is: specifying the exact features that make the new project unique, using historical data to analyze the impacts of these features on project costs and using these estimates to analyze cost overrun risk for the new project. For example, a new project might use one technology that has never been used in this kind of projects, but in some other projects. Project managers can first use regression models to analyze the impacts of this technology on project costs in previous projects. And use this estimate to 60 analyze the cost overrun risk for the new project. For example, if the regression analysis shows that the involvement of a certain technology in project may increase project cost overrun ratio by 5%. For the cost overrun ratio of a new project using this technology, is likely to be 5% larger than that of a similar project without using this technology. After all, projects have at least some features that can be found correspondingly in previous projects. Regression models can help test and quantifying the impacts of these features on project costs and therefore help cost estimation and cost overrun risk analysis for a new project. We suggest instead of relying solely on personal experiences, project managers can improve the reliability and accuracy of cost overrun risk analysis by using quantitative analysis models. 5. Summary and Conclusions Based on project risk source analysis, this thesis focuses on proposing risk analysis models to analyze cost overrun risk for transportation infrastructure projects. Making use of historical data in addition to experiences of project evaluators, the probability distribution models and regression models are used in project cost benefit analysis and improve on project evaluators' cost estimation in terms of reliability. Based on private sector behavior analysis, the PPP scenario analysis and PPP cost overrun risk analysis models determine some conditions for the success of PPP projects, which may help the public sector design successful PPP contracts. Our point is that project cost overrun risk analysis should be conducted by relying on feasible historical data. Current despise on quantitative project risk analysis and solely dependence on the experiences of project evaluators should be changed, since no individual's experiences can be comparable to the information we can draw from a valid project database. As we discuss in the Risk Definition section of the Literature Review, risk comprises both the probability of a risk event and its potential impact. Failing to consider either of these has the potential to create severe misunderstanding of risks. The probability distribution model in this thesis provides probability estimation for project cost overrun risks and the regression model provides estimates of the impacts of the risk factor variables, as well as predictions of project costs. The probability distribution model and the regression model in this thesis are both data based, intuitive and easily implemented in practical project cost risk analysis. The primary purpose of this thesis is to develop models that provide better project cost estimation without complicated technical and modeling requirements. 61 References Akintoye, Akintola S & Malcolm J MacLeod, Risk analysis and management in construction, International Journal of Project Management Vol. 15, No. 1, pp. 31-38, 1997 Ayyub, Bilal M. Risk Analysis in Engineering and Economics, Chapman & Hall/CRC, 2003 Boardman, Anthony E & Greenberg, David H. Cost-benefit Analysis: Concepts and Practice, Prentice Hall, 1996 Chapman, Chris, Project risk analysis and management—PRAM the generic process, International Journal of Project Management Wo\. 15, No. 5, pp. 273-281, 1997 Cho, Hyo-Nam et al. A risk assessment methodology for incorporating uncertainties using fuzzy concepts, Reliability Engineering and System Safety 78 (2002) 173-183, 2002 ) Cooper, Dale & Chapman, Chris, Risk Analysis for Large Projects: Models, Methods & Cases, John Wiley & Sons, 1987 Flyvbjerg, Bent, et al. Megaprojects and Risks: An Anatomy of Ambition, Cambridge, 2003 Flyvbjerg, Bent, et al. What Causes Cost Overrun in Transport Infrastructure Projects, Transport Reviews, Vol. 24, No. 1, 3-18, January 2004 Frame, J. Davidson Managing Risk in Organizations: A guide for managers, Jossey-Bass, 2003 Grimsey, Darrin & Lewis M. K, Evaluate the Risks of Public Private Partnerships for Infrastructure Projects, International Journal of Project Management, Vol 20, pp. 107118,2002 Grimsey, Darrin & Lewis, M. K. Public Private Partnerships: The Worldwide Revolution in Infrastructure Provision and Project Finance, Edward Elgar, 2004 Jaafari, Ali, Management of risks, uncertainties and opportunities on projects: time for a fundamental shift, International Journal of Project Management 19 (2001) 89-101, 2001 Kartam, Nabil A & Kartam, Saied A, Risk and its Management in the Kuwait Construction Industry: a Contractor's Perspective, International Journal of Project Management, vol.19, pp. 325-335, 2001 62 Lyons, Terry & Skitmore, Martin, Project risk management in the Queensland engineering construction industry: a survey, International Journal of Project Management 22(2004)51-61,2004 Ministry of Employment and Investment, Province of British Columbia, Risk Analysis for Public Private Partnership, 1995 Pindyck, Robert S. Irreversibility, Uncertainty and Investment, Journal of Economic Literature, 29, 1110-1148, 1991 Pouliquen, Louis Y. Risk Analysis in Project Appraisal, World Bank Staff Occasional Papers Number Eleven, John Hopkins Press, 1970 Public Financial Management, Inc. Final Report Intergovernmental Financial Risks of New Rail Start Projects, DTUM60-87-C-71343 Task 4, February 7, 1989 Regnier, Eva, Activity Completion Times in PERT and Scheduling Network Simulation, Part II, DRMI Newsletter, April 8, 2005 Shapira, Zur, Risk Taking: A Managerial Perspective, Russell Sage Foundation, 1994 Sturk, R et al. Risk and Decision Analysis for Large Underground Projects, as Applied to the Stockholm Ring Road Tunnels, Planning and Development, S0886-7798 (96) 00019- 3,1996 Titarenko B P, 'Robust technology' in risk management, International Journal of Project Management Vol. 15, No. 1, pp. 11-14, 1997 Vermeulen Erik M. et al. Analyzing risk and performance using the multi-factor concept, European Journal of Operational Research 93 (1996) 173-184, 1996 Williams, Terry, A classified bibliography of recent research relating to project risk management, European Journal of Operational Research 85 (1995) 18-38, 1995 . Wooldridge, Jefferey M, Introductory Econometrics: A Modern Approach, Thomson South-western, 2000 Ministry of Employment and Investment, Province of British Columbia, Risk Analysis for Public Private Partnership, 1995 63 Appendix Appendix 1. Beta Distribution Beta probability distribution has some nice properties: • has finite limits, which fit the lower and upper bounds for many real-world random variables. • can be asymmetric. Many variables in reality are right or left skewed to some unlikely but significant result. • flexible and can be in different shapes. The probability density function of beta distribution is: /(*) = (x- A)"-'{B x) -' — ,A < x < B;P,Q - A) ' B(P,Q)(B - Q p+Q 1 > 0 P and Q are the shape parameters. Shape parameters define the shape of a distribution. Beta distributions with different P and Q values have different probability density functions. Some classes of distributions have pre-defined shapes. For example, all normal distribution are always bell shaped. The following figure shows different Beta probability density functions for different P and Q values. Beta P D F ( 0 5 . 0 Beta P D F ( 0 5 , 2) (Sources from http://www.itl.nist.gov/div898/handbOok/eda/section3/eda366h.htm) A and B are the lower and upper bounds, respectively, of the distribution, and B(P,Q) is the beta function. B(P,Q) = J '^ "'( 0 P 1 x) ~ dx Q x The beta function can take on different shapes depending on the values of the two parameters: 64 is U-shaped P < 1,Q >Tor P = 1,Q > lis strictly decreasing P = Q = 1 is the uniform distribution P = 1,Q <Tor P > 1,Q < 1 is strictly increasing; P > 1, Q > 1is unimodal, which has a single local maximum value — If P = Q, the density function is symmetric about 1/2. P <\,Q<\ The mean and variance are calculated as follows: li = A + (B-A)S 2 =(B-A)~ P P + Q (P + QfiP Mode = A + (B- A) P + Q.+ l) 1 Q-2 P + ,P>\,Q>\ Appendix 2 Maximum Likelihood Estimation for Beta Distribution Shape Parameters We can use the following equations to estimate P and Q: B-A B-A n ~X n = Number of observations y = Cost overrun ratio for project / ' yAx) is the digamma function of x, which is given by the logarithmic derivative of the gamma function. i d T(x) ax l(x) T(x) is the gamma function, which is defined as: T{x)=^rVdt T(x+\) r(x)=—-—- x>0 A x<o X Appendix 3 Regression Analysis Reports for the VIHP • Small Road and Highway Project Regression Model Tests of Assumptions Section Assumption/Test Test Value 65 Prob Level Is the Assumption Reasonable at the 0.2000 Level of Significance? Residuals follow Normal Distribution? Shapiro Wilk Anderson Darling D'Agostino Skewness D'Agostino Kurtosis D'Agostino Omnibus 0.9433 0.3338 0.2396 -0.6424 0.4701 0 559802 0 508876 0 810625 0 520586 0 790512 Yes Yes Yes Yes Yes 1.0542 0 331348 Yes 0.5023 0.804051 Yes Constant Residual Variance? Modified Levene Test Relationship is a Straight Line? Lack of Linear Fit F(8, 1) Test Estimated Model (-.981527612769659) + ( 1.00551872149618) * (Log_Small_R_H_Project_Budget) Analvsisof Source Sum of Variance Section . DF Squares Mean Square Intercept 1 569546.3 Slope 1 4796.69 Error 9 10.03733 Adj. Total 10 4806.727 Total 11 574353 s = Square Root(l. 115259) = 1.056058 569546.3 4796.69 1.115259 480.6727 F-Ratio Prob Level 4300.9663 0.0000 Medium Sized R/H Project Regression Model Tests of Assumptions Section Test Assumption/Test Value Residuals follow Normal Distribution? Shapiro Wilk Anderson Darling DAgostino Skewness DAgostino Kurtosis DAgostino Omnibus Prob Level Is the Assumption Reasonable at the 0.2000 Level of Significance? 0.9817 0.2089 0.1667 0.7604 0.6060 0.986787 0.863874 0.867590 0.447033 0.738614 Yes Yes Yes Yes Yes 0.1020 0.755472 Yes Constant Residual Variance? Modified Levene Test Relationship is a Straight Line? Lack of Linear Fit F(9, 2) Test 0.4364 66 0.843076 Yes Power ( 5 % ) 1.0000 Estimated Model ( 105.068296795956) + (.619730185497456) * (Log_Medium_R_H_Project_Budget) Analysis of Variance Section Sum of Source DF Squares 1 Intercept 965603.8 1 Slope 350.3859 11 Error 712.8448 12 Adj. Total 1063.231 13 Total 966667 s = Square Root(64.80408) = 8.050098 Mean Square 965603.8 350.3859 64.80408 88.60256 F-Ratio Prob Level Power (5%) 5.4068 0.0402 0.5637 Large R/H Project Regression Model Tests ofAssumptions Section Test Assumption/Test Value Residuals follow Normal Distribution? Shapiro Wilk Anderson Darling DAgostino Skewness DAgostino Kurtosis DAgostino Omnibus Prob Level Is the Assumption Reasonable at the 0.2000 Level of Significance? 0.8199 7.1029 2.4162 4.3707 24.9410 0.000000 No 0.000000 No 0.015682 No 0.000012 No 0.000004 No 0.0436 0.835035 Yes Constant Residual Variance? Modified Levene Test Relationship is a Straight Line? Lack of Linear Fit F(51, 47) Test 0.9606 0.557187 Yes Estimated Model ( 1.65721935221895) + (.99547546003812) * Analysis of Variance Section Sum of Source DF Squares Intercept Slope 1 1 1.024448E+07 35932.66 67 (Log_Large_R_H_Project_Budget) Mean Square F-Ratio 1.024448E+07 35932.66 10065.5253 Prob Level Power (5%) 0.0000 1.0000 Error 98. 349.8477 Adj. Total 99 36282.51 Total 100 1.028076E+07 s = Square Root(3.569875) = 1.889411 3.569875 366.49 Residual Histogram Plot Histogram of Residuals of L o g J a g e J \ J H J ^ q e c t _ C c 6 t -4.0 ' 0.0 4.0 Residuals of Log J ^ g e J ^ J H J ^ c j e d C o s t Bridge and Tunnel Regression Model Tests of Assumptions Section Test Assumption/Test Value Residuals follow Normal Distribution? Shapiro Wilk Anderson Darling DAgostino Skewness DAgostino Kurtosis DAgostino Omnibus 0.6985 4.6111 2.4304 3.1520 15.8418 Prob Level 0.000001 0.000000 0.015082 0.001622 0.000363 Is the Assumption Reasonable at the 0 . 2 0 0 0 Level of Significance? No No No No • No • Constant Residual Variance? Modified Levene Test 1.1781 0.286386 Yes Relationship is a Straight Line? Lack of Linear Fit F(22, 8) Test 0.9236 0.589459 Yes Estimated Model (-.983040868152081) + ( 1.0033133225583) * (Log_large_B_T_Project_Budget) Analysis of Variance Section Source Intercept Slope Error DF 1 1 30 Sum of Squares 2999413 10899.37 8.849884 68 Mean Square F-Ratio 2999413 10899.37 36947.4954 0.2949961 Prob Level Power (5%) 0:0000 1.0000 Adj. Total 31 10908.22 Total 32 3010321 s = Square Root(0.2949961) = 0.5431355 351.878 Histogram of Residuals of LogJarge_BJTJ^cject_Cost A /^ - -1.5 -0.6 1.1 0.3 2.0 Residuals of L o g _ l a r g e _ B J J r q e c t _ C o s t 3 Appendix4. Distribution Summary in Arena Distribution: Beta Expression: 0.04 + 1.96 * BETA(7.18, 6.73) Square Error: 0.000274 Chi Square Test Number of intervals =16 Degrees of freedom =13 Test Statistic =147 Corresponding p-yalue< 0.005 Data Summary Number o f Data Points==17049 Min Data Value = 0.215 Max Data Value = 1.89 Sample Mean = 1.05 Sample Std Dev 0.254 Appendix5. Multiple Regression Report Run Summary Section Parameter Dependent Variable Number Ind. Variables Weight Variable R2 Adj R2 Coefficient of Variation Mean Square Error Square Root of M S E Completion Value log cost 3 None 0.9930 0.9929 0.0095 8.421432 2.90197 Parameter Rows Processed Rows Filtered Out Rows with X's Missing Rows with Weight Missing Rows with Y Missing Rows Used in Estimation Sum of Weights Completion Status 69 Value 163 0 0 0 0 163 163.000 Normal Ave Abs Pet Error 0.529 Regression Section Equation Regression Regression Independent Coefficient Variable b(i) Intercept 9.6613 (C39=1) -2.7525 Cost_Overrun_Ratio_Frequency -4.1109 log_budget 0.9723 Standard Standard Error Sb(i) 2.6249 1.1504 T-Value T-Value to test H0:B(i)=0 3.681 -2.393 1.9515 0.0083 -2.106 117.217 Prob Level 0.0003 0.0179 Reject Reject HO at 5%? Yes Yes Power Power of Test at 5% 0.9553 0.6621 0.0367 0.0000 Yes Yes 0.5532 1.0000 Estimated Model 9.66126733146649-2.7525117238345*(C39=1 )4.11086840238232*Cost_Overrun_Ratio_Frequency+ .972297221530233*log_budget Regression Coefficient Section Independent Regression Standard Lower Variable Coefficient Error 95% C.L. Intercept 9.6613 2.6249 4.4771 (C39=1) -2.7525 1.1504 -5.0246 Cost_Overrun_Ratio_Frequency -4.1109 1.9515 -7.9651 log_budget 0.9723 0.0083 0.9559 Note: The T-Value used to calculate these confidence limits was 1.975. Analysis of Variance of Variance -0.2566 0.9887 -0.0141 0.9840 Section Source Intercept Model Error Total(Adjusted) Analysis Upper Standardized 95% C.L. Coefficient 14.8455 0.0000 -0.4804 -0.0202 DF 1 3 159 162 Detail Sum of Mean Squares Square F-Ratio 1.528971E+07 1.528971E+07 0.9930 189704.8 63234.95 7508.811 0.0070 1339.008 8.421432 1.0000 191043.8 1179.283 R2 Prob Level Power (5%) 0.0000 1.0000 Prob Level Power (5%) 0.0000 0.0179 1.0000 0.6621 0.0367 0.0000 0.5532 1.0000 Section Model Term DF Intercept 1 Model 3 C39 1 Cost_Overrun_Ratio_Frequency 1 log_budget 1 Error 159 Total(Adjusted) 162 Sum of Mean Squares Square F-Ratio 1.528971 E+07 1.528971E+07 189704.8 63234.95 7508.811 0.9930 48.20848 48.20848 5.724 0.0003 R2 0.0002 0.6057 0.0070 1.0000 37.36802 115708.7 1339.008 191043.8 70 37.36802 4.437 115708.7 13739.785 8.421432 1179.283
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Transportation infrastructure project cost overrun...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Transportation infrastructure project cost overrun risk analysis : risk factor analysis models Wu, Qing 2006
pdf
Page Metadata
Item Metadata
Title | Transportation infrastructure project cost overrun risk analysis : risk factor analysis models |
Creator |
Wu, Qing |
Date Issued | 2006 |
Description | The main purpose of this thesis is to analyze some common risk factors and to propose several useful analytical models for cost overrun risk analysis in transportation infrastructure investments. Probability models and regression models are proposed and partially (due to data insufficiency) applied using the VIHP (Vancouver Island Highway Project) data. The VIHP case study shows that cost overrun ratio increases as project sizes increase for small road/highway projects (budget < $0.25 million) and bridge/tunnel projects (budget >$0.85 million). However for road/highway projects with budgets over $0.25 million, cost overrun ratio decreases as project size increases. Using the VIHP database, results of a distribution fitting model and a Monte Carlo simulation model are compared. Compared with the distribution fitting model, the Monte Carlo simulation model is shown to underestimate both the upper bound value of project cost overrun ratio and slightly the probabilities of cost overrun. The distribution fitting model and regression model are shown to have close estimates of project costs and cost ranges at each confidence level. |
Genre |
Thesis/Dissertation |
Type |
Text |
Language | eng |
Date Available | 2010-01-16 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
DOI | 10.14288/1.0092815 |
URI | http://hdl.handle.net/2429/18377 |
Degree |
Master of Science - MSc |
Program |
Business Administration |
Affiliation |
Business, Sauder School of |
Degree Grantor | University of British Columbia |
Graduation Date | 2006-11 |
Campus |
UBCV |
Scholarly Level | Graduate |
Aggregated Source Repository | DSpace |
Download
- Media
- 831-ubc_2006-0720.pdf [ 6.29MB ]
- Metadata
- JSON: 831-1.0092815.json
- JSON-LD: 831-1.0092815-ld.json
- RDF/XML (Pretty): 831-1.0092815-rdf.xml
- RDF/JSON: 831-1.0092815-rdf.json
- Turtle: 831-1.0092815-turtle.txt
- N-Triples: 831-1.0092815-rdf-ntriples.txt
- Original Record: 831-1.0092815-source.json
- Full Text
- 831-1.0092815-fulltext.txt
- Citation
- 831-1.0092815.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0092815/manifest