International Construction Specialty Conference of the Canadian Society for Civil Engineering (ICSC) (5th : 2015)

A review of the current knowledge and practice related to project progress and performance assessment Orgut, Resulali E.; Zhu, Jin; Batouli, Mostafa; Mostafavi, Ali; Jaselskis, Edward J. Jun 30, 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52660-Orgut_R_et_al_ICSC15_083_Project_Progress_Assessment.pdf [ 401.29kB ]
52660-Orgut_R_et_al_ICSC15_083_Project_Progress_Assessment_slides.pdf [ 2.37MB ]
Metadata
JSON: 52660-1.0076386.json
JSON-LD: 52660-1.0076386-ld.json
RDF/XML (Pretty): 52660-1.0076386-rdf.xml
RDF/JSON: 52660-1.0076386-rdf.json
Turtle: 52660-1.0076386-turtle.txt
N-Triples: 52660-1.0076386-rdf-ntriples.txt
Original Record: 52660-1.0076386-source.json
Full Text
52660-1.0076386-fulltext.txt
Citation
52660-1.0076386.ris

Full Text

5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   A REVIEW OF THE CURRENT KNOWLEDGE AND PRACTICE RELATED TO PROJECT PROGRESS AND PERFORMANCE ASSESSMENT Resulali E. Orgut1, Jin Zhu2, Mostafa Batouli2, Ali Mostafavi3 and Edward J. Jaselskis1 1 Department of Civil, Construction and Environmental Engineering, North Carolina State University, USA 2 Department of Civil and Environmental Engineering, Florida International University, USA 3 OHL School of Construction, College of Engineering and Computing, Florida International University, USA Abstract: Assessment of true project progress and performance is of critical importance in the successful delivery of construction projects. Major challenges related to measuring project progress and performance are the lack of consistent, reliable, and objective metrics and indicators and the lack of appropriate interpretation of these data for establishing suitable corrective action plans. The objective of this paper is to provide a review of existing applied knowledge and practices pertaining to methods, metrics and indicators for progress measurement, performance assessment and forecasting, as well as performance influencing factors, evaluating the shortcomings of the current approaches, and providing recommendations for improvement. The findings of this paper are primarily based on a comprehensive literature review and limited discussions with industry experts in the following areas: (1) methods and metrics used for progress measurement, (2) metrics and indicators used for performance assessment and forecasting, and (3) other metrics that can influence project progress and performance (e.g., risk, safety, and quality). Several industry and academic publications are reviewed including the reports from the Construction Industry Institute (CII), guidelines developed by professional organizations (e.g. Project Management Institute, Association for the Advancement of Cost Engineering International), and scholarly publications. Industry experts serving on the CII research team (RT-322) also provide their insights. Based on the extensive review of the relevant literature, this paper identifies limitations of various measures, metrics and indicators across different project control levels. A framework depicting the current project control process is provided along with a gap analysis related to the problems associated with this approach. 1 INTRODUCTION Performance inefficiency on construction projects has been adversely affecting the industry. Only 1 in every 20 projects is able to meet both its authorized cost and schedule within an acceptable margin (CII, 2012). Besides, Construction Industry Institute (CII) member industrial-sector EPC contractors are not able to generate any profit in 3 out of 5 projects (CII, 2014). One of the major reasons for the low efficiency in construction project performance is the incapability of the existing methods in providing a “true” measure of project progress and. A review on the existing methods reveals that there is not a systematic, consistent and efficient approach to identifying and interpreting progress and performance assessment metrics in the construction industry. 083-1 This results in the true performance of construction projects being potentially misrepresented and causes misunderstandings since the true project progress and performance is not known with certainty. Lack of an integrated approach and standards for assessment of progress and performance prevents not only evaluating the quality of metrics and indicators, but also effective interpretation as a basis for improvement. Absence of objectivity in progress measurement and performance assessment misleads perception regarding the true status of the project.  In a field with a large number of studies and established methods and approaches, a systemic review and synthesis enables gap analysis that inform current and future research. Hence, the first step towards improving progress and performance assessment is to analyze the gaps and limitations in the existing methods and metrics. Despite a vast body of literature in the area of project progress and performance assessment, there are limited studies that provide systemic review and gap analysis related to the existing metrics and methods. To this end, this review paper addresses the following questions: (1) what are the components of progress and performance assessment, (2) what methods and metrics are used for each component, and finally (3) what are the strengths and limitations of the existing methods and metrics for a robust assessment. Section 2 explains the methodology of this literature review study. Section 3 introduces a framework to classify project control functions at a single project level. Section 4 provides an overview of existing literature pertaining to the methods, metrics and indicators used for progress measurement, performance assessment, performance influencing factors and performance forecasting. Section 5 discusses gaps in the existing body of knowledge and practice, as well as identifying areas for improvement. Finally Section 6 provides the recommendations and concludes the paper with highlighting the needed research tasks to address the identified gaps. 2 METHODOLOGY The research methodology included three steps: (1) identifying relevant work; (2) collecting and summarizing information; and (3) conducting gap analysis. Two sources of information were used: academic and professional literature, and expert opinion from industry professionals with extensive experience in project management and project control. (1) Identifying relevant work: Through a comprehensive review of literature, 116 relevant documents were identified and reviewed: 64 professional organization reports [e.g., Construction Industry Institute, Project Management Institute (PMI) and Association for the Advancement of Cost Engineering (AACE)], 41 scholarly articles, and 11 government agency reports (e.g., reports from the Department of Energy and Department of Transportation). The identification and selection of documents started with utilizing specific keywords (e.g., performance assessment, KPI, and project control) and continued through snowball method. The search for relevant work stopped when additional searches did not lead to identification of new information. (2) Collecting and summarizing information: The articles and reports identified in the previous step were reviewed, analyzed and summarized to satisfy the research objectives. The collected information was used to identify the components of a framework for project controls and the methods and metrics associated with each component. The project control framework was then refined based on the feedback from industry professionals (i.e., individuals from eight owner and eight contractor companies, with over 100 years of combined experience) involved with this study through the channels of CII were essential in advancing the framework. Based on the knowledge and experience of industry representatives, the framework was iteratively improved to illustrate the project controls cycle as it occurs in projects.  (3) Conducting gap analysis: The framework created in the previous step was used to identify the existing gaps in the body of knowledge related to metrics and methods for improving project control and management to provide “true” insight about project progress and performance. The identified gaps were further analyzed and refined through multiple rounds of panel discussions with the industry professional participating in this study. The information collected from the panel discussions were analyzed through mind-mapping techniques to systemically evaluate the existing gaps. 083-2 3 A FRAMEWORK FOR PROJECT CONTROLS Core components of a project control structure (i.e., progress measurement, performance assessment, performance influencing factor metrics and performance forecasting) take place in a cyclical nature, which is repeated for every new reporting period, until the project is completed. Figure 1 illustrates the control cycle of a project for a single reporting period. This framework depicts the project control cycle in current practice, obtained through extensive literature search and insights from industry experts serving on Research Team 322 of CII.  Figure 1: Current practice framework for project controls Although the terms “operational”, “tactical” and “strategic” are frequently used to define different control levels of construction projects, there is no consensus among construction industry stakeholders over what these terms represent in a traditional project controls setting. PMI recognizes operational, tactical and strategic project levels as organizational layers of a portfolio management structure and offers guidelines to identify the work performance structure through information and data flow between project control levels (PMI, 2013). AACE International approaches “strategic” and “tactical” levels as different layers of the investment decision making process, whereas the term “operational” defines ongoing endeavors or activities (AACE International, 2012). However, these approaches lack a single project emphasis. Concentration on project controls structure at a single project level would help identify areas of improvement. Figure 1 matches project levels with project control functions from a single project perspective. Operational, tactical and strategic project levels are characterized mainly as they relate to metrics and indicators for progress measurement, performance assessment, performance influencing factors and performance forecasting, of a single project. The operational level basically serves the purpose of understanding the current state of the project. It mainly contains measurement of progress in terms of cost and schedule. Some observations and inspections for performance influencing factors also take place at operational level, such as measuring safety and quality aspects of the project. In the tactical level, data from progress measurement metrics and indicators obtained in the operational level are compared against the authorized baseline values in order to identify cost, schedule and efficiency variances and to understand the true current state. Also at the tactical level, performance influencing factor metrics are evaluated, as they relate to interpreting reasons behind why the project is at its current state. Finally, at the strategic level, the future state is predicted based on the current project trajectory.  083-3 4 METHODS, METRICS AND INDICATORS RELATED TO PROJECT PROGRESS AND PERFORMANCE 4.1 Progress Measurement The successful execution of a project hinges on various factors, but one fundamental question that should be answered as the first step in the control process is: What has been accomplished so far? Progress measurement addresses this question at the operational level and serves as the basis for assessing, forecasting, and improving project performance. In this paper, progress measurement is defined as the measurement of outcomes and throughput of a project at a certain point in time. It provides the fundamental information related to “where are we at?” There are several metrics used to measure construction project progress depending on the activities and work items. Although the names and definitions of these metrics could vary for different projects, they belong to one of the major progress measurement methods. The major methods for progress measurement include: units completed, incremental milestone, weighted or equivalent units completed, resource expenditure, and judgment (supervisor opinion) (CII, 1987; AACE International, 2012). Table 1 provides the definitions, applicable conditions, and implementation examples for each method.  Table 1: Definitions, applicable conditions and examples of progress measurement methods Method Definition Application Conditions Implementation Example Units completed quantity surveys or physical measurement of work items homogenous units of work linear-feet of wire pulling, cubic-yard of concrete placed Incremental milestone a percent completion be credited for completion of key incremental tasks one deliverable with multiple activities performed in sequence; output for each subtask cannot be easily measured Equipment installation, alignment and testing (e.g., 50, 70, and 100 percent of completion, respectively) Weighted or equivalent units completed a hybrid of units completed and incremental milestones a major effort involving a long period of time; composed of two or more overlapping subtasks, each with a different unit of work measurement Subtasks in structural steel erection with different units of measure are converted to equivalent tons and then the weighted percent complete is calculated Resource expenditure the percent of the total planned or forecast duration hours, or cost spent for the control account no discrete deliverables or milestones in the work package; a relatively constant level of effort Measurement for tasks such as project management, quality assurance, contract administration, and project controls Judgment the person responsible for the work package estimates the percent complete based on his or her informed judgment only for relatively minor tasks where development of a more discrete method cannot be used Measurement for tasks such as painting, dewatering, constructing support facilities, installing architectural trim, and landscaping Some of these methods are identified with different names in practice. For example, resource expenditure is occasionally labelled as Level of Effort (LOE) or cost ratio. Judgment is also recognized as supervisor opinion. Variations of these five basic progress measurement methods also exist. For example, start/finish is a variation to the incremental milestone method in which the only milestones are starting and finishing. Activities in a construction project such as flushing and cleaning, testing and major rigging 083-4 operations do not have readily definable intermediate milestones. Workers know when the work starts and when it is finished, but not the percentage completion in between. For this type of activities, start/finish is the most appropriate progress measurement method. Each progress measurement method has its own strengths and weaknesses. For example, the units completed method provides the most detailed and accurate progress information. However, it can only be applicable to homogenous units of work. Also, it might take a significant amount of time for data collection. On the other hand, the judgment method takes the least amount of time and effort. However, this subjective approach is highly dependent on the experience of the supervisor, and could be inaccurate and misleading. In general, there is a trade-off relationship between accuracy/consistency and efficiency in progress measurement (Chin et al., 2004). Table 2 summarizes strengths and weaknesses of these progress measurement methods. Table 2: Strengths and weaknesses of existing progress measurement methods (Thomas, 2000) Method Strengths Weaknesses Units completed • Most detailed and accurate • Does not rely on subjective opinions or evaluations • Claimed output can be readily verified • Time for data collection might be lengthy, especially if not applied correctly Incremental milestone • Easy to use • Simple to understand • Long periods may elapse before an intermediate milestone is reached Weighted or equivalent units completed • Detailed and objective • Provides ability to compare and summarize several different subtasks and activity groups • May be inaccurate, especially if there are few items and the activity durations are lengthy • Weighing or equivalency conversions and calculations might be complex, as well as requiring attention Resource expenditure • Greater detail and objectivity than simply estimating how much work was done • Less expensive than counting or measuring the units completed of subtasks • Requires much more effort than simply estimating the percent complete Judgment • Simple • Inexpensive • Quick • Can be very inaccurate and misleading Since one complete project usually includes different types of tasks, multiple progress measurement methods are often used on a project. One limitation in the existing body of knowledge is the lack of an integrated framework which facilitates applying various progress measurement methods systematically in the project. The implementation of different progress measurement methods are usually based on the project manager’s experience instead of objective criteria. The subjectivity may cause problems in providing true measurement of project progress in an efficient and effective way. For example, using the incremental milestone method for measuring the progress of landscaping as a minor component of the project instead of judgment method could improves the accuracy, however, it requires more time and effort to develop the criteria and collect the data. 083-5 4.2 Performance Assessment Progress measurement is essential for successful project execution, although it is not adequate by itself for effective project management and control. It is of vital importance to compare measured progress against the baseline in order to evaluate the current performance of the project. This paper identifies performance assessment as an evaluation of the existing outcomes and results of the project cost, schedule and efficiency at a certain point in the project cycle. It answers the question of “Where should we be at?” There are several metrics and indicators used to assess performance of construction projects based on different approaches. The most commonly used performance assessment method in the construction industry is Earned Value Management (EVM). Due to certain shortcomings of EVM, other methods such as Earned Schedule Method (ESM) and Earned Duration Management (EDM), are offered as viable alternatives. Table 3 explains commonly used methods and their related metrics.  Table 3: Definitions, related metrics and calculations of main performance assessment methods Method Related Metrics Calculations Earned Value Management (EVM) Cost Variance (CV) CV = EV – AC Cost Performance Index (CPI) CPI = EV / AC Schedule Variance (SV) SV = EV – PV Schedule Performance Index (SPI) SPI = EV / PV Earned Schedule Management (ESM) Earned Schedule (ES) * ES(t) = t + EV−PV(t) PV(t+1)−PV(t) Earned Schedule Performance Index (SPI (t)) SPI(t) = ES(t) Actual Duration Earned Duration Management (EDM) Earned Duration (ED) ** EDi = BPDi × APIi Earned Duration Index (EDIi) *** EDIi =  EDi  PDi  Critical Chain Project Management (CCPM) Critical Chain Completed (CCC) number of days of critical chain work completed total number of days on critical chain Buffer Consumption Rate (BCR) percentage of project buffer consumed percentage of critical chain completed * “t” stands for the time status that ES is calculated for, whereas “t+1” is for the following time period. ** BPDi stands for Baseline Planned Duration of scheduled activity I, whereas APIi is the Activity Progress Index, measured schedule progress of an activity through whichever method is preferred in the project. *** PDi is for Planned Duration of an activity i EVM methodology uses metrics to answer questions such as: what are the actual and planned costs of a project at a given point in time in relation to its earned value (Vanhoucke 2009). At the operational level, EVM captures three major values namely Actual Cost (AC), Planned Value (PV) and Earned Value (EV) (Fleming and Koppelman, 2000). These values represent the measured progress in terms of cost. One significant drawback of EVM is its inability to capture the schedule performance, especially over the last third of the project (Lipke 2009). In order to improve the schedule assessment ability of EVM, ESM was proposed (Lipke, 2003). ESM relies on the same operational-level performance indicators as EVM (i.e. PV, AC and EV); however it uses a new set of time-based performance assessment metrics to identify Earned Schedule (ES) as the time at which the amount of earned value should have been accomplished. Although ESM solves some limitations of time management in EVM, it still uses cost as a proxy to measure schedule performance of a project. Therefore, when a disparity exists between time and cost 083-6 profiles of a project, ESM fails to capture the distinctive schedule behavior of the project. To address this limitation of EVM and ESM, Earned Duration Management (EDM) was developed, which decouples schedule and cost performance measures. EDM also identifies a number of indices to measure progress and performance of schedule and cost (Khamooshi and Golafshani 2013).  EVM, ESM and EDM treat delays on critical and none-critical paths equivalently. Also the correlations between time and cost of different activities are not taken into consideration in any of these methods. In terms of performance forecasting, the adaptive behaviors of the project managers is not considered. And finally the assessment of the project in all these methods is limited to time and cost and other performance measures such as quality of the final products are not part of the analysis (Hall 2012; Hazır 2014). Critical Chain Project Management (CCPM), an emerging approach for performance assessment, focuses on resource utilization and minimizing idle capacity (Goldratt, 1997; Leach, 2000). However, issues related to the stability of the critical chain and the network structure, as well as resource efficiencies and multitasking are considered problematic aspects of CCPM.  For different project types and phases, various performance assessment metrics and indicators should be utilized, sometimes in combination with one another. However, one general weakness of current performance assessment methodologies is lack of consistency and implementation guidelines for metrics throughout the construction industry, which eventually diminishes reliability of metrics and indicators. 4.3 Performance Influencing Factor Metrics Performance influencing factor metrics are characteristics of the project or project organizations that influence the performance outcomes of the project (e.g., scope change, risk management, and resource management by answering the question “Why are we here?” Therefore, these metrics explain reasons behind the current state of the project by allowing managers to (1) know where to look if a problem related to cost, schedule or efficiency performance occurs, and (2) interpret performance assessment and forecasting more accurately and develop proactive response plans and corrective actions. Lack of unified guidelines to tackle problems as they relate to project progress and performance through influencing factor metrics not only leads to inconsistent and unreliable project information, but also makes it difficult to objectively compare one project to another. One exception to this problem is related to safety metrics. Almost every project uses nearly the same metrics and indicators for safety: OSHA Recordable Incident Rate (ORIR), Days Away, Restrictions and Transfers (DART) rate, number of near-miss incidents, people based safety observation, new hire count-vs-recordable injury rate, Total recordable incident rate (TRIR), Lost time incident rate (LTIR) and Total severity rate (TSR) (NRC, 2005; COAA, 2007). In addition to being related to life and death situations, safety is also highly regulated through guidelines. This highlights the importance and necessity of having established guidelines for using other common performance influencing factor metrics in the construction industry. 4.4 Performance Forecasting Performance forecasting uses current state performance information to predict the future outcomes and results. It basically answers the question of “Where will we be at?” Table 4 lists performance forecasting metrics for cost and schedule estimations.  Earned Value Management (EVM) offers several useful metrics to forecast what the expected cost values are for the remainder of the project (Vanhoucke, 2009). These metrics predict the final cost of the project based on the actual performance at the time of assessment and the planned cost of the remaining work. However, these projections are highly dependent on consistency of performance assessment metrics used and management’s opinion on some forecasted elements. (Fleming and Koppelman, 2000). The EVM methodology consists of metrics for schedule performance forecasting as well, similar to the cost metrics described above. However, similar problems can be observed with schedule performance forecasting metrics of EVM. At this point, ESM offers alternative schedule forecasting metrics (Lipke 2009). The same pattern with performance assessment metrics follows here; even though ESM metrics generate better results for schedule, they are yet to offer a robust forecasting alternative due to using cost as a proxy to estimate schedule of a project. 083-7 Table 4: Forecasting metrics, definitions and related methods  (Fleming and Koppelman, 2000; Vanhoucke, 2009; Lipke, 2009) Metric Definition Calculation Cost/Schedule Method Estimate at Completion (EAC)  manager’s projection of total cost of the project at the end of the project EAC=BAC/CPI Cost EVM Estimate to Completion (ETC) estimated cost required to complete the remainder of the project ETC=EAC-AC Cost EVM Variance at Completion (VAC) variance on the total budget at the end of the project VAC= BAC-EAC Cost EVM To Complete Performance Index (TCPI) future required cost efficiency needed to achieve a target BAC (budget at complete) or EAC (estimate at complete) TCPI(BAC)= (BAC-EV)/(BAC-AC) or TCPI(EAC)= (BAC-EV)/(EAC-AC) Cost EVM Independent Estimate at Completion (IEAC) a metric to project total cost using the performance to date to project overall performance. IEAC= AC+(BAC-EV)/CPI Cost EVM Estimate at Completion (time) [EAC(t)] manager’s projection of total duration of the project at the end of the project EAC(t)=PD/SPI(t) Schedule ESM Estimate to Completion (time) [ETC(t)] estimated time required to complete the remainder of the project ETC(t)=EAC(t)-AT Schedule ESM Variance at Completion (time) [VAC(t)] predicted variance on the total schedule at the end of the project VAC= PD – EAC(t) Schedule ESM To Complete Schedule Performance Index (TSPI) required time efficiency needed to achieve a target PD or EAC(t). (The TSPI provides a projection of the anticipated performance required to achieve either the PD or the EAC(t)) TSPI(PD)= (PD-ES)/(PD-AT) or TSPI[EAC(t)]= (PD-ES)/(EAC(t)-AT) Schedule ESM Independent Estimate at Completion (time) [IEAC(t)] an independent second opinion of the final project duration IEAC= AT+(PD-ES)/SPI(t) Schedule ESM BAC: Budget at Completion PD: Planned Duration AT: Actual Time  5 GAP ANALYSIS AND DISCUSSION In this study, mind mapping was used to abstract the tacit knowledge of the industry panel regarding two central topics: (1) the problems related to the existing project progress and performance assessment metrics and methods, and (2) the required improvements. In the first step of mind mapping, each individual on the industry panel provided his/her perspective regarding the central topic. The information 083-8 was collected and visualized into a preliminary mind map with a tree structure using Free Mind software. In the second step, the preliminary mind map was presented to the industry panel. The panel was asked to cluster the branches in the mind map into categories of problems and required improvements related to the existing metrics and methods. This step was concluded when the panel reached a consensus. Mind map in Figure 2 clearly depicts problematic areas and needed improvement identified through this study. Review of the current knowledge and practice revealed that there is no consistent set of progress measurement metrics and performance assessment indicators. Due to this lack of consistency, it is difficult to have a true comparison of one project performance to another. Also, there is potential for misrepresentation of progress and performance information, caused by unreliability of metrics and indicators. The existing use of metrics and indicators vary across the industry and depend on the level of the experience of project personnel. There is also limited understanding regarding the importance and effects of performance influencing factor metrics. Hence, there is a need for guidelines identifying what core metrics and key performance indicators (KPIs) should be used consistently on construction projects. Development of such guidelines can help reduce subjectivity in the metrics. Furthermore, in order to improve progress and performance assessment, identification of recommended practices for effective interpretation is necessary. Finally, understanding the significant impacts of performance influencing factor metrics and indicators to progress and performance can provide greater insight for performance improvement and foresight for forecasting project outcomes.  Figure 2: Mind map for identified problems and improvement areas 6 CONCLUSIONS AND RECOMMENDATIONS The purpose of this paper was to review common project progress, performance assessment and forecasting methods used in the construction industry. A framework was presented describing the operational, tactical, and strategic levels of project controls. After reviewing the literature and current practices in project controls, it is apparent that construction industry needs certain actions to improve project progress and performance assessment. This paper recommends to: (1) identify core metrics and key indicators, (2) develop guidelines for improving the reliability of metrics and indicators, and (3) establish recommended practices for interpreting metrics and indicators, forecasting outcomes, and responding to variance to plan as they relate to progress and performance assessment. In order to satisfy the industry needs and fulfill recommended courses of action, CII Research Team 322 (RT-322) is working on a rigorous three-step data collection and validation plan: (1) a web-based survey that uses the findings from the gap analysis presented in this paper to identify core methods, metrics and indicators, (2) case studies to understand the significance of core metrics and how reliability of these metrics can be improved and (3) Delphi method to validate findings using expert opinions.  083-9 Problems and suggested improvements identified through this study, can inform future research on applied project controls. Consequentially, real-life applications of these findings have the potential to advance many construction industry practices. Acknowledgements This research has been supported by the Construction Industry Institute. The opinions expressed in the paper represent those of the authors and not necessarily those of the Construction Industry Institute. References Association for Advanced Cost Engineering International (AACE International) 2012. Total Cost Management Framework: An Integrated Approach to Portfolio, Program, and Project Management. Revised 1st ed. AACE International, Morgantown, WV, USA. Chin, S., Yoon, S., Kim, Y. S., Jung, Y., Park, S. C., and Chung, M. 2004. A project Progress Measurement and Management System. 21st International Symposium on Automation and Robotics in Construction, Jeju, South Korea. Construction Industry Institute (CII). 1987. Research Summary 6-5: Project Control for Construction. CII, Austin, TX, USA. Construction Industry Institute (CII). 2012. Performance Assessment 2012 Edition. CII, Austin, TX, USA. Construction Industry Institute (CII). 2014. Performance Assessment 2014 Edition. CII, Austin, TX, USA. Construction Owners Association of America (COAA). 2007. CII/COAA Benchmarking & Metrics: Performance Metric Formulas and Definitions. COAA, Austell, GA, USA. Fleming, Q., and Koppelman, J. 2000. Earned value project management 2nd ed., Project Management Institute, Newton Square, PA, USA. Goldratt, E. 1997. Critical chain. 1st ed., North River Press, Great Barrington, MA, USA. Hall, N. G. 2012. Project management: Recent developments and research opportunities. Journal of Systems Science and Systems Engineering, 21(2): 129-143. Hazır, Ö. 2014. A review of analytical models, approaches and decision support tools in project monitoring and control. International Journal of Project Management. Kenley, Russell, and Olli Seppänen. 2010. Location-based management for construction: Planning, scheduling and control. 1st ed., Spon Press, London, UK. Khamooshi, H., and Golafshani, H. 2013. EDM: Earned Duration Management, a new approach to schedule performance management and measurement. International Journal of Project Management, 32(6), 1019-1041. Leach, L. 2000. Critical chain project management. 2nd ed., Artech House, Boston, MA, USA. Lipke, W. H. 2003. Schedule is different. Measurable News, 2003(March), 10-15. Lipke, W. H. 2009. Earned schedule. Lulu Publishing, Raleigh, NC, USA. National Research Council (NRC). 2005. Measuring Performance and Benchmarking Project Management at the Department of Energy. The National Academies Press, Washington, DC, USA. Project Management Institute (PMI). 2013. A guide to the project management body of knowledge: (PMBOK® guide). 5th ed., Project Management Institute, Newtown Square, PA, USA. Thomas, H. 2000. Principles of Construction Labor Productivity Measurement and Processing: Final Report. University Park, PA, USA: Pennsylvania Transportation Institute, Pennsylvania State University. Vandevoorde, S., and Vanhoucke, M. 2006. A comparison of different project duration forecasting methods using earned value metrics. International Journal of Project Management, 24(4): 289-302. Vanhoucke, M. 2009. Measuring time: Improving project performance using earned value management. Springer, New York, NY, USA.   083-10  5th International/11th Construction Specialty Conference 5e International/11e Conférence spécialisée sur la construction    Vancouver, British Columbia June 8 to June 10, 2015 / 8 juin au 10 juin 2015   A REVIEW OF THE CURRENT KNOWLEDGE AND PRACTICE RELATED TO PROJECT PROGRESS AND PERFORMANCE ASSESSMENT Resulali E. Orgut1, Jin Zhu2, Mostafa Batouli2, Ali Mostafavi3 and Edward J. Jaselskis1 1 Department of Civil, Construction and Environmental Engineering, North Carolina State University, USA 2 Department of Civil and Environmental Engineering, Florida International University, USA 3 OHL School of Construction, College of Engineering and Computing, Florida International University, USA Abstract: Assessment of true project progress and performance is of critical importance in the successful delivery of construction projects. Major challenges related to measuring project progress and performance are the lack of consistent, reliable, and objective metrics and indicators and the lack of appropriate interpretation of these data for establishing suitable corrective action plans. The objective of this paper is to provide a review of existing applied knowledge and practices pertaining to methods, metrics and indicators for progress measurement, performance assessment and forecasting, as well as performance influencing factors, evaluating the shortcomings of the current approaches, and providing recommendations for improvement. The findings of this paper are primarily based on a comprehensive literature review and limited discussions with industry experts in the following areas: (1) methods and metrics used for progress measurement, (2) metrics and indicators used for performance assessment and forecasting, and (3) other metrics that can influence project progress and performance (e.g., risk, safety, and quality). Several industry and academic publications are reviewed including the reports from the Construction Industry Institute (CII), guidelines developed by professional organizations (e.g. Project Management Institute, Association for the Advancement of Cost Engineering International), and scholarly publications. Industry experts serving on the CII research team (RT-322) also provide their insights. Based on the extensive review of the relevant literature, this paper identifies limitations of various measures, metrics and indicators across different project control levels. A framework depicting the current project control process is provided along with a gap analysis related to the problems associated with this approach. 1 INTRODUCTION Performance inefficiency on construction projects has been adversely affecting the industry. Only 1 in every 20 projects is able to meet both its authorized cost and schedule within an acceptable margin (CII, 2012). Besides, Construction Industry Institute (CII) member industrial-sector EPC contractors are not able to generate any profit in 3 out of 5 projects (CII, 2014). One of the major reasons for the low efficiency in construction project performance is the incapability of the existing methods in providing a “true” measure of project progress and. A review on the existing methods reveals that there is not a systematic, consistent and efficient approach to identifying and interpreting progress and performance assessment metrics in the construction industry. 083-1 This results in the true performance of construction projects being potentially misrepresented and causes misunderstandings since the true project progress and performance is not known with certainty. Lack of an integrated approach and standards for assessment of progress and performance prevents not only evaluating the quality of metrics and indicators, but also effective interpretation as a basis for improvement. Absence of objectivity in progress measurement and performance assessment misleads perception regarding the true status of the project.  In a field with a large number of studies and established methods and approaches, a systemic review and synthesis enables gap analysis that inform current and future research. Hence, the first step towards improving progress and performance assessment is to analyze the gaps and limitations in the existing methods and metrics. Despite a vast body of literature in the area of project progress and performance assessment, there are limited studies that provide systemic review and gap analysis related to the existing metrics and methods. To this end, this review paper addresses the following questions: (1) what are the components of progress and performance assessment, (2) what methods and metrics are used for each component, and finally (3) what are the strengths and limitations of the existing methods and metrics for a robust assessment. Section 2 explains the methodology of this literature review study. Section 3 introduces a framework to classify project control functions at a single project level. Section 4 provides an overview of existing literature pertaining to the methods, metrics and indicators used for progress measurement, performance assessment, performance influencing factors and performance forecasting. Section 5 discusses gaps in the existing body of knowledge and practice, as well as identifying areas for improvement. Finally Section 6 provides the recommendations and concludes the paper with highlighting the needed research tasks to address the identified gaps. 2 METHODOLOGY The research methodology included three steps: (1) identifying relevant work; (2) collecting and summarizing information; and (3) conducting gap analysis. Two sources of information were used: academic and professional literature, and expert opinion from industry professionals with extensive experience in project management and project control. (1) Identifying relevant work: Through a comprehensive review of literature, 116 relevant documents were identified and reviewed: 64 professional organization reports [e.g., Construction Industry Institute, Project Management Institute (PMI) and Association for the Advancement of Cost Engineering (AACE)], 41 scholarly articles, and 11 government agency reports (e.g., reports from the Department of Energy and Department of Transportation). The identification and selection of documents started with utilizing specific keywords (e.g., performance assessment, KPI, and project control) and continued through snowball method. The search for relevant work stopped when additional searches did not lead to identification of new information. (2) Collecting and summarizing information: The articles and reports identified in the previous step were reviewed, analyzed and summarized to satisfy the research objectives. The collected information was used to identify the components of a framework for project controls and the methods and metrics associated with each component. The project control framework was then refined based on the feedback from industry professionals (i.e., individuals from eight owner and eight contractor companies, with over 100 years of combined experience) involved with this study through the channels of CII were essential in advancing the framework. Based on the knowledge and experience of industry representatives, the framework was iteratively improved to illustrate the project controls cycle as it occurs in projects.  (3) Conducting gap analysis: The framework created in the previous step was used to identify the existing gaps in the body of knowledge related to metrics and methods for improving project control and management to provide “true” insight about project progress and performance. The identified gaps were further analyzed and refined through multiple rounds of panel discussions with the industry professional participating in this study. The information collected from the panel discussions were analyzed through mind-mapping techniques to systemically evaluate the existing gaps. 083-2 3 A FRAMEWORK FOR PROJECT CONTROLS Core components of a project control structure (i.e., progress measurement, performance assessment, performance influencing factor metrics and performance forecasting) take place in a cyclical nature, which is repeated for every new reporting period, until the project is completed. Figure 1 illustrates the control cycle of a project for a single reporting period. This framework depicts the project control cycle in current practice, obtained through extensive literature search and insights from industry experts serving on Research Team 322 of CII.  Figure 1: Current practice framework for project controls Although the terms “operational”, “tactical” and “strategic” are frequently used to define different control levels of construction projects, there is no consensus among construction industry stakeholders over what these terms represent in a traditional project controls setting. PMI recognizes operational, tactical and strategic project levels as organizational layers of a portfolio management structure and offers guidelines to identify the work performance structure through information and data flow between project control levels (PMI, 2013). AACE International approaches “strategic” and “tactical” levels as different layers of the investment decision making process, whereas the term “operational” defines ongoing endeavors or activities (AACE International, 2012). However, these approaches lack a single project emphasis. Concentration on project controls structure at a single project level would help identify areas of improvement. Figure 1 matches project levels with project control functions from a single project perspective. Operational, tactical and strategic project levels are characterized mainly as they relate to metrics and indicators for progress measurement, performance assessment, performance influencing factors and performance forecasting, of a single project. The operational level basically serves the purpose of understanding the current state of the project. It mainly contains measurement of progress in terms of cost and schedule. Some observations and inspections for performance influencing factors also take place at operational level, such as measuring safety and quality aspects of the project. In the tactical level, data from progress measurement metrics and indicators obtained in the operational level are compared against the authorized baseline values in order to identify cost, schedule and efficiency variances and to understand the true current state. Also at the tactical level, performance influencing factor metrics are evaluated, as they relate to interpreting reasons behind why the project is at its current state. Finally, at the strategic level, the future state is predicted based on the current project trajectory.  083-3 4 METHODS, METRICS AND INDICATORS RELATED TO PROJECT PROGRESS AND PERFORMANCE 4.1 Progress Measurement The successful execution of a project hinges on various factors, but one fundamental question that should be answered as the first step in the control process is: What has been accomplished so far? Progress measurement addresses this question at the operational level and serves as the basis for assessing, forecasting, and improving project performance. In this paper, progress measurement is defined as the measurement of outcomes and throughput of a project at a certain point in time. It provides the fundamental information related to “where are we at?” There are several metrics used to measure construction project progress depending on the activities and work items. Although the names and definitions of these metrics could vary for different projects, they belong to one of the major progress measurement methods. The major methods for progress measurement include: units completed, incremental milestone, weighted or equivalent units completed, resource expenditure, and judgment (supervisor opinion) (CII, 1987; AACE International, 2012). Table 1 provides the definitions, applicable conditions, and implementation examples for each method.  Table 1: Definitions, applicable conditions and examples of progress measurement methods Method Definition Application Conditions Implementation Example Units completed quantity surveys or physical measurement of work items homogenous units of work linear-feet of wire pulling, cubic-yard of concrete placed Incremental milestone a percent completion be credited for completion of key incremental tasks one deliverable with multiple activities performed in sequence; output for each subtask cannot be easily measured Equipment installation, alignment and testing (e.g., 50, 70, and 100 percent of completion, respectively) Weighted or equivalent units completed a hybrid of units completed and incremental milestones a major effort involving a long period of time; composed of two or more overlapping subtasks, each with a different unit of work measurement Subtasks in structural steel erection with different units of measure are converted to equivalent tons and then the weighted percent complete is calculated Resource expenditure the percent of the total planned or forecast duration hours, or cost spent for the control account no discrete deliverables or milestones in the work package; a relatively constant level of effort Measurement for tasks such as project management, quality assurance, contract administration, and project controls Judgment the person responsible for the work package estimates the percent complete based on his or her informed judgment only for relatively minor tasks where development of a more discrete method cannot be used Measurement for tasks such as painting, dewatering, constructing support facilities, installing architectural trim, and landscaping Some of these methods are identified with different names in practice. For example, resource expenditure is occasionally labelled as Level of Effort (LOE) or cost ratio. Judgment is also recognized as supervisor opinion. Variations of these five basic progress measurement methods also exist. For example, start/finish is a variation to the incremental milestone method in which the only milestones are starting and finishing. Activities in a construction project such as flushing and cleaning, testing and major rigging 083-4 operations do not have readily definable intermediate milestones. Workers know when the work starts and when it is finished, but not the percentage completion in between. For this type of activities, start/finish is the most appropriate progress measurement method. Each progress measurement method has its own strengths and weaknesses. For example, the units completed method provides the most detailed and accurate progress information. However, it can only be applicable to homogenous units of work. Also, it might take a significant amount of time for data collection. On the other hand, the judgment method takes the least amount of time and effort. However, this subjective approach is highly dependent on the experience of the supervisor, and could be inaccurate and misleading. In general, there is a trade-off relationship between accuracy/consistency and efficiency in progress measurement (Chin et al., 2004). Table 2 summarizes strengths and weaknesses of these progress measurement methods. Table 2: Strengths and weaknesses of existing progress measurement methods (Thomas, 2000) Method Strengths Weaknesses Units completed • Most detailed and accurate • Does not rely on subjective opinions or evaluations • Claimed output can be readily verified • Time for data collection might be lengthy, especially if not applied correctly Incremental milestone • Easy to use • Simple to understand • Long periods may elapse before an intermediate milestone is reached Weighted or equivalent units completed • Detailed and objective • Provides ability to compare and summarize several different subtasks and activity groups • May be inaccurate, especially if there are few items and the activity durations are lengthy • Weighing or equivalency conversions and calculations might be complex, as well as requiring attention Resource expenditure • Greater detail and objectivity than simply estimating how much work was done • Less expensive than counting or measuring the units completed of subtasks • Requires much more effort than simply estimating the percent complete Judgment • Simple • Inexpensive • Quick • Can be very inaccurate and misleading Since one complete project usually includes different types of tasks, multiple progress measurement methods are often used on a project. One limitation in the existing body of knowledge is the lack of an integrated framework which facilitates applying various progress measurement methods systematically in the project. The implementation of different progress measurement methods are usually based on the project manager’s experience instead of objective criteria. The subjectivity may cause problems in providing true measurement of project progress in an efficient and effective way. For example, using the incremental milestone method for measuring the progress of landscaping as a minor component of the project instead of judgment method could improves the accuracy, however, it requires more time and effort to develop the criteria and collect the data. 083-5 4.2 Performance Assessment Progress measurement is essential for successful project execution, although it is not adequate by itself for effective project management and control. It is of vital importance to compare measured progress against the baseline in order to evaluate the current performance of the project. This paper identifies performance assessment as an evaluation of the existing outcomes and results of the project cost, schedule and efficiency at a certain point in the project cycle. It answers the question of “Where should we be at?” There are several metrics and indicators used to assess performance of construction projects based on different approaches. The most commonly used performance assessment method in the construction industry is Earned Value Management (EVM). Due to certain shortcomings of EVM, other methods such as Earned Schedule Method (ESM) and Earned Duration Management (EDM), are offered as viable alternatives. Table 3 explains commonly used methods and their related metrics.  Table 3: Definitions, related metrics and calculations of main performance assessment methods Method Related Metrics Calculations Earned Value Management (EVM) Cost Variance (CV) CV = EV – AC Cost Performance Index (CPI) CPI = EV / AC Schedule Variance (SV) SV = EV – PV Schedule Performance Index (SPI) SPI = EV / PV Earned Schedule Management (ESM) Earned Schedule (ES) * ES(t) = t + EV−PV(t) PV(t+1)−PV(t) Earned Schedule Performance Index (SPI (t)) SPI(t) = ES(t) Actual Duration Earned Duration Management (EDM) Earned Duration (ED) ** EDi = BPDi × APIi Earned Duration Index (EDIi) *** EDIi =  EDi  PDi  Critical Chain Project Management (CCPM) Critical Chain Completed (CCC) number of days of critical chain work completed total number of days on critical chain Buffer Consumption Rate (BCR) percentage of project buffer consumed percentage of critical chain completed * “t” stands for the time status that ES is calculated for, whereas “t+1” is for the following time period. ** BPDi stands for Baseline Planned Duration of scheduled activity I, whereas APIi is the Activity Progress Index, measured schedule progress of an activity through whichever method is preferred in the project. *** PDi is for Planned Duration of an activity i EVM methodology uses metrics to answer questions such as: what are the actual and planned costs of a project at a given point in time in relation to its earned value (Vanhoucke 2009). At the operational level, EVM captures three major values namely Actual Cost (AC), Planned Value (PV) and Earned Value (EV) (Fleming and Koppelman, 2000). These values represent the measured progress in terms of cost. One significant drawback of EVM is its inability to capture the schedule performance, especially over the last third of the project (Lipke 2009). In order to improve the schedule assessment ability of EVM, ESM was proposed (Lipke, 2003). ESM relies on the same operational-level performance indicators as EVM (i.e. PV, AC and EV); however it uses a new set of time-based performance assessment metrics to identify Earned Schedule (ES) as the time at which the amount of earned value should have been accomplished. Although ESM solves some limitations of time management in EVM, it still uses cost as a proxy to measure schedule performance of a project. Therefore, when a disparity exists between time and cost 083-6 profiles of a project, ESM fails to capture the distinctive schedule behavior of the project. To address this limitation of EVM and ESM, Earned Duration Management (EDM) was developed, which decouples schedule and cost performance measures. EDM also identifies a number of indices to measure progress and performance of schedule and cost (Khamooshi and Golafshani 2013).  EVM, ESM and EDM treat delays on critical and none-critical paths equivalently. Also the correlations between time and cost of different activities are not taken into consideration in any of these methods. In terms of performance forecasting, the adaptive behaviors of the project managers is not considered. And finally the assessment of the project in all these methods is limited to time and cost and other performance measures such as quality of the final products are not part of the analysis (Hall 2012; Hazır 2014). Critical Chain Project Management (CCPM), an emerging approach for performance assessment, focuses on resource utilization and minimizing idle capacity (Goldratt, 1997; Leach, 2000). However, issues related to the stability of the critical chain and the network structure, as well as resource efficiencies and multitasking are considered problematic aspects of CCPM.  For different project types and phases, various performance assessment metrics and indicators should be utilized, sometimes in combination with one another. However, one general weakness of current performance assessment methodologies is lack of consistency and implementation guidelines for metrics throughout the construction industry, which eventually diminishes reliability of metrics and indicators. 4.3 Performance Influencing Factor Metrics Performance influencing factor metrics are characteristics of the project or project organizations that influence the performance outcomes of the project (e.g., scope change, risk management, and resource management by answering the question “Why are we here?” Therefore, these metrics explain reasons behind the current state of the project by allowing managers to (1) know where to look if a problem related to cost, schedule or efficiency performance occurs, and (2) interpret performance assessment and forecasting more accurately and develop proactive response plans and corrective actions. Lack of unified guidelines to tackle problems as they relate to project progress and performance through influencing factor metrics not only leads to inconsistent and unreliable project information, but also makes it difficult to objectively compare one project to another. One exception to this problem is related to safety metrics. Almost every project uses nearly the same metrics and indicators for safety: OSHA Recordable Incident Rate (ORIR), Days Away, Restrictions and Transfers (DART) rate, number of near-miss incidents, people based safety observation, new hire count-vs-recordable injury rate, Total recordable incident rate (TRIR), Lost time incident rate (LTIR) and Total severity rate (TSR) (NRC, 2005; COAA, 2007). In addition to being related to life and death situations, safety is also highly regulated through guidelines. This highlights the importance and necessity of having established guidelines for using other common performance influencing factor metrics in the construction industry. 4.4 Performance Forecasting Performance forecasting uses current state performance information to predict the future outcomes and results. It basically answers the question of “Where will we be at?” Table 4 lists performance forecasting metrics for cost and schedule estimations.  Earned Value Management (EVM) offers several useful metrics to forecast what the expected cost values are for the remainder of the project (Vanhoucke, 2009). These metrics predict the final cost of the project based on the actual performance at the time of assessment and the planned cost of the remaining work. However, these projections are highly dependent on consistency of performance assessment metrics used and management’s opinion on some forecasted elements. (Fleming and Koppelman, 2000). The EVM methodology consists of metrics for schedule performance forecasting as well, similar to the cost metrics described above. However, similar problems can be observed with schedule performance forecasting metrics of EVM. At this point, ESM offers alternative schedule forecasting metrics (Lipke 2009). The same pattern with performance assessment metrics follows here; even though ESM metrics generate better results for schedule, they are yet to offer a robust forecasting alternative due to using cost as a proxy to estimate schedule of a project. 083-7 Table 4: Forecasting metrics, definitions and related methods  (Fleming and Koppelman, 2000; Vanhoucke, 2009; Lipke, 2009) Metric Definition Calculation Cost/Schedule Method Estimate at Completion (EAC)  manager’s projection of total cost of the project at the end of the project EAC=BAC/CPI Cost EVM Estimate to Completion (ETC) estimated cost required to complete the remainder of the project ETC=EAC-AC Cost EVM Variance at Completion (VAC) variance on the total budget at the end of the project VAC= BAC-EAC Cost EVM To Complete Performance Index (TCPI) future required cost efficiency needed to achieve a target BAC (budget at complete) or EAC (estimate at complete) TCPI(BAC)= (BAC-EV)/(BAC-AC) or TCPI(EAC)= (BAC-EV)/(EAC-AC) Cost EVM Independent Estimate at Completion (IEAC) a metric to project total cost using the performance to date to project overall performance. IEAC= AC+(BAC-EV)/CPI Cost EVM Estimate at Completion (time) [EAC(t)] manager’s projection of total duration of the project at the end of the project EAC(t)=PD/SPI(t) Schedule ESM Estimate to Completion (time) [ETC(t)] estimated time required to complete the remainder of the project ETC(t)=EAC(t)-AT Schedule ESM Variance at Completion (time) [VAC(t)] predicted variance on the total schedule at the end of the project VAC= PD – EAC(t) Schedule ESM To Complete Schedule Performance Index (TSPI) required time efficiency needed to achieve a target PD or EAC(t). (The TSPI provides a projection of the anticipated performance required to achieve either the PD or the EAC(t)) TSPI(PD)= (PD-ES)/(PD-AT) or TSPI[EAC(t)]= (PD-ES)/(EAC(t)-AT) Schedule ESM Independent Estimate at Completion (time) [IEAC(t)] an independent second opinion of the final project duration IEAC= AT+(PD-ES)/SPI(t) Schedule ESM BAC: Budget at Completion PD: Planned Duration AT: Actual Time  5 GAP ANALYSIS AND DISCUSSION In this study, mind mapping was used to abstract the tacit knowledge of the industry panel regarding two central topics: (1) the problems related to the existing project progress and performance assessment metrics and methods, and (2) the required improvements. In the first step of mind mapping, each individual on the industry panel provided his/her perspective regarding the central topic. The information 083-8 was collected and visualized into a preliminary mind map with a tree structure using Free Mind software. In the second step, the preliminary mind map was presented to the industry panel. The panel was asked to cluster the branches in the mind map into categories of problems and required improvements related to the existing metrics and methods. This step was concluded when the panel reached a consensus. Mind map in Figure 2 clearly depicts problematic areas and needed improvement identified through this study. Review of the current knowledge and practice revealed that there is no consistent set of progress measurement metrics and performance assessment indicators. Due to this lack of consistency, it is difficult to have a true comparison of one project performance to another. Also, there is potential for misrepresentation of progress and performance information, caused by unreliability of metrics and indicators. The existing use of metrics and indicators vary across the industry and depend on the level of the experience of project personnel. There is also limited understanding regarding the importance and effects of performance influencing factor metrics. Hence, there is a need for guidelines identifying what core metrics and key performance indicators (KPIs) should be used consistently on construction projects. Development of such guidelines can help reduce subjectivity in the metrics. Furthermore, in order to improve progress and performance assessment, identification of recommended practices for effective interpretation is necessary. Finally, understanding the significant impacts of performance influencing factor metrics and indicators to progress and performance can provide greater insight for performance improvement and foresight for forecasting project outcomes.  Figure 2: Mind map for identified problems and improvement areas 6 CONCLUSIONS AND RECOMMENDATIONS The purpose of this paper was to review common project progress, performance assessment and forecasting methods used in the construction industry. A framework was presented describing the operational, tactical, and strategic levels of project controls. After reviewing the literature and current practices in project controls, it is apparent that construction industry needs certain actions to improve project progress and performance assessment. This paper recommends to: (1) identify core metrics and key indicators, (2) develop guidelines for improving the reliability of metrics and indicators, and (3) establish recommended practices for interpreting metrics and indicators, forecasting outcomes, and responding to variance to plan as they relate to progress and performance assessment. In order to satisfy the industry needs and fulfill recommended courses of action, CII Research Team 322 (RT-322) is working on a rigorous three-step data collection and validation plan: (1) a web-based survey that uses the findings from the gap analysis presented in this paper to identify core methods, metrics and indicators, (2) case studies to understand the significance of core metrics and how reliability of these metrics can be improved and (3) Delphi method to validate findings using expert opinions.  083-9 Problems and suggested improvements identified through this study, can inform future research on applied project controls. Consequentially, real-life applications of these findings have the potential to advance many construction industry practices. Acknowledgements This research has been supported by the Construction Industry Institute. The opinions expressed in the paper represent those of the authors and not necessarily those of the Construction Industry Institute. References Association for Advanced Cost Engineering International (AACE International) 2012. Total Cost Management Framework: An Integrated Approach to Portfolio, Program, and Project Management. Revised 1st ed. AACE International, Morgantown, WV, USA. Chin, S., Yoon, S., Kim, Y. S., Jung, Y., Park, S. C., and Chung, M. 2004. A project Progress Measurement and Management System. 21st International Symposium on Automation and Robotics in Construction, Jeju, South Korea. Construction Industry Institute (CII). 1987. Research Summary 6-5: Project Control for Construction. CII, Austin, TX, USA. Construction Industry Institute (CII). 2012. Performance Assessment 2012 Edition. CII, Austin, TX, USA. Construction Industry Institute (CII). 2014. Performance Assessment 2014 Edition. CII, Austin, TX, USA. Construction Owners Association of America (COAA). 2007. CII/COAA Benchmarking & Metrics: Performance Metric Formulas and Definitions. COAA, Austell, GA, USA. Fleming, Q., and Koppelman, J. 2000. Earned value project management 2nd ed., Project Management Institute, Newton Square, PA, USA. Goldratt, E. 1997. Critical chain. 1st ed., North River Press, Great Barrington, MA, USA. Hall, N. G. 2012. Project management: Recent developments and research opportunities. Journal of Systems Science and Systems Engineering, 21(2): 129-143. Hazır, Ö. 2014. A review of analytical models, approaches and decision support tools in project monitoring and control. International Journal of Project Management. Kenley, Russell, and Olli Seppänen. 2010. Location-based management for construction: Planning, scheduling and control. 1st ed., Spon Press, London, UK. Khamooshi, H., and Golafshani, H. 2013. EDM: Earned Duration Management, a new approach to schedule performance management and measurement. International Journal of Project Management, 32(6), 1019-1041. Leach, L. 2000. Critical chain project management. 2nd ed., Artech House, Boston, MA, USA. Lipke, W. H. 2003. Schedule is different. Measurable News, 2003(March), 10-15. Lipke, W. H. 2009. Earned schedule. Lulu Publishing, Raleigh, NC, USA. National Research Council (NRC). 2005. Measuring Performance and Benchmarking Project Management at the Department of Energy. The National Academies Press, Washington, DC, USA. Project Management Institute (PMI). 2013. A guide to the project management body of knowledge: (PMBOK® guide). 5th ed., Project Management Institute, Newtown Square, PA, USA. Thomas, H. 2000. Principles of Construction Labor Productivity Measurement and Processing: Final Report. University Park, PA, USA: Pennsylvania Transportation Institute, Pennsylvania State University. Vandevoorde, S., and Vanhoucke, M. 2006. A comparison of different project duration forecasting methods using earned value metrics. International Journal of Project Management, 24(4): 289-302. Vanhoucke, M. 2009. Measuring time: Improving project performance using earned value management. Springer, New York, NY, USA.   083-10  A REVIEW OF THE CURRENT KNOWLEDGE AND PRACTICE RELATED TO PROJECT PROGRESS AND PERFORMANCE ASSESSMENT Resulali E. Orgut1, Jin Zhu2, Mostafa Batouli2, Ali Mostafavi3 and Edward J. Jaselskis1  1 Department Of Civil, Construction And Environmental Engineering, North Carolina State University, USA 2 Department of Civil and Environmental Engineering, Florida International University, USA 3 OHL School of Construction, College of Engineering and Computing, Florida International University, USA  2015 Canadian Society for Civil Engineering Construction Specialty Conference 1 Problem Definition 1 Performance Inefficiency is a major challenge in construction industry Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Only 1 out of 20 Construction Projects Met Authorized Cost and Schedule (CII, 2012) 1 Problem Definition 2 One of the major reasons for the low efficiency in construction project performance is the incapability of the existing metrics and methods in providing a “true” measure of project progress and performance. If you cannot measure it, you cannot manage it. -Peter Drucker Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 1 Problem Definition 3 Essential questions that CII RT-322 wants to answer: How can we identify and better interpret the “true” measures of project progress and performance?  What are the more useful parameters and indicators? Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 2 Research Objectives 4 Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Approach for defining research objectives: Form an industry panel Interview and survey with individuals in the panel Preliminary mind map  Consensus  Final mind map  Yes Mind map review in the industry panel No Revise 2 Research Objectives 5 Mind map for identified problems and needs for improvement Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 2 Research Objectives 6 •  Objective #2: Identify guidelines for improving the reliability of metrics and indicators. •  Objective #3: Identify recommended practices for interpreting metrics and indicators. core metrics and key indicators guidelines for improving reliability tolerances for variance to plan for these metrics Overarching objective Improve project progress and performance assessment •  Objective #1: Identify the core metrics and key indicators. Objectives of CII RT-322 Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 3 Research Framework 7 Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Where are we at? Where should we be at? Where will be at? 4 Gap Analysis 8 Identifying relevant work Collecting and summarizing information Conducting gap analysis Three-step methodology for gap analysis Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 4 Gap Analysis 9 Summary for Literature Review Source Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 38	  26	  11	  41	  0	   5	   10	   15	   20	   25	   30	   35	   40	   45	  CII	  Reports	  Professional	  Organiza<on	  Reports	  Government	  Agency	  Reports	  Scholarly	  Ar<cles	  CII	  Reports	   Professional	  Organiza<on	  Reports	   Government	  Agency	  Reports	   Scholarly	  Ar<cles	  Series1	   38	   26	   11	   41	  33%	  22%	  9%	  35%	  CII	  Reports	  Professional	  Organiza<on	  Reports	  Government	  Agency	  Reports	  Scholarly	  Ar<cles	  Directly Related •  RT 244 – Global Project Control and Management Systems (2006–09) •  RT 220 – Leading Indicators to Project Outcome (2004-06) •  RT 107 – Predictive Tools (1993-95) •  RT 6 – Cost/Schedule Controls (1984-88) •  SD 91 – Early Warning Signs of Project Changes (1993) •  SD 80 – Project Performance Modeling: A Methodology for Evaluating Project Execution Strategies (1992) •  SD 24 – Cost and Schedule Control in Industrial Construction (1986) Indirectly Related •  RT 193 – Radical Reduction in Project Cycle Time (2002-04) •  RT 172 – Supply Chain Management Concepts (2000-02) •  RT 153 – Effects of Field Rework (1998-2001) •  RT 124 – Re-Engineering the EPC Process (1995-97) •  RT 6 – Cost/Schedule Controls (1984-88) •  DOE (Department of Energy) •  DOT (Dep rtment of Transportation •  DOD (Association for Advanced Cost Engineering Internati nal) •  ANSI (American National Standards Instit te) •  NASA (American Society of Civil Engineers) •  NRC (National Research Council of the National Academies) •  PMI (Project Management Institute) •  AACEI (Association for Advanced Cost Engineering International) •  CPM (College of P rf rmance Management) •  ASCE (American Society of Civil Engineers)   C AA (Construction OwnerAsso iation of America) •  IPA (Independent Pr ject Analysis) •  IPMA (International Project Management Association) •  International Journal of Project Management •  Journal of M nagement in Engine ring •  Automation in Construction •  Journal of Building and Environment •  European Journal of Operational Research 4 10 Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Gap Analysis 4 Gap Analysis: Progress Measurement 11 Method Metric Examples Units completed •  linear-feet of wire pulling, cubic-yard of concrete placed Incremental milestone •  Equipment installation, alignment and testing (e.g., 50, 70, and 100 percent of completion, respectively) Weighted or equivalent units completed •  Subtasks in structural steel erection with different units of measure are converted to equivalent tons and then the weighted percent complete is calculated Resource expenditure •  Measurement for tasks such as project management, quality assurance, contract administration, and project controls Judgment •  Measurement for tasks such as painting, dewatering, constructing support facilities, installing architectural trim, and landscaping Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Methods and Metrics for Progress Measurement 4 Gap Analysis: Progress Measurement 12 Method Strengths Weaknesses Units completed •  Most detailed and accurate •  Does not rely on subjective opinions  •  Time for data collection might be lengthy Incremental milestone •  Easy to use •  Simple to understand •  Long periods may elapse before an intermediate milestone is reached Weighted or equivalent units completed •  Detailed and objective •  ability to compare and summarize different subtasks •  May be inaccurate, especially if there are few items and the activity durations are lengthy •  Weighing or equivalency conversions and calculations might be complex Resource expenditure •  Greater detail and objectivity •  Less expensive than measuring the units completed of subtasks •  Requires much more effort than simply estimating the percent complete Judgment •  Simple, quick and Inexpensive •  Can be very inaccurate and misleading Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Ø  Guidelines for selecting the most appropriate progress measurement metrics for different conditions and requirements Strengths, Weaknesses, and Needs for Improvement related to Progress Measurement Metrics 4 Gap Analysis: Performance Assessment 13 Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Conventional Performance Assessment Approach: Earned Value Management “I have one interesting observation I found during an audit for a few projects last month. Looking at the S-curves, engineering looks very good, procurement looks bad, and construction looks very bad. However, my root-cause analysis was that most of the blame should go to engineering focusing on earning any credit not earning the right credit.” Quote by an experienced project control manager 4 Gap Analysis: Performance Assessment 14 Method Related Metrics Calculations Earned Value Management (EVM) Cost Variance (CV) CV = EV – AC Cost Performance Index (CPI) CPI = EV / AC Schedule Variance (SV) SV = EV – PV Schedule Performance Index (SPI) SPI = EV / PV Earned Schedule Management (ESM) Earned Schedule (ES)  ES(t) = t + EV−PV(t) PV(t+1)−PV(t) Earned Schedule Performance Index (SPI (t)) SPI(t) = ES(t) Actual Duration Earned Duration Management (EDM) Earned Duration (ED)                       EDi = BPDi × APIi Earned Duration Index (EDIi) EDIi =  EDi  PDi  Critical Chain Project Management (CCPM) Critical Chain Completed (CCC) number of days of critical chain work completed total number of days on critical chain Buffer Consumption Rate (BCR) percentage of project buffer consumed percentage of critical chain completed Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Methods and Metrics for Performance Assessment 4 15 Method Strengths Weaknesses Earned Value Management (EVM)  •  represent the measured progress in terms of cost •  inability to capture the schedule performance, especially over the last third of the project Earned Schedule Management (ESM)  •  addresses limitations of EVM using a new set of time-based performance assessment metrics  •  still uses cost as a proxy to measure schedule performance  •  when a disparity exists between time and cost profiles of a project, ESM fails to capture the distinctive schedule behavior Earned Duration Management (EDM) •  decouples schedule and cost performance measures •  treat delays on critical and none-critical paths equivalently •  the correlations between time and cost of different activities are not taken into consideration •  assessment of the project is limited to time and cost Critical Chain Project Management (CCPM) •  focuses on resource utilization and minimizing idle capacity •  Incapable of capturing the stability of the critical chain, the network structure, as well as resource efficiencies Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Gap Analysis: Performance Assessment Strengths, Weaknesses, and Needs for Improvement related to Performance Assessment Metrics Ø  What metrics can tell us the true insight on project performance?  4 16 Gap Analysis: Performance Forecasting Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Method Related Metrics Calculations Earned Value Management (EVM) Estimate at Completion (EAC) EAC=BAC/CPI Estimate to Completion (ETC) ETC=EAC-AC Variance at Completion (VAC) VAC= BAC-EAC To Complete Performance Index (TCPI) TCPI(BAC)=(BAC-EV)/(BAC-AC) or TCPI(EAC)=(BAC-EV)/(EAC-AC) Independent Estimate at Completion (IEAC) IEAC=AC+(BAC-EV)/CPI Earned Schedule Management (ESM) Estimate at Completion (time) [EAC(t)] EAC(t)=PD/SPI(t) Estimate to Completion (time) [ETC(t)] ETC(t)=EAC(t)-AT Variance at Completion (time) [VAC(t)] VAC= PD – EAC(t) To Complete Schedule Performance Index (TSPI) TSPI(PD)=(PD-ES)/(PD-AT) or TSPI[EAC(t)]=(PD-ES)/(EAC(t)-AT) Independent Estimate at Completion (time) [IEAC(t)] IEAC=AT+(PD-ES)/SPI(t) Methods and Metrics for Performance Forecasting 4 17 Gap Analysis: Performance Forecasting Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Method Strengths Weaknesses Earned Value Management (EVM)  •  forecast the final cost of the project based on the actual performance at the time of assessment and the planned cost of the remaining work •  the projections are highly dependent on consistency of performance assessment metrics used and management’s opinion on some forecasted elements Earned Schedule Management (ESM)  •  offers alternative schedule forecasting metrics  •  Incapable of offering a robust forecasting due to using cost as a proxy to estimate schedule of a project Ø  What performance forecasting metrics can provide better prediction for future performance?  Strengths, Weaknesses, and Needs for Improvement related to Performance Forecasting Metrics 4 18 Gap Analysis: Influencing Factor Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Influencing Factor Metrics Examples Scope/Change management Number of pending change notices, timeliness of responses to RFIs Risk  Risk management plan update frequency Resource Team turnover rate, equipment utilization Quality Percent of rework, error/omission related design change notices (DCNs) Safety OSHA recordable incident rate, DART rate Ø  What are the metrics that really matter?  Metrics related to Influencing Factors 4 19 Potential Innovative Metrics from Gap Analysis Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work Ø  Goal: Identify metrics that matter   Examples of potential innovative metrics: –  Number of sequence activities –  SPI(t) (based on earned schedule) –  Total float consumption index –  Number of backlog tasks –  Critical path length index  5 Ongoing Work 20 • Understand	  exis<ng	  prac<ce	  and	  iden<fy	  knowledge	  Gaps	  Literature	  Review	  • Inves<gate	  metrics	  and	  methods	  used	  in	  projects	  Survey	  • Further	  iden<fy	  the	  importance	  and	  rela<onships	  between	  metrics	  Case	  Study	  • Analyze	  and	  evaluate	  metrics	  through	  an	  expert	  panel	  Delphi	  • Develop	  recommended	  prac<ces	  for	  selec<ng	  and	  interpre<ng	  metrics	  	  Implementa<on	  Tool	  Development	  Problem Definition Research Objectives Research Framework Gap Analysis Ongoing Work 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52660.1-0076386/manifest

Comment

Related Items