7.1 INTRODUCTION

7.1.1 Purpose. The analysis section of the Criteria is of particular importance because it is intended to ensure that contract data may be used to generate information to measure contract performance.

7.1.2 The Criteria, per se, do not require the submission of data or reports from the contractor to the customer. The criteria only set forth characteristics which contractors' systems must possess, and specify the type of data which should be derived from them.

7.1.3 Data Elements. Five basic data elements are identified in the criteria: ACWP, BCWS, BCWP, Budget at Completion (BAC) and Estimate at Completion (EAC). ACWP was discussed previously under Accounting and Indirect Cost Management (CHAPTERS 5 and 6). BAC was discussed in CHAPTER 4: Planning and Budgeting. This section discusses BCWS, BCWP, BAC, EAC, and analysis of variances resulting from comparisons of these five basic elements.

7.2 BUDGETED COST FOR WORK SCHEDULED

7.2.1 For the total contract, BCWS is normally the negotiated contract cost, or the contractor's nominated equivalent in a Fixed Price contract (see 4.7.2), plus the estimated cost of authorised but unpriced work, less any management reserve. It is time-phased by the assignment of budgets to scheduled increments of work.

7.2.2 Determination of BCWS. For any given time period, BCWS is determined at the cost account level by totalling the budgets for all work packages scheduled to be completed, plus the budget for the portion of in-process work (open work packages) scheduled to be accomplished, plus the budgets for LOE and apportioned effort scheduled to be completed during that period.

7.2.3 Anticipated Learning. A contractor must utilise anticipated learning when developing the time-phased BCWS. Any recognised method used to apply learning is acceptable as long as the BCWS is established to represent as closely as possible the expected ACWP that will be charged to the cost account/work package.

7.2.4 Consideration of BCWP. In developing BCWS, consideration should be given to the methods planned for taking credit for BCWP (refer to Budgeted Cost for Work Performed, below).

7.3 BUDGETED COST FOR WORK PERFORMED

7.3.1 BCWP consists of the budgeted costs for all work actually accomplished during a given time period. At the cost account level, BCWP is determined by totalling budgets for work packages actually completed, plus the budget applicable to the completed in-process work within open work packages, plus the budgets for LOE and apportioned effort associated with completed work.

7.3.2 Determination of BCWP. The major difficulty encountered in the determination of BCWP is the evaluation of in-process work (work packages which have been started but have not been completed at the time of cut-off for the report). As discussed in CHAPTER 3, the use of short-span work packages or establishment of discrete value milestones within work packages will significantly reduce the work-in-process evaluation problem and procedures used will vary depending on work package length. For example, some contractors prefer to take no BCWP credit for a short-term work package until it is completed, others take credit for fifty percent of the work package budget when it starts and the remaining fifty percent at completion. Some contractors use formulae which approximate the time-phasing of the effort, others use earned standards, while still others prefer to make physical assessments of work completed to determine the applicable budget earned. For longer work packages, many contractors use discrete milestones with pre-established budget or progress values to measure work performance.

7.3.3 The criteria do not specify any particular method as the technique used will largely depend on work package content, size, and duration. However, the use of arbitrary formulae, as described above, should be limited to very short work packages. At all times BCWP must be computed using the same labour, overhead, and other rates as for BCWS.

7.4 DATA ANALYSIS

7.4.1 BCWS & BCWP. Comparisons of BCWS with BCWP relate work completed to work scheduled during a given period of time. While this provides a valuable indication of schedule status, in terms of dollars worth of work accomplished, it may not clearly indicate whether or not scheduled milestones are being met since some work may have been performed out of sequence or ahead of schedule. A formal time-phased scheduling system must, therefore, provide the means of determining the status of specific activities and milestones.

7.4.2 BAC & EAC. Comparisons of BAC with EAC are required internally at the cost account level and provide estimated variances expected at the completion of the contract. Cost Account Managers need to be constantly alert to circumstances which will cause the estimate at completion and, therefore, the variance at completion, to change. Such changes must be reported monthly.

7.4.3 BCWP & ACWP. Comparisons of BCWP and ACWP will clearly show whether completed work has cost more or less than was planned for that work. Analysis of these differences should reveal the factors contributing to the variances, such as poor initial estimate for the task, technical difficulties requiring application of additional resources, the cost of labour or materials different than planned, personnel efficiency different than planned, or a combination of these or other reasons.

7.4.4 Cost Account Level Comparisons. Comparisons of BCWP with BCWS and with ACWP are required at the cost account level. Cost accounts consist of an aggregation of work packages which are the responsibility of a single organisation. Managerial authority and responsibility for corrective action should exist at this point making the cost account a key management control point in the system. It is important that the performance measurement baseline be maintained at this level since comparisons of planned versus actual performance are of little value if the measurement base is subject to uncontrolled fluctuation and change. Since higher level management information consists of direct summaries of the results of such comparisons, there is less need for further calculations at high levels to determine program status.

7.4.5 Subcontractor - Performance Reporting. When a sub-contractor is required to comply with the Criteria and also to provide a Cost Performance Report (CPR), sub-contractor data should be capable of summarisation by the prime contractor for performance measurement purposes. If a sub-contractor is not required to comply with the Criteria, the prime contractor should establish procedures which tie the planned and actual accomplishment of the sub­contractor to valid indicators, such as the proposed payment schedule or completion of identified work segments. As a general guide, if a sub-contract (including fixed or variable price) has a value of $1M or more, has at least six months between the beginning of work and the first significant delivery, and is scheduled to receive progress payments, then the prime contractor would normally be expected to measure in-process performance of that sub-contractor prior to receipt by the prime of the product(s) being produced by that sub-contractor.

7.5 DATA SUMMARISATION

7.5.1 Variance Summary Levels. BCWS, BCWP, ACWP and associated variances should be summarised directly into both the CWBS and organisational structure from the appropriate level (cost account or below) to provide both contract status and organisational performance at all levels of management. Because favourable variances in some areas are offset by unfavourable variances in other areas, higher level managers will normally see only the most significant variances at their own level. However, the accumulation of many small variances which may be adding up to a large overall cost problem not attributable to any single major difficulty will also be evident. Similarly, summation of positive and negative variances may cloud the true picture. The same is true of the information to be reported to the Project Authority.

7.5.2 If required by the contract, the CPR provides data to the Project Authority at a summary level as specified in the contract. Information as specified in the contract will also be reported at the total contract level for major functional categories which reflect the contractor's organisational structure, such as engineering, manufacturing, subcontractor, etc. While only problems having significant cost or schedule impact on the contract will appear on this report due to the wash-out effect of favourable and unfavourable variances, all significant variances should be explained in the problem analysis portion of the report. The reason for reporting only summary level information to the Project Authority is that as long as contract performance is proceeding according to plan, there should be no need to report additional detail. If actual performance begins to deviate from the plan, the contractor's system should provide the capability for tracing the variances to their source to isolate the causes of the deviation.

7.5.3 It should be recognised that this method of performance measurement is only one of the management tools available to contractors and to the Project Authority. Many major problems are disclosed through methods other than monthly cost reports. For example, failure to meet closely monitored schedule, manpower, or technical achievement plans and requirements should promptly alert contractor management that a problem exists. However, the contractor's internal cost performance reports and the CPRs to the Project Authority should indicate the overall cost and schedule impacts of such problems on the contractor.

7.6 SIGNIFICANT VARIANCES

7.6.1 Variance Thresholds. It is important to establish reasonable variance thresholds that will cause problem analyses and narrative reports to be prepared. Careful selection of these thresholds is necessary to prevent unnecessary work associated with preparing an excessive number of written analyses. The analysis of every cost and schedule variance is usually unnecessary and unproductive. Therefore, the contractor should establish internal cost and schedule variance thresholds and analyse only those variances which are significant: that is, those which exceed the thresholds. These thresholds may vary with respect to the level of the CWBS element, the level of the organisational element, the amount of work remaining, etc. It is essential that these internal variance thresholds be so established that all significant variances will be analysed while at the same time avoiding an excessive number of variance analyses.

7.6.2 Generally, thresholds are established requiring a variance analysis for any cost or schedule variance that exceeds a certain percentage of BCWS or BCWP and exceeds an established dollar minimum (for example, __% of cumulative BCWS, and $________). When initially establishing the thresholds, it may be advisable to provide for tightening these thresholds as the contract progresses, in view of the increased cumulative values of BCWS, BCWP and ACWP.

7.6.3 Another approach is to establish the thresholds as a percentage of the Budget at Completion (BAC) rather than as a percentage of BCWS or BCWP (for example, 100(BCWP­ACWP) divided by BAC for cost variance threshold; 100(BCWP­BCWS) divided by BAC for schedule variance threshold). This results in a threshold which becomes a progressively smaller percentage of cumulative BCWS and BCWP as the contract progresses. Since this type of variance threshold may be relatively loose early in the contract, the threshold for early variances may be limited by adding a threshold based on a percentage of cumulative BCWS (for example, _____% of BAC or _____% of cumulative BCWS, whichever is less).

7.6.4 Variance Analysis. Whenever a variance exceeds the prescribed threshold, analysis and explanation are required. Consideration may be given to establishing higher thresholds for underrun or ahead-of-schedule conditions to minimise the generation of analyses and explanations of variances that do not have potential for adverse impact.

7.6.5 The selection of thresholds should avoid the explanation of variances that are unimportant, while not missing variances that are significant. It should be recognised that no particular approach or set of thresholds is "best" for all circumstances. It may be appropriate to use different thresholds for different levels of management, for different organisational elements, and for reporting to the customer. Whenever, during the performance of a contract, it becomes apparent that the thresholds are no longer appropriate, they should be revised. Too few or too many variance analyses in relation to the performance status of the contract may indicate improperly set thresholds which require adjustment.

7.7 TECHNICAL ACHIEVEMENT

7.7.1 Technical versus Cost & Schedule Accomplishment. The key to meaningful correlation of technical achievement with cost and schedule control is the proper definition, organisation and supervision of effort. If a CWBS matches the specification tree and also reflects the manner in which the contractor actually does the work, the problem of correlation is greatly simplified. In correlating cost, schedule and technical achievement, it is apparent that unfavourable cost or schedule conditions are usually caused by technical difficulties. Therefore, quantitative information as to technical status is desirable and should be supplemented by narrative reports.

7.7.2 As work on a contract progresses, the contractor determines the adequacy and quality of the work performed by making inspections, tests, and other types of technical measurements. If the results are satisfactory and no corrective action is required, the work is then allowed to proceed further. If, on the other hand, deficiencies are found, the contractor considers various alternatives for corrective action; for example, redesign, scrap and remake, rework, etc. When considering these alternatives, the impact on cost and schedule must be weighed in addition to the technical considerations. After one of the alternatives is selected as the desired course of action to correct the deficiencies, it may be necessary to plan the additional work in terms of new work packages or additions to existing unopened work packages and to change the schedules affected. In some cases, the contractor may subject to appropriate controls provide additional budget to the responsible organisation. Thus, there is a close relationship between technical achievement (that is, inspection and test) and its impact on cost and schedule.

7.8 ESTIMATE AT COMPLETION

7.8.1 EAC. The Criteria require the contractor periodically (normally at least annually) to develop comprehensive estimates of costs at contract completion. In developing the EAC, the contractor should use all available information to arrive at the best possible estimate of costs for all authorised contract effort.

7.8.2 The procedure should be systematically and consistently used from period to period, with adequate consideration given to performance to date. The EAC procedures should provide for the formulation or updating of an estimate of cost to completion, time-phased to the extent necessary to reflect projected rates. This is necessary to ensure that resource requirements are realistic and phased in accordance with projected performance. In addition, the estimate at completion should be examined monthly for accuracy as a routine cost management function and should be updated, as warranted. Such an examination is required to ensure reliable and timely EAC status reporting consistent with contractor reporting requirements.

7.8.3 The EAC process focuses on the Cost Account Manager, but it should also provide for regular input from functional and other managers which may impact contract performance.

7.8.4 Both the comprehensive EACs and the monthly updates are essential as a basis for management decision making by both the contractor and the Project Authority. While no specific time period for developing the comprehensive EAC is established by the Criteria, it is expected that a comprehensive estimate will be prepared on an annual basis as a minimum, usually in support of the business plan update, or more frequently when performance indicates that the current estimate is invalid. The EAC submitted to the Project Authority should be reconcilable with internal cost reports and any amendment to the payment schedule (funds) reported to the Project Authority. EACs should be established without regard for contract ceilings.

7.8.5 During CSCSC reviews the Review Team evaluation of a contractor's EAC process will concentrate on the adequacy of the EAC policy, procedures, and methods, and the consistency with which they are followed. EAC policy and procedures, and methods should be described in the System Description. The EAC procedures must be responsive to, and comply with, all the elements required by the ACSIG. Checklist Item IV.6 (see ANNEX D).

7.8.6 If the accuracy of the quantified results becomes a concern during the review process, the system description and procedures, and the way they are used, will be re-checked to ascertain that they are adequate. If they are adequate, then the accuracy concerns will be documented and forwarded promptly to the Project Authority for detailed examination, and will be included in the official CSCSC review report. The completion of the CSCSC review will not be impacted, unless non compliance with or inadequacy of the procedures is an issue.

7.8.7 Subcontract EAC Reporting. When a subcontractor's EAC is in excess of the subcontractor's fixed-price or ceiling in an incentive contract, the data portion of a prime contractor's CPR should show what the performance of the subcontract(s) will cost the prime. This means the CPR Formats 1& 2 (see ANNEX G) will include in the prime's EAC and the subcontract(s) EAC up to the contract price for firm price contract or ceiling price for an incentive contract. When available, the prime's best estimate of the subcontractor actual final cost (if it is over that contracted) will be included in the CPR Format 5 Narrative Analysis. If there is a CSCSC flowdown or CSSR requirement on a subcontract, and if the prime wants the subcontractor's actual EAC on a firm or incentive contract, the prime should negotiate this with the subcontractor.

7.9 CRITERIA

7.9.1 General. The remainder of this CHAPTER is devoted to discussion of the Analysis Criteria. The objective is to clarify the requirements of the Criteria as an aid to interpretation for both Review Teams and contractors. Criterion Analysis 2 deals with Indirect Costs and is discussed in CHAPTER 6. Further amplification is found in the Evaluation/Documentation Review Checklist in ANNEX D which contains check-list questions used by Review Teams to evaluate performance measurement systems.

IDENTIFY AT THE COST ACCOUNT LEVEL ON A MONTHLY BASIS USING DATA FROM, OR RECONCILABLE WITH, THE ACCOUNTING SYSTEM: BCWS AND BCWP; BCWP AND APPLIED (ACTUAL WHERE APPROPRIATE) DIRECT COSTS FOR THE SAME WORK; VARIANCES RESULTING FROM THE ABOVE COMPARISONS CLASSIFIED IN TERMS OF LABOUR, MATERIAL, OR OTHER APPROPRIATE ELEMENTS, TOGETHER WITH THE REASONS FOR SIGNIFICANT VARIANCES.

The intent of this Criterion has several facets. First is to establish the fact that analysis, to remain viable, must be accomplished on a regular, periodic basis. Since most contractor's accounting and budgeting systems are established on a monthly basis, analysis should be accomplished on this same periodic interval.

Secondly, it is the intent of this Criterion to establish the fact that analysis efforts must begin, as a minimum, at the cost account level. Since the cost account is the lowest level where full management and control responsibility exists for specific CWBS increments of work, the cost account is the logical point for not only the planning, scheduling, budgeting and accounting efforts but also for the analysis effort as well. A Cost Account Manager would not have full management and control responsibility if his span of authority did not include the requirement to analyse the work performance and associated costs against the Performance Measurement Baseline. Since the cost account represents the lowest level of the CWBS and OBS, by virtue of this requirement for analysis to begin at this level, summarisation of analytical data and trends can be validly accomplished.

Another intent of this Criterion is to make it perfectly clear that all data to be analysed must come directly from, or be reconcilable with, the accounting system. This represents yet one more effort on the part of the criteria to ensure that completely comparable data is analysed; this minimises the amount of distortion that would otherwise be rampant if data from a "second set of books" were used for comparison/analysis purposes.

Lastly, this Criterion establishes the minimum content of any cost analysis effort. It implies that the following data elements must be identified, on a monthly basis, at the cost account level: BCWS, which represents the amount of work planned each month; BCWP which represents the amount of work actually accomplished each month as well as the budgeted value of that work progress; and ACWP, which represents the actual cost of the work accomplished each month. Given these three data elements, this Criterion requires that two comparisons be made with them: BCWP minus BCWS results in the cost account's Schedule Variance (SV) expressed in budgetary terms. BCWP minus ACWP results in the cost account's Cost Variance (CV) expressed in dollars relative to the budget. Since budgets and actuals are required by previous criteria to be established/accumulated by element of cost, the above data elements, comparisons, and variances can (and should) also be identified by those same elements of cost. Lastly, where variances (schedule or cost) exist, the Cost Account Manager is required by this Criterion to identify the reasons for those which are significant. (The significance of a variance is usually established by its relative size in comparison to a variance threshold or tolerance band.)

SUMMARISE THE DATA ELEMENTS AND ASSOCIATED VARIANCES LISTED IN CRITERIA ANALYSIS 1 AND 2, THROUGH THE CONTRACTOR ORGANISATION AND CWBS TO THE REPORTING LEVEL SPECIFIED IN THE CONTRACT.

The intent of this Criterion is to ensure that the data being used by the contractor's managerial staff is the same data that is reported to the Project Authority. Since the CWBS and OBS exist as a formal and disciplined framework for work and responsibility definition they become the ideal vehicles for summarisation of data from the cost account level to the reporting level. Using the CWBS and OBS for summarisation purposes ensures that data on all work elements and organisational elements is included in the reporting data base. An additional benefit of this requirement for CWBS and OBS summarisation is that it helps to identify the significant problem areas from among all levels of the contractor organisation and the contract scope of work. When significant variances exist in the monthly data report, it is possible to track down through the CWBS and OBS to identify the causal factors involved in that variance. This auditability allows analysis of corrective action procedures and impact evaluation, both of which are important in decision-making at the Program Office level.

IDENTIFY ON A MONTHLY BASIS SIGNIFICANT DIFFERENCES BETWEEN PLANNED AND ACTUAL SCHEDULE ACCOMPLISHMENTS TOGETHER WITH THE REASONS.

Because a comparison of BCWS to BCWP provides a dollar-valued schedule variance determination, it is often misassumed that this is the extent of schedule management and analysis required by the Criteria. To be sure, such a schedule variance is extremely important to any performance measurement system, but it cannot stand alone. What can it mean, for example, when a contractor has a negative schedule variance of $1.5 million? How far behind schedule has the contractor actually slipped? Is this schedule slippage retrievable? How much time does this $1.5 million represent? And more importantly, as a summary-level schedule variance, what is the schedule status of the work packages of the underlying CWBS and OBS elements? The intent of this Criterion is to ensure duplicity in schedule management and analysis. Schedules must be managed by the work they represent as well as by the cost by which they are depicted in a BCWS or BCWP data element. Reasons for schedule variances must address work tasks in order for corrective action to be relevant. The qualifying of schedule variance in dollar terms is simply a means of addressing the trend of a schedule variance. It does not address the time factor of the work discrepancy. Reference to the actual schedules is necessary to determine actual slippages. This Criterion ensures that this time factor is not overlooked.

IDENTIFY MANAGERIAL ACTIONS TAKEN AS A RESULT OF CRITERIA ANALYSIS 1 THROUGH 4.

The intent of this Criterion is to ensure that after the contractor's managers have analysed the performance measurement data (in accordance with the four foregoing Criteria) they then take the necessary management action. This management activity should be identified, documented, and followed-up for effectiveness, and reported to the Project Authority (where significant variances are involved).

BASED ON PERFORMANCE TO DATE, ON COMMITMENT VALUES FOR MATERIAL, AND ON ESTIMATES OF FUTURE CONDITIONS, DEVELOP REVISED ESTIMATES OF COST AT COMPLETION FOR CWBS ELEMENTS IDENTIFIED IN THE CONTRACT AND COMPARE THESE WITH THE CONTRACT BUDGET BASE AND, WHERE REQUIRED BY CONTRACT, THE LATEST STATEMENT OF FUNDS REQUIREMENTS REPORTED TO THE COMMONWEALTH.

The Estimate at Completion (EAC), or as it is known to others, the Latest Revised Estimate (LRE), is the contractor's vehicle for telling the Commonwealth where each CWBS element, OBS element, and the total contract is going with respect to cost. At the beginning of a contract, the EAC is usually equal in value to the Budget at Completion (BAC) and these values are usually less than the Negotiated Contract Cost (NCC) and Contract Budget Base (CBB). At this point in time, optimism normally prevails. As a contract progresses, problems arise which have to be reacted to and neutralised. As these problems cause "significant" variances, the responsible manager must assess the impact of the problem. The most commonly used way of assessing impact is to develop a revised EAC. If, for example, a cost account had a BAC and EAC of $500,000 prior to the occurrence of a significant variance, and the EAC was revised to $600,000 as a result of the problem, the Cost Account Manager is telling you that the impact of that problem is $100,000 of additional expenses expected to be incurred to get back on track. With all managers using EACs to forecast impact of their problems, the EAC becomes a barometer to the Project Authority by which on program decisions can be taken before the problems blow out of control. The intent of this Criterion is to ensure that EACs be constructed properly and that they be compared to the amount of work authorised and the latest estimates of funds requirements reported to the Project Authority.