Industry Resources
Earned Value Management
EVM Home | EV Primer | Policy Documents | Training| FAQs | Contacts
Defence Requirements for Performance Management Using Earned Value
If you know nothing about Earned Value, you might like to take a quick tutorial before proceeding!
- Introduction
- Management Control Systems
- Performance Measurement
- The Earned Value Performance Management Process
- Where Are The Rules And The Guidance?
- What Do The Standards Require?
- Overview of Defence Policy For EVM
- What Is The International Focus?
- So What Does The Earned Value Approach To Performance Management Offer?
Introduction
TOP
This page provides an overview, from a Defence Department perspective, of what the earned value performance measurement process is about, why and how they are applied, and some of the advantages of implementing Earned Value Management. Good management of projects essentially consists of measuring how the project
is performing and making appropriate management decisions. Thus:
Performance Measurement + Decision Making = Performance Management
Management Control Systems
TOP
Every company or organisation will have developed systems to manage the functions it performs. In large organisations, these systems may be formalised in a family of documents that set out the company's organisation, policies, objectives, financial systems, inventory control methodology, etc. In smaller organisations, the "system" may be well understood although little documentation may exist.The Department of Defence seeks to be assured that contractors possess adequate performance management in their control systems appropriate for the circumstances. This means that the contractors' systems should be neither too stringent nor too informal; they should be appropriate for the complexity and scale of work to be performed.
For major acquisition contracts, Defence requires contractors to comply with the Australian Standard AS 4817-2006 and the Defence Supplement to AS 4817-2006 as part of the contract requirements. The application of these contract requirements will vary depending on the criticality of the procurement, the anticipated contract duration, and the project risk profile.Performance Measurement
TOP
The Australian Standard and Defence Supplement describe sound management practices in general use. A novel aspect, however, relates to the measurement of progress. ”Progress" can be measured in many ways and points of view will differ depending on whether you are the customer, the contractor, a finance specialist, a designer, the manufacturer or the potential user. For example, the contractor's Finance Manager may consider early payment to be real progress while the end user is only interested in the timely delivery of an end item that works well in service.In project management, we are concerned with all types of "progress", but in terms of Performance Measurement, we are referring to "Earned Value" which is the measurement and monitoring of real project progress in terms of cost (or price), time and technical/physical accomplishment. It is an objective methodology for measuring and reporting actual achievement in terms of the way the work was planned to be performed.
We are also interested in the relationship between progress and payments effected under a contract.The Earned Value Performance Measurement Process
TOP
Under the requirements for Earned Value Management (EVM), contractors' performance management systems are required to meet several key principles. These are promulgated in the Australian Standard AS 4817-2006 and the Defence Supplement and are based on the simple approach to plan the work and work the plan. These requirements can also be arranged in five categories as follows:- Organisation. Definition of work, company organisations and delegation of authority and responsibility.
- Planning and Budgeting. Scheduling work, establishing budgets and methods to measure achievement, and developing a baseline against which to measure contract performance.
- Accounting. Recording costs at appropriate levels and the ability to summarise them by relevant cost category and element to the contract total.
- Analysis. Contract performance measurement and the utilisation of this information within the company and in reports to Defence.
- Revisions and Access to Data. Maintaining the integrity of the baseline, the timely incorporation of changes to the contract and access to EVM data by the customer.
Revisions and Access to Data. Maintaining the integrity of the baseline, the timely incorporation of changes to the contract and access to EVM data by the customer.
Where Are The Rules And The Guidance?
TOP
The requirements are contained in Australian Standard AS 4817-2006, the Defence Supplement to AS 4817-2006 and the ASDEFCON Strategic Contracting template Statement of work. Additional insight into Defence’s implementation of the standards can be found in the following documents:- EVM System Review Handbook
- IBR Handbook
Copies of these documents can be obtained electronically under the Policy Documents page within the EVM web site.
What Do The Standards Require?
TOP
Organisation
The first step in the development of any effective management control system is the organisation of the work and the people to undertake it. The standards require that all work to be performed under the contract be defined within the framework of a Contract Work Breakdown Structure (CWBS) and assigned to individuals responsible for undertaking that work. The latter is displayed on an Organisation Breakdown Structure or OBS. Put simply, you need to define all the work and the individuals responsible for it.A CWBS is simply a tool to ensure that all work tasks have been defined. It is a product-oriented breakdown of the contract into its component parts. The level to which a CWBS is broken down depends on the complexity of the contract and the level of detail required for effective management.
The CWBS:
- provides a formal structure for identifying the products, components, tasks and services required to produce the end product to the contractual technical specifications;
- provides a framework for integrating management subsystems;
- aids development of a logical workplan to meet contractual requirements; and
- reflects the reporting structure for management information required by the customer.
Further guidance on the use of Work Breakdown Structures can be found in DEF(AUST) 5664.
While the CWBS identifies the work, the OBS identifies the people to undertake it.
The meshing of the CWBS and the OBS (ie, drawing together the work and those responsible for that work) can be visualised as a matrix. This Responsibility Assignment Matrix, or RAM, assists in the identification of each organisation with its allocated work during the planning phases of a contract.
The assignment of work elements to responsible organisations identifies a key point for management control and cost collection purposes. This control point, or Control Account, represents the point where a Control Account Manager (CAM) is assigned responsibility to plan work, determine budget, and measure performance and manage variances to planned performance.
Control Accounts have the following characteristics:
- they have separately identified budgets that are time-phased over the life of the Control Account;
- they summarise directly into the CWBS and the OBS to the total contract level;
- they always have a clearly defined manager (the CAM); and
- they are further broken down into identifiable elements of measurable work (Work Packages), level of effort tasks and work commencing at a future time (Planning Packages).
Planning and Budgeting
While the Organisation group outlines the basic framework for defining and organising work, the Planning and Budgeting group addresses the issues of scheduling and budgeting. In simple terms, this requires that all work to be performed is properly scheduled and that appropriate budgets are assigned to units of work.
Scheduling
The prime purpose of the scheduling is to ensure that all work is scheduled, critical interfaces are considered, visibility into work progress is possible and management is provided with valid and timely schedule information.
First and foremost, the standards do not require the use of any specific scheduling system or methodology. Rather, they require that the scheduling system used be formal (ie. documented and understood by those using it), complete and consistent. Given the scale or scope of contracts to which the requirements are applied, a contractor's system should contain a summary, or Master-level, schedule and component subordinate schedules. These should document a logical sequence and highlight interdependencies from the summary to the detailed Work Package levels.Budgeting
The Standards require that a contractor:
- establish and maintain a project budget baseline against which performance can be measured;
- ensure that all project work is budgeted and that this budget is accurately assigned to identified units of work; and
- establish budgets that can be directly related to milestones and effort identified in the scheduling system.
Budgets are planning targets for the use of resources. They may be expressed in monetary terms or other resources (eg. manhours).
Within a budgeting system, a number of defined elements exist. The more important are:
- Project Budget (PB). The total negotiated cost of the contract (less any profit or fee).
- Performance Measurement Baseline (PMB). The time-phased budget plan against which performance is measured. This is composed of the combined budgets of all identified work elements within the contract.
- Management Reserve (MR). An amount of the budget held aside by contractor management to cover certain unforseen requirements. MR does not form part of the PMB.
- Undistributed Budget (UB). Budget which cannot be identified to elements of work at or below the reporting level (ie. work that has not yet found a "home" within the CWBS).
The relationship of these budgets to each other and Control Accounts is illustrated below:
Accounting
In supporting performance measurement, the contractor's internal accounting system must be able to relate costs incurred to work accomplished.
This group basically requires a contractor's accounting system to accurately record all costs applicable to the contract and summarise these costs from the level where they occur to the contract total through both the CWBS and the OBS.
To achieve this, the costs should be accumulated at the Control Account level (at least). These costs, known as Actual Cost, or AC, are then summarised to higher levels for reporting or consolidation purposes.
These costs are further broken down into Indirect (those costs not readily identifiable to a specific contract) and Direct Costs. Typically, Direct Costs are further broken down into:
- Labour,
- Material,
- Other Direct Costs, and
- Subcontractor Effort.
Analysis (or Performance Measurement)
The “Organisation”, “Planning” and “Budgeting” categories contain the EVM system requirements, to plan the work and establish a baseline against which to measure performance. Accounting contain the requirements for an EVM system to accurately accumulate costs arising from work undertaken. Analysis builds on these foundations and outlines requirements for the measurement of performance and reporting this performance up the management chain of the contractor. Although the standards do not specifically require the submission of data or reports, they do set out the characteristics a system must possess and the type of data that should therefore be available.
The basic objective is to require the contractor to identify and correct or accommodate deviations to the plan. This includes recording, analysing, summarising and reporting data so that management (both the contractor and Defence) are able to make informed and timely decisions.
To achieve this, five basic data elements are required. These are:
- Planned Value (PV). The sum of the budgets for all work effort planned to be accomplished (ie. the plan).
- Earned Value (EV). The sum of the budgets for all work effort actually accomplished.
- Actual Cost (AC). The actual costs incurred to date.
- Budget At Completion (BAC). The sum of all budgets allocated to the contract. This does not include the contractor Management Reserve.
- Estimate at Completion (EAC). Actual costs to date plus an estimate for work remaining.
Analysis of these data elements provides managers with the information needed to effectively monitor progress. This monitoring is achieved by analysis of Cost Variance (CV), Schedule Variance (SV), and Variance at Completion (VAC). These are determined as follows:
CV = EV - AC
SV = EV - PV
VAC = BAC - EAC
Revisions
The last group is concerned primarily with maintaining the integrity of the PMB and ensuring the timely and orderly incorporation of changes to the contract. Throughout their life, all contracts are subject to change. Keeping track of these changes is a vital facet of project management. Under these criteria, the integrity of the baseline is maintained through traceability of changes while timely incorporation of the changes ensures managers are using up to date information in the decision making process.
Revisions commonly encountered are of three types:
- External Changes . Those contract changes directed by or negotiated with Defence.
- Internal Replanning . Revisions to the contract work plan undertaken by the contractor within the budget, scope and schedule of the contract.
- Formal Reprogramming . A major replan of the contract undertaken where the current baseline is clearly unrealistic or unworkable. This does not necessarily require any change to the contract price.
However, if the proposed new PMB exceeds the Contract Budget Base (CBB) the result is referred to as an Over-Target Baseline (OTB). Customer approval is required before implementation of an OTB.
Overview of Defence Policy For Earned Value Management
TOP
Having provided a brief overview of the Criteria, where is Defence policy detailed and where is it applied?Defence policy for Performance Management is set out in:
- Australian Standard AS 4817-2006, Project performance measurement using Earned Value;
- Defence Supplement to AS 4817-2006; and
- ASDEFCON Strategic Contracting Template.
When Does Defence Require Earned Value Management?
As a general guide, Defence will seek to apply Earned Value Management to contracts which are likely to exceed $20m. However, the risk of the contract will be used to determine whether these performance management requirements will be levied and what level of reviews will be required. Defence may also seek the application of these requirements to subcontracts based on subcontract risk.
Reporting Performance
Although the standards themselves do not require or specify any reporting formats, contracts specifying Earned Value Management normally require the contractor to report performance to Defence via a monthly Earned Value Performance Report (EVPR).
The EVPR consists of five formats or separate reports:
- Format 1. Cost and schedule status by CWBS reporting element.
- Format 2. Cost and schedule information by company functional breakdown (eg, Design, Manufacturing, etc.).
- Format 3. Current status of the PMB including details of any amendments that have taken place during the reporting period.
- Format 4. Forecasts of manpower requirements over the life of the projects.
- Format 5. An explanation of all variances (both positive and negative) to the plan that exceed set thresholds.
Defence requires that these reports contain cost-level (as opposed to price) information even where the contract may be of a fixed priced nature.
Defence will also require baseline and statused schedule information direct from the contractor's scheduling system in the form of networks, Gantt charts or other appropriate reports.
These reports, while useful, do not substitute for day-to-day management and contact between the contractor and customer. However, the reports are valuable for confirming and quantifying known or suspected problems and they force recognition of emerging cost problems. In addition, the data is invaluable in forecasting future performance.
How Does Defence Assess a Contractor's System?
When performance management requirements are set out in a contract, Defence must assure itself that the specified requirements are being met and that "the system" is supplying valid reporting data.
Details of the reviews and related processes are contained in the Earned Value Management System Review Handbook and the Integrated Baseline Review Handbook. It is recognised that not all contracts and Contractors are alike. Some contracts will be low risk, others high. Some Contractors will have a mature EVMS in place while others will not. The following are typical EVM reviews and associated activities. The requirements for, content of, and timings of each will be dependent on the level of contract risk.
-
The first activity will be an Implementation Workshop (IW). This workshop is focused on the Contractor’s proposed implementation strategy and is generally necessary where there is no existing EVMS. The IW will be held either during the Offer Definition stage or no later than one month after the contract effective date (ED+1M). The IW is not a review. It provides the Project Office and Contractor with the opportunity to gain a common understanding of the EVMS requirements and provides the Contractor the opportunity to explain how these requirements will be met.
-
The Integrated Baseline Review (IBR) is a risk-based process enabling a technical and schedule review of the planned effort. It focuses on the assignment, definition, scheduling and resourcing of work, thus establishing early visibility into the acceptability of the Contractor’s contract planning. The IBR also reviews the methods and metrics used to measure contract performance or progress. The process involves a review of pertinent documentation, an on-site review of the Contractor’s proposed plan, a review of some of the management systems directly related to the establishment of the PMB and interviews/discussions with relevant managers. An IBR is usually scheduled for ED +3 to 6 months.
-
The EVMS Review assesses the Contractor’s practical implementation of an EVMS for compliance with the contractual requirements and to ensure it is capable of generating accurate and reliable data. This review is tailored for each project based on project contract characteristics, the Contractor’s prior EVM experience, and past performance of the Contractor in applying EVM to other Defence contracts or other contracts where the equivalent EVM requirements have been applied. The review focuses on the Contractor’s documented process and procedures. The on-site portion allows the Contractor to demonstrate that its EVM System is compliant with specified requirements (this will include AS 4817-2006, the Defence Supplement to AS 4817-2006, and the negotiated contract).
- Lastly, System Assurance Reviews will be conducted as an ongoing exercise to maintain compliance with the requirements. This type of review will allow the Project Office to assess the integrity, implementation and use of the system by the Contractor’s staff. The depth and frequency of these reviews will be based on a risk assessment. For example if the data being provided by the Contractor becomes doubtful, or quality deteriorates, etc., then a System Assurance Review is necessary.
An important point to remember with this process is that the above represents only the formal aspects. Defence encourages and supports ongoing discussions and cooperation between Contractors and the Project Office throughout the life of the project.
What Are The Costs And Benefits Of The Review Process?
Firstly, undergoing reviews of your performance management system will cost in terms of time, effort, people and dollars. However, this is offset by benefits.
Having agreed in the contract to put in place a system meeting the requirements, the review process serves to verify your system does meet the requirements of the contract, including the national standard. Not only do you know it works but so does Defence.
And what does this provide? Verification of your performance management system’s compliance to these standards provides your customers with confidence in your ability to manage complex projects and reduces the perceived risk in your ability to manage future projects.
What Is The International Focus?
TOP
The principles in the Standards as used by Defence originally arose in the USA Department of Defense. This process is now internationally accepted and has become accepted in several countries including Canada, Sweden, and the USA.The Australian Department of Defence is a member of the International Performance Measurement Council (IPMC), a body made up of Australia, Canada, Sweden, the USA, UK, New Zealand and Japan. The IPMC continually monitors the principles of performance management and is a forum for exchange of information.
Since November 1995, the Department of National Defence of Canada, the Australian Department of Defence, and the US Department of Defense have been participants in an International Memorandum of Understanding (MoU) to foster the mutual acceptance of cost, schedule, and technical performance management objectives using Earned Value as the integrating tool. The original MoU was for a period of ten years and is approaching its expiration date. All three participating countries have agreed to a ten year extension.
This MoU establishes a comprehensive framework for the purpose of implementing mutually accepted cost and schedule performance management principles on complex defence projects in accordance with each Participant’s laws, policies and national priorities.
So What Does The Earned Value Approach To Performance Management Offer?
TOP
The Standards provide a sound basis for effective project management. Some attributes are:- meaningful information for decision making;
- discipline through the use of formal procedures;
- objectivity, again through formalised processes;
- consistency in performance measurement;
- accountability through the allocation of tasks and responsibilities to those actually doing the work; and
- effective management of change.
It should be remembered that these advantages are not Defence-specific; they are common to any industry requiring quality management.
Last updated: 16 June 2008
