SIGIR 05-010: Audit Award Fee Memo Report

SPECIAL INSPE CTOR GENE RAL FOR IRAQ RECONSTRUCTION July 26, 2005 MEMORANDUM FOR DIRECTOR, PROJECT AND CONTRACTING OFFICE - IRAQ COMMANDING GENERAL, JOINT CONTRACTING COMMAND – IRAQ/AFGHANISTAN SUBJECT: Interim Briefing to the Project and Contracting Office - Iraq and the Joint Contracting Command – Iraq on the Audit of the Award Fee Process (Report No. SIGIR-05-010) The purpose of the award fee is to reward contractors for excellence in performance, as measured by the criteria provided and established in the contract. The importance of good criteria, including defined metrics, assists both the contractor and government evaluators in their assessment as to the level of performance of the contractor. Further, having a solid methodology that encourages contractors to strive for excellence in performance is the key to achieving the intent of an award fee for contractors. This memorandum transmits to you the slides, “Interim Briefing to the Project and Contracting Office - Iraq and the Joint Contracting Command – Iraq, Audit of the Award Fee Process, July 19, 2005,” used to brief Project and Contracting Office – Iraq and Joint Contracting Command – Iraq officials on issues related to the award fee process. As one of the Special Inspector General’s goals is to provide timely information so that prompt action can be taken when necessary on improvements to program management, this interim briefing is being provided to you so that you may initiate timely action on opportunities for improvements to: • award fee plans • metrics used to measure the contractors’ performance • methodologies used to award contractors for excellence in performance • overall recordkeeping required in this process This interim briefing made five recommendations. As agreed at the briefing, Project and Contracting Office – Iraq and Joint Contracting Command – Iraq management officials provided oral comments. In summary, officials of both organizations agreed with our recommendations to improve the documentation for the award fee process. Those officials stated that they were already taking additional actions to improve the documentation of the award fee process and associated files. They also provided us with a draft of a new policy and procedure document for the award fee process. However, both Project and Contracting Office - Iraq and Joint Contracting Command - Iraq officials stated that they were going to assess the impact of our recommendation for changing the award fee methodology on the basis of how contractors are currently awarded fees. They requested an opportunity to brief us on the results of their assessments of the impact prior to the completion of the subject audit. We agreed. 400 Army Navy Drive • Arlington, Virginia 22202 As noted in our briefing to Project and Contracting Office – Iraq and Joint Contracting Command – Iraq officials, we are continuing our assessment of other U.S. government organizations that have award fees included in contracts funded by the Iraq Relief and Reconstruction Fund. We will report on those results and actions taken by those organizations in a future audit report. We performed the audit in accordance with our statutory duties contained in Public Law 108- 106, as amended, which mandates the independent and objective conduct of audits relating to the programs and operations funded with amounts appropriated or otherwise made available to the Iraq Relief and Reconstruction Fund. Public Law 108-106, as amended, requires that we provide for the independent and objective leadership and coordination of and recommendations on policies designed to promote economy, efficiency, and effectiveness in the administration of such programs and operations and to prevent and detect waste, fraud, and abuse. Our review of these issues was performed in accordance with generally accepted government auditing standards. We are sending copies of this memorandum to the Secretary of State, the Secretary of Defense, selected Congressional Committees, and other interested parties. This memorandum will also be available on the Office of the Special Inspector General for Iraq Reconstruction website at http://www.sigir.mil. We appreciate the courtesies extended to the staff. For additional information on this interim briefing report, please contact Mr. James P. Mitchell at jim.mitchell@sigir.mil or at (703) 428-1100. Stuart W. Bowen, Jr. Inspector General SPECIAL INSPECTOR GENERAL FOR IRAQ RECONSTRUCTION Interim Briefing Interim Briefing to the to the Project and Contracting Office Project and Contracting Office - - Iraq Iraq and the and the Joint Contracting Command Joint Contracting Command - - Iraq Iraq Audit of the Award Fee Process Audit of the Award Fee Process July 19, 2005 July 19, 2005 1 Background • May 2003 - the Secretary of the Army was designated as the Executive Agent for support of the Office of Reconstruction and Humanitarian Assistance. • June 2003 - the Army’s support role was expanded to include the Coalition Provisional Authority. • November 2003 - the Congress appropriated $18.6 billion for the Iraqi Relief and Reconstruction Fund of which $18.4 billion was designated for Iraq. • January 2004 - the Army’s role was clarified to include acquisition and program management support to the Coalition Provisional Authority and any successor entity. • June 2004 - the President continued the Department of Defense support role by creating the Project and Contracting Office. • The Project and Contracting Office - Iraq (PCO-I) is responsible for all activities associated with program, project, asset, construction, and financial management of that portion of the reconstruction effort undertaken by the U.S. • The Joint Contracting Command - Iraq (JCC-I) provides operational contracting support to the Chief of Mission, the Multi-National Forces – Iraq, 2 and the PCO-I. Definitions • Cost-Plus-Award-Fee contract. The Cost-Plus-Award-Fee contract is a cost-reimbursement type contract with an estimated contract amount and special fee provisions. The fee established in a Cost- Plus-Award-Fee contract consists of: – a fixed amount (the base fee) which does not vary with performance. The base fee is a fixed amount paid to the contractor and is designed to compensate the contractor for factors such as risk assumption, investment, and the nature of the work – an award fee pool 3 Definitions (cont’d) – The award fee pool represents an additional dollar amount available to the contractor to earn for performance that demonstrates quality efforts on a single contract toward accomplishing the tasks and functions cited in the contract. – The award fee pool is awarded for excellence in performance as measured by the criteria defined and established in the contract. – The award fee period is based on contract terms, most terms are semi-annual, commencing upon contract award. 4 Definitions (cont’d) • Award Fee Evaluation Board (AFEB) – Evaluates the contractor’s overall performance and recommends the amount of the award fee to the Award Fee Determination Official. • Award Fee Determination Official (AFDO) – A senior official appointed by the Principal Assistant Responsible for Contracting to review the AFEB assessment and determine the amount of the award fee to be paid to the contractor. • Performance Monitor – An individual with daily or frequent cognizance of the contractor’s performance. 5 Additional Acronyms • ACA – Army Contracting Agency • AFARS – Army Federal Acquisition Regulation Supplement • D-001 – Contract W914NS-04-D-0001 • D-002 – Contract W914NS-04-D-0002 • DFARS – Department of Defense Federal Acquisition Regulation Supplement • FAR – Federal Acquisition Regulation 6 Scope • In June 2005, we reviewed 18 Cost-Plus-Award-Fee contracts, awarded from January 2004 to March 2004, by 6 different procuring organizations. • The 18 contracts reviewed consisted of 11 design-build contracts valued at almost $6.75B, and 7 program management and support (PMAS) contracts valued at more than $200M. • We reviewed the contract files at JCC-I. • We reviewed the AFDO award fee assessment files . 7 Methodology • Reviewed the FAR, the DFARS, and the AFARS for guidance concerning award fees. • Conducted limited research of guidance from other audit agencies auditing award fees and reviewed the Army Audit Agency’s report “Program Management in Support of Iraq Reconstruction,” issued May 26, 2005. • Reviewed procedural guides from the ACA. • Discussed contract and task order procedures with key personnel at the PCO-I and the JCC-I. • Reviewed the “hardcopy” contract files maintained at the JCC-I. • Reviewed the JCC-I electronic contract files; including emails, WORD documents, EXCEL spreadsheets, and PDF files not contained in hardcopy format. • Reviewed the AFDO documentation from the AFEB evaluations conducted from the 1st and 2nd Award Fee periods. 8 Criteria • FAR 16.405-2, Cost-plus-award-fee contracts • DFARS 216.405-2, Cost-plus-award-fee contracts • AFARS 5116.405-2, Cost-plus-award-fee contracts • ACA Acquisition Instructions • ACA Award Fee Contracts • ACA Award Fee Member Guide • ACA Contract Incentive Guide 9 Objective • Our over-arching objective is to determine whether award fees are adequately reviewed, properly approved, and awarded according to established standards. 10 Our Specific Objectives Were to Ascertain if: • An AFEB and pertinent policies and adequate procedures have been established and consistently applied to the evaluation of award fees. • Award fee plans clearly identify the specific award fee evaluation criteria for assessing contractor performance and determining the amount of the award fee. – The performance indicators were properly established. – The performance requirements were properly defined. 11 Our Specific Objectives Were to Ascertain if: (cont’d) • AFEB recommendations are supported by appropriate evaluations of contractor performance. • The award fee determination is documented in sufficient detail to show that the integrity of the award fee determination process has been maintained. 12 Results of Review Was an AFEB established? Yes Were pertinent policies and adequate procedures established? Yes Were the policies and procedures consistently applied? No 13 Summary of Inconsistent Application of Policies and Procedures • Documentation could not be located for the appointment of the AFEB Chairperson, other AFEB board members, and performance monitors in all the contracts. – AFEB Chairperson • 1st Award Fee Period – 9 out of 18 contracts reviewed did not document an appointment • 2nd Award Fee Period – 7 out of 18 contracts reviewed did not document an appointment – AFEB Members • 1st Award Fee Period – 14 out of 18 contracts reviewed did not document any appointments • 2nd Award Fee Period – 12 out of 18 contracts reviewed did not document any appointments – Performance Monitors • 1st Award Fee Period – 13 out of 18 contracts reviewed did not document any appointments • 2nd Award Fee Period – 12 out of 18 contracts reviewed did not document any appointments 14 Summary of Inconsistent Application of Policies and Procedures (cont’d) • Limited evidence was found supporting that the monthly contract assessments are performed. – Performance Monitor’s Monthly Reports • 1st Award Fee Period – 13 out of 18 contracts reviewed showed no evidence of monthly monitoring being performed • 2nd Award Fee Period – 12 out of 18 contracts reviewed showed no evidence of monthly monitoring being performed – AFEB Monthly Meetings • 1st Award Fee Period – 16 out of 18 contracts reviewed showed no evidence of monthly meetings • 2nd Award Fee Period – 13 out of 18 contracts reviewed showed no evidence of monthly meetings 15 Summary of Inconsistent Application of Policies and Procedures (cont’d) • Self-assessment from contractors not always in the file. • 1st Award Fee Period – 9 out of 18 contracts reviewed showed no evidence of the contractor’s self-assessments • 2nd Award Fee Period – 5 out of 18 contracts reviewed showed no evidence of the contractor’s self-assessments 16 Example Noncompliance Award Fee • Contract W914NS-04-C-0007 to Foster Wheeler • Award fee plan mandated monthly performance meetings with feedback to the contractor. • At the end of the 1st award period (March – September 2004), the AFEB awarded a score of 15% which equated to zero dollars. • Contractor appealed the score based upon no monthly performance meetings or performance feedback. • U.S. Government subsequently awarded a score of 50%, that resulted in an award fee in the amount of $439,145 being paid from the award fee pool of $878,291. • Contract modification which allowed payment of the award fee was signed February 5, 2005. 17 Results of Review Did the award fee plans clearly identify the specific award fee Did the award fee plans clearly identify the specific award fee evaluation criteria and were the performance indicators properly evaluation criteria and were the performance indicators properly established? established? No No • • Evaluation criteria was subjective with few metrics being used. Evaluation criteria was subjective with few metrics being used. Defined metrics would include specified time periods, acceptable Defined metrics would include specified time periods, acceptable error rates, etc. error rates, etc. 18 Examples of Requirements Without Metrics from the Award Fee Plans • Schedule Adherence Criteria – Used words such as “timeliness, accuracy, and completeness of performance” as the requirement. • Cost Control Criteria – Used words such as “maintain effective cost control measures . . .” as the requirement. • Quality Control Criteria – Used words such as “minimization of rework” as the requirement. • Training Criteria – Used words such as “effective hiring and integration of Iraqi women” as the requirement. 19 Conclusion • The effect of stated evaluation criteria without established definable metrics could result in over-inflated contractor performance evaluations. • This conclusion is similar to the results reported in the Army Audit Agency’s report “Program Management in Support of Iraq Reconstruction,” issued May 26, 2005. 20 Performance Scores of the Second Contract Evaluation Period From the Award Fee Determination Officer • For the 11 Design-Build contracts reviewed: – 1 below average rating of 66.9% – 1 average rating of 67.3% – 4 above average ratings of 84.7% to 86.6% – 5 excellent ratings of 93.8% to 96% • For the 7 PMAS contracts reviewed: – 1 very good rating of 78.3% – 6 outstanding ratings of 93.5% to 98.7% 21 Results of Review • Does the award fee plan provide incentive for the contractor to strive for quality efforts towards a superior performance? 22 Current Award Fee Plan Variances in Fee Calculations • 16 of the 18 award fee plans require awards based on the percentage of score (e.g. a 70% performance score results in receiving 70% of the award fee pool, an 80% score results in receiving 80 % of the award fee pool, etc.) • 2 of the 18 award fee plans for D-001 and D-002 require awards based from no fee awarded for a performance score less than or equal to 70%, to receiving 40% of the award fee pool with a performance score of 80%, and receiving 80% of the award fee pool with a performance score of 90%. 23 Current Award Fee Plan Criteria • Per AFARS 5116.405-2 “. . . contractors should not receive award fee (above the base fee) for simply meeting contract requirements....” 24 Current Award Fee Plan Result • Base fee component is 3% for 16 of the 18 contracts, which is the highest base fee allowed by DFARS. • For 7 of the 18 PMAS contracts, the current award fee plans allow awards of an additional 60% -70% of the award fee pool for some performance above standard and with several performance weaknesses. • For 9 of the 18 design-build contracts, the current award fee plans allow awards of 50% - 74% of the award fee pool for average results. 25 Current Award Fee Plan Percent of Award Fee Pool That Can be Earned 100 Percent of fee awarded 80 PMAS Contracts (7 of 18) Design-Build Contracts (9 of 18) D-001/D-002 Contracts (2 of 18) 60 40 20 0 0 10 20 30 40 50 60 70 80 90 100 Performance Score 26 Alternate Method for Distributing Award Fees • ACA Award Fee Contracts Guide identifies the Cubic scale as one that properly rewards superior performance. • This scale can be applied through an unilateral modification to existing contracts. 27 Percent of Award Fee 100 80 Comparative Analysis of the Current Scoring Scale to the Cubic Scale D-001/D-002 Cubic PMAS Design-Build 60 40 20 0 40 50 60 70 80 90 100 Performance Rating 28 Comparison of Award Fees Received Using the Cubic Strategy Versus the Current Method Award Fees Using Cubic Strategy for Awarding Fee Actual Award Fees Using Current Fee Strategy For the 11 Design-Build contracts Reviewed: 1 below average rating of 66.9%* 1 average rating of 67.3% $ 0 309,436 $ 0 1,666,772 4 above average ratings of 84.7% to 86.6% 5 excellent ratings of 93.8% to 96% For the 7 PMAS contracts reviewed: 1 very good rating of 78.3% 6 outstanding ratings of 93.5% to 98.7% 21,884,504 20,536,016 27,642,137 21,062,281 49,787 9,327,239 86,667 9,479,620 29 Conclusion • The PCO-I and the JCC-I have been proactive in implementing corrective actions. Notable examples include: – Contract files have been indexed and filing system improved. – The JCC-I has conducted additional training regarding award fees and the process to use. – The AFDO and the JCC-I have drafted AFEB policies to provide guidance on the responsibilities for award fee evaluation and administration. Those policies include the provision that the Contracting Officer is a member of the AFEB to ensure contract integrity, contract compliance, and centralization of contract files. • Opportunity to leverage award fee pool by applying an alternate methodology (e.g., Cubic scale) to determine award fees. That encourages contractors to demonstrate quality efforts toward accomplishing the tasks and functions cited in the contract to reward performance excellence.30 Based Upon our Interim Results, We Recommend that the PCO-I and the JCC-I: • Document appointments in the contract files made for the AFEB Chairperson, the AFEB board members, and performance monitors. • Comply with the award fee plan to ensure that: – monthly assessments are performed – self-assessments are obtained from the contractors, where required – meetings with the contractors to discuss the monthly evaluations are documented • Centralize all award fee documentation. 31 Based Upon our Interim Results, We Recommend that the PCO-I and the JCC-I: (cont’d) • Revise award fee plans to more clearly identify the specific award fee evaluation criteria for assessing contractor performance (i.e., more metrics and/or quantifiable criteria to distinguish between poor performer and an excellent performer). • Modify the current contracts to utilize a method of applying the award fee to provide additional incentive for contractors to achieve quality results. 32 Audit Team Members • The Office of the Assistant Inspector General for Auditing, Office of the Special Inspector General for Iraq Reconstruction prepared this presentation. Staff members who contributed include: – Joseph T. McDermott – Tony L. Adams – James A. Carrera – Ronald L. Rembold 33 Questions/Comments? 34