SIGIR 05-017: FINAL Award Fee Report

O OF FF FI IC CE E O OF F T TH HE E S SP PE EC CI IA AL L I IN NS SP PE EC CT TO OR R G GE EN NE ER RA AL L F A A AW W C C CO O ON N I I IR WA NT R RA A AR T TR R A AQ Q R RD RA A D D F F AC C Q R R FE CT RE E EE E E T TO O OR R E EC C CO O P P RS S ON PR S I N NS R RO I IN N S ST O OC NV V T TR R R RE E EP P PO O OR R RT T T FO OR R I IR RA AQ Q R RE EC CO ON NS ST TR RU UC CT TI IO ON N C CE E ES VO O OL R RU U UC S SS L LV V C CT S S F F VE E T TI I IO O FO ED D D ON O OR R R I I IN N N N N N N NU U UM M MB B BE E ER R R S S SI I IG G GI I IR R R- - -0 0 05 5 5- - -0 0 01 1 17 7 7 O O OC C CT T TO O OB B BE E ER R R 2 2 25 5 5, , , 2 2 20 0 00 0 05 5 5 SPECIAL INSPE CTOR GENE RAL FOR IRAQ RECONSTRUCTION October 25, 2005 MEMORANDUM FOR COMMANDING GENERAL, JOINT CONTRACTING COMMAND – IRAQ/AFGHANISTAN SUBJECT: Award Fee Process for Contractors Involved in Iraq Reconstruction (Report No. SIGIR-05-017) We are providing this report for your information and use. We initiated an audit of the award fee process for contractors involved in Iraq reconstruction in June 2005 as a result of concerns previously brought to our attention by staff of the United States Army Corps of Engineers, Gulf Region Division, Iraq. We performed the audit in accordance with Public Law 108-106, which mandates the independent and objective conduct of audits relating to the programs and operations funded with amounts appropriated or otherwise made available to the Iraq Relief and Reconstruction Fund. Public Law 108-106, as amended, requires that we provide for the independent and objective leadership and coordination of and recommendations on policies designed to promote economy, efficiency, and effectiveness in the administration of such programs and operations and to prevent and detect waste, fraud, and abuse. We considered comments received from the Joint Contracting Command - Iraq/Afghanistan on a draft of this report when preparing the final report. We appreciate the courtesies extended to the staff. For additional information on this report, please contact Mr. Joseph T. McDermott by email at joseph.mcdermott@sigir.mil or at (703) 428-1100, or Mr. James Carrera by email at james.carrera@sigir.mil or at (703) 428-1100. For the report distribution, see Appendix E. Stuart W. Bowen, Jr. Inspector General 400 Army Navy Drive • Arlington, Virginia 22202 Special Inspector General for Iraq Reconstruction Report No. SIGIR-05-017 (Project No. SIGIR-2005-10) October 25, 2005 Award Fee Process for Contractors Involved in Iraq Reconstruction Executive Summary Introduction. This audit report discusses the Award Fee Process for contracts awarded for projects that are funded with the Iraq Relief and Reconstruction Fund. The goal of award fees in contracting is to motivate a contractor’s performance by offering financial incentives in areas critical to program success that are susceptible to measurement and evaluation. Upon execution of a contract that contains an award fee arrangement, the contracting officer initiates a process that provides incentives to a contractor to improve performance and requires the Government to record assessments of the contractor’s performance upon which award fees will be based. Title II of Public Law 108-106, “Emergency Supplemental Appropriations Act for Defense and for the Reconstruction of Iraq and Afghanistan, 2004,” authorized $18.4 billion for security, relief, rehabilitation, and reconstruction projects in Iraq funded through the Iraq Relief and Reconstruction Fund (IRRF). Objective. The overall objective of this audit was to determine whether award fees provided to contractors performing IRRF-funded projects are adequately reviewed, properly approved, sufficiently substantiated, and awarded according to established standards. Specifically, we sought to determine whether: • An award review board has been established, and pertinent policies and adequate procedures have been consistently applied to the evaluation of award fees. • Award fee plans clearly identified the specific award fee evaluation criteria for assessing contractor performance and determining the amount of the award fee. • The performance indicators were properly established, and the performance requirements were properly defined. • Award review board recommendations and determinations were supported by appropriate evaluations of contractor performance. • The award fee determination was documented in sufficient detail to show that the integrity of the award fee determination process has been maintained. Results. We reviewed the 18 cost-plus award-fee contracts funded with IRRF monies. The contracts and the associated contract files were located at the Project Contracting Office (PCO) and the Joint Contracting Command - Iraq/Afghanistan (JCC-I/A). The 18 contracts reviewed consisted of 11 design-build (DB) contracts valued at up to $6.75 billion and 7 program management and support (PMAS) contracts valued in excess of $200 million. These 18 contracts were awarded through six different procuring activities. i In general, cost-plus-award-fee contracts include a base award fee (for simply meeting contract requirements) and a merit-based award fee for performance that exceeds contract expectations. For 16 of the 18 contracts, the base fee component was 3 percent, which is the highest base fee allowed by the Department of Defense Federal Acquisition Regulation (FAR) Supplement. To receive merit-based award fees, management should evaluate and measure contractors’ performance against specific award fee evaluation criteria, which should be provided in the contracts. However, the 18 contracts did not contain the required criteria with definable metrics. This missing component creates the potential for inflated contractor performance evaluations. Further, the Army FAR Supplement 5116.405-2 states that “contractors should not receive award fees (above the base fee) for simply meeting contract requirements.” For 9 of the 11 DB contracts, the award fee plans allowed awards of an additional 50 percent to 74 percent of the award fee pool for average results. For the 7 PMAS contracts, the award fee plans permitted awards of an additional 60-70 percent of the award fee pool for some performance above standard while still allowing several weaknesses in performance to remain. In addition, we reviewed award fee files and found that the Award Fee Evaluation Board recommendations and determinations of fees were not documented in sufficient detail to show that the integrity of the award fee determination process has been maintained. The documentation we reviewed in contract files was insufficient to substantiate the award fees that were approved. On July 19, 2005, we provided the JCC-I/A, the PCO, and the U.S. Army Corps of Engineers Gulf Region Division Award Fee Determining Official a briefing on the interim results of our audit.1 At that time, these officials advised us that they were implementing new procedures that would address many of the corrective actions we identified. Since our July 19th briefing, the JCC-I/A and PCO have been proactive in implementing corrective actions we identified during our review, including interim recommendations. In fact, on July 23, 2005 JCC-I/A issued a new Award Fee Board Policy. Consequently, we consider two of the conditions noted in the interim briefing corrected, however, during the finalization of our audit, we identified conditions that continue to need to be addressed and make the following additional recommendations. Recommendations. We recommend that the Commanding General, Joint Contracting Command - Iraq/Afghanistan: 1. Ensure that the appointments of the members of the Award Fee Evaluation Board, specifically the Chairperson of the board, the board members (both voting and non- voting), and the contract performance monitors are documented. 2. Continue to revise award fee plans to more clearly identify the specific award fee evaluation criteria for assessing contractor performance by providing clearer metrics and more quantifiable criteria that permit better performance evaluations. ii 1 Interim Briefing to the Project and Contracting Office - Iraq and the Joint Contracting Command - Iraq on the Audit of the Award Fee Process, SIGIR-05-010, July 26, 2005. 3. Continue to review and modify the current contracts to utilize a method of applying the award fee to provide additional incentive for contractors to achieve quality results. 4. Ensure that the Award Fee Determination Officer’s and the Award Fee Evaluation Board’s recommendations and determinations are fully documented as to their rationale. Management Comments and Audit Response. The Joint Contracting Command – Iraq/Afghanistan concurred with four of five recommendations made in a draft of this report, and provided a reason for the non-concurrence. For management comments, see page 22. We agreed and changed our report accordingly. iii Table of Contents Executive Summary i Introduction Background 1 Objective 2 Finding Award Fee Process 3 Appendixes A. Scope and Methodology 11 B. Acronyms 13 C. Contract Files Reviewed 14 D. JCC-I/A Award Fee Board Policy, dated July 23, 2005 15 E. Report Distribution 19 F. Audit Team Members 21 Management Comments Joint Contracting Command - Iraq/Afghanistan 22 Introduction Background This audit report discusses the Award Fee Process for contracts funded from the Iraq Relief and Reconstruction Fund (IRRF). Public Law 108-106, as amended, mandates the Special Inspector General of Iraq Reconstruction (SIGIR) to engage in the independent and objective conduct of audits relating to the programs and operations funded with amounts appropriated or otherwise made available to the IRRF. It also requires SIGIR to provide for the independent and objective leadership and coordination of, and recommendations on, policies designed to promote economy, efficiency, and effectiveness in the administration of such programs and operations, and to prevent and detect waste, fraud, and abuse. Reconstruction Program. Title II of Public Law 108-106, “Emergency Supplemental Appropriations Act for Defense and for the Reconstruction of Iraq and Afghanistan, 2004,” authorized $18.4 billion for security, relief, rehabilitation, and reconstruction projects in Iraq funded through the IRRF. The Coalition Provisional Authority (CPA) established the Project Management Office (PMO) to execute the Iraq infrastructure reconstruction program. During its initial planning stages for the reconstruction of Iraq, the PMO identified over 5,000 potential Iraq infrastructure reconstruction projects that it potentially could execute. From that initial review, approximately 2,311 projects were selected for funding through the IRRF. Project and Contracting Office. The CPA was the authority responsible for the temporary governance of Iraq from April 2003 until June 28, 2004. Along with the CPA, the PMO ceased to exist on June 28, 2004, and the Project and Contracting Office (PCO) was established as its successor organization. The PCO is responsible for over $12 billion dollars of IRRF funds and has management control of hundreds of projects in the electricity, water, oil, health, transportation, education, security, and justice sectors. Joint Contracting Command - Iraq/Afghanistan. The Joint Contracting Command - Iraq/Afghanistan (JCC-I/A) provides operational contracting support to the Chief of Mission, PCO, and the Multi-National Force - Iraq (MNF-I). The Principal Assistant Responsible for Contracting to the Commanding General, JCC-I/A, has responsibility for the award fee plan and appointing the Award Fee Determination Official (AFDO). The AFDO is a senior procurement official with responsibility for appointing the Award Fee Board Evaluation Chairperson, Award Fee Evaluation Board, and performance monitors. United States Army Corps of Engineers, Gulf Region Division. United States Army Corps of Engineers, Gulf Region Division (GRD) provides construction management functions such as: construction execution and closeout; construction administration; property management; safety; quality assurance; schedule analysis; Administrative Contracting Officer delegation; and construction progress reporting to PCO. GRD also provides some construction management services to the MNF-I and the Multi-National Security Transition Command - Iraq. Federal Acquisition Regulation. Federal Acquisition Regulation, Subpart 16.3 and 16.4, entitled “Cost Reimbursement Contracts,” sets specific award fee requirements for cost-reimbursement contracts that provide for: (1) a base amount fixed at inception of the contract; and (2) an award amount that the contractor may earn in whole or in 1 part during performance and that is sufficient to provide motivation for excellence in such areas as quality, timeliness, technical ingenuity, and cost-effective management. The amount of the award fee is determined by the Government’s judgmental evaluation of the contractor’s performance based on the criteria stated in the contract. This determination and the methodology for determining the award fee are decisions that rest solely within the discretion of the Government. Defense Federal Acquisition Regulation Supplement (DFARS). Defense Federal Acquisition Regulation Supplement, Subpart 216.405-2, states that a cost-plus-award- fee (CPAF) contract is suitable for level-of-effort contracts where mission feasibility is established but measurement of better-than-expected achievement must be by subjective evaluation rather than objective measurement. Army Federal Acquisition Regulation Supplement (AFARS). The Army Federal Acquisition Regulation Supplement Sub-part 5116.4052, entitled, “Cost-plus-award- fee Contracts,” states the intended goal of award fee contracting is to motivate the contractor’s performance in those areas critical to program success that are susceptible to measurement and evaluation. By entering into an award fee arrangement, the contracting officer initiates a process that provides incentives to a contractor to perform at a superior level and requires the government to record assessments of the contractor’s performance upon which the award fee will be based. Objective The overall objective of this audit was to determine whether award fees are adequately reviewed, properly approved, sufficiently substantiated, and awarded according to established standards. Specifically, we sought to determine whether: • An award review board has been established, and pertinent policies and adequate procedures have been consistently applied to the evaluation of award fees. • Award fee plans clearly identified the specific award fee evaluation criteria for assessing contractor performance and determining the amount of the award fee. • The performance indicators were properly established, and the performance requirements were properly defined. • Award review board recommendations and determinations were supported by appropriate evaluations of contractor performance. • The award fee determination was documented in sufficient detail to show that the integrity of the award fee determination process has been maintained. For a discussion of the audit scope, methodology, and a summary of prior coverage, see Appendix A. For definitions of the acronyms used in this report, see Appendix B. For the list of contract files reviewed see Appendix C. For a list of the audit team members, see Appendix F. 2 Award Fee Process Award Fee Process Administration Criteria The 18 contracts and associated contract files reviewed consisted of 11 design-build (DB) contracts with a not-to-exceed value of up to $6.75 billion, and 7 program management and support (PMAS) contracts with a value in excess of $200 million. The contracts, contract files, and the Award Fee Evaluation Board’s (AFEB) summaries and supporting documentation were located at PCO and JCC-I/A. The Award Fee Plan for each contract describes the policies and procedures for determining award fees and outlines the duties and responsibilities of personnel associated with the award fee process. Documentation could not be located for the required appointment of the AFEB Chairperson, performance monitors, or for the other board members in the contracts. Moreover, the AFEB recommendations and determinations were not documented in sufficient detail to show that the integrity of the award fee determination process had been maintained. Additionally, the Award Fee Determination Official’s (AFDO) decisions were not documented in sufficient detail to show that the integrity of the award fee determination process has been maintained during the entire period reviewed. These conditions occurred because the PCO did not establish adequate administrative processes, controls, and criteria to properly administer the Award Fee Evaluation Board. We looked at two award fee periods governing the 18 contracts awarded in January 2004 through July 2005. For those periods, we found deficiencies in the following areas for each contract file reviewed: AFEB Chairperson: • 1st Award Fee Period: 9 of 18 contract files reviewed did not document the required chairperson appointment. • 2nd Award Fee Period: 7 of 18 contract files reviewed did not document the required chairperson appointment. Contract Performance Monitor: • 1st Award Fee Period: 14 of 18 contract files reviewed did not document any monitor appointments. • 2nd Award Fee Period: 12 of 18 contract files reviewed did not document any monitor appointments. AFEB Members: • 1st Award Fee Period: 13 of 18 contract files reviewed did not document any appointments. • 2nd Award Fee Period: 12 of 18 contract files reviewed did not document any appointments. 3 Moreover, limited evidence was found indicating that the required monthly contract assessments were performed. Performance Monitor’s Monthly Reports: • 1st Award Fee Period: 13 of 18 contract files reviewed showed no evidence of monthly monitoring being performed. • 2nd Award Fee Period: 12 of 18 contract files reviewed showed no evidence of monthly monitoring being performed. AFEB Monthly Meetings: • 1st Award Fee Period: 16 of 18 contract files reviewed showed no evidence of monthly meetings. • 2nd Award Fee Period: 13 of 18 contract files reviewed showed no evidence of monthly meetings. Additionally, self-assessments from contractors were not always documented in the contract files. • 1st Award Fee Period: 9 of 18 contract files reviewed showed no evidence of the contractor’s self-assessments. • 2nd Award Fee Period: 5 of 18 contract files reviewed showed no evidence of the contractor’s self-assessments. Because of the poor administration of contract files for award fees, contractors have successfully appealed adverse decisions made by the AFEB. For example, Contract No. W914NS-04-C-0007, awarded to Foster Wheeler, received an evaluation score of 15% at the end of the 1st award period (March – September 2004), which should have resulted in no award fee to the contractor. However, Foster Wheeler appealed the decision because of management’s failure to perform monthly performance meetings and the absence of the required performance feedback. The U.S. Government subsequently awarded Foster Wheeler a score of 50%, which resulted in an award fee of $439,145 that was paid from the award fee pool of $878,291. The final decision approving the award fee was issued in a contract modification that allowed payment of the award fee in February 2005. Award Fee Plan Criteria The Award Fee Plan in each contract describes the policies and procedures for determining award fees and outlines the duties and responsibilities of personnel associated with the award fee process. However, Award Fee Plans we reviewed did not clearly identify specific award fee evaluation criteria and performance indicators were not properly established. Specifically, we found that the evaluation criteria were subjective with few metrics being used. Schedule Adherence Criteria used undefined terms such as “timeliness, accuracy, and completeness of performance,” as the standard of review. Cost Control Criteria used general terms such as “maintain effective cost control measures” as the standard of review. Quality Control Criteria used measurement terms such as “minimization of rework” as the standard of review. The effect of having evaluation criteria without clearly defined metrics could have the potential to result in inflated contractor performance evaluations and consequently inappropriately approved award fees. 4 We reviewed 11 award fee contract files for design-build contractors and 7 award fee files for program management and support contractors. We found the following performance ratings for the 2nd award fee period, as approved by the AFEB for the 18 contracts: For the 11 Design-Build contract files: • 1 below average rating of 66.9%. • 1 average rating of 67.3%. • 4 above average ratings of 82.0% to 86.6%. • 5 excellent ratings of 93.8% to 96%. For the 7 PMAS contract files: • 1 very good rating of 78.3%. • 6 outstanding ratings of 93.5% to 98.7%. Cost-plus award-fee contracts allow the government to evaluate a contractor’s performance according to specific criteria and to grant an award amount within designated parameters. Award fees can serve as a valuable tool to help control program risk and encourage excellence in contract performance. The current award fee plan does not provide proper incentives for contractors to strive for better-than-expected results. For example, 16 of the 18 Award Fee Plans reviewed allow awards tied the amount of the award directly to the performance grade (e.g., a 70 percent performance score results in a 70 percent award from the fee pool), whereas the other two Award Fee Plans3 were scaled, allowing no award fee for a score less than 70 and rising proportionally from there. Per the Army FAR Supplement (AFARS 5116.405-2) “contractors should not receive an award fee (above the base fee) for simply meeting contract requirements.” The base fee is three percent of the negotiated contract/task order cost for 16 of the 18 contracts we reviewed; which is the highest base fee percentage allowed by the DFARS. For 7 of the 18 contracts (the PMAS contracts), the award fee plans allowed awards of an additional 60-70 percent of the award fee pool for performance above standard (but still with several weaknesses). For 9 of the 18 contracts (design-build contracts), the award fee plans allowed awards of 50-74 percent of the award fee pool for simply average results. The award fee plans allowed 50-60 percent of the award fee pool to be earned for meeting minimum requirements. For example, for the seven PMAS contracts, a minimum of 60 percent of the award fee pool is awarded if the contractor receives a performance score of 60 (which equates to some part of the performance above standard but with several weaknesses) and earns up to 100 percent of the award fee pool for earning a performance score of 100. This is illustrated in Chart 1. For 9 design-build contracts, a minimum of 50 percent of the award fee pool is awarded if the contractor receives a performance score of 50 (which equates into average results) 3 Contracts W9126G-04-D-0001 and W9126G-04-D-0002 have criteria that performance scores less than 70 percent receive no fees, and award fees begin at zero percent for meeting the specifications and increasing to 100 percent of the award fee for a performance score of 100. The remaining 16 contracts receive a percentage of the award fee pool equal to their performance score, provided the score is not unsatisfactory (i.e. performance score of 75 received 75 percent of award fee pool, 91 performance score receives 91 percents of the award fee pool, etc.) 5 and earns 100 percent of the award fee pool for earning a performance score of 100. The chart also illustrates the award fee pool awards used by W9126G-04-D-0001 (D- 001) and W9126G04-D-0002 (D-002) for their performance rating. Chart 1 - Based Upon Performance Rating Current Award Fee Plan - Percent of Award Fee Pool That Can be Earned 100 PMAS Contracts (7 of 18) Design-Build Contracts (9 of 18) Percent of fee awarded 80 D-001/D-002 Contracts (2 of 18) 60 40 20 0 0 10 20 30 40 50 60 70 80 90 100 Performance Score The Army Contracting Activity (ACA) Award Fee Contracts Guide identifies a more common and apt award fee mechanism called the “cubic scale.” This alternate method for distributing award fees more properly rewards superior performances and complies more fully with the Army FAR Supplement (AFARS 5116.405-2), which requires superior performance for an award fee. As illustrated in Chart 2, the cubic scale is similar to the award used for D-001 and D-002, although the cubic scale is slightly more generous in awarding fees at the lower scores than D-001/D-002. 6 Chart 2 - Comparative Analysis of the Current Scoring Scale to the Cubic Scale D-001/D-002 Cubic 100 80 PMAS Design-Build Percent of Award Fee 60 40 20 0 40 50 60 70 80 90 100 Performance Rating The cubic award fee scale or another award fee scale as deemed appropriate by the Contracting Officer could be applied through a unilateral modification to existing contracts prior to the start of a performance period. The effect of using the cubic scale to compute the award fee pool to be awarded for a corresponding performance score is shown by the following table, which illustrates the award fee that would have been awarded using the cubic award fee scale for awarding fee versus using the current methodology of awarding fee. Table 1: Comparison of Cubic Award Fee Scale verses Current Award Fee Methodology Award Fees Actual Award Using Cubic Fees Using Strategy for Current Fee Awarding Fee Strategy For the 11 Design-Build Contracts Reviewed: 1 below average rating of 66.9%* $ 0 $ 0 1 average rating of 67.3% 309,436 1,666,772 4 above average ratings of 84.7% to 86.6% 21,088,041 27,616,696 5 excellent ratings of 93.8% to 96% 20,536,016 21,062,281 For the 7 PMAS contracts reviewed: 1 very good rating of 78.3% 49,787 86,667 6 outstanding ratings of 93.5% to 98.7% 9,327,239 9,479,620 * No award fee is awarded for below average performance. 7 As is evident from Table 1, the contractors who exceed contract expectations make award fees approaching the maximum. The contractors whose award fees are affected most are those who perform at a marginal level or average level. Documentation of the AFEB’s Determination Criteria To determine whether the AFEB recommendations and determinations are supported by appropriate evaluations of contractor performance, we reviewed the supporting documentation provided to the AFDO to support the AFEB’s determination. The AFEB recommendations and determinations were not documented in sufficient detail to show that the integrity of the award fee determination process has been maintained. Specifically, we found that: • 15 of 18 contracts reviewed showed no evidence of either the monitor’s monthly reviews or the AFEB board monthly meetings with the contractor during the first award fee period as required by the award fee plan. • 12 of 18 contracts reviewed showed no evidence of either the monitor’s monthly reviews or the AFEB board monthly meetings with the contractor during the second award fee period as required by the award fee plan. In addition, we reviewed the method used by the AFEB to determine the award fee on the Design Build contracts that involved multiple Task Orders (TO). Each TO has its own award fee pool, and thus, each TO should be evaluated individually. For the first award fee period, the contracts had not matured to the point of having significant TOs under each contract, so we took no exception for this period. However, during the second award fee period that we reviewed, we found no evidence that the AFEB looked at the TOs on an individual basis and found no evidence that the AFEB weighed the TOs as to their significance in award dollars (to the total contract value) in 10 of the 18 contracts reviewed. Thus, without sufficient evidence to document and support the awarding of award fees under each TO, we were unable to ensure the validity of the fees or the integrity of the award fee determination process. Documentation of the Award Fee Determination Criteria Overall, the award fee determinations we reviewed were not documented in sufficient detail to demonstrate that the integrity of the award fee determination process had been maintained. • 16 of the 18 contracts reviewed showed no evidence that the AFDO decisions were documented in sufficient detail to maintain the integrity of the Award Fee Process for the first award fee period. • 7 of the 18 contracts reviewed showed no evidence that the AFDO decisions were documented in sufficient detail to maintain the integrity of the Award Fee Process for the second award fee period We did note that during the second award fee period, improvements had been made regarding the level of contract file documentation maintained by the AFDO. In addition to the written correspondence that either approved or disapproved the AFEB decision, the current AFDO documented the meetings held with the AFEB regarding their recommendations, the rationale behind his decisions, and made his personal electronic and written files available for our review. 8 Management Actions JCC-I/A and PCO have been proactive in implementing corrective actions that we identified. Notable examples include: • Contract files have been indexed and filing system improved. • JCC-I/A has conducted additional training regarding Award Fees and the process to use for personnel involved in evaluating contractor performance. • JCC-I/A issued an Award Fee Board policy dated July 23, 2005, to provide guidance on the responsibilities for award fee evaluation and administration – see Appendix D. This policy include provisions that: o Emphasize that the contractor earns an award fee by performing the work requirements as stated in the contract and by excelling in the areas specified in the award fee plan. o The contractor begins each evaluation period with 0% of the available award fee and works up to the evaluated fee for each evaluation period, the PMAS contractors serve primarily as performance monitors and provide input to the AFEB on contractor performance. o The AFEB shall be composed of only Government personnel and that performance monitors and other technical experts serve in advisory capacity only and may not participate during AFEB discussions. o Rollover of fees to successive periods is normally not permitted other than for reasons such as Government-caused delay or changes to the performance work statement. o The Contracting Officer is an AFEB member to ensure contract integrity, contract compliance, and centralization of contract files. • JCC-I/A has coordinated with PCO and the AFDO, and has notified the contractors of the intent to change the award fee plans for the next award fee evaluation period to ensure that issues such as property management, inventory control, and changes in the rating and applications of award fees will be incorporated into future award fee plans.4 • JCC-I/A is in the process of revising the award fee scales to provide added incentives to encourage excellence in contract performance; a performance evaluation of less than good will no longer earn award fee. Because management actions were still ongoing at the time our audit work was concluded, we could not assess the ultimate efficacy of all those actions. In our opinion, however, two of the cited conditions have been corrected and the results of these remaining actions probably will aid in the correction of the other conditions identified in this report. Conclusion Although JCC-I/A and PCO have been proactive in improving the award fee process, we believe there are additional opportunities that can be made by leveraging the award fee pool by applying an alternate methodology (e.g., cubic scale, improved metrics and criteria) that is transparent to determine award fees in a way that 4In response to Interim Briefing to the Project and Contracting Office - Iraq and the Joint Contracting Command - Iraq on the Audit of the Award Fee Process, dated July 19, 2005. 9 encourages contractors to demonstrate above-expectations efforts toward accomplishing the tasks and functions cited in the contract to reward performance excellence. Since our July 19th briefing, the JCC-I/A has been proactive in implementing corrective actions we identified during our review, including interim recommendations. In fact, on July 23, 2005, JCC-I/A issued a new Award Fee Board Policy (see Appendix D). Consequently, two of our recommendations in the interim report have already been implemented. However, during the finalization of our audit, we identified conditions that continue to need to be addressed and make the following additional recommendations. Recommendations, Management Comments, and Audit Response We recommend that the Commanding General, Joint Contracting Command - Iraq/Afghanistan: 1. 2. 3. 4. Ensure the appointments of the members of the Award Fee Evaluation Board, specifically the Chairperson of the board, the board members (both voting and non-voting), and the contract performance monitors are documented. Continue to revise award fee plans to more clearly identify the specific award fee evaluation criteria for assessing contractor performance (i.e. more metrics, and/or quantifiable criteria to distinguish between a poor performer and an excellent performer). Continue to review and modify the current contracts to utilize a method of applying the award fee to provide additional incentive for contractors to achieve quality results. Ensure that the Award Fee Determination Official’s and the Award Fee Evaluation Board’s recommendations and determinations are appropriately documented as to their rationale. Management Comments. The Joint Contracting Command - Iraq/Afghanistan concurred with four of five recommendations made in a draft of this report, and provided a reason for the non-concurrence. For management comments, see page 22. Audit Response. We agreed and changed our report accordingly. 10 Appendix A. Scope and Methodology We initiated this audit because Mr. Stuart W. Bowen, Jr., Special Inspector General for Iraq Reconstruction met with the AFDO, Mr. James Dalton, Director of Business Management, United States Army Corps of Engineers, Gulf Region Division (GRD), to discuss the Award Fee Process. Mr. Dalton identified several deficiencies occurring in the award fee process during the meeting as follows: • The current procedure of awarding fees by the AFEB is that the contractor starts at 100 percent of the award fee pool, decreases the percentage if problems are found with the contractor’s performance. This is contrary to the concept of award fees. The award fee process should operate according to the award fee plan, that is, award fees are awarded based upon the performance score which ranges from average performance to excellent (performance scores of less than average receive no award fees.) • Previous AFEB comments have not been sufficient to justify the contractor’s performance, either good or bad. • Unearned fee was carried over into the next evaluation period by the previous AFDO. Mr. Dalton did not believe this was the correct process, that the contractor should do their best to earn the entire award fee during the period being rated, and no carryover will be provided, except for elements of the award fee plan that cannot be performed during the rating period. Mr. Dalton identified the 18 contracts that were Cost Plus Award Fee (CPAF) contracts. We decided to review all 18 contracts. The 18 contracts reviewed consisted of 11 design-build contracts valued at up to $6.75 billion, and 7 program management and support (PMAS) contracts valued in excess of $200 million. These 18 contracts were awarded by six different procuring activities. The on-site review of the AFDO’s files was conducted from the 1st and 2nd Award Fee periods, at the PCO Compound in June 2005. We also evaluated documents, procedures, and processes at the JCC-I/A. We reviewed the hard contract files and electronic contract files including emails, WORD documents, EXCEL spreadsheets, and PDF files not contained in hardcopy format at the JCC-I/A between June 11, 2005 and June 25, 2005. We conducted interviews with JCC-I/A managers and staff to discuss contract and task order procedures with key personnel at both JCC-I/A and the PCO. As a basis for our review and evaluation, we used: • Provisions of Public Law 108-106, “Emergency Supplemental Appropriations Act for Defense and for the Reconstruction of Iraq and Afghanistan, 2004”. • Federal Acquisition Regulation Part 16.3 and 16.4, “Cost-reimbursable Contracts”. • Defense Federal Acquisition Regulations 216.405-2, Cost-plus-award-fee. • Army Federal Acquisition Regulations 5116.405-2, Cost-plus-award-fee contracts. • Procedural guides from the Army Contracting Agency. 11 We conducted interviews with the Head of the Contracting Activity and the Principal Assistant Responsible for Contracting, a Sector Chief, five contracting officers, and two contract specialists. Our review included contracts with effective dates from January 2004 through July 2005. In addition, we made inquires to the other recipients of IRRF monies, specifically, the U.S. Agency for International Development, the Multi-National Security Transition Command – Iraq, the Air Force Center for Environmental Excellence, Gulf Region Division, and the Department of State. Except for the Department of State, who did not respond to our inquiries, none of the recipients utilized CPAF contracts at the time of this review. However, based upon our review of available IRRF contract databases, we believe that any CPAF contracts awarded by the Department of State were not significant to this review. Thus, we used our universe of 18 contract files to test the Award Fee process by determining whether the minimally required documents were contained in each file. We performed this performance audit in accordance with generally accepted government auditing standards. Use of Computer-Processed Data. perform this audit. Prior Coverage. We did a search of the Government Accountability Office (GAO), the Office of Inspector General Department of Defense (DoDIG), and the Special Inspector General for Iraq Reconstruction (SIGIR - formerly the Office of the Inspector General, Coalition Provisional Authority), NASA, and Army Audit Agency web sites for “Cost Plus Award Fee,” type reports. The GAO reports can be accessed over the Internet at http://www.gao.gov. The DoDIG reports can be assessed at http://www.dodig.osd.mil/audit/reports. SIGIR reports can be located over the internet at http://www.sigir.mil. The search was conducted for dates July 2000 to August 2005, using “Cost Plus Award Fee,” as the search keywords. There were no recent GAO, DoDIG, or SIGIR reports on Cost Plus Award Fee contracts. One Army Audit Agency report was provided in hardcopy and is referenced below. Army Audit Report No. 01-169, “Best Practices for Using Award Fees.” February 20, 2001. We did not use computer-processed data to 12 Appendix B. Acronyms ACA Army Contracting Activity AFDO Award Fee Determination Official AFEB Award Fee Evaluation Board CPA Coalition Provisional Authority CPAF Cost Plus Award Fee DB Design-Build FAR Federal Acquisition Regulation GRD Gulf Region Division IRRF Iraq Relief and Reconstruction Fund JCC-I/A Joint Contracting Command – Iraq/Afghanistan MNF-I Multi-National Force - Iraq PCO Project and Contracting Office PMAS Program Management and Support PMO Project Management Office SIGIR Special Inspector General for Iraq Reconstruction TO Task Order 13 Appendix C. Contract Files Reviewed The 18 contract files reviewed during this audit are listed below. Contract Value Contract Number Contractor Sector 7 June 2005 W914NS-04-C-0001 AECOMS PCO Services $ 40,466,723 W914NS-04-C-0002 Iraq Power Alliance PMAS Electrical 49,202,499 W914NS-04-C-0003 CH2M Hill/Parsons PMAS Public Works & Water 67,001,082 W914NS-04-C-0004 Berger, URS PMAS Facilities & Transportation 3,000,000 W914NS-04-C-0005 Berger, URS PMAS Facilities & Transportation 4,000,000 W914NS-04-C-0006 Berger, URS PMAS Facilities & Transportation 5,553,043 W914NS-04-C-0007 Foster Wheeler PMAS Oil 29,802,165 W912G6-04-D-0001 KBR Oil 763,077,645 W912G6-04-D-0002 Parsons/Iraq Oil 267,139,474 W914NS-04-D-0003 Fluor-AMEC Electrical Generation 193,839,567 W914NS-04-D-0005 Lucent Facilities, Transportation. & Communications 142,497,133 W914NS-04-D-0006 Parson's Delaware Facilities, Transportation. & Communications 382,492,138 W914NS-04-D-0007 Washington International Public Works & Water 86,131,671 W914NS-04-D-0008 Fluor-AMEC Public Works & Water North 41,557,713 W914NS-04-D-0009 Parson's Delaware Facilities, Transportation. & Communications 381,192,583 W914NS-04-D-0010 Washington International Electrical T/D North 368,515,403 W914NS-04-D-0011 Perini Electrical T/D South 291,323,523 W914NS-04-D-0022 Fluor-AMEC Public Works & Water South 15,768,393 TOTAL $ 3,331,586,267 14 Contract Issue Date 3/10/2004 3/10/2004 3/10/2004 3/10/2004 3/10/2004 3/10/2004 3/11/2004 1/16/2004 1/16/2004 3/11/2004 3/23/2004 3/25/2004 3/11/2004 3/23/2004 3/26/2004 3/12/2004 3/12/2004 3/23/2004 Contract Issued By Pentagon Renovation Office Pentagon Renovation Office Pentagon Renovation Office Pentagon Renovation Office Pentagon Renovation Office Pentagon Renovation Office Pentagon Renovation Office United States Agency for International Development Army Corps of Engineers Coalition Provisional Authority Coalition Provisional Authority Department of the Army Coalition Provisional Authority Naval Facilities Engineering Coalition Provisional Authority Coalition Provisional Authority Coalition Provisional Authority Naval Facilities Engineering Appendix D. JCC-I/A Award Fee Board Policy US DEPARTMENT OF THE ARMY JOINT CONTRACTING COMMAND-IRAQ/AFGHANISTAN APO AE 09316 Reply to: JCC-I/A/A-R 23 July 2005 MEMORANDUM FOR: JCCI-R Sector Chiefs / Contracting Officers / Award Fee Determining Official / Award Fee Board Chairmen / PCO Program Managers SUBJECT: Award Fee Board Policy 1. The purpose of this policy is to provide information and guidance on the responsibilities for award fee evaluation and administration on JCC-I/A/A-R CPAF contracts. Contracting officers and those who participate during award fee management in the Iraq theater may change several times during the extended performance of IDIQ contracts for reconstruction and construction management services. This policy is an effort to maintain continuity during transitions which will inevitably occur. This policy does not restrict or deviate from any requirement of the FAR, DFARS, or AFARS. A more in-depth review of award fee contracts is provided in the ACA Handbook for “Award Fee Contracts” and is posted on the ACA web site at http://aca.saalt.army.mil/. We strongly encourage that anyone administering a CPAF contract or participating on an AFB to read or review this guide periodically. 2. Under the terms of a cost reimbursement contract, the Government reimburses for a “best effort” where the Government assumes all or most of the performance risk. Award fee provisions are added to a cost reimbursement contract to provide an incentive to excel in critical performance areas emphasized in an Award Fee Plan (AFP). The award fee process evaluates a contractor’s performance level and rewards superior performance by providing a “bonus” based upon evaluated factors incorporated in the contract AFP. 3. The CPAF type contract provides for a fixed portion of the fee (base fee) to be paid for compliance when meeting all term and conditions of the contract and by an additional award fee to be earned based upon level of performance tied to critical evaluation factors contained in the AFP. It is important to emphasize that the contractor earns an award fee by performing the work requirements as stated in contract and by excelling in the areas specified in the AFP and not by doing what the program manager or other government personnel may want of the contractor on a particular day. Government personnel must avoid the temptation to try to get “extra” work using the subtle threat of withholding award fee. 4. Available award fee during the evaluation period must be earned. The contractor begins each evaluation period with 0% of the available award fee and works up to the evaluated fee for each evaluation period. Contractors do not begin with 100% of available fee and have deductions withdrawn to arrive at the evaluated fee. 15 5. The AFP and the managing structure should avoid becoming an overly structured process. Simplicity is the key – a simple plan is more direct, easier to follow and simpler to defend. It should be based on a few critical factors that best reflect desired end product. A simple plan also reduces layers of review. The AFP establishes the methods that the Government will use to assess performance and includes performance evaluation criteria. The AFP is a living document and, due to changing priorities, may change during contract performance. The AFP may be changed unilaterally by the Government if the change occurs at least 30 days before the beginning of the next evaluation period. If the change occurs during the evaluation period, then it must be bilateral, i.e. a negotiated agreement between the parties. The plan also establishes the standard to measure how well a contractor is performing. 6. AF evaluations and earned fee determinations are based upon subjective performance evaluations provided by the Award Fee Evaluation Board (AFEB) and an Award Fee Determining Official (AFDO). It is important that the personnel involved with the AF process understand their roles and responsibilities. This team includes the program manager, performance monitors (PMAS contractors or CORs), the AFEB and the AFDO. 7. The program manager has responsibility for the performance work statement (PWS) technical function. The PM maintains overall technical responsibility for the mission and provides technical oversight and guidance to the performance monitors. 8. The PMAS contractors serve primarily as performance monitors and provide input to the AFEB on contractor performance. The monitors provide continuous evaluation and assessment of the contractor’s daily performance. This daily monitoring and regular reporting are the foundation of the award fee evaluation process. The monitor must be familiar with the contract requirements and the rating criteria of the AFP. Performance monitors are not members of the AFEB. 9. The AFEB is responsible for review of the performance monitors’ evaluations, the contractor’s self-evaluation, and any other information to arrive at an overall objective and impartial position on the contractor’s performance. The AFEB shall be composed of only Government personnel. The AFEB will consist of a chairperson, the contracting officer and up to five additional voting members. The contracting officer will be a voting member. All members will be experts in their fields. In addition there will be a recorder who is a non- voting member. The recorder coordinates the administrative actions required by the AFDO, AFEB and performance monitors. The AFEB members must be familiar with the award fee process, contract requirements, and the AFP. Performance monitors and other technical experts serve in advisory capacity only and may not participate during AFEB discussions. 10. The AFDO is independent of the AFEB and makes the final decision that determines the amount of award fee earned during the evaluation period. The AFDO will be appointed in accordance with the AFP and should be a staff member at a senior level. The AFDO will be designated by position title rather than by name in the AFP. The AFDO becomes involved in the evaluation process only after the AFEB has met and agreed on a recommended award fee amount. The AFDO’s role is to independently evaluate the AFEB recommendation, to consider any contractor self-assessment, and to render a final decision on the amount of fee earned for the period. The AFDO’s final decision must be documented and provided to the contractor. If the decision varies upward or downward from the AFEB’s recommendation, the rationale for the change shall be documented in the contract file (not with the decision letter) and related back to the AFP. The AFDO letter should be provided to the contractor within 5 days after the decision and simultaneously with the modification signed by the contracting officer to obligate the fee amount. The contractor can then immediately invoice for the earned fee. 16 11. The board processes and timelines will be established in the AFP. All members should be prepared and available for the scheduled session(s). The chairperson or recorder will generally set the meeting date and assure that all documents are ready to review. The contracting officer will notify the contractor of the date, time and location where the contractor may present a self-evaluation to the AFEB. The performance monitors provide surveillance observations for the period to the program manager who will then provide this information to the AFEB. The AFEB chairperson should meet monthly with the performance monitors to ensure that they are documenting performance. Feedback should also be provided to the contractor on a monthly basis to identify negative performance trends and to give the contractor an opportunity to take corrective action. 12. The AFEB will convene within two weeks after the end of the rating period in order to complete the process within 45 days from the end of the rating period. The chairperson should arrange for refresher training or training of new board members prior to the meeting if needed. Once the board is in session the chairperson should provide a brief overview of the evaluation criteria, purpose of the meeting, and introduction of any new members. The contractor will be allowed to present a self-evaluation to the AFEB with a follow-up written analysis. The board members may ask questions, but will not discuss the Government surveillance reports with the contractor. Performance monitors will not attend the board meeting. This is a fact-finding meeting and should be conducted in a non-adversarial manner. 13. After completion of the contractor self-evaluation, the board will reconvene and discuss the performance monitor’s evaluation report along with any other information relating to contractor performance. The task is then to decide how well the contractor performed in each area using the metrics provided in the AFP. Voting members will individually score contractor performance in writing. These individual reports will be submitted to an assigned individual for consolidation into a single weighted and rated recommendation for award fee. The consolidated rating will be included in the AFEB report to the AFDO along with a synopsis of strengths and weaknesses, the contractor’s self-evaluation, and any other key factors that were considered. 14. The AFDO will review the recommendation for award fee and, if necessary, request clarification of issues. The AFDO will determine the dollar amount and provide a written analysis of the contractor’s strengths, weaknesses and award fee amount for distribution to the contractor. This process should be complete within 45 days after the end of the rating period. Once the decision has been made the Contracting Officer will unilaterally modify the contract to obligate funds. (NOTE: In accordance with DFAS-IN 37-1, a CPAF contract is to be obligated upon award less the award fee amount.) Once the AF determination is made, the Prompt Payment Act becomes effective so timely modification is critical. 15. The definition of the award fee pool is the total dollars available for each evaluation period. The award fee allocated for each evaluation period is the maximum amount that can be earned during that period. This amount is established prior to award of the contract or task order and included in the AFP. Changes to this amount may occur only as a result of new work negotiated via supplemental agreement to an existing task order or definitization of a new task order. 16. Rollover of fees to successive periods is normally not permitted other than for reasons such as Government-caused delay or changes to the performance work statement. The reason for rollover of fee must be documented in the file. If the unearned fee is rolled over, the contracting officer must monitor the appropriation for violation of the “bona fide need rule”. However, the Government may move a portion of fee from one performance period to another 17 so long as it is not associated with failure to earn fee in previous evaluation periods. may be done unilaterally if done before the start of the next evaluation period. This 17. Continuous communication with the contractor during the evaluation period is critical. Meetings should be conducted monthly and attended by the contractor, performance monitors and members of the AFEB. Informal monthly evaluation reports will be prepared; however, these reports will not be scored or assigned any type of rating. These reports should be used when the AFEB convenes for the formal evaluation at the end of the award fee period. /s/ Edwin H. Martin Colonel, USA Principal Assistant Responsible for Contracting 18 Appendix E. Report Distribution Department of State Secretary of State Senior Advisor to the Secretary and Coordinator for Iraq U.S. Ambassador to Iraq Director, Iraq Reconstruction Management Office Inspector General, Department of State Department of Defense Secretary of Defense Deputy Secretary of Defense Director, Defense Reconstruction Support Office Under Secretary of Defense (Comptroller)/Chief Financial Officer Deputy Chief Financial Officer Deputy Comptroller (Program/Budget) Inspector General, Department of Defense Department of the Army Assistant Secretary of the Army for Acquisition, Logistics, and Technology Principal Deputy to the Assistant Secretary of the Army for Acquisition, Logistics, and Technology Deputy Assistant Secretary of the Army (Policy and Procurement) Director, Project and Contracting Office Commanding General, Joint Contracting Command – Iraq/Afghanistan Assistant Secretary of the Army for Financial Management and Comptroller Chief of Engineers and Commander, U.S. Army Corps of Engineers Commander, Gulf Region Division Auditor General of the Army U.S. Central Command Commanding General, Multi-National Force – Iraq Commanding General, Multi-National Security Transition Command – Iraq Commander, Joint Area Support Group – Central Other Defense Organizations Director, Defense Contract Audit Agency 19 Other Federal Government Organizations Director, Office of Management and Budget Comptroller General of the United States Inspector General, Department of the Treasury Inspector General, Department of Commerce Inspector General, Department of Health and Human Services Inspector General, U.S. Agency for International Development Mission Director – Iraq, U.S. Agency for International Development Congressional Committees and Subcommittees, Chairman and Ranking Minority Member U.S. Senate Senate Committee on Appropriations Subcommittee on Defense Subcommittee on State, Foreign Operations and Related Programs Senate Committee on Armed Services Senate Committee on Foreign Relations Subcommittee on International Operations and Terrorism Subcommittee on Near Eastern and South Asian Affairs Senate Committee on Homeland Security and Governmental Affairs Subcommittee on Federal Financial Management, Government Information and International Security Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia U.S. House of Representatives House Committee on Appropriations Subcommittee on Defense Subcommittee on Foreign Operations, Export Financing and Related Programs Subcommittee on Science, State, Justice and Commerce and Related Agencies House Committee on Armed Services House Committee on Government Reform Subcommittee on Management, Finance and Accountability Subcommittee on National Security, Emerging Threats and International Relations House Committee on International Relations Subcommittee on Middle East and Central Asia 20 Appendix F. Audit Team Members The Office of the Assistant Inspector General for Auditing, Office of the Special Inspector General for Iraq Reconstruction, prepared this audit report. The staff members who contributed to the report include: Tony L. Adams James A. Carrera Ronald L. Rembold 21 Management Comments Joint Contracting Command - Iraq/Afghanistan 22 23