Audit of the Canadian Surface Combatant Project

November 2015

7050-11-52 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

ADM(Mat)

Assistant Deputy Minister (Materiel)

ADM(RS)

Assistant Deputy Minister (Review Services)

AOPS

Arctic Offshore Patrol Ship

C Navy

Chief of the Naval Staff

CFDS

Canada First Defence Strategy

CID

Capability Investment Database

CRS

Chief Review Services

CSC

Canadian Surface Combatant

DND

Department of National Defence

DRDC

Defence Research and Development Canada

ELMS

Engineering Logistics Management Support

LCC

Life Cycle Cost

NATO

North Atlantic Treaty Organization

OPI

Office of Primary Interest

RCN

Royal Canadian Navy

SOR

Statement of Operational Requirements

SRB

Senior Review Board

US

United States

NSPS

National Shipbuilding Procurement Strategy

Results in Brief

Overall Assessment

As the largest procurement in Canadian history, the CSC project is in the midst of balancing the capabilities of a fleet of ships that have not yet been designed within a known project budget. Opportunities to advance the project schedule will need to be exploited, with regular updates to key stakeholders as better information becomes available.

In 2010, Chief Review Services (CRS) 1 conducted the Analysis of Capital Equipment Projects to determine which projects warranted audit attention. The Canadian Surface Combatant (CSC) project was determined to be of higher risk due to a number of factors. The objective of this audit was to assess the adequacy of the governance processes and management controls in place for the CSC project to ensure a cost-effective and timely operational capability.

The CSC project objective was to recapitalize the Canadian Forces surface combatant fleet by replacing three Iroquois Class destroyers and 12 Halifax Class ships in accordance with the 2008 Canada First Defence Strategy (CFDS). In June 2010, the National Shipbuilding Procurement Strategy (NSPS) was initiated to renew Canada’s federal fleet while ensuring stability of the Canadian shipbuilding industry. In October 2011, the shipyard selected to build the CSC vessels was announced. The definition phase, which aims to further define the CSC requirements and deliver a design for construction, began in June 2012. There is considerable design uncertainty at the outset of the project in an environment with emerging technologies and a changing threat for ships that may not be delivered until the late 2020s. The design of the ships, which will take at least five years, is a continual process that will consider the affordability of the ships’ requirements. With known cost and capability pressures due to a project cap of $26.2 billion, set in 2008 without the benefit of a ship design, there was a change to the project scope in November 2013 from the acquisition of “15 ships” to “up to 15 ships.” 2

During the project options analysis phase, project staff benchmarked with allies and conducted an options analysis study to determine a rough order magnitude estimate 3 in 2008. A number of operational research studies were conducted to determine the preliminary Statement of Operational Requirements (SOR), which was endorsed in 2010 and continues to evolve. Market surveys were conducted in 2011 to determine the affordability of the project. Once the definition phase was approved, one year of technical consultations began in 2013 with 140 marine industry companies to develop more detailed technical specifications.

Since the selection of the designated shipyard as the intended prime contractor in January 2015, the project office has been developing an integrated project schedule with the shipyard. As well, the shipyard is now involved in the project governance activities. Two consulting companies are being employed by the shipyard to reconcile the requirements with the project budget. With the April 2015 decision on a CSC procurement strategy, documentation has been prepared to pre-qualify contractors who are capable of integrating the ships’ combat systems and other contractors capable of designing the ship. Solicitation documentation is in preparation in order to select one of each type of contractor to participate in a design contract that should be in place by 2017, for the largest procurement in Canadian history. Construction of the first ship should begin in the early 2020s.

Findings and Recommendations

Impact of Fleet Size. The impact has not yet been fully assessed and communicated, should the CSC project deliver fewer than 15 ships. Design options and the affordability of capability requirements are now being assessed by the intended prime contractor, the project office and the Royal Canadian Navy (RCN). Fleet size may also be impacted by further operational research on such subjects as anti-submarine warfare.                                                                                                                                                                                                                                                                                 This could                                                                                                                                     simultaneously maintaining the                                                                  

It is recommended that Chief of the Naval Staff (C Navy) complete additional operational research as the CSC design evolves, then assess and communicate the impact of different fleet sizes to the Chief of the Defence Staff, and develop options to facilitate future decision making.

Preliminary SOR. The preliminary SOR will continually evolve into the early 2020s. Since 2008, several significant changes have been made to the CSC requirements, which are now well substantiated by extensive operational research, as shown in the most recent SOR. However, there is no definition of the size of the naval task group, which is the fundamental tactical formation for the RCN. As the design of the CSC evolves, this definition will be a key consideration in determining the size of the CSC fleet.

It is recommended that C Navy revise the SOR to address the various sizes and compositions of naval task groups as the capability of the CSC evolves.

Project Schedule. The project team is actively strengthening the schedule management controls. However, continuous rigour is required to determine whether the project is on track to meet the early-2020s construction milestone. For example, when compared to the less complex Arctic Offshore Patrol Ship (AOPS) project schedule, the CSC milestones reflect a compressed schedule. Many of the schedule revisions for ship delivery were due to factors beyond the project office’s control, such as the building of two combat ship projects at the same shipyard. It was decided in 2013 that the AOPS would be constructed before the CSC fleet. At that time, these factors postponed the planned delivery of the first CSC ship from 2020 to 2025. Ultimately, in due time the Navy will have to manage the schedule gap between the extended Halifax Class hull design life and its replacement by the CSC fleet,                                                     4                      

It is recommended that Assistant Deputy Minister (Materiel) (ADM(Mat)) continue to focus rigour on the schedule, track the impact of project changes, and identify time-saving measures with regular briefings to the Project Leader. The Senior Review Board (SRB) should also be apprised of schedule issues on a routine basis, along with the impact of changes on long-term milestones.

Information for Decision Making. There has been collaboration with the NSPS Office to gain further specialized technical support for the project. However, the content of the most recent project brief could be improved. To expedite the project’s approval, the brief should include the definition of a naval task group size, the impact of delays on the Halifax Class and Iroquois Class ships’ replacement schedule gaps, and the limitations of the Esquimalt jetties’ length. Moreover, at the time of the audit, project information in the Capability Investment Database (CID) monthly progress report and in costing documents was outdated. For the project implementation phase, the project office will be smaller than benchmarks with project office models. The Canadian Patrol Frigate project office, which delivered the 12 Halifax Class ships, was almost double the size of the CSC project office during the implementation phase.5

It is recommended that the organization of the project office for the implementation phase be compared to that of the Canadian Patrol Frigate project office, to ensure that all key roles and responsibilities are accounted for. The following project documentation should also be updated to address the audit observations on the project brief: the CID monthly progress report and the life cycle cost (LCC) estimate assumptions.

Contract Management. Holdback and timesheet terms and conditions in the Engineering Logistics Management Support (ELMS) contract were not enforced. The timesheet and holdback clauses were included to provide cost and quality controls.

It is recommended that ADM(Mat) ensure that the ELMS contract provisions for timesheets and holdbacks are enforced.

Note: Please refer to Annex A—Management Action Plan for the management response to the ADM(RS) recommendations.

1.0 Introduction

1.1 Background

NSPS. The CSC project is one of the several Department of National Defence (DND) and Canadian Coast Guard projects governed by the NSPS interdepartmental deputy minister and assistant deputy minister committees chaired by the contracting authority.6 The last major shipbuilding effort in Canada was during the 1990s, with the delivery of 12 Halifax Class ships. The NSPS was announced in June 2010 to renew Canada’s federal fleet, while ensuring continuity of the Canadian shipbuilding industry. This initiative to build the ships in Canada was estimated by the Canadian Association of Defence and Security Industries to generate approximately 15,000 jobs and $2.4 billion in economic benefits over the next 30 years.7 The NSPS competitive process selected the two designated shipyards in 2011—one for combat ships, such as the CSC, and another for non-combat ships. The combat shipyard will first build the AOPS, starting in 2015, and begin CSC construction in the early 2020s. In January 2015, the combat shipyard was selected as the intended prime contractor for the CSC fleet. With respect to the CSC procurement strategy, in April 2015, it was decided to compete for a single design team rather than tender for two competing ship designs.8 The non-combat shipyard will build the Joint Support Ships along with a number of Canadian Coast Guard ships. While major ship construction will be directed to the two designated shipyards, small vessels worth an estimated $2 billion will be competitively tendered amongst Canadian shipyards other than those selected to build large ships.9

Project Scope. The CSC project obtained revised project approval identification in January 2009. The CFDS included the requirement to replace the 3 Iroquois Class destroyers and 12 Halifax Class frigates for a total of 15 ships.10 The definition phase, necessary to further define the requirements and design the ships, began in June 2012. This definition phase will reconcile the capability requirements with affordability in a continual process that results in detailed design and production specifications, and it should be completed in the early 2020s. As reported in the May 2013 CRS Internal Audit of Capital Project Cost Estimation and the fall 2013 Office of the Auditor General audit of NSPS,11 the project faces considerable cost pressures. A budget cap at $26.2 billion was set in 2008 for a ship project that has no existing design and that is the largest procurement in Canadian history. Recognizing the cost constraints, in November 2013, there was a change to the project scope from the acquisition of “15 ships” to “up to 15 ships.”12

Project Activities. Since 2006, RCN staff have been dedicated to the CSC project with a view to determining its high-level requirements and rough order magnitude estimates. By 2008, a benchmark with allied ship projects combined with the CSC Options Analysis Study led to an initial cost estimate of $26.2 billion. The draft high-level requirements were endorsed by the Joint Capability Requirements Board by 2009, and an early version of the preliminary SOR was endorsed by Chief Force Development in 2010. To further develop the SOR, the CSC project office requested operational research studies be conducted by Defence Research and Development Canada (DRDC), the Canadian Forces Maritime Warfare Centre, and the ELMS contractor. A market survey was conducted in 2011 to determine the affordability of the project within the budget cap.

The definition phase began in 2012, when the project office started consultations with industry to develop a procurement strategy and to write CSC technical specifications. By 2015, the project office had completed procurement and technical consultations with 140 companies. With input from industry, further refinement of the requirements resulted in version 7.0 of the SOR in October 2014, which will continue to evolve into the early 2020s. Since January 2015, the project office has been working with the designated shipyard to develop an integrated master schedule. An effort to reconcile requirements and affordability has also been initiated by the intended prime contractor, while working with the project office and the RCN. The project office plans to have a design contract 13 in place for 2017, and has been developing solicitation documentation to begin the tendering process. A preliminary design is planned to be complete by 2019 with a view to beginning construction of the first ship in the early 2020s.

Rationale for Audit. ADM(RS) conducts risk analyses to identify equipment projects with attributes of higher risk that warrant an audit. The CSC project was so identified in the 2010 Analysis of Capital Equipment Projects. Due to the high value of the project, the Departmental Audit Committee directed CRS to include the CSC project in the 2011/12 to 2013/14 Risk-Based Audit Plan. The 2013 Office of the Auditor General audit of the NSPS delayed the conduct of this audit until 2014.

1.2 Objective

The objective of the audit was to assess the adequacy of the governance processes and management controls in place for the CSC project to ensure a cost-effective and timely operational capability.

1.3 Scope

The audit scope included the 2006 CSC Options Analysis Study along with a focus on future planned activities, including in-service support and in-process documentation up to January 2015. Included in the audit scope was an impact assessment of interdependent projects, such as the Maritime Helicopter, Joint Support Ship, and the Halifax Class Modernization/Frigate Life Extension, on the CSC project. Payments worth $36.8 million made by the project office between November 2008 and March 2014 were considered for review. The conduct phase of the audit took place between June 2014 and January 2015. Subsequent events up to May 2015 were considered in the finalization of the audit.

The audit excluded an assessment of risk management, which was assessed in a horizontal CRS Internal Audit of Project Risk Management Practices in 2013. Likewise, a review of CSC project acquisition cost estimates was excluded from this audit due to the 2013 CRS horizontal audit of Capital Project Cost Estimation, which reviewed this project’s acquisition cost estimation in great detail. Vendors were also not audited, as that is the role Public Works and Government Services Canada.

1.4 Methodology

The following methodology was used to conduct the audit:

  • interviews with C Navy staff members, Vice Chief of the Defence Staff, ADM(Mat) staff, and other stakeholders;
  • examination of project documents and contract management practices;
  • data analyses or review of Defence Resource Management Information System and Dynamic Object Oriented Requirements System;
  • site visits to end users—Halifax and Iroquois Class ships; and
  • sample of payments worth $10.3 million, which is 28 percent of the project office’s expenditures made between November 2008 and March 2014.

1.5 Audit Criteria

The audit criteria can be found at Annex B.

1.6 Statement of Conformance

The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit thus conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.

2.0 Findings and Recommendations

2.1 Impact of Fleet Size

The impact on the RCN’s ability to fulfil the roles and missions identified in the CFDS has not yet been fully assessed and communicated, should the CSC project deliver fewer than 15 ships,  as design options continue to be assessed.

Good Practices

  • The August 2010 DRDC Fleet Mix Study included a risk assessment of the inability to deploy adequate capabilities if the CSC fleet consisted of fewer than 15 ships.
  • Another DRDC study requested by the RCN in September 2014 aims to relate the number of CSCs procured to the output that the RCN can deliver.

The RCN is expected to contribute to the three roles of the Canadian Armed Forces as defined in the CFDS: defend Canada, defend North America, and contribute to international peace and security. The annual Force Posture and Readiness Directive provides a four-year forecast of the RCN resources that could be generated for each of the following six CFDS missions, potentially at the same time:

  • daily domestic and continental operations;
  • support to major international events in Canada;
  • response to a terrorist attack;
  • support to civilian authorities during a crisis in Canada;
  • leading and/or conducting a major international operation for an extended period; and
  • deployment of forces elsewhere in the world for shorter periods.

The CFDS renewal of core equipment programs included the need to monitor and defend Canadian waters and make significant contributions to international operations with a fleet of 15 new ships. However, in June 2011, the project office estimated that the constrained CSC budget                                                                                           14 Although a few fleet size studies have been performed as a starting point, more work is necessary. Since the project scope was changed in November 2013, from the acquisition of “15 ships” to “up to 15 ships,” the impact of a smaller fleet has not yet been fully assessed and communicated to senior management.

2.1.1 Fleet Size Requirements

Capability-based planning at DND examines the most likely scenarios that will be encountered in the six CFDS missions. Equipment projects deliver the number of combat systems needed to satisfy the operational requirements of these anticipated scenarios. For example, the Maritime Helicopter Project determined the number of helicopters to deliver based on the RCN’s most likely tasks, force structure, and the helicopter sustainment requirements. Once the capability requirements of the CSC have been determined, the three major factors that will influence the CSC fleet size requirements are the naval task group size, the RCN tasks, and the ship maintenance cycle.

Naval Task Group Size

The size of the CSC fleet required by the RCN to fulfil its assigned tasks is influenced by the RCN’s basic tactical force structure, namely, the naval task group. The task group includes the complementary combat capabilities necessary for successful naval warfare. There is some flexibility in the size and composition of a naval task group as it was defined in the 1994 Defence White Paper as consisting of “up to four combatants (destroyers, frigates, or submarines) and a support ship.” This original definition is still in use in other major Crown projects, reflecting current fleet capabilities. Over 80 percent of the CSC mission scenarios require maritime helicopter support, which is a major factor in the size of a naval task group.                                                                                                                                                                                                                                                                                                                                                         15                                                                                                                                                                                                                                                                                                     16 However, the DRDC study on helicopter support for the CSC fleet also recommended further operational research on the use of maritime patrol aircraft, submarines, and unmanned air vehicles in an anti-submarine scenario that impacts naval task group air resource requirements.

RCN Tasks

According to recent Reports on Plans and Priorities, the RCN is expected to generate two naval task groups capable of domestic or international operations.17 As well, single ships capable of international operations are to be deployed with or without allied task groups. These naval forces are the most common Navy commitments in the Force Posture and Readiness directive. The 2013/14 Report on Plans and Priorities also specified the following: a force structure of one navy task group, with up to four combatants and a support ship; a second navy task group with up to three combatants and a support ship; and two high-readiness ships for NATO or United Nations operations. 18

Depending on the size, composition, and response times of the naval task group with maritime helicopter support, the RCN could be expected to generate                                                                                                                                                                                                                         Should optimal helicopter support not be required, then the size of the task group may be smaller.

Fleet Maintenance Cycle

Although not yet determined for the CSC fleet, the ship maintenance cycle will also impact the availability of the CSCs and the fleet size. It is assumed that the CSC maintenance cycle would be similar to the Halifax Class, as the CSC project is replacing this fleet.19 For example, as shown in Annex C,                                                                                                                                                                                                                                                                                                                                                                                        

 2.1.2 Operational Impact

As portrayed in Annex C,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           20                                                                                                                                                                                                                                                                                                                                                                                                                                

 2.1.3 Summary

The project was capped early on at $26.2 billion, even though that figure was a rough order magnitude estimate, where cost estimates can range between plus or minus 40 percent. The quantity of ships that can be delivered within the established budget is directly affected by their capabilities. Although the intended prime contractor is working in consultation with the RCN and other departmental officials to determine if the requirements are affordable, decisions on capability trade-offs and optimal fleet size cannot be made until operational research studies are finalized and a preliminary design is complete.

ADM(RS) Recommendation

1.         C Navy should complete additional operational research as the CSC design evolves, then assess and communicate the impact of different fleet sizes to the Chief of the Defence Staff and develop options to facilitate future decision making.

2.2 Preliminary SOR

Naval task group size, which is a major factor in determining the number of CSCs required by the RCN to fulfil its missions, is not yet specifically addressed in the project SOR.

2.2.1 Requirement Changes

Good Practices

  • A version of the project’s preliminary SOR was formally sent to the Chief of Force Development for review by many naval Chief of Force Development personnel.
  • The preliminary SOR included references to over 25 operational research studies.
  • For traceability purposes, the project requirements database has captured the sources of the operational capabilities stated in the SOR.

The project presented draft requirements in December 2008 that were endorsed by the Joint Capability Requirements Board, a body whose role it was to reach a consensus on the preliminary SOR. Since then, the requirements have evolved due to technological changes and further operational research studies, and will continue to do so over the next five years. The most recent version of the SOR includes 15 revisions that reflected prior CRS audit observations, adding clarity to the requirements definition. The major operational requirement changes since 2008 with respect to accommodation, CSC range, and helicopter carrying capacity, are now well substantiated in the preliminary SOR. However, the affordability of these three requirements is now being assessed by the intended prime contractor, and may not be determined until 2019, with the anticipated completion of the preliminary design.

2.2.2 Naval Task Group Definition

The review of the CSC SOR found that the size of a naval task group was not addressed. Although three different defence scenarios in the SOR describe the need for a task group, the composition of the task group was not specified. Other ship-related projects have included the size of the naval task group in their requirements.21                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                                                                    22                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      

2.2.3 Summary

The naval task group is the RCN’s basic tactical formation, with a size and composition mandated in the Force Posture and Readiness directive. The size of a naval task group will influence the size of the CSC fleet, and depends on the mission as well as the capability of the CSC—which have yet to be determined. Therefore, no scale of task group size has been developed in the preliminary SOR to address the expected threat for the three CSC task group scenarios, which may vary in their intensity of operations.

ADM(RS) Recommendation

2.         C Navy should revise the SOR to address the various sizes and compositions of naval task groups as the capability of the CSC evolves.

OPI: C Navy

2.3 Project Schedule

Continuous schedule management rigour is required to determine whether the project is on track to meet the milestone for ship construction. The planned implementation phase of the CSC project will result in a schedule gap for the Halifax Class ships replacement                                                             The magnitude of the schedule gap was briefed to the RCN and they are aware of its significance.

2.3.1 NSPS Schedule

For the most part, the schedule revisions in the CSC project were caused by factors external to the CSC project office. In early 2010, before the approval of the NSPS, the delivery of the first ship was scheduled for 2020. This date was changed to 2025 in March 2013, due in part to the sequencing of the two combat ship construction projects at the designated combat shipyard for the NSPS. As the AOPS design was further advanced, and since it was not as complex, it was decided that this fleet would be built first, followed by the CSC fleet. During the conduct phase of the audit, the CSC construction milestone was set for 2020, to coincide with the anticipated decrease in AOPS construction activity. Revision of the CSC construction milestone to the early 2020s was necessary due to the design complexity and the availability of more detailed scheduling information.

2.3.2 Internal Schedule Controls

Good Practices

  • The procurement strategy consisted of streamlining the design schedule by 12 to 24 months, which was due to choosing a single design team, and was approved in April 2015.
  • The new project schedule being developed will apply best practices from the Project Management Body of Knowledge, the United States (US) Coast Guard, and Naval Sea Systems Command in the US.
  • In January 2015, the project office started to work with the intended prime contractor on an integrated master schedule.

The initial assumptions related to key CSC project milestones were sensitive to pending decisions such as the detailed procurement strategy. The 2020 target date for the construction of the first CSC ship was to avoid a gap between AOPS and CSC construction activity.                                                                                                   23                                                                                      Until the design contract is awarded, anticipated in 2017, the contractor’s detailed schedule for the CSC preliminary and detailed design will not be known. Therefore, the project office plans to focus its efforts on the detailed work breakdown structure and scheduling up to the design contract award milestone. At the time of the audit, the project office had contracted expertise to develop a three-month detailed schedule that will be extended to five months and will continue with a rolling wave approach. Given major project decisions that require interdepartmental concurrence, there is a need to make appropriate risk adjustments to take account of events beyond DND’s control.

The AOPS project schedule was compared to the CSC schedule, as both combat ships are developmental in nature and being built by the same shipyard. The CSC schedule is much more aggressive by forecasting three months between project approval and the milestone to begin construction. In comparison, the AOPS project had scheduled nine months duration for the same activities for a far less complex ship.

Co-chaired by C Navy and ADM(Mat), the role of the SRB is to perform a challenge function on project risk, cost, schedule, and performance. The March 2014 CSC SRB timetable indicated that the design work would take place between spring 2016 and spring 2018. However, subsequent project office consultations with industry found that only the preliminary design would be complete at this time, and that another 24 months would be necessary for a functional and detailed design. As the schedule is a major concern of the C Navy, improved visibility into the major milestones would provide greater insight into the project and its inherent scheduling risks, and would provide greater opportunities for timely decision making on the project.

2.3.3 Implementation Phase Milestones

The 12 Halifax Class ship replacements are currently scheduled to take place between 2029 and 2043. The extended                   hull design life for the Halifax Class will be exceeded between                                   Based on the project schedule during the audit conduct phase, when it was assumed that the project would deliver 15 ships and the first ship would be delivered by 2025, there will still be at least an average                                                                

While the service life of allied ships is approximately 32 years, the Halifax Class will not be replaced by the CSC for                       Historically, Canadian ships have remained in service for 34.5 years on average. However, the Iroquois Class has been life extended to 42 years of service.

One cause of this schedule gap is the length of the planned CSC implementation phase to build up to 15 complex ships in Canada. The 12 Halifax Class ships were delivered in a four-year period (mid-1992 to mid-1996), but the replacements will be delivered over 14 years due in part to the capacity of the combat shipyard and the available cash-flow. To deliver the CSC fleet earlier would require more shipyard capacity but could compromise the NSPS’s continuous build program.

2.3.4 Summary

                                          and the scheduling of the construction of the AOPS fleet before the CSC fleet have, in part, resulted in a five-year revision to the anticipated delivery of the first CSC. The project office will have difficulty tracking its progress to meet milestones and understand the impact of changes external to the project office on the previously-targeted 2020 construction date. The delayed approval of the CSC design procurement strategy contributed to the subsequent revision of the construction milestone to the early 2020s. The project mitigated this delay by performing other work concurrently while awaiting a decision.                                                                                                                                                                                                                                                                          Based on the DND economic model escalation rate of 1.5 percent, the project loses approximately                                                                                                    

The Halifax Class replacement schedule is a long-term concern of the RCN, which has limited recourse to address this issue at this time. The magnitude of the schedule gap was briefed to the RCN, and the RCN chain-of-command is aware of its significance. As the gap might occur only                     from now, the RCN will need to develop options for managing the gap sufficiently in advance.

ADM(RS) Recommendation

3.         ADM(Mat) should continue to focus rigour on the schedule, track the impact of project changes, and identify time-saving measures with regular briefings to the project leader. The SRB should also be apprised of schedule issues on a routine basis, along with the impact of changes on long-term milestones.

OPI: ADM(Mat)

2.4 Information for Decision Making

Improvements in the accuracy and completeness of project documentation are required to manage project risks and expedite the CSC project.

2.4.1 The Project Brief

The project brief is a living document that provides an overview of the project for the approval of funding at each phase. It contains key information that is relied upon by other organizations to remain informed about a project and to aid in decision making. The content of the most recent version of the CSC project brief that was endorsed by the Program Management Board in October 2014 could be improved as follows:

Good Practices

  • To expedite staffing of the CSC project office, the project has hired a former public service executive to produce a human resources plan.
  • There has been collaboration with the NSPS Office to gain further specialized technical support for the project.
  • The project office has shared resources/expertise with the AOPS project.
  • The intended prime contractor has been included in the project governance activities since January 2015.
  • The CSC project scope would benefit from the current definition of the size and composition of a naval task group as this is the basic force structure that operates within the RCN and that will likely impact the size of the CSC fleet.
  • In order to identify the urgent need to recapitalize the Halifax Class, the project brief should note that the hull design life may result in a                                                       on average prior to replacement. Routine inspections and hull strengthening practices have been used in the past to extend the hull life.
  • The RCN’s September 2014 announcement to reduce the Iroquois Class from three ships to one ship should be included to highlight this capability gap.
  • The assumption that the A and B jetties in Esquimalt will accommodate the CSC contradicts an October 2014 CRS audit report. That audit found that the A Jetty will only accommodate ships that are 140 metres in length, not the assumed length of CSC, which could be as long as                     24

2.4.2 Project Office Size Substantiation

At the time of the audit, approximately 20 of the 84-person CSC project office establishment positions for the definition phase were not filled. A benchmark analysis conducted by the audit team determined that the 235 project personnel identified in the project brief attachment for the implementation phase may be insufficient as the benchmark analysis suggests that the average size of the CSC project office should be 442 personnel. The optimal size of a project office is also influenced by external support. Notwithstanding the smaller CSC project office, it has made use of contracted support, early involvement of the intended prime contractor, the NSPS secretariat, the AOPS project lessons learned, and it will have access to US Navy shipbuilding costing and project management expertise.

2.4.3 CID Progress Report

The CID was established in 2002 to help DND manage the Investment Plan. The CID monthly progress report is required for all equipment projects implemented by ADM(Mat). Notwithstanding the frequent changes in the project and in the other project office reporting requirements, the CSC project’s monthly progress report was incomplete in the following sections:

Project Status. A concise description of the current stage/status of the project was not included in the CID. Therefore, departmental planning staffs are not provided with an up-to-date status of the project.

Project Issues. There are 12 standard project issues that reflect the Project Management Body of Knowledge practices in the progress report. Although the CSC project has reported risks in these areas in other documentation, this part of the report was not completed. This information from the project issues section provides the root cause for the “yellow” rating 25 in the cost, schedule, and technical performance criteria within the CSC progress report.

Lessons Learned. Although the CSC project office has been in place for over six years, the lessons learned section of the project’s progress report had no entries. The project office states that it has shared lessons learned with other Navy projects, and at Director General Major Project Delivery (Land and Sea) meetings. However, utilizing the central repository of the CID would allow other naval project offices, or anyone in the Department, to benefit from CSC’s specific lessons learned.

2.4.4 LCC Estimate Assumptions

The project office’s estimate has not been recently updated as the Department is expecting a government-wide policy on LCC. Until this policy is approved, the LCC estimate becomes increasingly invalid as planning assumptions, such as those regarding both crew size and the rate at which ships are to be built, have changed. Additionally, more appropriate ship repair and betterment escalation rates could have been chosen.

2.4.5 Summary

The project brief information regarding the force structure, the duration of the Halifax Class replacement schedule gap, and the jetty limitations, would be useful for decision making. Without up-to-date project information in the CID, it is difficult for Department analysts to monitor the progress of the capital acquisition program. Moreover, the fact that the CSC project office size is smaller than other project office models suggests                                                                                                                                                    

ADM(RS) Recommendation

4.         ADM(Mat) should ensure that the project office implementation organization is compared to the Canadian Patrol Frigate project office to ensure that all key roles and responsibilities are accounted for.The following project documentation should also be updated to address the audit observations:

  1. the project brief,
  2. the CID monthly progress report, and
  3. the LCC estimate assumptions.

OPI: ADM(Mat)

2.5 Contract Management

Holdback and timesheet terms and conditions in the ELMS contract were not enforced.

Good Practices

  • Contract management staff has requested that the ELMS contractor improve the content of monthly progress reports to monitor the status of the work.
  • CSC project staff recently updated a series of financial management standard operating procedures.
  • For six of ten ELMS claims reviewed, the challenges of the contract management staff had reduced the claims by $62,000.

The audit sample included payments for the ELMS contract that supports several major ship projects. At the time of the audit, the CSC project tasks under this contract had amounted to $6.4 million. The invoices were supported by monthly progress review reports and progress review meetings. However, two clauses that existed in the contract were not being enforced—the submission of contractor timesheets and the use of holdbacks.

2.5.1 Timesheet Clause

The contract contains a clause that requires the ELMS contractor to support its labour charges with timesheets. It was explained by contract management staff that the contractor’s timesheets included the contractor work order numbers instead of the DND task identifiers.

2.5.2 Holdback Clause

The contract and the individual task authorizations required the contractor to invoice no more than                   of the amount claimed each month, with the balance to be paid upon task completion. The purpose of this clause was to influence contractor cash flow to ensure the work would be done on time, but it was not being enforced.

2.5.3 Summary

Future risk still exists as the contract is only slightly over half spent and could be extended from 2016 to 2019 through option years. The timesheet and holdback clauses were included in the contract to provide cost and quality controls. If the timesheet provisions are not enforced, the risk of double billing for labour charges cannot be mitigated. Without holdbacks, the Crown has less leverage to ensure that deliverables are satisfactory and delivered on time.

ADM(RS) Recommendation

5.         ADM(Mat) should ensure that the ELMS contract provisions for timesheets and holdbacks are enforced.

OPI: ADM(Mat)

3.0 General Conclusion

The CSC project represents an important procurement initiative due to its fundamental capacity to replace the RCN’s major surface fleet that supports the RCN and to its being the largest component of the NSPS. As such, successful delivery of the project is of strategic importance. At the time of the audit, there remained at least five years in the definition phase to finalize the design of the ship. The CSC project is in the midst of balancing the capabilities of a fleet of ships that have not yet been designed with a known project budget. Opportunities to advance the project schedule will need to be exploited, with regular updates provided to key stakeholders as better information becomes available.

While much attention has been paid to the project, and a number of good practices have been identified, the audit has identified some areas for improvement. Specifically, should a smaller fleet size be delivered, the operational impact would need to be fully assessed and made available to facilitate strategic decision making. The SOR should be revised to address different naval task group sizes so as to help determine the CSC fleet size as the CSC design evolves. Given the potential                   schedule gap between the delivery of the CSCs and the ships they replace, particular attention should be paid to project scheduling issues. Also, noting the dynamic nature of this exceptionally complex project, information in support of decision making and timely delivery would likely benefit from a review of its key documentation. The implementation of these audit recommendations, along with the one regarding contract management, should help better position the project for successful delivery.

Annex A—Management Action Plan

ADM(RS) uses recommendation significance criteria as follows:

 

Very High—Controls are not in place. Important issues have been identified and will have a significant negative impact on operations.

High—Controls are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.

Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.

Low—Controls are in place but the level of compliance varies.

Very Low—Controls are in place with no level of variance.

Impact of Fleet Size

ADM(RS) Recommendation (High Significance)

1.         C Navy should complete additional operational research as the CSC design evolves, then assess and communicate the impact of different fleet sizes to the Chief of the Defence Staff and develop options to facilitate future decision making.

Management Action

As the design and capability of CSC evolves throughout the project definition phase, the RCN will continue operational research into the impact of different fleet mix options on the operational output of the RCN. These options include the generation and sustainment of naval task groups and the ability to fulfil all CFDS missions. The RCN is also participating in ADM(Mat)-initiated studies with industry that are helping to further refine CSC requirements. The results of these various studies will be communicated through established governance processes to facilitate future decision making.

OPI: C Navy

Target Date: June 2016

Preliminary SOR

ADM(RS) Recommendation (High Significance)

2.         C Navy should revise the SOR to address the various sizes and compositions of naval task groups as the capability of the CSC evolves.

Management Action

The RCN’s task group definition is under review by Director Naval Strategy. The RCN will assess the impact on the CSC SOR upon completion of this review and operational research into the impact of different fleet mix options while the CSC’s capability evolves. Concurrent with this review, the RCN is working with the project management office, the intended prime contractor, and their subcontracted external agents to review the CSC SOR. The CSC project plans to conduct initial CSC requirement reconciliation in accordance with the recently announced Most Competitive Procurement Strategy for CSC and report to the NSPS Governance and Independent Review Panel for Defence Acquisition. If the results generate changes to the CSC SOR, Chief of Force Development will be consulted for Defense Capability Board consideration.

OPI: C Navy

Target Date: June 2016

Project Schedule

ADM(RS) Recommendation (High Significance)

3.         ADM(Mat) should continue to focus rigour on the schedule, track the impact of project changes, and identify time-saving measures with regular briefings to the project leader. The SRB should also be apprised of schedule issues on a routine basis, along with the impact of changes on long-term milestones.

Management Action

Currently, the project office has developed a work breakdown structure for the definition phase, which includes qualification, selection, and preliminary design work, as well as an associated integrated master schedule. This work breakdown structure includes elements of work by the Crown (Public Works and Government Services Canada, the CSC project office, many stakeholders and support staff), as well as by contracted work. Working with the intended prime contractor, the project has developed and is maintaining a joint schedule of key milestones to ensure that work efforts are aligned between the project and the intended prime contractor. This will encompass the complete design effort. It is the intention of the CSC project office that these schedule products will be sustained throughout the project and serve for reporting schedule information.

In a related vein, the project maintains a regular risk management program that identifies triggers and strategies to positively influence the schedule and to minimize the impact of any delays that may emerge. The CSC project office will ensure that risks that have an impact on the schedule and opportunities to advance milestones are identified, tracked, mitigated to the extent possible, reflected in the CSC project baseline schedule, reported in a timely and more comprehensive manner to the project leader prior to defence procurement meetings, and included as a separate item in reports to the SRB.

OPI: ADM(Mat)/Director General Major Project Delivery (Land and Sea)

Target Date: December 2015 (Baseline schedule risks will be presented to the project leader routinely, prior to Defence Procurement Strategy governance meetings and to SRBs.)

Information for Decision Making

ADM(RS) Recommendation (High Significance)

4.         ADM(Mat) should ensure that the project office implementation organization is compared to the Canadian Patrol Frigate project office to ensure that all key roles and responsibilities are accounted for.The following project documentation should also be updated to address the audit observations on:

  1. the project brief,
  2. the CID monthly progress report, and
  3. the LCC cost estimate assumptions.

Management Action

Action 4-1: The project office will work with the sponsor to update those portions of the project brief concerning the employment of the CSC, the transition from the current fleet, and the infrastructure to support the CSC. However, due to the sensitivities regarding the competitive nature of the project at present, some of the information required to be held in the CID is under cover letter only, and readers are advised that information is available upon request to the project office, provided it is based on a need-to-know. The project office will take the necessary steps to keep the information up-to-date.

OPI: ADM(Mat)/Director General Major Project Delivery (Land and Sea)

Target Date: September 2015

Action 4-2: Assumptions with respect to the project’s LCC will be developed in consultation with Director Costing Services and key CSC LCC stakeholders once the revised CSC LCC framework and supporting processes are in place. In the interim, the implementation of the revised project LCC framework, its underlying cost breakdown structure, and definition of project supporting processes—including cost generation, validation, consolidation, and reporting—is currently underway. Furthermore, the project team has been working closely with Director Costing Services and, through participation in the LCC Working Group, intends to recommend additional NATO and US Department of Defence Naval Systems Sea Command LCC cost breakdown structure best practices.

OPI: ADM(Mat)/Director General Major Project Delivery (Land and Sea)

Target Date: November 2016 (CSC LCC assumptions prepared for next project submission)

Action 4-3: The CSC project office will endeavour to fill the vacancies of their organization and perform yearly assessments on: (1) the adjustment of resources against work-load of the project, and (2) the adjustments required for the organization by creating new positions for new work items or subject areas requiring different skill sets. The Naval Evaluation, the Test Establishment, and the US Naval Sea Systems Command organizations have all conducted studies on behalf of the project to validate the project team’s size and composition. Findings of those studies, which looked at staffing models from several similar national and international naval combatant program offices, have shaped the plan for the development of the project’s current and future organization. The project is currently executing the plan in accordance with all applicable human resources processes. Moreover, the project is leveraging resources from other government agencies, naval project offices, and industry contractors to greatly supplement its expertise and capacity in key areas.

OPI: ADM(Mat)/Director General Major Project Delivery (Land and Sea)

Target Date: November 2016 (organizational needs prepared for next project submission)

Contract Management

ADM(RS) Recommendation (Low Significance)

5.         ADM(Mat) should ensure that the ELMS contract provisions for timesheets and holdbacks are enforced.

Management Action

Action 5-1: As per Standard Acquisition Clauses and Conditions C0705C and C0710C, discretionary audits and timesheet verifications are included in the contract at Article 6.3. Existing task Statements of Work will be negotiated to reflect the requirement that timesheets will be required and will be available on request. Furthermore, all new ELMS tasks will include this requirement. Discretionary audits of timesheets will be conducted periodically.

Action 5-2: Additionally, the provisions of the contract shall be amended to include the requirement for the contractor to submit a detailed monthly summary of the                                                                                                                                                                                                                                                                                                                         Currently, each of the ELMS tasks are paid up to                     of the total value, with the remaining                     released only upon signature of the Certificate of Compliance, which indicates that the task is completed and has been accepted by the task technical authority.

The proposed solution to address the comments of the ADM(RS) audit is to amend the contract to reflect the current method by which payments are made under the contract. The contract shall be amended to stipulate that the Crown shall pay monthly costs incurred, from                                                 while the internal monitoring process pays up to                     of the individual task value until a Certificate of Compliance is signed.

These solutions have been discussed with the ADM(RS) auditors and the chain of command within Director General Maritime Equipment Project Management and have been accepted as sufficient to address the comments of the audit. Contract amendment 15 details each of the changes that will be formally implemented. The amendment has been agreed in principle with the contractor, and is currently in draft version. It is anticipated that the amendment will be formally adopted with the May 2015 progress claim.

OPI: ADM(Mat)/ Director General Maritime Equipment Project Management

Target Date: June 2015

Annex B—Audit Criteria

Criteria Assessment

The audit criteria were assessed using the following levels:

Assessment Level and Description

 

Level 1: Satisfactory

Level 2: Needs Minor Improvement

Level 3: Needs Moderate Improvement

Level 4: Needs Significant Improvement

Level 5: Unsatisfactory

Project Schedule

1.   Criterion. Project schedule is achievable, and is managed to avoid impact on operational requirements. (Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors March 2011—Stewardship 1, 15, 16. Program Management Book of Knowledge 2011, Annex A)

Assessment Level 3—Project schedule management is currently undergoing large revisions; however, the audit noted that it needs to include additional detail up to the 2017 design sub-contract award. Design phase milestones need to be reported to SRB to understand the level of effort required in the aggressive schedule to meet the early 2020s milestone for ship construction to begin. There is a gap between the Halifax Class replacement schedule in the CSC project that the RCN will need to manage in due time.

Financial Management

2.   Criterion. Financial management is in accordance with the Financial Administration Act and DND and Treasury Board regulations, while ensuring lowest total cost of ownership and facilitated with reliable and relevant cost estimates. (Audit criteria related to the Management Accountability Framework: A Tool for Internal Auditors March 2011—Stewardship 5, 6, 10, 12, and 13)

Assessment Level 2—The June 2012 LCC estimate assumptions need to be updated once the government-wide LCC policy is promulgated. Timesheets and holdbacks should be enforced in the ELMS contract.

Information for Decision Making

3.   Criterion. An adequate monitoring process and governance structures are in place that utilize high-quality, up-to-date, and accurate information as the basis for decision making. (Audit criteria related to the Management Accountability Framework: A Tool for Internal Auditors March 2011—Governance and Strategic Direction 6, Results and Performance 2. DND Project Approval Directive)

Assessment Level 3—The Project Brief content needs improvement to expedite project approval, and there is a need to improve the CID monthly progress report information. Project Requirements

Project Requirements

4.   Criterion. Operational requirements are in accordance with defence policy. They are clearly defined, complete, prioritized in due time, consistent, and traceable throughout the project activities from SOR development to test, evaluation, and training plans. (Audit criteria related to the Management Accountability Framework: A Tool for Internal Auditors March 2011—Policy and Programs 2, Risk Management 7, Governance and Strategic Management 4.)

Assessment Level 3—The impact on the roles and missions of the CFDS has not been formally assessed and communicated to senior management, should a smaller CSC fleet be delivered, as design options continue to be assessed. Naval task group size, a major factor in determining the number of CSC required, is not specifically addressed in the project SOR.

Annex C—CSC Fleet Size Impact

Table C-1                                                                                                                        

Table Summary

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

 

           

 

           

       

                

           

         

               

     

                                                   

                           

                       

                         

                         

                   

             

                   

                     

          
           
           
            
                 
            
         
           
          
          

                                                                                                                                                                                                                                                                                                                                                         

 

 

___________________________________________________________________________________________________________________________

Footnote 1 Former designation of Assistant Deputy Minister (Review Services (ADM(RS)).

Footnote 2 Backgrounder on the National Shipbuilding Procurement Strategy – Year 2: A Status Update, November 2013: http://www.tpsgc-pwgsc.gc.ca/app-acq/sam-mps/ddi-bkgr-10-eng.html Last consulted on April 15, 2015.

Footnote 3 This term is defined in the DND Costing Handbook, chapter 5, as estimates that can range between plus or minus 40 percent.

Footnote 4 This assumes that the project will deliver all 15 ships on the first CSC target delivery of 2025.

Footnote 5 Although the Canadian Patrol Frigate project office only delivered 12 ships, construction of the first batch of 6 ships took place in two shipyards, which meant that additional oversight was required.

Footnote 6 Public Works and Government Services Canada officials are the chairs of the NSPS governance committees.

Footnote 7 See: http://www.tpsgc-pwgsc.gc.ca/app-acq/sam-mps/ddi-bkgr-10-eng.html. Last consulted on April 15, 2015.

Footnote 8 Known as the most competitive procurement strategy.

Footnote 9 See: http://www.tpsgc-pwgsc.gc.ca/app-acq/sam-mps/ddi-bkgr-10-eng.html. Last consulted on April 15, 2015.

Footnote 10 CFDS, 2008, page 4.

Footnote 11 Office of Auditor General Fall Report 2013.

Footnote 12 See: http://www.tpsgc-pwgsc.gc.ca/app-acq/sam-mps/ddi-bkgr-10-eng.html Last consulted on April 15, 2015.

Footnote 13 The Crown will be involved with the first tier of subcontracts regarding the competitive selection of the ship designer and systems integrator. NSPS Technical Briefing on CSC, May 1, 2015.

Footnote 14 Briefing note to the Deputy Minister of National Defence, Costing – Canadian Surface Combatant, June 2011.

Footnote 15 The CSC SOR refers to the August 2013 DRDC study on Cyclone helicopter support.

Footnote 16                                                                                                                                                                                                                                                                                                                                                                                              

Footnote 17 The most recent Report on Plans and Priorities for fiscal year 2015/16 is less specific on the number of task groups and single ship missions but the resources for sub-sub-program activity 3.1.1 are very similar to those in fiscal year 2014/15.

Footnote 18 Combatants could be frigates, destroyers, or submarines. The response time for the two task groups is different; one task group has a much faster response time than the other.

Footnote 19 The “short work period,” lasting one month and to be undertaken each year, was not included in the fleet maintenance cycle.

Footnote 20 Navies in the Asia-Pacific region have increased their submarine presence by 50 percent in the last seven years. See The Future Security Environment, 2013-2040, chapter 5.

Footnote 21 The Joint Support Ship SOR provided the sustainment requirements for a task group. The Halifax Class Modernization/Frigate Life Extension project charter, dated March 2008, included the 1994 White Paper definition “up to four combatants (frigates, destroyers, submarines) and a support ship.”

Footnote 22 DRDC. Analysis of the Number of Cyclone Helicopters Needed to Provide Organic Helicopter Support to a Future Fleet Canadian Naval Task Group, August 22, 2013.                                                                                                                                                                                                                                                                                                                                                                                                                                                              

Footnote 23                                                                                            

Footnote 24 In response to the October 2014 CRS Audit of the A and B Jetty Recapitalization Project, the Navy updated the jetty requirements on how it will accommodate the CSCs.  The Navy noted that it will continue to review the scenarios for the jetty as the CSC ship length will not be known until a CSC design is selected. The Navy also noted that having ships overhang the jetty length is common practice, even routine in smaller ports.

Footnote 25 Yellow ratings indicate that project contingencies will be used, project schedules have slipped up to 10 percent, and technical studies, decisions, and management documents are difficult to resolve.

Date modified: