Evaluation of Environmental Protection and Stewardship

July 2014

1258-208 (CRS)

Reviewed by ADM(RS) in accordance with the Access to Information Act.  Information UNCLASSIFIED.

Acronyms and Abbreviations

ADM(Mat)

Assistant Deputy Minister (Materiel)

CA

Canadian Army

CAF

Canadian Armed Forces

CEPA

Canadian Environmental Protection Act

CFB

Canadian Forces Base

CRS

Chief Review Services

DAOD

Defence Administrative Orders and Directives

DES

Defence Environmental Strategy

DEW

Distant Early Warning

DFO

Department of Fisheries and Oceans

DND

Department of National Defence

EEE

Electronic and Electrical Equipment

EnvO

Environmental Officer

EP&S

Environmental Protection and Stewardship

FHR

Federal Health Regulations

FSDS

Federal Sustainable Development Strategy

FCSAP

Federal Contaminated Sites Action Plan

ADM(IE)

Assistant Deputy Minister (Infrastructure and Environment)

DGIEES

Director General Infrastructure and Environment Engineering

FTE

Full-Time Equivalent

FY

Fiscal Year

GHG

Greenhouse Gas

L1

Level 1

NRC

National Research Council

NU

Nunavut

PAA

Program Activity Architecture

PAP

Project Approval Process

PCB

Polychlorinated Biphenyls

RCAF

Royal Canadian Air Force

RCN

Royal Canadian Navy

RDPA

Regulations Designating Physical Activities

RTA

Range and Training Area

SDS

Sustainable Development Strategy

SK

Saskatchewan

UXO

Unexploded Explosive Ordnance

VCDS

Vice Chief of the Defence Staff

W/B/F

Wing/Base/Formation

DRMIS

Defence Resource Management Information System

DGIEGPS

Director General Infrastructure and Environment Governance Policy and Strategy

Executive Summary

Overall Assessment

  • There is evidence of an ongoing and demonstrable need for EP&S, a role that is clearly aligned with Government of Canada and DND roles, responsibilities and priorities.
  • The Program has met expectations.
  • There is some concern with respect to rate of remediation and mitigation activities, particularly for unexploded explosive ordnance. In addition, the level of ongoing support to the sustainability of ranges and training areas (RTAs) will require further attention.

This report presents the findings and recommendations of the evaluation of the Department of National Defence (DND) Environmental Protection and Stewardship (EP&S) Program. The evaluation was conducted by the Chief Review Services (CRS) between June 2013 and February 2014, in compliance with the Treasury Board Policy on Evaluation (2009). As per the policy, the evaluation examines the relevance and the performance of the Program over a five-year period (2008-2013).

Program Description

The EP&S program ensures that DND complies with all environmental legislation and polices in all activities. This includes the protection, mitigation and remediation of the environment, sustainable practices, and the management of unexploded explosive ordnance (UXO), on both historic and current DND property.

EP&S is executed by all DND/Canadian Armed Forces (CAF) organizations and by the Assistant Deputy Minister Infrastructure and Environment (ADM(IE)) as the functional authority.

To deliver these services, these organizations collectively dedicate approximately 140 military and civilian personnel, and an overall annual expenditure of approximately $91.6 million.1

Relevance

The evaluation determined that the need for the EP&S program is of continuing relevance and is aligned with federal Government and departmental roles, responsibilities and priorities. Throughout the year, the CAF, through the conduct of its operations, may create circumstances that are harmful to the environment. This includes physical damage, chemical contamination, pollution, as well as the use of fossil fuels and other non-renewable materials. Furthermore, many bases and training areas (both current and legacy) owned or leased by the Department may have been contaminated by past practices and require environmental remediation to ensure that they no longer pose a health, safety or environmental risk to both current and future users. At present there are 1,156 sites2 within Canada that require environmental remediation from past CAF activities. In addition, another 840 sites3 may potentially contain UXO, which are munitions utilized on training areas that failed to explode; some may have remained in the ground—in some cases since World War I.

Performance

With respect to performance, during the period of the evaluation the EP&S program generally met expectations and delivered on its mandate. The Program was seen to be most effective in the remediation of contaminated sites, having established and implemented a sound process that involves all stakeholders, including local community involvement, while at the same time meeting all requirements for successfully remediating contaminated lands. The Program also has ensured that DND is complying with environmental legislation and regulations, with relatively few violations considering the extent of its operations. Furthermore, EP&S has established a credible and effective process for the management of UXO on legacy sites. Lastly, the Program is seen to be very affordable—representing less than 0.5 percent of the Defence budget.

Going forward, however, there are some concerns:

The actual implementation rate of the UXO mitigation plan is questionable. At the current pace it will take over eighteen years to complete.4 There does not appear to be any formal risk assessment completed by the Department to determine if this rate is acceptable.

In addition, the amount of funding in support of environmental remediation projects has declined by over 28 percent during the evaluation period. While this reflects the completion of some large projects, the funding has not been transferred into a larger number of new, smaller projects.

With respect to ensuring sustainable RTAs, the impacts of decades of use and more restrictive standards have combined to begin to affect the availability of some training areas at certain times of year. If unchecked, this could impact readiness training.

With respect to efficiency and economy, during the five-year review period the overall budget of the program area declined—falling from approximately $125.5 million in fiscal year (FY) 2009/10 to $91.6 million5 in FY 2012/13. This can be directly attributed to reduced remediation expenditures in the Federal Contaminated Sites Action Plan (FCSAP) program and as large projects, such as the Distant Early Warning (DEW) line, approach completion while replacement projects of equivalent value have not occurred. With respect to efficiency, for remediation projects the overall efficiency was seen to decline as the expenditure reductions mostly involved project funding as opposed to overhead and personnel costs. This, however, would be expected as management of large projects is typically more efficient than that of smaller ones. Other areas, such as expenditures towards sustainable RTAs, and the corporate environmental program, have seen efficiency gains due to increased CAF project expenditures and a ten percent reduction in EP&S program staff as part of the Deficit Reduction Action Plan.

These are the evaluation team’s findings and recommendations:

Findings and Recommendations

Key Finding 1: There is a continued need for the EP&S program.

Key Finding 2: The EP&S program aligns with federal roles and responsibilities.

Key Finding 3: The EP&S program aligns with DND/CAF priorities.

Key Finding 4: While the evaluation team determined that there exists a sound process and plan to address the threat of UXO, the extent to which the process has been implemented raises some concern.

Key Finding 5: There is a need for clear direction as to what areas are to be deemed priorities for the Program to focus upon.

Key Finding 6: DND/CAF has demonstrated effective remediation of contaminated sites.

Key Finding 7: Long-term funding for the monitoring of remediated sites is uncertain.

Key Finding 8: Environmental issues have impacted the use of some key training areas. While these are largely the result of the cumulative effect of activities over the past decades on the RTA, they now require the CAF to implement mitigation activities, which reduces training flexibility and affects operational readiness.

Key Finding 9: A significant proportion (88 percent) of Base/Wing Environmental Officers (EnvOs) felt that there is a significant lack of support with regard to legislative and environmental policy direction, and must often look elsewhere for assistance.

Key Finding 10: The evaluation team found it difficult to determine whether or not sustainability and green procurement targets were met. The issue is a lack of substantive baselines and targets within the Sustainable Development Strategy (SDS) to adequately measure performance. 

Key Finding 11: Meaningful performance measures must be developed for the Department’s Environmental Strategy.

Key Finding 12: For only approximately 0.5 percent of the total Defence budget, the EP&S program fulfills a critical need. As such, the EP&S program can be deemed to be an affordable program for DND.

Key Finding 13: The Department has reduced overall expenditures on environmental remediation projects as large projects near completion, and projects of equivalent value have not occurred.

Key Finding 14: Between 2009 and 2013, the efficiency of project delivery declined as the value of projects delivered was reduced, while Full-Time Equivalent (FTE) and associated overhead expenditures remained relatively constant.

Key Finding 15: Recent FTE reductions in EP&S as a response to the Deficit Reduction Action Plan have increased efficiency in corporate environmental program support.

Based upon these findings, the evaluation recommends that:

Recommendation 1: ADM(IE) should conduct a formal risk-based assessment on the rate of priority ranking and assessment of known UXO sites.

Recommendation 2: ADM (IE) should develop a multi-year plan with clear rankings to determine the order in which UXO sites will have mitigation activity conducted.

Recommendation 3: ADM(IE) should develop means to ensure that EnvOs have access to the information and support they require to ensure that legislation and regulations are effectively met.

Recommendation 4: ADM(IE) should review all sustainability and green procurement targets and develop a performance measurement strategy for these areas.

Note: Please refer to Annex A—Management Action Plan for the management responses to the CRS recommendations.

1.0 Introduction

1.1 Profile of Environmental Protection and Stewardship

An evaluation of DND’s EP&S program was conducted by CRS from June 2013 to February 2014, as a component of the DND/CAF Five-Year Evaluation Plan (2012/13 to 2016/17). The evaluation was performed in compliance with the Treasury Board Policy on Evaluation (2009) and, as such, examines the relevance and performance of EP&S between FY 2008/09 and FY 2013/14.

Although no previous evaluations of this area have been performed, CRS has conducted an Evaluation of the DND/CAF Ammunition Safety Program (2005), and audits were conducted of Hazardous Materials Management (2012), the Defence Sustainable Development Strategy (2011), and the DEW Line Clean-up (2008). In addition, in conjunction with this evaluation, CRS completed an evaluation of the $30 million Contribution to the Province of Ontario for the Mid-Canada Line Clean-up. That review is given as a separate report.

1.1.1 Background

The EP&S program exists to ensure that DND complies with all environmental legislation and polices in all of its activities. This includes the protection, mitigation and remediation of the environment, sustainable practices, and the management of UXO on both historic and current DND property.

Environmental issues impact garrisons, RTAs, and may also impact operational and mission capability. While environmental issues within the RTAs are of concern to all DND/CAF organizations, environmental issues have a particularly strong link on delivery of the Canadian Army (CA) mission, given that the RTAs are the primary training platform.

EP&S is executed by all DND/CAF organizations, and by the ADM(IE) as the functional authority.

Within DND, environmental protection and stewardship may be coarsely separated into three groups:

  • protection of the environment and sustainability;
  • management of unexploded explosive ordnance; and
  • assessment and remediation.

1.1.2 Program Description

EP&S provides DND/CAF with knowledge, data and tools needed to mitigate environmental risk and to provide environmental oversight. The key EP&S activities were seen to consist of:

  • management of legacy UXO sites;
  • management of contaminated sites (both active and legacy);
  • promoting sustainable Wing, Base and Formation (W/B/F) garrisons, RTAs6; and
  • ensuring compliance with federal rules and regulations.

ADM(IE) is the functional authority for the Program, with the following7 areas of oversight:

  • for environmental compliance and sustainability, the Director General Infrastructure and Environment Governance Policy and Strategy (DGIEGPS);
  • for large-scale contaminated site projects, the DGIEES; and
  • for UXO programs, the Director General Portfolio Requirements (DGPR).

Within Assistant Deputy Minister (Materiel) (ADM(Mat)), and the CA, Royal Canadian Navy (RCN) and Royal Canadian Air Force (RCAF), EP&S activities are undertaken by EnvOs and staff who rely upon DGIEGPS to provide environmental policy support. Areas where policy support is provided by DGIEGPS are hazardous materials, species at risk, halo and hydrocarbons, polychlorinated biphenyls (PCBs), and sustainable RTAs.

All DND/CAF organizations are responsible for prioritizing their efforts to address environmental impacts that pose the greatest risk. To assist in this, DGIEGPS is responsible for promoting environmental protection and stewardship and serves as the primary office for the DND/CAF Environmental Strategy and programs.

1.1.3 Program Objectives

Many activities undertaken by DND/CAF can negatively impact human health and the environment. The objective of EP&S is to minimize and/or mitigate those impacts to ensure that all training and operations are conducted in a manner that complies with environmental legislation and regulations, including sustainability.

For the purposes of the evaluation, the program objectives (outcomes) to be assessed were seen to be:

  • Canadians are safeguarded against UXO threats;
  • priority contaminated sites are remediated to acceptable levels;
  • training and operations are conducted in an environmentally responsible manner, resulting in sustainable garrisons and RTAs; and
  • environmental targets are met.

1.1.4 Stakeholders

EP&S is delivered within DND/CAF by several groups8:

  • ADM(IE) is the functional authority for environmental matters, and provides environmental policy interpretation, support, and advice;
  • ADM(Mat) implements Green Procurement policies; and
  • Chief of the Air Force Staff, Chief of the Army Staff, Chief of the Navy Staff, CMP and the Vice Chief of the Defence Staff (VCDS) implement activities to ensure environmentally sustainable W/B/F garrisons and RTAs.

Partners in the delivery of EP&S are Environment Canada, Fisheries and Oceans, Natural Resources Canada, Health Canada, private sector contractors, and Defence Construction Canada with respect to environmental issues, and Transport Canada with respect to explosives and the transport of dangerous goods.

1.2 Evaluation Scope

The evaluation team considered issues of coverage and responsibility, resources, relevance, and performance (effectiveness and efficiency).

1.2.1 Coverage and Responsibility

Within the 2013 Program Activity Architecture (PAA) the EP&S program is found in sub-program 4.3 “Environment Protection and Stewardship”.

These Program Activities are linked directly to the Strategic Outcome: “Care and Support to the Canadian Forces and Contribution to Canadian Society.”

1.2.2 Resources

The total expenditures of the EP&S program were approximately $91.6 million in FY 2012/13. This included civilian and military personnel salaries, overhead expenses, operations and maintenance, and the actual conduct of environmental services (professional engineering, abatement, and remediation). Approximately 140 civilian and military FTE are involved in these activities. Financial details are provided in section 2.5 of this report.

1.2.3 Evaluation Issues and Questions

In accordance with the Directive on the Evaluation Function, the core issues to be evaluated were relevance and performance.

With respect to relevance, the evaluation team asked:

  1. To what extent has the EP&S program addressed a demonstrable need and is responsible to the needs of Canadians?
  2. Did the EP&S program align with federal government roles and responsibilities?
  3. Did the EP&S program align with federal government priorities and DND/CAF strategic outcomes?

With respect to performance, the evaluation team asked:

  1. To what extent did the EP&S program meet expected outcomes?
  2. Was the funding of the EP&S program the most efficient or economic means of achieving the expected outputs and outcomes?

Details of the methodology utilised for the evaluation are included in Annex B—Methodology and Annex C—EP&S Logic Model.

2.0 Findings and Recommendations

Evaluation findings and recommendations are outlined in Sections 2.1 through 2.5.

2.1 Continued Need

Key Finding 1: There is a continued need for the EP&S program.

Throughout the year, the CAF, through the conduct of its operations, may create circumstances that are harmful to the environment. This includes physical damage, chemical contamination, pollution, as well as the use of fossil fuels and other non-renewable materials. Furthermore, many bases and training areas (both current and legacy) owned or leased by the Department may have been contaminated by past practices and require environmental remediation to ensure that they no longer will pose a health, safety or environmental risk to both current and future users. At present, there are 1 1569 sites within Canada that require environmental remediation from past CAF activities. In addition, another 84010 sites may potentially contain UXO, which are munitions utilized on training areas that failed to explode; some may have remained in the ground—in some cases since World War I.

As such, to address these concerns there is a need for the federal government to actively apply an environmental protection and stewardship program.

2.2 Alignment with Federal Roles and Responsibilities

Key Finding 2: The EP&S program aligns with federal roles and responsibilities.

The evaluation team concluded that the EP&S program aligns with federal roles and responsibilities.

Environmental acts and regulations of Canada require that DND/CAF monitor all of its actions and mitigate and minimize the impact of those activities on the environment.

In protecting the environment, DND/CAF must respect, among others, the Canadian Environmental Protection Act, Fisheries Act, Canadian Environmental Assessment Act, 2012, Species at Risk Act, Migratory Birds Convention Act, and Arctic Waters Pollution Prevention Act. These acts require that no one “intentionally or recklessly causes a disaster that results in a loss of the use or the non-use value of the environment”, “shows wanton or reckless disregard for the lives or safety of other persons and thereby causes a risk of death or harm to another person”, or carries on “any work, undertaking or activity that results in the harmful alteration or disruption, or the destruction, of fish habitat”.

In ensuring that UXO is treated appropriately, the EP&S program develops and administers appropriate measures to ensure the safety of Canadians. The Transport of Dangerous Goods Act and the responsibilities delegated to the Minister of National Defence by the Explosives Act and the Regulations emanating from it support these actions.

In assessing the environment the DND/CAF must respect the Canadian Environmental Assessment Act, 2012, which may require DND/CAF to perform an Environmental Assessment.

2.3 Alignment with DND/CAF Priorities

Key Finding 3: The EP&S program aligns with DND/CAF priorities.

The EP&S program aligns with one of DND’s four current priorities,11 the priority of Ensuring Defence Affordability. The EP&S program aligns with these two sub-elements of this Priority:

Priority Element: The ongoing requirement of integrating environmental considerations into the wide breadth of activities undertaken within Defence in support of the four pillars upon which Canada's military capabilities are built: personnel, equipment, infrastructure and readiness.

Priority Element: Defence must demonstrate compliance with architectural, fire protection, engineering, environmental and related legislation, regulations and standards, including an overall risk-based compliance and monitoring framework.

The DND/CAF EP&S program contributes to the delivery of these priority elements by developing and implementing a Defence Environmental Strategy (DES) that promotes health and safety12 and supports sustainable development on both DND/CAF lands and when deployed on operations. The Program delivers environmental protection and stewardship compliant with existing legislation and federal policy.

2.4 Achievement of Expected Outcomes (Effectiveness)

To assess the Program’s effectiveness, the evaluation team gauged its ability to meet the following expected outcomes13:

  • Canadians are safeguarded against UXO threats;
  • priority contaminated sites are remediated to acceptable levels;
  • training and operations are conducted in an environmentally responsible manner; and
  • departmental environmental policy targets are met.

Data for performance measures was obtained from program reports and documentation, case studies, financial information and interviews with program managers and other stakeholders.

2.4.1 Canadians are Safeguarded against Unexploded Explosive Ordnance

Key Finding 4: While the evaluation team determined that there exists a sound process and plan to address the threat of UXO, the extent to which the process has been implemented raises some concern.

Key Finding 5: There is a need for clear direction as to what areas are to be deemed priorities for the Program to focus upon.

The Minister of National Defence has a responsibility for all explosives used by the DND/CAF within the Canadian land mass and territorial waters regardless of its origin. This includes UXO. The DND/CAF response to the risk posed by UXO is governed by the principle of due diligence. In the event of an accident, DND/CAF must be capable of demonstrating that all reasonable measures14 were taken to prevent its occurrence.

To assess the Program’s effectiveness to safeguard Canadians against UXO, the evaluation considered the following performance measures:

  • qualitative rating of UXO management practices and procedures; and
  • comprehensiveness of application of practices.

Assessment of UXO Management Practices

Based upon an examination of program documentation, interviews with program managers, and a review of actual practices,15 the evaluation team determined that an effective process is in place to successfully assess the risk posed by UXO, and to conduct activities to mitigate such threats.

The management process of UXO Legacy Sites is well defined and consists of: site prioritization, risk assessments, and risk mitigation.

Not all sites (past or legacy) may contain UXO. Site prioritization determines the potential UXO risk, allowing DND/CAF to determine if further review is required, and to manage any such sites on an ongoing basis. The scope of assessment includes desktop reviews (high resolution radar data, map data, stakeholder information, and online resources) and a prescriptive, module-based “litmus test for the conditions that may result in UXO.”16 A priority ranking is subsequently assigned (namely, No Issue, Low Priority Site, and Priority Site).

Once assigned a priority ranking, DND/CAF applies risk formulas and ratings, adapted specifically for munitions activities by Directorate Ammunition and Explosive Regulation. These steps assess the probability of an encounter that may result in a UXO detonating.

Risk mitigation and/or management activities are conducted to safeguard against any identified threats. Activities include shoreline sweeps, construction support, subsurface and underwater clearance, and the use of warning signs and other types of communications.

In the event of an immediate threat (i.e., if an explosive device is found), DND/CAF has measures in place for an immediate response. The CAF explosive disposal units are the first responders when such a call is received. There are arrangements for contractors to perform surveys and clearance activities on short notice, typically within 48 hours. Examples of successful clearance operations leading to reduction of risk include former Canadian Forces Bases (CFB) at Rivers, Manitoba; Granby, Québec; and McGivney, New Brunswick (See Annex F for a complete list of assessed and cleared sites).

No UXO related deaths or injuries were noted during the evaluation period.

2.4.2 Comprehensiveness of Application of Practices

While the evaluation team determined that a sound process and plan to address the threat of UXO exists, the extent to which the process has been implemented raises some concern.

At the time of the evaluation, 840 sites were identified as potentially containing a UXO threat. Only 11617 of them have to date been assessed and assigned a Priority Ranking and risk assessment. At the current assessment rate of 4018 sites per year, the remaining un-assessed sites will not have been examined until FY 2032/33.19

The evaluation team was unable to find any documentation that confirms whether this is an acceptable rate. A range of opinions was found among stakeholders. Interviews revealed that some desire to proceed more expeditiously on conducting the priority ranking and risk assessment of the remaining 724 sites. Others felt that maintaining the current pace of site prioritization (40 per year) is acceptable, and any additional effort should be directed towards the actual mitigation of areas already deemed to be mid- to high-level risks.

However, until all sites are risk assessed, and mitigation and/or management plans are implemented at required sites where UXO are deemed present, the potential risk to human health remains unknown.

Interviewees also expressed concern over insufficient allocation of resources to some known high and medium-risk sites, and gaps in understanding the exact requirements where potential issues may exist. There appears to be no clearly defined process in place to determine the order in which to mitigate sites that have been assessed with risks. For example, the evaluation team found that, at times, areas deemed to be “low risk” may have mitigation activities performed ahead of high-risk sites.

CRS Recommendation

1.         ADM(IE) should conduct a formal risk-based assessment on the rate of priority ranking and assessment of known UXO sites.

OPI: ADM(IE)

CRS Recommendation

2.         ADM (IE) should develop a multi-year plan with clear rankings to determine the order in which UXO sites will have mitigation activity conducted.

OPI: ADM(IE)

2.4.3 Priority Contaminated Sites Are Remediated

Key Finding 6: DND/CAF has demonstrated effective remediation of contaminated sites.

Key Finding 7: Long-term funding for the monitoring of remediated sites is uncertain.

A second key expectation of the Program is that land contaminated by the Department is remediated effectively, in conformance with environmental legislation and government regulations. Since record-keeping began in the mid 1990s, approximately 1,844 sites have been identified in Canada as having possibly been contaminated at some point by DND/CAF operations. Approximately 287 of those sites have to date been remediated to the satisfaction of stakeholders, and each year a further 80 sites are remediated successfully. Approximately $40 million is spent annually on site remediation activity.

The following table shows the number of sites remediated during the period covered by the Evaluation (FY 2008/09 – FY 2012/13).

Data is derived from the Federal Contaminated Sites Inventory database. Note the drop in the number of remediated sites from FY 2009/10 to FY 2010/11. This actually represents a change in classification as opposed to a reduction in work. Prior to FY 2010/11, a “remediated” site was one where remediation had begun. Commencing with FY 2010/11, a “remediated” site was one where the work had been completed and the site was closed.

For FY 2012/13, no data was available, since during that year ADM(IE) had switched from entering data into EcoNet to entering data into the Defence Resource Management Information System (DRMIS). The data conversion from EcoNet to DRMIS was faulty, and subsequent data entry into DRMIS was in error. The evaluation team was unable to determine if resources had been assigned to correct the data or to enter data for subsequent years.

Table 1. Remediated Sites.

Table Summary.

This table shows the number of contaminated sites remediated by Fiscal Year. The fiscal years are listed in the left-hand column. Read across each row for the number of remediated sites.

 

FYSites Remediated
2008/09 345
2009/10 362
2010/11 74
2011/12 84
2012/13 n/a

To assess if the Program’s remediation activities were effective, the evaluation team considered these criteria:

  • conformance to legislation and regulations;
  • local stakeholder acceptance; and
  • compliance with the contractual terms and conditions.

Three case studies of remediation projects were examined to make the assessment. These included the DEW Line, the Mid-Canada Line, and the Goose Bay remediation projects. A summary of the key findings about each case study follows. In all the mentioned cases, the evaluation team found clear evidence of:

  • effective project management;
  • excellent cash management and expenditure control;
  • extensive use of stakeholder engagement;
  • appropriate use of oversight and management committee meetings, and
  • adherence to the terms and conditions of contracts.

Case Study 1: DEW Line

The DEW Line remediation was a $583.3 million project to remediate 21 contaminated sites across the far north.20 Work began in 1989, and is planned to be completed by March 2015.21

The 21 sites cleared by the DEW Line project all met the applicable environmental regulations. Continuous monitoring of the 21 sites to ensure ongoing compliance and remediation is underway. Monitoring is to continue up to FY 2033/34, although funding for it has not yet been put in place.

All stakeholders were deemed to be satisfied with the remediation. The use of local Aboriginal communities for labour and other services contributed greatly to stakeholder satisfaction. As a result, there are now several Inuit companies capable of undertaking large construction projects costing more than $30 million, and lasting several years (e.g., Mary River, Baffin Island22 or the DND contaminated sites on Ellesmere Island).

Oversight of the project was conducted by the DND Nunavut Tunngavik Inc. Steering Committee, formerly chaired by the Director General Environment.23 Performance measures were placed in each contract and included a schedule with milestones, delineation of tasks and responsibilities, cash flows, performance bonds, and human resource allocations with respect to the use of Aboriginal workers. Evidence indicated compliance with all of these.

Environmental Disclosure Reports are being prepared for each of the 21 sites, and documentation will be archived in accordance with ADM(IE) procedures.24 Each such Report contains a record of the sites’ condition at the start of remediation, and their condition at the end of remediation. A final Environmental Disclosure Report (Book 22) contains a record of human resource and financial issues for the entire project.

Case Study 2: Mid-Canada Line

The Mid-Canada Line remediation project was a Contribution Agreement of $30 million to assist the Province of Ontario in the remediation of 11 contaminated sites in northern Ontario. The project was successful. As this is a Contribution Agreement, so subject to conditions for review under the Financial Administration Act, this project is reported by CRS separately.

Case Study 3: Goose Bay

The Goose Bay project will remediate sites contaminated with hydrocarbons, PCBs, chemical waste, and human garbage. Planning for the remediation began in 2004, and the original upper limit for which spending approval was received was $300 million. Revised estimates and experience from work commenced indicate that the total project spending over the period 2010 to 2020 will be approximately $220 million. Beginning in 2010, work commenced on eight sub-projects, and another two sub-projects are in the planning phase. Remediation work is expected to be completed in 2020. Funding for long-term monitoring after 2020 has not been approved.

The evaluation team found that remediation to date has been effective. Although there were scoping issues, this is typical for work of this type, and sound practices were put in place to control costs and ensure that proper assessments and management were being instituted.

While none of the ten sub-projects has been closed to date, the evaluation team found no evidence at this early stage to indicate that stakeholder acceptance will not be forthcoming.

2.4.4 Sustainable Ranges and Training Areas

RTAs are used for a variety of purposes, ranging from military training to communications. They host various users, including local militia and community associations, and undergo maintenance, construction, preservation and other activities.

To assess the Program’s effectiveness in ensuring sustainable use of RTAs, it was examined in terms of the following criteria:

  • extent to which RTAs are not available due to environmental concerns; and
  • extent to which RTAs monitor and comply with regulations.

Environmental Concerns and Impacts on RTA

Key Finding 8: Environmental issues have impacted the use of some key training areas. While these are largely the result of the cumulative effect of activities over the past decades on the RTA, they now require the CAF to implement mitigation activities, which reduces training flexibility and affects operational readiness.

Interviews with program managers and user/stakeholders stated that the commanders’ operational requirements for an available, accessible and adaptable RTA25 are generally met. An analysis of the frequency of RTA closures confirmed that RTAs are usually available, and that commanders are conforming to environmental issues.26

That being said, environmental issues have impacted the use of some key training areas. While these are largely the result of the cumulative effect of activities over the past decades on the RTAs, they are now requiring the CAF to implement mitigation activities, which are reducing training flexibility and affecting operational readiness budgets.

For example, the demolition range located in Sector 6 of the Valcartier RTA had a moratorium ordered in May 2012 on the use of RDX (Trinitrotrimethylenetriamine)27 explosives. The moratorium was implemented due to the proximity of Sector 6 to the Garrison boundaries and municipal water wells and due to the fact that high levels of RDX (exceeding provincial guidelines) had been measured in both surface water and groundwater. Valcartier has been able to mitigate the operational impacts of this moratorium by conducting demolition activities in other locations of the RTA, although the moratorium has reduced the number of sites on which demolition training can occur.

Similarly, erosion issues at 5 Canadian Division Support Group Gagetown have resulted in sedimentation of watercourses within the RTA, resulting in non-compliance with the federal Fisheries Act and the New Brunswick provincial Clean Water Act. DND began to invest significantly under the Sedimentation and Erosion Control project to address these issues. However, recent funding cuts to the Sedimentation and Erosion Control project have resulted in the need to alter military training. Depending on environmental conditions, the Base closes roads and ranges and places restrictions on mounted maneuvers. For example, the 2014 iteration of the Combat Team Commander Course may be moved to the fall since spring is a high-risk period for sedimentation contamination.

RTA Monitoring and Compliance with Regulations

Key Finding 9: A significant proportion (88 percent) of Base/Wing EnvOs felt that there is a significant lack of support with regard to legislative and environmental policy direction, and often must look elsewhere for assistance.

An effective EP&S program will ensure that all RTAs comply with appropriate environmental regulations. Non-compliance not only reduces CAF’s ability to act as a sound environmental steward, but can also result in fines and or closures that impact the ability of the CAF to train. To assess compliance, the evaluation team examined the RTA’s use of environment assessments, in-out clearance procedures, pollutant monitoring, and the frequency of compliance violations.

In doing so, the evaluation team noted that there appears to be an issue concerning the availability of advice and support to Base/Wing EnvOs. A survey of these individuals noted that 88 percent of the respondents felt that there was a significant lack of support with regards to legislative and environmental policy direction. Many felt that this has been exacerbated since the recent re-organization of ADM(IE). Respondents indicated that, when they had a question about environmental policy, they now went to Base/Wing staff within DND/CAF, or to their respective contacts at Environment Canada, instead of to ADM(IE).

This may be a communications issue, as management within ADM(IE) have indicated that, post re-organization, all the same resources remain in place to support the Base/Wing EnvOs.

Environmental Assessments

Program data tracking the need to conduct Environmental Assessments was not always complete; however, it appears that for the most part they are being conducted. In some instances, however, a lack of qualified EnvOs had prevented assessments from being conducted.

In-Out Clearance

Evidence indicated that in-out clearance operations (through the use of Range Standing Orders) are conducted to ensure compliance and/or conformance to environmental requirements before and after training operations. The evaluation team found minor variances (garbage disposal, driving off track) from Range Standing Orders, but for the most part there was compliance.

Pollutant Monitoring

The Department was found to be in compliance with all pollutant monitoring requirements.

Some pollutants, such as PCBs, continue to be found in areas under DND/CAF administrative responsibility,28 although the Department is striving to eliminate them. ADM(IE) staff stated that the use of PCBs within DND/CAF has shown a downward trend since the 1980s, as the following data indicates:

Table 2. PCB Destruction by Fiscal Year.

Table Summary.

This table shows the amount of PCBs by their weight in kilograms for calendar years 2009 to 2012. Columns show amount of PCBs by weight. Rows show calendar year. Read across for the weight for that year of PCBs that were in use, in storage, sent for destruction, and destroyed.

 

Year Kilograms
In Use In Storage Sent for Destruction Destroyed
2009 29 1,220 3,256 3
2010 0 658 2,528 1,890
2011 0 557 3,660 100
2012 0 640 73 1,090

For many pollutants, no national targets or standards exist. As such, there is often little monitoring of use, and in many instances the only manner in which the Department becomes aware of a non-conforming practice is when a regulatory authority flags a violation. This includes the use of halocarbons and wastewater and effluent, as well as approximately 21 other chemicals listed in National Pollutants Release Inventory.

There are also currently no national performance requirements or associated targets with storage tanks. Interviewees stated that, as per the regulatory requirements, the majority of work to ensure that DND/CAF storage tanks are registered in the national database (Federal Identification Registry for Storage Tank Systems) has been completed—yet no data was available to substantiate this.

Compliance Violations

The Environmental Program tracks cases of non-compliance as well as environmental enforcement actions taken against DND/CAF. The number of non-compliance cases varies from year to year, although the number of violations found between 2008 and 2011 (the study period of the evaluation) appears to be higher than the number of violations noted for the previous four-year period (24 versus 19). Furthermore, during the evaluation period there was a significant workforce reduction of environmental inspectors at Environment Canada, which may impact the data. Data from 2001 to 2013 is given in the following table. Data for 2012 and 2013 was not available to the evaluation team.

Table 3. Non-Compliance Actions by Fiscal Year.

Table Summary.

This table lists the number of violations for fiscal years 2001 to 2013. The years are indicated in the first column and are sub-divided by the category of violation. For each calendar year, read across to determine the number of violations per category.

 

YearFHR, 2003PCB RegulationsCEPA, 1999Sulfur & Diesel FuelFisheries ActStorage Tank Systems RegularSKNUTotal
2013   No data
2012   No data
2011 1         1     2
2010 3     1 2 2     8
2009 2           1   3
2008 6   1 1     1 2 11
2007 5   1         1 7
2006   1 1   1       3
2005 2               2
2004 4 2     1       7
2003 2 1     1       4
2002 1               1
2001 3               3
Total 29 4 3 2 5 3 2 3 51

 

CRS Recommendation

3.         ADM(IE) should develop means to ensure that EnvOs have access to the information and support they require to ensure that legislation and regulations are effectively met.

OPI: ADM(IE)

2.4.5 Compliance with Sustainability Policy

Key Finding 10: The evaluation team found it difficult to determine whether or not sustainability and green procurement targets were met. The issue is a lack of substantive baselines and targets within the SDS to adequately measure performance.

The evaluation team assessed DND/CAF’s ability to achieve environmental targets against requirements of DND’s SDS and the Green Procurement Policy. Overall, the Department was seen to be either in compliance with policies or moving towards compliance as implementation plans are being developed. 

The need to integrate operations, finance, health and safety, economic development, and the environment is recognized in the SDS. The objectives of the SDS include:

  • developing and implementing the concept of sustainable use of military training areas;
  • planning and conducting military and non-military activities on DND land and marine training areas, such that adverse impacts are minimized and military training can occur without compromising the capacity for future training; and
  • preserving biodiversity, in particular species at risk.

The evaluation team noted several issues with respect to the ability of the Department to meet the objectives of the SDS. The first objective (developing and implementing the concept of sustainable use of military training areas) has not been achieved, as the Department has yet to define a concept of sustainability within the Department which encompasses both environmental and operational sustainability. 

The second objective has been discussed in section 2.4.3 of this report.

Based upon program documentation, the third objective (preserving biodiversity, in particular for species at risk) is being met. However, there are concerns that this may increasingly conflict with operational training needs as more species may be at risk due to impacts unrelated to DND/CAF activities—e.g., global warming, droughts, natural succession, etc.

The Policy on Green Procurement29 requires departments to use “environmentally preferable goods and services.” These are defined by the Policy as those that have “a lesser or reduced impact on the environment over the life cycle of the good or service, when compared with competing goods or services serving the same purpose.” Environmental performance considerations include, among other things: reducing greenhouse gas emissions and air contaminants; improving energy and water efficiency; reducing waste while supporting reuse and recycling; using renewable resources; reducing hazardous waste; and reducing toxic and hazardous substances.

Annex E provides a detailed table of Green Procurement results achieved. Based on these criteria, the evaluation team examined the frequency and performance measures associated with each relevant initiative. Departments were left to develop and report on performance targets at their discretion. The DND/CAF targets focused on establishing capacity for green procurement. Overall, the evaluation team found it difficult to determine whether or not sustainability and green procurement targets were met. The issue is a lack of substantive targets within the SDS to adequately measure against outcomes that reflect sound environmental protection and stewardship.

CRS Recommendation

4.         ADM(IE) should review all sustainability and green procurement targets and develop a performance measurement strategy for these areas.

OPI: ADM(IE)

2.4.6 Defence Environmental Strategy

Key Finding 11: Meaningful performance measures must be developed for the DES.

The DES30 is intended to integrate all environmental sustainability objectives, goals, and targets. The Strategy was approved by the Deputy Minister/Chief of Defence Staff in May 2013. It addresses the Federal Sustainable Development Strategy (FSDS) requirements and reflects the broader impacts on the environment that are unique to DND/CAF’s mandate. The Strategy makes due diligence and environmental sustainability integral components to the decision-making process.

In the evaluation team’s review of the documentation it was noted that the Strategy lacks a meaningful performance measurement strategy—as such, a means of measuring the initiative’s progress and impacts is severely limited.

2.5 Demonstration of Efficiency and Economy

The following section examines the extent to which the EP&S Program provides value for money by using the most appropriate, efficient, and economical means to achieve its expected outcomes. The evaluation team considered the processes and mechanisms that are in place to monitor efficiency and economy, and expenditures related to the EP&S Program. This information was gathered from an analysis of program financial data, through program business plans, management documents and key informant interviews.

2.5.1 Economy

Key Finding 12: For only approximately 0.5 percent of the total Defence budget, the EP&S program fulfills a critical need. As such, the EP&S program can be deemed to be an affordable program for DND.

Key Finding 13: The Department has reduced overall expenditures on environmental remediation projects as large projects near completion and projects of equivalent value have not occurred.

In FY 2012/13, the annual expenditure of the EP&S program was approximately $91.6 million. This represented environmental remediation, UXO management and mitigation, RTA environmental compliance and sustainability, and associated operating expenditures and salaries from ADM(IE), the RCN, CA and RCAF.

This value represents approximately 0.5 percent of DND’s overall annual expenditures. In return, EP&S ensures that the Department meets legislative and regulatory requirements, and, in doing so, mitigates any health or environmental risks to both DND personnel and those of surrounding communities. Furthermore, it strives to put into place conditions to ensure the long-term usability of training areas, without which CAF would be very hard-pressed to conduct most of its training and readiness preparation needs. Accordingly, it plays a critical role for a very small portion of the Defence budget. As such, for DND the EP&S program can be deemed to be an affordable program.

Table 4. EP&S Spending by Fiscal Year.

Table Summary.

This table shows program expenditures for FYs 2009/10 to 2012/13. Columns show the fiscal years. Rows show the expense categories.

 

 FY 2009/10FY 2010/11FY 2011/12FY 2012/13
DEW Line Remediation $62,168,747 $43,681,780  $41,763,978  $26,296,411 
FCSAP $21,781,969 $9,760,874  $6,476,754  $3,733,167
Goose Bay Remediation $5,580,068 $7,108,846  $6,317,422  $9,865,343 
Grants and Contributions $2,769,854 $4,957,000  $7,500,000 $5,500,000
UXO Program $10,729,070 $12,031,255  $11,713,804 $12,739,667 
Corporate Environmental $933,473  $2,201,524 $6,567,436 $7,675,577 
CAF Salary Wage Envelope $9,191,269 $10,385,540  $10,164,730  $9,366,269 
CAF Project Total $12,411,479 $13,365,544 $20,909,392  $16,485,043 
PROGRAM TOTAL $125,565,928 $103,492,363  $111,413,516 $91,661,478

Source: ADM(IE), FCSAP and Mid-Canada Line Radar Sites—DRMIS. C Air Force, C Army and C Naval—PAA. The evaluation team found discrepancies in overall program expenditures for FY2009/10, 2010/11, 2011/12, 2012/13, as reported by ADM(IE).

As per Table 4, the annual EP&S funding decreased from $125.5 million in FY 2009/10 to $91.6 million in FY 2012/13. This was due to a reduction in expenditures as various FCSAP, DEW Line and Mid-Canada Line Remediation projects neared completion, and additional projects of equivalent value were not initiated.

The expenditures for other areas have essentially remained constant over the past four years, with the exception of the expenditures made by the CAF in support of RTA, which have seen an increase of approximately $4 million dollars, or 33 percent. Given the need to ensure that the RTA remains available for the long-term and the issues facing their operation, this appears to be a much needed investment.

2.5.2 Efficiency

Key Finding 14: Between 2009 and 2013, the efficiency of project delivery declined as the value of projects delivered was reduced, while FTE and associated overhead expenditures remained relatively constant.

Key Finding 15: Recent FTE reductions in EP&S as a response to the Deficit Reduction Action Plan have increased efficiency in corporate environmental program support.

Over the past five years, the amount of civilian FTE dedicated to the program areas has declined by ten percent as a result of the deficit reduction action plan. This represented a reduction of from 46 FTE to 43 FTE. At the same time, the overall value of outputs from the Program decreased as the value of remediation project expenditure declined by 28 percent. As such, the overall efficiency of the Program could be seen to be declining. It is acknowledged, however, that the amount of overhead required to manage a project does not change linearly with the change in project size, and as such the indicator of a ratio of salary to project value should only be used as a rough measure. Accordingly, this does not appear to be an area of significant concern.

The amount of staff dedicated to supporting the UXO program appears reasonable, given the total project expenditures, with salary amounting to approximately six percent of the project’s costs.

Military contributions in support of sustainable ranges have increased over the past five years, while essentially maintaining the level of FTE involved. As such, the efficiency of the operation, measured as the cost of administering projects as opposed to the project value, can be seen to be improving.

Annex A—Management Action Plan

CRS Recommendation

1.         ADM(IE) should conduct a formal risk-based assessment on the rate of priority ranking and assessment of known UXO sites.

OPI: ADM(IE)

Management Action

ADM(IE) will conduct a formal risk-based assessment to determine if the current rate of priority ranking and assessment of known UXO sites adequately addresses potential departmental liability. This risk-based assessment will be conducted by end of FY 2014/15, with the outcome forming the basis of the programs options analysis and resultant funding submission to the Chief of Programme scheduled for this fiscal year. 

OPI: ADM(IE)

Target Date: March 2015

CRS Recommendation

2.         ADM(IE) should develop a multi-year plan with clear rankings to determine the order in which UXO sites will have mitigation activity completed.

OPI: ADM(IE)

Management Action

ADM (IE) is developing a multi-year investment plan for UXO sites that will provide a clear ranking to determine the order in which sites will have assessment and mitigation activities conducted. This work is currently well underway, and will be completed by end of FY 2014/15. The investment plan is an integral component of the programs options analysis and resultant funding submission to CProg scheduled for this fiscal year. 

OPI: ADM(IE)

Target Date: March 2015

CRS Recommendation

3.         ADM(IE) should develop means to ensure that EnvOs have access to the information and support they require to ensure that legislation and regulations are effectively met.

OPI: ADM(IE)

Management Action

As a result of the centralization of Real Property, ADM(IE) will continue to communicate and work with EnvOs through the Level 1 (L1) chain of command. As an interim measure, ADM(IE) updated its 2011 contact list in 2013/14, thereby providing a list of key environmental staff and their responsibilities, from ADM(IE), the L1s and bases and wings.

OPI: ADM(IE)

Target Date: Completed


ADM(IE) will continue to streamline environmental committees and working groups. The committee structures were approved in FY 2013/14, and the working groups and sub-committees will be consolidated in FY 2014/15. This will ensure that EnvOs have an efficient and effective venue to engage and provide input.

OPI: ADM(IE)

Target Date: March 2015


ADM(IE) is rationalizing its policy suite and other tools as part of the departmental initiative to streamline policy direction. This is expected to be completed by FY 2016/17, and will ensure that consistent direction and support is provided.

OPI: ADM(IE)

Target Date: March 2017

CRS Recommendation

4.         ADM(IE) should review all sustainability and green procurement targets and develop a performance measurement strategy for these areas.

OPI: ADM(IE)

Management Action

ADM(IE) promulgated the DES in 2013, to include the FSDS, while also reflecting the broader and unique defence activities and their impacts on the environment. The targets associated with the FSDS were addressed in the DES. 

ADM(IE) will continue to rationalize measures that contribute to the overall Departmental Performance Management Framework, as well as those related to the DES/FSDS. This approach in consultation with L1s is expected to be completed by FY 2015/16.

OPI: ADM(IE)

Target Date: March 2016

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

The Evaluation team used multiple lines of evidence and complementary qualitative and quantitative research methods as means to help ensure the reliability of information and data supporting findings. The methodology established a consistent approach in the collection and analysis of data to support evaluation findings, conclusions and recommendations. Based on the evidence from available sources, the evaluation team reviewed the achievement of expected outcomes and the program’s efficiency and economy to develop a balanced picture of the relevance and performance of the EP&S. Information and data were correlated to each evaluation question and corresponding indicators. To ensure the validity of the data captured, the evaluation team used a data triangulation approach.

1.1 Evaluation Design

A planning phase was conducted to provide an understanding of the scope, complexity, and context of DND/CAF EP&S. During this planning phase, the logic model and evaluation framework were developed based on research conducted by the evaluation team and information and feedback gathered from stakeholders. The Logic Model was confirmed by the stakeholders.

1.2 Data Collection Methods

The evaluation team used multiple lines of evidence and complementary research methods as means to ensure the reliability of information and data to be collected. The evaluation methodology was both qualitative and quantitative.

Data collection methods were selected based on the data required to address performance indicators. The following data collection methods were used to gather qualitative and quantitative data for each type of operation in the evaluation:

  • literature and document reviews;
  • outside information (experts, literature, Allies—United States, Australia and United Kingdom—and academia);
  • defence EP&S key performance indicators;
  • key informant interviews;
  • survey; and
  • administrative and financial data reviews.

1.2.1 Document and File Review

A preliminary document review was conducted, as part of the planning phase of the evaluation, to garner a foundational understanding of the EP&S. A comprehensive document review was undertaken as part of the conduct phase of the evaluation, focusing on the relevance and the performance of the EP&S activities.

The following documents were reviewed during the conduct phase of the evaluation:

  • Foundation Documents: Defence Environment Strategy, Army Environmental Strategy, Federal Sustainable Development Strategy.
  • Corporate Documents: Reports on Plans and Priorities, Departmental Performance Reports, Canada First Defence Strategy and Speeches from the Throne.
  • Legal Documents: Acts and Regulations.
  • Other Documents: Evaluations, audits, other internal studies.

The document review was conducted using a customized template organized according to the evaluation questions and indicators.

1.2.2 Outside Information

Experts in the field of EP&S, literature in the public domain, Allies and academia were consulted to determine the issues affecting EP&S both in Canada and abroad.

1.2.3 Key Informant Interviews

Key informant interviews and information sessions scheduled with EP&S stakeholders, who were directly or indirectly involved in the program delivery, served as an important source of qualitative information.

Interviewees were emailed an interview guide prior to the interview. Individual interviews were conducted in person or by telephone. Follow-up questions were posed and answered by email. The interview guide was explained to interviewees before the beginning of the interview, where they were encouraged to be open and candid in their responses to the questions.

Notes were taken by the evaluators during the interviews, with the authorization of the interviewees. The evaluators transcribed the notes taken during the interviews and compared them with one another with a view at reaching a common record.

A bilingual (English and French) survey on their knowledge of environmental policy was provided to the 30 Base/Wing/Formation/Command EnvOs. Twenty-four responses were received.

The following table lists the organizations where interviews were conducted, and the numbers of interviews held and personnel involved.

Table B-1. Number of Interviewees by Organizations.

Table Summary.

This table shows the number of interviews conducted and the total number of interviewees per organization. The name of the organizations are listed in the left-hand column, followed by a column for the number of interviewees, and another column for the number of total interviews conducted. For each row, read across to determine the number of interviewees and total number of interviews per organization.

 

OrganizationsIntervieweesInterviews
ADM(IE) Environmental staff 44 16
ADM(Mat) Environmental staff 2 3
VCDS 1 1
RCAF Environmental staff 1 1
CA Environmental staff 5 7
RCN Environmental staff 1 1
5 Wing Goose Bay 7 3
CA Petawawa 4 1
RCN Halifax 1 1
1 Canadian Air Division 1 1
CA Western Region 5 6
Total 72 41

1.2.4 Survey of Environmental Policy support and direction

A survey of Base/Wing/Formation/Command EnvOs was conducted to determine their knowledge of environmental policy support and direction. The survey asked four questions:

1. When you have a question on Environmental Policy, who do you ask?

2. Are you comfortable with applying Environmental Policy in your work? If not, why not?

3. Have you received sufficient support and direction (in understanding and executing Environmental Policy)? If not, please provide examples of when, and with respect to what issue, you have needed support but in your opinion it has not been forthcoming.

4. Do you have any further comments?

The survey was sent to 30 individuals and 24 responses were received.

1.2.5 Review of Financial and Administrative Data

The EP&S financial data was reviewed in order to determine the degree of efficiency and economy of the activities and outputs. The data, covering five years from 2009 to 2013, was extracted from the PAA, mapped to data provided by Assistant Deputy Minister (Finance and Corporate Services), and then compared to the ADM(IE) financial reports for accuracy and compliance.

1.2.6 Case Studies of Samples of Projects

Case study methodology was applied to gain full depth of analysis and specific examples of remediation of contaminated/UXO legacy sites to acceptable levels. The case studies illustrated the extent of progress towards achieving the expected outcomes, the issues and concerns arising from the project, and lessons learned.

The case studies included interviews, on-site visits (to Goose Bay and Rockcliffe), and document and data reviews on the following:

  • DEW Line Clean-Up
  • Mid-Canada Line Clean-Up
  • Rockcliffe Air Base (remediation)
  • Lac St-Pierre (UXO)
  • Goose Bay (remediation)
  • CFB Valcartier (remediation)

2.0 Limitations

The following table shows the limitations related to the sources and the mitigation strategies applied to them.

Table B-2. Evaluation Limitations and Mitigation Strategies.

Table Summary.

This table lists the limitations of the evaluation and the corresponding mitigation strategies. The left-hand column lists the limitations and the right-hand column lists the mitigation strategies that the evaluation team adopted.

 

LimitationMitigation Strategy
The reorganization of the environment function within ADM(IE) in the period 2012 – 2014 reduced the number of staff responsible for environmental policy and administration Interviews were conducted with staff who had left DND
After the re-organization most of the environment staff were new to their positions Interviews were conducted with staff who had previously been responsible for the environment portfolio
Attribution of activities and outputs of the program to ultimate outcomes was difficult due to the high level of influence of external factors and lack of data More focus was placed on measuring the immediate and intermediate outcomes, and from these findings deductions were made to assess the achievement of the ultimate outcome
Possibility of the interviewees to provide biased information and only positive stories about their program A comparison was made between interviewees with other people from the same organization or group and information from other sources (documents and files).
The PAA attribution of monies to EP&S was not consistent with the monies reported by the ADM(IE) Comptroller A detailed analysis of accounts was made by ADM(IE) Comptroller to arrive at the cost of the Program
Different terminologies used for the mandate of the EP&S The evaluators referred to Environment Canada documentation to determine the specific terminology

Annex C—Logic Model

Figure C-1. Logic Model for the Environmental Protection and Stewardship

Text description for Figure C-1.

Inputs are people and partners, government funding allocations, information in the form of government direction and Defence requirements, information management and information technology, and processes.

Activities use these inputs to manage UXO sites, remediate contaminated sites, perform environmental sustainability, and conduct federal sustainable development.

Outputs from these activities are site assessments and classification, environmental strategies and plans, and environmental targets.

Immediate outcomes are that measures and activities are conducted to safeguard against UXO threats, remediations are conducted, training and operations are conducted in an environmentally responsible manner, and FSDS and SDS targets are met.

Intermediate outcomes are that Canadians are safe from UXOs, contaminated sites are remediated to acceptable levels, training and operations are conducted in an environmentally responsible manner, and FSDS and SDS targets are met.

The ultimate outcome is that Defence activities on human health and the environment are minimized.

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance.

Table Summary.

This table indicates the data collection methods used to assess the evaluation issues/questions for determining the performance of the Program in terms of its relevance. The three evaluation questions related to relevance are listed in the left-hand column. Read across the rows to determine the indicators used to assess each evaluation question and the lines of evidence used for the indicators and questions.

 

Evaluation Issues/QuestionIndicatorsDetails
RELEVANCE
1 1.1 To what extent does the EP&S continue to address a demonstrable need? 1.1.1 Evidence of continuous need for environmental protection and stewardship for DND/CAF
  • Key informant interviews
  • Program documents and reports
  • Acts and Regulations
1.1.2 Evidence of continuous need for environmental protection and stewardship for Canadians
  • Key informant interviews
  • Acts and Regulations
2 2.1 Does the federal government (and DND specifically) continue to have roles and responsibilities in the delivery of the EP&S activities? 2.1.1 Degree of alignment with federal jurisdiction and with environmental policies
  • Program documents and reports
  • Acts and Regulations
2.1.2 Evidence of complementarities or duplicative initiatives within DND/CAF
  • Program documents and reports
3 3.1 Do the EP&S activities align with the current federal government’s priorities and DND strategic outcomes? 3.1.1 Degree of alignment with the federal government’s priorities
  • Program documents and reports
  • Acts and Regulations
  • Throne Speeches
3.1.2 Degree of alignment with DND strategic outcomes
  • Program documents and reports

 Table D-2. Evaluation Matrix—Performance (Effectiveness).

Table Summary.

This table indicates the data collection methods used to assess the evaluation issues/questions for determining Program's performance in terms of its achievement of outcomes, or effectiveness. The eight evaluation questions related to effectiveness are listed in the left-hand column. Read across the rows to determine the indicators used to assess each evaluation question and the lines of evidence used for the indicators and questions.

 

EvaluationIssues/QuestionIndicatorsDetails (information sources/data requirements)
PERFORMANCE (Effectiveness) 
Immediate Outcomes 
4 4.1 To what extent are measures implemented and activities conducted to safeguard against immediate UXO threats? 4.1.1 Comprehensiveness of environmental protection and stewardship
  • Program documents and reports
  • Key informant interviews
4.1.2 Degree of satisfaction of stakeholders with Environmental Policy
  • Key informant interviews
  • Survey
4.1.3 Degree of satisfaction of stakeholders with the mitigation of environmental damage
  • Key informant interviews
4.2 To what extent is environmental remediation conducted? 4.2.1 Perceived degree of increased knowledge of the stakeholders on these issues
  • Key informant interviews
Intermediate Outcomes
 4 4.3 To what extent are Canadians safeguarded against UXO threats? 4.3.1 Number of deaths and/or injuries resulting from UXO
  • Documentation
  • Key informant interviews
4.4 To what extent are priority contaminated sites remediated to acceptable levels? 4.4.1 Degree to which contaminated sites are remediated
  • Site closure documentation
  • Key informant interviews
4.5 To what extent are training and operations conducted in an environmentally responsible manner? 4.5.1 Degree to which environmental considerations are taken into account when planning and executing training
  • Key informant interviews
4.5.2 Number of training operations cancelled/affected by environmental considerations
  • Key informant interviews
 
4.6 To what extent are ranges and training areas sustainable? 4.6.1 Extent to which the EP&S policies led to sustainable training areas
  • Program reports and documentation
  • Key informant interviews
 4.7 to what extent are FSDS and SDS targets met? 4.7.1 Extent to which FSDS and SDS targets are met
  • Program reports and documentation
  • Key informant interviews
Ultimate Outcome
4 4.8 To what extent is the impact of Defence activities on human health and the environment minimized? 4.8.1 Evidence of activities affecting human health and the environment implemented by the DND/CAF
  • Program reports and documentation
4.8.2 Perceived extent to which the EP&S policies contribute to these changes
  • Key informant interviews

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy).

Table Summary.

This table indicates the data collection methods used to assess the evaluation issues/questions for determining the relevance and performance of the program. The single evaluation question related to efficiency and economy is listed in the left-hand column. Read across the rows to determine the indicators used to assess the evaluation question and the lines of evidence used for the indicators and questions.

 

EvaluationIssues/QuestionIndicatorsDetails (information sources/data requirements)
PERFORMANCE (Resource Utilization: in relation to outputs and in relation to outcomes) (Efficiency and Economy)  
5 5.1 Is the funding of the EP&S the most efficient or economical means of achieving the expected outputs and outcomes for the DND/CAF? 5.1.1 Appropriate processes and mechanisms are in place for monitoring
  • Key informant interviews
  • Program documents and reports
5.1.2 Expenditures related to the EP&S
  • Program documents and reports
5.1.3 Comparison to other nations
  • Key informant interviews

Annex E—Green Procurement Themes

Table E-1. Green Procurement Themes.

Table Summary.

This table shows the initiatives, performance measures, targets and results for the themes. Themes are by column. The rows list the initiatives, performance measures, targets and results for the themes.

 

ThemeInitiativePerformance measureTargetResultComment
Green Procurement The Department will incorporate environmental performance considerations in its procurement decision-making processes. Participation in federal interdepartmental commodity group management teams. As indicated. Achieved. DND participated in 37 of 37 federal interdepartmental commodity group management teams. Performance measures and intended targets relevant to SDS green procurement requirements centered upon participation in these commodity groups. The evaluation team was unable to obtain specific examples of DND's contribution to the development and implementation of green procurement policy, procedures, tools and contracts.
Green procurement capacity building. Number of procurement and materiel management staff with formal green procurement training. 100 percent completion of the online Canada School of Public Service Green Procurement course taken by approximately 21,203 personnel by 31 March 2011.

Not achieved.

24.5 percent

The initial target of training 100 percent was ambitious, given the number of personnel to be trained (approximately 8,100 civilian and 17,500 military members).
Number of acquisition cardholders with formal green procurement training. Training provided to all staff, including acquisition cardholders. Achieved. Targets were revised from 100 percent to 25 percent of civilians and members having completed the green procurement course.
Each department will establish at least three Specific, Measurable, Achievable, Realistic, Time-bound green procurement targets to reduce environmental impacts AND; Each department will establish SMART targets for training, employee performance evaluations, and management processes and controls, as they pertain to procurement decision making. DND/CAF will procure green vehicles for the light commercial fleet where it is cost-effective and operationally feasible. As indicated. Exceeded.

Green procurement targets were reported on track.

Although the DND/CAF does a conscientious job of complying with the Green Procurement Policy, the evaluation found that the integration of environmental measures in contracting, acquisition, equipment disposal, and the power to enforce compliance in contracts, is limited.

The Green Policy sometimes produces aberrations, such as the demand that all vehicles must use Flex-Fuel. The evaluation found that ethanol-based gasoline is not generally available near the W/B/F, so the cost premium paid for Flex-Fuel vehicles is wasted.

DND/CAF will implement the conversion to an environmentally friendly hydraulic fluid common within the 15 aircraft fleets by March 31, 2014. As indicated. Exceeded.  
DND/CF will apply best practices to eliminate from use or restrict in application specified, nationally managed high-risk hazardous materials identified by 31 December each year based upon Risk Management Actions from Environment Canada and Health Canada. As indicated. Exceeded.  
Green Procurement Objectives will be part of Performance Agreements/Management Agreements for relevant personnel . As indicated. Achieved.  
DND/CAF will have in place policies that enable departmental personnel to perform and enforce best practices related to materiel acquisition and support functions As indicated. Achieved.  
Training for select employees.All current DND/CAF employees and members performing Financial Administration Act Sections 32 and 34 responsibilities will complete the Green Procurement Course. As indicated. On track. Opportunity for improvement in training was noted in 2011/12.
Greenhouse Gas Emissions (GHG) The federal government will take action now to reduce levels of GHG from its operations to match the national target of 17 percent below 2005 by 2020.  Percentage of absolute reduction in GHG emissions by FY 2020/21, relative to fiscal year 2005/06.  Ten percent GHG reduction.  On track: 10 percent.  FY 2012/13 data shows the DND/CAF on track to meet its seventeen percent reduction in GHG by 2020.  
Surplus Electronic and Electrical Equipment (EEE) Increase disposal of all departmentally generated EEE.   Existence of an implementation plan for the disposal of all departmentally generated EEE.   As indicated.   Yes. Approved in December 2011.  The evaluation team was unable to determine if the plan had been implemented. 
Green Buildings          Pursuant to departmental strategic frameworks, new construction and build-to-lease projects, and major renovation projects, will achieve an industry-recognized level of high environmental performance.    Number of completed new construction, build-to-lease and major renovation projects in the given fiscal year, as per departmental strategic framework. Nine.  On track. Due to operational priorities, the Department reported that the plans developed to meet the Green Building target were amended and delays may be expected. 
Number of completed new construction, build-to-lease and major renovation projects that have achieved an industry-recognized level of high environmental performance in the given fiscal year, as per the departmental strategic framework. Nine.  On track.         
Existence of strategic framework. As indicated. The strategic framework was approved in December 2011.
New lease and lease renewal projects over1000m2, where the Crown is the major lessee, will be assessed for environmental performance using an industry-recognized assessment tool.   Number of completed lease and lease renewal projects over 1000m2 that were assessed using an industry-recognized assessment tool.  One. On track.
Number of completed lease and lease renewal projects over 1,000 m2 that were assessed using an industry-recognized assessment tool in the given fiscal year, as per the departmental strategic framework.  One. On track.
Existence of strategic framework.  As indicated. Yes. Approved in December 2011.
Fit-up and refit projects will achieve an industry-recognized level of high environmental performance.   Number of completed fit-up and refit projects in the given fiscal year, as per the departmental strategic framework. No target cited. On track.
Number of completed fit-up and refit projects that have achieved a  high, industry-recognized level of environmental performance in the given fiscal year, as per the departmental strategic framework No target cited. On track.
Existence of strategic framework. As indicated. Yes. Approved in December 2011.
Existing Crown buildings over 1,000 m2 will be assessed for environmental performance using an industry-recognized assessment tool. Number of buildings over 1,000 m2, as per departmental strategic framework. 1,513  Opportunity for improvement. 
Printing Unit Reduction Achieve ratio of office employees to printing units.  8:1 average ratio of employees to units.  As indicated.  Achieved  DND/CAF achieved its required targets. 
Paper Consumption By March 31, 2014, each department will reduce internal paper consumption per office employee by 20 percent. Each department will establish a baseline between FY 2005/06 and 2011/12, and an applicable scope.   Each department was required to establish a baseline between FY 2005/06 and 2011/12, and an applicable scope.   As indicated.  Unknown.  The evaluation team was unable to determine if the baselines were established, nor if DND/CAF is on track to meet this target.
Green Meetings By March, 2012, each department will adopt a guide for greener meetings.  Presence of a green meeting guide—i.e., leading to less  As indicated.  Achieved. Approved in 2011.   A Guide was produced. The evaluation was unable to determine if the Guide is followed. 

Annex F—Key Examples of Successful Clearance Operations Leading to Reduction of Risk Level

  • The clearance at former CFB Rivers, Manitoba, of approximately twenty-six hectares of farmland is the location of the most recent incident involving the public and UXO that resulted in an injury. The Program conducted clearance operations in FY 2007/08, and site work is considered complete. The risk is categorized as low; no further action is expected at the site.
  • A further example of successful clearance took place at what was a mortar manufacturing facility during World War II, about two kilometers south-east of downtown Granby, Quebec. The site is located near public zones, with a residential area on the eastern side of the site and a road on its western boundary. Some 80,000 smoke mortars were cleared from the site in FY 1982/83, as a result of improper disposal by the company after the war. The legacy sites program completed an additional clearance in FY 2012/13, to ensure that no UXO remained, and in advance of the site’s potential development to residential use. Site work is considered complete and the risk is categorized as low.
  • In Kamloops, British Columbia, the Program conducted clearance operations of a disposal pit at a former munitions storage magazine in FY 2011/12. The land was being prepared for development when a pit of buried munitions scrap was discovered. The Program surveyed the site and cleared the single known pit. Site work is considered complete and the risk is categorized as low.
  • In FY 2010/11, the Program conducted clearance operations near the village of McGivney, New Brunswick. A former CAF service member contacted CFB Gagetown regarding UXO he had encountered on the property while trapping near the former disposal/demolition area. A site clean-up was conducted in November 2011. The resulting debris was removed from the site: 580 kilograms of munitions scrap, 3,511 kilos of munitions containers and 76,700 kilos of non-munitions debris. The risk is categorized as low. The program managers reported being unsure if further work is required.
  • The following three sites were locations of known munitions debris sold prior to a moratorium on DND’s sale of munitions scrap (imposed during mid- to late-1990’s). All sites are completed with a report of no further action required. Risk levels are categorized as low for both sites.
    • Medicine Hat-Gas City Metals—completed 2010;
    • Medicine Hat-Intercity Salvage—completed 2010; and
    • Toronto-Solway Metal Sales—competed 2013.
  • Within the lands of the Tsuu T'ina First Nation, Alberta, a mortar scrap pit was first identified in 2002 in response to an Explosive Ordinance Device Call. Unconfirmed anecdotal information suggests that the mortar pit was created from items stockpiled during agricultural activities on adjacent areas, rather than originating as a discrete military dumpsite. It is suspected that the pit had been in use for 40-50 years for both range and domestic waste. Clearance was completed in 2010. This site lies within the Greater Calgary management Area. While the mortar pit site is cleared, issues remain in the Calgary Area.
  • The evaluation team also reviewed suspected UXO sites where detailed risk assessments and clearance operations have not taken place. This may impede land development, incur First Nation claims, and create liability to the Crown. Issues remain in Calgary. In July 2013, two unexploded shells were found on the shores of the Elbow River in the Weaselhead area, exposing a legacy of UXO that lies just beneath the surface of a portion of southwest Calgary, including the potential route of its southwest ring road. The route is considered a critical piece of economic development for the province, and impacts land development for the Tsuu T’ina First Nation. Current efforts are underway to conduct detailed risk assessments at suspected sites, with possibility of clearance operations. A further example is land owned by the Okanagan Indian Band in Goose Bay Lake, British Columbia. The land has potential for real estate development. The band contends that there has been little effort to decontaminate the area despite an agreement DND signed in 1952 to remove any UXO from the area.31
  • In Vernon, British Columbia, a developer is suing the federal government for allegedly not disclosing that a former military camp might be contaminated with UXO from World War II training activities. The lawsuit alleges that the government violated the Explosives Act by abandoning UXO and not informing the developer. Since 1945, eight people have been killed and three wounded by UXO left at the camp and on nearby land, including two boys who died in 1963 while trying to pry open a UXO. In response to the lawsuit, DND “denies that it intentionally concealed facts relating to the possible existence of UXO …in the lands.”32 The case is expected to be heard by the British Columbia Supreme Court in the fall of 2014.

___________________________________________________________________________________________________________________________

Footnote 1 Includes approximately $26 million in CAF expenditures.

Footnote 2 Source: Director General Infrastructure and Environment Engineering (DGIEES).

Footnote 3 Source: Director Environmental Engineering Management (DEEM).

Footnote 4 Ibid.

Footnote 5 Includes CAF contributions of approximately $25.8 million towards sustaining RTA.

Footnote 6 There is a difference between “sustainable RTAs” and “sustainable training.” Sustainable training does not necessarily result in sustainable RTAs. RTAs are not nature preserves. They are pieces of land used by the CAF to train soldiers. Sustainable RTAs are defined in a way that permits the CAF to conduct training in perpetuity.

Footnote 7 Active at the time of the evaluation.

Footnote 8 Subject to legislation, regulations and Defence Administrative Orders and Directives (DAODs), such as DAOD 1001-0 and DAOD 4003-0.

Footnote 9 Source: DGIEES.

Footnote 10 Source: Director Environmental Engineering Management.

Footnote 11 DND Report on Plans and Priorities 2013-14.

Footnote 12 In conjunction with the VCDS.

Footnote 13 See the Methodology Section in Annex B and the Logic Model in Annex C.

Footnote 14 Interviews with ADM(IE) staff and Canadian Forces Legal Advisor staff.

Footnote 15 Examples include the discovery and clearance of UXO at Lac St-Pierre, Québec, Churchill, Manitoba, and Tracadie, New Brunswick, and detailed risk assessments and survey operations conducted in Calgary’s southwest ring road development and on lands surrounding the T’Suu T’ina First Nations, Alberta.

Footnote 16 UXO and Legacy Sites Program Risk Tools. November 8, 2013.

Footnote 17 Source: DEEM.

Footnote 18 Ibid.

Footnote 19 CRS calculation.

Footnote 20 A further 21 sites were transferred to the responsibility of the Department of Indian and Northern Development. Approximately one third of those sites have been remediated.

Footnote 21 The remediation work was not completed in 2013, as weather hampered completion (viz., winter ice developed faster than expected, and hindered work).

Footnote 22 The Mary River Property is an iron ore mine located on North Baffin Island, in the Qikqtani Region of Nunavut. It is one of the largest and richest undeveloped iron ore projects in the world, and involves the construction, operation, closure and reclamation of an open pit mine. Based on the current defined mineral reserve at Deposit Number 1, the total annual designed output will be optimized to between 18 and 30 million tonnes per year of ore. This Project is expected to operate 12 months of the year.

Footnote 23 Since the reorganization of ADM(IE), it is unclear who now chairs this Committee.

Footnote 24 1262-1 (Director General Environment), Request to Cease the Destruction of Records Relating to Decommissioned DND Facilities, May 11, 1999.

Footnote 25 Including Garrison areas supporting an RTA.

Footnote 26 The evaluation team learned about only one instance of an exception, when a commander took vehicles off the RTA without washing them down (to remove invasive plant species) before exiting the RTA.

Footnote 27 Hawley's condensed chemical dictionary – Hawley, Gessner Goodrich, 1905. New York: Wiley, c1997. xiii, ISBN 0442023243.

Footnote 28 Some equipment still requires the use of PCBs (as per the exemptions provided in the Regulations) and some artifacts that contain PCBs (such as the Enigma Machine) will remain intact to conserve their historical value.

Footnote 29 http://www.tpsgc-pwgsc.gc.ca/ecologisation-greening/achats-procurement/politique-policy-eng.html.

Footnote 30 Prepared by Director General Environment, Strategic Governance and Direction Group, January 2012.

Footnote 31 “The permittee agrees to remove, or dispose of all duds from the permit area at the termination of any firing proactive during which said duds were fired on said permit area.”

Footnote 32 World War II explosives trigger lawsuit by British Columbia developer, CBC News, May 30, 2013.

Date modified: