Evaluation of the CASARA Contribution Program

November 2014

1258-216 (CRS)

Reviewed by ADM(RS) in accordance with the Access to Information Act.  Information UNCLASSIFIED.

Acronyms and Abbreviations

CAF

Canadian Armed Forces

CASARA

Civil Aviation Search and Rescue Association

CDS

Chief of Defence Staff

CRS

Chief Review Services

DIV

Division

DND

Department of National Defence

DPR

Departmental Performance Report

FY

Fiscal Year

GC

Government of Canada

ICSAR

Interdepartmental Committee on Search and Rescue

IAMSAR

International Aeronautical and Maritime Search and Rescue

JRCC

Joint Rescue Coordination Centre

MOU

Memorandum of Understanding

NIF

New Initiatives Fund

NSS

National Search and Rescue Secretariat

PAA

Program Alignment Architecture

RBAF

Results-based Audit Framework

RCAF

Royal Canadian Air Force

RPP

Report on Plans and Priorities 

SAR

Search and Rescue

RMAF

Results-based Management and Accountability Framework

USCG

United States Coast Guard

Executive Summary

Overall Assessment

  • The contribution CASARA provides to the DND/Canadian Armed Forces (CAF) SAR response is still needed and is an effective support organization despite its decline in use and membership in recent years.
  • Operating costs for CASARA have increased significantly compared to the usage rates. This is partially due to external influences such as fuel costs. There may, however, be some savings opportunities by using simulators for training and refining the governance/administration model.
  • Overall the CASARA Program continues to provide good value to the DND/CAF; therefore, funding should be renewed for another five year term.

This report presents the findings and recommendations of the evaluation of the Department of National Defence (DND) Contribution Agreement with the Civil Air Search and Rescue Association (CASARA) Program. Chief Review Services (CRS) of DND conducted the evaluation in compliance with the Government of Canada (GC) Policy on Evaluation (2009).

Program Description

CASARA is a non-profit volunteer organization that provides an organized and trained volunteer pool to:

  • support DND Search and Rescue (SAR) operations;
  • promote SAR prevention; and
  • support Canada’s SAR Program.

The CASARA Contribution Program is administered from within DND by the Royal Canadian Air Force (RCAF). The contribution to CASARA falls under section 2.2 of the DND Program Alignment Architecture (PAA) 2013 and involves annual expenditures of $3.1 million. The terms and conditions for CASARA were approved in March 2010 and are effective until March 2015.

Relevance and Performance

The CRS evaluation team determined that the need for DND/CAF to employ the capability generated by CASARA is of continuing relevance and aligns with federal government and departmental roles, responsibilities and priorities. While the contribution to CASARA is a very small portion of DND/CAF annual spending, the impact of its contribution to overall departmental strategic objectives and priorities remains high. CASARA can directly impact and improve the safety of Canadians and provide public confidence in the Government’s ability to protect Canadian citizens at home.

Tasking of CASARA for SAR missions has declined in recent years due in part to fewer major search missions and/or shorter search times required on SAR incidents. In spite of this, the Program’s impact is still significant, especially in more remote areas of the country. Despite the reduced membership levels, it has maintained the level of training and capabilities required for year-round support to DND/CAF and the National SAR Program.

In recent years, efficiency measures have been implemented, particularly in training, despite rising external expenses, such as the cost of fuel and aircraft rental or charter fees. Potential for additional savings may also exist in the current governance model of CASARA, which maintains a number of membership organizations and governing boards. Outreach and awareness activities by CASARA are also limited, reduced to being mostly local and regionally based, without a broader national strategy or coordination nor any demonstrable impact. Consideration should be given to discontinuing promotion and safety awareness activities.

Key Findings and Recommendations

Key Finding 1: Continued need is demonstrated through approximately 900 SAR taskings for CASARA over the five year evaluation period. These incidents, however, required only 35 percent of the number of flying hours compared to five years ago. This could be an indication that changes in technology and safety awareness are having an effect on the length of time required for searches. 

Key Finding 2: CASARA is aligned with the SAR priorities of DND and the GC by supporting SAR missions with air resources and air-search capabilities.

Key Finding 3: CASARA is aligned with the roles and responsibilities of the federal government by augmenting mandated DND/CAF SAR activity.

Key Finding 4: Although membership is down overall and in certain geographic regions, this does not appear to be having a negative effect on mission support to DND/CAF for SAR responses.

Key Finding 5: The number of available aircraft declined by approximately 10 percent over the evaluation period for both member-owned, and rented and chartered aircraft.

Key Finding 6: CASARA training hours have been reduced in recent years and in response to this reduction, CASARA has adopted mechanisms such as using simulators to accommodate for training shortfalls.

Key Finding 7: Over the five year evaluation period, CASARA contributed to 18 percent on average of RCAF SAR incident responses. This value has been decreasing annually from 20 percent of all RCAF SAR responses in 2010 to 12 percent in 2013. 

Key Finding 8: The performance reporting of CASARA’s outreach and prevention activities do not measure longer term outcomes or effectiveness. Therefore, it is uncertain if it has had any impact on improved safety.

Key Finding 9: To address increased fuel costs CASARA has several initiatives to drive efficiencies in training.

Key Finding 10: The administration/governance of the Program1 has multiple layers and represents almost 30 percent of the Program costs.

Key Finding 11: Should the amount of hours CASARA is tasked by DND/CAF continue to decline, the ongoing value of the Program may be reduced. As it currently exists, the real value of the Program is its ability to augment and enhance DND/CAF’s SAR search capabilities and response time.

Recommendation 1: DND/CAF should develop targets for the membership levels and distribution of CASARA resources relative to the tasking level and hours required for flying (for training and missions). DND/CAF should also respond if the level falls below a minimum acceptable level, which needs to be determined.

Recommendation 2: Increase the use of alternative training methods, such as using the SAR simulator, across all membership organizations in CASARA.

Recommendation 3: Unless performance measures can demonstrate the impact of prevention and safety awareness activities, consider redirecting these financial resources towards training expenditures.

Recommendation 4: CASARA should streamline its management and governance structure to reduce administrative and meeting costs.

Recommendation 5: Increase efforts on outreach beyond 1.3 percent of spending if recruitment targets that need to be established are not met.

Recommendation 6: Despite the decline in use, overall the Contribution to CASARA remains good value for the department and the Program should be renewed.

Note: For a more detailed list of CRS recommendations and management responses, please refer to Annex A—Management Action Plan.

1.0 Introduction

1.1 Profile of CASARA

1.1.1 Background

In accordance with the GC’s Policy on Evaluation (2009), CRS evaluated the DND/CAF Contribution Agreement for CASARA. The evaluation began in November 2013 and covered the period between 2009 and 2014. DND last evaluated the CASARA Contribution Agreement in April 2009. The purpose of this evaluation is to assess the relevance, design, implementation and outcomes to 2014 of the funded activities for CASARA since the last evaluation.

1.1.2 Program Objectives

The specific activities, outputs and outcomes of CASARA are captured in a program logic model and integrated into the Results-based Management and Accountability Framework (RMAF) and Results-based Audit Framework (RBAF) that was developed in 2010. The following immediate, intermediate and ultimate outcomes were identified in the RMAF:

Immediate Outcome

  • A national distribution of search capable resources that augment and compliment the efforts of CAF SAR response
  • A nationally distributed volunteer pool able to deliver SAR Awareness briefings to the general aviation community

Intermediate Outcomes

  • A reduction in the time and effort required to resolve CAF SAR coordinated  SAR incidents
  • Increased public awareness of CASARA and the federal SAR response mandate

Ultimate Outcomes

  • An aeronautical SAR response that is as effective as practicable
  • A general aviation community that understands how the SAR system serves their needs and is proactive in increasing their chances of survival

1.1.3 Program Description

CASARA is a nation-wide, non-profit volunteer organization formed in 1986 and funded by the Department of National Defence. It was founded as a means to provide an organized and trained volunteer support to DND/CAF for SAR operations. CASARA’s mission is promote SAR prevention and to support Canada’s SAR Program.

Funding is provided to CASARA for mission support through DND/RCAF budgets, in addition to funds specific to SAR missions. This funding is for $15 million over five years with a maximum of $3.1 million annually. An additional contingency of $735,300 has been earmarked for additional initiatives in the Arctic. Internal funding of $400,000 from DND has also been provided since 2010 for CASARA outreach and promotion, which Transport Canada previously provided.

CASARA has approximately 2,300 members across Canada including pilots, navigators, spotters, search and rescue coordinators and ground support personnel. They operate over 400 privately-owned fixed wing aircraft. CASARA supports the CAF by providing trained and organized personnel to assist with setting up and operating a deployed search headquarters.

Funding from the Contribution Agreement is for the following activities:

  • training operations;
  • insurance (comprehensive and liability);
  • administration and organization costs; and
  • promoting SAR response awareness (and special projects).2

Funding for CASARA supports training and necessary equipment such as the emergency locator transmitters, ground-to-air radios, computer equipment/programs, telephone pagers and Global Position Systems. The costs for flying hours and rented/chartered aircraft are also reimbursed for training activities. CASARA also provides certified spotters on CAF aircraft, if needed. It also provides a rapid local response to locate missing aircraft or emergency locator transmitters before CAF primary SAR aircraft can be airborne.

1.1.4 Program Governance and Liaison

CASARA has 13 membership organizations, or chapters, across Canada. The member organizations are overseen by the CASARA National and operate as federally incorporated, not-for-profit volunteer organizations. CASARA’s integration with the CAF is a critical factor in its success and the success of the National SAR Program. CASARA is administered and managed by the Director of Air Force Readiness. The chain of command continues up to the Chief of the Air Staff and the Chief of Defence Staff (CDS) and then to the Deputy Minister. The CAF A3 Transport and SAR Readiness Staff are responsible to the Commander, 1 Canadian Air Division, for the CASARA Liaison Officer and co-ordination of CASARA/CAF activities.3

1.1.5 Key Stakeholders and Beneficiaries

The following key stakeholders and beneficiaries were identified in the 2010 CASARA integrated RMAF/RBAF.

Key Stakeholders

The key CASARA stakeholders include:

  • CAF;
  • RCMP and co-deliverers of ground SAR services including provinces and territories;4
  • volunteer organizations; and
  • the National SAR Secretariat (NSS).

Beneficiaries

Beneficiaries to the contribution funding for CASARA include:

  • persons and operators of aircraft and air vehicles using Canadian airspace and facilities, private and commercial;
  • recreational aviators and their families; and
  • persons in distress situations who are able to be located by air.

1.2 Evaluation Scope

1.2.1 Scope

This evaluation includes all relevant data and information from 2009 to 2014. It focuses on the relevance, delivery and performance of those activities funded by the Contribution Agreement for CASARA.

1.2.2 Issues and Questions

To assess the relevance and performance of the Program, the CRS evaluation team considered the following core evaluation issues:

Relevance: The extent to which the Program is still relevant to the need and priorities of the federal government and its partners and is aligned with federal roles and responsibilities.

  • Continued Need. Does the Program continue to address a demonstrable need and is it responsive to the needs of Canadians?
  • Alignment with Government Priorities. Is the Program aligned with federal government priorities and DND/CAF Strategic Outcomes?
  • Alignment with Federal Roles and Responsibilities. Does the Program align with current federal roles and responsibilities?

Performance (effectiveness): The extent to which the Program was successful in achieving its expected outcomes and has been functioning effectively and efficiently; that is:

  • To what extent does CASARA provide nationally distributed, trained, qualified and eligible members; and aircraft, vehicles and equipment that are capable of supporting CAF SAR response activities?
  • To what extent has CASARA contributed to reducing the overall time and effort required to resolve SAR CAF coordination of SAR incidents?
  • To what extent has CASARA contributed to increased public awareness and SAR prevention mandates?

Performance (efficiency and economy): Is the Program using appropriate and efficient means to achieve its functions? How cost effective is CASARA to DND/CAF and the National SAR Program?

2.0 Findings and Recommendations

2.1 Relevance—Continued Need

The relevance of CASARA is determined by assessing

  • the continued/ongoing need of the Program;
  • its alignment with the priorities and objectives of the department and the GC; and
  • its alignment with the roles and responsibilities of the federal government.

Evaluation question: Does the Program continue to address a demonstrable need and is it responsive to the needs of Canadians?

Key Finding 1: Continued need is demonstrated through approximately 900 SAR taskings for CASARA over the five year evaluation period. These incidents, however, required only 35 percent of the number of flying hours compared to five years ago. This could be an indication that changes in technology and safety awareness are having an effect on the length of time required for searches.

The national SAR system responds to more than 15,000 calls each year of which the RCAF, through the Joint Rescue Coordination Centres (JRCCs), responds to over 1,000 air response incidents.5 CASARA members contribute to the RCAF SAR response for approximately 15 percent of all aeronautical incidents across all regions of Canada.

Regarding hours flown, however, CASARA’s contribution to RCAF responses has declined over the past five years, demonstrating a diminishing need. Between 2003 and 2008 CASARA responded to 842 taskings for a total of 4,018.9 hours flown. This evaluation period, 2008–2013, found a similar number of CASARA taskings (912) but they required only about 35 percent of the number of flying hours (1,446). This could be an indication that changes in technology and safety awareness are having an effect on the length of time required for searches. 

While the amount of hours flown and thereby contributed by the Program has declined significantly, it is important to note that CASARA remains an important asset, particularly for its surge potential. The Canadian area of responsibility pertaining to SAR is immense, covering 18 million square kilometers of land and water including three coastal/littoral regions and a population of approximately 34 million people.6 Although the majority of the population resides in the southern part of the area of responsibility, tourism and economic development activity in the northern regions is increasing.7 This increased activity in remote and northern regions may lead to an increase in incidents that require a SAR response. CASARA represents a key resource that can support the DND/CAF SAR capability in these northern and remote areas.

2.2 Relevance—Alignment with Government Priorities

Evaluation question: Is the Program aligned with federal government priorities and DND/CAF Strategic Outcomes?

Key Finding 2: CASARA is aligned with the SAR priorities of DND and the GC by supporting SAR missions with air resources and air-search capabilities.

SAR is identified as a high priority for DND/CAF and the GC, as reflected in the Report on Plans and Priorities (RPP),8 DPR9 and CDS Guidance to CAF.10 Operational SAR is included in the RPP, under the priority, ensuring sustainable operational excellence both at home and abroad, and under Defence priority, Readiness to Respond.11

SAR is included in the DND Program Alignment Architecture (PAA) under section 2.2, Defence Services for Canadian Safety and Security, subsections 2.2.3, National Search and Rescue Program and 2.2.4, Search and Rescue Operations12and Strengthening Canadian Identity (p. 25, 2008/09). SAR is also included in departmental priorities in the Canada First Defence Strategy under section, Defending Canada—Delivering Excellence At Home,13 and continues to be a priority for CAF as indicated in the Royal Canadian Air Force (RCAF) Vectors document.14 CASARA supports these priorities by providing support to SAR activities and objectives.

2.3 Relevance—Alignment with Federal Roles and Responsibilities

Evaluation question:  Does the Program align with current federal roles and responsibilities?

Key Finding 3: CASARA is aligned with the roles and responsibilities of the federal government by augmenting mandated DND/CAF SAR activity.

The GC has federal responsibilities pertaining to SAR operations through its participation in international organizations and as required, by the following signatory treaties, conventions and agreements:

  • Convention on International Civil Aviation (1944): provides assistance to aircraft in distress through a federal aeronautical SAR service (International Aeronautical and Maritime Search and Rescue (IAMSAR) Manual, Volume 4, April 2004: Article 25, A1–3);
  • International Convention on the Safety of Life at Sea (1974): regarding the “adoption and pursuance of common laws and regulations . . . to ensure, from the point of view of safety of life, that a ship is fit for the service for which it is intended” which includes “coast watching and the rescue of the persons in distress at sea and around its coasts” (IAMSAR Manual, Volume 4, April 2004: A1–2; and Canadian Aeronautical and Maritime Search and Rescue Manual, Chapter 1.03.2);
  • International Convention on Maritime Search and Rescue (1979, in effect in Canada since 1985): “to facilitate co-operation between governments and to facilitate co-operation between those participating in SAR operations at sea,” in effect in Canada since 1985 (IAMSAR Manual, Volume 4, April 2004: A1–2); and
  • Memoranda of Understanding:
    • Maritime Forces Pacific (CAF) and the United States Coast Guard (USCG) for mutual assistance;
    • CAF, CCG, USCG and UK federal departments (Coast Guard, Defence, Civil Aviation division of Environment Transport and Regions) defining operational and administrative responsibilities and coordination; and
    • Maritime Forces Atlantic (CAF) and USCG for off-shore petroleum exploration.

DND/CAF’s responsibilities for SAR also include: the Minister of DND as lead Minister for the National SAR Program; the NSS; and, the Interdepartmental Committee on Search and Rescue. CASARA is a volunteer organization that assists the federal government and other partners (provincial, territorial and municipal) when required. It has an MOU15 with DND/CAF and receives direction and tasking orders from the JRCCs across Canada.

2.4 Performance—Effectiveness

Evaluation question: The extent to which the Program was successful in achieving its expected outcomes and has been functioning effectively and efficiently; that is:

  • To what extent does CASARA provide nationally distributed, trained, qualified and eligible members, and aircraft, vehicles and equipment that are capable of supporting CAF SAR response activities?
  • To what extent has CASARA contributed to reducing the overall time and effort required to resolve SAR CAF coordination of SAR incidents?
  • To what extent has CASARA contributed to increased public awareness and SAR prevention mandates?

To determine the performance of CASARA, the CRS evaluation team focused on the following immediate and intermediate outcomes:

  • the extent that a nationally distributed, trained, qualified and eligible CASARA membership, and its aircraft, vehicles and equipment are capable of supporting CAF SAR response activities is available;
  • the extent of CASARA’s contribution to SAR missions; and
  • increased public awareness of CASARA and SAR prevention mandates.

2.4.1 Availability of CASARA’s Resources Capable of Supporting DND/CAF SAR Response Activities

CASARA's ability to support DND/CAF SAR response activities is represented by the following criteria:

  • the number of volunteers and membership organizations available across the country;
  • the appropriate amount and type of equipment and aircraft;
  • the distribution of resources; and
  • the level of appropriate training acquired.

Membership Levels

Key Finding 4: Although membership is down overall and in certain geographic regions, this does not appear to be having a negative effect on mission support to DND/CAF for SAR responses.

In 2013 the membership of CASARA was 2,347 in all key positions, including pilots, navigators, spotters and SAR coordinators. CASARA’s membership numbers dipped between 2010 and 2012, but have started to rise again recently. Table 1 shows the total membership levels.

Table 1. CASARA Total Membership

Table Summary

This table shows the total number of CASARA members per Fiscal Year (FY).

 

FYTotal Membership
2009/10 2,802
2010/11 1,809
2011/12 1,807
2012/13 2,284
2013/14 2,347

Membership levels have been decreasing in past years partly because of aging demographics (the average age of CASARA volunteers in 2009 was 55 and older)16 and challenges in attracting new volunteers. Interviews also noted that the high cost of owning and operating a small aircraft in Canada can be another barrier to new members joining CASARA as well as the time commitment required. Although the sale of small personal aircraft in Canada has increased over the past five years, the increasing cost of aircraft may be prohibiting people from volunteering their services. Also not all private aircraft purchased is suitable for CASARA use.17

Despite the decline in CASARA memberships, the CRS evaluation team has not noted any reduced ability to respond to taskings during the evaluation period. This is mainly due to fewer flights and less time/hours required for searching in support of DND/CAF’s SAR response over the past five years.

Ownership, Rental and Charter of CASARA Aircraft

Key Finding 5: The number of available aircraft declined by approximately 10 percent over the evaluation period for both member-owned, and rented and chartered aircraft.

Having available aircraft and equipment to perform searches is vital for CASARA support. In regions or areas where CASARA member-owned aircraft is not available for missions or training, CASARA rents or charters aircraft and is reimbursed by DND/CAF.

As Table 2 shows, the number of available aircraft declined by approximately 10 percent over the evaluation period, for both member-owned, and rented and chartered aircraft. The proportion of rented or chartered aircraft has remained relatively steady and represents 40 percent of all CASARA aircraft used for missions and training over the past five years.18

Table 2. CASARA Air Assets—Owned, Rented or Chartered

Table Summary

This table summarizes the number of air assets owned, rented or chartered by CASARA members for FYs 2009/10 to 2013/14.

 

FYMember-ownedRentedCharteredOtherTotal
2009/10 250 107 18 30 405
2010/11 222 105 8 36 371
2011/12 232 103 8 36 379
2012/13 229 106 8 37 380
2013/14 226 97 7 41 371

Similar to the decline in memberships, the reduced availability of aircraft does not appear to have reduced the ability to respond to taskings during the evaluation period.

Distribution of CASARA Member Organizations and Resources

A major element of CASARA’s contribution to SAR response is the geographic location of resources, through placement of the air assets and the trained and organized SAR crews that are ready to respond. CASARA has 13 membership organizations, including one in each province or territory, and members and units in regions across Canada. Although coverage exists from coast to coast, membership has been the lowest in the northern territories, followed by the Atlantic coast. The ratio of current to certified members is also the lowest in those same regions, with the Atlantic region as the lowest when comparing certified versus current member numbers.

Table 3 shows the number of members by region and the number of CASARA taskings per region.

Table 3. CASARA Membership by Region

Table Summary

This table summarizes the percentage of total CASARA membership and the percentage of reported JRCC tasking per region of the country.

 

Region% of total CASARA Membership% CASARA-Reported JRCC Tasking
Pacific 33 42
West 26 14
Central 25 31
Atlantic 8 7
North 8 5

Although membership is lowest in the Atlantic and northern regions, tasking and incidents are also the lowest in these regions respectively. As per Table 3, the distribution of CASARA members appears to be appropriate with tasking levels. Despite this, interviews noted some concern that a tasking will not occur from a JRCC unless there are resources available.

Historically, SAR incidents reported in the North have been low, representing approximately 2 to 5 percent of total incidents. Despite this, CASARA has recently increased its efforts at outreach, training and recruiting of northern members particularly in the Nunavut regions of Pond Inlet and Arctic Bay. These communities are small and without dedicated SAR aircraft. CASARA provides an additional resource in these northern communities that will bolster CAF’s SAR capabilities and response time. 

Recommendation

1.        DND/CAF should develop targets for the membership levels and distribution of CASARA resources relative to tasking levels and hours required for flying (for training and missions). DND/CAF should also respond if the level falls below a minimum acceptable level, which needs to be determined.

OPI: RCAF

Training Levels

Training time (i.e., the number of hours) and actual exercises have gone down since 2009/10 from approximately 43,445 hours to 27,751 hours and 976 exercises in 2009/10 to 647 in 2013/14. This decline is due in part to rising costs associated with training from RCAF, which impacts the amount of joint training CASARA can perform. Although the actual budget for SAR training (of CASARA members) has remained stable over the past five years, the costs associated with training (such as fuel) have risen, leading to fewer exercises overall. As per Table 4, the trend in hours of training per member is decreasing.

Table 4. Number of Exercises and Hours Spent on Training

Table Summary

This table summarizes the number of exercises and hours that CASARA members spent on training.

 

FYNumber of ExercisesVolunteer HoursTotal MembersHours per Member
2009/10 976 43,445.9 2,802 15.51
2010/11 809 34,808.8 1,809 19.24
2011/12 667 23,884.2 1,807 13.22
2012/13 664 23,638.8 2,284 10.35
2013/14 647 27,751.7 2,347 11.82

 

Key Finding 6: CASARA training hours have been reduced in recent years and in response to this reduction, CASARA has adopted mechanisms such as using simulators to accommodate for training shortfalls.

Recommendation

2.       Increase the use of alternative training methods, such as using the SAR simulator, across all membership organizations in CASARA.

OPI: CASARA

2.4.2 Frequency of CASARA Use to Support SAR Missions

Key Finding 7: Over the evaluation period, CASARA contributed to 18 percent on average of RCAF SAR incident responses. This value has been decreasing annually   from 20 percent of all RCAF SAR responses in 2010 to 12 percent in 2013. 

A strong measure of CASARA’s contribution to SAR is demonstrated by the number of SAR related incidents it has assisted on over the past five years. Over the five year evaluation period, 5,175 incidents required a CAF SAR response with 912 of these missions supported by CASARA—representing 18 percent of the total. This shows a significant contribution to the DND/CAF SAR response as it allowed for a reduction in flying hours by primary CAF SAR aircraft. In some cases, CASARA assets facilitated quicker response times to SAR taskings due to their geographic proximity of to the search area.

The 18 percent contribution represents an average for the evaluation period. Program data indicates, however, that the frequency of CASARA tasking has decreased over the past five years from 225 in 2009 to 148 in 2013 (20 percent to 12 percent). In addition, the number of hours spent on missions has also decreased. This may be due in part to better aviation and SAR safety and changes to search technology and locating devices. Interviews indicated that the duration of CASARA searches may not be as long as in the past because of the reduction in major searches over the same period as evaluated. In the past, major searches could sometimes take a number of days, but with increased use of the 406 beacon and other locating devices, search duration has decreased, in some cases, significantly.

Table 5 provides the annual number of taskings of CASARA and CAF resources for 2009–2013. It is noted that the frequency of tasking for CASARA in the Halifax SAR Region is quite small compared to other regions. The cause of this appears to be that the SAR responses for the maritime region frequently involve searches over open water or in extremely poor weather (e.g., fog). Under these conditions, the ability of commercial, volunteer pilots and aircraft to contribute is limited.

Table 5. Number of CASARA Taskings Relative to CAF from 2009 to 2013

Table Summary

This table compares the number of times the RCAF and CASARA were tasked from 2009 to 2013 by SAR Region (SRR). (Data is based on JRCC reported taskings. It may include some duplication between CAF and CASARA utilizations.)

 

YearSRRRCAF UtilizedCASARA Utilized% Use of CASARA
2009 Halifax 407 10 2.46
Trenton 405 122 30.12
Victoria 319 93 29.15
Sub-total 1,131 225 19.89
2010 Halifax 370 30 8.11
Trenton 396 139 35.10
Victoria 239 79 33.05
Sub-total 1,005 248 24.68
2011 Halifax 326 15 4.60
Trenton 381 93 24.41
Victoria 250 54 21.60
Sub-total 957 162 16.93
2012 Halifax 364 7 1.92
Trenton 377 91 24.14
Victoria 270 31 11.48
Sub-total 1,011 129 12.76
2013 Halifax 363 12 3.31
Trenton 423 83 19.62
Victoria 285 53 18.60
Sub-total 1,071 147 13.82
Totals 5,175 912 17.62
2.4.3 Effectiveness of Prevention and Awareness

Key Finding 8: The performance reporting of CASARA’s outreach and prevention activities do not measure longer term outcomes or effectiveness. Therefore, it is uncertain if it has had any impact on improved safety.

The following indicators were analysed to assess the overall effectiveness of CASARA’s prevention and awareness activities from 2009–2014:

  • the number of prevention and awareness presentations and sessions provided by CASARA;
  • the number of volunteer hours spent providing these sessions; and
  • the number of non-CASARA people directly exposed to a target message.

Based on the reporting information provided for this evaluation, Table 6 shows a relatively small number of presentations that are provided, which has also been decreasing over the last five years. Safety and awareness presentations by CASARA have decreased from 21 presentations in 2009/10 to 10 presentations in 2013/14. It is unclear, based on reporting data provided what the level or quality of prevention and safety awareness information was. A review of one of the main presentations used in recent years indicated the use of key aviation and SAR safety messages. 

Table 6 also indicates that the number of hours spent on prevention and awareness presentations given by CASARA has decreased significantly from 244 to 83 hours in the past three years. Interviews also indicated that partnerships and outreach activities are also often only local or regional based, lacking a national strategy and coordination.

Table 6. Awareness and Outreach Activities Provided by CASARA from FY 2009/10 to 2013/14

Table Summary

This table summarizes the number of presentations and volunteer hours CASARA members spent on awareness and outreach activities for FYs 2009/10 to 2013/14.

 

FYNumber of PresentationsVolunteer Hours
2009/10 21 244.3
2010/11 10 70.5
2011/12 16 289.4
2012/13 11 95.9
2013/14 10 63.4

Other examples of CASARA’s promotion and awareness activities include maintaining partnerships and cooperation with other SAR providers at the regional and chapter level. In some cases, MOUs or agreements with provinces or territories are in place for other SAR activities. In addition, CASARA engaged in the annual SARscene Conference by sending members each year. CASARA also helps to promote the use of the 406 beacon and raises awareness about the discontinuance of the previous 121.5 beacon. 

Partnership and collaboration with the NSS over the evaluation period has been limited and most activities are local or regionally based without connection to a broader national CASARA strategy.

Recommendation

3.       Unless performance measures can demonstrate the impact of prevention and safety awareness activities, consider redirecting these financial resources to training expenditures.

OPI: RCAF/CASARA

2.5 Performance—Efficiency and Economy

Evaluation question: Is the Program using appropriate and efficient means to achieve its functions? How cost effective is CASARA to DND/CAF and the National SAR Program? The efficiency and economy of CASARA was assessed by the following three indicators:

  • cost of inputs versus outputs and a detailed analysis of annual expenses;
  • efficiencies found through lessons learned and best practices; and
  • cost savings analysis of CASARA’s contribution to the DND/CAF for SAR responses.

2.5.1 Costing Breakdown

Table 7 shows the details of CASARA’s expenditures from 2010/11 to 2013/14. As per the table, CASARA spends over 50 percent of its annual budget on training activities. The second and third highest expenditure areas were in Equipment Repair and Replacement, and Professional Fees. These areas largely consist of expenditures for charter aircraft and represent approximately 20 percent of the Program budget. As discussed previously, efficiencies are down overall as external costs such as fuel and other expenses have increased while actual tasking and training for CASARA has reduced.

Key Finding 9: To address increased fuel costs, CASARA has several initiatives to drive efficiencies in training.

CASARA has implemented some efficiencies and lessons learned in recent years. For example, efficiencies were found through the use of simulators for training, and by requiring all flight training exercises to include other CASARA team members, such as spotters and navigators. These measures maximize training and flying time and improve the training experience levels for all. In northern or remote regions, where assets are limited or unavailable to members, the rental and charter of aircraft has produced cost savings. It costs more to own and maintain aircraft for training and mission support in these regions.

Another way CASARA has helped to reduce the number of training hours and exercises has been through using flight simulators and developing and using a new SAR simulator program, called the CASARA Virtual Trainers. This program was developed in partnership with Carleton University and the SAR New Initiatives Fund (NIF).19 This new SAR simulator has only recently been developed and distributed to select CASARA member organizations.

A principal concern is the cost of administration and travel related to the organizational structure. As per Table 7, almost 30 percent of the budget each year is dedicated to those activities. This in part may be due to what appears to be too many layers of administration. CASARA has 13 membership organizations, or chapters, across Canada representing each of the provinces and territories. Each member organization has an elected/appointed executive consisting of provincial/territorial directors, deputy directors, secretaries and treasurers. In addition there is a National Board of Directors with one director elected/appointed by each member organization. The CRS evaluation team questions if it is necessary to have so many governing bodies, particularly when the National SAR Program is divided into only three geographic regions.

Table 7. Annual Expenditures 2010–2014

Table Summary

Table 7 is a financial table of CASARA’s annual expenditures in dollars for FYs 2010/11 to 2013/14. 

 

Cost Centre2010/112011/122012/132013/14Total%
Flying Training 1,099,862 1,148,174 1,105,955 1,166,150 4,520,141 48.50
All Other Training 131,368 108,651 130,423 105,150 475,592 5.10
Equipment Repair and Replacement 216,796 237,009 175,695 367,466 996,965 10.70
Professional Fees/Services 176,245 223,511 304,797 210,973 915,525 9.80
Meetings—Zone 11,426 20,591 10,284 17,316 59,617 0.60
Search Master Course 4,685 7,845 4,827 11,213 28,570 0.30
Training Officers’ Meetings 30,295 25,808 25,019 23,085 104,206 1.10
Unscheduled Committee Meetings 22,066 3,505 0 16,458 42,029

0.50

Treasurer’s Meetings 0 0 0 7,130 7,130 0.10
Unscheduled Travel 26,783 36,314 15,946 6,417 85,461 0.90
Meetings—Provincial 157,502 148,878 162,578 128,458 597,416 6.40
AGM/Fall Directors’ Meetings 70,575 67,647 69,645 72,768 280,634 3.00
National Executive Meetings 4,260 13,413 21,864 21,471 61,009 0.70
Past President Association 0 0 0 0 0 0.00
Communications 117,995 114,191 113,174 113,694 459,052 4.90
Member 93,673 100,643 94,609 96,628 385,553 4.10
Public Relations 37,242 27,838 30,304 21,235 116,618 1.30
National Office Expenses 40,562 50,728 47,102 44,723 183,115 2.00
Grand Total 2,241,332 2,334,745 2,312,221 2,430,334 9,318,632 100

 

Key Finding 10: The administration/governance of the Program has multiple layers and represents almost 30 percent of the Program costs.

Recommendation

4.      CASARA should streamline its management and governance structure to reduce administrative and meeting costs.

OPI: CASARA

Analysis of the financial information shows spending on Public Relations represents only 1.3 percent of total spending. This figure is considered low given CASARA’s prevention and safety awareness component and its need to attract and recruit new members. Furthermore, when compared with the financial information from the previous CRS evaluation of CASARA (2009), Public Relations spending has remained the same despite falling recruitment numbers and consistent prevention and awareness requirements.

Recommendation

5.      Increase efforts on outreach beyond 1.3 percent of spending if recruitment targets that need to be established are not met.

OPI: CASARA

2.5.2 Economy/Value for Money

Key Finding 11: Should the amount of hours CASARA is tasked by DND/CAF continue to decline, the ongoing value of the Program may be reduced. As it currently exists, the real value of the Program is its ability to augment and enhance DND/CAF’s SAR search capabilities and response time.

The annual expenditures for the CASARA Program are $3.1 million, which, over the evaluation period provided an average of 280 SAR mission hours. However, throughout the evaluation period the total mission hours used declined steadily. For example, in 2008, the Program delivered 551 mission hours, while in 2012 only 116 hours were utilised.

Table 8 compares the cost of the RCAF SAR platforms and the hours they delivered to the cost of the Program. The RCAF Cost Factors Manual was used to analyze the cost of personnel, maintenance and fuel per hour of operation for each aircraft type. As shown in the table, the cost of an RCAF SAR response is, on average for 2012, $29,144 per hour flown. Based upon the hours for CASARA for the same year, the average CASARA cost per hour is very similar, at $26,724.

Table 8. Hourly Cost of SAR Aircraft in 2012

Table Summary

This table shows the hourly cost (in dollars) of SAR aircraft as reported in the Cost Factors Manual (i.e., estimates) and the cost per hour flown (i.e., Force Employment) by type of RCAF aircraft used at each SAR squadron.

 

SquadronAircraftFE SAR HoursCFM (Full Costs)Cost per Hour
Comox (442) CC115 Buffalo 500 12,393,000 24,786
CCH149 Cormorant 656 21,273,424 32,429
Greenwood (413) CC130 Hercules 587 18,322,618 31,214
CH149 Cormorant 419 13,587,751 32,429
Gander (103 RU) CH149 Cormorant 519 16,830,651 32,429
Trenton (424) CC130 Hercules 615 19,196,610 31,214
CH146 Griffon 406 4,897,984 12,064
Winnipeg (435) CC130 Hercules 672 20,975,808 31,214
RCAF SAR Response 4,374 127,477,846 29,144
National CASARA Search 2012 116 3,100,000 26,724
CASARA Search 2008 551 3,100,000 5,626

The RCAF platforms present far greater capabilities than the CASARA aircraft and only cost slightly more. For example, when comparing CASARA’s 2008 total hours flown, the cost per hour is 20 percent less than using an RCAF aircraft. Essentially since the cost of the CASARA Program is fixed, the less the Program is actually called upon, the greater the cost.

Should the amount of hours CASARA uses continue to decline, the ongoing relevance and value of the Program may be reduced. As it currently exists, the real value of CASARA is its potential to augment the CAF’s SAR response especially in remote and northern regions. Should a major search operation occur in which beacons are not used, having over 400 CASARA aircraft to support DND/CAF’s SAR activities and the SAR system as a whole is a significant asset.

Recommendation

6.       Despite the decline in use, overall the Contribution to CASARA remains good value for the department and the Program should be renewed.

OPI: RCAF

Annex A—Management Action Plan

Recommendation

1.         DND/CAF should develop targets for the membership levels and distribution of CASARA resources relative to the tasking level and hours required for flying (for training and missions). DND/CAF should also respond if the level falls below a minimum acceptable level, which needs to be determined.

Management Action

Despite decreased membership in southern Canada, CASARA continues to meet its mandate for SAR readiness. Recent developments in CASARA North have seen net increases in membership and future CASARA North growth is planned for 2015.RCAF will work with JRCCs and CASARA to identify potential gaps and weaknesses in readiness caused by membership challenges. Nationally, optimal membership levels will be reviewed.

OPI: RCAF

Target Completion Date: Spring 2015

Recommendation

2.         Increase the use of alternative training methods, such as using the SAR simulator, across all membership organizations in CASARA.

Management Action

The SAR NIF sponsored CASARA Virtual Trainer has been deployed across all southern member organization units. Distribution to existing and future CASARA North locations is already in progress. CASARA is investigating new learning software and management programs to enhance and replace its existing CASARA Management System. This new system will incorporate alternative training functions for spotters, navigators and pilots with the intent to reduce aircraft flight time.RCAF will monitor the progress of the CASARA Management System replacement. CASARA will pursue 2015 SAR NIF program for funding for new training tools.

OPI: RCAF/CASARA                     

Target Completion Date: Summer 2015

Recommendation

3.         Unless performance measures can demonstrate the impact of prevention and safety awareness activities, consider redirecting these financial resources to training expenditures.

Management Action

CASARA is a response organization, however, when conducting operations and participating at SARscene and various SAR exercises, it actively engages in SAR awareness and outreach activities. The responsibility for prevention and safety needs further definition through the Interdepartmental Committee on Search and Rescue.

CASARA will continue to engage in aeronautical SAR awareness and outreach activities. Re-allocation of funds to Arctic sustainment/training and technology initiatives will be considered.

OPI: CASARA                                 

Target Completion Date: Summer 2015

Recommendation

4.         CASARA should streamline its management and governance structure to reduce administrative and meeting costs.

Management Action

Administration and meeting expenses are consistent with similar not-for-profit organizations. However, potential cost reductions exist by reducing meeting frequency and attendance requirements, and employing Internet-based tools, which are seen as the preferred options for future cost savings.

CASARA will investigate reducing administrative and meeting costs through several options. Additionally, CASARA will explore leveraging prominent SAR events such as SARscene to conduct multiple SAR functions/training sessions/theme meetings and programs. Governance will be reviewed by CASARA National, however, broad changes may significantly impact existing and proposed MOUs with provinces and territories.

OPI: CASARA

Target Completion Date: Summer 2015

Recommendation

5.         Increase efforts on outreach beyond 1.3 percent of spending if recruitment targets that need to be established are not met.

Management Action

Recruitment targets for CASARA North are proceeding well. FY 2013/2014 has seen the standup of four new units in the communities of Pond Inlet, Arctic Bay, Hall Beach and Gjoa Haven. This has resulted in an increase in membership of over 50 personnel.

CASARA will increase recruitment in the Arctic with two new locations in 2015—an addition of 25 to 30 new members. Recruitment targets in the south will be evaluated and monitored by RCAF/CASARA.

OPI: CASARA                                 

Target Completion Date: Update 2015

Recommendation

6.         Despite the decline in use, overall the Contribution to CASARA remains good value for the department and the Program should be renewed.

Management Action

RCAF to OPI agrees with the recommended renewal of the CASARA Contribution Agreement for five years (April 1, 2015 to March 31, 2020).

OPI: RCAF

Target Completion Date: Winter/Spring 2015

Annex B—Logic Model

Figure B-1. CASARA Logic Model (2010). This flowchart depicts the link between CASARA inputs, activities and outputs, and the immediate, intermediate, and ultimate outcomes.

Text description for Figure B-1:

This flowchart depicts the link between CASARA inputs, activities and outputs, and the immediate, intermediate and ultimate outcomes. Beginning with a starting point called inputs, the flowchart divides into five groupings.

The CASARA logic model shows five main activities that lead to outputs, immediate and intermediate outcomes, and to the ultimate outcomes of: (1) An aeronautical SAR response that is as effective as practicable, and (2) A general aviation community that understands how the SAR system serves their needs and is proactive in increasing their chances of survival.

This logic model is a series of inputs and activities that lead to four main outputs. These outputs result in two immediate outcomes, two intermediate outcomes, and two ultimate outcomes. The logic model components are broken down as follows:

The inputs are People, Funds, SAR Manual, and SAR Network (i.e., GDMSS, partnerships, other enablers). These inputs lead to five activities. These are:

  • SAR training
  • Program administration
  • Insurance premiums
  • Promoting SAR awareness
  • Special projects

These activities flow into two streams or starting points as follows:

Starting Point A

Outputs:

  • Trained and qualified crews
  • Trained and qualified crews with access to aircraft and specialized equipment to support SAR activities

Immediate outcome:

  • A national distribution of search capable resources that augment and compliment the efforts of CAF SAR response

Intermediate outcome:

  • A reduction in the time and effort required to resolve CAF SAR-coordinated SAR incidents

Ultimate outcome:

  • An aeronautical SAR response that is as effective as practicable

Starting Point B

Outputs:

  • Briefings and outreach events for the general aviation community
  • Meetings, program activities, outreach, new units (North)

Immediate Outcome:

  • A nationally distributed volunteer pool able to deliver SAR Awareness briefings to the general aviation community

Intermediate Outcome:

  • Increased public awareness of CASARA and the federal SAR response mandate

Ultimate Outcome:

  • A general aviation community that understands how the SAR system serves their needs and is proactive in increasing their chances of survival

___________________________________________________________________________________________________________________________

Footnote 1 Unless otherwise indicated, references to “the Program” are to the CASARA Contribution Program.

Footnote 2 CASARA RMAF (2010) page 10.

Footnote 3 CASARA RMAF (2010) page 8.

Footnote 4 CASARA also has Memorandum of Understanding (MOU) and agreements with some provinces and territories to provide support to ground SAR activities. These activities are outside of the scope of this evaluation.

Footnote 5 Departmental Performance Report (DPR) 2011/12, page 24 and Posture Review, page 2, Canadian Joint Operations Command 2013.

Footnote 6 Quadrennial SAR Review (December 2013), ADM(PA) DGM–24913–965, page 4.

Footnote 7 Canadian Forces Defence Strategy (2008), page 8.

Footnote 8 DND RPP 2013/14, page 51.

Footnote 9 DND DPR 2012/13, page 84.

Footnote 10 SAR is mentioned as a key effort for Excellence in Operations by the CDS, in CDS Guidance to the CAF, June 2013).

Footnote 11 Namely, “to provide security to Canadians, Defence will continue to provide prescribed levels of SAR response capability in collaboration with SAR mission partners within the Canadian SAR region” (RPP 2013/14).

Footnote 12 PAA 2013/14, page 3.

Footnote 13 Canada First Defence Strategy, page 7.

Footnote 14 Air Force Vector (2014); see: http://airforce.mil.ca/canairgens/2014/201916ZFEB14_e.htm. Last retrieved December 4, 2014.

Footnote 15 MOU, April 1, 2010 to March 31, 2015.

Footnote 16 CRS CASARA Evaluation 2009.

Footnote 17 Reference annual sales data from Transport Canada/Statistics Canada on ownership/registration of private aircraft in Canada 2008–2013.

Footnote 18 CASARA annual reports.

Footnote 19 http://carleton.ca/ace/projects-and-publications/casara-simulation-training/.

Date modified: