6 Audit of the Transportation Safety Board of Canada

Table of Contents

6.1 This audit covers the Transportation Safety Board of Canada (TSB)’s appointment activities for the period between August 31, 2010, and September 1, 2011. The objectives of the audit were to determine whether the TSB had an appropriate framework, practices and systems in place to manage its appointment activities and whether appointments and appointment processes complied with the Public Service Employment Act (PSEA), the Public Service Employment Regulations (PSER), the Public Service Commission (PSC) Appointment Framework and related organizational appointment policies.

6.2 The TSB is an independent agency whose objective is to advance transportation safety. It fulfills this mandate by conducting independent investigations into selected transportation occurrences to identify the causes and contributing factors of the occurrences as well as the underlying safety deficiencies. The TSB then makes recommendations to improve safety and reduce or eliminate risks to people, property and the environment.

6.3 The jurisdiction of the TSB includes all aviation, marine, rail and pipeline transportation occurrences, in or over Canada, that fall under federal jurisdiction. The TSB may also represent Canadian interests in foreign investigations of transportation accidents involving Canadian registered, licensed or manufactured aircraft, ships or railway rolling stock. In addition, the TSB carries out some of Canada’s obligations related to transportation safety at the International Civil Aviation Organization and the International Maritime Organization.

6.4 The TSB consists of up to five board members, including a chairperson (who is the deputy head of the TSB). The TSB business plan stated that, as of March 31, 2011, it had 222 employees. Of these, 214 (96.4%) were indeterminate. The TSB operates offices in seven provinces; its headquarters is located in Gatineau, Quebec.

6.5 The organization carried out 33 appointments within the scope of this audit. As part of our audit, we conducted interviews, analyzed relevant documentation and audited a representative sample of 24 appointments. This sample of 24 appointments provides an estimate of the population accurate within +/- 10%, 90% of the time, given a deviation rate of 20% or less. This single sample is sufficient given that the objective was to provide an overall estimate of compliance.

Observations on the Appointment Framework

The Public Service Employment Act and the Public Service Commission’s delegated authorities

A sub-delegation process was in place

6.6 The PSEA provides the PSC with the authority to appoint employees in the public service. The PSC delegates its appointment authorities to deputy heads, who in turn may sub-delegate the exercise of these authorities. The PSC expects deputy heads to have a sub-delegation instrument in place that is well managed and communicated across the organization.

6.7 We found that the deputy head of the TSB signed an agreement with the PSC to accept the delegation of staffing authority. The deputy head established a sub-delegation instrument to sub-delegate this authority to specific employees on the deputy head’s behalf. The instrument identified the requirements that employees had to meet in order to be sub-delegated, including mandatory training. The agreement with the PSC and the sub-delegation instrument were posted on the TSB intranet site and were accessible to all employees and bargaining unit members.

6.8 The TSB’s sub-delegation instrument, the Staffing Sub-delegation Framework, was consistent with PSC requirements. This framework had four elements: the TSB sub-delegation of staffing authority policy; the staffing sub-delegation authority chart; the sub-delegation staffing process chart; and the staffing sub-delegation monitoring directive. Sub-delegated managers, and ultimately the deputy head, were accountable for appointments made by the TSB.

6.9 The TSB had mandatory training to provide sub-delegated officials with the knowledge needed to carry out their appointment-related responsibilities. A variety of training tools and communication mechanisms were also available on TSB’s intranet site to provide information to sub-delegated officials on staffing policies and appointment-related decisions.

Appointment policies

Appointment policies were in place

6.10 The PSC expects deputy heads to establish mandatory appointment policies for area of selection, corrective action and revocation, as well as criteria for the use of non-advertised processes. The PSC also expects other appointment policies that organizations develop to be compliant with the PSEA, the PSER and the PSC Appointment Framework.

6.11 We found that the deputy head of the TSB established the mandatory appointment policies and criteria for the use of non-advertised appointment processes. However, we noted that their Policy on Area of Selection and Policy on a Non-advertised Appointment Process were missing the guiding value of representativeness. During this audit, the mandatory policies were revised to include this value.

6.12 We found that stakeholders were consulted in the development and revision of the three mandatory policies. The stakeholders included sub-delegated officials, human resources (HR) advisors, bargaining agents and Executive Committee members. During the scope of the audit, the TSB’s organizational policies were communicated and made accessible through the organization’s intranet to all sub-delegated officials, other employees and bargaining agents.

Planning for staffing

Strategies were in place and actively monitored

6.13 Organizational staffing strategies describe planned organizational staffing priorities and how and when they will be achieved. The PSC expects deputy heads to establish staffing strategies to address the priorities of senior management. Organizational staffing strategies and priorities must be communicated, monitored and adjusted, when required.

6.14 We found that the deputy head of the TSB had two business plans covering the period of the audit and outlining strategies to be used to achieve its staffing priorities, such as knowledge transfer, capacity building and training program review. The approved TSB Business Plans were posted on the organization’s intranet. We also found that an e-mail was sent to all staff, inviting them to familiarize themselves with the plan and with upcoming projects, activities and staffing opportunities.

6.15 The deputy head of the TSB monitored the results of its staffing strategies by utilising semi-annual staffing actions and gap reports. Analysis of the variance between planned and actual staffing activities was performed and reported to the Executive Committee. Adjustments to the staffing plan were made throughout the year, as required, and on an annual basis, as part of the business planning cycle.

Capacity to deliver

Roles were clearly defined

6.16 The PSC expects deputy heads to ensure that those who have been assigned a role in appointment processes have been informed of their responsibilities and have the support to carry out this role.

6.17 The deputy head of the TSB clearly defined the roles, responsibilities and accountabilities of sub-delegated officials, the head of HR and HR advisors in relation to appointment and appointment-related authorities. These roles, responsibilities and accountabilities were detailed in the sub-delegation instrument that was communicated and made accessible to all employees.

Monitoring

Mandatory monitoring was conducted, but quality of documentation on file was weak

6.18 Organizational monitoring is an ongoing process that allows deputy heads to assess staffing management and performance related to appointments and appointment processes. Monitoring makes it possible to identify the need for early corrective action, manage and minimize risk and improve staffing performance. The PSC expects deputy heads to undertake the mandatory monitoring outlined in the PSC Appointment Framework and adjust practices accordingly.

6.19 We found that, during the scope of the audit, the deputy head of the TSB established a monitoring reporting and review mechanism to meet mandatory PSC requirements. Year-to-date results of staffing were presented to the Executive Committee at mid-year.

6.20 The deputy head of the TSB assigned the responsibility to HR advisors for the ongoing monitoring and maintenance of staffing files. In 2010, the TSB conducted its own monitoring exercise on appointment files from 2008 and 2009. The TSB presented its findings and action plan to the Executive Committee only in 2011. The monitoring exercise identified inadequate documentation as a main area of concern. An action plan was developed. We found that most of the action items in the plan were identified as being completed, including the establishment of a file review checklist.

6.21 During our audit of the appointment files, we observed that the new checklist was being used as stated in the action plan. However, most of the appointment files continued to lack important information, despite the completed checklist on file indicating that the information was present. We found that letters of offer, applicants’ résumés, second language assessment results, assessment board reports, proof of education and others were missing. Most of the missing documents were eventually found at the request of the auditors; however, the use of the checklist as a monitoring mechanism was not achieving the results intended in the action plan.

6.22 We found that the deputy head of the TSB did not have a mechanism to control the quality of the documentation on file. We found that 67% (16 out of 24) of the appointments we reviewed had various deficiencies, such as the following:

  • The statement of merit criteria was either not identical in both official languages or showed differences between the advertisements and the notifications posted;
  • The appointment was made from a pool, when the original advertisement did not indicate that a pool would be created; and
  • The justification of the reasons for the appointment decision (right fit) was not compliant with the PSC Appointment Framework, or there was no documentation of a right-fit decision on file.

6.23 Active monitoring was conducted by the TSB; however, the measures taken to address the issues found did not result in the required key documentation being placed in the appointment files. These documents, either missing or incomplete, have resulted in some processes where the guiding values of the PSEA have not been demonstrated. Refer to recommendation 1 at the end of this report.

Observations on compliance

Poor assessment tools were the main cause of merit not demonstrated or not met

6.24 The PSEA establishes that all appointments must be made on the basis of merit. Merit is met when the Commission is satisfied that the person to be appointed meets the essential qualifications for the work to be performed, as established by the deputy head, and, if applicable, any other asset qualifications, operational requirements and organizational needs established by the deputy head.

6.25 We found that merit was met in 67% (16 out of 24) of the appointments in our sample. However, we found that 25% (6 out of 24) of appointments did not demonstrate merit. The main cause of merit not being demonstrated was weaknesses in the assessment tools and inaccurate documentation to show how merit was met. We were unable to conclude whether merit was demonstrated in these appointments because there was no clear link between the assessment conducted and every qualification, or there was no evidence that each qualification was assessed.

6.26 We also found that, in 8% (2 out of 24) of appointments audited, merit was not met. Merit was not met because the person appointed did not meet one or more of the essential qualifications for the work to be performed. Table 1 provides a summary of our observations concerning merit for the appointments audited. Table 2 provides a further breakdown of the reasons for which merit was not demonstrated.

Guiding values were also affected by poor assessment tools

6.27 We found that another 25% (6 out of 24) of appointments audited had weaknesses in the demonstration of the guiding values. We found some examples where applicants were not screened consistently, or the rating tools were not consistently applied throughout the screening process. In one case, we were not able to conclude whether the assessment tools were consistently applied to all applicants, as an assessment board report was not available.

6.28 The weaknesses found in the development and use of the assessment tools resulted in merit not being demonstrated or not met in one third of the TSB’s appointments. The deputy head placed the responsibility of the selection and development of the assessment tools on the sub-delegated officials, with advice from the HR advisors. Weaknesses in the assessment tools may be attributable to the infrequency of sub-delegated officials using their authority, and sub-delegated officials not seeking the advice of HR in the development and use of the tools. Refer to recommendation 2 at the end of this report.

Priority requests had errors in 50% of the appointments

6.29 The PSEA and the PSER provide an entitlement, for a limited period, for certain persons who meet specific conditions to be appointed in priority to others. The organization must take into consideration persons with priority entitlements, and must also obtain a priority clearance from the PSC before making an appointment. One of the roles of the HR advisors identified in the TSB’s sub-delegation staffing process chart was the preparation of the priority clearance requests.

6.30 We found that, in 38% (9 out of 24) of appointments we reviewed, one or more criteria such as essential qualifications, tenure, position number, group and level, linguistic profile or conditions of employment used to obtain priority clearance were not the same as those used to make the appointment decision. In 13% (3 out of 24) of the instances, priority clearance was not requested from the PSC.

6.31 Since in total 50% (12 out of 24) of the appointments reviewed had errors in the priority clearance request, or the request was not made, this could have resulted in priority persons not being referred for further consideration for a vacant position. Refer to recommendation 3 at the end of this report.

Recommendations

  1. The deputy head of the Transportation Safety Board of Canada should improve its monitoring mechanism at the appointment level to ensure that documentation is complete, accurate and compliant with the Public Service Employment Act, the Public Service Employment Regulations, the PSC Appointment Framework and other governing authorities. The findings of this monitoring should be reported to senior managers for appropriate corrective measures in a timely manner.
  2. The deputy head of the Transportation Safety Board of Canada should ensure that sub-delegated managers have the knowledge and support from human resources to develop assessment tools that demonstrate merit, and that the guiding values are considered in managerial appointment decisions.
  3. The deputy head of the Transportation Safety Board of Canada should ensure that human resources advisors complete priority clearance requests for each transaction, and that the request contains accurate and complete information.

Conclusion

6.32 The first objective of the audit was to determine whether the TSB had an appropriate framework, systems and practices in place to manage its appointment activities. We found that the deputy head of the TSB had established appropriate policies and staffing strategies. The strategies were monitored and adjusted, as required. We found weaknesses in the TSB’s monitoring practices over appointments to ensure that adequate documentation was on file, assessment tools demonstrated merit and priority clearances were appropriately performed.

6.33 The second objective was to determine whether appointments and appointment processes complied with the PSEA, the PSER, the PSC Appointment Framework and related organizational policies. We found that most appointments and appointment processes complied with the PSEA, the PSER and the PSC Appointment Framework. Although the majority of appointments we reviewed met merit, there were some appointments in which merit was not met or not demonstrated. We found that the majority of the appointments we reviewed had issues with demonstrating the guiding values. We noted that administrative errors, deficiencies in assessment tools and inadequate consideration of priorities had an impact on the demonstration of the guiding values.

Action taken by the Public Service Commission

The PSC will monitor the TSB’s follow-up action to the audit recommendations through its regular monitoring activities, including the annual Departmental Staffing Accountability Report; as a result, the PSC has decided not to amend the existing delegation agreement with the deputy head of the TSB.

Overall response by the Transportation Safety Board of Canada

The TSB agrees with the findings and recommendations of this report and has developed a comprehensive Management Action Plan that addresses recommendations outlined in this report. Some measures have already been implemented and the remaining will be implemented by September 2012.

The TSB has already made improvements to its monitoring processes to improve the quality of documentation on staffing files. Any issues identified through monitoring will be raised to the attention of senior management and corrective actions will be taken in a timely manner.

Revised training and forms have been developed for use by sub-delegated managers to ensure merit and the guiding values are more clearly demonstrated in managerial appointment decisions.

The TSB had already identified administrative issues with priority clearances prior to the start of the audit. Corrective actions have since been taken and an additional level of review was implemented to ensure the integrity and effectiveness of the priority clearance process.

Appendix

Table 1: Observations on merit
Observations Total appointments
Merit was met Assessment tools or methods evaluated the essential qualifications and other merit criteria identified for the appointment; the person appointed met these requirements. 16 (67%)
Merit was not met The person appointed failed to meet one or more of the essential qualifications or other applicable merit criteria identified. 2 (8%)
Merit was not demonstrated Assessment tools or methods did not demonstrate that the person appointed met the identified requirements. 6 (25%)
Total appointments audited 24 (100%)

Source: Audit and Data Services Branch, Public Service Commission

Table 2: Observations on merit not demonstrated
Merit was not demonstrated Reasons for merit not demonstrated
No assessment performed Assessment tool did not evaluate all of the appointment criteria Assessment was not applied as per tool Organization was unable to provide documentation that supports merit
Totals 0 5 1 0

Source: Audit and Data Services Branch, Public Service Commission

Table of Contents