Public Service Commission

www.psc-cfp.gc.ca

Common menu bar

• Français
• Home
• Contact Us
• Help
• Search
• canada.gc.ca

Breadcrumb

1. Home >
2. Audit and Data Services >
3. Completed Reports – 2009

Study on the data collection of non-advertised appointment processes

A study by the Public Service Commission of Canada

October 2009

Public Service Commission of Canada
300 Laurier Avenue West
Ottawa, Ontario K1A 0M7
Canada

Information: 613-992-9562
Facsimile: 613-992-9352

© Her Majesty the Queen in Right of Canada, represented by the Public Service Commission of Canada, 2009

Table of Contents

• Summary
• Introduction
  • Background
  • Issue of Study
• Focus of the study
  • Approach
  • Context
• Observations
  • Overview
  • Relevance: The information gathered should meet the needs of the clients
  • Accuracy: The information gathered should correctly describe appointment information
  • Timeliness: The information gathered should be available to clients in a reasonable amount of time
  • Accessibility: The information gathered should be easily accessed
  • Interpretability: The meaning of the leading indicators should be easily understood
  • Coherence: The information gathered by different organizations should be comparable
• Concluding remarks
• Study team
• Appendices
  • Appendix 1: About the study
  • Appendix 2: Reporting methodology
  • Appendix 3: Leading indicators of long-term success
  • Appendix 4: Glossary

Summary

The 2005 Public Service Employment Act (PSEA), as well as the related appointment policies, provides deputy heads with the necessary flexibility to meet the needs of their organization, notably the choice of appointment process. The core values of merit and non-partisanship, as well as the guiding values of fairness, transparency, access and representativeness, will influence the choice of appointment process.

Non-advertised appointment processes have been assessed by the Public Service Commission (PSC) as a risk area which needs to be more closely monitored by deputy heads to ensure respect of the core and guiding values. Further, the PSC has expressed a need to collect more rigorous and detailed information about non-advertised appointment processes.

The ratio of non-advertised appointment processes to total appointment processes is used by the PSC as an indicator of organizational effectiveness and adherence to the guiding value of access. However, with incomplete information about the circumstances under which non-advertised appointment processes occurred, the ability to assess the risks associated with such appointment processes is limited.

The purpose of the study was to describe data collection practices related to non-advertised appointment processes and the limits of the data available, as well as to explore associated risks and areas for improvement for the PSC and organizations.

The study focused largely on the non-advertised appointment activities of four organizations: the Canadian International Development Agency, Citizenship and Immigration Canada, Library and Archives Canada and Transport Canada. It included approximately 50 files from each of the selected organizations, consisting of 10 advertised appointment processes and 40 non-advertised appointment processes for the period of April 1, 2007, to March 31, 2008.

While all four of the organizations examined had a formal human resources (HR) system, three of the organizations did not rely solely on their HR system to provide information to the PSC. The Canadian International Development Agency, for example, maintains a separate Excel-based database to help manage the hiring process. In the case of Citizenship and Immigration Canada, data entry into PeopleSoft is highly decentralized, requiring the organization to cross-reference data in PeopleSoft with data from an Excel-based system. At Library and Archives Canada, the current version of the Human Resources Information System (HRIS) does not allow for customized queries; therefore, staff rely on the process number or on a manual review of the appointment file to determine the type of appointment process. Transport Canada is the only organization that uses its HR system to manage the hiring process as well as position information.

While the calculation of the leading indicators related to access is easy to understand and is well documented, organizations were unanimous in their concern that the ratios of non-advertised appointments (ACC-1 and ACC-3) do not provide a meaningful measure of the value of access to public service positions. (Refer to Appendix 3 for a complete list of the leading indicators related to effectiveness and adherence to the staffing values.)

The study found that accuracy varied among organizations and that issues of both over-coverage and under-coverage exist. Errors in coding were also found. The amount of time and effort required for submitting data to the PSC and the capacity to respond to changing requests vary greatly among organizations.

There is a limited level of coherence of information reported to the PSC for extensions to acting appointments. One organization treats such extensions to acting appointments as subsequent non-advertised appointments, while the remaining organizations treat such extensions as administrative measures. However, for the purpose of the quarterly reports to the PSC, two organizations consider these as appointments.

The PSC is committed to clarifying the policy interpretation surrounding extensions to acting appointments. Further, the PSC will explore other options and methods of data collection from organizations.

Introduction

Background

The 2005 Public Service Employment Act (PSEA), as well as the related appointment policies, provide deputy heads with the necessary flexibility to meet the needs of their organization, notably the choice of appointment process. A process can be internal or external advertised, or internal or external non-advertised. The core values of merit and non-partisanship, as well as the guiding values of fairness, transparency, access and representativeness, will influence the choice of appointment process.

As per the Public Service Commission (PSC) Policy on Choice of Appointment Process and related documents, due consideration should be given not only to the core and guiding values and to human resources (HR) planning but also to the position requirements, operational realities, workforce characteristics and other considerations related to short-term situations. However, the PSC considers that the core and guiding values should lead to advertised processes more often than to non-advertised processes.

As part of their accountability, deputy heads are required to establish organizational criteria to identify the circumstances under which the use of non-advertised processes is appropriate for their organization. As per the Public Service Commission 2007-2008 Annual Report, non-advertised appointment processes may be appropriate under certain conditions and would likely be selected by the manager for appointments following a reclassification or promotions within developmental programs.

Issue of study

Over the years, non-advertised appointment processes have been assessed by the PSC as a risk area which needs to be more closely monitored by deputy heads to ensure respect of the core and guiding values. Therefore, the PSC has expressed a need to collect more rigorous and detailed information about non-advertised appointment processes. There is also a requirement for continuous monitoring to ensure a reliable information base for the assessment of organizational performance and respect of the core and guiding values.

As noted in its annual report for 2007-2008, the PSC estimates that 43% of all appointments, internal and external, made under the PSEA in 2007-2008 were done through non-advertised processes. This figure is at 28% for external appointments. It is worth noting that for acting appointments only, 70% were made through non-advertised processes.

The PSC’s 2006 Report on Preliminary Data Collection Activities Related to Advertised versus Non-advertised Appointments identified problems with the data quality of appointment information reported by organizations.

There are various means by which organizations can provide the information requested by the PSC. They have the option of supplying the information through a download from their HR system, or manually reporting the information using a PSC Excel template or other format.

As indicated in the report by the Council of Systems Cluster Groups entitled A Retrospective and Current Analysis of the Corporate Administrative Systems Clusters in the Government of Canada 2007-2008, a variety of administrative systems is available to public service organizations for the management of HR information. The two most common systems include the Government of Canada Human Resources Management System (PeopleSoft) and the Human Resources Information System (HRIS); however, there is an array of other HR systems, including,Transport Integrated Personnel System (TIPS) and Systems Applications Products (SAP).

Currently, organizations must submit data to the PSC quarterly, or bi-annually for small organizations, on the use of advertised and non-advertised appointment processes, indicating as well whether the process was internal or external. With this information, the PSC calculates the ratio of external non-advertised appointment processes to total appointment processes, which is referred to as the Access 1 leading indicator (ACC-1); and the ratio of internal non-advertised appointment processes to the total appointment processes, which is referred to as the ACC-3 leading indicator. (Refer to Appendix 3 for a complete list of the leading indicators related to effectiveness and adherence to the staffing values.) The PSC uses the ratio of non-advertised appointment processes to total appointment processes as an indicator of organizational effectiveness and adherence to the guiding value of access. However, with incomplete information about the circumstances under which non-advertised appointment processes occurred, the ability to assess the risks associated with such appointment processes is limited.

Focus of the study

Approach

The purpose of the study was to describe data collection practices related to non-advertised appointment processes and the limitations of the data available, as well as to explore associated risks and areas for improvement for the Public Service Commission (PSC) and organizations.

This study was conducted using Statistics Canada’s Quality Assurance Framework, which defines the six dimensions of data quality: relevance, accuracy, timeliness, accessibility, interpretability and coherence. A full definition of each of these dimensions can be found in Appendix 2.

The study focused largely on the non-advertised appointment activities of four organizations: the Canadian International Development Agency, Citizenship and Immigration Canada, Library and Archives Canada and Transport Canada. The organizations selected are subject to the Public Service Employment Act.

Source: Audit and Data Services Branch, Public Service Commission of Canada

The study activities included interviews with key stakeholders from the four organizations in order to obtain an understanding of their data collection process and related risks. Interviews were also conducted with PSC stakeholders to identify the organizations’ reporting requirements and key reporting issues. The study also included an analysis of appointment and system documentation as well as other information, including organizational monitoring reports. Finally, the study included observation of the data collection process in the organizational human resources (HR) systems and the file review of a random sample of appointment processes.

The study included approximately 50 files from each of the selected organizations, consisting of 10 advertised appointment processes and 40 non-advertised appointment processes for the period of April 1, 2007, to March 31, 2008. This sample allowed for an assessment of general data quality for each organization for non-advertised processes; the combined advertised processes from each organization allowed for an estimation of general data quality across all four organizations.

For more information regarding the study objectives and methodology, see Appendix 1.

Context

There are various reasons for not soliciting applications for a process and for proceeding with a non-advertised appointment process. For example, for developmental programs, potential participants must meet the conditions of participation and the required qualifications, as defined in the organizational entry requirements. The subsequent non-advertised promotions are based on the participant meeting the requirements of his/her current level and thereby qualifying for the next level. Promotion to the next level takes place after an assessment has been made against the relevant competency profile and after other program requirements have been met. As demonstrated, the impact on the core and guiding values will depend on the circumstances surrounding the choice of a non-advertised appointment process.

Observations

Overview

While all four of the organizations examined had a formal human resources (HR) system, three of the four organizations did not rely solely on their HR system to provide their quarterly reports to the Public Service Commission (PSC). As a result, these organizations maintain a separate system — typically using Excel-based systems designed in-house — to manage this information. HR systems such as PeopleSoft, System Applications Products (SAP) and Human Resources Information System (HRIS) are used primarily for the storage of position information, and not for the management of the appointment process. The HR system is normally used to generate a process number at the beginning of the hiring process, while the remaining information would be captured subsequent to the appointment.

Based on the file review of a random selection of files, almost half of all non-advertised processes are either acting appointments of four months or over, or are extensions of acting appointments. Table 1 lists, in order of frequency, the various types of non-advertised processes that the four organizations conducted during the 2007-2008 fiscal year.

Source: Audit and Data Services Branch, Public Service Commission of Canada

As per the Appointment Policy Questions and Answers for the Choice of Appointment Process, the PSC has indicated a preference for advertised appointment processes to ensure respect of the guiding values. Respect of the value of access ensures that persons have a reasonable opportunity to apply and be considered for employment. However, under certain circumstances, non-advertised appointment processes provide the necessary flexibility to deputy heads. Appointments resulting from developmental programs, reclassifications, the special assignment pay plan (SAPP) and incumbent-based appointments, as well as some appointments of designated employment equity (EE) group members, will usually result in a non-advertised appointment process. For other types of appointments the use of non-advertised processes may have a greater impact on the guiding values.

Relevance: The information gathered should meet the needs of the clients

The relevance of statistical information reflects the degree to which it meets the needs of clients. Indicators ACC-1 and ACC-3 reflect the ratio of appointment processes that are non-advertised to external and internal appointment processes, respectively. The ratio of processes that are non-advertised is used as an indicator of access to federal government positions. High ratios of ACC-1 and ACC-3 may contribute to poor performance measures for organizations.

There are common criteria for the use of non-advertised appointment processes among the four organizations. For example, all have included reclassification, acting for less than 12 months, developmental programs, student bridging and highly specialized skills as criteria for using a non-advertised appointment process. There are further criteria that are common among more than one organization, including SAPP, EE appointment, conversion of tenure of employment from term to indeterminate and emergency situations. Based on the above organizational criteria, the type of non-advertised appointment processes is captured electronically in three of the four organizations.

Discussions with stakeholders within the PSC and the four selected organizations have raised concerns regarding the relevance of these indicators, given the manner in which they are calculated. Organizations have been critical of the fact that high ratios do not necessarily indicate reduced access to positions, given that non-advertised appointments may be used for promotions as part of a developmental program, when appointing a person to his/her reclassified position or when bridging a student into an indeterminate or term position. Organizations consider that these non-advertised processes do not represent a risk to the value of access. As a result, in February 2009, the PSC revised the type of appointment process applicable to student bridging appointments from non-advertised to advertised.

As part of its own internal risk management strategy, the Canadian International Development Agency differentiates non-advertised processes into two separate categories: those considered to be of higher corporate risk (e.g. any appointments to
EX-level positions, acting appointments over 12 months or promotions) and those that present lower risk (e.g. reclassifications, promotions within an approved developmental program or student bridging). In addition to having separate approval processes for these two categories of non-advertised processes, the Canadian International Development Agency also produces separate statistics for each category, as they believe that it is more important to monitor and restrict the higher risk processes.

The opinion that the current method of calculating ACC-1 and ACC-3 produces an ambiguous measure of an organization’s performance was expressed by all of the selected organizations. In addition, the PSC’s division responsible for handling organizational inquiries regarding the data analysis for leading indicators has noted that this opinion is expressed uniformly across government organizations. Similarly, the PSC’s division responsible for compiling the leading indicators and reporting them to organizations has expressed the opinion that more detailed information regarding the types of non-advertised processes would improve the relevance of the leading "Access" indicators. The ratios (ACC-1 and ACC-3) that are calculated by the PSC are discussed with the various organizations to determine whether or not they are relevant reflections of access to positions. This, as well as other information, is taken into account by the PSC in determining the final rating for these leading indicators.

To illustrate the impact of redefining ACC-1 and ACC-3, the variables were recalculated using the data from the sample of files selected for the file review. Based on the current definition, ACC-1 (external non-advertised processes) for all four organizations combined is 24% and ACC-3 (internal non-advertised processes) is 61%. If, for example, we calculate using only acting, acting extensions, promotions, conversion of tenure of employment from term to indeterminate (prior to the conversion period), shortage group and other appointments, the ratios become 9% and 38%, respectively.

Accuracy: The information gathered should correctly describe appointment information

Accuracy is the degree to which information correctly describes the phenomena that it was designed to measure. Two elements of accuracy that are relevant for administrative data are coverage and validity. Coverage is the degree to which the list of appointments provided by organizations to the PSC is a true reflection of their population of appointments. Over-coverage occurs if the list of appointments includes appointments from outside the population. Under-coverage occurs if the list of appointments is missing appointments from within the population. Validity is the accuracy of the data found within the dataset. Both coverage and validity are necessary conditions for generating accurate indicators.

The PSC receives organizations’ quarterly reports which list the characteristics of all specified appointments needed for calculating ACC-1 and ACC-3. These include appointments to the public service, promotions and acting appointments of four months or over. As a quality control procedure, the lists of appointments from the organizations are compared with a list of all known appointments from the PSC Fiscal Database, a database generated by the PSC from payroll information. The result of this comparison illustrates the degree of under-coverage within the quarterly reports.

Overall, federal government organizations sent the PSC 69% of all appointments listed in the PSC Fiscal Database for the 2007-2008 fiscal year. Of the four organizations examined, Transport Canada provided a relatively complete list of appointments (87%). Library and Archives Canada and Citizenship and Immigration Canada had near-average under-coverage rates (68% and 67%), and the Canadian International Development Agency had the lowest under-coverage rate of the four selected organizations (57%).

As part of the file review, the percentage of over-coverage was calculated. The quarterly reports sent from organizations to the PSC for the 2007-2008 fiscal year contained processes that should not have been included, such as deployments and term extensions. Only Transport Canada had an over-coverage rate of 0%, while the three remaining organizations had over-coverage rates that ranged from 7% to 12%.

Of equal importance to coverage is the validity of the information that is sent to the PSC. The file review demonstrated that this information contained some errors. While the Canadian International Development Agency had the lowest coverage rate, we found no errors in the data they sent. Conversely, Transport Canada sent the most complete information; yet, it had the highest error rate (14%). It should be noted, however, that more than half of the errors discovered in Transport Canada files were likely due to a systemic problem, which has since been corrected. Some advertised processes were coded as non-advertised. Specifically, while the first appointment from an advertised process was coded as advertised, subsequent appointments under that same process number were coded as non-advertised.

Of the 40 files reviewed from the sample that organizations indicated were advertised, only 1 (2%) was actually non-advertised. Of the 152 files that organizations indicated were non-advertised, 16 (11%) were found to be advertised. Six of these cases were subsequent appointments. This ambiguity between advertised and non-advertised appointments in the context of the collective staffing processes raised a number of questions in the HR community. It was clarified in January 2008 in a Letter to Heads of HR (08-02) that, in order to maximize flexibility, managers can make advertised appointments from a pool if the advertisement includes information clearly indicating how, by whom and for what occupational groups and/or level(s) the results of the collective appointment process may be used. When appointments (i.e. the first and all subsequent appointments) made from the pool are consistent with the information provided in the advertisement, they are considered advertised appointments.

Files were also assessed for their accuracy relative to being internal or external appointments. Of the 44 files that organizations listed as external appointments,
1 (2%) was actually internal and, of the 148 files that organizations listed as internal appointments, 7 (5%) were, in fact, external.

Timeliness: The information gathered should be available to clients in a reasonable amount of time

Timeliness refers to the delay between the end of a reference period to which the information pertains and the date on which the information becomes available. In the case of the PSC quarterly reports, timeliness is affected by two stages of data processing. The first is the abstraction or entry of information into electronic format; the second is the extraction of this information into a format that can be used by the PSC, and any required quality control measures.

Based on the interviews conducted with organizations, the delays associated with the entry of data into electronic formats is not a major factor affecting the timeliness of data. In the four organizations examined, information was typically entered into HR databases on a timely basis, with minimal backlogs.

The delays associated with the extraction and preparation of data showed extremely wide variances. While Transport Canada could produce the PSC quarterly report in a matter of days and with minimal effort, Citizenship and Immigration Canada estimated that it required approximately one month of effort from 1.5 full-time equivalents to prepare one quarterly report. This level of effort is due in part to the fact that much of their HR data management is decentralized, a characteristic associated with the structure of the organization and the nature of the work they perform.

Library and Archives Canada and the Canadian International Development Agency also reported that the preparation of PSC quarterly reports requires a great deal of effort. Organizations are requested to submit their quarterly reports one month after the end of each quarter. In the first quarter of the 2008-2009 fiscal year, approximately 15% of all federal government organizations were not able to meet this deadline. This statistic does not illustrate the differing levels of effort required by organizations to produce the PSC quarterly reports, but it does reveal that many organizations are not managing data in such a way as to easily access it. During the previous fiscal year, two of the four selected organizations asked for extensions for submitting their quarterly reports.

Accessibility: The information gathered should be easily accessed

Accessibility refers to the ease with which statistical information can be obtained. Reduced accessibility of data can be associated with reduced accuracy, including poor coverage and validity. Data that are easily accessible are likely to be accessible through electronic means and to require minimal manipulation. The data is therefore accessed in a more standardised manner (i.e. through electronic queries). When errors occur in the performance of an electronic query, they tend to be systemic, that is, the same type of error will occur multiple times. Such an error requires only one correction to the HR system to eliminate the occurrence of that type of error. Alternatively, errors that are associated with manual manipulation can result from inexperience, fatigue, indifference or human error. As a result, the variance of this type of error is much greater and more difficult to detect and correct.

As seen in the four organizations examined, there is an array of effort required to access information. In the case of the Canadian International Development Agency, information stored in SAP is not easily accessible given the amount of time and effort required to create customized queries (i.e. the Canadian International Development Agency estimates that the time necessary to adjust SAP to generate the PSC quarterly reports is approximately 18 months). As a result, the Canadian International Development Agency relies on an Excel-based system to manage the appointment process as a source of information for the PSC quarterly reports. As needed, the organization may refer to the appointment file to collect the required information.

At Citizenship and Immigration Canada, in order to help ensure data quality, production of the quarterly reports begins with reconciliation between data from their PeopleSoft system and an Excel-based system. Anomalies are then manually reviewed.

Library and Archives Canada’s current version of HRIS does not allow for customized queries. Staff rely on the interpretation of the process number to determine the type of process or, in many cases, on a manual review of the original appointment file to gather all the data needed for the PSC quarterly reports. This organization is currently in the process of developing an Access database for managing appointment processes.

Transport Canada is the only organization that uses its HR system to manage hiring process as well as position information. The organization has the capacity to easily modify its system and produce customized queries. The PSC quarterly reports are completed through electronic queries.

Interpretability: The meaning of the leading indicators should be easily understood

Interpretability refers to the availability of supplemental information in order to properly understand the leading indicators. The organizations are responsible for providing data from which the PSC calculates the leading indicators. As such, it is the PSC’s responsibility to provide the supplemental information for interpreting the leading indicators. Both ACC-1 and ACC-3 are relatively simple variables, and our examination found sufficient internal PSC documentation explaining how the statistics are calculated.

Coherence: The information gathered by different organizations should be comparable

Coherence of statistical information reflects the degree to which it can be successfully brought together with other statistical information. In this case, a high degree of coherence among the organizations in producing PSC reports is critical for being able to bring the data together and make valid inter-organizational comparisons. All organizations need to have standard concepts and classifications or groupings of data in order to produce reliable statistics.

The 2006 Report on Preliminary Data Collection Activities Related to Advertised versus Non-advertised Appointments indicated that there were inconsistencies in the interpretation of certain appointment processes. As noted earlier, the PSC issued a letter in January 2008 to clarify when a collective staffing appointment should be reported as an advertised or a non-advertised appointment.

However, a further data definition issue was identified following the file review of extensions to acting appointments that were originally of four months or over. One organization treated such extensions to acting appointments as subsequent non-advertised appointments, while the remaining organizations treated such extensions as administrative measures. However, for the purpose of the quarterly reports to the PSC, two organizations considered acting extensions as appointments. The Public Service Staffing Tribunal decisions indicate that extensions to acting appointments are considered appointments. However, there is currently no PSC policy interpretation on this matter. This data definition issue contributes to a reduced level of coherence of the data collected and reported by the PSC; further,  it can affect inter-organizational comparisons, as well as inflate the ratio of non-advertised appointments for certain organizations. The PSC will examine this issue as part of its policy review and its five-year review of the Public Service Employment Act (PSEA).

Concluding remarks

The quality of data from organizations depends on several interrelated factors, including the size and structure of the organization, the type of human resources (HR) system being used and the availability of resources to maintain HR information. This study illustrates how the challenges to producing quality data vary greatly among organizations.

Transport Canada has a system that can be used to manage appointments and easily produce data. Although data is entered in several regions, it is managed centrally. This system can be modified to meet Transport Canada’s needs and those of central agencies.

The Canadian International Development Agency’s current HR system, Systems Applications Products (SAP), is used in few other organizations. There are few resources available with the capacity to modify the system, limiting its responsiveness.

Library and Archives Canada is a medium organization which has limited resources available for maintaining a complex HR system.

The management of Citizenship and Immigration Canada’s HR data is decentralized and must be cross-referenced to a secondary tracking system, contributing to the difficulty of reporting information regarding select appointments.

In three of the four organizations examined, detailed information regarding non-advertised processes is captured electronically but, for two, it remains difficult to access. If the data were accessible, a more relevant measure of access to positions could be calculated. Organizations, however, already seem to have great difficulty meeting the current demands of the Public Service Commission (PSC) quarterly reports.

The PSC is committed to clarifying the policy interpretation surrounding extensions to acting appointments. Further, the PSC will explore other options and methods of data collection from organizations.

Study team

Vice-President, Audit and Data Services Branch:
Jean Ste-Marie

Director General, Government-wide Audit and Evaluation Directorate:
Terry Hunt

Director, Audit and Studies Division:
Ewa Ochmann

Manager, Audit and Studies:
Manon LeBrun

Manager, Systems and Methodology:
Paul Pilon

Functional Expert:
Maria Catana

Analysts:
Guillaume Fontaine
Linda McNamara
Renée-Anne Ouellet

Student:
Natalie Desimini

Appendices

Appendix 1 – About the study

Objective

The purpose of the study was to describe data collection practices related to non-advertised appointment processes and the limitations of the data available, as well as to explore associated risks and areas for improvement for the Public Service Commission (PSC) and organizations.

Approach

The study focused on the non-advertised appointment activities of four organizations for the period of April 1, 2007, to March 31, 2008. Organizations were selected based on the following criteria: (1) obtaining a critical mass of non-advertised appointment processes; (2) obtaining organizations that use different human resources (HR) information management systems; (3) obtaining a mix of medium and large-sized organizations; and (4) obtaining both centralized and decentralized organizations.

As a result of the above selection process, the following organizations were included in the study:

• the Canadian International Development Agency;
• Citizenship and Immigration Canada;
• Library and Archives Canada; and
• Transport Canada.

Multiple lines of evidence were used to address the study objectives, including the following:

• Interviews with key stakeholders, including the following representatives within each organization: HR operations, HR oversight and HR systems specialists;
  
• Examination of the organizational information of appointment processes and its potential use for data analysis;
• Observation (walk-through) of the data collection process from the appointment file to the HR information management system;
• Documentation review, including various monitoring reports and criteria for non-advertised appointment processes; and
• File review of a representative sample of appointment processes.

Sampling method

The quarterly reports of all appointments from the 2007-2008 fiscal year for the four selected organizations were used as the sampling frame for the selection of processes for file review Forty non-advertised processes and ten advertised processes were randomly selected from each organization, with the exception of Library and Archives Canada, who had only thirty-six non-advertised appointment processes during that fiscal year. Files that were out of scope, such as deployments or term extensions, were replaced with other randomly selected files. Table 2 summarizes the populations, sample sizes and number of files found out of scope for each organization.

Source: Audit and Data Services Branch, Public Service Commission of Canada

Confidence intervals from samples of non-advertised appointment processes for individual organizations are +/-10% or better, given a 90% confidence level and deviation rates of 20% or less. The confidence interval from the sample of non-advertised appointment files for all organizations combined is approximately +/- 5%, given a 90% confidence level and deviation rates of 20% or less. The confidence interval from the sample of advertised appointment files for all organizations combined is approximately +/- 10%, given a 90% confidence level and deviation rates of 20% or less. When necessary, weighted ratios are calculated to account for non-proportional sampling.

Appendix 2 – Reporting methodology

The study made use of the Statistics Canada’s Quality Assurance Framework to define data quality and organize the lines of inquiry for the study and the report findings. This framework defines six dimensions of data quality.

Source: Statistics Canada’s Quality Assurance Framework, September 2002

Appendix 3 – Leading indicators of long-term success

Intended effects: Progress in improved short-term outcomes

Results: Effectiveness and adherence to staffing values

Merit

MER-1: Managers’ satisfaction with the quality of hires;

MER-2: Percentage of non-imperative appointments where the person does not meet the official language profile within the time periods prescribed by regulations;

MER-3: Candidates’ perceptions on whether actual job requirements match advertised requirements;

MER-4: Candidates’ perceptions on whether the posted qualifications and criteria for positions are bias-free and barrier-free;

MER-5: Trends in founded Public Service Staffing Tribunal complaints and Public Service Commission investigations into staffing; and

MER-6: Trends in in-house investigations and founded complaints.

Non-Partisanship

NP-1: Managers’ perception of external pressure to select a particular candidate; and

NP-2: Number of founded complaints related to political influence in staffing.

Representativeness

REP-1: Staffing strategies — employment equity (EE) [distribution of hires versus work-force availability]; and

REP-2: Number and percentage of internal and external staffing advertisements with EE provisions — to enhance access of EE group members to public service positions.

Access

ACC-1: Ratio of non-advertised external processes to total;

ACC-2: Ratio of external processes advertised nationally to total;

ACC-3: Ratio of non-advertised internal processes to total; and

ACC-4: Trends of managers’ selection of staffing process.

Fairness

FAIR-1: Candidates’ perceptions of the fairness of the assessment process; and

FAIR-2: Percentage of acting appointments that becomes indeterminate in the same or similar position in the same organization.

Transparency

TRANS-1: HR plans and staffing strategies are communicated on organizations’ Web sites, and contents are clearly communicated to employees and managers;

TRANS-2: Organizations demonstrate consultation and communication of staffing-related information to unions;

TRANS-3: Length of time that job openings are posted; and

TRANS-4: Candidates’ perceptions of openness and transparency in internal staffing.

Appendix 4 – Glossary

Access
One of the guiding values of the Public Service Employment Act requiring that persons from across the country have a reasonable opportunity, in their official language of choice, to apply and to be considered for a public service employment.

Acting appointment
The temporary promotion of an employee.

Acting extension
The increase in the length of the temporary performance of the duties of another position by an employee.

Advertised appointment process
When persons in the area of selection are informed of and can apply to an appointment opportunity.

Appointment
An action taken under the Public Service Employment Act to hire or promote someone.

Appointment values
Core values (merit and non-partisanship) and guiding values (fairness, access, transparency and representativeness).

Collective staffing process
An approach that allows for one staffing process to fill several similar positions within or between departments and agencies, as opposed to several individual processes.

Confidence interval
An interval which has a known and controlled probability (generally 95% or 99%) to contain the true value.

Conversion of tenure of employment from term conversion
When a person has been employed in the same organization as a term employee for a cumulative working period of three years without a break in service longer than sixty consecutive calendar days, the organization must appoint the employee indeterminately at the level of his/her substantive position.

Deployment
The movement of a person from one position to another in accordance with Part 3 of the Public Service Employment Act. A deployment cannot be a promotion and cannot change the tenure of employment from specified term to indeterminate. A person who is deployed is no longer the incumbent of his or her previous position.

Designated groups
As defined by the Employment Equity Act, women, Aboriginal peoples, persons with disabilities  and members of visible minorities. Often referred to as employment equity designated groups.

Developmental programs
Programs that offer learning and leadership development by providing hands-on work assignments and formal learning components.

Deviation rate
The total variation displayed by a set of observations, as measured by the sums of squares of deviations from the mean, may in certain circumstances be separated into components associated with defined sources of variation used as criteria of classification for the observations.

External appointment process
A process in which persons may be considered, whether or not they are employed in the public service.

Fairness
One of the guiding values of the Public Service Employment Act requiring that decisions be made objectively and free from political influence or personal favouritism, that policies and practices reflect the just treatment of persons, and that persons have the right to be assessed in their official language of choice.

Incumbent-based appointment
An internal appointment process within the Research and University Teaching groups, if there is a career progression framework established by the deputy head.

Indeterminate employment
Employment of no fixed duration, whether part-time, full-time or seasonal.

Internal appointment process
A process for making one or more appointments for which only persons employed in the public service may be considered.

Leading indicators
Infrastructure, processes, practices and actions, as articulated under the Staffing Management Accountability Framework’s Key Change Areas and associated indicators, that are necessary for the achievement of positive staffing results.

Merit
One of the core values of the Public Service Employment Act. An appointment is made on the basis of merit when a person appointed meets the essential qualifications for work to be performed, as established by the deputy head, including official language proficiency. The sub-delegated manager may also take into account any current or future asset qualifications, operational requirements and organizational needs.

Non-advertised appointment process
An appointment process that does not meet the criteria for an advertised appointment process.

Non-partisanship
One of the core values of the Public Service Employment Act, non-partisanship ensures that appointments and promotions to and within the public service are made free from political influence and supports the capacity and willingness of employees to serve governments, regardless of political affiliation.

Over-coverage
Arises from the presence in the frame of units not belonging to the target population and of units belonging to the target population that appear in the frame more than once.

Promotion
An appointment to a higher-level position.

Public service
As defined by the Public Service Employment Act, the organizations named in Schedule I to the Financial Administration Act, the organizations named in Schedule IV to that Act and the separate agencies named in Schedule V to that Act.

Reclassification
The reclassification of a position occurs when the evaluation of its work description results in a change to either or both its occupational group and level, due to a significant change in the work assigned to it. A reclassification may be upward, if the work becomes more demanding, or downward, if the work becomes less demanding.

Representativeness
One of the guiding values of the Public Service Employment Act requiring that appointment processes be conducted without bias and without creating systemic barriers in order to help achieve a public service that reflects the Canadian population it serves.

Shortage group
Lack of persons available having the skills, abilities or training required for the functional community.

Special assignment pay plan
Organizations may assign a specified number of employees, for a period of up to three years, to duties and responsibilities that have not been classified or that have been classified at a lower level to which the employee is appointed.

Specified term employment
Part-time or full-time employment of a fixed duration.

Staffing activities
Include internal and external appointments to the public service, deployments, assignments and secondments.

Student bridging
A mechanism that allows managers to hire qualified recent post-secondary graduates who have previously worked in the public service through student employment programs, such as the Federal Student Work Experience Program, the Co-operative Education and Internship Program or the Research Affiliate Program, in accordance with the organizational criteria or policies.

Term extension
The increase in the length of the employment of a fixed duration.

Term to indeterminate
The non-advertised appointment of the employee indeterminately at the level of the employee’s substantive position prior to the term conversion period.

Transparency
One of the guiding values of the Public Service Employment Act, requiring that information about strategies, decisions, policies and practices be communicated in an open and timely manner.

Under-coverage
Results from the omission from the frame of units belonging to the target population.

Primary navigation (left column)

Date Modified:

2009-10-09