By Jack Abraham, EdD
If you have taken a moment to ponder which promotion process is best, you may simply shrug your shoulders and say, “It depends.” It depends, of course, on your perspective and experience with promotions.
If you are a chief officer, you probably went through a promotion gauntlet to get to your rank. Members new to the profession have the gauntlet to look forward to.
This article will review eight promotion processes and identify the strengths and weaknesses of each process.
Officers in fire departments with established promotion processes may find this information useful in making improvements in how vacancies in ranks are filled. Recently formed fire departments or newly appointed fire chiefs may find this information useful in implementing a promotion process where none existed before or an existing promotion process can be improved.
Cultural and legal significance
There should be little debate about the importance of fire department promotions. It certainly is important to the members who get promoted and those who do not get promoted in the context of their career advancement. It certainly is important to the management of the fire department when members are promoted to supervisory ranks. And technical skill positions, such as driver/operators and code enforcement officials, must be filled with members who are prepared for those responsibilities before being promoted. Promotions also say a lot about the culture of the fire department and the leaders who make the ultimate promotion decisions.
How promotions are made also has important legal implications for all concerned. Litigation has resulted because the civil rights of department members were alleged to have been violated. (It should be noted that this author is not an attorney and not qualified to give legal advice.)
Depending on the fire department’s local government authority, the promotion process may need to be approved by a higher authority, like a city manager or board of commissioners and legal counsel.
Start with a solid job description
So where does the promotion process begin? In my opinion, it should begin with a well-written, accurate, legally defensible job description for each rank. The job description will document the skills necessary for each rank, qualifications necessary to execute those job skills, and who is eligible for each rank.
While a detailed discussion of job descriptions is outside the scope of this article, the job description should distinguish between ranks such as captain and lieutenant and have justification why different qualifications are necessary for the rank. If one rank requires “X” years of experience and the other rank requires “X+2” years of experience, why? If one rank requires a college degree and one rank does not, why?
The reasons why the job description is the beginning of the promotion process include validity and reliability:
- Validity means that the promotion process is based on requirements that are related to the job description.
- Reliability means that the process results in consistently good promotion decisions.
For example, the promotion process for an apparatus driver/operator would be based on the knowledge, skills and abilities necessary to drive and operate fire apparatus. The valid promotion process may include a demonstration of the candidates’ ability to establish a draft from a portable drafting tank while flowing 200 gpm. The reliable promotion process would result in candidates being promoted who are ready to do the job, in accordance with the job description, on the first day of the candidate’s promotion and that promoted members perform successfully in their new rank. Promoting a member to a supervisory rank who has never had supervisory responsibility or training/education would have questionable reliability.
Another important component of the promotion process is a written standard operating procedure (SOP). An example of a written procedure would be different schedules for promotion testing, like if the promotion process is implemented on an as-needed basis (when a vacancy occurs) or if the promotion process is a calendar event regardless of openings. The calendar method results in a list of prequalified members who can be immediately promoted if/when a vacancy needs to be filled.
The promotion procedure must be available to all members to read. Any deviation from a written procedure can result in a grievance or legal challenge by the members of the department.
Promotion process options
Let’s now review the eight types of promotion processes. Keep in mind that there are two basic types of decision-making criteria: subjective and objective. Subjective decision-making is basically the opinion of the decision-maker(s). Objective decision-making is measurable and allows for the ranking of scores by candidates. Regardless, the promotions should result in a justifiable reason why selected members are promoted.
The order in which the following promotion processes alternatives are presented is intended to progress from the simplest to the most complicated.
1. Appointment: An appointment occurs when the promotion decision is made without a formal comparison process or competition between the candidates. In other words, a member is simply selected for promotion. This may be a very subjective decision.
Advantages: Quick, inexpensive, and the chief (or decision-maker) can place their subordinate members in ranks that will support the chief.
Disadvantages: The chief may have a personal or professional bias against certain members in which case the promotion decision is based upon what can be perceived to be in the best interest for the chief and not the fire department as a whole. There is an important legal consideration that applies to appointments, and will apply to all other processes: If there is a pattern of decisions that discriminates against Equal Employment Opportunity Commission (EEOC)-protected class individuals, there may be a violation of federal law. For example, if only white males are chosen for promotion when qualified African-Americans, women or other protected class individuals are available, there may be a justified legal challenge to the promotion decisions.
2. Seniority: Promotions based on seniority will go to members who have been in the firefighting profession – or a member of the fire department – for the longest period of time. This is a decision based on objective criteria.
Advantages: The seniority promotion process is predictable and easy to understand. Members are rewarded for staying with the fire department, and members who know with certainty they will be promoted may take the opportunity to prepare for their new rank.
Disadvantages: Time in service does not necessarily result in progressive development for promotion. There is the saying that a member can have 10 years of experience or one year of experience 10 times. A promotion process based solely on seniority may result in a culture of entitlement for those members who simply wait for their promotion turn.
3. Resumes: Resumes provide a convenient way to compare candidates on paper. If this process is used, the resumes should be standardized in format so each resume is identical with the exception of the content of the individual candidate’s resume. This can be accomplished by having all resumes prepared by the same person, such as an administrative assistant. The candidates would provide the information necessary to complete the resume – education, work experience, certified training, additional responsibilities, etc. Each candidate should sign the resume stating that the information is true and accurate.
Once the resumes have been prepared, they can be graded in the context of the job description. One method of doing this is having professionals fire service evaluators outside the fire department do the grading. The resumes can be assigned a serial number so the evaluators do not know the name of the candidates they are grading.
Advantages: The process of creating a resume prepares candidates for upper-level ranks. Resumes also allow for objective comparisons among the candidates. Members participating in the promotion process will also have the opportunity to list their accomplishments and job-related activities that may distinguish those members who go above and beyond the minimal job requirements.
Disadvantages: The preparation of resumes, especially if there are a large number of promotion candidates, will be time-consuming. A standardized grading process will need to be established. If multiple evaluators of the resumes are used, they should not collaborate on their grading to eliminate possible group-think bias.
4. Performance evaluations. Using performance evaluations in the promotion process is similar to doing background checks on new firefighter applicants. The information on the evaluation form can be very subjective and difficult to compare when different officers are evaluating their subordinates. Some officers may be prone to a “halo effect” where all of their personnel are evaluated as being above average. Other officers may evaluate most of their subordinates as average.
Another performance evaluation technique is called a 360-degree evaluation. This survey technique requires everyone who works with the candidate to do an evaluation using characteristics such as leadership, dependability, safety, etc.
Advantages: The best predictor of future performance is often past performance. Using performance evaluations or peer evaluations can help distinguish among candidates’ current job performance and work attitude.
Disadvantages: The subjective nature of most performance evaluations can be a problem. This may be controlled by requiring officers who rate subordinates above or below average to justify the rating in writing. If 360-degree evaluations are used, it may be necessary to eliminate the highest and lowest few evaluations to control for members who may rate their peers as perfect (halo effect) or terrible (hate effect).
5. Interviews: The purpose of promotion interviews is to have a face-to-face discussion with the candidates to evaluate their interpersonal skills and their ability to respond to questions in a pressure setting. Promotion interviews have been referred to as the candidates’ “day in court.”
There are two types of interviews: structured and unstructured.
- Structured: All candidates are asked the same questions by the same interview panel member in the same order.
- Unstructured: Candidates can be asked any question by any panel member.
All interview questions should be based on the job description and past job performance. There should be an evaluation form to “grade” each interview, and the scoring of interviews and may be both subjective (opinion) and objective (numerical).
Advantages: Some members who may not score well in other promotion processes, such as taking a written test, may do very well in an interview. An interview panel will give multiple evaluations (based on the number of interviewers) of each candidate.
Disadvantages: Constructing interviews is time-consuming and requires the scheduling of the interview panel and the candidates. The panel has to agree on the correct answers to the interview questions so scoring of the interview will be valid. Also, confidentiality of the interview questions between the first candidate and the last candidate interviewed can be a problem.
6. Assessment centers: Assessment centers are job-related activities that evaluate the ability of the candidates to perform given tasks in a given situation. The situation may include troubleshooting a pumper that cannot pull a draft, initiating a disciplinary action, prioritizing multiple messages during an in-basket exercise, or similar structured situations. The scoring by the evaluators may be both subjective and objective.
Advantages: Candidates are evaluated based on their performance during the job-related situations. As long as the evaluators agree on what the correct actions of the candidate should be, the scoring should be valid. Candidates who are not prepared for the rank they are trying to be promoted into will probably not do well in an assessment center.
Disadvantages: Assessment centers – like interviews – can be time-consuming and expensive to administer. Security of the exercises will be a challenge. The optimal answer(s) to the assessment center should be based on the job description and written SOPs. Also, discussion among evaluators may influence scoring.
7. Written tests. Written tests permit the objective comparison of candidates based on test scores. After a written test is administered and graded, all candidates who exceed the minimum passing grade would be considered eligible for promotion. Therefore, establishing a justifiable passing grade is important if the written test is going to be the sole decision-making criteria for promotion.
If the “passing score” is 75 and not 70, why? It is important to accept that there is no EEOC or affirmative action policy that requires the promotion of an unqualified candidate, regardless of protected class status. There is also no legal requirement to promote the individual with the best score. This type of decision-making criteria would be an important part of the written promotion SOP.
Testing of any type exposes the department to compliance with the Americans with Disabilities Act (ADA). If a member has a certified learning disability – such as dyslexia or attention deficit disorder – and presents written documentation of this disability, then the prescribed accommodation must be provided. Accommodations may include extra time to take a test, reading the test to the candidate or taking the test in a private setting instead of an open classroom. The written promotion procedure should include a statement of ADA compliance and how the affected members should present documentation to the individual responsible for administering the test prior to the test date.
Advantages: Presuming that the written test is valid (job-related) and reliable (previous promotions have been successful), the ranking of the candidates by test grade will determine who is most knowledgeable of the promotion material based on the questions asked on the test. A test that is constructed in-house can have a high degree of what is called criterion validity if the test is directly related to the fire department’s job descriptions, SOPs and training program.
Disadvantages: The construction of valid and reliable written tests is very difficult. A written test that results in a disparate (unintended) impact on protected class individuals may be challenged in court. If half of the candidates taking the promotion test are protected class individuals but very few pass the test, there may be a disparate-impact problem. The individual(s) who constructed the test will have to defend the methods that were used to validate the test and used to determine the test’s reliability.
One method to determine the validity of a test created in-house is to have a trusted professional outside of department take the test as a pilot before it is administered in the promotion process. This way the test can be checked for grammatical errors, clear wording, correct answers, etc.
An alternative to constructing in-house written promotion tests is to purchase written tests that are available from numerous vendors. Make sure that a purchased written test accurately evaluates the candidates in the context of the job description of the rank the candidates are being tested for. A captain’s promotion test for a large urban fire department will not be valid for a captain’s test in a small community department.
Make sure that the vendor of the purchased test will defend the test in court if it is challenged.
Ask to see the validity and reliability studies of any test before it is purchased. The studies will be a statistical analysis of the previous times the test has been administered. Consult with a math professor at a local college or high school if you have difficulty understanding this information. Pay specific attention to the demographic information on how different demographic groups scored on the test.
Before the results of a written test are announced, a thorough test-analysis should be performed. If a multiple-choice test used, an appropriate test analysis would include how many times each question is missed. If everyone taking the test missed the same question, then it is either a bad question (meaning that the question is not based on valid criteria) or the grading sheet was mismarked. The actual scoring of the test may change if these or similar types of errors have occurred.
8. Elections: Elections are common in volunteer departments. Written job description-type qualifications will help members to prepare to be qualified for their next rank. It is no less important to have a written procedure on how the election is to be administered.
Advantages: Followers like to follow a popular leader who they have helped get elected. If a member gets reelected, presumably they are doing a good job. Also, the opportunity to remove an ineffective officer by election can be can advantage.
Disadvantages: A closely contested election can be divisive among the department membership. An unqualified member may be elected if the promotion candidates are running only on popularity or how close they live to the fire station, for quick response times.
Cheating
Cheating is a fact of life. In today’s high-tech environment, be aware of the use of smartphones and watches, micro-cameras and transmitters/recorders. Access to these devices should be carefully controlled while candidates are being evaluated for promotion. Make sure that there is security of written tests before and after administration.
Final steps
There are seven final steps in the promotion process:
- Double check all test scores before test results are announced. It is a good idea to have an independent person, such as an auditor or Certified Public Accountant, verify the accuracy or the final tabulations.
- Some jurisdictions may require that the supervisory authority over the fire department review the promotion results and decision(s) before a public announcement.
- Review the test results with each candidate. It is important for the candidates to learn their strengths and weaknesses whether or not they are promoted. Be prepared to answer questions about why certain scores were earned by the candidate.
- Conduct a critique of the promotion process. The review of promotions allows for continuous improvements to be made. Candidates who went through the process can be asked to submit anonymous comments.
- Written document retention is important. Keep all documents related to how the promotion decision is made for the legally required time which will vary by the jurisdiction’s record retention policy. Consider document security; you do not want copies of the test to get circulated before the test is administered in the future.
- Promoted members should serve a probationary period during which time their job performance is evaluated. The purpose of the probationary period is to allow the member to become adjusted to his or her new job and to coach the individual as needed. Promoted members who frequently underperform during the probationary period may indicate a reliability problem with how promotion decisions are made.
- The promotion decision should be publicized to all members as simultaneously as possible. It is also an opportunity for positive publicity for the fire department and the members who are promoted if there is a news release in the local newspaper and webpage with pictures, biographical information and a brief description of the promotion process.
Determining the best process
So, which of the above is the best process? The answer lies in the desired outcome – a valid and reliable evaluation of the candidates’ readiness for their next rank.
It is possible, of course, to use a combination of different methods. If objective scores are the outcome of using multiple processes, then a weighted average may emphasize the relative importance of some methods when compared to others in overall process. For instance, a performance evaluation may be worth 10% of the final evaluation score while a written test is worth 60%, and an assessment center is worth 30%. It is very important that the promotion SOP specifies how the overall promotion process will be scored.
To-do list
Here’s a simple checklist for any department to follow when evaluating their promotional processes:
- Verify an accurate job description exists for all ranks.
- Review/develop written promotion SOP. Administer the process as written.
- Verify promotion evaluation methods are reliable and valid.
- Check for scoring errors after the evaluation is completed and before the results are announced.
- Review results with each member who participated in the process.
- Make promotion decision(s). Announce promotion of fire department members and media.
- Evaluate promoted members during probationary period.
- Maintain document retention, including demographic analysis of evaluation methods.
- Maintain test document security.
- Critique promotion process for future improvement opportunities.
About the Author
Dr. Jack Abraham, EdD, is an assistant professor of business at Erskine College in South Carolina. He has a bachelor’s degree in business administration from Virginia Tech, a master’s degree from Webster University, and a doctorate in education from Clemson University. Abraham has 25 years of experience teaching business in higher education as an adjunct faculty member of several institutions, including Clemson, the University of Memphis, and the National Fire Academy. He retired after 37 years in the fire service, including being the fire chief at three different departments. He has numerous published articles on business and the fire service in professional periodicals.