How To Rate Your Fire Department

Twenty-six years ago, the book How to Judge Your Fire Department: 500 Questions With Commentary was written by Warren Y. Kimball, former chief fire service specialist for the National Fire Protection Association (NFPA). The book, based on a 526-item questionnaire "which local fire departments could use to check themselves against recommended good practices," was published in 1972 by NFPA and has been long out-of-print.

Photo by Chris E. Mickal Volunteer firefighters participate in LPG training exercises at the Jefferson Parish, LA, Training Center. Does your fire department have at least an annual training plan designed to meet its needs? That's one of many criteria members can use in making a formal assessment of a department's personnel, equipment, procedures and policies.

The book was the first publication which presented a comprehensive approach to evaluating departments. Its 500 questions, each with an explanatory paragraph, were divided into seven sections: apparatus, equipment and facilities; fire prevention and investigation; life safety services; management; operations; private fire protection; and command. Of course, some questions reflected practices which today are different. But they still can be a helpful tool in a department's self-improvement program.

Years ago, fire departments typically were rated by only two groups: the members of the department itself and - once every five or 10 years - an official fire rating organization such as the Insurance Service Office (ISO) or a state fire rating board. Today, departments are being rated on a continuous basis by municipal finance directors; city, town and county managers; elected officials; and taxpayers. Money is talking, and in a noticeable number of departments some members have been walking. It may be that at least some of these local evaluations have little to do with fighting conflagrations but much to do with cost control and increased community willingness to assume risk by decreasing department size and strength.

A question of growing importance, then, is what methods are available to fire officers to assess their own departments, both to help educate and negotiate with those who control resource allocations, and - just as important - to form the basis for department planning? There are several ways to conduct a rating project but each calls for work on the part of department personnel. The primary purpose of this article is to present details for a preliminary approach, which I call the "Rate Your Department Overview," and which I've been using as part of department studies.

Let Others Do The Work

The first, and perhaps the easiest, way to obtain a department rating is to let an ISO or state rating board do most of the work. Advantages are that the method of scoring each item is well-tested, the raters generally are experienced and the process is reasonably short. Disadvantages are that the primary purpose is to establish key rates for fire insurance, not to improve the department, some items of importance to the department are not reviewed and a poor showing can cause an increase in fire insurance premiums.

Because ISO evaluations are designed to provide a basis for setting fire insurance rates, the items typically checked are linked directly to the ability of a department to receive and transmit alarms, flow water, and arrive at a fire with sufficient resources and in a timely way.

Some states, using their own rating organizations, follow more closely the older ISO rating schedule - changed in 1980 - which included such additional major topics as fire prevention. Also, approximately 65 cities and counties in the United States - never get an ISO team visit but are "experience/statistically rated."

Although I sometimes hear fire officers express low interest in ISO and state board ratings, I've noticed an increase in attention paid to them by municipal managers and elected officials. I suspect there are three reasons for this. First, until very recently there were no other "official" ways to obtain a general view of a department except, perhaps, for two NFPA standards: 1201, Standard for Developing Fire Protection Services for the Public; and 1500, Fire Department Occupational Safety and Health Program. Second, many municipal officials relate the ratings to the department's firefighting capability, and straight firefighting runs are on the decline; therefore, some budget keepers view "firefighting" ratings as having the potential to point toward reductions in the size of fire departments. Third, an improved rating (grade 4 to grade 3, for example) lets elected officials announce, happily, widespread community savings in certain insurance premiums.

Over the past years, changes have occurred in the 1980 ISO rating schedule. For example, departments which pass the qualifying test now can get credit for rural water supply operations. Rating your department means keeping up with the insurance-related rating schedules. If you need help, you may review a copy of Dr. Harry E. Hickey's book, Fire Suppression Rating Schedule Handbook, available from the Society of Fire Protection Engineers (SFPE).

Self-Assessment Model

The second way to rate your department is by using the newest, most comprehensive and what should be the most useful long-term assessment program. This is the Commission on Fire Accreditation International (CFAI) Fire and Emergency Services Self-Assessment Manual and accreditation process, which has the potential to lead a department to a voluntary fire and emergency service award of accreditation. This comprehensive program recognizes the great gain obtained when citizens, local officials, and fire and other emergency personnel assess and evaluate their level of service delivery and make desirable adjustments. Led by a team of fire service personnel within the International Association of Fire Chiefs (IAFC), the program was in development for more than a decade before being put into use. The program also was supported throughout its development by the International City/County Manage-ment Association, which continues to be active in the CFAI.

Following an initial test of the assessment program and accreditation model in a southwestern city, the program was tested further in 12 diverse departments over eight months in 1993 and 1994. Agencies using the manual evaluate their performance in 10 categories:

1. Governance and administration
2. Assessment and planning
3. Goals and objectives
4. Financial resource
5. Programs
6. Physical resources
7. Human resources
8. Training and competency
9. Essential resources
10. External systems relationship

Hiring Consultants

The third approach to rating your department is to contract with an individual consultant or consulting firm. There are experienced firms which conduct several assessments (or "audits" or "studies") each year but even with these you can expect to spend time digging through records and other materials to provide necessary documentation.

The most important step in this process is designing the "request for proposals"(RFP). This provides the consultant with what you want the nature and scope of the study to be. Even the best consultants can't read your mind, so determine what you want rated before the work begins.

Overview Rating

The fourth method for conducting a rating project, and the focus of this article, is what I term a "Rate Your Department Overview." Here is a quick way to see where the department is generally and in which areas it needs to move forward.

The purposes of the Overview Rating are to develop a department profile that indicates what needs improvement and serve as a preliminary guide to more developmental planning for improvement.

To begin, list the major components of the department and the services delivered to customers. These lists may include:

Structural Components

• Personnel
• Facilities
• Equipment
• Health and safety provisions
• Personnel practices and relations
• Support Services
• Alarm, dispatch and communications
• Pre-planning

Service Components

• Fire prevention and public safety education
• Fire suppression
• Emerging medical and healthcare services
• Technical rescue
• Hazardous materials response
• General customer safety services
• General customer Support

For each component compose a list of key questions. This probably is the most difficult aspect of the process, since the questions must lead directly to value judgments concerning the department's place on a satisfactory/unsatisfactory scale. Coming up with an accurate self-assessment answer to each question also is difficult.

1. The questions must be geared directly to the type of department and the nature of its community.
2. The questions must simultaneously observe two very important considerations -
3. 1. Safety for both department personnel and customers.
   2. A defined level of assumed risk which is understood and accepted community-wide.
4. If there have not been community-wide risk level decisions, the department must prepare tentative, temporary ones based on local experience and available standards.
5. Documents such as NFPA standards plus national and state regulations often provide "must do" or "must aim for" levels.
6. In the interest of time, each major component should have no more than 10 questions assigned to it.

This Overview Rating provides only a general department profile. The IAFC Self-Assessment Accreditation process provides many detailed measurements to furnish a total assessment.

For each of the Overview Rating categories it helps to envision a vertical scale on which there are four indicating points. This produces the department "profile." From low to high, these are:

• Unacceptable level (UL). Serious, drastic and immediate remedial measures need to be taken. This is the lowest point on the scale. (An example might be a volunteer department that requires three or four daytime pages before a two-person ambulance crew can be assembled at the station.)
• Adequate level (AL). This describes a barely adequate to acceptable level for a condition or situation of the department. (An example might be requirements for a medical examination and physical fitness test prior to entry with a minimum follow-up required every two years thereafter.);
• Reasonable maximum attainment level (RMAL). This is likely reachable by the department with planning, effort and a level or modestly increased annual budget. With some components this is the same as the current level; with others it is higher. (An example might be the planned addition of a hazmat response unit.)
• Ideal maximum attainment level (IMAL). This would be an excellent condition or situation for the community and the department but most often cannot be reached without major changes in the operating environment. (An example might be a modern emergency communications system dedicated solely to the fire department.)

After a department has listed the major components and categories it wishes to "score" in the Overview Rating process, and after it scores each one, a graphic profile of its condition and situations can be constructed. For example, a hypothetical department scoring itself in six major components might be profiled as shown above - for this department, major plans and immediate efforts are needed to upgrade apparatus and the ability to respond adequately and safely to hazardous materials incidents; the training program and the fire prevention program can be maintained at their current level; and some focus of attention and some resources should be devoted to improving technical rescue capability and the always challenging condition of personnel relations.

Many of the components are interrelated. To illustrate, if the training officer begins to build new programs to improve hazmat and technical rescue capabilities, other training areas may suffer, thus lowering the "training" rating while raising the "technical rescue" and "hazmat" capability levels.

The purpose of the Overview Rating process is to provide an easy-to-obtain, general view of where a department stands on major components and where improvement is needed. Detailed study conducted internally or by outsiders can provide the basis for developmental planning.

The best Overview Rating questions for a department to ask itself under each component or category have to be designed specifically for that department's unique circumstances. These circumstances include, but are not limited to, the following:

• Known strengths and weaknesses of the department.
• Organizational type.
• Depth and breadth of the service delivery items needed by the community.
• Degree of risk desired by the taxpayers.
• Status of the local economy.
• Strengths and interests of the members.
• Emergency run workload.
• Availability of automatic or on-call mutual aid.
• Type and condition of facilities, apparatus and equipment.
• Volume of necessary inspection services.
• Nature of the emergency communications system.
• Strength and appropriateness of support services.
• Degree of support from local officials.
• Degree of support from local taxpayers.
• Volume and strength of competition for local dollars.
• Willingness of the organization to change.
• Willingness of the community to encourage change.
• Strength of department leadership.

The key questions generally should be broad based because this process seeks an overview of the current status of the department, thus providing a profile of strengths and weaknesses.

The following illustrate the types of check-list questions that can provide an overview of eight of the major department components used to construct the basic profile of a department, whether volunteer, career or combination.

STATIONS

1. Are fire stations located so that high-hazard areas are within four minutes running time of first-due apparatus?
2. Are unstaffed stations located in areas where responding personnel are readily available?
3. Are stations large enough for safe apparatus boarding and movement?
4. Are the living quarters, work areas and ready-rooms clean and healthy areas?
5. Can station drills, classes and individual studying be carried out conveniently?
6. Are volunteer stations set up to encourage frequent use by members?

APPARATUS

1. Are all the various types of apparatus necessary for initial attack or initial operations in the municipality available in the station(s)?
2. Are required pumping and laddering capacity plus apparatus for sustained operations available, at least through automatic or quick response on-call mutual aid?
3. Do all response vehicles meet relevant NFPA standards?
4. Are all vehicles maintained so that they are safe, reliable and in good working order?
5. Are reserve vehicles readily available?
6. Is there a reasonable apparatus replacement plan which is honored by the municipality?

TRAINING

1. Are qualified personnel given the responsibility for coordinating and conducting training sessions?
2. Is there at least an annual training plan designed to meet defined department needs?
3. Are individual lesson plans prepared and standardized?
4. Are expert instructors used elsewhere or brought in to conduct highly specialized training?
5. Is the success of training measured by performance standards?
6. Are detailed training records kept for individual personnel, companies and the department as a whole?

PERSONNEL MANAGEMENT

1. Are all aspects of personnel management assigned to trained, experienced and competent individuals?
2. Are detailed personnel records maintained?
3. Is there a relevant and sensible performance-based assessment of each member applied on a regular basis?
4. Has the department prepared internal personnel goals and objectives as well as customer service goals?
5. Is there widespread mutual respect and support among the members?

HEALTH AND SAFETY

1. Are there sound, realistic medical and physical fitness requirements at the entry level?
2. Is there a physical fitness and medical program which continues during the years of membership?
3. Is there a functioning health and safety committee?
4. Is a competent safety officer present at all incidents?
5. Are all necessary safeguards operating for emergency medical and rescue personnel?
6. Are "safe operations" stressed in standard operating procedures, training, emergency operations, critiques and performance assessments?

RESPONSE CAPABILITY

1. Can sufficient personnel and equipment arrive at all high life-hazard targets in time for reasonable rescue operations, as determined by fire officers?
2. Can sufficient personnel and equipment arrive at 90% of the district in time for safe and effective offensive firefighting operations, as determined by fire officers?
3. Is automatic-response mutual aid operating for any areas or selected targets where local initial attack forces are insufficient?
4. Are there any special services such as hazmat mitigation, confined space entry or tanker operations which cannot be provided within a reasonable time frame, as determined by fire officers?
5. Are sufficient provisions made for sustained, long-term emergency operations?
6. Do local officials and citizens know, understand and agree to the strength and capability level of local and mutual aid emergency forces?

INCIDENT MANAGEMENT

1. Is every operating company/ team being supervised by a competent officer?
2. Does each working incident have an overall incident commander, in addition to company officers?
3. Does a recognized incident command system (ICS) operate at working incidents?
4. Is a qualified safety officer operating at incidents?
5. Is radio communication available and properly operating at incidents?
6. Are safe operations as well as effective tactics stressed at incident critiques and fed back into the training program?

DEPARTMENT LEADERSHIP

1. Are department leaders operating effectively at their proper responsibility levels?
2. Are department leaders technically competent for their assignments?
3. Are department leaders sufficiently skilled in personnel management so that task-oriented department members have reasonably high morale?
4. Do department leaders recognize the pressures for change and respond proactively?
5. Do department leaders work to enable every member to operate at full horsepower and develop as future leaders?
6. Do department leaders define power as the ability to do a better job for the customers rather than the ability to make members obey?

Facing Up To Challenges

Knowing where your department stands in its ability to serve its community and its members is a necessary first step in meeting the challenges of today and tomorrow. To quote a former president of AT&T, "When the pace of change outside an organization becomes greater than the pace of change inside it, the end is near."

A proactive assessment can make the difference between a planned future and one determined solely by others. Our world is changing quite rapidly, and fire departments are being changed with it. However you accomplish it, rate your department and take it ahead before it gets done for you.

Dr. John A. Granito is a consultant in fire-rescue services and emergency management, and is principal public safety consultant to MMA Consulting Group Inc., Boston. He has served as safety consultant to the Port of New Orleans since 1992 and has been consultant to the IBM Corp., Strategic Air Command, Agency for International Development, International City Management Association and Federal Emergency Management Agency. Dr. Granito is coordinator of the Urban Fire Forum, composed of chief fire executives from 30 of the largest cities in the U.S., Canada, Great Britain, Germany and Australia. He is chair of the National Fire Protection Association's Technical Committee on the Organization, Deployment and Evaluation of Fire and Emergency Medical Services. Dr. Granito has been supervisor of fire training for New York State and is a member of the International Association of Fire Chiefs. He holds a doctorate in leadership studies and is a professor emeritus and retired as vice president for public service of the State University of New York.