When I was an officer at the fire station nearest the University of Colorado, our station had more after-dark calls than any other station in the city – and many were false fire alarms.
When we set about to reduce the number of false fire alarms, the data initially told us a few things that reinforced our initial impressions: We ran on a lot of false fire alarms; many of them occurred at night; and fraternity houses seemed to account for large portion of them.
The data was accurate, but it only identified the problem. It did not give us the information we needed to truly understand the problem so that we could solve it.
I addressed part of this issue in my recent column “False alarms and fraternities: The importance of solving problems upstream,” including a discussion about what can be gleaned about problem-solving from Dan Heath’s book “Upstream: The Quest to Solve Problems Before They Happen.” Beyond the need to identify the real source of the problem, Heath talks about two kinds of data – data for inspection and data for learning – and how to employ them in problem-solving. Both are important, yet many organizations only go as far as dealing with the first.
Data for inspection
Data for inspection is about counting and measuring things against a standard. How many fire alarms did we run last year? What percentage of emergency responses arrived within five minutes of dispatch? How many fire department vehicles were involved in crashes or collisions in the past year? Are on-duty injuries increasing or declining over the past five years?
Data for inspection provides a quantitative framework for addressing problems. It can lead to the establishment of goals that can be used as standards for measurement. We want to have at least 80% of all emergency responses arriving at scene within five minutes. We want to reduce on-duty injuries by 50% in the next two years.
This kind of data is important for identifying problems, but if it is the only way data is used, it can be difficult to solve problems. Data for inspection only tells you what and how many. It does not go deeper to investigate what factors contribute to the problem – why it might be happening.
Trying to solve problems at this level alone means that an organization is always reacting to things – not “going upstream” – to discern the reason that the problem exists in the first place. This kind of problem-solving can also have some serious unintended consequences.
For example, say a fire department has analyzed data from incident reports and discovered that only 60% of emergency response arrivals were within five minutes. This is obviously a problem, as fires grow exponentially with each additional minute of delay, and the viability of critical patients is likewise negatively affected.
In response to this data analysis, the department sets a clear standard: 90% of emergency responses will occur within the five-minute window. If the response takes longer, the officer must document the reason why it did.
On the surface, this is a worthy goal and will improve service and public safety in the community. But only addressing the problem at this level does not understand it. And it can lead to more serious problems as a result.
Why are responses taking longer? Are crews delaying in the stations? Are traffic patterns or street conditions affecting response times? Is there a disconnect between dispatch and firefighters? All these issues go much deeper and cannot be solved by just telling the crews to be faster in their response.
Also, if there is a clear penalty for a longer response time (and needing to formally document something is almost always seen as a penalty), officers might look for a way to avoid that outcome. Maybe they will push firefighters to get on the rigs before they are fully dressed in their turnout gear, forcing them to continue putting on gear en route. This could lead to diminished seatbelt use and distraction – serious safety risks for firefighters.
An officer might encourage the driver to go faster or not slow down as much at intersections – actions that significantly increase the risk of vehicle accidents while responding. Or the officer might put the engine on scene when still two blocks away, leading to inaccurate reporting and confusion with subsequent-arriving crews.
The way to avoid these negative outcomes is to prioritize data for learning instead of simply for measurement. These two ways of using data are complementary: Data for inspection identifies problems, and data for learning seeks to understand them.
Data for learning
Using data for learning happens at two levels. First, data must be analyzed in depth and from more than one perspective. Other sources of data can be used to cross-reference potential causes of problems.
For example, when investigating longer response times, analyze the data across many different factors: time of day, type of call, area of response, normal response route, etc. This information should be accessible from a standard incident report but is only useful if incident reports are fully and accurately filled out. If crews know that the information they include in reports will be used in a way that could benefit them, they are more likely to commit to a more detailed approach to reporting.
In addition, look for other sources of data: When and where was major construction occurring in the city during that time? Were there any usual events happening? What was the weather like? Go beyond just numbers to look at qualitative factors, such as what is included in the narrative portion of reports.
Then take it a step further. Talk to crews. Find out what factors affect their ability to respond in the most efficient manner. If there is a high level of trust and people can be honest, you might be surprised by what you learn.
For this kind of research to be valid, there must be trust built into the effort. If people feel that they will be punished or ostracized for telling the truth, they will find a way to alter that reality. Gathering data must be seen by everyone as a positive tool rather than as a way of potentially punishing someone.
Open, positive data-collection is key
Good data is necessary for good decisions. When people see that deep, accurate data will help them rather than be used against them, they will they be willing to invest in collecting it.
Only when data is framed as a learning tool can it be used to resolve and prevent problems rather than simply reacting to them and potentially making them worse. The focus on acquiring and using data in an open and positive way is something that no leader can afford to overlook.
Editor’s note: How is your department using data to improve operations? Share in the comments below.
