Key Performance Indicators in IT – part 1

I was asked recently by a company as part of their first-round hiring process a question very similar to this:

How would you establish measurement of the departments under you in such a way that you would have an indication that they are operating well?  What would these things mean?  How would you go about setting them up?

Well, first off:  I was surprised.  Not at the nature of the question, but that I was being asked it at all.  It’s the first time in over 10 years that someone has broached the topic of Key Performance Indicators (notoriously known as KPIs in business-speak) and actually understood what they meant.  Most companies understand that there are metrics one can use to establish the overall health of a company or department – and that they are wildly different from department to department to company.

Some just don’t get it at all.  As an example, at the company I am in the process of leaving, our MD thinks that a KPI is a to-do list.  Yes, he’s that far away from knowing how to run a business.  Which might be one of the reasons why it went down… but that’s another story.

First, it deserves note – why do KPIs matter?  The whole point in measuring a KPI is to identify critical factors in your business or department and watch them with the intent of fixing them when they show times of trouble, or improving them to better your unit’s output.  Distilling this concept:  KPIs provide visibility into the heartbeat of the business unit.  Many people (myself included) don’t look at a spreadsheet and say “Oh, I see, got it” right off the bat (in my defense, it takes a few minutes 🙂 ).  But if you can show someone a graph with bars or lines on it?  Click.  They get it.  This is especially important for board members or executive meetings – not because they won’t “get” the excel sheet (most will, but some wont), but because they don’t have time to dick around pretending they care about the nitty-gritty details of who made what sale or which developer is pumping out the most code with fewest bugs.  They are being paid to handle a much broader picture, and wasting their time means wasting company money (and it also means probably pissing them off, which will result in first their asking for your excel slide to be removed from the deck, and later probably asking for you to follow your excel slide).

I’m going to hit up a few different sides of the KPI angle, aimed at different departments within IT – and will probably do an overview of one at a company-level as well, in what is to become my first series article here.  Before I begin though, it deserves a little definition of what a KPI really, truly is.

Let’s start by breaking down its name:  KPI, is Key Performance Indicator.

KEY:  the item, whatever it is, represents a key portion of your business.  Without it, things don’t work.  It’s like your car – if you don’t have the key, you’re screwed.  You can get by on three wheels, or no seats, or broken windows, for a little while and still get where you’re going.  But if you don’t have the key, you aren’t going anywhere (notwithstanding the smart-ass “hot-wiring” comment that will no doubt follow, stretching my metaphor too far).

PERFORMANCE:  performance represents accomplishment.  Miles per gallon for your car.  Consistent placement of a round from a pistol or rifle.  The oven produces good heat per unit of gas or electricity given to it.

INDICATOR:  a metric used to measure something.  An indicator light on the dashboard of your car, the CPU meter on some of your screens right now, the thermometer outside your window.  These are all examples of indicators.

So…when you put it all together, you get an indicator, a measuring stick, of the performance of an item that your department or business can’t succeed without.

There will only be a few items that are “Key” – not everything can be key, because when everything is critical, nothing is.  Same thing goes for problems and bugs and features (as some readers from past companies will remember me saying this), and it’s worth making an axiom out of.  In fact, I’m quite certain other people have said it already, but it bears being made into a T-Shirt:

WHEN EVERYTHING IS A CRITICAL EMERGENCY, THEN NOTHING IS.

Some time I’ll go over to Cafe Press and put some shirts together.

I’ve worked at two firms (it could count as three, but I went to work for one of them twice) where I came in the door to a fire-fighting culture that viewed every single issue as a critical make-or-break.  The situations were each different, but bore resemblance to one another in that they were each driven by some form of “Oh my god, we’ve got a bug, if the customer doesn’t get a fix by five pee-emm then the sky is going to rain menstrual blood all over us!!” and of course the development staff was tied up doing nothing but fix-fix-fix, and every fix for one customer was considered inconsequential or a nuisance to others, where later it would become yet another hellfire-and-brimstone case.

In their defense, it’s very easy to end up with a bad case of tunnel-vision when that happens.  As an outsider coming in, I could see very clearly that they weren’t prioritizing and identifying what was good for their product.  But inside?  They were so caught up in the day-to-day (and at least one person in each case had made a career out of creating the spaghetti solutions which added more fuel to the fear-mongering fire – which that person used as job security), they couldn’t see an end to the tunnel. Breaking out of that long enough to see how to fix it is hard.  Which is why the firefighting culture is often fatal for a company.

So KPIs are a very few things that actually are critical.  For example, if a company doesn’t sell anything and makes no money, it will die (because eventually investors are going to tell the company to go screw itself).  So revenue generation would be a good target to choose for a company KPI.

They’re also indicators.  As such, they must be measurable.  That’s not an option.  I repeat:  they must be measurable. If you can’t measure it, it’s not an indicator, and it’s out the window.

And they measure performance.  Performance can be tuned.  That’s also important – if you’re measuring something you have no control over, it’s not a KPI.  That doesn’t mean it isn’t worth measuring – many factors over which we have no control exert influence over our businesses.  It just means it’s not a KPI.

Let’s sum that up now:

A KPI is a measuring stick used to monitor one of a few critical aspects of the health of your business unit with the intent of proactively identifying problems or aiding in the improvement of your unit.

I’ll crank out another article on this in a day or five, I’ll hit up measurement of KPIs as they apply to a Tech Support department.  I’ll probably add a few items that it will have in common with other call-center type environments as well.

Update:  I forgot to mention something that KPIs are not:  immediate.  The time frame over which KPIs show change is not daily.  It is at best monthly – and more likely quarterly.  Most data used by KPIs is strategic – this is not a tactical tool.  You may observe a shift over a one-week span, but at the rate of weeks, such shifts are indistinguishable from normal noise.  If you see a shift that looks alarming, by all means pay close attention to the situation, but unless it repeats or worsens over coming weeks it should not be taken as an impactor of the KPI.

Use of a KPI should produce a trending line, which is what gives you something to bring action to.  Spikes and troughs in the data are to be expected, and can usually be explained by occurrences that have specific causes and durations (i.e., “Call wait time spiked there because we had two members of the team resign while three more were on holiday – so we were temporarily understaffed.”)

Update: part II can be found here