Performance Based Budgeting workshop at the Texas Municipal League in Austin, February 26. Watch here for more information.

Performance Measures

November 2009

Here comes the hard part. I now have to come up with relevant and measurable performance measures to make sure I am on track with my goals.

Most performance measure professionals will tell you that metrics can be process-based or outcome-based. Process-based metrics include input, efficiency, and output metrics. Outcome-based metrics talk about the ultimate positive results of conducting a process.

For a quick example, if I was trying to measure the success of a county clinic’s vaccination program, I could look at the process:

• Input—
• how many people are eligible for services
• how many nurses are on staff
• how many hours the clinic is open
• Efficiency—
• how long—on average—it takes to do each vaccination
• how much—on average—a vaccination costs
• Output—
• how many children are vaccinated

Or to quote from the performance audit standards of the GAO’s Yellow Book:

7.15d. Efforts: Efforts are the amount of resources (in terms of money, material, personnel, etc.) that are put into a program. These resources may come from within or outside the entity operating the program. Measures of efforts can have a number of dimensions, such as cost, timing, and quality. Examples of measures of efforts are dollars spent, employee-hours expended, and square feet of building space.

7.15 f. Outputs: Outputs represent the quantity of goods or services produced by a program. For example, an output measure for a job training program could be the number of persons completing training, and an output measure for an aviation safety inspection program could be the number of safety inspections completed.

7.15 g. Outcomes: Outcomes are accomplishments or results of a program. For example, an outcome measure for a job training program could be the percentage of trained persons obtaining a job and still in the work place after a specified period of time. An example of an outcome measure for an aviation safety inspection program could be the percentage reduction in safety problems found in subsequent inspections or the percentage of problems deemed corrected in follow-up inspections. Such outcome measures show the progress made in achieving the stated program purpose of helping unemployable citizens obtain and retain jobs, and improving the safety of aviation operations. Outcomes may be influenced by cultural, economic, physical, or technological factors outside the program. Auditors may use approaches drawn from other disciplines, such as program evaluation, to isolate the effects of the program from these other influences. Outcomes also include unexpected and/or unintentional effects of a program, both positive and negative.

So, the GAO calls input and efficiency metrics “effort metrics.” Nice and simple.
For further explanation of this family of metrics, please see my newsletter dated January 2006.

OK, those all seem fine. But why bother with a process, anyway? You are trying to accomplish something, right? You are trying to keep children healthy. And as a byproduct of keeping children healthy, you are keeping the community healthy.

So an outcome measures would sound something like:

• reduction in outbreak of contagious disease
• reduction in number of school days missed due to illness

The balanced scorecard has room for all of these types of metrics. The process metrics are best applied down at the internal business process level and the outcome metrics are best measured at the customer and finance level. Theoretically, happy financial metrics and happy customer metrics result from having the internal business processes tied down.

Relevant performance measures

If I told you that I created two new products last year, would you be impressed? It is hard to determine if this is a good number without knowing other information, right? You’d want to know how long it takes to create a product, how much it costs to create a product, and whether the products are actually selling, wouldn’t you? And that is just the beginning of the questions you might have.

The State of Texas ran into this problem in reporting performance measures to the legislature. The agencies would tell the legislature, “We served 3,500 citizens this year.” And the legislature replied, “Is this enough? Is this number good or bad?”

These kinds of metrics—ones where you just count how many you finished—are known as OUTPUT metrics. They are easy to collect and measure so they are the go-to metric for most managers. But in order to appreciate the output, you also need to know how much effort went into creating the output and whether the effort was worth it.

Let’s take an example from my product development processes listed in last month’s newsletter:

1. Product development

• Idea generation
• Learning objective development
• Drafting
• Editing for technical correctness
• Editing for logic
• Edit for readability
• Edit for grammar
• Format

You see that creating a new product takes much more than just typing it up and sending it out there. I am very good at idea generation. That would be an INPUT to the process, wouldn’t it?

And the objective development and drafting isn’t hard; it just takes a while. Depending on what else is on my plate, it might take me six months. The length of the drafting process could be an EFFICIENCY metric.

And now, the real cost and time begin. I am no good at editing my own work (I always think it looks fine, and it doesn’t!) so I hire others to do it for me. But, I have found out that there are different types of editors—technical editors (to make sure I’m not lying to my professional audience), logic editors, and grammar editors. And that one person cannot and should not do it all. Someone who is great at catching grammar errors is generally not that great at logic editing and vice versa. And each of these editors takes time and costs money. Again, EFFICIENCY metrics.

Again, the OUTPUT metric is the number of products created each month or quarter.

And then for the real point of doing all of this work: does it sell? We won’t know that until it has been out on the market for a few months or maybe a few years. Whether the product sells in accordance with my expectations would be an OUTCOME metric.

I am tired from just thinking about all of this, much less actually measuring it in real life. I can think of dozens of metrics for these processes and dozens of desired outcomes, but only some of them will resonate with me as being relevant and interesting. I would like to know all of the information above.
Consider another area from my internal business processes list from last month:

• Accounting
• Output—number of royalty reports sent to authors on time
• Output—number of financial statements generated
• Input—number of transactions to account for each month
• Output—percentage of transactions properly classified

As a finance weenie, I could go on and on and on with happy accounting metrics, but I only feel any pain or concern around the first one. The rest of these metrics are horribly boring. I’ve been working with my bookkeeper for years now, and she and I have a way to correct misclassifications. It isn’t worth my effort to hold her to a metric counting the number of misclassifications she inputs.

Was it beneficial for me to think through the metrics, the goals, and the processes? You bet. Will it be worth it for me to track my progress regarding these metrics? I guess that depends on how complex things get. The bigger and less centralized the business becomes, the more important metrics will be for making sure everything is on track.

Since I haven’t even sold my first book on Yellowbook-cpe, I don’t have a performance measure czar on the payroll yet. The frequency and accuracy of information costs time and money.

For those of you who are working for large organizations, I encourage you to dedicate some resources to tracking at least a few key metrics. They go a long way to enhancing transparency and accountability. So next month, let’s discuss what qualities a meaningful metric possesses.

-----

Leita Hart-Fanta, CPA, CGFM
Resides in Austin, Texas and can be reached at www.happycashflow.com