What You Measure Will Improve

A concrete measure provides the focus and sense of accomplishment needed to improve continuously.

Last week my head of technical engineering called me after 9 p.m. on a Friday to share an emerging idea for how to cut our AWS costs dramatically. We were both excited by the possibility!

The episode got me thinking about the genesis of his creativity. He is a brilliant technologist who solves complex problems in elegant ways. He is cost-sensitive. He architects solutions that are lean, robust, and resilient. Our current applications are all those things already. So what caused him to propose a significant architecture change?

A few weeks ago, I noted that our AWS costs were growing to a level that required intentional management, so I asked for an “AWS cost estimator.” I wanted a forecasting spreadsheet that would allow us to estimate our costs on a forward-looking basis as we continued to ramp up our usage. Soon we had worked through connecting the dots between AWS’ byzantine pricing models and our anticipated usage. We came up with a way to visualize how our business-critical activities were related to their associated costs. Taking the time to measure and forecast revealed the major cost drivers, and showed how our AWS choices were influencing the bottom-line.  The natural result was to give my head of technical engineering the ability to see the impact of architecture choices. It also gave him a reason to consider technological solutions that are less technically elegant yet could potentially save the business substantial sums.  It also gave him a basis for justifying the risk and cost of making a significant change.

This story illustrates a principle I learned long ago as a management consultant:  what you measure will improve.  Having a specific measurement of something that is happening provides insight. That insight leads to a natural impulse to improve things according to the measure. The AWS cost estimator story is an example of this principle in action. We built the forecast model to measure what was happening with one of our significant costs. The emerging pattern helped us identify optimization opportunities and motivated us to pursue them. 

Here are a few other examples of my experiences with this principle in action:

• Product development progress:  One of the challenges faced by pre-commercial startups is how to communicate progress to your Board of Directors. The tendency is to describe tasks accomplished, but that makes it very difficult for your investors to assess the progress trend. Upon joining a startup developing novel turbine-free wind energy technology as CEO, I worked closely with the technical team to find a measure of performance we could consistently report. We chose watts per meter squared because this measure would reflect progress on the underlying economics of the technology and was independent of the particular prototype embodiment. It became our primary progress metric over several years of development.
  

• Manufacturing quality and cost:  As we were commercializing a sophisticated life science research tool, establishing consistency in manufacturing was a critical challenge. The heart of our instrument was a flow cell where a complex mix of laser beams, lenses, photodetectors, and fluidics all need to align within tolerances measured in microns. Consistently measuring our flow cell manufacturing failure rate drove us to invest in developing a series of progressively more sophisticated measurement tools. These tools gave us visibility into more detailed measurements within the flow cell manufacturing process. That allowed us to consistently make flow cells and reduce waste.
  
  
• Sales incentive plans:  Incentive plans for salespeople are typically structured with specific sales targets within particular periods. Salespeople are some of the very best at optimizing around what their incentive plan measures.  I remember seeing our average selling price being depressed by the combination of number-of-units-sold sales targets combining with end-of-period price discounts. Customers and salespeople quickly learned the pattern, to the detriment of the overall business.  Another example was a new product that did not get enough attention because the sales incentives did not recognize the extra work involved in making the new sale, and it was easier to hit the targets using the established products. One model that did work exceptionally well was giving a pure revenue target to each salesperson that they could meet in a variety of ways. This incented them to get creative with bundling products to meet customer needs as well as add-ons like service contracts that cost us little to support.

Because worthwhile measurements are not always easy to acquire and have such an impact on behavior, one of a startup CEO’s most important jobs is deciding what to measure.  Measurements often require work to design and set up, as well as ongoing monitoring commitments.  Too many measurements create confusion and undermine focus.  What you choose to focus on as the performance measurements for your startup will drive the behavior of your team, so choose with wisdom and intention.