Running a proof of value to test a change makes perfect sense - just avoid the bear traps.

Proof of (Limited) Value

“Half a league, half a league,
Half a league onward,
All in the valley of Death
Rode the six hundred.
“Forward, the Light Brigade!
Charge for the guns!” he said.
Into the valley of Death
Rode the six hundred.”

The Charge of the Light Brigade by Alfred, Lord Tennyson

While the consequences of a typical process change cannot be compared to the suffering that ensued from the Charge of the Light Brigade, anyone who has been hurled headlong into a change without running a limited trial beforehand may be forgiven for thinking that they had been sent into Tennyson’s valley of death.

A proof of value (PoV), proof of concept (PoC), Test & Learn or whatever you may choose to call it, all have their roots in Deming’s PDCA cycle {Plan, Do, Check, Act). In essence:

  1. Define and understand the problem you are trying to solve.
  2. Design the counter measures to address the root cause of the problem.
  3. Test your countermeasures and hypotheses in a limited way.
  4. Study the results. If in line with expectations, roll out the plan; if not, go back to the start.

The obvious benefits include:

  • reduced implementation risk
  • lower cost of innovation
  • less waste in the innovation and improvement process
  • better outcomes for customers, shareholders and staff.

But at a more fundamental level, PDCA is a system for instilling a problem-solving mindset, a critical ingredient to building a workforce capable of continuous improvement and innovation. Who wouldn’t want a self-directed workforce, capable of solving problems and continuously innovating with limited risk of blowing up the farm?

As with all things, the devil is in the detail. PoVs have their own risks and can lead you to conclude that:

  1. You should proceed, when in fact you shouldn’t.
  2. You should stop or go back to the planning stage, when in fact you should press ahead.

To reduce these risks, you need to be very clear about what you are trying to prove i.e. the way you design the experiment matters!

If you want to test whether a specific process change reduces the error rate, there is no point in just asking the best performer in the team, who rarely produces errors to test the new process. A representative cross-section will give you a far better indication of the likely outcome of rolling the change out across the entire process.

For most types of change a team would like to test, the design considerations are relatively simple. However, there are a couple of service scenarios to be mindful of, where the design of the PoV is anything but straightforward.

Three specific examples spring to mind:

  1. Multi-Skilled v Specialist Workforce: if every staff member in an organisation could perform every task, the resulting flexibility and capacity release potential would be enormous. Intuitively, however, we know it is a fool’s errand to try and train everyone to do everything. The human memory has its limitations. So how much cross-skilling can we do? The proof point may take many months to present itself. You need to gather evidence of the quality and productivity outcomes from infrequently performed tasks that adequately reflect the consequence of people either failing to remember, partially remembering or remembering incorrectly. The results of a three-month proof of concept will not adequately correlate with the real outcome over an extended period of time.
  2. Scale Effects: in many cases it is one thing to prove a case “in the lab”, it’s another to industrialise the process and run it at scale. The unit cost produced in the PoV will only ever reflect the process under those conditions. Running it at scale may introduce new costs e.g. communication, duplication and office politics costs i.e. diseconomies of scale. These cannot be tested in a limited “live” pilot.
  3. Hawthorne Effect: the behaviour of the team members in the trial will be different to what you can expect to see “in the wild”, merely because their behaviour is being observed. In addition, the people participating in the trial are usually the best performers with a deep understanding of the problem to be solved – clearly not representative of running in business-as-usual mode.

So, think very carefully about how you design your PoV. For certain types of solutions, it is extraordinarily complex to design a “live” trial. These are better suited to simulation modelling, where the longer timeframe or distribution patterns of behaviour are easier to replicate. Other options are to “chunk” the problem down into smaller parts, which are easier to run through limited “live” trial.

There is no such thing as perfect information and it is both technically and economically impossible to test for all eventualities. However, a well thought through and designed PoV is a powerful addition to any leader’s toolkit. It’s worth remembering that “one small step for man, one giant leap for mankind.” was in fact just a series of PoVs – a perfect example of continuous improvement and innovation in action, with each stage designed to test a specific, limited set of parameters.

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkdIn
Share on pinterest
Share on Pinterest

Leave a comment

How to transform your business

Hire external or train existing staff? This is one of the major dilemmas facing any mature organisation embarking on digital transformation. With the odds of success so low, the stakes are high.

What’s the key to successful digital transformation?

This people-oriented approach makes transformation more achievable and delivers benefits along the way.

The importance of day-to-day delivery teams to fund digital transformation

Digital Transformation is hard. Bringing the day-to-day delivery teams into the transformational tent can play an important role in helping fund the transformation and increase the odds of success.