Author: Sarah Janes – Layer 8
How will you measure the Return on Investment?
It’s a question that perplexes many of us when it comes to security awareness, and a lot of people I speak to struggle to provide an answer that is both clear and concise. I used to be no different. But, because building secure organisations through people is my business, I made it our mission to find a way. At Layer 8, we’re constantly researching, testing and improving the measurement of change.
We all need effective measurement processes that are simple to carry out. But what should we measure? And what does success look like? One answer to these questions is to create people-centred KPIs and here’s how…
5 Steps to creating clear and concise people-centred KPIs
I’m as guilty as the rest for locking myself away somewhere ‘quiet’ when doing something complicated or new, but I’m going to ask you to try something quite different.
STEP 1 – Talk to people. Over the next 2 weeks drop this question into as many conversations as possible:
“Imagine you have been away from our business for 3 years, and when you return our security culture has developed enormously and people are taking responsibility for security like never before. As you walk through the business you start to notice how things have changed. Look at what people are DOING, what’s different about their behaviour? Hear what people are SAYING, what’s different about their conversations?”
STEP 2 – Note down what you hear. Your question is a provocation to imagine improvement, and how people respond to it will indicate their priorities and what they see as the most pressing changes that need to happen. The key words in this question are doing and saying. That’s because culture is a product of what we say and do daily to demonstrate what’s important – and what people do and say is measurable.
STEP 3 – Analyse the answers. If you’ve had plenty of conversations, you’ll have a wealth of info to draw on now. Look for patterns and trends. Are some things repeated? Can you cluster responses from certain locations, roles or levels of seniority?
STEP 4 –Start to write your KPIs. Dominant values and concerns will have emerged, and collectively desired behaviours and standards will be apparent. This is the raw material to formulate your KPIs, and now it’s up to you to find clear, concise ways of expressing them. Remember that each statement you arrive at must be something measurable. I recommend dropping the KPIs into your existing security compliance framework. Then you can see how behaviours and compliance align with each other.
STEP 5 – Identify measures. We can now identify the types of measures we would need: the number of patches applied, whether AV is up to date, click-throughs on phishing emails, for example. If you’re specific with your KPI statements, it’s easier to identify measures, for example, ‘10% or less service downtime through security failure.’
We’ve learnt a few things along the way that will help:
- Be specific – Often, KPIs are not specific enough. We hear things like, ‘We want people to take responsibility for security and do the right thing’. But that statement isn’t granular and doesn’t provide any clues to how it might be measured. How do people show that they have taken responsibility through their conversations and behaviours?
- Align with your business – Some security KPIs are too ‘niche’ and ignore the concerns of the rest of the business. For example: ‘100% of people must go through the security induction CBT’. In this case the CFO might say, “So what! How does that make my business better?” Look at your business’ KPIs; typically, they will include, productivity, quality, profitability, cost saving and growth. A KPI as in step 5, above – ‘10% or less service downtime through security failure’ – is a good example of a KPI aligned to productivity that other key colleagues and executives will buy into.
- Be realistic – Whilst achieving 100% might be optimal, it’s unlikely to be achievable. Set a realistic goal, and when you’ve cracked it, deliver improvements year on year.
- Include rich data – We are programmed to measure using numbers, but sometimes the measure might be qualitative. You might collect stories of change from your security champions, collate process improvements from each department, and gather feedback from security training sessions.
Give it a go! A bonus of this approach – with conversations as your starting point – is that you’re likely to achieve more than a set of useful KPIs; you’ll have started a process of collaboration, developing a shared vision, building relationships and garnering buy-in.
Now help me understand
We’re always curious to find out what you think, so…
How important is measuring your people programme? And, what methods do you use?