Knowledge work covers a bewildering range of jobs from analysts to artists, money traders to managers. Unlike physical or clerical work, the outcomes of knowledge work are often hard to measure. The measurement challenge is interesting because there’s the possibility of wide variation. In repetitive work, like a cashier, the best performer might be a bit faster than average but a good programmer might be many times faster than an average one. What’s the difference between an average marketer and a great marketer? Between an average oncologist and the best oncologist?
Knowledge worker performance is almost always assessed using the standard performance management paradigm of setting objectives and having the manager make a judgement of how well the person did relative to those objectives.
Narrowing the problem of measurement
We have to go back to the fundamentals of HR analytics to approach the problem of measuring knowledge work. The first step is to recognize that we need to narrow down the analysis. As we said, there are a bewildering range of jobs that consist mainly of knowledge work. There’s no universal solution, so we have to address what’s most important.
What business decisions are you trying to make?
Perhaps the main decision is this whether to make a disproportionate investment in talent because of the high value of getting the best knowledge workers. For example, if we learn that our best business analysts are twenty times more valuable than average ones, it would make sense to invest a great deal of effort in recruiting and retaining the best ones. You might put aside compensation dollars to create special deals for the best analysts and, instead of posting the job on a job board, hire a specialist search firm. Compensation and cost of hire may skyrocket, but that’s a smart investment if it brings in high-performing analysts.
Identifying the knowledge work that matters
How do you identify these jobs where the variation in performance is so wide that it justifies special treatment? You can go a long way by determining what jobs have this pivotal nature. It’s important to push back against cavalier or ill-thought out answers as managers give input through this process.
Managers need appropriate justification for their views on positions that have great variation in performance. Beyond that, they need to articulate the cause of the variation. These decisions aren’t always easy, but having a comprehensive understanding of the roles with varying performance will leave you confident that you’ve found the jobs where the variation is big enough to really matter.
When you do ask the manager about the cause, look for one of three types of answers:
- The cause is a variation in natural talent. For example, the best marketers are simply far better than average. The word “genius” may be used.
- The cause of variation is a winner-takes-all effect. In popular music, if you hit the top ten, you’ll sell far more records than someone at number 11, even if the song is not noticeably better. It’s a winner-takes-all world in music. This effect isn’t common in office jobs, but you might find cases where a job needs someone who knows everyone in the industry and there are only a few such people.
- The cause of variation high leverage in the job. In software projects, the business analyst can be a bottleneck because they translate user needs into the specifications of what needs to be developed. They don’t need to be a genius. If they’re even moderately more skilled than average their impact on how quickly the project is delivered may be huge because they speedily answer questions without having to go back to the user for clarification.
Doing this analysis doesn’t solve the general problem of measuring knowledge work, but we’ve done something more important: We’ve identified those jobs where HR efforts will make a big difference to organizational performance.
Measurement using a mosaic
Once you’ve identified the knowledge workers that matter most – the pivotal roles – you’ll want to get more accurate in assessing performance. The first step is to start gathering a mosaic of measures. Depending on the job, you might find that measuring performance is straightforward, but more likely you’ll need a whole range of measures to get a handle on performance. For example, with a business analyst you might look at feedback from the users requesting the software, feedback from the programmers writing the software, and data on whether milestones are reached on time.
After gathering the mosaic of measures, it will take judgment to decide how big the difference is between average and the best. We want to know this difference because it will guide how much we’ll be willing to invest to recruit and retain high-performers. We can make these judgment calls by:
- Getting a range of opinions. Many perspectives are better than one.
- Using a structured format to guide the assessment. This is the idea behind doing structured interviews: List the factors you want to consider rather than asking for an overall assessment.
- Making reviewers defend their assessment. When people know they’ll need to explain the reasons behind their assessment, they make better decisions.
Turning your analysis into action
Once you've identified the pivotal jobs:
- Design special reward deals for the top performers
- Work with talent acquisition to determine what it will take to attract the best candidates
- Use what you’ve learned about high performers to improve development
We started this post with a difficult and broad question about knowledge work. We made progress by narrowing the problem, drawing on the concept of pivotal jobs. The two main ideas we landed on, the mosaic of measures and reducing bias in making judgments, are relevant to many jobs. However, we still need to focus our efforts on the knowledge work where variation is very high: That’s what will create the biggest business impact.