• Edwin and George

Can You Rate People Accurately? – A Short Exercise

In short: No.


Everyone at EGM knows what I think of yearly performance reviews - I've written about it before in Performance Management – and its ‘Very Tricky Problem’. A "rule-based, bureaucratic process existing as an end to itself rather than actually shaping performance" (yes I went to town a little).


Having witnessed bad leaders in large organisations with no leadership skills, qualifications or training rate their people, here's another not-so-fake example that some of you might relate to:



Six team leaders, thirty staff to discuss. How was Sally going to get through this?


The annual talent calibration meeting – HR has a lot to answer for.


"The meeting to determine who gets a pay rise, who gets promoted and who gets left behind".  


Sally looks at her papers. It’s been a tough job rating her team individually against the six corporate behaviors (their "framework"...fancy). She’d listened closely on the course last month, but still found it hard to understand the criteria. It all seemed – well, a bit abstract and ‘hit and miss.’ And then, to average all the scores out...and give each individual an overall rating...Sally dreaded the 1-2-1 meetings next week with her team.


Mike starts proceedings. "I think my team has done really well this year. Most of them will come out as 2s. They’re all well above average. Perhaps Davinia is a 1, but let’s discuss".

"OK" says Alice the senior manager. "I’m worried we won’t have enough 4s or 5s. We need to have some below average performers."


"I think Margaret, Polly and Don are 2s’ says Mike ‘no-one can disagree with that. They’ve come on great in my team. I think Dave is a 2 also, but he’s possibly a weak 2." 

"Is Dave the one who’s always smoking at the back of the building?" asks Brian. 


"Yes’ says Mike. I’ve followed the criteria and he’s coming out as a 2. He scored 2 in each of the categories."


"But you’ve got Dave down as a 2 for business awareness, tenacity and working to our values," says Flo. "Dave worked for me about 5-years ago and I would rate him a 4 in each of those."


"We’re going to run out of 2s at this rate" says Alice. "We need the correct number of 1s, 2s, 3s 4s and 5s otherwise our curve won’t look right and I’II have HR after me. If we have too many 2s, you lot will have to be 3s and 4s so we get the proper shaped curve."


"I would give Dave a 3," says Patricia, "He might or might not be good at his job but I think he has too many smoke breaks. He’s always first out of the car park every day as well. Is Don the nice young man with the grey suit and polished black shoes?" 


"Yes," replies Mike.


"He definitely deserves a 2," says Flo. "He always looks so smart and those shoes are immaculate. I think he gets them from David Jones in Melbourne."


"I agree," says Alice. "A smart appearance should be rewarded. Give Don a 2. I would give Dave a 4. What about Margaret and Polly?"


"Well," says Mike pensively. "I’m not exactly sure about Margaret as her job is so specialist. She always sounds as though she knows what she’s doing and she’s pretty switched on so I thought I’d better rate her 2. She did a great job organizing the office bake -off as well." 


"As for Polly, she does a fabulous job and I don’t know what we’d do without her," continues Mike, "I was thinking of rating her 1. By the way, there’s no truth in the rumors that I heard in the pub last week. I’ve been happily married for many years."


"Give Don a 2, Dave a 4 and Margaret and Polly 3s," says Alice. "We need to move on so give Davinia a 3 as well. Tell her she’s done really well and we’ll be looking out for her next year." 


"Yes but…" says Mike.


"Now Mike," says Alice, you started off with 4 2s and a potential 1. You’ve ended up with 1 rated 2, 3 3s and a 4. That’s really good work Mike – well done."


"Now to you Rita…"


Rita starts. "I think my team have done really well this year. I think most of them will come out as 2’s. Perhaps Laura is a 1 but let’s discuss."


Your Task: 


On the evidence so far, please rate Mike on a scale of 1 (high) and 5 (low) against one of the company competency areas:


‘Influencing and Persuasion’ – ‘the ability to use fact based arguments to get the result you require.’


Correct Answer


Mike was rated as a 1 in all competency areas including ‘Influencing and Persuasion.’ As it turns out, Mike has been happily married for many years, as he says, – to Alice.


The Serious Bit


Research shows that people can’t rate others accurately. (For example, ‘Getting rid of Performance Ratings: Genius or Folly:’ Industrial and Organizational Psychology; University of Cambridge June 2016). 


Over the last 40-years, we have tested and retested the ability of people to rate others and the inescapable truth is humans cannot reliably rate other human beings on anything at all’ (‘9 Lies About Work:’ Marcus Buckingham).


If the judges at the world skating championships can’t give consistent ratings, when all they’re doing is watching one skater after another doing the same routine, what chance do we have in business? 


So why do we attempt to do this?

718 views

© 2020 EGM Partners

Executive Search and Interim Management

Recruitment and OD Services

EGM Recruitment Pty Ltd

ABN: 24613658389

Mail: GPO BOX 2309, Adelaide, 5001

Office location: L4/33 Pirie Street, Adelaide 5000