Resources How-to's How to set up an effective 360 degree multirater evaluation process


How to set up an effective 360 degree multirater evaluation process

Planning to introduce a 360 degree multirater evaluation process in your organization? That's great! Gathering 360 degree multirater feedback can be one of the most effective ways to ensure employees get broader, fairer feedback that supports improved performance and ongoing development. But to get the desired results from your process, you need to design it properly from the start. To do that you need to ask and answer some fundamental questions.

Why do you want to gather 360 degree multirater feedback?

There are three common reasons companies choose to gather 360 degree multirater feedback:

  1. To gain insight into the performance and potential of current and future leaders
  2. To gain broader insight into the development needs of employees
  3. To gather broader feedback for performance appraisals, helping ensure their fairness, especially where the manager does not have direct, firsthand knowledge of their employees' performance

Your reason for gathering multirater feedback will greatly influence the design of your program.

Companies that have employees who work for remote managers, who work for several managers, who work different shifts than their manager or who work on project teams often find they need to gather multirater feedback as input for their performance appraisal process in order to give their managers the insight they need into employee performance. Whenever an employee's manager is not in a position to directly observe the employee's performance, 360 degree evaluations help make performance appraisals fairer, and the feedback given to employees more comprehensive and helpful.

In making the decision why, it's important to consider organizational needs, as well as organizational culture. A company with a more collaborative, collegial and supportive culture may well benefit from gathering multirater feedback for performance appraisals, while one that is highly competitive and individualistic might risk doing harm and further polarizing their workforce. These companies should consider starting with a development focus.

It's important to be clear on your purpose and to communicate it to all involved in order to ensure effective participation and engagement with the results.

What is a 360-degree evaluation?

A 360-degree evaluation is an assessment tool that provides employees with the opportunity to receive performance feedback from supervisors, peers, co-workers and also external parties such as suppliers and customers. The 360-degree evaluation process also enables organizations to gain insight into the performance and potential of future leaders, and to determine the development needs of employees.

Who should be involved?

This is actually a two part question: who should you gather feedback on, and who should you gather feedback from? The answers depend on your reason for conducting 360 degree multirater evaluations and your work culture.

If you're trying to gain insight into the performance and potential of current and future leaders, you'll first want to determine which levels of management to target. Are you only interested in assessing and developing your C-level executives or your entire management team? Do you have an effective means of identifying high-potential employees and assessing them as part of their grooming/development for leadership or management positions? Next you need to decide who to include as assessors. One important factor to consider is whether you want to gather "upward" feedback, that is, feedback from a manager's direct and/or indirect reports. Potential assessors include peers, suppliers, customers, managers and staff.

If you are trying to gain broader insight into the development needs of your employees, you'll likely want to conduct 360 degree multirater evaluations on all your employees. Some organizations aim to gather feedback on just a select group, such as their top performers, lower performers, or employees who work in high impact areas. Here again you'll also need to decide who it's most appropriate to gather feedback from.

When deciding who to gather feedback from, you also need to consider:

  1. What is an optimal and/or practical number of assessors? The answer will depend on your organization size and structure, the purpose of the evaluations, and who you've chosen to allow as assessors. Some studies find 8-15 assessors an optimal number for development focused evaluations. If you're using 360 degree evaluations to get broader feedback for performance appraisals, you'll likely need fewer assessors. The larger the number of assessors you use, the less likely they may be to have worked directly with the employee being assessed.
  2. Do you want to establish a minimum number of required assessors? If anonymity is important, you need to ensure you have more than 5 assessors for each employee being assessed. You might also want to consider whether you want to establish requirements for the type of assessor. For example, you might require feedback from at least 1 customer, one supplier and one manager.
  3. Do you want to establish a maximum number of required assessors? More is not always better. You may gather so much feedback that the information becomes overwhelming to deal with or too diluted to be meaningful.
  4. Who will select the assessors? If you let employees choose their own assessors, they're more likely to take the feedback more seriously. However, they are also likely to choose assessors with whom they have a good or favorable working relationship. It may be ideal to have the employee and their manager or HR choose the assessors together to ensure more objective selection.
  5. Will the feedback be kept anonymous? Most programs keep the feedback anonymous, especially to the employee who is being assessed. It might also be important to keep the feedback anonymous to the manager or coach who'll be reviewing the feedback with the employee if you follow this practice. Anonymous feedback tends to be more honest, because the assessor does not stand to suffer or benefit from the feedback. You do need to be careful though as some employees use the protection of anonymity to sabotage others. Here again, you need to consider your corporate culture to determine the best approach. If you are soliciting "upward" feedback from employees on their manager, it's imperative to ensure anonymity in order to be able to gather honest feedback and protect employees from any repercussions. Bear in mind, maintaining anonymity can be more difficult if you are gathering qualitative evaluations.

Once you've decided who will be involved, you need to ensure all participants know why this is being done, what their role is and how the information will be used, today and going forward. You also need to make sure that those providing feedback have some training in that skill.

What aspects of performance will be assessed?

Here again, the answer depends on your purpose for conducting 360 degree multirater evaluations.

If you're using them to identify leadership or general development needs, you normally just solicit feedback on competencies. You would typically use the same core and/or leadership competencies assessed as part of your performance appraisal process. Some organizations who are conducting 360 degree multi rater evaluations for leadership development purposes elect to assess a different or broader set of leadership skills. The risk with this approach is you may dilute your performance management efforts by trying to assess and develop too many competencies at once.

If you're using 360 degree multirater feedback to gather broader feedback for your employee performance appraisal process, you should be evaluating the same set of competencies with both processes. Some organizations also choose to gather feedback on the performance of goals - especially when the manager does not work closely with the employee.

When gathering feedback on competencies and/or goals, you need to decide if you want to gather quantitative feedback, qualitative feedback, or both. Gathering quantitative feedback only makes it easier to maintain the anonymity of assessors, but can fail to deliver the richer qualitative details that explain the reason for the quantitative ratings. You can decide separately if the employee being assessed will be allowed to see the comments/ratings, or only their manager or HR, and also whether they can see individual comments/ratings or only aggregates.

If you're conducting 360 degree multirater evaluations to gain a broader perspective for employee performance appraisals, it's important to use the same rating scale as you do on your performance appraisal forms. If you're using them to identify development needs or for career development or succession planning/talent assessment purposes, you might want to consider whether you want to avoid neutral ratings by choosing an even numbered scale (e.g. 4-point). You should also consider whether you want to allow a Not Applicable (N/A) or Insufficient Knowledge option.

When should you conduct 360 degree multirater assessments?

If you're conducting your 360 degree multirater evaluations to assess the development needs of current or potential leaders, or to determine development needs for your employees, you can conduct the evaluations at any time. However, you might want to gather the feedback either shortly before, or after your performance appraisal process so that you couple the associated development planning with your performance appraisals.

With development focused multirater evaluations, you also need to consider how often to reassess staff. You'll only reap the benefits of your 360 degree evaluations if you include follow up development planning and regular reevaluations as part of the process to ensure skill gaps are addressed and development is occurring.

If you are gathering 360 degree multirater feedback to provide broader feedback for performance appraisals, you simply need to allow enough time for participants to complete their evaluations in time for managers to use them as part of their employee performance appraisals.

How will you deal with the results?

It's also critical to think ahead about how you will deal with the results. In this regard, you need to decide:

  • Who will receive the 360 degree multirater feedback?
  • Who will deliver the feedback to the employee (if they do not receive the feedback directly)?
  • Will the feedback from individual assessors be viewable and by whom?
  • Will the feedback be aggregated and viewable by whom?
  • Do you need the ability to sort the results and analyze trends?
  • What follow-up or actions plans will be put in place to deal with the results?

When, where and how the 360 degree multirater feedback gets communicated to the affected employee is an important consideration. Depending on the purpose of your program, and your organizational culture, you might find it better to have the employee's manager, a coach or an HR representative present the results to the employee and help them interpret them.

  • Some programs simply aggregate the feedback, remove any assessor identification and deliver it to the employee in the form of a report.
  • If you are conducting 360 degree evaluations as part of a leadership development initiative, typically the candidate requests the evaluation and HR runs the process then delivers the results and required coaching to the employee.
  • If you are using 360 degree multirater feedback for development planning only, you should ensure that the results are not viewed before or even consulted during the performance appraisal since they might skew the managers' perceptions/ratings.

Human nature makes it difficult to receive negative feedback. Employees often need help to interpret the feedback they are given and see it in a positive light. They also often need help to consider personality type and social styles when interpreting feedback. How we perceive others is largely a measure of who we are. Assessors may in part be reacting to differences in personality and social style in providing ratings. This needs to be taken into account when considering feedback and factored in to any associated development planning.

Finally, it's vital to set up a process for taking action based on the feedback, and following up to ensure actions have been taken and have been effective in impacting performance. Without this follow through, the process of collecting 360 degree multirater feedback becomes meaningless.

How will you administer the process?

One vital decision you need to make is how you will administer your 360 degree multirater feedback process. Though seen as a tremendously valuable tool, many organizations shy away from conducting 360 degree evaluations because of the paperwork and administration involved in collecting and collating all the feedback.

Automating the process eliminates the hassles for everyone involved, saves tremendous time, energy and effort, and lets you extract all the value from the feedback.

Read how others have implemented successful 360 degree multirater evaluation processes

At award-winning CPA and consulting firm Clark Nuber, employees typically work for multiple managers in the course of a year, making it hard to conduct fair and accurate performance appraisals. Now they can easily gather feedback and seamlessly incorporate it into review documents, giving employees a complete and accurate picture of their performance.

Success with Talent Management made easy. Experience it for yourself.

Get Started