Over the course of the past two decades, I have participated in, created, and re-created dozens of performance calibration processes. This activity can be simultaneously one of the most dreaded and most valuable exercises a group can go through. And a critical portion of this process is the Performance Calibration meeting - where a set of supervisors come together to discuss the performance of employees and achieve agreement on performance appraisal ratings.
This doc goes through a few different models for running this calibration and some thoughts on when to apply each technique.
Getting Started: Set up Employees and your Calibration Framework
All the processes suggested require starting with a list of employees and a set of ratings:
When a team is small enough to just linearly run through the list, this is a simple framework for calibration. You can filter by level, manager, etc or just go straight through.