More on Performance Evaluation

Note: this was originally a comment in a LinkedIn Discussion. The question was "Two of the problems I currently face: 1. If work standards and numerical goals get eliminated, we have to establish a new performance monitoring system. How do we evaluate performance fairly ? Leadership is hard to measure. 2. How do we design a fair compensation / reward system after that? Any suggestion?"

I think we all wrote reams on this in the DEN several years ago.
The problems are many, but mostly they have to do with an understanding of what it means to measure, and what it means to know. I had a conversation with a young HR person a couple of years ago about the criteria they were using to put people into a training program. I asked whether the peoples' managers actually knew them well enough to be able to predict whether an individual would be successful in the program. She said, "Well, I hope they are using performance evaluations to select them...you know, some objective data instead of just a manager's opinion." I asked her where they got the "objective data" from, and (of course) she told me they came from scores and rankings assigned by the managers. Then I asked, "How is that different from manager opinions?" She looked at me as though I were an alien or an idiot or both, and said, "They assign NUMBERS!"
It's always dangerous to try to make an inherently subjective task objective, but in this case, it's impossible. Anyone with a modicum of understanding of General Systems Theory knows that you can't separate a person acting in a system's performance from the performance of the system. It's one of the primary lessons from the Red Bead. Deming usually pointed out that it's trying to solve an equation with two unknowns.
It also ignores variation theory, although in many cases, the schemes put forward by management attempt to use variation theory as an excuse for some of the bad practices. In any group of people, for any given measure, there will be a distribution of performance. Because of system effects and interactions, the distribution is mostly the result of random variation. Some people will be at the higher end in some years and the lower end in other years, due to random variation.
Some people may end up in the upper tail of the distribution, more than three sigma away from the average. They may even stay there for a few cycles. These are people who are doing better than the system, and should be studied to find out how the system could be improved. They are the ones who should probably be rewarded more. If you asked everyone in the organization "who's our number one person?" all fingers would point at that person.
Some people might end up in the lower end of the distribution...they have managed to underperform the system. If you asked anyone who should go, they would point at that person. Those people are in need of help...maybe a different job in the same organization, maybe some training or a different manager, maybe a job in another organization.
Those people in the tails are relatively easy to identify. They are also rare. Most of the rest of the people are doing their best, and their performance is a result of random variaiton. Rating and ranking them is a step away from reality, and can't be done on any sound rational basis.
If you want to try some strategies that make sense, add "Abolishing Performance Appraisals," by Tom Coens and Mary Jenkins, to your reading list. They provide a very comprehensive treatment of the downsides, but also offer some great suggestions for "what to do instead."