How numbers can make you make mistakes

When I've moved into new jobs, positions or projects, one of the first things I'll do is look to see what data there is about the process and what people are doing with it. Time and again it amazes me what huge amounts of data there is - or the complete lack of it. Having lots of data is not always good because the majority of it is completely useless and the rest is never looked at.
People have an obsession with numbers and apply a lot of significance to them (IE baseball stats) without thinking through what the numbers are truly telling us. The three biggest sins when measuring and analyzing numbers would be: a) they don't measure anything controllable, b) the measure at the wrong frequency (too often or not often enough) and c) they measure the wrong step of the process.
Measuring too often or not often enough can be quite a significant factor but focus on only that particular point of the process. The two other sins have a tremendous impact on the total process and on the business itself if the measurements are used to make significant business decisions. Again, those two sins are measuring the wrong step in the process and measuring uncontrollable factors.
Imagine a company making widgets. The company has been doing well and they know this because they look at their sales numbers on a monthly basis. One month, the sales drop. Second month, they drop as well and phone calls by Sales to customers aren't being returned. The owner is furious and fires the Sales Manager.
The new Sales Manager comes in but sales continue to fall. She gets smart though and starts looking for other measures. Hmm, complaints about the number of defective parts being received has escalated in the past 6 months. Speaking to the plant manager, who in only measured on number of cases leaving the plant, reveals that an old machine was put back on the line about 7 months ago so that demand for product could be met.
Together, they look at the old machine and start to measure the number of defective parts making it into the cases. Turns out, there is an 80% defective rate on a machine that produces 1/3 of all production. Over 26% of the product shipped from the plant to customers is defective.
The Sales Manager does not have any control over the quality of the product, which is getting customers upset and causing them not to purchase product. Yet, she is being measured on it via sales numbers. The Plant Manager is being measured on how much product is going out the door, not on how good that product is - measuring the wrong point in the process and after the fact.

Another way to look at is the dieter. They want to be healthier and figure if they measure their weight everyday, they'll know how healthy they are. Between the error inherent in the scale and the natural daily variation in weight, they won't get the true story. Plus, as they exercise more and make better nutrional choices, the fat will decrease and the heavier, more dense muscle will increase, potentially causing an increase in actual weight. BUT, the person is actually healthier and if they measured the type of food they were eating, how long they can now run for and how long they can hold the plank position, they would know that they are healthier! Especially as the jeans wouldn't be so tight! A much better, truer measure!