what happens when a test company gets it wrong

[Les Bell]

(10-19-2014 03:28 PM)lrdheat Wrote:  Concerning the lion in the cage, I saw the probabilistic way to catch a lion stated as: Place an open cage in lion country. There is a non-zero probability of a lion being in the cage at time >0. Wait!

Which reminds me of an old, old humour piece:

Pachydermic Personnel Prediction

by

Peter C. Olsen

A bold new proposal for matching high-technology people and professions.

Over the years, the problem of finding the right person for the right job has consumed thousands of worker-years of research and millions of dollars in funding. This is particularly true for high-technology organizations where talent is scarce and expensive. Recently, however, years of detailed study by the finest minds in the field of psychoindustrial interpersonnel optimization have resulted in the development of a simple and foolproof test to determine the best match between personality and profession. Now, at last, people can be infallibly assigned to the jobs for which they are truly best suited.

The procedure is simple: Each subject is sent to Africa to hunt elephants. The subsequent elephant-hunting behavior is then categorized by comparison to the classification rules outlined below. The subject should be assigned to the general job classification that best matches the observed behavior.

CLASSIFICATION GUIDELINES

Mathematicians hunt elephants by going to Africa, throwing out everything that is not an elephant, and catching one of whatever is left. Experienced mathematicians will attempt to prove the existence of at least one unique elephant before proceeding to step 1 as a subordinate exercise. Professors of mathematics will prove the existence of at least one unique elephant and then leave the detection and capture of an actual elephant as an exercise for their graduate students.

Computer scientists hunt elephants by exercising Algorithm A:
1. Go to Africa.
2. Start at the Cape of Good Hope.
3. Work northward in an orderly manner, traversing the continent alternately east and west.
4. During each traverse pass,
a. Catch each animal seen.
b. Compare each animal caught to a known elephant.
c. Stop when a match is detected.

Experienced computer programmers modify Algorithm A by placing a known elephant in Cairo to ensure that the algorithm will terminate. Assembly language programmers prefer to execute Algorithm A on their hands and knees.

Engineers hunt elephants by going to Africa, catching gray animals at random, and stopping when any one of them weighs within plus or minus 15 percent of any previously observed elephant.

Economists don't hunt elephants, but they believe that if elephants are paid enough, they will hunt themselves.

Statisticians hunt the first animal they see N times and call it an elephant.

Consultants don't hunt elephants, and many have never hunted anything at all, but they can be hired by the hour to advise those people who do. Operations research consultants can also measure the correlation of hat size and bullet color to the efficiency of elephant-hunting strategies, if someone else will only identify the elephants.

Politicians don't hunt elephants, but they will share the elephants you catch with the people who voted for them.

Lawyers don't hunt elephants, but they do follow the herds around arguing about who owns the droppings. Software lawyers will claim that they own an entire herd based on the look and feel of one dropping.

Vice presidents of engineering, research, and development try hard to hunt elephants, but their staffs are designed to prevent it. When the vice president does get to hunt elephants, the staff will try to ensure that all possible elephants are completely prehunted before the vice president sees them. If the vice president does see a nonprehunted elephant, the staff will (1) compliment the vice president's keen eyesight and (2) enlarge itself to prevent any recurrence.

Senior managers set broad elephant-hunting policy based on the assumption that elephants are just like field mice, but with deeper voices.

Quality assurance inspectors ignore the elephants and look for mistakes the other hunters made when they were packing the jeep.

Salespeople don't hunt elephants but spend their time selling elephants they haven't caught, for delivery two days before the season opens. Software salespeople ship the first thing they catch and write up an invoice for an elephant. Hardware salespeople catch rabbits, paint them gray, and sell them as desktop elephants.

VALIDATION

A validation survey was conducted about these rules. Almost all the people surveyed about these rules were valid. A few were invalid, but they expected to recover soon. Based on the survey, a statistical confidence level was determined. Ninety-five percent of the people surveyed have at least 67 percent confidence in statistics.

ACKNOWLEDGMENT

This study has benefited from the suggestions and observations of many people, all of whom would prefer not to be mentioned by name.