For a test that has a large enough number of test takers, the marks distribution will be a normal curve as shown.

Tests fail when:

1. The methodology used to identify the right candidates is misplaced

   A real story: When Sunitha’s parents were visiting her in Bangalore, she wanted a driver who could show them around the city. She hired one after asking him these questions: “Do you know the route to MG Road , Lalbagh and Forum?” and “Do you have a valid driver's license?” Both of which criteria, of course, the driver satisfied. The second day on the job, he met with a road accident injuring both parents. A classic example of evaluation gone wrong; it tested the stated objective but was not completely relevant. Most tests created “in-house” are ad-hoc. The creator of the test tries is usually always looking to come up with 'clever' or 'difficult' questions rather than trying to identify the skills that need to be tested by a question.

2. 'Unsuitable' and 'Border-line' candidates are able to make it to the 'Right' candidates region

   A typical campus written test of 1 hour has 40 multiple-choice questions with a cut-off around 18 marks. Now, consider a candidate who knows the answers to 11 questions and gets another 7 right through sheer guess work. At the interview stage, questions from any given panel tend to repeat across interviews. If this candidate is interviewed towards the end of the line-up, s/he could clear this process because of access to historical information.

3. Conventional linear negative marking approach is used

   To prevent guess work, typical tests (including CAT) penalise a test taker with negative marks for wrong attempts. Negative marks are awarded for each wrong attempt and are a fraction of the marks awarded for the right answers. This system is inherently flawed. To begin with, no attempt is made to differentiate between a student who actually solves the question and gets a wrong answer and a guess maker. Seconly, the resources (ex:time) used by a test-taker are not taken into consideration. For example, a test-taker who attempts 20 questions and gets 4 wrong and a test-taker who attempts 40 questions to get 4 wrong are penalised equally, whereas the first test taker had the luxury of more time per question compared to the second one. pH Test uses a handicap based negative marking approach to address such issues. Read more about pH test's pioneering methodologies.

4. High preparedness of the candidates

   Typically, the test format or question types are extremely predictable for a company. A candidate could prepare for such an exam over a long period of time utilizing historic data and clear the evaluation. In an endorsement of the phenomenon, the IITs recently identified prolonged preparation by candidates as a reason for the falling quality of intake.