ADVERTISEMENT
  About the SA Blog Network













Guest Blog

Guest Blog


Commentary invited by editors of Scientific American
Guest Blog HomeAboutContact

Cornell Student Scrapes Indian Exam Results, Exposes the System’s Flaws

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Exam results are often posted on bulletin boards in schools and colleges in India

Exam results are often posted on bulletin boards in schools and colleges in India

When Debarghya Das’ friends approached him to obtain their final grades from the education board before the official announcement, he gave it a try and got nowhere. Das, a computer science student at Cornell, had just finished his own exams and was to begin an internship at Google in a few days. Having graduated from the same education system as his friends a few years before, he had always suspected that the board was doing something fishy, but had never found any proof of it — he’d only heard rumors.

Intrigued by the prospect of analyzing the data of thousands of students, he began poking around on the results page after the grades were announced and found that the web security was rather lax. In a few hours, he slipped past the education board’s poorly-designed website, scraped together the results of the thousands of students who had written the exam that year and found potential evidence that the grades were being tampered with.

The quality of education in India depends heavily on the kind of syllabus followed by the school, which in turn depends on the state the school is located in. But the exception to this are the Central Board of Secondary Education (CBSE) controlled by the central government of India and the Indian Certificate of Secondary Education (ICSE or ISC) run by the independent Council for the Indian School Certificate Examinations (CISCE).

The CISCE does not have governmental oversight and compared to the other state-run systems, it has a smaller, economically well-off student population. This year over 200,000 students took the CISCE’s tenth and twelfth grade exams. Their results were distributed to external media sources and published on various websites. The security on these websites was almost non-existent and since they had posted the data in the public domain, Das was able to scrape it easily.

The graphs of six subjects overlap perfectly with the same 33 scores missing

The graphs of six subjects overlap perfectly with the same 33 scores missing

When Das analyzed the “26 megabytes of pure, magnificent data,” as he puts it in his blog post, he found that in six different subjects — English, Hindi, computer application, science, math and history, civics and geography — not one student had scored over half of the possible grades attainable to pass the exam. The tests are scored out of 100 and the minimum score required to pass is 35, which means students should theoretically be able to get any of the 66 numbers between 35 and 100 to pass. However, Das’ data showed 33 missing scores.

In other words, of the nearly 150,000 tenth grade students who had taken the exam, no one had scored 33 of the possible 66 marks. Not one student, in all six subjects! It resulted in the sharp peaks shown in the graph above, instead of the expected smooth bell-shaped curve. And even more shockingly, the same 33 numbers were missing in all six subjects. Coincidence?

“It’s statistically impossible,” says Bhubaneswar Mishra, a professor of computer science at NYU. And Das agrees; having heard stories of students complaining about how they expected higher grades, he had always suspected there might be some tinkering going on.

But Gerry Arathoon, the chief executive and secretary of the CISCE, denied Das’ allegations. In a statement to the Times of India, he said, “in keeping with the practice followed by examination conducting bodies, a process of standardization is applied to the results.”

Since the difficulty level of exams varies every year, many examination systems like the SAT and GRE, compute a raw score based on the number of correctly answered questions, which is then converted into a standardized score. If the exam is tougher than previous years, the standardization corrects for it and students receive a higher final grade. However, if the exam is easier, then the grades are lowered.

“But the inherent flaw with the council’s standardization is that they’ve been passing off the standardized score as the raw score,” says Das. Arathoon’s statement also doesn’t explain how the exact same scores could be missing in six different subjects’ results.

In addition, it’s common practice amongst standardized testing bodies to release both the raw and standardized score, which the CISCE doesn’t follow. The SAT, for instance, provides statistical data including averages and percentile distributions in order to reveal how students did on the whole without violating privacy laws. It makes the SAT a lot more transparent in its dealings when compared to the CISCE.

Das compared the CISCE’s system to a big black box. “Things go in and things come out but we don’t know what’s happening inside. It’s unfair to the students. They deserve to know more,” he said.

The Indian education system places a lot of importance on tenth and twelfth grade marks. It decides what subjects you’ll be allowed to study, the colleges you’ll be accepted in and how much you’ll pay to enroll in the major you’re interested in. Colleges announce “cut-off marks” for every major and only those students who have scored above the minimum required score are considered by the college’s admission office. Often the cut-off marks are as high as 99% or incredibly, even 100%.

The pressure to do well is intense during the final years of high school and a shift of one or two points on an exam can make the difference between studying on scholarship or paying thousands of dollars in capitation fees or “donations” to guarantee a place in the university. In such a competitive environment, the CISCE’s lack of transparency can lead to a lot of psychological and financial stress for students as well as their families.

Despite the significant coverage of Das’ allegations by the Indian media, it would be unrealistic to expect any major changes in the CISCE’s system. Since Arathoon has openly accepted that results are standardized, without acknowledging the other discrepancies that Das’ analysis revealed, it’s highly unlikely that the CISCE will change its methodology. At best, they might improve the security on the websites where the results are released.

Image credits: IgnitionMind (top) and Debarghya Das (bottom).

Naveena Sadasivam About the Author: Naveena Sadasivam is a graduate of NYU’s Science, Health and Environmental Reporting program and is currently interning at ProPublica. Her work has appeared on Scienceline.org and InsideClimate News. Follow on Twitter @NaveenaSivam.

The views expressed are those of the author and are not necessarily those of Scientific American.






Comments 11 Comments

Add Comment
  1. 1. sreeancheri 9:52 am 06/13/2013

    Since the errors are systematic, isn’t it unlikely that it is a human error? Isn’t it a programming error?

    Link to this
  2. 2. terribletony 10:22 am 06/13/2013

    Could this be the result of something as simple as each question being worth two points, or taking a 50-point grade and multiplying it by two to map it to a 100-point scale?

    Both of those would result in half of the possible scores being impossible to achieve.

    Link to this
  3. 3. OgreMk5 12:19 pm 06/13/2013

    I’d be very interested in seeing a comparison for this data against the entry marks needed for various post secondary schools. I’d even be willing to make some predictions about this. It would appear that the majority of schools have some cutoffs around the 85, 83, 70, and 65 marks.

    India obviously has a very different educational system than the US. Also, reading the original blog, they don’t have anything like FERPA, which would make the board’s ‘security’ totally illegal. The school I was at (in Texas) had a FERPA violation charged against them because a teacher told a student that her husband passed the same test that she just failed.

    Anyway, after reading this and the original blog, I don’t see how these results are not manipulated. Whether this is done in a systematic fashion (i.e. “we can bump students 1 point if they are one below the cutoff for x university”) or in a algorithmic fashion, I don’t know. But the chances of not a single student (140,000) getting 33 scores out of 5 tests is very small.

    Now, improbable things do happen, but the original blog posts indicates that this has been happening, in the exact same fashion, for over a decade.

    It almost must be systematic.

    Link to this
  4. 4. syzygy 1:52 pm 06/13/2013

    My guess is that it comes about due to some form of corruption.

    I have a patent pending in India but have no intention of licensing a company there. The Indian patent examiner for example did a copy and paste, word for word, of the EU patent examiners first response for the same international patent application.

    I realize it is anecdotal evidence, but my experience is that ethics in India is determined by the highest bidder.

    Link to this
  5. 5. dadster 1:05 am 06/14/2013

    I tend to agree with the view that that its unlikely to change ICSE exam system simply for want of a more effective system of doing it .Till they device one, they will do with the working model
    But, Those who are interested to know more in detail about this analysis , here is it from the horse’s mouth.
    http://deedy.quora.com/Hacking-into-the-Indian-Education-System?srid=d5K&share=1

    Link to this
  6. 6. @Nancy 2:34 am 06/14/2013

    From my experience,some examination systems in computer probably fair,such as SAT and TOEFL ,especially the standardized scored maybe changed because of differ tough in various exams, however all the students who enter for the same exam face to same examination systems.

    Link to this
  7. 7. Jerzy v. 3.0. 1:33 pm 06/17/2013

    Rather curious statistical mystery.

    This extreme hair comb or zig-zag pattern suggests that most questions are scored several points, so it is impossible to get intermediate numbers.

    To get such pattern by fraud, supposed fraudsters would have to tamper almost 100% of answers, and fraudsters themselves would have to be remarkably one-minded. By itself extremely unlikely.

    Memo to the Cornell student: often examining statistic data is the simple part, interpreting results in real life is the real prize.

    However, the student discovered more serious flaw: that education servers in India are so poorly protected. This is real worry. And rather strange as informatics in India is really well developed.

    Link to this
  8. 8. karl 9:16 pm 06/20/2013

    I wonder what are the chances of something like “I’d like to be a milionaire” mindset acting here?
    coded like “if province.caste=Pariah, grade=grade-5″
    ok, perhaps on a better coding, and not so crude, but…

    Link to this
  9. 9. sciedupress 4:05 pm 06/24/2013

    Thank you for making the effort to go over this, Personally i think strongly about this and love learning more about this subject. If at all possible, while you gain expertise, can you mind upgrading your site with increased information? It’s very useful for me personally.
    macrothink
    institute

    Link to this
  10. 10. eashan 11:55 am 08/24/2014

    Campusbulls – One of the Best Education Portals in India providing Education news, Institute details with Ranking & Reviews, Educational Products, Services, resources, Education Job Board.

    Link to this
  11. 11. moris01 7:28 am 09/30/2014

    I’ve been thinking a lot about empathy lately. Some of these buy assignment services are just spectacular.

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Holiday Sale

Give a Gift &
Get a Gift - Free!

Give a 1 year subscription as low as $14.99

Subscribe Now! >

X

Email this Article

X