August 14, 2011
Two polls, one styled as the State of the Nation Poll by the CNN-IBN and another titled as the “Mood of the Nation” poll by the India Today over the past week have left everyone confused, if not about the real mood of the nation, then certainly about the ability of pollsters to gauge it correctly.
While the CNN-IBN poll says that the UPA is likely to better its performance at the national level by winning as many as 280 seats, the India Today poll says, contrarily, that the UPA may lose as many as 70 seats and drop to a tally of just 187-197 seats. In terms of popular vote, the CNN-IBN poll says that the UPA is gaining two percentage points nationally, while the India Today poll says that the UPA is losing as much as 6.7 percentage points in votes.
Bizarre divergences indeed!! One of them is horribly wrong.
Of the two, the national projections of CNN-IBN poll appear flawed as voters cannot be expected to give a resounding victory to a coalition after expressing a strong desire for change of the same coalition government at the Centre! While people may not necessarily vote for a party that they believe has performed well (as in the case of Vajpayee regime in 2004), it is utterly fallacious to think that people will vote in GREATER NUMBERS for a regime that has lost their goodwill and which they desperately want replaced. Tell me one such instance where a party has returned to power with a greater vote share after becoming thoroughly discredited among people.
As far as I can remember, no two opinion polls at the national level have ever shown such divergence even in terms of whether a national alliance is gaining or losing votes. The CNN-IBN poll was conducted by the CSDS and the India Today poll was conducted by Nielsen almost around the same time. Both are noted pollsters and have been conducting these polls for many years. It is therefore difficult to explain the differences.
Huge Divergence in State Projections
The two polls also differ on the projections for most of the states. For instance, the CNN-IBN poll says that the Congress party has a sizeable lead in a Lok Sabha election in Madhya Pradesh, Karnataka and a narrow lead in Delhi, whereas the India Today poll shows the BJP sweeping all these three states.
Paradoxically, CNN-IBN poll finds that the BJP is neck and neck with the Congress in a contest for the Madhya Pradesh assembly when it own poll shows that the state’s chief minister Shivraj Singh Chouhan is one of the most popular chief ministers in the country with an approval rating of over 75%. To me it seems the wishful thinking of the pollsters rather than a reflection of ground realities. The BJP has won almost every single election and by-election in Madhya Pradesh and wrested even the SAFEST seats of the Congress in the last two years, and all indications are that the BJP is all set to post a big victory in 2013 assembly elections.
Nielsen’s UP Bouncer
India Today poll says that the BJP is leading in Uttar Pradesh by a wide margin in both assembly and Lok Sabha elections. It shows the Samajwadi Party struggling at the fourth place with just 11 percent of the popular vote while the CNN-IBN poll places the SP in the lead in the state!
India Today poll’s findings in Uttar Pradesh appear to be a result of poor sampling or bad field work by Nielsen in the state rather than a miraculous turnaround in the BJP’s fortunes. The same problem seems to be the result of the CSDS’s findings in Madhya Pradesh, Karnataka, Delhi etc.
Sample Size small for State Projections
CNN-IBN poll has an all-India sample size of about 20,854 voters and the India Today poll has a sample of 12,000 voters. These are large enough samples for making national/ regional projections. Hence, both the polls should have arrived at broadly similar projections at the national level even if their projections varied somewhat at the state levels. Such state level variations in projections do not greatly affect the national projections as errors at state level tend to get cancelled out when they are aggregated to make national projections. Only when the pollsters have inherent biases that this statistical factor does not come into play and all the errors (mostly deliberate) are committed only in one direction.
Pollsters ought to exercise abundant caution when they put out findings based on small samples. The sample size in the CNN-IBN poll was less than 1500 in all states except U.P and perhaps as low as 600 in some states. The average sample size in the India Today poll was even smaller.
Vote and seat projections with such samples are fraught with serious limitations. Bizarre Uttar Pradesh projections by the India Today poll and Madhya Pradesh projections of CNN-IBN poll are a victim of this greed to overanalyze data. Pollsters are generally aware of these limitations and usually it is the editor’s overzealousness to go over the top that results in such adventurist projections. ‘Startling” findings do make good headlines but they kill whatever faith is left in the accuracy of opinion polls.
Rahul Gandhi is leading in the prime ministerial rankings of both the polls and this is largely on account of the fast depleting political stock of PM Manmohan Singh and non-projection of any leader by the Opposition as a PM candidate so far.
Here again, there are some inexplicable differences. India Today poll finds that the PM ratings of Rahul Gandhi have dropped from 28% last year to 21% this year, whereas the CNN-IBN poll has seen an increase in his ratings from 6% in 2009 to 19% this year. So is Rahul Gandhi’s popularity raising or falling? Make your own best guess.
Narendra Modi has received the highest prime ministerial rating for an opposition leader in both the polls with the India Today poll giving him a rating of 12%, compared to a meager 5% given by CNN-IBN poll.
With the Indian elections increasingly looking like presidential contests, at the time of elections people are likely to compare the prime ministerial candidates of different parties/ coalitions. These ratings would then become redundant and one-on-one comparative assessments will come into play.
A poll undertaken by the LensOnNews sometime ago sought to make such comparative assessments. A nationwide poll across 40 parliamentary constituencies showed that Rahul Gandhi was preferred over Manmohan Singh but Narendra Modi was preferred over Rahul Gandhi by a wide margin.
Asked who would make a better prime minister between Rahul Gandhi and Narendra Modi, 53% favoured Modi against only 38% who rooted for Rahul. Therefore, media going gaga over Rahul Gandhi’s ratings may prove to be ephemeral should the BJP choose to promote Narendra Modi as the prime ministerial candidate.
Opinion polls do not influence the way people vote in elections. Else, we would not have had so many polls going wrong in the past. While the pollsters in general had better success in projecting state election outcomes, the last two Lok Sabha elections held in 2004 and 2009 have proved all pollsters (including myself) wrong by a wide margin. Voters’ pulse is increasingly becoming difficult to gauge in national elections, and hence these poll findings should be taken not with a pinch but dollops of salt.