Back

Note: This article represents the view of the author and not the University of Hong Kong.

The credibility of the media is discussed from time to time, sometimes in reaction to some media malpractices, and sometimes as part of the broader issue of press freedom. According to a regular survey conducted by POP, on a scale of 0-10, the credibility rating of our local news media in general dropped to a record low of 5.46 marks by the end of last year.

The reason for the decline in our media's credibility has again become a topic of discussion recently. This article will hopefully contribute to the discussion by focusing on some of the common mistakes made by the news media in reporting public opinion polls. Should our local media really wish to anchor themselves to the world, in order to construct a better China, they must uphold their own standards, including the reporting of opinion polls properly.

According to the author's observation, the majority of the local new media, as well as some opinion researchers, may not be aware of some recognized standards in releasing or reporting opinion polls. Hong Kong hitherto does not have such a code, but there are many such standards around the world, which we can refer to. According to these standards, the media should observe the so-called "principle of minimum disclosure" in reporting polls.

Generally speaking, when reporting opinion polls, the media should also give the name(s) of sponsoring or research organization(s), date of survey, target population and the sampling method, sample size, response rate, margin of error, and the exact wording of the questions. The requirements on broadcast media may be slightly relaxed, due to the amount of information which they can conveniently carry. Using such standards, basically every media in Hong Kong fails.

Settling for less, it still seems that many journalists have, when handling simple figures, forgotten some ABCs of statistics which they have learned in their junior high school. Let's have a quick revision and correct at least a few mistakes!

First of all, take our recent press release as an example, to say that "14% of the respondents were satisfied with CE's policy direction" is not the same as saying that "14.4% of the respondents were satisfied with CE's policy direction", nor "around 15% of the respondents were satisfied with CE's policy direction", nor "one in seven respondents was satisfied with CE's policy direction". Each expression has very specific meanings in terms of precision level. For statistics obtained from a random sample of 1,050 respondents, and having a sampling error of plus/minus 2 percentage points for the relevant figure, it would be over-precise to use "14.4%". Using "14%" would be much more appropriate. The author's usual practice is to give one decimal place only in the tables for easy reference, and rounded to whole numbers in all press releases and related texts. All sampling errors are also clearly stated in order to avoid misunderstanding. The author would suggest all media to use whole numbers to report percentages in opinion polls.

For rating figures obtained from questions using 0-10 or 0-100 rating scales, however, 2 and 1 decimal places respectively should be used. This is because the sampling errors of these statistics depend on the actual distribution of the rating figures, and are normally very small. These measurements, therefore, should be more refined.

This brings us to another common mistake, that of mixing up percentages with rating scores. When CH Tung's average rating was 45.5 marks, with a standard error of 1.4 marks at 95% confidence level, it does not mean that "46% of the respondents supported Tung". Take a hypothetical example, if all respondents did not support a certain person, and all gave him a failure score of 40, his support rate would be 0%, whereas his average rating would be 40 marks. To say that he has a 40% support rate is far from true.

The last common mistake is the confusion, sometimes deliberate, of change in percentage points with percentage of change. Take two of our recent surveys as examples, the proportion of respondents who believed that the policies suggested in the Policy Address could not revitalize our economy has increased from 35% to 59%. Should we say that the negative figure has increased by 24 percentage points, or should we say that there was a 69% ( 59%/35% - 100% ) increase? The author does not think it is appropriate to exaggerate changes by using percentage of change, unless there is no upper limit to the base of such figures, like most of the economic indices. Just imagine, if the support rate of a certain person increased from 2% to 10%, should we say that he has made a five-fold improvement, or 400% increase, or just an increase of 8 percentage points?

The three common mistakes pointed out in this article is only the tip of an iceberg. Nevertheless, if our local media can correct all of them in the near future, the author would be more than satisfied.