Back

Note: This article represents the view of the author and not the University of Hong Kong.

　　Thank you CE Donald Tsang's emphasis on the public sentiment and opinions, now the society became more aware of the functions of the public opinion surveys again. The recent SAR government constitutional development and the West Kowloon proposals have both quoted the government-commissioned polls findings to support their own proposals. Lately, the author has discussed the limitations of using such survey finding in various formats and occasions, and also raised a question to the government in which still remains unanswered. This question is about the questionnaire design and who is in charge of the assessment duties, in order to ensure the level of responsibility and independence of the organization.

The question that the author raised is not to put the government or other organizations in a difficult position, but this is to hope that they understand the importance of the scientific aspects and professional ethics of conducting public opinion polls. If the public opinion debates and comparisons of different poll findings on constitutional development and the West Kowloon proposals are dealt appropriately, this in fact can raise Hong Kong's social intelligence and public awareness. Under this circumstance, mass medias and the academic people in Hong Kong cannot deny this responsibility.

On 31 October, Ming Pao reported a self-conducted IVR survey finding in great details, and their news headline was 'Rapid drop in supporting the constitutional development', the editorial's title was 'Change of public opinion and cancellation of appointment system in order to create triple-win situation'. On 2 November, Wong Ka-ying, Associate Professor from the Hong Kong Institute of Asia-Pacific Studies, CUHK, wrote in Hong Kong Daily News to criticize those related surveys have not met the basic scientific standard. He explained it by listing three reasons:

1. The successful response rate of IVR surveys is extremely low; there is still a big gap to meet the expectation of a professional opinion survey. Those who are willing to be interviewed most probably belong to a specific socio-economic group of people; hence the sample has lack of representativeness.

2. When conducting IVR surveys, unlike telephone survey with interviewers, there is no direct contact with the respondents and it is impossible to judge the suitability of the respondents to answer the questions, this further lower the effectiveness of the sample.

3. IVR surveys seldom ask the demographics of the respondents in detail, in this case, the results obtained from the sample is hardly comparable with the population, which also means it is hard to confirm whether the sample deviates from the inferred population structure.

The author not only agrees and supports Prof Wong Ka-ying's viewpoint, there are two more questions the author would like to raise here, in order to specify the inappropriateness of those related surveys.

Firstly, the author has emphasized a couple of times, whenever any institutions release the survey findings, they should follow the international ethical and professional standard by providing its research methodology, sample information, full questionnaire, survey period, sampling method, response rate, rating method and standard error, etc. in detail. The IVR surveys released by Ming Pao Daily did not provide the response rate, sample information and standard error. It is possible that the survey did not record the respondents' background information, that is why none of such information can be provided. Nevertheless, according to the knowledge of the author towards IVR surveys, its operation system should be able to record the frequency of the dialed numbers and also the category of the contact response in order to calculate the response rate of the survey. In fact, Ming Pao Daily occasionally does supply the response rate of the IVR surveys, but not this time. Regarding the standard error, local medias usually simply ignore it. This is not solely the problem of one news reporting agency.

Secondly, in order to verify the figures from the government, Ming Pao Daily intentionally took out 5 questions from the commissioned surveys by the government and conducted the surveys repeatedly in IVR method. Repetitive verification is supposed to be a very scientific method, but since they are totally different interpretation, this scientific method is not valid at all here. A better method would be using similar questions and conducting similar type of surveys, and then compare the results. If Ming Pao Daily's analyses and findings are based on two similar IVR surveys, some of the problems raised from the research methodologies may be controlled, and the trends of the opinion analyses should be more 'reliable'. But it is important to note that in statistics, reliability and validity are two different concepts, they are also called 'level of reliability' and 'level of validity', they cannot be explained by only a few words here. Findings from IVR surveys can possibly be 'reliable' but 'invalid'; these concepts have to be explained in another piece of article.

After the author read the opinion survey figures in Ming Pao Daily, regardless of those typos in the questions, the author cannot get any conclusion such as 'After some discussion in the society, the overall supporting rate of the government policies has dropped and the supporting rate of the constitutional development have declined tremendously', nor conclusion like 'Cancellation of the voting rights of those appointing councilors in the Legislative Council and Chief Executive election would gain support from more than half of the citizens towards the government policies'.

Luckily, in Ming Pao Daily's report, there is one piece of news quoting a message from the government, suggesting that the levels of seriousness of IVR surveys and professional surveys are not identical, hence cannot be compared directly. This is finally a balanced comment. However, as Wong Ka-ying has pointed out, some people who always criticize the government conducting unscientific research seem to never doubt about the faults of IVR survey findings, but they have agreed on and given explanations and commentaries based on those results.

These double standards can possibly due to an extremely high expectation towards the government by the general public. But since the law is a bit lenient now, this may reflect the general public's non-acceptance and problem of credibility of the government itself. The same rule applies, if this problem does not exist in a newspaper in which self-labeled as 'the most credible' newspaper, the author may simply ignore the issue.

Same as Wong Ka-ying, the author hopes while everybody are competing for the supports from the public, they should also focus on the scientific aspects of the public opinion surveys, and to understand the limitations of different types of polls, so that politics never got mixed with science.