Market research – how good is the data?

Everywhere you look there is a company willing to pay people to participate in their surveys. It seems like a win win situation, the participants get paid for providing their opinion, the market research company gets paid for conducting the research and the companies, that fund the research, gathers valuable data.

But, how valuable is that data?

Well run surveys are still vital to those companies who wish to research a market and obtain feedback on particular products or services. Many people are willing to express an opinion, and a small incentive like a chance to win a prize or a small payment is often all that is required to encourage people to give up their time and participate.

For years market research companies have assembled groups of participants into survey pools or survey panels, groups of people willing to provide an opinion on a range of subjects. Often incentives of nominal value were offered as a 'thank you' for the participant's time; however the explosion of 'paid for taking surveys' websites has spawned a new profession - the professional survey participant (PSP).

For companies that are relying on market research generated by panels of PSP's the question they must ask is how flawed is the data that they are relying on? Does the average PSP represent a cross section of society or a niche? What social group do they really belong to and is it smart for a company to base research on groups of people that are willing to comment part time, and sometimes, full time, on anything and everything?

Not all panels should be tarnished with a negative brush after all customer and employee surveys to name just two rely on a survey pool. However, customers and employees are to some degree stakeholders in a company's research, by participating customers will benefit from better products and services, employees will be able to voice concerns and help make improvements to working conditions and methods, more importantly the views of the customer and the employee are valuable. Likewise panels that are made up of hand picked participants chosen for their knowledge, expertise and experience are equally valid and in some cases a necessity.

The panels that should be questioned are those that are assembled by surveys that rely on large volumes of the general public whose motivation for participating is only the reward they will receive, they will often have no knowledge, understanding or interest as to the 'who', 'why' or 'what' of the research that is being carried out.

Research companies that attract either part-time or full time PSP's rarely make any reference about the importance of the research they conduct but instead concentrate on offers of easy money and extra income.

Some survey methods will be more vulnerable than others, the most vulnerable perhaps being the online surveys itself. Company's that use PSP research data have to ask themselves how many surveys would it take for each PSP to realise that the more surveys they take the more cash or other incentives they receive. Will they after completing three or four surveys even think about the question or just click on any random response?

Although telephone surveys and focus groups will be less prone to the problem of people responding in a random or unconsidered manner with each telephone survey they take, and focus group they attend, they are becoming less like a member of the general public and more like a professional respondent.

A survey can have many purposes and not all surveys are conducted with a view to being impartial and fair. Like a tobacco company that funds research into the effects of nicotine, the findings can be manipulated to reinforce a particular line of argument. Panels and PSP's have their place but if a company is relying on good and accurate market research data – beware the survey panel!