Best Practice Guide: International product testing – ensuring your research is a success!
By Helen Lester, Sagitta Research
Previously we considered why Product Testing Research is more important than ever before (see blog ‘The importance of product testing research in our e-society’). This month we look at what steps you need to take to ensure multi-country studies are a success.
There are a number of key steps that should be taken when carrying out international product testing research which, if you don’t plan them, can quickly grow into hurdles that then have to be overcome. At the very least, this could mean your project may overrun and/or cost more, and at worst, the validity of your data could be compromised.
Once you have established your research objectives and decided which countries you want to survey, you have to consider your target market in each country. Indeed, a product may have a different customer profile in the US to what it has in some European countries; perhaps your customers tend to be older or younger in one country for instance? When setting demographic quotas, it is therefore important to consider the market profile of each individual country you are researching, rather than simply setting quotas based on one country and assuming they are appropriate for all. Indeed, an alcoholic beer company may sell undistilled alcoholic beverages to 16 year olds in Germany and Belgium (where the minimum legal age for drinking beer is 16) whereas in the US, the age is likely to be higher (given the minimum legal age is 21). Consequently, due attention needs to be given to not only who is eligible to take part in the survey in each country, but also the individual quotas which will be set for each country. It is likely you will want to make direct comparisons between specific cells (e.g. age groups), in which case it is imperative you interview sufficient numbers in each cell to ensure that statistical comparisons can be made. In addition, you may need to structure the quota groups in such a way that meaningful comparisons can be made between countries. For example, in the case of beer drinkers, you may require additional quotas (16-20 years and 21-25 years, rather than 16-25 years) to take account of Germany and Belgium, where the minimum drinking age is lower than in countries such as the US (where you may set a quota group of 21-25 years). This will then enable you to evaluate differences by age as well as the potential effect of approaches to legal drinking at a slightly later life stage.
The next step to consider is the timing of your research project. Ideally, fieldwork would take place at the same time in all countries. Apart from consistency, it means the research can be processed and analysed in a timely manner. However, if so, it is important to take into account country-specific biases which may occur. For example, staying with the beverage theme, in an annual report, Carlsberg refers to the fact that beer consumption is affected not only by demographics, but also by seasonality, weather and a whole host of different factors. Given this, if carrying out a multinational beer study in Australia and the UK, you may like to establish a fieldwork period which is not at the height of the summer/winter season in either country in case it affects a respondent’s perception of the product they are testing.
The questionnaire and other survey materials (e.g. interviewer instructions, showcards, etc.) must be properly finalised before sending them for translation. We recommend commissioning our own in-house translation agency – D.Code Translations – as they use native speaking translators and also provide a proofing service within the cost of the translation (so in effect two linguists are involved in producing each translation). Irrespective of which translation agency you use, you need to allocate sufficient time in your study schedule for translations – in terms of translating them and also checking the document is an accurate translation of the original. If it is not, you will not be able to make reliable cross-country comparisons.
When processing the data, it is imperative the data are entered in the same way for each country. We would advise pre allocating set data positions (column numbers) on the English questionnaire (including a code for ‘country’ of course!), so all translations include the same information. Whilst the translations need to be checked to ensure none of the data position labelling has been lost / altered during the translation stage, it saves time and removes the chance of misaligned data later on.
As you can see, project organisation is paramount when carrying out international product testing research. Apart from developing a detailed schedule, all fieldwork documents should be carefully scrutinised, as otherwise any mistakes will be multiplied across all the countries and languages involved – a costly error for a one-country study, let alone a multinational project.
We are experienced international researchers, so if you are planning some product testing research in multiple countries, give Sagitta MR a call. We have the understanding, professionalism and flexibility to ensure your project is a success!