Every day, we are bombarded with data from a variety of sources. Data posing as objective syndicated research appears in media, on industry websites, across our social feeds and more. Yet, opinions and statements may not always be truly objective syndicated research. It is important for travel brands and organizations to understand the methodology of the research they are reporting – especially when investing in data that is used to make key marketing decisions or to report success.
DEFINING SYNDICATED RESEARCH
Syndicated research is funded, conducted and owned by an independent research firm. It is sold or licensed to individuals or organizations interested in the data. The research firm has no vested interest in the results of the data, but it should be able to analyze the data to derive useful insights for its clients. The primary uses of syndicated research are typically to track performance and for competitive comparisons.
KEY ELEMENTS WHEN EVALUATING SYNDICATED RESEARCH
A syndicated research firm should strive to develop and use the best possible objective methodologies for collecting, processing and analyzing data. Trending performance is dependent upon stable and consistent data collection methodologies that stand the test of time. This does not mean that data collection and analysis methodologies do not change; however, there must be compelling reasons to do so, and appropriate testing must be employed to understand potential impacts on the trends. The longer a research firm has been conducting syndicated research in a particular field, the more guidance they can provide.
The syndicated research survey must provide an objective measurement of the population of interest. If, for example, understanding the travel behavior of U.S. residents is the goal, then you must ensure the sample is representative of the U.S. population. Thus, when determining visitor volumes, surveys should be indicative of all residents of a population – both travelers and non-travelers. Additionally, when looking at visitation to a specific destination, the most accurate methodology is to ask survey respondents to list their destinations visited (unaided recall) without the aid of dropdown menus or preselected response options (aided recall). Aided responses create the potential for error – forcing the respondent to choose an option that may be nearby, but not the exact destination visited.
Sample size is important. Sample size should always be considered based on the number of completed surveys, not just the number of surveys distributed. For example, we may invite 20,000 people to participate in a survey, yet only 5,000 may actually complete the survey. Thus, the data reported are based on 5,000, not 20,000 respondents. This is important when determining the statistical validity of the data – especially when looking at cross sections of data. For example, the total sample of respondents visiting a destination may be 600 respondents. If, however, we only want to know about specific behaviors of business travelers, who make up only 20 percent of total travelers to a destination, then we are looking at data based on a much smaller sample of 120 respondents. These data can be very valuable, particularly for directional trends; just make sure you understand the amount of error being introduced by smaller samples. While it is not always possible, a good rule of thumb for any analysis is a sample of 400 respondents, as the increases in variability are very small for samples of greater than 400.
Understand the source. The source for who is invited to complete the survey is rarely considered when evaluating data, but extremely important in the quality of respondent. This is particularly true when a web-based survey method is used. Some surveys are just “put out there,” and anyone who happens to see it is asked to participate. This is called a river sample and produces the least representative and most biased samples. There are other panel sources that use a combination of several panels to obtain the desired number of completed surveys. These are better than river samples, but typically not as good as panels managed by a single source. Companies that own and manage their panels understand the characteristics of their panel participants and can more accurately generate samples representing the target audience. The most representative of all are panels constructed using address-based methodology.
Consider the timing of the survey. Survey timing is important for two reasons. First, the more frequently a survey is conducted, the more likely it is that recent behavior will be observed and that the effects time has on memory will be mitigated. The second has to do with the behavior being measured. For example, we know that travel behavior changes depending on the season of the year or even in specific months of the year and that these time frames can differ by destination. Thus, monthly is the optimal survey frequency for syndicated travel research.
With questions, please contact us.