The New Face of Research Surveys: Opportunities and Challenges
Surveys have historically been one of the most important tools available for conducting empirical research. Today, they continue to be prominent within social and behavioral science research and play growing roles in areas such as clinical outcomes research. However, surveys are taking novel forms that afford new opportunities but face new challenges.
Online survey platforms like SurveyMonkey offer a means to cost-effectively gather survey data. What were once census-like paper and pencil surveys are now being virtually distributed. Given the ease with which such surveys can be configured, distributed, and completed, their general appeal to researchers is clear.
Other surveying platforms are also on the rise, such as Amazon’s MTurk, which crowdsources survey completion. Surveys relying on MTurk, for example (which is touted as a “one stop shop” for survey research), tap into a pool of over 500,000 users at very little cost. One study exemplifies this reach, pulling 500 participants to answer a two-question survey for 1 cent, all within 33 hours. Consequently, the crowdsourcing approach generally has an obvious logistical appeal.
However, care must be taken when relying on virtual surveying. While MTurk populations tend to be more diverse than traditional survey populations, they are not representative of the general population and are increasingly biased toward participants from non-Western countries. Also, crowdsourced survey results tend to be more biased toward “socially desirable” answers. And unlike laboratory-based surveys, online surveying offers little to no environmental control, and generally cannot be supplemented with other data (e.g., physiological measurements).
With the rise of personal computing and mobile technologies, electronic diaries (EDs)—e.g., surveys you complete on your smartphone app when prompted—are increasingly useful. EDs are particularly advantageous in their ability to capture additional dimensions of data. Not only can they record survey results, but they can do so frequently over time and in real-world settings. Particularly with surveys collected through smartphones, survey data can also be supplemented with additional metrics, such as location data recorded via GPS or physiological measurements recorded by a Fitbit.
However, capturing additional dimensions of data can make the analysis task more difficult. Longitudinal survey data can be challenging to conceptualize. For example, a decision needs to be made about studying daily data based on actual recorded time or by the “social day” timescale (where a day is between waking time and bedtime). Intuitively, achieving statistical significance may also be more difficult when survey-takers complete ED entries while subject to real-world influences (versus tightly controlled laboratory environments).
Addressing the Open Issues
To support the analysis of modern survey data, Prometheus Research’s RexRegistry offers an acquisition pipeline that collects data from various sources on a large scale, and transforms it into usable forms. The configurability of various querying and automated informatics tools that are integrated into RexRegistry means customized reports can be produced to assess the quality of crowdsourced survey data. For example, checks can be configured to flag when a crowdsourced survey population is increasingly biased towards or against participants from Western nations. Such flagging can be done in real-time. Similarly, RexRegistry’s flexibility means it can handle highly dimensional data sets, such as those associated with smartphone-based surveys. Customizable calculations can also help synthesize such complex survey data, such as to automatically group longitudinal data into social days instead of calendar days.
Altogether, the flexibility of RexRegistry’s database structure, the customizability of integrated informatics tools, and the expert analysts backing these technologies can help mitigate the challenges of modern surveying, while maximizing their opportunities.