You've made the decision to move your research study online, but perhaps you are not familiar or entirely comfortable with the various tools available. We've compiled a list of some of the more popular ones here and take a look at the pros and cons of each.
Step 1: Moving Online
The first step to moving online is transferring the study materials to an online platform. The best platform to use depends on the type of study being conducted. If the study is a survey, then the questions can be pasted into a survey-hosting platform. Popular platforms include Qualtrics, SurveyGizmo, SurveyMonkey, and Google Forms. You may have access to at least one of these platforms through your institution.
If a study employs a more complex behavioral task, consider using an experiment-hosting platform. Popular platforms include Gorilla, Inquisit, and Pavlovia. These platforms allow researchers to put together experimental tasks, either from a library of common tasks or individually programmed by the user. These platforms also host the experiment on their servers so researchers do not have to set up their own websites to host studies.
Gorilla
Gorilla is an experiment-hosting platform for behavioral tasks that cannot be hosted in a survey. Gorilla has a graphical interface, which researchers can use to set up studies using a library of experimental tasks. Researchers pay per participant response.
Pros
No programming knowledge is required to use Gorilla.Cons
Complicated tasks can require coding if they are not already included in the built-in libraries.Inquisit
Inquisit similarly hosts complex behavioral tasks on its platform. Researchers pay for a license to conduct unlimited experiments for a certain amount of time.
Pros
ProsIt requires participants to download software on their own computers, increasing confidence that participants have a standardized experience.Inquisit enables the researcher to see if a participant has navigated off of the experiment window.
It can be cost-effective for running a lot of experiments with a lot of participants.
Cons
Many participants will not be willing to download new software to complete a study.
Using Inquisit requires knowledge of computer programming, as researchers write their tasks in a language similar to JavaScript.
It can be expensive to use in a less active research lab.
Pavlovia
Pavlovia is also an experiment-hosting site for complex behavioral tasks. It supports tasks written in Python or JavaScript using the libraries PsychoPy, jsPsych, or lab.js. Researchers can choose between paying for a license or paying per participant.
Pros
It can be easy to use if you already have experience with Python or JavaScript.Other researchers' code is often public.
Uses open-source programming languages and libraries, which means researchers can save their code and host the experiments elsewhere in the future.
Cons
Pavlovia requires comfort with computer programming languages and tools.Researchers have to do the bulk of coding themselves, which can be time-consuming.
Step 2: Recruit Participants
The second step is to recruit people to participate in the study. Researchers with funding to compensate participants will often recruit from online participant pools. Popular platforms include Amazon Mechanical Turk and Prolific. Researchers at academic institutions with an undergraduate participant pool can disseminate their online studies through platforms specific to their school, such as Sona Systems. Another option is to use social media to recruit individuals who are not signed up for a participant pool.
Most hosting platforms will store the data collected. Depending on the platform, data can be downloaded in a variety of formats, such as CSV and SAV.
Amazon Mechanical Turk (MTurk)
MTurk is the most popular recruitment platform for academics. Amazon founded Mechanical Turk to establish a workforce to complete discrete tasks that computers are unable to do.
Pros
There are a lot of workers on MTurk, making it possible to collect hundreds of responses within hours.
MTurk does not have any restrictions on participant payment.
Cons
Researchers have raised concerns about bots on the platform.
MTurk offers limited prescreening options and researchers must pay extra to use prescreening criteria.
The interface can be difficult to use, so many researchers use a platform called Cloud Research (formerly TurkPrime) to recruit from MTurk.
Prolific
Prolific is an online participant pool established to support academic research and used only for data collection.
Pros
The staff at Prolific closely manages the participant pool, including verifying the phone number on each account and strategically releasing studies to participant accounts.
Prolific includes an extensive prescreening questionnaire, which researchers can use to target populations at no extra cost.
Cons
Minimum compensation rate of £5.00 (or $6.50) per hour.
As participants only sign up for the platform if they're interested in research studies, there may be concerns about participant non-naiveté.
Sona Systems
Sona Systems is a participant management system. Schools or departments typically purchase and manage a site for their members to use. Institutions most frequently use Sona to manage undergraduate student participant pools, though some maintain a pool of paid participants as well.
Pros
Sona allows a lot of control over participant management.
Paid and student pools can be managed on the same platform.
With a license, unlimited studies can be run without paying a commission for each participant.
Cons
Using Sona requires an on-site administrator to manage the website.
Finding participants to sign up for the Sona pool rather than individual studies can be time-consuming.
It is cost prohibitive for individual researchers to obtain a license.
Barbara Toizer is a PhD student at the University of Kansas.
Grad Student's Guide to AI: Top Platforms to Boost Your Academic Success (Part 2)
Grad Student's Guide to AI: Top Platforms to Boost Your Academic Success (Part 1)
