Moving Psychological Science Forward with the Accelerator

Christopher Chartier is an associate professor of Psychology at Ashland University in Ashland, OH. He recently received the 2018 SIPS Leadership award for his work creating the Psychological Science Accelerator and Study Swap. SPSP recently chatted with Chris about the Accelerator project and what is next for this significant undertaking. (This interview has been edited for brevity and clarity.)

What is the Psychological Science Accelerator?

I like to describe it as a standing distributed network — “standing” in that it’s not bound to a single project or one-time point, and “distributed” in that the labs are all over the world. The PSA currently has 350 labs at over 300 institutions in more than 50 countries. These labs have committed, at least in principle, to collecting data for democratically selected, crowdsourced studies. Anyone in the world can submit an idea. We then select the most promising or interesting to pursue and collect huge amounts of geographically and culturally diverse samples of data.

What led you to develop this?

I was really energized after attending my first meeting of the Society for the Improvement of Psychological Science (SIPS) in 2016. So many folks there wanted to improve things in the field. I immediately started working on StudySwap. After the second SIPS meeting in 2017, I had all these interesting ideas bouncing around in my head. I had listened to a podcast about the Voyager mission and then went on a great mountain bike ride. I was probably a little too inspired for what I should have been proposing at that point in my career, but when I came home from the bike ride, I fired off this blog post - Building a CERN for Psychological Science. The post went academic viral, and we quickly had over 100 labs signed up and enthusiastically emailing me. I had this “Oh crap” moment, which was both amazing and terrifying.

What has the year since been like?

Our introductory paper with all of the policy decisions we've made, establishing the institution and outlining how we operate, has been accepted by APS’s AMPPS (Advances in Methods and Practices in Psychological Science) journal. We also have our first four empirical studies in various stages of progress and have 350 labs actively involved. It's still early days for this — the whole thing might crash and burn, but we're going to give it a shot!

Who are some of the other key players?

Early on, we established an interim leadership team which has morphed into a series of directors who are heavily involved in the day-to-day operations of the organization. Some of the other folks who were integral in the development of the project include Lisa de Bruine (University of Glasgow), Hans IJzerman (Université Grenoble Alpes), Charlie Ebersole (grad student, University of Virginia), Heather Urry (Tufts), and Hannah Moshontz (grad student, Duke), and too many more to list!

As an international team, how do you communicate and share information?

I've had to get intimately knowledgeable on global time zones and electronic communication! We do a ton of Google Hangouts and use Google Docs for sharing agendas, meetings notes, etc. For our more hour by hour conversations, we have a very active Slack workspace with more than 100 members. We've got all sorts of conversations going on there.

How are you funding the Accelerator?

We're not! We got this thing started with zero funding. When we started recruiting labs, we said we need folks who can recruit participants without funding at first. The currency of the Accelerator right now is authorship on academic papers. We have Physics-style papers with hundreds of people listed as contributing authors. We use the CRediT taxonomy that started in Biology to break down categories of contribution, so if somebody contributes to multiples of these, they’ve earned intellectual authorship. Our first grant was one of SPSP’s Small Research Grants which we will use to collect some mTurk data for a study. More recently, we've been having conversations with NSF, NIH, DARPA, the Laura and John Arnold Foundation, and the Templeton Foundation. I think the Accelerator can work really well for a couple years without funding, but then we're going to need more infrastructure and more incentives to keep people involved.

What big needs in the field right now do you see the Accelerator filling?

The Accelerator was born out two big needs that go hand in hand. First is the so called “replication crisis.” When we have an interesting question that the field thinks is important, whether that's a replication study or a novel study or even exploratory, we need huge amounts of data to answer it adequately. The second big need that the field hasn't been paying as much attention to is the WEIRD samples problem in psychological science. The vast majority of our publications are based upon participants from Western, educated, industrialized, rich and democratic societies. We need to collect more generalizable, global data. The findings that come out of the Accelerator are often going to be much more generalizable because they'll be collected on all six populated continents, from dozens of countries with different kinds of cultural contexts.

Have there been any unexpected benefits of your work on the Accelerator?

We realized early on that we're figuring out ways of doing research that are going to benefit not just our studies, but anyone engaging in these types of collaborations. We've developed a data management plan that is going to be really helpful to others. The big one for me is the processes we've put in place for translation. When we conduct these studies in dozens of countries, in dozens of different languages, we need to translate the source document into the target language, and then back-translate to look for consistency. So we are getting a ton of practice on that. And sharing those documents on the OSF will be a big unforeseen benefit.

And there are so many meta-scientific questions. For example, how different institutional review boards respond to the same source material. We send the exact same study to 120 different IRBs and get a huge range of responses or requests back. So there's some interesting meta-scientific products that might come out of this that other researchers can look at and benefit from.

What’s next for the Accelerator? In what ways do you hope to develop the product?

Once we have some big funding, we can expand the network—the types of labs we can work with and the types of studies we can support. The hope is that we can move into behavioral studies, or studies that require more specialized equipment. Eventually even neuroscience studies. Right now we are relatively limited in the types of studies that we can effectively conduct. Secondly, we need to find ways to support labs in developing nations—Africa, South America, Southeast Asia —and find ways to get them more involved.

How can interested individuals or labs get involved in the project?

We're always welcoming more people to get involved. They can contribute via data collection, but they can contribute in other ways if they don't have data collection resources —reviewing submissions, improving protocols, helping with translation and material prep. We want to welcome in everyone in the field with this earnest interest in improving psychological science. It requires a broad set of involvement from everyone in the field. Hopefully, people will sign up so we can keep the momentum going!

To learn more about the Psychological Science Accelerator or to get involved, visit https://psysciacc.org