Having been thrust centre stage during the COVID-19 pandemic, online methods are here to stay. During lockdown, the benefits of adopting online experimentation to study behaviour and cognition have become increasingly clear and the technology to do so is improving rapidly. The potential of these methods is huge. However, the online paradigm is still new for everyone and the question is, how can we make this potential a reality?
I recently attended the BeOnline conference, which brought together researchers from academia and industry who are grappling with this question to share best practice and discuss challenges. The range of talks on offer covered everything from working with blind participants remotely to the technicalities of eye and mouse tracking. There were also demos of some fantastic new development tools, including the Gorilla game builder that will allow researchers to develop immersive research games without the need for any code!
As part of this conference, I delivered a short talk on the benefits of online methods for driver behaviour and transport research and my experience developing a game-based online experiment. This talk highlighted the role that online experiments can play in researching driver and road user behaviour, the challenges of rapidly upskilling to adopt online methods, and the pivotal role the community can play in supporting research planning and skill development.
Here are three of my reflections on the event and the challenges facing online research:
Do online research, not simply research online
Online research should be more than simply taking the same old protocols from the lab and implementing them remotely or building yet another survey. After all, you can run an online experiment in the lab, on the bus, or in a field (4G permitting, of course), and the true utility of this approach is the flexible and powerful experiment development, recruitment, and data acquisition tools that are available. We need to transition from “if I can’t do it in person, I’ll have to move it online” to incorporate these methods into the empirical toolkit with the same flexibility, not as location-dependent extensions of existing approaches.
Move from 2D to 3D methodology
However, as with anything new, there are questions surrounding online methods. How can we establish validity, particularly when the protocols are novel? Do participants pay attention during experiments? Will reviewers take gamified experiments seriously?
While these questions are both reasonable and important, they are by no means confined to online research. Indeed, the replication crisis that is upending psychology is arguably a crisis of fuzzy theory and unclear methodology. As Dr Joshua Blasters discussed in his keynote talk at the start of the conference, we are perhaps unjustified in viewing lab-based research as the “ground reality”, especially when we consider the questionable replicability of many lab-based findings. Here, online methods could be part of the solution – adding a third dimension to the traditional division between ecologically valid observational studies and controlled lab experiments.
The fact that the same study can be run in multiple contexts, in and out of the lab, means that they could provide the ideal framework to advance scientific rigour. Being able to use these methods not only to probe novel research questions, but also to test some of the assumptions that exist around empirical research by triangulating findings across contexts and between fundamental and applied levels means we could deepen our understanding of methodology as a whole.
Be ambitious!
The potential of these tools is vast, but so too is the challenge. Or at least that’s how it can seem sometimes. Integrating multiple platforms that no one had even heard of a few years ago will doubtless present learning curves, but these are not insurmountable. Seeing the breadth and depth of expertise in the community over the two days of the conference was inspirational and it is just this sort of knowledge sharing that will make incorporating new technologies into empirical behavioural science possible. Applying the principles of open science to how we learn methodology, as well as how we report our methods, and having difficult conversations about what works, what doesn’t, and the theoretical and ethical questions involved in our work is not only vital, it is refreshing.
This new paradigm has the potential to not only provide new methods but also to reshape methodology. Now is the time to roll our sleeves up and do it!
