Journaling my experience with remote research
It’s been nine months since we have started working from home and as for the pandemic, the end is nowhere in sight. Most researchers found themselves in uncharted waters, trying to overcome new challenges of social distancing, working to stay relevant in such unprecedented times. Although remote research has existed during the past years, COVID-19 has made this approach the only valid option till the foreseeable future. With the new normal and the challenge of having little to no experience in remote research, I had to try and test out what would work out for the team and help us get back on track.
Remote research has proven advantageous when it comes to approaching a diverse group of users to be a more budget-friendly approach. It has also become more convenient for the users, keeping in mind their availability and preference. But before you even consider starting remote research; it’s essential to keep in mind some methods, tips for conducting remote research and setting up on the right foot.
☞ Don’t misunderstand it to be a short-cut
Remote research is no different from planning one before the pandemic struck. It is unjust to equate it to be a quick and cost-effective approach. Treat remote research similar to in-person research, following all the processes with extreme planning and purpose.
☞ Start by gaining clarity
It is incredibly essential to understand the basics of remote research and study about the various means available before you start practising it. Conducting such a benchmarking would help you select the right tool for a suitable method that would work well for you and your team.
☞ Plan and might as well over plan
Remote research is 2x harder to coordinate than in-person research. Make sure you are focusing on your research plan.
Key points to remember:
- Have enough buffer time before and after each session in case of technical problems, session overruns and enough breaks from the constant screen time.
- Have a set questionnaire in place so that you are covering all the relevant points during the conversation.
- Set responsibility for note-taking and ensure that you have blocked calendars of observers for each session.
- Create a research tracker for each research project so that your key stakeholders are on track with the research timelines and interview schedules.
Be more willing to accept that things might not go the way they are supposed to work. So, be more informed of what could go wrong and how would you overcome them.
Back-up tips:
- Have an extra pair of headphones/earphones ready.
- Make sure that you charge all your devices.
- Have an alternative video conferencing platform to fall back on in case there is a problem.
- Have makeshift network options to rely on like a mobile hotspot, wireless routers etc.
- Familiarise your team about the interview process so that they can step-in during these times.
- Recruit backup participants in case of last-minute dropouts. Over recruit about 2–3 participants.
- Lastly, I know that any tech challenges can raise anxiety levels. So stay calm! Breathe. 😊
☞ Choose the right method and tool.
Research in times of social distancing has narrowed the number of methods we can use. But that doesn’t mean that we stop collecting user data. After experimenting with various methods, we came up with a few that worked well for the team so far:
a) Get back to the old friend; Telephone calls
There was a lot of experimentation and approvals required before we plunged into remote research methods and tools. During that span, the team had no better alternative but to resort to quick feedback calls to understand changes in user behaviour, sensing the problem and how to overcome it. Although we have mostly neglected telephonic calls, in a pre-COVID world, they proved to be rather useful and quick during such unpredictable situations.
Telephone calls also work with users who don’t have access to good connectivity or technology. It is the best you can do under the circumstances; it’s any day better than omitting people without access to technology from your participant pool.
b) Remote usability tests
Remote usability is an excellent way of observing user interaction and identifying any pain points.
You can use remote testing tools like Lookback.io, which allows you to run real-time sessions with your users or unmoderated sessions. It records the screen, along with the face, voice of the user and their taps during tasks. Zoom and Google Meet are alternate options to achieve similar outputs. These sessions can be recorded or streamed live to be accessible to the team/ stakeholders.
c) Desirability test
Desirability test helps you analyse the aesthetic response to a design. Remote desirability test is done by sharing a presentation with your users, adding videos or image stills of the design in a slide and a bunch of reaction cards in the next one. Let the users pick the words that describe the design the best and reason it out with you.
d) Concept tests
Concept testing allows you to assess user thoughts about a concept or an idea and validate the market fit. Divide the session into understanding your user’s problems and proposing a solution crafted for the problem. Propose the solution either by giving a description of it or showing your users low-fidelity screens/ blueprints to grasp their understanding of the solution and how it solves the problem at hand. During the lack of time or bandwidth, you can do concept testing by sending a survey or converting in-person sessions to unmoderated sessions.
e) Card sorting
Card sorting helps to evaluate information architecture. During the session, participants organise topics into categories that make sense to them and group them accordingly. One can facilitate card sorting activities by using tools such as Google Slides, Miro, Mural. You can invite participants to the board and request them to share their screen or share your screen and move the cards as per the user’s request.
f) Diary Studies
Remote diary studies help inform a journey map, transitioning to be an efficient tool to capture natural user behaviour in the actual context.
Diary studies again can be done using Lookback, after onboarding users to share videos/images of different tasks. You can use Google Forms as a to back up for diary studies by asking users to keep a regular journal over a period. If getting access to Lookback is a blocker for you, try out the same exercise with alternative methods like WhatsApp, by asking your participants to share photos/ videos regularly.
☞ Revive conducting pilot sessions
Research in the pandemic world has constrained many researchers from exploring and trying out the various known research methodologies. But while we tackle the muddle we find ourselves in; it’s easy to let go of the basics, especially running pilots or dry runs. Pilots served to be purposeful before and are more than needed nowadays to assess your plan. It is essential to pilot your virtual research approach to check how it unfolds while you are in conversation with a stranger on the other side of the screen. You will also be able to test the technology you are using and manage any issues.
☞ Set expectations with participants
As you don’t have the chance to meet your participants, shake their hands and greet them, it’s necessary to find other ways to build the rapport. It’s not easy for anyone to show up on a video call with a stranger. The key is to attempt to create personal connections with things such as pre-interviews, e-mail communication, and ensuring a quick run-through, so the participants understand what to expect. This way, you can start the research session on a more relaxed tone.
☞ The zoom fatigue is real
Realise that everyone is in this mess together and equally trying to overcome particular challenges. Be mindful of the situation and consider times when you can make do without a video, mostly around the conversational parts.
☞ Rethink online interviews
With the constant meetings in a day and endless conversations, it’s a little hard for a participant to focus on the planned session. Designing the session like just another meeting would affect the concentration span of the participant. Instead, it’s now time to break away from the conventional methods of research and accept new and fun interactive activities.
Leverage online activities to understand the underlying drivers of the user’s behaviour. Pick whiteboard tools like Miro, Google Jamboard and Mural that have easily understandable drag-and-drop interaction.
Activities give participants moments of self-reflection and enable them to play an active part in your investigation.
Some tried activities used during our research sessions are as follows:
- Circles of Me
- User journey mapping
- Design your Ideal
- Graffiti wall
- Reaction cards
- Word clouds
- Card sorting and prioritisation
☞ Master the sound and silence relationship
With the season of remote research, a researcher would encounter 2x times the awkward silence moment than in-person interviews. One would fill in these moments of silence continually speaking. But this leads to nothing meaningful coming out of the conversation. Please resist the urge to talk and create moments of silence to allow the participant to share their stories.
These are stressful times for all. We need to be empathetic towards our users and ourselves.
☞ Be invested
It has happened a couple of times with me when I have done 2–3 tasks together while conducting the interview. I sent a mail, replied to a colleague on Slack and messaged some of my stakeholders to join the call. In a virtual research session, it already gets challenging to record facial cues and emotions and distractions could lead to losing out important information. It’s best to close all other platforms and applications and focus on the user and the research.
☞ Keep constant checks with the user
We need to research with a little more care during such sensitive times; people are overwhelmed with additional duties, they have lost their social connections, and some have even lost their jobs. It is necessary to have more clarity and understand their situations.
- Keep introductions and ice breakers to a little longer than usual to really understand the users, their plans, their space and experience.
- During the conversation, maintain constant checks of how are they finding the session or if they have any questions or concerns until this point.
- If they speak about something that has been really hard, feel free to empathise with them.
- Give them the option to cut short the session if they want to if it really gets too much.
☞ Understand what tool works best for your user
In some cases, users might not be comfortable with using unheard software such as Lookback. Sometimes, they might not use digital products like Google Meet or don’t have suitable devices or technology to support the call. Such obstacles might refrain you from conducting your remote sessions with your users. While running an initial call with them to set the expectations, do ask a few things:
- Which device would they be using to join the call?
- If this call needs to happen on a particular device, Do they have that device?
- Request them if they are willing to install the software for joining the call? If not, what would be their preferred platform of communication?
Since not everyone would be comfortable with online tools, create a friendly guide explaining to assist them in navigating the software. Identify how you can help them in terms of setting up the call, joining the video call, sharing their screen, and unmuting themselves during the remote usability test. Spare some time to help users who struggle with new tools.
☞ Accept interruptions and respect the participant’s space
COVID has cut off researchers’ access to fancy labs and artificially created research locations. And with participants being at home, comfortable in their environment, interruptions are going to be a reality. There would be food preparations, family members, kids and pets. Such situations demand that researchers reduce the stress of the users, accepting these external influencing factors.
- Ensure that your users aren’t apologising for any disturbances or technical issues. Instead, take time to understand these issues and how they deal with them.
- Be prepared to make last-minute changes to your plan, acknowledging the shared space and accepting the involvement of a family member during the session.
☞ Don’t forget to think about yourself
Finally, with the whole ‘work from home’ fiasco, one can easily neglect their health and daily routine. I have been in situations speaking to five users in a day, without any breaks. It’s natural to feel that you need to be available all the time, but that would take a toll on your mental health.
It’s important to be kind to yourself, take sufficient breaks after each session, be realistic on how much can you achieve in a day and plan accordingly. Remember to relax when your work is done and stay hydrated.
These past couple of months have been very different and challenging, but it’s an excellent opportunity to adopt remote research. I feel that as a team, we are still far from perfecting our approach, but there is no justification to stop speaking to users, and everyone is doing what’s best for them right now. Don’t hesitate to ask for help, learn from people’s experiences and find out what works for your projects. We still have a long way to go! 🙌🏽
