How to pick the best software, the first time
When you’re doing human behavior research, funding a new idea often gets to a point where a stakeholder wants pilot data or a proof-of-concept before committing substantial resources. Many experienced researchers are familiar with this Catch-22: you need software to run a pilot study…. so that you can get funding,…to potentially buy better software…to run the project.
In a best case scenario, you puzzle together some rough data that indicates that your idea is promising. Along the way, you learn which software specifications are most important to your study and will help you optimize your study design and research workflow. But now, you have to justify to your stakeholders why you need to buy more software even though you are presenting (what seems like) decent data from software you already have.
In this article, we will discuss why you should think about software as a long-term purchase, even when you are in the early phase of acquiring pilot data. We will also discuss considerations that will help you decide whether a software solution will suit your long-term needs.
Why should you think about research software as a long-term purchase?
To put it simply, changing software is hard. Training users and finding an optimal workflow takes time, research, consensus and workflow optimization. Additionally, there may be resistance from users that have conducted their previous research with different software.
Does your lab keep old computers or have some computers disconnected from the internet because they store precious studies and someone is afraid of system updates? Have you ever collected pilot data using trial software but later found it difficult to move that study to different software?
What do researchers usually look for in behavior research software?
Many researchers start with browser searches, software comparison sites, and AI tools to evaluate and compare software options. This is a good way to compare specifications, but similar descriptions arise again and again.
When choosing software, people often have similar lists of what they are looking for.
- Intuitive user interface. This makes it easy to learn and easy to teach future users.
- Ability to integrate with certain hardware, survey tools, or other software they already use.
- Assurance regarding their participants privacy and data security, especially in online studies.
- Universal data export formats compatible with endless data analysis tools.
- Transparent analyses, so that they understand how the raw data is converted into metrics and can consider whether these analyses are scientifically sound.
These are all important features for human behavior research, but this list is not enough to tell you which software is a good long term investment for your research. Many researchers also rely on recommendations from other researchers or methodology from comparable publications. Asking long-term users about their experience with their research software is a great place to start.
Now, we will go over some key questions to keep in mind when deciding on human behavior research software. First, we start with some software specifications to add to your search list. Next, a discussion on how to get the most out of software demonstrations. Finally, we discuss how your interactions with the vendor or developing company is key to long term success.
Think Ahead: Which software specifications are important for future-proofing your lab?
When choosing software, consider how your research will evolve.
Questions to ask:
- Is the software scalable? Beyond pilot studies, collecting an adequate sample size to test your hypothesis can be a logistical nightmare. Be sure to understand the costs and requirements for adding data collection stations to a physical lab or increasing the number of online participants. Ask about scaling data collection and data collection capabilities. Sometimes, these are different processes with different costs.
- Is the software expandable? Once you go beyond self-report metrics to collect psychophysiological metrics and you have access to real-time data and nonconscious insights, coming up with new research questions is easy. Invest in a software solution that can grow with you. Think about which sensors you might be interested in using in the future. Look for multimodal or mixed-method software solutions.
- Is the software flexible? If you decide to change your study design completely, will this software still work for you? If you decide to pool resources and share it with a neighboring lab or collaborator, is it also going to work for them?
- Is the software adapting?
- Would it be hard to change software? Can you import data from other studies? Can you re-analyze data you collected previously?
- Would it be hard to change hardware? Depending on the sensor you are using, hardware technology may advance quickly or your needs might change in terms of portability or accuracy. Do you need new software every time you get new hardware?
- Is the software keeping up with advances in the field? How are they handling operating system updates? How often do they add new metrics? Are they keeping up to date with privacy and security regulations?
Get the most out of demos
Ask for a software demonstration, (often referred to as a “demo”). If your study design is more complex or your questions are more technical, your demo might include a software expert. They should have extensive experience in setting up different types of labs and be able to come up with creative configurations.
A demo will take you beyond your current understanding, get you thinking about workflow (the key to your daily sanity and resource management), and help you understand possibilities you may not have considered.
Our Suggestions:
- During the demo, rather than asking about where to click, ask about available resources for how to use the software and whether the software company provides training.
- Software companies usually have a collection of premade demo studies and will try to choose a study that aligns with your research goals. The more information you can give about what you want to investigate, the more helpful the demo will be.
- Even better, if you have some examples of data from your previous studies or stimuli and you can share them, do it! Some companies are happy to personalize your demo using what you provide.
- Ask for a sample export. See if the format is compatible with your data analysis strategy and software.
- Ask for a demo of a technology you are not looking into using now, but you think might be interesting to use in the future. Can this software grow with you, or is it only good for the method you are currently using?
From this day forward, it’s a relationship
If you choose software that can grow with you, there is a good chance you will have this software for a long time. It is important to consider the company that makes the software.
Questions to ask:
- How is their scientific credibility? Is this software mentioned in publications? Are they using analysis strategies accepted by your research field and how transparent are they about how the data is handled?
- How hard is it to get answers? If you like to figure things out yourself, are there abundant resources you can search through? If you want to talk to someone about your specific situation, how hard is it to get in touch with a person? Many tech companies use chatbots to filter questions to the right department or automatically provide answers to frequently asked questions. Are you directed to a person or a resource that can help you? How long does it take them to direct you to the right place?
- After I purchase, can I still get in touch with a person? Find out how the software company handles tech support (their availability and response time) and whether you will have an account manager that you can go to for questions and clarifications.
- How often is the software updated? A lot of behavior research software comes as a subscription. You should expect that the company keeps up to date with relevant metrics, operating system updates, and remains compatible with the tools you are using. Ask for an overview of software updates from previous years so you can see changes the company has made in their software over time. Ask which changes clients had to pay for and which were included in their subscription.
Final thoughts
Choosing behavior research software is often a long-term commitment, so thorough evaluation is crucial. Look beyond the specifications you need today and make sure the software is scalable, expandable, flexible, and adaptable. Get the most out of your software demo by providing information about your study design and research goals. Remember that selecting software is the start of a relationship, requiring ongoing attention and communication with the provider to maximize its value and longevity.
