Promoting independence through smart tech innovation: Designing a safe and reliable device
Case Study
Camino
Camino is the world’s first Smart Walker: An intelligent electric-powered mobility device that boosts, breaks, and detects movements.
About
- Website: https://caminomobility.com/
- Industry: AgeTech
- Timeline: 6 weeks
Services
- Formative Usability Testing
- Summative Usability Testing
Deliverables
- UX (Usability Reports)
- Dovetail Project
- SUS (System Usability Scale) survey report
Team
- Bryan
- Lisa
Project Info
Challenge
Camino Robotics designed a walker for people living with mobility issues. The initial tests for the device prototype were complete, and they were ready to move on to the next step of becoming compliant according to the International Electrotechnical Commission (IEC), International Standardization Organization (ISO), and Food and Drug Administration (FDA) regulations. They reached out to Rill Insights to conduct formative and summative usability tests. Our goal was to identify any Critical Task usability issues and ensure the device meets user needs and FDA safety requirements.
Solution
Rill Insights created and conducted formative and summative usability tests to evaluate the overall experience and safety of the device. We recruited participants from senior centers and physical therapy offices to complete the tests. The final reports identified any usability issues and provided recommendations of improvements needed to stay in compliance with the IEC, ISO, and FDA requirements.
Rill Insights were a great partner for our AgeTech user testing program. Lisa has tremendous knowledge and experience working with older adults with empathy and care for their safety throughout the process. Bryan has a deep understanding of all elements of product and service development and testing. It is a challenge to engage with older adults and have them agree to participate in the user testing of physical mobility devices, which they worked very hard and creatively to complete. Operating user testing of physical devices with this group requires great attention to user safety at all times, and they were assiduous about this. It is also a challenge to keep users engaged and secure meaningful user feedback, and they achieved this.
The feedback was provided using a rigorous system of video, tagging, and logging video of all points of interest. This provides a valuable and accessible repository for ongoing product development and solid evidence for FDA submissions and audits. The final report was comprehensive. They were a pleasure to work with.
Our Process
Title
Kickoff
Our collaboration with Camino began with an incredibly tight timeline. They needed to pass regulatory safety standards within 6 weeks to stay on track with their go-to-market strategy. With a need for fast recruitment and real world testing, it was all hands on deck for our local research team.
Research Design
Usability tests for medical devices require a much stricter protocol than many other products. Building upon definitions of critical safety risks and their severity, we designed testing protocols that evaluated each potential risk within the framework of elders setting up and using the Camino smart walker. Our research design included testing people in natural contexts of use to emulate real usage as closely as possible. In addition to the usability tests, we designed training protocols that every tester was given, along with two survey: Fall Efficacy Scale (FES) and System Usability Scale (SUS).
Field Testing
We launched the study with a grassroots recruitment effort at local Senior Centers and a physical therapy practice. These were the contexts where we executed the usability studies, because they had slopes and ramps we needed to evaluate the AI assisted “Smart” functionality.
In true “Guerilla ethnography” style, we set up our field recording equipment in public spaces, capturing the influence of the natural world… and the interest of everyone passing by. Lisa moderated the studies while Bryan operated the audio/ video gear and took observational notes. Each study consisted of an introductory interview and FES survey, a training session, a comprehensive review of the instructions and quick guide, usability testing of 24 tasks, and a follow-up interview with the SUS survey.
Analysis & Reporting
With 17 testers and each test taking 60-90 minutes, there was a mountain of data to analyze. We leveraged Dovetail, a qualitative data analysis tool, to transcribe and code each recording. From there we created insights for each of the 24 tasks, scoring success, highlighting any usability issues and safety risks, and providing supporting evidence through quotes and highlight reels.
Deliverables included two separate reports for the formative and summative studies, with scoresheets and data visualizations of combined results. We concluded the study with concrete recommendations for how to improve the product.
Results
The Camino was overwhelmingly loved, nicknamed the “Cadillac of walkers” by the community. Our study identified two critical risks that have since been fixed, and the product is now entering the market as a safe and compliant device.
UX Research
Screenshots of Dovetail, the space for analysis and synthesis. Insights are created by coding transcipts and looking for themes. In this project we organized all highlights by task, and created insights for each.
Human Factors Usability standards
These are the standards that medical devices must meet in order to be considered safe and easy to use.
