Last week, we had the pleasure of talking with Shannon Hoste, President of Agilis Consulting Group, about her expertise in human factors and usability. Shannon led the CDRH Human Factors team and is also on the AAMI HE – Human Factors Engineering Committee. Here is the second part of our interview with Shannon. Read Part 1 of our interview.
OHP: If a company comes to you and has a product ready to go to market but needs human factors (HF) data, what do you do?
SH: First, you need to understand how the product is used, who uses it, where and for what purposes. You build the use-related risk analysis. Through this analysis, you learn what steps could cause potential use errors or difficulties and identify which ones could lead to harm; and this determines the focus of your study.
Once you get to the summative (aka validation) test, it is essentially a dry run of your post-market experience. Fifteen users for each user group, and they all use it as they would in the real world. This is where it is important to understand what your representative use scenarios are; as you will want to simulate them for the study. For example, in a simulated study, HF validation for home-use, participants would be provided the product as they would receive it as a patient, such as “you picked this up at the pharmacy and then came home.” Similarly, if there is training that is regularly delivered with that product, you would replicate that process as well (along with representative training decay).
When considering your user groups, characteristics such as age and demographics come into play. With 15 people per user group, I am considering who the product users are throughout its market lifecycle. Then within those users, are there any distinct characteristics that would create a new user group, such as: pediatric, teen, adult, education, training, physical capabilities, sensory capabilities, etc. Based on progressive health conditions, you might distinguish some of the impairments into separate user groups, which can make the testing quite large when you need to really breakdown the user groups. It is also important to recruit diversity in your study.
OHP: Do you have to witness and watch use, and if so, does that need to be done in the setting or can it be done by video?
SH: Most of it is live since you are trying to replicate the environment. That said, over the last two years, there has obviously been more remote testing. Remote studies can be interesting for home use because you are sending the product to them and they are using it in their own personal environment. When someone takes a device into their home, it’s a different (but very real) experience.
In all testing, we try to create an environment where people are comfortable so that when they get into the right mind set and start using the product, you can see the pieces that trip people up and identify where the difficulties are. This would be objective data, how did they perform. Then you can ask questions and probe deeper to understand where they had difficulties and better understand why. This would be subjective data.
After the study, you are going to look at each and every one of the observed or reported use errors and difficulties and see why they happened. You will also want to explore: were that to occur in actual use, what could be the resulting harm? Now, given the nature of observational studies, you may also see some things happen because of the study itself. This whole analysis is where training and experience in human performance, memory, learning, perception etc. are leveraged to identify root causes and potential solutions.
OHP: Do you always start with a hypothesis?
SH: I actually try not to have a hypothesis. Part of human factors and usability is keeping an open mind to see the patterns that exist, and trying not to go in with any preconceived notions. You definitely start to see trends emerge, which is helpful.
An example of a trend is that people generally don’t do well when you ask them to do math in a high stress environment. For example, there was a medication that was on the market and the new product was the same medication but a 5x higher concentration, requiring some math to get to the correct dosage level. In early studies, nurses were noted in this calculation to multiply by five rather than divide by five! This would be a 25x overdose. That information was extremely valuable to learn in a simulated study, and actionable.
OHP: Is human factors testing required for all pharma, too?
SH: It depends on risk. In combination products (devices with drugs or biologics) it is pretty common, yes. In that space, use-related risks can lead to medication error and this data becomes very important to support safety.
OHP: What else should we know about human factors and usability, specifically with regards to packaging?
SH: Packaging is considered part of the user interface, therefore it should be evaluated as part of the overall HF process, to understand and demonstrate safe and effective use. Additionally, I think there are missed opportunities; there are only so many options to get in front of the end user during use. What I mean by this is ideally, we want to provide information or cues to the user while they are using a product. Think about haptic feedback on your smart phone as an example. Now apply that to a mechanical device, how do you get information that is noticeable and perceivable to the user while they are using the product? Well, since they have to interact with the packaging, there is potential opportunity to utilize this phase of the user interaction to support use. You see examples of this in some medication packaging, and I’ve also seen great use of packaging in diagnostic kits for home sample collection. Even in the operating room environment, I was excited to learn about Oliver’s HDPE CleanCut Cards. I think there is a lot of potential value there.
There are also regulations. There is a line item about the MDR – Section 11 Annex that includes a clause around packaging. It states “it shall be ensured that the integrity of the packaging be evident to the user.” This is something that can be better understood and demonstrated through your human factors activities.
OHP: Are there any trends you’re seeing that may speak to the future of human factors?
SH: I think the trends we will see in human factors will follow the trends we see in technology. For example, take artificial intelligence. As AI advances or you have a device that integrates AI, how do you establish and understand the use and safety questions around this? How does my user know when to trust the AI and when it should not be trusted? We are evaluating how humans and technologies interact. We must continue to study and understand potential harm or risks.
Overall, our future with human factors and usability involves many puzzle pieces—and it’s an ever-changing puzzle.
OHP: Thanks so much for all of this insight! It’s so interesting to think about what’s ahead.
SH: It’s my pleasure. I love what I do. Working in this field is rewarding because you are making an impact and helping people. If you are working on any aspect of a medical device, you are making a global impact, which is very gratifying from the standpoint of doing something that is meaningful. Referring to product development, human factors and use-related risk is like pulling together puzzle pieces to make it work. Whether you are guiding a project or a team of people, to be able to step back and make the connections that move things forward is rewarding.
