Odds are with user testing, you’re doing it wrong. Eye tracking? Based on pure conjecture. Focus group testing? To easy influenced by outside variables. User surveys? Purely subjective and descriptive. Your project needs research. It’s not that these tactics aren’t helpful, but they need to be accompanied by quantified data. That’s why I’d like to introduce you to GOMS.
Introduction to GOMS
When humans use computer systems, they perform tasks, which can be decomposed into smaller goals, operators, methods, and selection rules. In an ideal system, each goal is achieved as efficiently as possible. It is important that research is conducted on how users will interact with a system (human factors) based on how it is designed (system design). One way of creating predictions about how an individual will interact with a system is by using GOMS. GOMS is an engineering model that can produce “quantitative predictions of performance at an earlier stage in the development process than prototyping and user testing” . This means that designs can be tested, exposing potentially troublesome areas that could affect goal completion.
GOMS stands for the following: Goals (G), Operators (O), Methods (M), and Selection rules (S), which is a decomposed behavior sequence to explain how an individual performs a task. Goals exist at the highest level and can contain subgoals. For example, an individual could have the goal “I need to drink some water.” Operators are a low level analysis of the actions needed to achieve a goal. For example, in reaching the goal of getting a drink of water, an individual might need to locate a glass. Methods are further decompositions of a goal. For example, an individual might need to decide what type of water: bottled, tap water, or filtered water from the fridge. Lastly, selection rules help an individual decide further execution when there are multiple methods available. For example, an individual may choose to drink tap water because the filter in the fridge needs replacing. Or he may choose to drink bottled water because there are no clean glasses. An example of an HCI-related goal might be composing and sending an email or completing a checkout process in an online store.
Types of GOMS Concepts
There are four different types of GOMS concepts: CMN-GOMS, Keystroke-Level Model (KLM), NGOMSL, and CPM-GOMS. Although all of these concepts give predictive information about how individuals use computer systems, they all highlight different parts of the task completion process. In choosing a GOMS concept to assess a design, one must know “the type of task the users will be engaged in and the types of information gained by applying the technique” . For example, when evaluating how a user would interact with a system for the first time, it would be more appropriate to use NGOMSL instead of CPM-GOMS. NGOMSL measures the time it takes users to learn a system and provides insights on the number of steps taken to do so. But CPM-GOMS assumes the user is already an expert at the system and can do multiple tasks at once.
Advantages and disadvantages of GOMS
Like any tool for measuring human behavior, implementation of GOMS has its advantages and disadvantages. First, it gives both qualitative and quantitative measurements, which can provide powerful insight on how users will approach a design. Additionally, because it is a model, it does not require any actual users. Often, usability testing can be both expensive and difficult in order to produce accurate results with physical subjects. Because GOMS is goal-directed, it is ideal for the Agile work process commonly used in many development processes. Agile increments the build of large products into “sprints” of smaller tasks. For example, in building an online banking tool, a project manager may schedule an Agile sprint only for the user task of changing personal contact information. GOMS data would be useful at this level of the build. Additionally, once GOMS data is dissected and implemented to a change in the design, it is easy to modify the model to create future iterations until the design is optimal.
However, regardless of what GOMS concept is applied, “none of them automate any part of the modeling process” . This means that a GOMS model for a given design must be manually crafted by hand. Additionally, because no automation exists, the model must be constructed by someone already skilled in the applied GOMS concept. This makes GOMS both time- and labor-intensive. Additionally, GOMS can only measure tasks that are goal directed. This makes it not applicable in many cases. For example, a GOMS model could be used in predicting the time it would take a user to place an online payment on a checkout screen. But it would not be useful in predicting the shopping behavior it took to get to the payment screen. Because GOMS does not address UX/UI issues, other types of human behavior and usability models, like eye tracking or focus group testing, should be used with GOMS to create a more usable system.
Examples of GOMS implementation
GOMS is used today to predict issues with routine task behavior. In one research study on mental workload and its effect on pupillary response , a partial GOMS analysis of task completion time showed pupil size correlates to changes in mental workload among sub tasks. The research exposed from this GOMS model could help to “forecast a user’s mental workload” amongst smaller sub-tasks in a design. In a more recent study, KLM was used to study the accessibility of graphical user interfaces for visually impaired users . It assessed the difference in time in task completion between two distinct user interfaces, one providing audio-feedback to the user. The GOMS KLM model was effective. When compared to actual user data, it showed a difference in time of only 15%. This was helpful in this particular study because the researchers’ subject group was comprised of only 4 people.
John, Bonnie E., and David E. Kieras. “Using GOMS for User Interface Design and Evaluation: Which Technique?” ACM Transactions on Computer-Human Interaction 3.4 (1996): 287-319. Web. doi:10.1145/235833.236050
John, B., Vera, A., Matessa, M., Freed, M., & Remington, R. “Automating CPM-GOMS.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2002): 147–154. Web. doi:10.1145/503376.503404
Iqbal, S., Zheng, X., Bailey, B. “Task-Evoked Pupillary Response to Mental Workload in Human-Computer Interaction.” ACM Transactions on Computer-Human Interaction (2004): 1478-1480. Web.
Ginn, S. CLIsis: “An Interface for Visually Impaired User of Apache Isis Applications.” University of Amsterdam. (2016). Web.