Ask the Right Question, by Jennifer Fraser and Scott Plewes

Abstract

What is the right way to evaluate the current “state” of UX? In a world where business stakeholders are focused on numbers and metrics to ascertain value, how does UX play a role in this STEM-obsessed world? We can start by asking the right questions and instead of looking within, look at how well we are adapting to our ecosystem at this moment in time.

Ask the Right Question

Ask the Right Question
– Jody Foster (as Detective Liz Danvers, True Detective - Season 4)

What is the “right” way to evaluate the state of UX?

In the TV Series “True Detective”, the protagonist (Detective Liz Danvers) is trying to make sense of puzzling observations. These observations seem contradictory, or may be dismissed as irrelevant by others, but the detective persists so that she can, eventually, understand what really happened in the system. A system where people, technologies, and institutions interacted in a complex yet explainable manner. It is a lot of work and takes patience, as well as highly developed skills of observation, analysis and deduction.

In situations like this, there is often an antagonist who wants a quick resolution. A resolution that sounds “good enough”, ie: people will “buy it”, if they don’t look too closely at the data, worry about what data is relevant, or whether you are asking the right questions. The antagonist’s approach may be better adapted, in some ways, to the needs of the other stakeholders in the system. They might also want a quick resolution, something that matches their preconceived notions, and lets them make decisions and move forward. The more predisposed stakeholders are to these appealing characteristics the less they want to listen to the “true” detective.

UX is, at least partially, in a type of crisis because mimicking “true UX” is, in many cases, better adapted to success in business than doing “true UX”.

This adaptation consideration is important because a limited way to talk about the state of something is by describing, or measuring, its attributes i.e.: the particular condition that something is in at a specific time. For example, a person is so tall at a certain age, or a ball is being thrown so fast. Therefore, to talk about the state of UX in this way we would look at its internal attributes. This could be, for example, what it does, what it offers, or how many people have been hired into UX positions in the last year.

But, another way to think about the state of something is how well adapted it is to its current ecosystem at this moment in time. If we instead use this definition, then the state of UX would be defined by both what the UX community does and offers, but also its relationship within its broader ecosystem; in particular to what degree it is thriving within this ecosystem. This means that the state of UX would need to take into account the relationship between UX practices, their value to business, and how UX is influenced - and at times in competition with - other elements in the ecosystem. This would include competition that comes from data science, LLM’s, and practices “mimicking” UX, but not following its rigor. If, for the moment, we think of these elements as UX’s competition, then UX is not doing great.

The relationship between UX and its value to business is highly influenced by an increasing desire for quantitative-based insights and metrics, and an amplified misunderstanding of UX.

Quantitative UX is important, and has a critical role to play, but not to the exclusion of Qualitative UX. Quantitative data can “feel” better because it seems more rigorous, more scientific, and therefore more certain. The fact that the data, or the data analysis, may be irrelevant to the problem or goal at hand is often overlooked. This is exacerbated by the wide-scale availability of data and the outputs of various LLMs feeding this desire, unfortunately it is not accompanied by a clear understanding of the data itself, its provenance, and its implicit bias.

Instead, we see people pushing the data to give an answer without considering whether or not it is the "right" answer, or whether that answer might be obfuscating a more important insight. For example, it may seem insightful to know your customers are going to abandon your product, on average, after a certain number of calls to customer support. But, does that really help if you don’t know why and therefore how to intervene effectively to keep them?

But in spite of this, social science-based UX is losing out to math and algorithms, because it's not adapted to what business stakeholders perceive as valuable. In a STEM obsessed world, numbers rule.

In order to successfully intervene in a complex system, we need to understand the interplay between humans, technologies, and institutions, and that understanding should not only be based on numbers. This is because the elements that define the state within a human system do not share characteristics that are often true in physical systems. In physical systems we have: absolute laws; identically behaving particles; finite and manageable variables; well understood boundary conditions; emergent variables and relationships that are mathematically connected to smaller scale aggregate behaviours; easier extension of experimental internal to external validity; and more apparent causal relationships. These characteristics of physical systems make them amenable to almost exclusively mathematically based analysis.

Math will continue to work fantastic for physics until electrons start having emotional responses, and be somewhat limited in human endeavors until people start acting like electrons.

Traditional social science-based UX plays a critical role in the understanding of human systems, but it is being replaced by numerical models of understanding that are limited, and/or biased, and these limitations are not being recognized as people are often not taking into account the broader system and its inherent complexity.

So, is UX in a losing competition with equations and algorithms? Only if we choose to be. The competition isn’t inherent in the ecosystem, it’s a choice people make. And we can choose to combine our quantitative and qualitative approaches.

The rich data we have access to, and the math and computing we can apply to it, can provide us with unprecedented understanding and innovative solutions. Added to a broader understanding of the ecosystem and the interplay between the elements in the ecosystem to contextualize it, the opportunities for a new approach to UX are almost boundless. This is not just about mixed methods. It is about connecting multiple approaches - qualitative and quantitative - to understanding business ecosystems and the people within them.

We need to ask the right question, and by doing so, we would stop defining the state of UX by looking within, but instead, by looking at the health of its interaction with the other elements in its ecosystem.

We postulate that the state of UX is at an inflection point, and inflections can go in two directions, they can take off upward, or plummet downward.

Partnering with math and computing is the path upwards.

Written by Jennifer Fraser and Scott Plewes.

About Jennifer Fraser
Jennifer Fraser is an accomplished executive with deep experience in the field of Experience Strategy and Design. She has designed products for many different target markets, including: healthcare; security, aviation; and finance. As a consultant, she builds teams and leads experience strategy and design across the spectrum of Customer, Employee, Product and User, as they are all intertwined. She worked as an architect in her former life, which relates to her interest in systems design and finding connections between things.

Links

LinkedIn: https://www.linkedin.com/in/jenniferfraser/
Website: https://jfxd.ca/

About Scott Plewes
Over the past thirty years, Scott Plewes has worked in the areas of business strategy, product design and development in the high tech sector with a specialization in experience design. He has extensive cross-sector expertise and experience working with clients in complex regulated industries such as aviation, telecom, health, and finance. His primary area of focus over the last several years has been in product and service strategy and the integration of multi-disciplinary teams and methods. Scott has a master's degree in Theoretical Physics from Queen's University.

Links