There are many factors that contribute to human-to-human trust: accountability, integrity, confidentiality, to name a few. In last week’s article, we shared our perspective on how Thread’s product and design team create transparent experiences to fan the initial embers of curiosity into a warm hearth of trust. While transparency may create the ideal environment in which trust can be kindled, a single interaction, no matter how clear and honest, is still not enough to dissolve the deep rooted skepticism many people still have around AI. So, what is the secret ingredient to cultivating deep rooted bonds of trust? The ingredient—or rather, a pattern—is consistency.
From a design point of view, how we cultivate reliability in Thread Service Desk is more about crafting a consistent and predictable experience than about delivering results with 100% accuracy all of the time. Can a technician or service desk admin expect the AI to greet their clients in the same style, tone and structure every time a client interacts with the agent? Can a repeat client expect the same or a familiar experience when reporting an IT issue? How might a service desk administrator test the behavior of an agent continuously to guarantee repeatable results? These are the questions the product and design team strive to answer through the features we deliver.
One of the core agentic products to Thread is the Triage Agent, an AI agent that acts as the first touchpoint with a client. When the client writes in a request through chat or email, the Triage Agent will greet them, ask questions to collect necessary information, and hand off the issue to a technician who will further analyze the problem and provide resolution steps.
As the service desk admin, evaluating the quality of Triage Agent’s responses in real-life scenarios before rolling the agent out to their clients is a mandatory piece of the user experience. The testing phase is crucial to increase an admin’s confidence in the tool, encourage them to roll the agent out to their clients, and, in the end, improve our overall Triage Agent adoption goals. As a result, the product and design team decided to prioritize improving the existing testing experience.
In design, we often emphasize the concept of “recognition over recall”, the principle of making information visible to decrease a user’s cognitive load. In the old implementation of the testing experience, the emulator would appear on the right hand side of the page. The challenge with the implementation was that the emulator was not sticky to the top of the page, which increases the user’s cognitive load as they needed to actively remember what configurations they had changed that were now out of view when using the emulator.
In the new implementation, we reworked the layout so that the emulator always persists, keeping context visible at all times.
In addition to improving the interaction quality, we also wanted to enhance the functionality of our testing capabilities to create a safer testing environment. In the legacy experience, there was no “sandbox” environment, or ability for the admin to test changes in the emulator without deploying the changes to any client that had access to Triage Agent. Without the ability to test without consequence, there was no way an admin could guarantee that their changes would achieve the intended result, cultivating hesitancy and fear rather than curiosity and experimentation. In response to this lack of functionality, we introduced a global “publish” action and individual “save” actions per configuration card so that admins could save their changes locally, test them and fine-tune them, all before publishing to live client environments. Through safe testing, admins can verify that any changes in configurations result in expected changes to make sure that clients have a reliable experience with the AI.
Ultimately, building trust in AI is a continuous practice that will always be relevant as people’s relationship with AI evolve over time. The improvements Thread has made thus far within the agent testing experience is only a small segment in the lengthy journey for us to create comfort and establish trust within Thread Service Desk. They reflect our deeper design philosophy that confidence in AI must be earned through repeated and verifiable behavior.
Once there is an established pattern of dependability where service desk admins can conduct experiments and improvements without fear of consequence, we move onto the final piece of the puzzle. With transparency as the foundation and dependability as the next layer, the admin now needs to feel a sense of agency and partnership with AI. In our last article of this series, we explore how we give admins what they should never have to sacrifice: control.