Dynamic Actions Application
Product vision and design, UX/UI, overview video (2022)
How can I improve conversation outcomes, handle critical and unpredictable business situations in conversations, at scale, in real time?
The problem
LivePerson's customers use bots and AI to route large number of conversations, to initiate workflows, to collect and store conversation context data, and much more. The problem is that in the past few years, they have seen a dramatic increase in messaging volume, so in response, they have invested heavily in automation to handle customer inquires and scale agents support. However:
-
most bot automations are scripted to handle easy or predictable situations/use cases. (E.g. customer information gather, pay bill, change seats for flights).
-
advanced automation workflows are very complex and expensive to build, requiring highly technical resources for long period of time
-
it's hard to track and manage if automations are working properly due to the very large (and growing) number of bots specialized in less common use cases
The goal
Understand the needs and expectations from Automation managers and builders, identify unsolved, complex or expensive use cases and build an application that:
-
monitor every conversation and recognize signals in conversations
-
enable advanced automations in real time
-
provide complete visibility and traceability for brands
1- Discover
Through constant conversations with Customer Success Managers (CSMs) and brands, our product manager put together a PR-FAQ/business requirements doc, which identified two good use cases that could benefit from this new product idea. It was clear how much complexity our customers faced when using our current platform capabilities to built more advanced automations,.
But I needed to know who does what and where, in order to understand our users task flow. So for this part of the research, I supervised a Jr. Designer that I mentor, so she could help with the interviews to map how our users utilize the eco-system during the creation of automations or routings. We also did persona development and interviewed several users, to learn about their motivations and expectations, pain points and common flows when using our platform for automations. After synthesizing the findings, I was able to validate 3 personas for this product: The automation manager, the builder, and the account manager.
Click on any image to enlarge
By the end of the discovery phase, I had the following learnings and conclusions:
-
Automation managers and account managers want an easy, low-to-no code way to build, deploy, manage, test, analyze the performance, and troubleshoot complex automations, all in one place
-
Builders want a powerful, well documented and efficient way to build, deploy, test, and troubleshoot advanced automations
-
The customer from our first use-case wants to simplify and eventually replace their current highly complex automation that addresses a few unpredictable situations. They needed a way to test and control the number of conversations affected by our solution in order to minimize impact on consumers and business,
-
Usually all users utilize a conversation transcript viewer as part of their workflow, in order to troubleshoot/refine intents, and bot dialogs in their automations
-
Competitors don't offer this capability
The proposed solution
Based on insights gained on how to solve the business and user needs, we (as a team) decided that for mvp we would:
-
Offer the ability to create rules based on 1 signal type (consumer intent) and 1 action (execute function) which would allow us to offer flexibility to solve a huge number of customer use cases from the start, as well as to launch quicker, so we could gather real usage data before investing in more capabilities. Unfortunately, it would not be a low-to-no code solution from the start as our users want. But I was committed to take this challenge to make that complex part of the experience as easy as possible.
-
Based on customer needs, introduce a gradual activation model, with an A/B test flavor to it. No other product within our platform offer this capability. It'd allow brands to test the dynamic actions performance and accuracy before affecting consumer conversations,
-
Offer insights with operational metrics, and logs to help builders to troubleshoot their rules.
-
Offer an in-product conversation transcript viewer to solve their frequent pain point of having to navigate all over the platform for this functionality. However, product and engineering were discussing whether we should build yet another transcript viewer from scratch, when users can go elsewhere for similar information. I pushed for offering it within our product because it is a more efficient, streamlined and improved customer experience. To me, it was clear that our users really valued this functionality, which would also help with product adoption.
2- Ideate and design
Product design principles applied to my designs
Based on insights from users, and my personal recommendation to elevate the quality of design at LP
Contextual
Help users and reduce complexity by showing only what they want when they want. E.g.: guided experience and tips that only appear when needed, extra data hidden unless clicked by user.
Simple
Less is more! Less steps, less pages, less texts, less clutter. More simplicity, easier and cleaner interface.
Consistent
Even though DA might not look the same as other products within the platform, within itself it is. E.g:. Active/passive components display consistent color palette across the product.
Accessible
DA was created using the LP new design system, which follows WCAG standards.
Innovative and engaging
We are not afraid to take risks! We are innovating with this product, setting new standards and UI patterns to be followed by other products moving forward! E.g.: gradual activation dial model. To improve brand consistency, I also created new illustrations for DA empty screens, based on LP 2022 brand strategy launched in 2022.
I worked closely with product manager during several brainstorming sections to define which features we'd offer to solve those user and business needs. I proposed a design that organizes the capabilities based on users' job-to-be-done, for a simple, easy to follow and scalable solution, as the product becomes more complex in the future.
As a result of our brainstorming sections, I've put together the long term vision Information Architecture for DA, and presented to stakeholders for approval since we would need a buy-in before proceeding. This was the base for our roadmap. After the approval, I created the Inf. architecture for MVP (shown below).
After those features got prioritized and agreed amongst leaders, they were added to the roadmap, I draw what the MVP experience would look like in a user flow document, in order to help the engineering team with their technical requirements, scoping and planning.
What should it be called?
In the meanwhile, marketing continued its discovery, so they led a brainstorming section to revise and define the product name which included me, product manager, and engineer manager. As a result, our product name changed from AI Triggers to Dynamic Actions. Because of this change, in order to maintain consistency throughout the experience, I led a discussion with my team to also change our policy unit, which was also called triggers. At first we changed to rules, but after a deeper investigation within our platform, we realized that it would make sense to call it dynamic actions.
Our logo
I wanted to create a "brand" for Dynamic Actions, so users would easily identify and distinguish our product amongst all the others shown in the Conversation AI hub page, which is also our product entry point.
My goal was to creatively represent DA's core functionality:
1. Constantly listens to conversations
2. Then, in real time, it triggers actions to these conversations
To make it more interesting and visually appealing, I decided to add animation in a loop which represents how our system listens to conversations, sends and receives data, non-stop. During the conceptualization phase, I gathered feedback from designers and stakeholders. After completion, I contact marketing to check for any compliance or branding issues. I was very happy how quick this got approved! :)
The next step was wireframing/ideating my vision. With the list of screens needed in hand, I started my early explorations, which I used during technical, estimation, and prioritization discussions with the rest of the team, resulting in more cuts/adjustments on scope as we learned more about the design.
For example, after technical investigation it was clear the challenge and high cost of making DA part of the core product as I wanted from the start. Due to tech stack differences and dev resources constraints, we decided that Dynamic Actions should be a separate app, which allowed us to build quicker, and be more scalable. Because of my long-term vision Information architecture document, the team knew how large and complex this product could become in the future, so we needed to be prepared. We also decided to remove non-operational metrics from MVP, which reduced the scope of the overview page significantly.
Example of wireframes and design ideations. Click on any image to enlarge.
DESIGN CHALLENGE #1: What should Dynamic Actions look like?
At that time, we were in transition between our old and a new design system, so my question was, should I go with the old one for consistency between our products, or should take the risk to use an incomplete and untested new one? I evaluated the pros and cons of both approaches, discussed technical effort with my front-end lead, and decided to become a pilot for the new design system. This would solve two problems:
-
DA would have a better experience, with more modern and cleaner look and feel compared to other products within the platform
-
I could test new components and designs before the DS was shared to all, and bring those feedback and learnings back to the team to improve our DS components
Win-win deal! My product can become the new standard and I can help to elevate the quality of design within LivePerson.
Also, using the components from the design system figma file allowed me to quickly create low-def mocks during the ideation phase, helping to sell DA with better visual quality to stakeholders early in the process.
After researching comparable products inside and outside LivePerson eco-system, I had a great understanding which patterns would work to address my user needs, and which ones I could re-use to maintain consistency within our platform experience. Each area of Dynamic Actions had its own challenges that required more or less ideation in order to find the best design. Below you can view some design objectives I had and some of the problems I was trying to solve.
Get started page
In the slideshow below you can see the evolution and some ideations of the welcome page.
I tried several different page layouts in order to find the best visual balance, content hierarchy while still solving user pain points. I tried adding illustration for additional visual appeal and brand recognition, since it follows the website style at that time.
I was not able to continue experimenting and validating this page, because I had to focus on the most important feature of the product: the new gradual activation model. Based on my experience, I decided to start with the simplest option for this page, and improve it once we have more user feedback from the early adoption phase.
Click to view ideations 1, 2, 3, and 4 of the Get Started page
The dynamic action creation flow
I established that this area should have:
-
less dependency on product documentation. This is a common pain point amongst our users so in order to improve our user experience, I added in-product guidance/tips, so the user has access to the info they need when they need during the task
-
easy to follow 3-step linear flow in a modal to create a dynamic action: Choose signal, Configure action, and Activate
DESIGN CHALLENGE #2: How can I make the "Configure action/Select a function" screen so user friendly that even non-technical users can succeed?
This screen is targeted to the builder persona, very developer-centric. I brainstormed with my dev to get his valuable insights on what a good UI would look like, and it ended-up very technical and confusing to me. I created a 2nd version with a more descriptive UI so I could test both versions. I also asked that we developed a template or code examples doc to display in the UI, so less technical users would not feel lost or discouraged during this step. My goal was to unblock as many users as possible during this step. Next, I just needed to validate the UI with our builders persona.
The Manage screen
In addition to the provided requirements, I established that this page should:
-
clearly indicate DA status, to help users to complete tasks and manage their DAs more efficiently
The Analyze screen
My goal for this page was:
-
to be the one-stop screen for troubleshooting and analyzing the DA performance
-
at first, display only the most important/actionable data, then, display specifics as per user needs/request
-
provide easy access to the conversation transcript to help users troubleshoot DAs and evaluate intent (signal) accuracy
Early example of wireframe/draft. Click to enlarge.
I decided to explore designs based on the Intent Manager app transcript experience because it is the 2nd most used in our platform (89% usage rate) and it is similar to what our DA users need. I improved the UX/UI and added a new pattern to indicate when signals got detected. Based on data from usability tests, the result was an improved, cleaner and more friendly viewer compared to the existing ones in the platform.
Platform existing pattern. Click to enlarge.
DESIGN CHALLENGE #3: Should we create our own conversation transcript window or reuse existing ones?
Based on my platform research, I found out that currently, our users have 4 different ways to access conversations transcript (1 via API, and 3 via UI in different products). And DA was about to create a new one!
Immediately those questions came to mind: Is it possible to repurpose one of them, to save engineering cost? Would it degrade the experience? Is there any DA specific functionality requirement that those don't offer? The answer for those were: no, we can't repurpose because DA is built in React, and those other ones were not. And DA transcript would need customization to add different elements/functionalities not shown on the other ones anyway.
3- Validate
My approach was continuous validation, even if the design was not complete. At first I focused on the builder persona, because I had access to a good number of devs. I gathered quantitative and qualitative data. The other studies were more qualitative. After each study I went back, ideated different solutions, designed it, and tested it again, until I reached at least 80% success rate on the most important areas of the product. Then, the MVP design was ready for development and approved by leadership/stakeholders.
Total of 26 participants (A/B test, survey and usability tests).
Study #1
Who: 12 users: 6 survey + 6 UT interviews
What: A/B test - select function screen, Get Started, Create DA modal and Manage page v1
A/B test results:
Tab design: 3 (25%)
Stacked design: 7 (58.3%)
No preference/other: 2 (16.6%)
Learnings:
-
Design suggested by developer was the winner.
-
Almost all builders asked for some guidance or examples on how to use the parameters field.
-
Even Sr, developers might not be familiar with LP functions. Don't assume all devs do.
-
All other areas of the design was a success with only minor changes needed.
Study #3
Who: 3 UT interviews
What: Activate step v2, Manage page v3 (w/ insights as a drill down page)
Learnings:
-
Activate screen is less confusing but also less appealing. Still needs improvements.
-
Insights page can be simplified with less content.
-
Minor verbiage improvements needed in other screens.
Study #2
Who: 5 UT interviews
What: Activate step v1, Manage page v2, Insights and analyze page.
Learnings:
-
Activate v1 design failed. Gradual activation concept is complex, unfamiliar and hard to understand.
-
Graph colors must be updated.
-
Logs needed extra data.
-
Free vs premium experience confusing/unnecessary.
-
Some verbiage and design system components issues.
____________________________________
Quotes about the gradual activation concept:
-
"I feel dumb... not sure what all of this mean"
-
"What does 20% active mean?"
Study #4
Who: 6 UT interviews with internal and external users
What: Activate step v3, Manage page v4 (w/ insights as a side panel)
Learnings:
-
Activate screen is clearer. Passed.
-
Tips throughout the experience is super valuable and necessary to unblock users.
-
View transcript functionality is highly valuable and desired by users.
-
Combining insights with the manage screen provides a simpler and cleaner experience. Success.
4- Build and Release
During the building phase I constantly synced with engineering and reviewed their UI/functionality progress, provided support on any missing designs needed, and helped with UAT. During this phase, the common challenge is to build/deliver what was promised, as resources and timelines change. During the team leads meeting, I guided them on which scope can be reduced, and what needed to be developed with the same quality as I designed.
At that time I also reviewed our early adoption documentation, and created the vision, the script+storyboard, and managed the end-to-end process of the Dynamic Actions overview video. The challenge was to secure marketing resources for this project and put our video on their roadmap. I was able to be flexible and build a successful relationship with that team, which allowed me to use them for script review and video production. I worked closely with the videographer, directing him on how to reach my vision/standards regarding the animations and narration. You can see the final video on the top of this page. : )
A full functional prototype of the final design can be presented upon request only.
5- Test and Analyse
For the next steps, I put together a usability test plan to gather quantitative and qualitative data to capture "how all 3 personas use Dynamic Actions" during the early adoption phase. I will use triangulation between Google analytics metrics and Usability Tests interviews to identify and adjust any issues on the experience before we go live in GA later next year.
Testing hasn't started yet.
TESTIMONIAL
“When I demoed Dynamic Actions to my brand, they were absolutely happy, and had no negative feedback. They loved how clean and simple it looked. They were happy how even non-technical users can easily do the setup with guided help throughout the experience."
T.W. Customer Success Mng. for large Telecom in Australia