In the event Design User Experience Driven by AI, Paul Lambert (Product Manager) and Annika Crowley (Interaction Designer) from Google gave us great talks on how Gmail designed the Smart Compose feature, and five strategies to create great user experience with AI (see sketchnotes).

I remember Smart Compose was a pretty hot topic when it was launched. It came out right around the time that people started to pay more attention to privacy. Also, a composer empowered by AI? How could that work — is Google going to compose emails for you?

So far, I’ve used Smart Compose many times and found it very convenient. There were a few times, it really surprised me with some suggestions (both pleasantly and not), because it seemed that it knew the content on what others responded in my email chain.

As a long-time Gmail user, privacy-wise, I’ve always been suspicious about Gmail’s capabilities to read and analyze our emails, given that its in-email Ads seem to be somewhat powerful and targeted. Obviously, I would not be able to find out answers from this event, but it’s good to listen to the Gmail team’s philosophy behind this feature.

Having worked on an AI-empowered product before, I found it took a lot of effort to win trust from users. When it comes to AI, users have high expectations to have a “natural” experience. What if your AI technology is not that smart?

In my opinion, the Smart Composer design represented very well the smart composing capability backed by AI. As a product designer, I’m curious about how the Gmail team approached this project, and the process they’ve been through to land on the final design direction.

The Gmail team shipped Smart Composer following this formula:

• Understand user problems
• Define success
• Iterations: Design -> Build -> Evaluate

Understand user problems

It all started from a problem: “How could we help people reduce time in emailing?” because:

• Email is time consuming.
• Professional writing is hard.

Smart Compose was not built in a day, or was even built in the first phase. The team started with a smaller project: Smart Reply. It is a quick and easy way for users to save time for short replies. Users could go with the suggested replies, or use them as a starter. 

Define Success

Once the team saw the usage of Smart Replies soar, they started working on Smart Compose.

The team sets Satisfaction Rate as their success metric. They wanted happy users. Only keeping a high bar can they achieve that.

Their initial test got a 40% satisfaction rate. This was far away from the team’s expectation. They’ve done some work and v2 bumped it to 70%. However, it was not until v3 with 80% satisfaction rate did they feel comfortable to release the feature. From there, happy users could help Smart Composer be smarter.

Iterations: Design -> Build -> Evaluate

Since people have high expectations for AI, the team started from “what’s the cheapest thing to create something magic?” and work their way up. The speakers went through a number of design directions they did and tested, and shared some really good learnings:

• The team discovered biases from the suggestions Gmail made, so they started paying attention to reducing bias and increase fairness.
• Outliers are guaranteed. The team approached this project in a conservative, and defensive manner, such as avoiding sensitive topics.
• It’s not for everyone. They acknowledge the fact that not everyone likes to have AI compose emails, so they let users turn off the functionality if needed. 

The team’s take aways:

• Maximize impact: On by default. Toothbrush journeys.
• A shared philosophy around product development process is important.
• With ML features, UI can be only a fraction of the UX.