From vision, to strategy, to execution: Ting Cai on building useful products with AI

The world of AI moves fast. In the blink of an eye, the latest GenAI models have become multilingual, understanding different dialogues and even emotional tones; and multimodal, accepting and outputting audio, video and more.

In such a rapidly changing landscape, how does one effectively build AI products? Rakuten Group’s Chief AI & Data Officer Ting Cai joined the 2024 edition of Rakuten Product Conference, Presented by Rakuten India, to share his perspective.

“Product managers are the catalysts that turn vision into strategy, strategy into execution, and execution into value,” Cai told the conference. “How can a product manager stay concrete, live and breathe like a user, but at the same time understand the technology so they can integrate what is technically possible, what the user wants, and come up with a vision and define the product?”

Start with a vision

“Rakuten has a lot of businesses across multiple industries,” Cai began. “This gives us fantastic unique data assets we can leverage to train efficient embeddings and efficient large language models (LLMs). Our data is like a rich farmland; with it we can carefully cultivate and grow our AI products. We have online and offline channels that reach millions of customers and businesses so they can take advantage of large language models.”

“Product managers need to think big to solve hard problems. They’re striving to accomplish mission impossible.”

Ting Cai, Rakuten Group Chief AI & Data Officer

Through its AI-nization initiative, Rakuten is looking to take full advantage of AI technology across the company, leveraging its advanced capabilities to deliver more value to Rakutenians, partners and customers.

“Our vision is to enable AI-nization across Rakuten by leveraging our data assets, our channels and ultimately building the flywheel so we can leverage every interaction with our customers and businesses to enhance our products so we can deliver more benefits to our customers.”

For product managers, it’s a tricky balance between dreaming big and staying grounded.

“Product managers need to think big to solve hard problems. They’re striving to accomplish mission impossible. If you solve an easy problem, you’re already subject to disappointing others because many other people are solving the same problem, and it’s hard to differentiate,” Cai told the conference. “We need to think big, but at the same time stay concrete. Be practical, think what AI can do to improve our everyday products.”

Turning vision into strategy

Cai presented five key approaches for product managers looking to leverage AI.

1. What should AI do?

“In order to come up with a good vision, it’s very important to start with an understanding of our customers. There’s a lot of conversation about the latest LLMs, what AI can do. But most importantly, in my opinion, is what AI should do. What are the right problems to solve?”

Understanding the user’s needs is the first step to building a good product. This is something Cai has pursued at Rakuten through a technology called semantic search on several of Rakuten’s services. This tech seeks to produce search results close to what the user means instead of the literal search terms used.

“In the world of search, a query represents user intent. So, query analysis is super important to delivering a top-notch search experience,” Cai explained. “We’ve been working on semantic search for Rakuten Fashion and Rakuten Ichiba, and we are expanding it to multiple products across the Rakuten Group. All of that is building a strong foundation for understanding the user’s intent through search query analysis.”

2. Actions speak louder than words

“It’s not enough just to do a survey and ask what users want. It’s more important to observe what they do,” Cai stressed.

“Of all of the data collection tools I’ve worked on, including search engines, the user survey is probably the least important. It’s more important to look at the actual user behavior. What are the things they query, what are the things they click, and what pages do they spend more time on? Observing what users do can help product managers identify the latent intent.”

“To be a good product manager, it’s very important to demonstrate leadership – to have a vision, bring clarity and focus on the big bets for the entire team.”

3. Integrate AI into existing habits

Cai spoke of some recent work undertaken with healthcare professionals to explore how AI could help in the medical field. He cited the example of pathologists trained their entire lives on microscopes, only to be told to use cloud technology and AI to analyze their samples.

“It’s like changing their life for them. They spend years going through this medical training, and their life is attached to this microscope,” he remarked. “In order to lower the barrier of adopting AI technology, we have to integrate it into existing habits.”

4. Does it have to be AI?

“Does this problem need an AI? Or is there a simpler solution?”

Cai cited the example of someone in a recent Zoom meeting who had a timer as their background – an elegant solution to managing precious meeting minutes.

“Can AI solve a problem differently? I have been in those meetings sometimes when people are passionately discussing a topic, when suddenly the human timer said, Time is up; you should stop! But maybe AI can actually detect the emotion, detect the discussion and let people continue their flow.”

5. Instill trust

Last but not least, build a brand that customers trust. “All of these visions are synthesized into product branding. Branding instills trust and emotion. It can deliver premium value for your product.”

Turning strategy into execution

“To be a good product manager, it’s very important to demonstrate leadership – to have a vision, bring clarity and focus on the big bets for the entire team. But it’s not enough to have only the vision and strategy. Most importantly, it’s about the execution.”

Iteration and finetuning are essential components of a product’s execution, especially in the fast-paced world of AI.

“As you can see, these AI developments are happening very quickly. So, I believe it’s important to develop the inner loop so you can experiment faster,” Cai emphasized. “Even when a product is developed, you can continue to tune it through in-context learning, through prompt engineering, through retrieval augmented generation. There’s so many things you can continue to iterate through the development process.”

During this iterative process, product managers must continue learning and trying new tools to get a feel for what can be applied to their products. “That in-depth technical knowledge, plus an understanding of user needs, in a continuously iterative fashion is the way to go for product development.”