Inside Rakuten AI: Zoey Zhao on AI agents and the quest for trust and usefulness

In this series, we sit down with Rakuten AI leaders for a deep dive into the stories behind this transformative technology and the inspiring individuals driving Rakuten’s vision of AI for all. Watch this interview and others in the Inside Rakuten AI series on our YouTube channel.

Artificial Intelligence has come a long way from simple chatbots to intelligent agents who can orchestrate workflows with multiple tasks, interpret intent based on context, and reshape how we engage with digital ecosystems. As a leader of homegrown AI solutions for the Japanese market, Rakuten is building its own AI agent platform that doesn’t just answer questions but understands the context of life in Japan and beyond.

On July 30, Rakuten announced the full-scale launch of Rakuten AI, a new, cutting-edge AI agent platform designed to enhance user experiences and streamline interactions across the Rakuten Ecosystem, as well as enrich and empower daily life.

We spoke to Zoey Zhao, Senior Product Manager at Rakuten AI and Data Division, who is deeply involved in Rakuten AI’s development. With a unique background in aerospace engineering, a deep interest in Natural Language Processing (NLP), and a passion for user-first design, she offers rare insight into how the product evolved from a helpful assistant to an intelligent and understanding AI agent platform for one of Japan’s most diverse ecosystems.

From engineering drones to building chatbots

Many AI professionals come from computer science or data science backgrounds. But Zoey’s route to tech began in the skies. “I did my bachelor’s and master’s degree studies in Singapore, but my undergrad study was actually in aerospace engineering. I started out my engineering career by building drones,” she said.

Her transition from aerospace engineering to AI began when she landed a job at a research lab jointly funded by Nanyang Technological University and Singtel. There, she joined the voice and NLP team.

“We were developing a smart transcript tool for call centers. The goal was to use AI to summarize calls, auto-file tickets, and make post-call services smoother. Looking back, it’s very similar to what Zoom, and other online communications platforms are doing today,” she recalled.

But Singapore’s unique linguistic landscape and cultural context posed new challenges for the young researchers. AI transcription engines built for American or British English struggled with Singlish-a local English variant filled with words from Malay and Chinese dialects, unique grammar, and lightning-fast speech patterns.

“Market solutions from global leaders just didn’t have the accuracy. Plus, the data was sensitive, so we couldn’t use cloud-based options,” she said. “We had no choice but to build our own voice engine with custom models.”

This experience gave her a front-row seat to the challenges of building tech from the ground up and deploying custom AI in the real world, a valuable lesson for her new career in product management.

Localizing AI products for Japan

In 2021, she joined Rakuten’s Asia Incubation team in Singapore, initially working on recommendation systems and visual search. But her interest in NLP continued, and she became involved with the original Rakuten AI Assistant project, helping shape it into a multi-functional tool that’s rolled out across the ecosystem, evolving into the Rakuten AI agentic platform.

“I joined the original AI Assistant team last year, before the annual Rakuten Optimism conference event. At the time, we were still validating pain points, doing interviews, and trying to find a product-market fit,” she recalled. At the 2024 event, Zoey joined Rakuten Group Chief AI & Data Officer Ting Cai onstage to demo the newly developed AI prototype. 

In addition to the Japanese market, the primary target market for Rakuten AI, another user demographic that Zoey’s team identified was a deeply underserved group in Japan: inbound visitors and non-Japanese residents.

She gave an example from her own experience, “If you go to a Japanese izakaya pub-restaurant and look at the handwritten Japanese menu on the walls, you don’t even know what to search online. You can’t type in Japanese, you can’t read it, and you don’t want to disturb others by speaking into your phone.”

To solve this, they developed a feature that interprets images and provides translations or contextual explanations without requiring the user to type or speak.

“We call it image analysis. You just upload a photo, and the AI tool tries to interpret what you need. It translates the text, reads the prices, even describes the dish. It’s not always perfect, but it helps a lot, especially for menus or product labels.”

This visual-first approach fits Zoey’s personal preferences as well, “I’m a very visual person. Sometimes I just take a picture and ask the AI to help me write a caption for Instagram. That’s our ‘Snap and Ask’ feature on Rakuten AI.”

But building for Japan’s domestic market comes with a whole different set of challenges. It’s not just about translation, it’s about lifestyle and behavior. Japanese consumers are hardly monolithic when it comes to tech adoption. “Some Japanese users are still using flip phones. Others are hardcore TikTok creators. Some won’t use credit cards, while others live entirely online. It’s incredibly diverse,” she explains.

Agentic gateway into the Rakuten Ecosystem

From the outset, Rakuten AI was never meant to be just a chatbot, rather the goal was for it to evolve into a central interface or hub that connects users with a wide range of Rakuten services, from online shopping to books, travel and even essential services like utilities.

She gave another personal example, “I moved from Singapore to Japan recently. During my move, I needed to set up my gas, electricity, internet services and it turns out that Rakuten has services for all of these. So, imagine if one AI platform could help you apply for everything through one chat.”

Currently, users can use Rakuten AI for personalized recommendations in shopping, fashion, music and other e-commerce services. But the goal is much bigger: to make Rakuten AI the premier gateway to the entire Rakuten ecosystem.

“To the user, it should feel like one entry point to everything Rakuten offers. That’s powerful,” she emphasized. “Agents can not only understand and connect you better, but they can also take actions for you so you can get more out of life.”

Quality control is also a key priority, with dedicated teams checking for everything from factual accuracy to offensive or harmful outputs. “We do unit testing, but we also test whether the reply is relevant, up-to-date and sounds natural. We check if the product suggestions match the query. And of course, we watch for hallucinations and other safety issues.”

From helpful assistant to trusted agent

Looking toward the future, the conversation naturally turns to one of the hottest topics in AI today: agents. So, what makes agents so exciting, and how are they different from assistants?

While assistants take orders, agents take action, moving from tasks to goal-oriented scenarios. The leap from assistant to agent may seem subtle, but it requires a radical change in architecture.

Zoey shares her take on the topic, “To me, agents are more domain specific. They have deep expertise in a certain vertical. But you also need a system that can smartly route the user’s query to the right agent.”

This routing layer, or what some call the “orchestrator,” is key to scaling usefulness without compromising accuracy.

“One complaint about AI assistants is that they’re too generic. Experts in a field don’t get real answers. With multi-agent systems, each agent focuses deeply, and another layer decides who answers what.”

But with great complexity comes great computing demand. “In the development stage, AI can be more unpredictable. You might ask the same question twice and get different answers,” she explained. “That’s not like traditional engineering, where things are binary. That’s why we need strong quality and infrastructure support.”

As a product manager, Zoey doesn’t consider technical issues as the biggest challenge, “It’s going from zero to one and finding the right use case that enough people care about and will use regularly. Technology can do a lot, but you need to solve a problem that’s real, valuable, and scalable.”

Zoey and her team are now focusing on scaling usage and refining their features based on real feedback. “Once we get more users, we get more data. And with that, we can iterate better and faster.”

Advice for women in tech or AI newbies

As a woman in AI product development, she’s keenly aware of the underrepresentation of people like her in the field. But her advice is refreshingly grounded: “Don’t overthink it. Just focus on solving real problems. The rest will come. All you need is curiosity and a laptop,” she emphasizes.

Her own experience in aerospace engineering taught her how technology levels the playing field and gives everyone an equal opportunity to succeed.

“When I was building drones, I had to carry mechanical parts and drill things. I wasn’t physically able to complete some tasks on my own. But in AI and software, there’s no such limitation. It’s an equal opportunity playground.”

Zoey is looking forward to the next iterations of Rakuten AI, “We’re exploring developing even more in the multi-agent direction. But for now, our goal is to find the right need, build the right product and scale it. That’s the most challenging and the most exciting part.”

Whether it’s snapping a photo of a handwritten menu, coordinating an outfit, planning a trip or setting up a home from scratch, Rakuten AI is learning and growing alongside its users. And behind it is a product manager and team who are doing exactly what tech was meant to do: solve real problems for real people.