Ting Cai on Rakuten’s embrace of AI 2.0

“There is no AI without humans. But humans with AI can achieve so much more.”

Speaking at the most recent Rakuten Technology Conference, Chief AI & Data Officer Ting Cai shared Rakuten’s strategic vision for artificial intelligence and addressed the unprecedented scale of industry investment, which has reached into the hundreds of billions of dollars.

Why are all these companies putting so much money into AI? Are they just throwing spaghetti at the wall to see what sticks? Or are they putting money in because of competitive pressure?

Cai highlighted a changing dynamic in the world of AI: While pre-2023 efforts centered on hardware development, model architecture and basic applications, the industry has now entered “AI 2.0” – a phase focused on practical implementation across diverse sectors.

“We are actually doing both,” Cai said, highlighting Rakuten’s dual approach of developing proprietary large language models while leveraging its ecosystem for wide-ranging AI applications.

Building AI into the foundation

“At Rakuten, we love to build things from the ground up. This foundational understanding of AI allows us to amplify its impact. That’s why we leverage our data to build our deep learning foundation.”

Using a technique called embedding, Rakuten can understand the fundamental nature of every person, place, thing and interaction in the ecosystem. Powerful AI – Rakuten’s deep learning foundation – understands how all of those things relate to each other, giving the company unique insight into what customers want and need. It also provides the data foundation to build language models, which enable customers and business partners to engage with Rakuten’s AI tools and services using natural language.

Rakuten’s large language models

In March 2024, Rakuten launched its first large language model (LLM), Rakuten AI 7B, which was followed just a few months later with Rakuten AI 2.0, the company’s first mixture-of-experts model, and Rakuten AI 2.0 mini, a small language model (SLM) designed for secure use on the edge. All three models were trained extensively on Japanese data and context to capture Japan’s unique linguistic, cultural, aesthetic, and regulatory nuances, and delivered best-in-class Japanese language performance at launch.

“Building a large language model is not a trivial undertaking,” Cai remarked. “It takes significant effort to improve data quality through: cleaning, filtering, identifying unique data, and removing duplicates or unhelpful information. It’s also very much about algorithms, models and tokenization – how to encode different languages efficiently, create new architectures and innovate beyond transformers.”

Building in-house language models is core to Rakuten’s AI strategy of ensuring that engineering teams and business customers have the AI tools they need to make smart tradeoffs between performance, security, latency, and cost.

“As you apply AI in various applications, it’s not enough to only use off-the-shelf models. You have to integrate with enterprise knowledge and your unique, personal context,” Cai told the audience. “So, how can you integrate all of the enterprise knowledge, your personal preferences and your conversation history into the language models so you can retrieve this information natively? Only after that, I believe, can you create truly useful applications in the era of AI 2.0.”

Leveraging Rakuten’s data for powerful search

One such application is semantic search, which uses the power of AI to understand what customers mean, not just what they say. Cai’s team rolled out Semantic Search to 11 Rakuten services in 2024, and the technology is already delivering results. Semantic Search reduced “zero-hit” results – a search with no relevant results – by up to 98.5%. 

“We launched semantic search last year, and now we are rolling it out to all Rakuten group services. Just in the second half of 2024, we launched semantic search on Rakuma, Rakuten PLAY, and many other services,” Cai explained. “By reducing the zero-hit ratio, we can show more relevant content. Users will then engage more, come back more often, and purchase more, helping our business grow and bringing more value to our merchants.” 

Cai’s team is also scaling the power of deep learning to deliver more relevant recommendations across 7 Rakuten properties, and introduce a new ad experience that’s more relevant, personalized, and useful.

Empowering businesses, empowering consumers

“At Rakuten, we have a vision to augment human creativity with the power of AI. We’re going to truly leverage our strengths in data, channels and, most importantly, people, organization and culture,” Cai said. “It’s more than just technology. We need to think about products, business execution and efficiency to pass real benefits to our customers.”

Rakuten’s unique ecosystem positions it to deliver personalized services across multiple sectors.

“The Rakuten ecosystem is really the only one of its kind in this world. I have never seen such a broad product portfolio in any other company. The spirit behind it is that we think about our customers and about what they need. If there are opportunities for us to create new value, we will go after that.”

It’s an approach Rakuten has taken time and time again – most recently in the telecommunications sector.

“Rakuten Mobile is a great example where we saw an opportunity to lower the cost for consumers on mobile subscription costs by leveraging the latest technology in cloud computing, virtualization and Open RAN. Now, with AI, we can offer cheaper and better services for our customers.”

The link to enhanced customer experiences

In November 2024, Rakuten Mobile introduced Rakuten Link AI, a new chat-based service which makes the full power of generative AI accessible to every Rakuten Mobile customer. With no new app to download, no new account to create, no additional subscription fee, and no complex commands to use, Rakuten Link is a powerful, easy way for everyone to experience how generative AI can augment their creativity and productivity.

“We launched Rakuten Link AI, which offers cutting-edge AI technology to millions of Rakuten Mobile users,” Cai remarked. “Technology alone is not enough – we need to create products that solve user problems, develop sustainable business models and make the technology accessible. That’s the idea behind Rakuten Link AI: providing low-cost services through the latest AI innovations.”

Of course, Cai’s vision of AI empowerment extends beyond customers to include Rakuten’s business partners as well.

Following the success of Rakuten Link AI, Rakuten Mobile launched Rakuten AI for Business in January 2025. The service, which allows companies in Japan to take advantage of generative AI, regardless of technical proficiency, boasts advanced language comprehension and task processing for business tasks such as translation, brainstorming and analysis. Another key advantage: Offering prompt templates that leverage a range of successful AI use cases from across Rakuten Group to boost efficiency and productivity.

“A couple of months ago, we also launched Rakuten Analytics. This is another service we provide to our business partners, that enriches client’s data analysis with Rakuten Ecosystem data. By leveraging AI to analyze client data and tapping into the Rakuten Ecosystem’s data power, we can explain the data in intuitive and easy-to-understand ways that help businesses engage with their customers more effectively while growing their revenue.”

Eating our own dog food

“This concept is truly important to me: As Rakutenians, we need to eat our own dog food. This means using our own software first to understand user pain points. By doing this, we can concretely identify what needs improvement.”

Over 30,000 employees have already utilized Rakuten AI for Rakutenians across multiple areas, including coding, image generation, knowledge retrieval, translation, meeting summaries and more. Now, more than 8,000 people are using it every single day. 

“By experimenting with these tools ourselves, we not only learn what’s possible but also how to refine and improve them.” Teams are also using Rakuten AI for Rakutenians to build low and no-code solutions that solve real-world problems. 

The Rakuten Advertising team at Rakuten International, for example, recognized that it was taking too long to migrate partners from a competitor’s ad platform to Rakuten. Using the power of Rakuten AI for Rakutenians, they streamlined migration of clients to the Rakuten Affiliate Network by creating a script code which reduced turnaround time by 85% and improved match rate by 15%. Similarly, they’re leveraging the power of LLMs to automatically craft more compelling, personalized titles for the thousands of products that hit a user’s feed. 

AI-nization is for everyone

Cai reiterated his vision of humans and AI working closely together, revealing that he regularly reminds his team of the importance of balancing innovation with practical results.

“We aim to leverage AI to augment human creativity. Rakuten has a long tradition of innovation. Many years ago, we rode the wave of the internet to create the Ichiba marketplace, connecting small merchants with customers. Today, we are riding the wave of AI to create even more value for our customers and society.”