How are AI voice agents changing customer service? Rakuten Mobile collaborates with OpenAI to integrate new technology

Today, a striking 76% of consumers still prefer calling customer service over the phone – but what if the voice on the end of the line isn’t human?

A new generation of AI voice agents are making their way into customer service operations, and they sound more human-like than ever before. At the forefront of this innovation is OpenAI, who introduced Realtime API in October 2024 – an advanced AI model that brings fluid, intuitive and proactive speech-to-speech conversations between businesses and their customers.

This new wave of AI-driven voice technology promises to reshape customer experience. Long wait times, often cited by 60% of consumers as the most frustrating part of service calls, could soon be a thing of the past. With AI voice agents available instantly, customers can now enjoy faster, smoother and more satisfying interactions with businesses.

Bringing AI-powered voice assistants to 10 million customers across Japan

At Rakuten Mobile, where around 30% of customer inquiries still happen over the phone, experts turned to AI to help bridge the gap between the business and its customers. Leveraging OpenAI’s Realtime API, the organization developed an AI-powered voice assistant making customer support quicker and more accessible for over 10 million subscribers and new customers across Japan.

The new voice agent integrates Realtime API with Rakuten’s homegrown Large Language Model (LLM), Rakuten AI 7B, to provide seamless, natural conversations in both Japanese and English and context-aware responses based on data from Rakuten Mobile’s business knowledge and customer interactions.

The new feature enables quicker signup, onboarding and other customer support cases for Rakuten Mobile’s services and is available to the public within Rakuten Mobile’s website and mobile app. Users can experience natural conversations with an agent that mirrors the experience of a human voice agent.

“Today, a third of our customer support cases are (still) from phone calls. By combining Open AI’s advanced speech-to-speech model with Rakuten AI 7B, we are able to provide empathetic, human-like voices that meet Rakuten Mobile customers where they are and solve their problems efficiently,” says Scott Wu, General Manager of AI for Customer Service and Director of AI Safety, Rakuten Group.

Using OpenAI’s Realtime API to build an advanced AI voice agent

Open AI’s Realtime API provides direct speech-to-speech interactions, as well as multimodal inputs (audio, images, and text) and outputs (audio and text), enabling fluid AI conversations that feel natural. With Realtime API, developers can build and successfully deploy voice agents that enhance customer service experiences for their businesses.

Traditionally, voice assistants have relied on multiple separate models – one to convert speech to text, and another to turn text back into speech. Realtime API takes a different approach. By handling both processes within a single model and API (application programming interface), it dramatically cuts down on response time, preserves the subtleties of human tone and delivers conversations that sound smoother and more lifelike.

How Rakuten AI 7B powers continuous learning and customer-centric AI

Behind the new voice assistant is Rakuten AI 7B – a high-performance, open large language model optimized for the Japanese language. The LLM is trained on English and Japanese internet-scale datasets, carefully curated and cleaned through an in-house multi-stage data filtering and annotation process. This ensures high quality, contributing to the enhanced performance of the models.

At launch, it achieved top marks among open-sourced Japanese LLMs in the LM Evaluation Harness benchmark – setting a new standard for Japanese-language AI innovation.

Rakuten’s engineers continuously refine the model by annotating new data and integrating customer feedback, ensuring it remains accurate, adaptive, and closely aligned with real-world interactions. This enables Rakuten Mobile’s AI voice assistant to improve over time and become better at understanding, anticipating, and addressing customer needs.

Security and privacy are fundamental to this design. By building the system on Rakuten AI 7B and hosting it securely within Rakuten’s own secure infrastructure, Rakuten Mobile ensures that all sensitive customer information remains protected. At the core of Rakuten’s AI strategy is the decision to build large language models in-house – allowing the company to carefully balance performance, security, latency and cost, while empowering engineers and business teams to scale AI-driven innovation across Rakuten’s 70+ businesses and services.

A new model of customer support in the AI era

Rakuten Mobile’s adoption of AI technology marks a new era in customer support where AI seamlessly bridges the gap between digital and human communication. Beyond voice agents, Rakuten is dedicated to advancing diverse AI solutions across its various business domains.

At Rakuten AI Optimism 2025, Rakuten Group Chairman and CEO Mickey Mikitani announced the launch of Rakuten AI, an intelligent agent seamlessly connecting Rakuten users across its various businesses and services from fintech, e-commerce, travel to telecom. With Rakuten AI, customers can browse for products on Rakuten Ichiba, generate videos and images, transcribe meeting notes, create personalized travel itineraries, and more – all within a matter of seconds.

Rakuten AI is available to the public at www.rakuten.ai.co.jp.

This holistic approach to AI, developing agentic AI services powered by the best partner and in-house technology, underscores Rakuten’s commitment to not just enhancing customer service, but fundamentally transforming the user experience.