Inside Rakuten AI: Maksim Tkachenko on the path to personalized AI with efficient Japanese LLMs

In this series, we sit down with Rakuten AI leaders for a deep dive into the stories behind this transformative technology and the inspiring individuals driving Rakuten’s vision of AI for all. Watch this interview and others in the Inside Rakuten AI series on our YouTube channel.

Large Language Models (LLMs) are at the forefront of the rapidly evolving artificial intelligence landscape. These powerful models are the engine driving AI and are reshaping how we interact with technology, promising a future where AI is not just a tool, but an intelligent force multiplier.

At Rakuten, a team of researchers is thinking outside the box and leveraging breakthrough innovation to push the boundaries of LLM development, aiming to create highly efficient models that leverage innovative architecture to understand user needs and become deeply integrated into their daily lives.

Leading this relentless drive is Maksim “Max” Tkachenko, General Manager of Frontier Research Department at Rakuten. His journey into AI began with a fascination for Natural Language Processing (NLP)-the intricate dance of explaining the fluid nature of human language to a computer through the rigid structure of mathematics.

Applying mathematical precision to language processing

Maksim “Max” Tkachenko, General Manager of Frontier Research Department, leads Rakuten’s LLM research

“Language is a very fluid concept,” Max explains. “But you actually can add a lot of structure to it. And that’s where it gets really interesting. So, how do we explain the fluidity of language with the rigidity of mathematics? This question propelled me throughout my years of research as I went on to do a PhD in language processing.”

Even in his current role overseeing Rakuten’s ambitious LLM initiatives, Max maintains a clear focus on the core principle behind LLMs. “Language processing is an area of research that has always been changing: now we have language models. When we talk about training these models, that doesn’t mean we just tell the model what a noun is and that it has to be followed by a verb, but that was the case before. Now, we don’t need to describe to AI models how human language works. Rather, we can give it a big collection of texts and challenge it to try to figure out how it works.”

“In this way, AI models learn by performing a simple task: predicting the next word in a sequence,” he explains. “This seemingly straightforward process has proven to be an incredibly effective way to train a type of machinery that enables AI to do quite intelligent things with human language.”

The training process for LLMs unfolds in two key stages. The first involves the model learning the inherent structure and operation of language. “That’s just part one,” Max clarifies. “The second part involves demonstrating to the language model how to function and how to approach specific tasks. This is achieved by providing the models with annotated data, like an English sentence paired with its Japanese translation, or examples of how to write an email. Through this process, the LLM learns to simulate an annotator, developing the ability to perform a wide array of tasks and engage in more directed communication.”

Rakuten’s unique advantage: A data-rich ecosystem

Rakuten is uniquely positioned to understand and address vertical tasks in many businesses and further develop general intelligence

Max joined Rakuten in 2022, initially focusing on machine translation. However, the emergence of easy to use and accessible generative AI technologies like ChatGPT was a watershed moment for the AI industry and a golden opportunity for him shift focus towards LLM development.

Today, he oversees a team dedicated to training and optimizing Rakuten’s own powerful models. This is a fundamental role with potential for outsized contributions and requires both strategic vision and deep domain knowledge. “Overseeing the entire language model development process involves multiple aspects including architectural design choices,” he explains. “We need to think about how we collect data and annotations.”

The diverse Rakuten Ecosystem, encompassing over 70 services in e-commerce, finance, telecom and more, provides a significant advantage in this endeavor. When discussing this unique positioning, Max starts from a high level, “Let’s step back a little bit and try to define what exactly is intelligence in language models. We can define intelligence through the set of different small tasks. We can categorize these tasks into different verticals.”

“So, for each domain we need to think about what kind of tasks we want to solve to achieve intelligence within this specific vertical, be it e-commerce or banking, and as we go on accumulating verticals we aggregate them into an intelligent machine that generalizes across multiple domains. So, in this case, I think Rakuten is uniquely positioned to understand and address vertical tasks in so many businesses and further develop general intelligence.”

The ability to leverage user behavior data, while safeguarding user privacy, is another key differentiator. “We can also access and gain understanding of an immense amount of data used in development. This data and domain knowledge comes from operating the businesses ourselves,” Max said. “For example, we can analyze purchase patterns to better understand how certain products relate to each other in the customer journey. This insight allows us to enhance personalization in our LLMs, enabling more relevant recommendations and tailored interactions.”

Driving towards personalized memory and lifelong conversations

Memory is a important key to personalization enabling concierge-like experiences with AI agents

As a leader, Max sets clear objectives for his team’s LLM development with an ambitious core mission, “We aim to deliver best-in-class Japanese large language models that understand the context of communication, cultural nuances of Japanese and deliver a personalized experience.” This overarching goal is pursued through aggressive innovation, constantly exploring cutting-edge research and making strategic architectural decisions to tailor LLMs to specific business verticals.

When asked about the key performance indicators (KPIs) for his team, Max emphasizes a focus on output rather than just speed of launch. “Our LLM KPIs are performance based,”he said. “Basically we’re looking at how accurate we can communicate in a variety of contexts, how accurate our personalization is.” This highlights a commitment to quality and relevance in every interaction.

The concept of personalization, a recurring theme in this discussion, is crucial. It’s not merely about the language an LLM uses, but its ability to deeply understand and match a user’s intent. Max provides a clear example, “If you’re talking to an AI assistant about ordering dinner but haven’t specified your preferred cuisine or delivery time, the chatbot will ask clarifying questions. However, if the system already knows your favorite restaurants, dietary restrictions, or usual order time, it can instantly suggest the perfect meal with minimal back-and-forth.”

This ability to “remember” previous interactions is paramount. Max paints a compelling picture of the possibilities of an intelligent agent with long-term memory, “For example, perhaps a year ago you mentioned that your anniversary is coming up and you’re looking for a gift. Why can’t we have a language model that remembers this detail and reminds you in advance?”

He imagines a scenario where an AI agent can tell you when your anniversary is coming up, then go ahead and recommend gifts and even tell you what you bought last time to avoid giving the same one again. This “memory,” he asserts, is a very important key to personalization as it enables concierge-like experiences where AI agents anticipate user needs across various aspects of their lives, from travel to shopping.

The Future is long context and architectural innovation

Rakuten’s LLMs include Rakuten AI 2.0, Rakuten AI 7B and the Rakuten AI 2.0 mini

Max and his team are exploring exciting developments, particularly in architectural design and training methodologies. The Rakuten 2.0 model utilizes the Mixture of Experts (MoE) architecture. The team constantly innovates to achieve the aim of developing tailored LLMs for specific business verticals which offer enhanced personalized experiences.

Moving forward, Max is focusing on enabling LLMs to support lifelong conversations with the user. This involves processing very long context, a significant challenge that could unlock even deeper levels of personalization and continuous interaction. Rakuten is also further developing its comprehensive portfolio of models to meet all requirements and resource considerations in terms of speed, cost and accessibiilty.

Currently, Rakuten’s LLMs include Rakuten AI 2.0 for general world knowledge, the faster and resource-efficient Rakuten AI 7B and the Rakuten AI 2.0 mini, a small language model (SLM) capable of running offline on edge or mobile devices.

“They all kind of deliver the same experience,” Max notes, “The difference is subtle. The smaller models, while faster, naturally have limitations on the sheer volume of information they can store. While all models offer secure chat experiences, their world knowledge varies. Right now, the larger Rakuten 2.0 has a much broader understanding, capable of answering complex questions about topics such as physics, whereas smaller models are better suited for more focused tasks.”

Max envisions a future where intelligent AI agents can remember us and proactively fulfill our needs

By taking a practical and nuanced approach, Rakuten can develop and deploy the most effective and efficient LLM for each specific application within its vast and varied ecosystem.

As Max and his team accelerate innovation in the LLM space, leveraging the data-rich Rakuten Ecosystem and pushing the boundaries of personalization, we draw ever closer to a future where AI agents are not only smart, but also remember us and proactively fulfill our needs.