From AI hype to real-world tools: Rakuten teams up with LangChain

In the rapidly evolving world of artificial intelligence, separating hype from practical business value remains a crucial challenge. At Rakuten, that challenge is being met head-on through an innovative partnership with LangChain, a startup that builds AI infrastructure tools to help companies turn powerful language models into practical business applications.

At a recent event at Rakuten’s Tokyo headquarters, LangChain CEO Harrison Chase and founding engineer Will Fu-Hinthorn addressed Rakuten employees, showcasing how their technology is bridging the gap between AI potential and business reality.

Building AI agents that work

The key to this transformation lies in something called agents – specialized AI tools built on top of large language models (LLMs).

“The key point is not about the model,” explains General Manager, AI for Business, Yusuke Kaji, who leads Rakuten’s LLM-related product development. “It’s more about the orchestration on top of the LLM, if you want to create a good service or tool or application. LangChain helps us address this problem of orchestration, connecting useful tools like web search, file search, or data analysis and coding assist tools.”

These tailored AI agents are far more sophisticated than standard chatbots. They comprise a chain of steps, including a planning stage and a reflection stage in which the agent assesses its own output and loops back to earlier in the chain if necessary.

They can tap into vast data resources through a process called retrieval augmented generation (RAG), which grounds AI responses in actual data rather than potentially hallucinated knowledge. This is particularly powerful when paired with large sets of data, like those managed by Rakuten.

Agents can also be built to work on a longer timescale than a standard LLM would allow, allowing for both short- and long-term memory of past interactions. Engineers can decide exactly how much human guidance is appropriate – too little, and the agent might go in an unwanted direction; too much, and the agent is no longer reducing the workload.

LangChain’s tools even allow engineers to peek behind the curtains and observe exactly what a model is doing. This transparency is crucial for debugging and improving AI systems that don’t always give consistent answers, making the technology more reliable for enterprise use.

From experiment to enterprise reality

For Rakuten, this technology has already shown impressive results.

“We initially created a platform that allows all Rakuten employees to create single-purpose agents themselves with no code,” Kaji shares. “Employees have built agents for sales, business analysis, customer support, translation, digital marketing support, legal work, HR and even coaching.”

The partnership started with internal tools but has quickly expanded.

“That’s what we did initially for internal employees,” Kaji notes. “Now, we are trying to do the same with agents for consumers and enterprise clients.”

Other use cases in play at Rakuten include tools to reduce the workload of Rakuten Ichiba merchants. One Rakuten engineer at the event outlined a scenario in which merchants could ask an agent to analyze customer reviews of a specific product, diving deep into specific attributes of that product.

“LangChain allowed us to transition from a proof-of-concept-like experiment to actual production.”

Agility and responsibility

What makes LangChain stand out in the crowded AI tooling space? Kaji points to two crucial factors.

“We were looking for a startup who can move quickly, and who also has trust from open-source communities,” he highlights. “In the LLM or the AI area, there are not many other AI startups who fulfill these conditions.”

Kaji has praise for LangChain’s agility: “They’re a small startup, but very quick and responsive to adapt to new trends, which is crucial since the generative AI industry is changing so fast.”

But even more important than this is the LangChain team’s sense of responsibility: “This second point is much more important for a large enterprise like Rakuten. If you’re just quick, you might be able to make a cool demo. But in the enterprise world, that only takes you so far. It needs to scale, and it needs to be trusted by our consumers, our enterprise clients.”

With LangChain’s tools, Rakuten is able to create reliable, robust infrastructure.

“They put a lot of effort into making their product stable, and their team is really responsible,” Kaji continues. “Even when we have very high expectations, they meet them.”

A vision of AI empowerment

Rakuten’s partnership with LangChain has highlighted how companies can move beyond AI hype and experimental projects to create real, practical applications. For Rakuten, this has brought a new set of challenges.

Rakuten currently boasts AI that enables Rakutenians to transform their work processes, and are actively sharing their AI learnings and success cases with clients and partners. Looking to the future, Rakuten Group Chairman and CEO Mickey Mikitani has a mission to democratize access to Al expertise.

“If you’re a small to medium-sized enterprise or merchant, you may not be able to hire the ten best engineers or marketing specialists in the world,” Kaji explains. “We are aiming to empower such businesses to get their work done by providing good AI specialists, whether that’s in engineering, marketing, sales or other white-collar work.”

This vision of AI democratization could cause a significant shift in how businesses operate. Smaller companies could leverage AI agents to access similar capabilities as their larger competitors, helping to level the playing field for businesses of all sizes.

This is the role Kaji sees Rakuten playing in the future: “Our vision is to empower our stakeholders – be that employees, clients, consumers or society.”