Zilliz Open Sources Industry-First Bilingual "Semantic Highlighting" Model to Slash RAG Token Costs and Boost Accuracy

REDWOOD CITY, Calif., Jan. 30, 2026 /PRNewswire/ — Zilliz, the company behind the leading open-source vector database Milvus, today announced the open-source release of its Bilingual Semantic Highlighting Model, an industry-first AI model designed to dramatically reduce token usage and improve answer quality in production RAG-powered AI applications.

This highlighting model introduces sentence-level relevance filtering, enabling AI developers to remove low-signal context before sending prompts to large language models. This approach directly addresses rising inference costs and accuracy issues caused by oversized context windows in enterprise RAG and RAG-powered AI deployments.

“As RAG systems move into production, teams are running into very real cost and quality limits,” said James Luan, VP of Engineering at Zilliz. “This model gives developers a practical way to reduce prompt size and improve answer accuracy without reworking their existing pipelines.”

Key Innovations and Technical Breakthroughs

Bilingual relevance by design: Optimized for both English and Chinese, the model addresses cross-lingual relevance challenges common in global RAG deployments. It is built on the MiniCPM-2B architecture, enabling low-latency, production-ready performance.
Sentence-level context filtering: Rather than scoring entire document chunks, the model evaluates relevance at the sentence level and retains only content that directly supports a user query before sending it to the LLM.
Lower token usage, higher answer quality: Zilliz reports that sentence-level filtering significantly compresses prompt size while improving downstream response quality, helping teams reduce inference costs and improve generation speed in production environments.

Availability

The Bilingual Semantic Highlighting Model is available today as an open-source release. To learn more about the training methodology and performance benchmarks, visit the Zilliz Technical Blog.

Download: : zilliz/semantic-highlight-bilingual-v1

About Zilliz

Zilliz is the company behind Milvus, the world’s most widely adopted open-source vector database. Zilliz Cloud brings that performance to production with a fully managed, cloud-native platform built for scalable, low-latency vector search and hybrid retrieval. It supports billion-scale workloads with sub-10ms latency, auto-scaling, and optimized indexes for GenAI use cases like semantic search and RAG.

Zilliz is built to make AI not just possible—but practical. With a focus on performance and cost-efficiency, it helps engineering teams move from prototype to production without overprovisioning or complex infrastructure. Over 10,000 organizations worldwide rely on Zilliz to build intelligent applications at scale.

Headquartered in Redwood Shores, California, Zilliz is backed by leading investors, including Aramco’s Prosperity 7 Ventures, Temasek’s Pavilion Capital, Hillhouse Capital, 5Y Capital, Yunqi Partners, Trustbridge Partners, and others. Learn more at Zilliz.com.