我的征尘是星辰大海。。。

The dirt and dust from my pilgrimage forms oceans of stars...

-------当记忆的篇章变得零碎，当追忆的图片变得模糊，我们只能求助于数字存储的永恒的回忆

作者:黄教授

手机视频列表

OceanBase_PowerRAG用RAG技术增强数据库检索2

视频

音频

原始脚本

全文语义检索向量融合实现从形似到神似。
传统数据库的全文检索仅能实现关键字匹配，如拼音相似、拼写纠错的形似匹配，无法捕捉文本背后的语义关联，常出现关键字相同但含义无关的无效结果。
而 OceanBase 融合 RAG 语义向量技术，彻底突破了这一局限。
通过关键字加语义双重校验，实现神似匹配。
先看传统全文检索的痛点。
假设数据库中存储两类文档段落。
水果种植文档，苹果是温带水果，适宜在疏松肥沃的土壤中种植，挂果期需充足光照。
电子产品文档，苹果公司的手机产品支持上门维修，保修期内非人为故障可免费更换零件。
当用户查询苹果产品的售后政策时，传统全文检索仅能匹配苹果这个关键字，会同时返回一和二两个段落，虽然都含苹果，但一与售后政策完全无关，需人工筛选，效率极低。
OceanBase 的语义检索突破 一、语义向量的构建逻辑。
针对上述两端文本，OceanBase 会先通过外部 Embedding 模型离线一次性处理，将其转为语义向量。
段落一的向量，核心维度偏向水果、种植、土壤、光照。
段落二的向量，核心维度偏向电子产品。
售后、维修、保修，这些向量与原文一同存储在数据库中，后续查询无需再依赖外部模型。
二、混合检索的精准匹配过程。
当用户查询苹果产品的售后政策时。
第一步，关键字匹配，形似。
通过 ik 分词器拆分查询为苹果、产品、售后、政策。
同时匹配到1和2中的苹果，初步筛选出两个候选段落。
第二步，语义向量匹配，神似。
将用户查询转为向量，核心维度偏向产品售后，分别计算语义段落向量的相似度，2的相似四度高达0。
85，语义高度相关，1的相似度仅0.12，语义无关。
第三步，RRF 算法融合得分，最终按关键字得分乘0，3加语义相似度得分乘0.7的权重计算总得分。
2的总得分远高于，因此仅返回2作为结果，彻底过滤无效信息。
传统全文检索的优化升级除了语义融合，OceanBase 对传统关键字检索也做了强化，支持多语言智能分词，如 中文、IK、英文、Space 分词器、自定义词典，可添加苹果公司售后政策等专业术语。
拼音字形模糊匹配，如苹果可匹配苹果，既保留了传统关键字检索的精准性，又解决了新词漏配、拼写错误等问题。
三、终极价值。
TP、AP、AI 混合负载适配，让程序员专注业务本身。
OceanBase 借用 RAG 技术的核心目标，是让数据库自身具备复杂模式匹配和全文语义检索能力。
同时实现 TP、事务处理、AP、分析处理、AI、智能检索、混合负载的原生适配，彻底解解放业务与数据库程序员，对企业而言 言，这意味着全场景覆盖的高效价值。
结合具体场景更易理解。
混合负载适配，一个数据库搞定全流程需求。
以银行反洗钱业务为例，OceanBase 可无缝支撑从实时交易到风险分析，再到智能检索的全流程。
TP 场景，实时交易，用户发起单笔800元凌晨转账，数据库毫秒级完成扣减余额。
记录流水的事务处理，同时实时更新用户行为向量的凌晨交易占比维度。
APP 场景批量分析，风控部门查询近3个月高频小额转账用户的交易总规模，数据库直接扫描历史数据完成聚合计算，无需导出至独立数仓。
 AI 场景，智能检索，需匹配反洗钱政策中高频小额存取的相关条款时，数据库通过语义向量检索快速定位政策段落，同时用模式向量匹配用户行为，一次性返回交易数据、加风险模式匹配结果、加政策依据。
技术透明化，程序员无需懂 AI 写 SQL 即可搞定所有。
对业务程序员而言，上述复杂流程仅需用熟悉的 SQL 就能完成。
向量检索、聚类算法等底层技术完全透明。
无需学习向量算法，仅需掌握 approximate 等少量扩展关键字，即可实现高维向量匹配。
无需协调多系统，结构化数据向量数据、文档数据同库存储，无需对接独立向量库、数仓或检索工具。
专注业务逻辑开发，例如实现反洗钱全流程检索，仅需关联交易表、风险模式表、政策文档表，用一条复合 SQL 即可完成，无需关注技术实现细节，合规与效率兼顾，彻底规避大模型风险。
检索过程全程可追溯，向量匹配的相似度计算，关键字得分的权重分配，政策条款的匹配依据，均可量化验证，完全适配金融行业合规要求。
毫秒级响应保障业务，所有计算由数据库原生引擎完成，即便同时处理 TP APP AI 负载。
仍能保持毫秒级查询延迟，远优于大模型秒级响应。
资源消耗更可控，无需为大模型部署昂贵的 GPU 集群。
数据库原生算法可高效利用硬件资源，降低企业成本。
结语，数据库的自我进化，而非对大模型的依附。
OceanBase Power RAG 的本质是数据库的自我升级。
它不仅借用 RAG 技术补全了传统数据库的关键短板，更实现了 TPAI 混合负载的原生适配，让数据库从只能精准查询升级为能懂模式能 懂语义，能称全流程的全能检索平台，而非成为服务大模型的附属工具。
这一突破清晰界定了数据库与 AI 技术的关系。
AI 技术是提升数据库能力的增强器，而非决定数据库属性的核心依赖。
未来，数据库的竞争将聚焦于如何用 AI 技术强化原生能力，覆盖全场景需求。
而 OceanBase 所引领的无大模型依赖原生集成 RAG 技术的路径，正是这一趋势的核心方向。

修正脚本

全文语义检索向量融合实现从形似到神似。
传统数据库的全文检索仅能实现关键字匹配，如拼音相似、拼写纠错的形似匹配，无法捕捉文本背后的语义关联，常出现关键字相同但含义无关的无效结果。
而 OceanBase 融合 RAG 语义向量技术，彻底突破了这一局限。
通过关键字加语义双重校验，实现神似匹配。
先看传统全文检索的痛点。
假设数据库中存储两类文档段落。
水果种植文档，苹果是温带水果，适宜在疏松肥沃的土壤中种植，挂果期需充足光照。
电子产品文档，苹果公司的手机产品支持上门维修，保修期内非人为故障可免费更换零件。
当用户查询苹果产品的售后政策时，传统全文检索仅能匹配苹果这个关键字，会同时返回一和二两个段落，虽然都含苹果，但一与售后政策完全无关，需人工筛选，效率极低。
OceanBase 的语义检索突破，一、语义向量的构建逻辑。
针对上述两段文本，OceanBase 会先通过外部 Embedding 模型离线一次性处理，将其转为语义向量。
段落一的向量，核心维度偏向水果、种植、土壤、光照。
段落二的向量，核心维度偏向电子产品、售后、维修、保修，这些向量与原文一同存储在数据库中，后续查询无需再依赖外部模型。
二、混合检索的精准匹配过程。
当用户查询苹果产品的售后政策时。
第一步，关键字匹配，形似。
通过 ik 分词器拆分查询为苹果、产品、售后、政策。
同时匹配到1和2中的苹果，初步筛选出两个候选段落。
第二步，语义向量匹配，神似。
将用户查询转为向量，核心维度偏向产品售后，分别计算语义段落向量的相似度，2的相似度高达0.85，语义高度相关，1的相似度仅0.12，语义无关。
第三步，RRF 算法融合得分，最终按关键字得分乘0.3加语义相似度得分乘0.7的权重计算总得分。
2的总得分远高于1，因此仅返回2作为结果，彻底过滤无效信息。
传统全文检索的优化升级除了语义融合，OceanBase 对传统关键字检索也做了强化，支持多语言智能分词，如 中文、IK、英文、Space 分词器、自定义词典，可添加苹果公司售后政策等专业术语。
拼音字形模糊匹配，如平果可匹配苹果，既保留了传统关键字检索的精准性，又解决了新词漏配、拼写错误等问题。
三、终极价值。
TP、AP、AI 混合负载适配，让程序员专注业务本身。
OceanBase 借用 RAG 技术的核心目标，是让数据库自身具备复杂模式匹配和全文语义检索能力。
同时实现 TP、事务处理、AP、分析处理、AI、智能检索、混合负载的原生适配，彻底解放业务与数据库程序员，对企业而言，这意味着全场景覆盖的高效价值。
结合具体场景更易理解。
混合负载适配，一个数据库搞定全流程需求。
以银行反洗钱业务为例，OceanBase 可无缝支撑从实时交易到风险分析，再到智能检索的全流程。
TP 场景，实时交易，用户发起单笔800元凌晨转账，数据库毫秒级完成扣减余额。
记录流水的事务处理，同时实时更新用户行为向量的凌晨交易占比维度。
AP 场景批量分析，风控部门查询近3个月高频小额转账用户的交易总规模，数据库直接扫描历史数据完成聚合计算，无需导出至独立数仓。
 AI 场景，智能检索，需匹配反洗钱政策中高频小额存取的相关条款时，数据库通过语义向量检索快速定位政策段落，同时用模式向量匹配用户行为，一次性返回交易数据、风险模式匹配结果、政策依据。
技术透明化，程序员无需懂 AI 写 SQL 即可搞定所有。
对业务程序员而言，上述复杂流程仅需用熟悉的 SQL 就能完成。
向量检索、聚类算法等底层技术完全透明。
无需学习向量算法，仅需掌握 approximate 等少量扩展关键字，即可实现高维向量匹配。
无需协调多系统，结构化数据向量数据、文档数据同库存储，无需对接独立向量库、数仓或检索工具。
专注业务逻辑开发，例如实现反洗钱全流程检索，仅需关联交易表、风险模式表、政策文档表，用一条复合 SQL 即可完成，无需关注技术实现细节，合规与效率兼顾，彻底规避大模型风险。
检索过程全程可追溯，向量匹配的相似度计算，关键字得分的权重分配，政策条款的匹配依据，均可量化验证，完全适配金融行业合规要求。
毫秒级响应保障业务，所有计算由数据库原生引擎完成，即便同时处理 TP、AP、AI 负载。
仍能保持毫秒级查询延迟，远优于大模型秒级响应。
资源消耗更可控，无需为大模型部署昂贵的 GPU 集群。
数据库原生算法可高效利用硬件资源，降低企业成本。
结语，数据库的自我进化，而非对大模型的依附。
OceanBase Power RAG 的本质是数据库的自我升级。
它不仅借用 RAG 技术补全了传统数据库的关键短板，更实现了 TP、AP、AI 混合负载的原生适配，让数据库从只能精准查询升级为能懂模式能懂语义，能撑全流程的全能检索平台，而非成为服务大模型的附属工具。
这一突破清晰界定了数据库与 AI 技术的关系。
AI 技术是提升数据库能力的增强器，而非决定数据库属性的核心依赖。
未来，数据库的竞争将聚焦于如何用 AI 技术强化原生能力，覆盖全场景需求。
而 OceanBase 所引领的无大模型依赖原生集成 RAG 技术的路径，正是这一趋势的核心方向。

英文翻译

Full-text semantic retrieval vector fusion achieves the transition from form similarity to semantic similarity.  
Traditional full-text retrieval in databases can only achieve keyword matching, such as form-based matching like pinyin similarity and spelling correction, but fails to capture the semantic associations behind the text, often returning irrelevant results where keywords are identical but meanings are unrelated.  
OceanBase, by integrating RAG semantic vector technology, completely breaks through this limitation.  
Through dual validation of keywords and semantics, it achieves semantic similarity matching.  
Let's first look at the pain points of traditional full-text retrieval.  
Assume the database stores two types of document paragraphs:  
A fruit cultivation document: "Apples are temperate fruits, suitable for planting in loose, fertile soil, requiring sufficient light during the fruiting period."  
An electronics document: "Apple Inc.'s mobile phone products support on-site repair, and non-human failures within the warranty period can have parts replaced free of charge."  
When a user queries the after-sales policy for Apple products, traditional full-text retrieval can only match the keyword "Apple," returning both paragraphs 1 and 2. Although both contain "Apple," paragraph 1 is completely unrelated to after-sales policy, requiring manual filtering, resulting in extremely low efficiency.  
OceanBase's semantic retrieval breakthrough:  
1. Construction logic of semantic vectors.  
For the two text paragraphs above, OceanBase first processes them offline in a single pass using an external Embedding model, converting them into semantic vectors.  
The vector for paragraph 1 has core dimensions偏向 fruit, planting, soil, and light.  
The vector for paragraph 2 has core dimensions偏向 electronics, after-sales, repair, and warranty. These vectors are stored in the database along with the original text, so subsequent queries no longer require external models.  
2. Precise matching process of hybrid retrieval.  
When the user queries "Apple product after-sales policy":  
Step 1: Keyword matching (form similarity).  
The ik tokenizer splits the query into "Apple," "product," "after-sales," and "policy."  
It matches "Apple" in both paragraphs 1 and 2, initially filtering two candidate paragraphs.  
Step 2: Semantic vector matching (semantic similarity).  
The user query is converted into a vector with core dimensions偏向 product after-sales. The semantic similarity between the query vector and each paragraph's vector is calculated: paragraph 2's similarity is 0.85 (highly semantically relevant), while paragraph 1's similarity is only 0.12 (semantically irrelevant).  
Step 3: RRF algorithm fusion scoring. The total score is calculated with weights: keyword score multiplied by 0.3 plus semantic similarity score multiplied by 0.7.  
Paragraph 2's total score is far higher than paragraph 1's, so only paragraph 2 is returned as the result, completely filtering out invalid information.  
Optimization and upgrade of traditional full-text retrieval: In addition to semantic fusion, OceanBase has also enhanced traditional keyword retrieval, supporting multilingual intelligent tokenization, such as Chinese, IK, English, Space tokenizers, and custom dictionaries, which can include specialized terms like "Apple Inc. after-sales policy."  
Fuzzy matching for pinyin and glyphs, such as matching "ping guo" to "Apple," preserves the precision of traditional keyword retrieval while solving problems like missed matches for new words and spelling errors.  
3. Ultimate value:  
TP, AP, AI hybrid workload adaptation enables programmers to focus on business itself.  
OceanBase's core goal in leveraging RAG technology is to equip the database itself with complex pattern matching and full-text semantic retrieval capabilities.  
At the same time, it achieves native adaptation for TP (transaction processing), AP (analytical processing), and AI (intelligent retrieval) hybrid workloads, completely liberating business and database programmers. For enterprises, this means high-efficiency value across all scenarios.  
Understanding specific scenarios makes it easier to grasp.  
Hybrid workload adaptation: one database handles the entire process.  
Taking bank anti-money laundering business as an example, OceanBase seamlessly supports the entire process from real-time transactions to risk analysis to intelligent retrieval.  
TP scenario: real-time transactions. When a user initiates a single transfer of 800 yuan at midnight, the database completes the balance deduction and records the transaction in milliseconds, while simultaneously updating the user behavior vector's dimension for the proportion of midnight transactions.  
AP scenario: batch analysis. When the risk control department queries the total transaction scale of users with high-frequency, small-amount transfers in the past three months, the database directly scans historical data to complete aggregation calculations without exporting to a separate data warehouse.  
AI scenario: intelligent retrieval. When needing to match relevant clauses in anti-money laundering policies regarding high-frequency, small-amount deposits and withdrawals, the database uses semantic vector retrieval to quickly locate policy paragraphs, while matching user behaviors using pattern vectors, returning transaction data, risk pattern matching results, and policy basis in one go.  
Technology transparency: programmers can handle everything by writing SQL without needing to understand AI.  
For business programmers, the complex process above can be completed using familiar SQL.  
Vector retrieval, clustering algorithms, and other underlying technologies are completely transparent.  
No need to learn vector algorithms; only a small set of extended keywords like "approximate" is needed to achieve high-dimensional vector matching.  
No need to coordinate multiple systems: structured data, vector data, and document data are stored in the same database, eliminating the need to interface with separate vector databases, data warehouses, or retrieval tools.  
Focusing on business logic development: for example, implementing anti-money laundering full-process retrieval only requires associating transaction tables, risk pattern tables, and policy document tables, using a single composite SQL. There is no need to worry about technical implementation details, achieving both compliance and efficiency while completely avoiding risks associated with large models.  
The entire retrieval process is traceable: similarity calculations for vector matching, weight allocation for keyword scores, and matching basis for policy clauses can all be quantified and verified, fully complying with financial industry regulatory requirements.  
Millisecond-level response ensures business needs: all calculations are performed by the database's native engine. Even when handling TP, AP, and AI workloads simultaneously, query latency remains at the millisecond level, far outperforming large models' second-level responses.  
Resource consumption is more controllable: no need to deploy expensive GPU clusters for large models. The database's native algorithms efficiently utilize hardware resources, reducing enterprise costs.  
Conclusion: Database self-evolution, not reliance on large models.  
The essence of OceanBase Power RAG is the database's self-upgrade.  
It not only uses RAG technology to compensate for key shortcomings of traditional databases but also achieves native adaptation for TP, AP, and AI hybrid workloads, upgrading the database from a tool that can only perform precise queries to a versatile retrieval platform that understands patterns, semantics, and supports the entire process, rather than becoming a subordinate tool serving large models.  
This breakthrough clearly defines the relationship between databases and AI technology.  
AI technology is an enhancer for improving database capabilities, not a core dependency that determines database attributes.  
In the future, database competition will focus on how to use AI technology to strengthen native capabilities and cover all scenario needs.  
The path of native integration of RAG technology without reliance on large models, pioneered by OceanBase, is the core direction of this trend.