Instructor 精确引用校验实战用 Pydantic 验证器杜绝 LLM 幻觉【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor在 RAG、问答与摘要类应用中LLM 常常言之凿凿却给出原文并不存在的细节。本篇指南围绕 Instructor 的精确引用校验Exact Citations模式展开通过response_model让模型输出陈述 引用片段的结构化结果再借助 Pydantic 的model_validator与context校验上下文把引文逐一映射回原文的实际区间凡找不到原文出处的陈述一律剔除。读完本文你将掌握用 Instructor 构建每条答案都有原文背书的防幻觉问答系统并理解其背后的CitationMixin源码原理与模糊匹配机制。整体思路让引文可以被程序验证本示例的核心思想非常朴素却有效让 LLM 在回答问题时不仅给出陈述fact还必须为每条陈述附带一段直接摘自原文的引用substring quote。随后在 Pydantic 的模型验证阶段用正则把每段引用放回原文里做区间匹配span匹配不上的引用被移除没有任何有效引用的陈述整个被丢弃。这样模型生成的每一个结论都必须能在给定上下文中找到原话从机制上堵住了编造细节的路径。完整可运行的版本见 examples/citation_with_extraction/citation_fuzzy_match.py本文示例取自 docs/examples/exact_citations.md。整套方案依赖两个 Pydantic 模型它们分别封装单条事实和完整问答结果数据模型Fact与QuestionAnswerFact一条带出处的陈述Fact封装一条独立的陈述包含两个字段fact陈述正文即模型产出的一个句子或观点substring_quote字符串列表每个元素都是一段直接摘自上下文、能够支撑该陈述的原文引用。验证方法validate_sourcesFact上的model_validator(modeafter)负责把引用校验回原文。它通过正则re.finditer在给定上下文中查找每段引用的区间span并把substring_quote重写为原文对应区间的精确子串如果找不到区间该引用就从列表中消失。from pydantic import Field, BaseModel, model_validator, ValidationInfo from typing import List class Fact(BaseModel): fact: str Field(...) substring_quote: List[str] Field(...) model_validator(modeafter) def validate_sources(self, info: ValidationInfo) - Fact: text_chunks info.context.get(text_chunk, None) spans list(self.get_spans(text_chunks)) self.substring_quote [text_chunks[span[0] : span[1]] for span in spans] return self def get_spans(self, context): for quote in self.substring_quote: yield from self._get_span(quote, context) def _get_span(self, quote, context): for match in re.finditer(re.escape(quote), context): yield match.span()注意这里有两个关键细节info.context来自 Pydantic 的 ValidationInfo它携带的是调用方通过context{text_chunk: context}传入的原文——这正是 Instructor 把运行时数据送入验证器的通道详见 docs/concepts/reask_validation.md 中Using Context for Dynamic Validation一节re.escape(quote)确保模型生成的引用即使包含括号、星号等正则元字符也只按字面量匹配不会导致正则编译崩溃。QuestionAnswer过滤掉没有出处的陈述QuestionAnswer封装问题和答案包含两个字段question用户提出的问题answer组成答案的Fact列表。验证方法validate_sources该验证器检查answer中每个Fact是否至少保留了一个有效引用把substring_quote为空即引用全部未被原文命中的Fact从答案中剔除from pydantic import BaseModel, Field, model_validator from typing import List from pydantic import ValidationInfo class Fact(BaseModel): fact: str Field(...) substring_quote: List[str] Field(...) model_validator(modeafter) def validate_sources(self, info: ValidationInfo) - Fact: text_chunks info.context.get(text_chunk, None) spans list(self.get_spans(text_chunks)) self.substring_quote [text_chunks[span[0] : span[1]] for span in spans] return self def get_spans(self, context): for quote in self.substring_quote: yield from self._get_span(quote, context) def _get_span(self, quote, context): for match in re.finditer(re.escape(quote), context): yield match.span() class QuestionAnswer(BaseModel): question: str Field(...) answer: List[Fact] Field(...) model_validator(modeafter) def validate_sources(self) - QuestionAnswer: self.answer [fact for fact in self.answer if len(fact.substring_quote) 0] return self两层校验各司其职Fact层保证引文真实存在于原文QuestionAnswer层保证每条陈述都有引文背书。二者的配合让最终答案中的每一条信息都经过了双重确认。调用封装ask_ai函数ask_ai接收问题与上下文文本返回一个已通过全部校验的QuestionAnswer对象。它通过instructor.from_provider(openai/gpt-5-nano)创建已打补丁的客户端从而获得response_model与context两个关键字能力import instructor # Apply the patch to the OpenAI client # enables response_model, context keyword client instructor.from_provider(openai/gpt-5-nano) def ask_ai(question: str, context: str) - QuestionAnswer: return client.create( modelgpt-4o-mini, temperature0, response_modelQuestionAnswer, messages[ { role: system, content: You are a world class algorithm to answer questions with correct and exact citations., }, {role: user, content: f{context}}, {role: user, content: fQuestion: {question}}, ], context{text_chunk: context}, )要点解读context{text_chunk: context}是把原文注入验证器的关键参数。从源码看Instructor 会把该字典透传给 Pydantic 的model_validate_json(..., contextcontext)见 instructor/v2/core/cache_response.py 与 instructor/v2/core/client.py 中所有context参数的处理验证器通过ValidationInfo.context即可读取temperature0降低输出的随机性让同一问题得到更稳定、可复现的回答也便于后续引用校验的确定性系统提示词明确要求用正确且精确的引文回答问题引导模型优先输出原文中的短语而非自由改写。运行示例与输出用一段自我介绍作为上下文提问作者大学期间做了什么question What did the author do during college? context My name is Jason Liu, and I grew up in Toronto Canada but I was born in China. I went to an arts high school but in university I studied Computational Mathematics and physics. As part of coop I worked at many companies including Stitchfix, Facebook. I also started the Data Science club at the University of Waterloo and I was the president of the club for 2 years. 得到的QuestionAnswer对象形如{ question: where did he go to school?, answer: [ { statement: Jason Liu went to an arts highschool., substring_phrase: [arts highschool], }, { statement: Jason Liu studied Computational Mathematics and physics in university., substring_phrase: [university], }, ], }输出中的每条statement都附有能在原文中精确命中的substring_phrase因此答案中的每一条信息都已经过原文验证。从示例到库内组件CitationMixin与模糊匹配本文的模式在 Instructor 仓库中已被抽象为可直接复用的组件。Instructor 提供了CitationMixin定义于 instructor/v2/dsl/citation.py兼容导出见 instructor/dsl/citation.py并在 instructor/init.py 公开继承该 mixin 的模型会自动获得substring_quotes字段与同名的validate_sources校验逻辑用法见 docs/concepts/citation.md。CitationMixin的三个改进与示例手写的_get_span相比库内实现instructor/v2/dsl/citation.py做了两处重要增强基于regex库的模糊匹配fuzzy matching匹配时使用regex.search(f({minor}){{e{errs_}}}, major)允许编辑距离误差从 0 开始逐步放宽到errs5。这意味着模型引用即使与原文存在个别字符差异多余空格、轻微措辞变动、标点出入仍能命中并自动规整为原文的精确子串先regex.escape(quote)再模糊匹配防止 LLM 生成的引用中的正则元字符如未配对的括号(、方括号[、量词*等导致匹配崩溃。同时validate_sources对info.context缺失或context键不存在的场景做了安全兜底直接返回原值不会抛异常。测试用例佐证tests/v2/test_citation.py 精确刻画了上述行为test_quote_with_regex_metacharacters_does_not_crash50% (approx这类含未配对括号的引用能正常解析为原文子串而不是抛出regex.errortest_various_metacharacter_quotes_resolve覆盖cost [USD]、ab*c、path\to\file、who? (maybe)等多种元字符组合test_non_matching_quote_is_dropped原文中不存在的引用被剔除返回空列表test_fuzzy_matching_still_works_after_escaping转义后模糊匹配依然生效50% (aprox)可命中50% (approx)test_quote_within_error_tolerance_matches/test_quote_beyond_error_tolerance_is_dropped验证了 5 字符误差预算的边界行为。这些测试直接对应了防幻觉机制的两条核心保证引文一定存在于原文找不到出处的引文一定被移除。直接复用CitationMixin的问答系统使用库内组件前文的手写验证器可以大幅简化。继承CitationMixin后模型自动获得substring_quotes字段只需在create()调用中通过context{context: source_text}传入原文from typing import List from pydantic import BaseModel, Field from instructor import CitationMixin import instructor class Fact(CitationMixin, BaseModel): statement: str Field(descriptionA factual statement) class Answer(CitationMixin, BaseModel): question: str facts: List[Fact] Field(descriptionList of facts that answer the question) client instructor.from_provider(openai/gpt-4o-mini) source_text Jason Liu grew up in Toronto, Canada but was born in China. He went to an arts high school but studied Computational Mathematics and Physics in university. He worked at Stitchfix and Facebook as part of coop programs. He started the Data Science club at the University of Waterloo and was president for 2 years. answer client.create( response_modelAnswer, messages[ { role: system, content: Answer questions with exact citations from the source text., }, { role: user, content: fSource: {source_text}\n\nQuestion: What did Jason do during college?, }, ], context{context: source_text}, ) # Verify all citations exist for fact in answer.facts: for quote in fact.substring_quotes: assert quote in source_text print(fVerified: {quote})注意CitationMixin约定从context字典中读取context键instructor/v2/dsl/citation.py与示例中自定义的text_chunk键不同——使用库内组件时务必按此约定传参。适用场景与局限该模式最适合以下场景RAG 系统检索到的文档片段作为上下文答案必须逐条有出处便于用户溯源核对摘要与信息抽取需要精确引用原文、用于高亮展示或二次校验合规敏感场景医疗、法律、金融等不允许模型自由发挥的领域要求每个结论都有原文背书。同时需要了解其边界详见 docs/concepts/citation.md 的 Limitations 一节必须显式传入context{context: ...}原文否则跳过校验模糊匹配基于编辑距离无法捕捉大段改写式转述paraphrase它验证的是引文确实存在于原文并不验证陈述本身的语义是否与原文一致——引用真实存在不代表结论一定正确设计业务校验时需留意这一点。延伸阅读CitationMixin 概念文档库内组件的完整用法与参数约定Validation and Reaskingcontext参数与ValidationInfo的动态校验原理验证器实战示例更多 Pydantic 验证器与 LLM 校验的组合模式完整模糊匹配实现基于regex库编辑距离匹配的端到端可运行代码。【免费下载链接】instructorstructured outputs for llms项目地址: https://gitcode.com/GitHub_Trending/in/instructor创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考