Harper 语法检查器中的封闭复合词回归测试misc_closed_compound_clean.md 的零误报验证机制【免费下载链接】harperOffline, privacy-first grammar checker. Fast, open-source, Rust-powered项目地址: https://gitcode.com/GitHub_Trending/har/harperHarperHarper是一个离线、隐私优先、用 Rust 编写的英语语法与拼写检查器其核心检查逻辑位于harper-corecrate 中。本篇以测试语料文件 misc_closed_compound_clean.md 为主体完整解读这份干净语料clean corpus的设计意图、它在 run_tests.rs 中的测试装配方式以及它正在守护的封闭复合词closed compound检查规则集的实现细节。读完本文你可以掌握如何用期望零 lint的语料验证检查器不误报、Harper 测试宏的完整工作链路以及封闭复合词规则从MapPhraseLinter到LintGroup的底层调用关系。1. 测试语料的完整内容与验证目标misc_closed_compound_clean.md 全文仅 15 行由 8 句以空行分隔的自然英文句子组成。完整内容如下Its hidden right here under the carpet. I got there after him. Go back there after dinner and finish it. Weve gotta go down right here. I hereby state that I got here by way of the issues link. Its over there in that box. Ill still be here after work. Ill meet you here after work.文件名中的clean是 Harper 测试语料的命名惯例*_clean.md表示该文档期望被检查后产生 0 条 lint与同目录下hex_basic_dirty.md期望 1 条这类脏语料相对。这份语料的验证目标非常明确——其中高频出现here、there、hereby、over there、gotta、after等词它们恰好与 Harper 的封闭复合词规则表中的条目见第 3 节高度相关。换句话说这个文件专门用来回答一个问题当句子中大量出现容易在词中间被错误拆出空格的封闭复合词时检查器会不会误报答案是一条 lint 都不应产生。逐句看语料刻意覆盖了多种易混淆场景right here、over there副词指示词的常见组合其中there既可能被误拆为the reright/there也各自是独立合法词after him、after dinner、after work介词after的三种不同句法位置hereby本身就是一个封闭复合词规则表中另有here forth一类词形的对照规则gotta口语化的封闭复合形式双引号包裹的issues顺带验证引号内 token 的解析不受干扰I got here by way of the issues linkby way of短语与got/got过去式共存避免触发过去分词误用类规则。每一句都在与 Harper 的不同规则维度擦肩而过这正是 clean 语料的价值单条规则的单测只保证该报时报而整句级 clean 语料保证不该报时绝对不报。2. 测试装配create_test! 宏如何消费这份语料语料本身只是静态文本它的执行由 run_tests.rs 中的宏调用驱动。该文件第 87 行注册了这条测试create_test!(misc_closed_compound_clean.md, 0, Dialect::American);第二个参数0就是期望 lint 数第三个参数指定美式英语方言。展开 run_tests.rs 中的create_test!宏后等价于生成如下测试函数#[test] fn lints_misc_closed_compound_clean_md_correctly() { let source include_str!(./test_sources/misc_closed_compound_clean.md); let dict FstDictionary::curated(); let document Document::new_markdown_default(source, dict); let mut linter LintGroup::new_curated(dict, Dialect::American); let lints linter.lint(document); assert_eq!(lints.len(), 0); // 确保每个 token 的 span 都能映射回真实字符 for token in document.tokens() { assert!(token.span.try_get_content(document.get_source()).is_some()); } }这条测试链路包含三个可验证的实现事实语料在编译期嵌入include_str!在编译阶段读取.md文件因此语料一旦修改测试行为随之改变无需运行时 IO走的是完整的 Markdown 解析 全量规则组路径Document::new_markdown_default使用默认 Markdown 解析器实现见 markdown.rsLintGroup::new_curated则装配了 Harper 内置的全部规则含拼写、语法、风格与封闭复合词规则。也就是说0 条 lint 是整个 curated 规则集的共同结论比单独测试某一个规则更严格附带 token span 完整性断言遍历document.tokens()断言每个 span 都能取回原文内容防止解析器生成指向不存在的字符范围的 token——这是所有create_test!用例共享的防御性检查。运行方式在仓库根目录cargo test -p harper-core --test run_tests lints_misc_closed_compound_clean_md_correctly3. 底层实现封闭复合词规则集与 MapPhraseLinter这份语料守护的核心规则位于 closed_compounds.rs。该文件定义lint_group()通过add_compound_mappings!宏声明一张错误写法 → 正确写法的映射表全部条目如下节选关键项完整清单见源文件 L23-L88add_compound_mappings!(group, { Anybody ([any body][..], anybody), Anyhow ([any how][..], anyhow), Anywhere ([any where][..], anywhere), Bypass ([by pass][..], bypass), Chalkboard ([chalk board][..], chalkboard), Codebase ([code base][..], codebase), Desktop ([desk top][..], desktop), Everybody ([every body][..], everybody), Everyone ([every one][..], everyone), Everywhere ([every where][..], everywhere), However ([how ever][..], however), Intact ([in tact][..], intact), Instead ([in stead][..], instead), Itself ([it self][..], itself), Keystroke ([key stoke, key stroke][..], keystroke), Laptop ([lap top][..], laptop), Middleware ([middle ware][..], middleware), Meanwhile ([mean while][..], meanwhile), Misunderstand ([miss understand][..], misunderstand), Misuse ([miss use][..], misuse), Myself ([my self][..], myself), Nonetheless ([none the less][..], nonetheless), Nothing ([no thing][..], nothing), Notwithstanding ([not with standing][..], notwithstanding), Nowhere ([no where][..], nowhere), Overall ([over all][..], overall), Overnight ([over night][..], overnight), Postpone ([post pone][..], postpone), Regardless ([regard less][..], regardless), Somebody ([some body][..], somebody), Someone ([some one][..], someone), Somewhere ([some where][..], somewhere), Straightforward ([straight forward][..], straightforward), There ([the re][..], there), Therefore ([there fore][..], therefore), Underclock ([under clock][..], underclock), Upward ([up ward][..], upward), Widespread ([wide spread][..], widespread), Without ([with out][..], without), Worldwide ([world wide][..], worldwide), Worthwhile ([worth while, worth-while][..], worthwhile), // ... 共 60 余条完整列表见源文件 });注意规则表中There ([the re][..], there)这一条它专门捕获there被错误地拆成the re的拼写事故。测试语料中反复出现的there/here正对应这类条目——语料验证的是这些词的正确拼写形式不应被任何规则误伤。规则注册入口在 lint_group/mod.rsLintGroup::new_curated内部调用out.merge_from(closed_compounds::lint_group())将整组规则并入 curated 规则集且该组通过group.set_all_rules_to(Some(true))默认全部启用。3.1 MapPhraseLinter::new_closed_compound 的匹配与替换机制每条映射在 map_phrase_linter.rs 中经MapPhraseLinter::new_closed_compound编译为一个MapPhraseLinterpub fn new_closed_compound( phrases: impl IntoIteratorItem impl AsRefstr, correct_form: impl ToString, ) - Self { let message format!( Did you mean the closed compound {}?, correct_form.to_string() ); let description format!( Looks for incorrect spacing inside the closed compound {}., correct_form.to_string() ); Self::new_fixed_phrases( phrases, [correct_form], message, description, Some(LintKind::Miscellaneous), ) }其内部调用链是new_fixed_phrasesL46-L65将多个错误写法如key stoke与key stroke两种拼法分别转换为FixedPhrase表达式再用LongestMatchOf组合成最长匹配优先的模式从而避免短错误形式遮蔽长错误形式。匹配命中后的替换逻辑在match_to_lintL114-L134取命中 token 的 span 及对应原文matched_text通过Suggestion::replace_with_match_case生成替换建议保留原文的大小写形态例如HOW EVER会建议HOWEVER而非小写however生成LintKind::Miscellaneous类型的 lint优先级 31消息模板为Did you mean the closed compound \X?。这套匹配 → 保大小写替换的机制正是 clean 语料需要保护的对象一旦FixedPhrase的 token 边界处理出现回归例如把the re之外的there也匹配进去本文第 1 节那 8 句话会立刻产生 lintassert_eq!(lints.len(), 0)随之失败。规则集自身的正向单测也写在 closed_compounds.rs 的文件内tests模块中采用assert_suggestion_result断言错误输入 → 修正输出例如#[test] fn it_self() { let test_sentence The project, it self, was quite challenging.; let expected The project, itself, was quite challenging.; assert_suggestion_result(test_sentence, lint_group(), expected); } #[test] fn how_ever() { let test_sentence This is true, how ever, details matter.; let expected This is true, however, details matter.; assert_suggestion_result(test_sentence, lint_group(), expected); }正向单测该报时报 clean 语料不该报时不报构成对同一规则集的两侧防线。4. clean/dirty语料对的回归测试范式misc_closed_compound_clean.md 并非孤例。run_tests.rs 中成对/成族的 clean/dirty 语料体现了 Harper 统一的回归测试范式语料文件期望 lint 数方言说明misc_closed_compound_clean.md0American封闭复合词高频词密集出现验证零误报hex_basic_clean.md0American十六进制内容不误报对照 hex_basic_dirty.md 期望 1 条yogurt_british_clean.md0British英式拼写语料验证零误报run_tests.rs L89title_case_clean.md0American标题大小写语料零误报对照title_case_errors.md期望 2 条issue_2054_clean.md0Britishissue 修复后的 clean 复测从这套登记方式可以推断 Harper 测试语料的维护流程当某条规则修复了误报 bug如 issue/PR 场景修复前用 dirty 语料钉住 bug 的期望 lint 数修复后再用 clean 语料如issue_2054_clean.md确认正确写法不再被击中。misc_closed_compound_clean.md属于同一思路的通用版——它不绑定单个 issue而是持续守护整组封闭复合词规则在正确拼写密集文本上的整体行为。5. 小结misc_closed_compound_clean.md 是一份 15 行、8 句的零 lint回归语料专门验证 Harper 的封闭复合词规则及整个 curated 规则集在here/there/hereby等高频词密集文本上不误报它由 run_tests.rs 第 87 行的create_test!宏消费展开后走编译期嵌入 → Markdown 解析 →LintGroup::new_curated全量检查 → 断言 0 条 lint 且 token span 完整的完整链路被守护的规则实现位于 closed_compounds.rs60 余条空格错拆 → 封闭形式映射与 map_phrase_linter.rsLongestMatchOf最长匹配 保大小写替换经 lint_group/mod.rs 的merge_from默认启用若你在 Harper 中新增封闭复合词规则推荐同步遵循 clean/dirty 语料对的做法正向用例写入closed_compounds.rs内的assert_suggestion_result单测并视情况在 test_sources 中补充 clean 语料防止误报回归。【免费下载链接】harperOffline, privacy-first grammar checker. Fast, open-source, Rust-powered项目地址: https://gitcode.com/GitHub_Trending/har/harper创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考