目录服务器上的三个决策模型健康检查LayaKevNanoJev服务器上这次的实测结果服务器上的三个决策模型服务器是Ubuntu没有显卡用 CPU 跑。本机的 8010、8011 已经被别的程序占用所以这三个服务用新端口并且只监听服务器自己的127.0.0.1。服务服务器上的地址权重Layahttp://127.0.0.1:18110多语言版3.22 亿参数Kevhttp://127.0.0.1:18008Kev-0.8BNanoJevhttp://127.0.0.1:18111NanoJev 0.6B健康检查curl -sS http://127.0.0.1:18110/health curl -sS http://127.0.0.1:18008/v1/models curl -sS http://127.0.0.1:18111/api/healthLayacurl -sS http://127.0.0.1:18110/v1/systemone \ -H content-type: application/json \ -d { state: {body: 三月被扣了两次款请今天退款不然我们就取消套餐。}, questions: { department: { type: choice, instructions: 这个工单应该交给哪个部门, criteria: { billing: 发票、扣款、退款, tech: 故障、bug, other: 其他 } }, urgent: {type: noul, instructions: 是否需要今天处理}, anger: { type: score, instructions: 客户有多着急, criteria: [不急, 尽快, 立刻] } } }看answers.department.choice部门、answers.urgent.noul“是”的概率、answers.anger.score0 是不急1 是尽快2 是立刻。响应结果 { model: laya-rl-agent, answers: { department: { type: choice, choice: billing, probabilities: { billing: 1, tech: 0, other: 0 }, confidence: 0.9997, answer_confidence: 1, action: { act_probability: 1 } }, urgent: { type: noul, noul: 0.8202, confidence: 0.8202, answer_confidence: 0.8202, action: { act_probability: 1 } }, anger: { type: score, score: 1.358, legend: { 0: 不急, 1: 尽快, 2: 立刻 }, probabilities: { 0: 0.0299, 1: 0.5822, 2: 0.3879 }, confidence: 0.2835, answer_confidence: 0.5822, action: { act_probability: 1 } } }, usage: { input_tokens: 162, output_tokens: 0 }, routing: { model: multilingual, repo: convaiinnovations/laya/multilingual, reason: non-Latin script (han, 100% of letters); the English checkpoint cannot read it, detection: { script: han, script_profile: { han: 1 }, language: null, is_english: false, language_undecided: true, diacritic_rate: 0, non_latin_fraction: 1 }, workflow: null } }Kevcurl -sS http://127.0.0.1:18008/v1/systemone \ -H content-type: application/json \ -d { state: 订单晚了两周鞋码也不对卡里还被扣了两次。, model: kev-latest, questions: { department: { type: choice, instructions: 哪个团队来处理, criteria: { returns: 退换货、尺码或损坏, shipping: 物流延误、丢件, billing: 扣款、发票、支付 } }, escalate: {type: noul, instructions: 需要马上人工处理吗}, frustration: { type: score, instructions: 客户挫败感有多强, criteria: [平静, 不满, 非常生气] } } }字段和 Laya 一样。latency_ms是这一次花了多少毫秒。CPU 上会比你的 Mac 慢。响应结果 { model: kev-latest, answers: { department: { type: choice, choice: returns, confidence: 0.1315, probabilities: { returns: 0.421, shipping: 0.4068, billing: 0.1722 } }, escalate: { type: noul, noul: 0.6857 }, frustration: { type: score, score: 1.4159, legend: { 0: 平静, 1: 不满, 2: 非常生气 }, probabilities: { 0: 0.0479, 1: 0.4884, 2: 0.4637 }, confidence: 0.7442 } }, usage: { input_tokens: 87, output_tokens: 212 }, latency_ms: 1358.8 }NanoJev是非题写boolean看p_true。curl -sS http://127.0.0.1:18111/api/evaluate \ -H content-type: application/json \ -d { states: [ { id: 工单1, state: 三月被扣了两次款请今天退款。, questions: { department: { type: choice, instructions: 这个工单应该交给哪个部门, criteria: { billing: 发票、扣款、退款, tech: 故障和 bug, other: 其他事项 } }, billing: { type: boolean, instructions: 这是不是账单问题, criteria: {true: 明确在说扣款或退款, false: 与扣款无关} }, anger: { type: score, instructions: 客户有多着急, criteria: [不急, 尽快, 立刻] } } } ] }结论在states[0].answers。响应结果 { schema_version: openjev-toy-inference-v1, checkpoint: { directory: /opt/decision-models/models/NanoJev, base_model: Qwen/Qwen3-0.6B, base_revision: c1899de289a04d12100db370d81485cdf75e47ca, set_head: attention }, temperature: { value: 1, fitted_by_this_command: false, note: 显式应用给定标量默认1不表示模型已校准。 }, execution: { device: cpu, parameter_storage: float32, precision: fp32, forward_autocast: disabled, states: 1, questions: 3, candidate_paths: 7, forward_passes: 1, batch_questions_limit: all, autoregressive_decode_steps: 0, prefix_sharing: false, max_length: 8192, disable_native_triton: true, network_model_calls: 0, persistent_model_load_count: 1, inference_call_index: 5, note: 上游入口只接受 CUDA本进程在 Apple MPS 上用同一份权重做本地推理 }, states: [ { id: 工单1, answers: { department: { type: choice, probabilities: { billing: 0.33446261286735535, tech: 0.4972914159297943, other: 0.16824595630168915 }, choice: tech, value: tech }, billing: { type: boolean, probabilities: { false: 0.15340343117713928, true: 0.8465965986251831 }, p_true: 0.8465965986251831, value: true }, anger: { type: score, probabilities: { 0: 0.18612158298492432, 1: 0.41676998138427734, 2: 0.3971083462238312 }, score: 1.2109866738319397, level: 1, value: 1.2109866738319397 } } } ] }服务器上这次的实测结果Laya部门billing概率 0.9997“是否今天处理”是 0.65。Kev “需要马上人工处理吗”是 0.69这一次约 1 秒。NanoJev“是不是账单问题”的p_true是 0.85value为 true。三个服务的安装位置 服务 配置文件 Laya /etc/systemd/system/laya.service Kev /etc/systemd/system/kev.service NanoJev /etc/systemd/system/nanojev.servicesystemctl status laya kev nanojev结论我的乌班图服务器是8核心16G的配置没有显卡接口响应时间再1秒内差不多。