0002: One LLM backend setting for every LLM call¶
- Date: 2026-11-09
- Status: accepted
Problem¶
Kairos makes four kinds of LLM call: fit scoring, JD analysis, resume tailoring and
skill extraction. Only fit scoring could run through Claude Code on the user's
subscription ([jobs] fit_scorer = "claude-cli"), through its own ClaudeCliFitScorer.
The other three always went through LiteLLM and needed a pay-per-token API key.
So a subscription-only install, like the reference deployment, scored jobs but failed
to tailor a resume. The backend choice sat under [jobs], so it looked like a scoring
setting, not an LLM one. And Claude CLI support lived in a scorer class instead of the
LlmClient port, so every new LLM feature would need its own CLI copy.
Options¶
- Add a CLI twin for each service (
ClaudeCliTailor,ClaudeCliAnalyzer, ...). Each is small, but each new feature needs two implementations, and the prompts can drift between them. - Implement the
LlmClientport a second time overclaude -p, and let one[llm] backendsetting choose the adapter for every call. The services and prompts stay single. - Keep LiteLLM only and require an API key. This is simplest, but it drops subscription users, who are the main audience of a self-hosted tool.
Choice¶
Option 2. kairos.llm has two adapters behind LlmClient:
| backend | adapter | billing | notes |
|---|---|---|---|
claude-cli (default) |
ClaudeCliClient (claude -p) |
the user's Claude subscription | --json-schema for structured output, --output-format stream-json for tailoring's streaming; no temperature control |
litellm |
LiteLlmClient |
per token, provider API key | model fallback, api_base, any LiteLLM model |
get_llm_client(llm, model=, effort=) is the only place that reads backend. The
services (FitScorer, Analyzer, Tailor, SkillExtractor) take any LlmClient.
Settings, all in [llm] except the scoring overrides:
backend:claude-cliorlitellm.model:""means the backend's default (sonnetfor claude-cli,claude-sonnet-4-20250514for litellm). A claude-cli model is an alias or a Claude model name. A LiteLLM id such asanthropic/...is refused there.effortandclaude_bin: used by claude-cli only.request_timeout: used by both backends.fallback_modelsandapi_base: used by litellm only.[jobs] fit_modelandfit_effortoverride model and effort for scoring only.""means the backend's scoring default (haiku/lowon claude-cli;[llm]on litellm).
Old configs keep working. [jobs] fit_scorer, claude_bin and fit_timeout_seconds
move to [llm] backend, claude_bin and request_timeout on load. A claude-cli
config that still holds the old LiteLLM default model gets the backend default instead.
The kairos watch entry point keeps its own settings; it is a separate process with its
own tool-using prompt.