Skip to content

0002: One LLM backend setting for every LLM call

  • Date: 2026-11-09
  • Status: accepted

Problem

Kairos makes four kinds of LLM call: fit scoring, JD analysis, resume tailoring and skill extraction. Only fit scoring could run through Claude Code on the user's subscription ([jobs] fit_scorer = "claude-cli"), through its own ClaudeCliFitScorer. The other three always went through LiteLLM and needed a pay-per-token API key.

So a subscription-only install, like the reference deployment, scored jobs but failed to tailor a resume. The backend choice sat under [jobs], so it looked like a scoring setting, not an LLM one. And Claude CLI support lived in a scorer class instead of the LlmClient port, so every new LLM feature would need its own CLI copy.

Options

  1. Add a CLI twin for each service (ClaudeCliTailor, ClaudeCliAnalyzer, ...). Each is small, but each new feature needs two implementations, and the prompts can drift between them.
  2. Implement the LlmClient port a second time over claude -p, and let one [llm] backend setting choose the adapter for every call. The services and prompts stay single.
  3. Keep LiteLLM only and require an API key. This is simplest, but it drops subscription users, who are the main audience of a self-hosted tool.

Choice

Option 2. kairos.llm has two adapters behind LlmClient:

backend adapter billing notes
claude-cli (default) ClaudeCliClient (claude -p) the user's Claude subscription --json-schema for structured output, --output-format stream-json for tailoring's streaming; no temperature control
litellm LiteLlmClient per token, provider API key model fallback, api_base, any LiteLLM model

get_llm_client(llm, model=, effort=) is the only place that reads backend. The services (FitScorer, Analyzer, Tailor, SkillExtractor) take any LlmClient.

Settings, all in [llm] except the scoring overrides:

  • backend: claude-cli or litellm.
  • model: "" means the backend's default (sonnet for claude-cli, claude-sonnet-4-20250514 for litellm). A claude-cli model is an alias or a Claude model name. A LiteLLM id such as anthropic/... is refused there.
  • effort and claude_bin: used by claude-cli only.
  • request_timeout: used by both backends.
  • fallback_models and api_base: used by litellm only.
  • [jobs] fit_model and fit_effort override model and effort for scoring only. "" means the backend's scoring default (haiku/low on claude-cli; [llm] on litellm).

Old configs keep working. [jobs] fit_scorer, claude_bin and fit_timeout_seconds move to [llm] backend, claude_bin and request_timeout on load. A claude-cli config that still holds the old LiteLLM default model gets the backend default instead. The kairos watch entry point keeps its own settings; it is a separate process with its own tool-using prompt.