LLM Integration¶
Kairos makes four kinds of LLM call: fit scoring, skill extraction, JD analysis and resume
tailoring. Each service depends on one port, LlmClient. One setting,
[llm] backend, picks the adapter behind it for every call
(decision 0002):
| backend | adapter | how it pays | structured output | streaming |
|---|---|---|---|---|
claude-cli (default) |
ClaudeCliClient, one isolated claude -p per call |
the user's Claude subscription | --json-schema |
--output-format stream-json |
litellm |
LiteLlmClient |
per token, a provider API key | forced tool use | LiteLLM streaming |
Architecture¶
graph TB
subgraph "kairos.core (ports)"
P[LlmClient]
T0[SkillExtractorService]
T1[JdAnalysisService]
T2[ResumeTailorService]
end
subgraph "kairos.llm (services, one copy each)"
FS[FitScorer]
SE[SkillExtractor]
A[Analyzer]
T[Tailor]
F["get_llm_client()\nreads [llm] backend"]
end
subgraph "kairos.llm (adapters)"
CLI[ClaudeCliClient]
LL[LiteLlmClient]
end
SE --> T0
A --> T1
T --> T2
FS --> P
SE --> P
A --> P
T --> P
F --> CLI
F --> LL
CLI -.implements.-> P
LL -.implements.-> P
CLI --> Claude["Claude Code (subscription)"]
LL --> Providers["Claude / GPT / Gemini / ... APIs"]
get_llm_client(llm, model=, effort=) is the only code that reads backend. Fit
scoring passes its own model and effort ([jobs] fit_model, fit_effort), so a cheap
model can score hundreds of postings while tailoring uses a stronger one.
Claude CLI adapter¶
- Each call is a fresh process with no tools, no MCP servers, no settings files and no saved session. Kairos's prompt replaces Claude Code's own system prompt, which cuts tokens about 14x.
- System messages become
--system-prompt, and the rest become one user turn on stdin. - There is no temperature control and no model fallback.
temperatureis accepted and ignored. - Usage comes from the reply envelope.
total_cost_usdis the API-price equivalent, whatever the subscription actually costs. - A signed-out CLI raises
ClaudeCliAuthError, anLlmAuthError, so a scoring run stops at once instead of failing three postings in a row. - It needs a signed-in
claudeon the API's PATH ([llm] claude_bin), orCLAUDE_CODE_OAUTH_TOKENin a container.
LiteLLM adapter¶
LiteLLM gives one interface to 100+ providers. To switch, change the model string:
from litellm import acompletion
# Claude
response = await acompletion(model="claude-sonnet-4-20250514", messages=[...])
# GPT
response = await acompletion(model="gpt-4o", messages=[...])
# Gemini
response = await acompletion(model="gemini/gemini-2.0-flash", messages=[...])
Fallback Chains (litellm)¶
Configure fallback models in case primary provider is unavailable:
[llm]
backend = "litellm"
model = "claude-sonnet-4-20250514"
fallback_models = ["gpt-4o", "gemini/gemini-2.0-flash"]
Provider API Keys¶
Each provider needs its own API key, configured via environment variables:
Only the key for the configured model (and fallbacks) is required. The claude-cli backend needs none of them.
Skill Extraction¶
Runs once during resume import to build the user's Skill Profile.
Prompt Strategy¶
Temperature: 0.0 (deterministic extraction)
The prompt instructs the LLM to: 1. Analyze each work experience and project section from the resume 2. Extract every technical skill mentioned (languages, frameworks, databases, tools, concepts) 3. Estimate proficiency (1-10) based on context clues: duration, depth of usage, role seniority 4. Estimate years of experience per skill 5. Cite evidence — which project/role demonstrates each skill
Output is parsed as a list[SkillEntry] via tool use with Pydantic schema. Users can review and adjust proficiency ratings after generation.
JD Analysis¶
Prompt Strategy¶
Temperature: 0.0 (deterministic extraction)
The prompt instructs the LLM to: 1. Extract structured data from the JD text 2. Identify required vs preferred skills with estimated proficiency level needed 3. Extract ATS keywords 4. Compare against the user's Skill Profile (proficiency-aware, not just binary) 5. Compute a match score (0.0 - 1.0) with per-skill breakdown and reasoning
Output is parsed as a JdAnalysis Pydantic model via structured output.
Resume Tailoring¶
Anti-Hallucination Rules (enforced in system prompt)¶
- Only rephrase, reorder, or emphasize existing content
- Never invent skills, experiences, achievements, or metrics
- Never change company names, dates, job titles, education details
- Only modify sections marked as tailorable
- Preserve the original format (LaTeX or Markdown)
Temperature: 0.3 (controlled creativity)
Validation¶
After LLM response, a validation step checks: - No new company names or job titles appeared - No new degree or institution names - Dates haven't changed - Section types match what was sent
If validation fails, retry with stricter prompt or flag to user.
Output Flow¶
flowchart TD
Input["Base Resume Sections\n+ JD Analysis"] --> LLM["LLM ([llm] backend)"]
LLM --> Raw[Raw LLM Response]
Raw --> Parse[Parse modified sections]
Parse --> Validate[Validate: no fabricated content]
Validate --> |Pass| Diff[Generate diff]
Validate --> |Fail| Retry[Retry with stricter prompt]
Diff --> User[Show to user for approval]
Structured Output Pattern¶
Every service asks for a Pydantic model through LlmClient.complete_structured, and
each adapter enforces it in its own way. LiteLLM forces a tool call whose parameters are
schema.model_json_schema(). Claude Code takes the same schema as --json-schema and
returns it as structured_output. Both validate the reply against the model, and a
mismatch raises LlmInvalidResponseError.
Cost¶
| Operation | Tokens (approx) | Cost at Sonnet | Frequency |
|---|---|---|---|
| Skill Extraction | 1,000-2,000 | ~$0.01-0.02 | Once per resume import |
| JD Analysis | 500-1,500 | ~$0.003-0.01 | Per job |
| Resume Tailoring | 1,000-3,000 | ~$0.01-0.03 | Per job |
LiteLLM reports cost via response._hidden_params["response_cost"]. Claude Code reports
total_cost_usd, the API-price equivalent. kairos usage compares that with the plan in
[usage] plan.
Model Configuration¶
[llm]
backend = "claude-cli" # or "litellm"
model = "" # "" = sonnet on claude-cli, claude-sonnet-4-20250514 on litellm
effort = "" # claude-cli only
claude_bin = "claude" # claude-cli only
request_timeout = 180 # seconds, both backends
fallback_models = [] # litellm only
api_base = "" # litellm only
temperature_extraction = 0.0 # litellm only
temperature_tailoring = 0.3 # litellm only
[jobs]
fit_model = "" # scoring only; "" = haiku on claude-cli, [llm] model on litellm
fit_effort = "" # scoring only, claude-cli; "" = low
A claude-cli model is an alias (haiku, sonnet, opus) or a Claude model name. A
LiteLLM id such as anthropic/... is refused there, so switching backends needs a
matching model, or "". KAIROS_LLM_MODEL overrides [llm] model. The Settings page
and kairos config set llm.backend litellm both change the backend.
Older configs load unchanged. [jobs] fit_scorer, claude_bin and fit_timeout_seconds
move to [llm] backend, claude_bin and request_timeout.