Skip to content

LLM Integration

Kairos makes four kinds of LLM call: fit scoring, skill extraction, JD analysis and resume tailoring. Each service depends on one port, LlmClient. One setting, [llm] backend, picks the adapter behind it for every call (decision 0002):

backend adapter how it pays structured output streaming
claude-cli (default) ClaudeCliClient, one isolated claude -p per call the user's Claude subscription --json-schema --output-format stream-json
litellm LiteLlmClient per token, a provider API key forced tool use LiteLLM streaming

Architecture

graph TB
    subgraph "kairos.core (ports)"
        P[LlmClient]
        T0[SkillExtractorService]
        T1[JdAnalysisService]
        T2[ResumeTailorService]
    end

    subgraph "kairos.llm (services, one copy each)"
        FS[FitScorer]
        SE[SkillExtractor]
        A[Analyzer]
        T[Tailor]
        F["get_llm_client()\nreads [llm] backend"]
    end

    subgraph "kairos.llm (adapters)"
        CLI[ClaudeCliClient]
        LL[LiteLlmClient]
    end

    SE --> T0
    A --> T1
    T --> T2
    FS --> P
    SE --> P
    A --> P
    T --> P
    F --> CLI
    F --> LL
    CLI -.implements.-> P
    LL -.implements.-> P
    CLI --> Claude["Claude Code (subscription)"]
    LL --> Providers["Claude / GPT / Gemini / ... APIs"]

get_llm_client(llm, model=, effort=) is the only code that reads backend. Fit scoring passes its own model and effort ([jobs] fit_model, fit_effort), so a cheap model can score hundreds of postings while tailoring uses a stronger one.

Claude CLI adapter

  • Each call is a fresh process with no tools, no MCP servers, no settings files and no saved session. Kairos's prompt replaces Claude Code's own system prompt, which cuts tokens about 14x.
  • System messages become --system-prompt, and the rest become one user turn on stdin.
  • There is no temperature control and no model fallback. temperature is accepted and ignored.
  • Usage comes from the reply envelope. total_cost_usd is the API-price equivalent, whatever the subscription actually costs.
  • A signed-out CLI raises ClaudeCliAuthError, an LlmAuthError, so a scoring run stops at once instead of failing three postings in a row.
  • It needs a signed-in claude on the API's PATH ([llm] claude_bin), or CLAUDE_CODE_OAUTH_TOKEN in a container.

LiteLLM adapter

LiteLLM gives one interface to 100+ providers. To switch, change the model string:

from litellm import acompletion

# Claude
response = await acompletion(model="claude-sonnet-4-20250514", messages=[...])

# GPT
response = await acompletion(model="gpt-4o", messages=[...])

# Gemini
response = await acompletion(model="gemini/gemini-2.0-flash", messages=[...])

Fallback Chains (litellm)

Configure fallback models in case primary provider is unavailable:

[llm]
backend = "litellm"
model = "claude-sonnet-4-20250514"
fallback_models = ["gpt-4o", "gemini/gemini-2.0-flash"]

Provider API Keys

Each provider needs its own API key, configured via environment variables:

ANTHROPIC_API_KEY=sk-ant-...      # Claude
OPENAI_API_KEY=sk-...             # GPT
GEMINI_API_KEY=...                # Gemini

Only the key for the configured model (and fallbacks) is required. The claude-cli backend needs none of them.

Skill Extraction

Runs once during resume import to build the user's Skill Profile.

Prompt Strategy

Temperature: 0.0 (deterministic extraction)

The prompt instructs the LLM to: 1. Analyze each work experience and project section from the resume 2. Extract every technical skill mentioned (languages, frameworks, databases, tools, concepts) 3. Estimate proficiency (1-10) based on context clues: duration, depth of usage, role seniority 4. Estimate years of experience per skill 5. Cite evidence — which project/role demonstrates each skill

Output is parsed as a list[SkillEntry] via tool use with Pydantic schema. Users can review and adjust proficiency ratings after generation.

JD Analysis

Prompt Strategy

Temperature: 0.0 (deterministic extraction)

The prompt instructs the LLM to: 1. Extract structured data from the JD text 2. Identify required vs preferred skills with estimated proficiency level needed 3. Extract ATS keywords 4. Compare against the user's Skill Profile (proficiency-aware, not just binary) 5. Compute a match score (0.0 - 1.0) with per-skill breakdown and reasoning

Output is parsed as a JdAnalysis Pydantic model via structured output.

Resume Tailoring

Anti-Hallucination Rules (enforced in system prompt)

  1. Only rephrase, reorder, or emphasize existing content
  2. Never invent skills, experiences, achievements, or metrics
  3. Never change company names, dates, job titles, education details
  4. Only modify sections marked as tailorable
  5. Preserve the original format (LaTeX or Markdown)

Temperature: 0.3 (controlled creativity)

Validation

After LLM response, a validation step checks: - No new company names or job titles appeared - No new degree or institution names - Dates haven't changed - Section types match what was sent

If validation fails, retry with stricter prompt or flag to user.

Output Flow

flowchart TD
    Input["Base Resume Sections\n+ JD Analysis"] --> LLM["LLM ([llm] backend)"]
    LLM --> Raw[Raw LLM Response]
    Raw --> Parse[Parse modified sections]
    Parse --> Validate[Validate: no fabricated content]
    Validate --> |Pass| Diff[Generate diff]
    Validate --> |Fail| Retry[Retry with stricter prompt]
    Diff --> User[Show to user for approval]

Structured Output Pattern

Every service asks for a Pydantic model through LlmClient.complete_structured, and each adapter enforces it in its own way. LiteLLM forces a tool call whose parameters are schema.model_json_schema(). Claude Code takes the same schema as --json-schema and returns it as structured_output. Both validate the reply against the model, and a mismatch raises LlmInvalidResponseError.

Cost

Operation Tokens (approx) Cost at Sonnet Frequency
Skill Extraction 1,000-2,000 ~$0.01-0.02 Once per resume import
JD Analysis 500-1,500 ~$0.003-0.01 Per job
Resume Tailoring 1,000-3,000 ~$0.01-0.03 Per job

LiteLLM reports cost via response._hidden_params["response_cost"]. Claude Code reports total_cost_usd, the API-price equivalent. kairos usage compares that with the plan in [usage] plan.

Model Configuration

[llm]
backend = "claude-cli"          # or "litellm"
model = ""                      # "" = sonnet on claude-cli, claude-sonnet-4-20250514 on litellm
effort = ""                     # claude-cli only
claude_bin = "claude"           # claude-cli only
request_timeout = 180           # seconds, both backends
fallback_models = []            # litellm only
api_base = ""                   # litellm only
temperature_extraction = 0.0    # litellm only
temperature_tailoring = 0.3     # litellm only

[jobs]
fit_model = ""                  # scoring only; "" = haiku on claude-cli, [llm] model on litellm
fit_effort = ""                 # scoring only, claude-cli; "" = low

A claude-cli model is an alias (haiku, sonnet, opus) or a Claude model name. A LiteLLM id such as anthropic/... is refused there, so switching backends needs a matching model, or "". KAIROS_LLM_MODEL overrides [llm] model. The Settings page and kairos config set llm.backend litellm both change the backend.

Older configs load unchanged. [jobs] fit_scorer, claude_bin and fit_timeout_seconds move to [llm] backend, claude_bin and request_timeout.