Skip to content

0012: Fit rules from a retro, and story drafts for unmatched criteria

  • Date: 2026-11-11
  • Status: accepted

Problem

Kairos keeps facts but does not learn from the owner's decisions.

  • Fit scores do not learn. The owner skips postings and applies to others, and many of those decisions disagree with the score: a 82 skipped, a 61 applied to. The score never hears about it. The only way to correct it is to edit [profile.fit] by hand, which means first noticing the pattern.
  • The evidence library does not grow. A cover letter or criteria response for a job needs one evidence story per criterion (decision 0008). A criterion with no matching story is left thin, and the owner has to write a story from scratch, even when the base resume already holds the facts for one.

Research

Agent memory projects reviewed on 2026-11-11:

  • mem0, Letta, Zep/Graphiti, LangMem. They extract facts from conversations into a vector or graph store. Kairos has no conversation: its signals are structured (a job's status and fit score), and PostgreSQL already holds them. A second store would hold copies of facts that have an owner here.
  • career-ops (MIT). Memory is files. Its retro mode reviews past evaluations and saves only the rules the user accepts; its Story Bank collects STAR stories across evaluations. Both put a person between what the model proposes and what is kept.
  • Devin. Its Knowledge items are short notes the agent suggests and the user approves.

The shared lesson: the model proposes, the owner approves, and only approved items reach a prompt. That is also how mail suggestions work already (decision 0006).

Choice

1. Fit rules

  • Signals. A decided job is one with a fit score whose status says what the owner thought of it: skipped (not for me), or any application status (worth applying). Rejection and withdrawal still count as applied: they say what the employer or the owner later did, not whether the posting was worth applying to. A disagreement is a skipped job scored at or above 70, or an applied job scored below 70 (70 is where _SCALE in the scoring prompt starts "good match").
  • Retro. kairos profile rules retro asks the LLM to read up to 40 recent disagreements (title, company, location, score, the score's reason and the owner's verdict) next to the rules already accepted, and propose up to 3 new rules. It needs at least 3 disagreements; with fewer it says so and calls no model.
  • Checks in code. A rule must cite at least 2 of the jobs it was shown; ids the model invents are dropped first. A rule is at most 300 characters. A rule whose text matches an open or accepted rule (case and spacing ignored) is dropped.
  • Review. Proposals are stored as proposed. The owner accepts, ignores, or later retires each one (kairos profile rules list | accept | ignore | retire). Nothing changes a score until a rule is accepted.
  • Use. The scorer reads the accepted rules at the start of each run and lists them in the system prompt as the candidate's own rules. Old scores stay until a rescore (kairos jobs score --rescore), so accepting a rule never triggers LLM calls by itself.
  • Owner of the fact. The fit_rules table. [profile.fit] stays the owner of the weights; a rule is an addition, not a weight.

2. Story drafts

  • Trigger. kairos evidence draft JOB reads the job's criteria (the stored ones, or read now, as kairos generate does) and picks the criteria that no library story matches (best_stories), essential ones first, at most 3.
  • Drafting. For each, the LLM writes one STAR story from the base resume only, with Source: Base resume, <entry heading>, or answers that the resume holds no evidence for it. A real gap is reported, never filled.
  • Checks in code. A draft must pass the evidence import checks (no email address, no internal host name) and the document fact check against the base resume alone: no new numbers, no new terms (the criterion's words do not count as the candidate's), no buzzwords or scope upgrades, one language. A draft that fails is reported with its failures and not stored.
  • Review. Drafts are stored as pending. Accepting one keeps the owner's Markdown file the source of the library (decision 0008): the CLI appends the story to the imported file, re-imports it, then marks the draft accepted. It refuses when the file is missing on this machine or has changed since the last import, so an unsaved edit is never overwritten. The owner may edit the story in the file afterwards and import again.
  • Owner of the fact. Drafts live in story_drafts until decided; an accepted story's owner is the Markdown file.

Not in scope

  • Running the retro or drafting on a timer, or reporting proposals on Telegram.
  • Learning from free-text notes, or semantic search over stories (pgvector).
  • Rescoring automatically when a rule is accepted.