0012: Fit rules from a retro, and story drafts for unmatched criteria¶
- Date: 2026-11-11
- Status: accepted
Problem¶
Kairos keeps facts but does not learn from the owner's decisions.
- Fit scores do not learn. The owner skips postings and applies to others, and many of
those decisions disagree with the score: a 82 skipped, a 61 applied to. The score never
hears about it. The only way to correct it is to edit
[profile.fit]by hand, which means first noticing the pattern. - The evidence library does not grow. A cover letter or criteria response for a job needs one evidence story per criterion (decision 0008). A criterion with no matching story is left thin, and the owner has to write a story from scratch, even when the base resume already holds the facts for one.
Research¶
Agent memory projects reviewed on 2026-11-11:
- mem0, Letta, Zep/Graphiti, LangMem. They extract facts from conversations into a vector or graph store. Kairos has no conversation: its signals are structured (a job's status and fit score), and PostgreSQL already holds them. A second store would hold copies of facts that have an owner here.
- career-ops (MIT). Memory is files. Its
retromode reviews past evaluations and saves only the rules the user accepts; its Story Bank collects STAR stories across evaluations. Both put a person between what the model proposes and what is kept. - Devin. Its Knowledge items are short notes the agent suggests and the user approves.
The shared lesson: the model proposes, the owner approves, and only approved items reach a prompt. That is also how mail suggestions work already (decision 0006).
Choice¶
1. Fit rules¶
- Signals. A decided job is one with a fit score whose status says what the owner
thought of it:
skipped(not for me), or any application status (worth applying). Rejection and withdrawal still count as applied: they say what the employer or the owner later did, not whether the posting was worth applying to. A disagreement is a skipped job scored at or above 70, or an applied job scored below 70 (70 is where_SCALEin the scoring prompt starts "good match"). - Retro.
kairos profile rules retroasks the LLM to read up to 40 recent disagreements (title, company, location, score, the score's reason and the owner's verdict) next to the rules already accepted, and propose up to 3 new rules. It needs at least 3 disagreements; with fewer it says so and calls no model. - Checks in code. A rule must cite at least 2 of the jobs it was shown; ids the model invents are dropped first. A rule is at most 300 characters. A rule whose text matches an open or accepted rule (case and spacing ignored) is dropped.
- Review. Proposals are stored as
proposed. The owner accepts, ignores, or later retires each one (kairos profile rules list | accept | ignore | retire). Nothing changes a score until a rule is accepted. - Use. The scorer reads the accepted rules at the start of each run and lists them in
the system prompt as the candidate's own rules. Old scores stay until a rescore
(
kairos jobs score --rescore), so accepting a rule never triggers LLM calls by itself. - Owner of the fact. The
fit_rulestable.[profile.fit]stays the owner of the weights; a rule is an addition, not a weight.
2. Story drafts¶
- Trigger.
kairos evidence draft JOBreads the job's criteria (the stored ones, or read now, askairos generatedoes) and picks the criteria that no library story matches (best_stories), essential ones first, at most 3. - Drafting. For each, the LLM writes one STAR story from the base resume only, with
Source: Base resume, <entry heading>, or answers that the resume holds no evidence for it. A real gap is reported, never filled. - Checks in code. A draft must pass the evidence import checks (no email address, no internal host name) and the document fact check against the base resume alone: no new numbers, no new terms (the criterion's words do not count as the candidate's), no buzzwords or scope upgrades, one language. A draft that fails is reported with its failures and not stored.
- Review. Drafts are stored as
pending. Accepting one keeps the owner's Markdown file the source of the library (decision 0008): the CLI appends the story to the imported file, re-imports it, then marks the draft accepted. It refuses when the file is missing on this machine or has changed since the last import, so an unsaved edit is never overwritten. The owner may edit the story in the file afterwards and import again. - Owner of the fact. Drafts live in
story_draftsuntil decided; an accepted story's owner is the Markdown file.
Not in scope¶
- Running the retro or drafting on a timer, or reporting proposals on Telegram.
- Learning from free-text notes, or semantic search over stories (pgvector).
- Rescoring automatically when a rule is accepted.