Zanbot Dev Diary, Pt. 1: Long-Term Memory and a Better Command Line

What I did today

First, a look at other products

Before starting, I wanted to understand why people praise Dots, Grok Bot, and Muse. There was a mix-up in the middle: I meant OpenAI's Dots, but the first search turned up New Computer's Dot, a companion app that already shut down in 2025.

After looking around, the three turn out to follow the same path: each agent gets its own cloud computer and works for you in the background. The difference is in how they earn trust. Dots has three permission levels (do it yourself / ask first / never), and Muse has a dedicated secure VM plus a separate Sentinel agent that guards sensitive actions. Either way, an assistant that wants to help proactively has to know the person first. So zanbot starts with long-term memory, and I'm not borrowing anything from Dots for now.

Long-term memory

Why FTS instead of embeddings

I didn't know FTS before, so I learned it today. FTS means full-text search: it uses an inverted index to quickly find records that contain a word. SQLite ships with FTS5, so there is nothing to install.

Chinese has no spaces, so the default tokenizer treats a whole sentence as one word. The trigram tokenizer fixes that by cutting text into overlapping three-character pieces. The cost is that a query needs at least three characters, so a two-character word like "咖啡" (coffee) has to fall back to LIKE.

Why not embeddings? Neither of my current providers offers them: Anthropic has no embedding API of its own (its docs recommend Voyage), and ChatGPT plan sign-in only exposes /models and /responses. Using another provider would mean managing one more API key. So the plan is:

• Version 1 uses FTS. While extracting a memory, the model also generates a few keywords (synonyms, broader terms) that get indexed with it. That way a word like "提神" (something to wake me up) can still find "美式咖啡" (Americano), without an extra model call.
• Version 2 can optionally use local fastembed, combined with FTS through RRF hybrid retrieval.

For now there are very few memories, so recall is just "send everything", trimmed to a 2,000-character budget:

# zanbot/memory.py:152
def recall(self, budget=RECALL_BUDGET, today=None):
    """Select memories for the system prompt without exceeding the character budget."""
    selected, used = [], 0
    for memory in self.active(today):
        cost = len(memory.content) + len(memory.kind) + 5
        if used + cost > budget:
            continue
        selected.append(memory)
        used += cost
    return selected

The SQL in active() already sorts by "profile first, then most recently updated". So when the model "remembers me", it's really because these notes are written into the system prompt it receives every turn.

Automatic extraction: a real model showed two problems right away

P2 runs one more model call on a background thread after each successful reply, and asks for add / update / delete operations as JSON. All offline tests passed, but a real model exposed two problems immediately.

I said "I've been learning OAuth lately" (in Chinese), then corrected myself: "actually, I've been learning FTS". The results:

1. It saved the assistant's words as facts about me. The note said "the user is learning OAuth, focusing on PKCE and OpenID Connect". But PKCE and OIDC came from the assistant's answer. I never mentioned them.
2. It didn't update the old note on a correction. When I said "actually it's FTS", it added a new FTS note and left the OAuth one alone. The two contradicted each other until the next turn, when I said outright "I'm not learning OAuth", and only then was it deleted.

The fix was to spell the rule out in the extraction prompt, with a right and a wrong example:

# zanbot/memory_extract.py:22
- Every note must be stated or clearly confirmed by the user in <user>. Read <assistant> only to understand what the user meant; never copy the assistant's explanations, suggestions, or lists into a note, and do not add details the user did not say.
  Example: the user says "我在学 OAuth" and the assistant explains PKCE and OIDC.
  Right: "用户最近在学 OAuth". Wrong: "用户在学 OAuth，重点关注 PKCE 和 OIDC".

I also added a rule: when the user says things like "actually" or "not X but Y", update the old note in the same turn. After retesting, both problems were gone.

The code can't fully trust the model's output either. The id in an update or delete must be one of the candidates shown to the model this time:

# zanbot/memory_extract.py:84
if op["op"] != "add":
    # Only ids shown to the model may change, so a bad reply cannot touch other memories.
    memory_id = parse_id(op.get("id"))
    if memory_id not in candidate_ids:
        raise ValueError("The extraction reply referenced an unknown memory id.")

One more small finding: the model would casually say "I'll remember that", even though it has no idea whether the background step saved anything. I later added an explanation of how zanbot's memory works to the system prompt, telling it not to say that, and not to invent features like "turn off memory in settings" either.

Command-line experience

Pull out a session layer first

Before, main.py mixed conversation logic with print calls, commands were spread across several places, and a typo like /memroy was sent straight to the model. C1 was a pure refactor:

main.py            argument parsing, wiring
zanbot/cli.py      terminal frontend: reads input, renders events
zanbot/session.py  ChatSession: history, memory, command dispatch, output through emit(event)
zanbot/commands.py command registry, used by dispatch, /help, and completion

The session doesn't know a terminal exists; it only emits events (TextDelta, ReplyFinished, MemoryNotice, and so on). For a future browser or desktop app, emit can simply push to a WebSocket instead. None of the original 57 tests changed in this step, and all of them passed.

Problem 1: Shift+Enter wouldn't insert a new line

After C2 brought in prompt_toolkit, I wanted Shift+Enter for new lines. Neither Shift+Enter nor Ctrl+Enter worked in Windows Terminal.

The first diagnosis was wrong: I assumed prompt_toolkit turns Ctrl+Enter into a new line on Windows. Reading its source later showed that Windows has two ways of reading the keyboard. Only Legacy mode does that conversion, and on Windows 10 and later prompt_toolkit uses VT mode by default.

Reading source was still guessing, so I wrote a small script that prints the raw key events Windows delivers:

In VT mode, Shift+Enter is identical to Enter, so no program can tell them apart. Legacy mode keeps the modifier state. So on Windows, zanbot now reads in Legacy mode and turns Shift+Enter into Meta+Enter itself (which is bound to a new line):

# zanbot/cli.py:79
class ModifierAwareReader(ConsoleInputReader):
    def _event_to_key_presses(self, ev):
        presses = super()._event_to_key_presses(ev)
        if (ev.VirtualKeyCode == VK_RETURN and ev.ControlKeyState & self.SHIFT_PRESSED
                and [press.key for press in presses] == [Keys.ControlM]):
            return [KeyPress(Keys.Escape, ""), presses[0]]
        return presses

The cost is relying on a prompt_toolkit internal class, so the dependency is pinned to <4, and the tests use the real key events I captured.

That fixed Windows Terminal, but the VS Code terminal still didn't work. VS Code's xterm.js already drops the Shift when it forwards the key, and VS Code keeps Ctrl+J for itself. The only option there is a keybindings.json entry that makes Shift+Enter send Escape+Enter, which is now in the README.

Problem 2: on Windows, Ctrl+C can't interrupt a waiting request

C3 needed "press Ctrl+C during a reply to cancel the turn". Just catching KeyboardInterrupt looks like enough, but on Windows Ctrl+C can't interrupt a blocking socket read. While a reply is still waiting for its first token (a reasoning model can take several seconds), pressing Ctrl+C does nothing at all.

So the request now runs on its own thread, and the main thread joins every 0.1 seconds, which keeps it responsive to Ctrl+C:

# zanbot/session.py:129
thread = threading.Thread(target=work, name="zanbot-reply", daemon=True)
thread.start()
try:
    self.wait_for(thread)
except KeyboardInterrupt:
    # The abandoned thread stops at its next text chunk, or ends on its own timeout.
    cancelled.set()
    self.emit(ReplyFailed("Cancelled.", cancelled=True))
    return

The abandoned thread stops at its next text chunk. run_turn only changes a copy of the history, and only a turn that wasn't cancelled adopts that copy, so a cancel leaves the history unchanged and doesn't trigger memory extraction.

Two small fixes

During testing, ChatGPT once returned a 504. A 504 is a gateway timeout on OpenAI's side, and resending fixed it, but the message at the time said "check the selected model", which was misleading. Now a 502/503/504/529 makes zanbot wait 1.5 seconds and retry once:

# zanbot/llm.py:18
def post_with_retry(send):
    """Send a request, retrying once after a transient status.

    The status arrives before any streamed text, so a retry can never repeat visible output.
    """
    response = send()
    if response.status_code in RETRY_STATUSES:
        response.close()
        time.sleep(RETRY_DELAY)
        response = send()
    return response

The status code arrives before any text is shown, so a retry never produces a repeated half-answer.

Branches

Starting today, each feature gets its own branch: feature/memory, feature/cli-ux, feature/retry-and-memory-prompt. Each phase is committed only after I've tried it in a terminal myself, then merged back to master. master always runs and passes all tests.

What I learned today

• Passing offline tests doesn't mean the behavior is right. Both extraction-prompt problems only showed up when a real model ran through a real conversation. Before changing a prompt, it's best to first reproduce the problem with a conversation that triggers it, then retest with the same conversation after the change.
• When a terminal key problem comes up, it's best to capture the raw key events first instead of guessing from the source. I guessed wrong the first time today.
• The same feature behaves differently in different terminals: it works in Windows Terminal, not in VS Code, and mintty can't even enter interactive mode. Before saying something is "supported", it's best to actually run it in each terminal.
• Pulling out the session layer before bringing in prompt_toolkit meant C2 and C3 barely touched the conversation logic while changing the interface.