Skip to content
SYS-01Systemv3.2Operational~14 min read

AI Second Brain: personal knowledge OS, in production.

Capture, embed, retrieve, distill, publish. The system the rest of the lab depends on. Hybrid retrieval with reranking, six components across four layers, documented as the system evolves. This page documents how it currently works — not how it was built. The build history lives in the Second Brain build log.

Version · v3.2Phase · OperationalOwner · SoloLiving system reference

The system survives on its retrieval quality, not its capture surface. Everything else is infrastructure in service of that constraint.

The AI Second Brain is the operational core of this lab. Every framework published here was drafted against it. Every build log entry was distilled through it. Every research thread that has become useful did so because it was captured, indexed, and retrievable at the moment it was needed — not six weeks later, in a folder that stopped being checked.

This page documents the current operational architecture: what each component does, how they connect, what the system requires to function correctly, and where the current limitations are. It is not a tutorial. It is not a build narrative. Both of those live elsewhere. This is the reference document for a running system — accurate as of v3.2, updated when the architecture changes.

The system operationalizes the Knowledge Forge framework at the infrastructure level. Where that framework describes Capture, Distill, Connect, and Publish as a conceptual pipeline, this system implements those stages as running software with defined interfaces, data flows, and failure modes. The framework is the thinking. This is the implementation.

§ ISystem Architecture

Four layers. Six components. One retrieval constraint that governs everything.

The system is organized into four horizontal layers, each with a distinct responsibility. The layers are not a pipeline — data does not flow strictly top to bottom. The intelligence layer pulls from storage on demand. The output layer draws from intelligence on demand. Ingestion writes continuously to storage. The architecture reflects this: the diagram below shows data flows, not execution order.

CodeComponentLayerStatusDescription
SYS-01-ACapture SurfaceIngestionOperationalUnified ingestion plane across web, audio, and notes with provenance tagging at entry.
SYS-01-BEmbedding StoreStorageOperationalDense vector index over all captured content. Chunked at paragraph level, embedded on ingest.
SYS-01-CBM25 IndexStorageOperationalSparse keyword index running in parallel. Primary retrieval signal for exact-match and named-entity queries.
SYS-01-DRetrieval OrchestratorIntelligenceOperationalQuery planner that routes to dense, sparse, or hybrid retrieval. Applies reranking and source attribution before returning results.
SYS-01-EDistillation AgentIntelligenceOperational (v3.2)Background worker that compresses raw captures into linked notes. Produces claims, connections, and draft distillations for human review.
SYS-01-FPublishing SurfaceOutputOperationalTranslation layer between private knowledge graph and public artifacts. Enforces the would-publish standard before export.
Figure 1 · System Architecture · SYS-01 v3.2
INGESTIONSTORAGEINTELLIGENCEOUTPUTSYS-01-ACAPTURE SURFACEWeb · Audio · Notes · Provenance tagged at entrySYS-01-BEMBEDDING STOREDense vector index · Paragraph-level chunksSYS-01-CBM25 INDEXSparse keyword · Exact-match signalSYS-01-DRETRIEVAL ORCHESTRATORQuery planning · Hybrid fusionReranking · Source attributionSYS-01-EDISTILLATION AGENTBackground workerClaims · Connections · DraftsSYS-01-FPUBLISHING SURFACEPrivate graph → public artifact · Would-publish standard enforcedembedindexasync
Fig. 1 · SYS-01 v3.2 · Four layers · Six components · Dashed lines indicate async / background flows

The architecture diagram makes one constraint visible that the component list does not: the Retrieval Orchestrator is the hub. Every downstream use of the system — whether a query during writing, a distillation job, or a publishing export — passes through it. The quality of the entire system is therefore the quality of the retrieval layer. Capture surface quality and index freshness matter insofar as they affect what the orchestrator can return. Nothing else matters more than that.

§ IICapture Surface

Single ingestion plane. Provenance is non-negotiable.

The capture surface is the ingestion layer: every piece of content that enters the system passes through it. The design principle is a single ingestion plane rather than source-specific pipelines. Web clips, audio transcriptions, manual notes, and imported documents all resolve to the same internal schema before anything downstream processes them.

Provenance is a hard constraint enforced at this layer, not downstream. Every captured chunk carries its source URL or reference, capture timestamp, and capture method. This is not a nice-to-have for citation purposes. It is a retrieval requirement: when the orchestrator returns a result, the practitioner needs to be able to evaluate its source without leaving the system. A retrieved chunk with no provenance is a retrieved claim with no verifiability. The system treats those as the same failure.

The current capture surface handles three input types operationally:

  • Web content. Browser extension captures selected text or full page. Provenance is the URL plus capture timestamp. Automatically queued for embedding on ingest.
  • Audio and voice. Transcribed via a local pipeline before entering the ingestion plane. The transcript carries the recording timestamp and a duration field. Quality degrades with background noise in ways that affect downstream chunk coherence — this is a known limitation, not a resolved one.
  • Manual notes. The highest-friction input type and, counterintuitively, the highest-value one. Notes that were worth writing manually tend to be the ones that the distillation agent produces the most useful output from. Friction at capture correlates with signal density in the captured content.

The capture surface deliberately does not evaluate relevance. That judgment belongs to distillation. Capture is as wide and as cheap as possible. Anything that produced a response during reading or thinking enters the system. The distillation agent handles the triage.

§ IIIEmbedding + BM25 Store

Hybrid index. Neither retrieval signal alone is sufficient.

The storage layer runs two indexes in parallel: a dense vector index over embedded chunks, and a BM25 sparse keyword index over the same corpus. The architecture is hybrid by design, not as a fallback. Each index has a distinct failure mode that the other partially compensates for.

Dense retrieval (embeddings) excels at semantic similarity: finding content that is conceptually related to a query even when the vocabulary does not overlap. It fails on exact-match queries, named entities, version numbers, specific technical identifiers — anything where the surface form of the query is the signal. A query for “GPT-4 function calling schema” may return semantically similar content about tool use in general when the specific schema documentation is what is needed.

Sparse retrieval (BM25) excels at exactly these cases: named entities, specific terminology, and exact phrases retrieve with high precision. It fails when the query vocabulary and the document vocabulary diverge — which happens constantly in a knowledge base that spans multiple authors, time periods, and terminological conventions.

The chunking strategy matters more than the embedding model for most queries. Chunks are currently set at paragraph level, with a sliding window overlap that preserves sentence boundary coherence. Smaller chunks increase retrieval precision at the cost of context. Larger chunks preserve context at the cost of precision. Paragraph level is the current calibration after testing both extremes. It is not fixed — the RAG research track is actively investigating whether dynamic chunking based on semantic coherence boundaries outperforms fixed paragraph splits under the query distribution this system actually sees.

§ IVRetrieval Orchestrator

The system is only as good as what the orchestrator returns.

The retrieval orchestrator is the intelligence hub. It receives a query, decides how to retrieve against the index, fuses the results from dense and sparse signals, applies reranking, and returns attributed chunks in ranked order. Every consumer of the system — the distillation agent, the publishing surface, interactive queries during writing — calls the orchestrator. Nothing queries the indexes directly.

The orchestrator operates in three modes depending on query characteristics:

  • Dense-first. Default mode for exploratory and conceptual queries. Retrieves from the embedding store first, uses BM25 results as a reranking signal.
  • Sparse-first. Used when the query contains named entities, version identifiers, or high-specificity terms. Retrieves from BM25 first, uses embedding similarity to filter out low-coherence results.
  • Hybrid fusion. Default for production queries with unknown characteristics. Runs both indexes in parallel, fuses results with reciprocal rank fusion, applies the reranker to the merged candidate set.

The reranking layer runs a cross-encoder over the top-k candidates from the fusion step. This is the component that most directly affects the quality of what surfaces. A good retrieval pass that returns fifty marginally relevant chunks is nearly useless without a reranker that can identify the three that actually answer the query. The reranker is also the most computationally expensive step — latency is acceptable for interactive use and becomes a constraint for high-volume distillation jobs.

Source attribution is enforced at the output of the orchestrator. Every returned chunk carries its provenance metadata intact from the capture surface. This is the principle the Technical Readiness framework calls trace literacy applied to a knowledge system: you should always be able to reconstruct where a retrieved result came from, when it entered the system, and what form it was in when it did.

§ VDistillation Agent

Where models earn their keep. Raw capture into linked notes.

The distillation agent is a background worker that processes the capture inbox continuously. Its job is to convert raw captured content into notes that represent the practitioner’s thinking rather than the source’s — the same distinction the Knowledge Forge identifies as the central operation of the Distill stage. The agent handles the mechanical part of that operation. The practitioner handles the judgment.

The agent produces three outputs per processed capture:

  • A claim extraction. The central assertion of the source, stated as a standalone sentence without context or qualification. This is the unit that travels through the graph as a linked note.
  • A connection proposal. A list of existing notes in the system that the new claim is in conversation with — agreeing, contradicting, extending, or qualifying. The agent generates the proposal. The practitioner confirms, rejects, or modifies each link before it is committed.
  • A draft distillation. A short note in the practitioner’s voice synthesizing the claim and its immediate context. This is a draft that requires review, not an autonomous output. The practitioner edits it before it enters the permanent store. The agent’s drafts that require the most editing are the most instructive — they expose where the source was ambiguous or where the practitioner’s thinking is incomplete.

The distillation agent writes back to the embedding store after confirmed outputs. This is the feedback loop shown as a dashed line in the architecture diagram: distilled notes re-enter the index, which means future retrievals can surface the practitioner’s processed thinking rather than only raw captured content. Over time, the index becomes denser with synthesis and sparser with unprocessed raw material. The quality of retrieval improves as a byproduct of the distillation workflow.

§ VIPublishing Surface

The seam between private knowledge and public artifact.

The publishing surface is the output layer. It is the mechanism by which private knowledge in the system becomes public artifacts on this publication. The translation from one to the other is not automatic and is not designed to be. Private knowledge is structured for retrieval. Public artifacts are structured for a reader who does not have access to the knowledge graph it was drawn from. Those are different structures, and the publishing surface enforces the translation work required to move between them.

The operational standard at this layer is the would-publish bar: content passes from private to public only when it can stand independently of its context in the knowledge graph. A note that is clear to the practitioner because they know the five surrounding notes it is connected to is not ready to publish. A note that carries its own navigational structure — claims stated without requiring prior context, connections made explicit rather than assumed — is.

The publishing surface also closes the compounding loop that the Knowledge Forge describes: public artifacts generate responses, challenges, and questions that re-enter the capture surface as high-quality inputs. The system is not linear. Every publish event seeds the next capture cycle. The loop is operational, not theoretical.

At the Education OS level, the publishing surface doubles as the evidence mechanism for the capability ledger: a published artifact that demonstrates operational understanding of a skill is the primary evidence that the skill has moved from Familiar to Operator in the ledger. The system and the OS share this evidence standard, which is why they are complementary rather than redundant.

§ VIIOperational Notes

What running this system actually requires.

Systems documentation that omits operational reality is incomplete. The following are the current known constraints, maintenance requirements, and failure modes of SYS-01 v3.2.

Index freshness. The embedding index and the BM25 index can drift from the capture store if ingestion jobs fail silently. The current monitoring is a daily freshness check: the count of captures in the last 24 hours is compared against the count of newly indexed chunks. A delta above threshold triggers a manual review. This is a manual process and the most operationally fragile part of the system.

Reranker latency. Cross-encoder reranking over large candidate sets (>50 chunks) produces latency that is acceptable for interactive queries but becomes a bottleneck for the distillation agent when processing large capture backlogs. The current mitigation is a top-k cap of 30 chunks for distillation jobs and 20 for interactive queries. This is a calibration, not a solution.

Audio transcription quality. Voice capture remains the lowest-quality input type. Transcription errors propagate into the embedding index and occasionally surface in retrieval results in ways that are difficult to trace without examining the original audio. Provenance helps here — the timestamp allows the original recording to be retrieved — but it does not eliminate the problem.

Distillation review backlog. The agent produces draft outputs faster than they can be reviewed during high-capture periods. The backlog is a feature, not a bug — it means capture friction has not been introduced to manage the review queue. But an unreviewed distillation is an unconfirmed link, which means the knowledge graph grows in apparent density without growing in actual verified connection density. The backlog is monitored and kept below 72 hours of lag.

Production readiness is a posture, not a checklist. The four rungs of Technical Readiness apply here as directly as to any AI system: the capture pipeline has evals, the system has defined quality thresholds, traces are readable, and the practitioner is accountable for it when it fails.