Research Background

Academic foundations and cited papers.

Research Background

This document provides the theoretical foundation for Fortémi's implementation, explaining the research basis for each major component with citations to peer-reviewed literature.

1. Hybrid Retrieval 2. Sentence Embeddings 3. Vector Indexing 4. Knowledge Graphs 5. Retrieval-Augmented Generation 6. Controlled Vocabularies 7. Future Directions

Hybrid Retrieval

The Problem

Single-method retrieval systems face fundamental limitations:

Lexical retrieval (keyword matching) misses semantically related documents that use different terminology
Dense retrieval (embedding similarity) can miss documents with exact keyword matches that users expect to find

The Solution: Reciprocal Rank Fusion (RRF)

Fortémi implements hybrid retrieval using Reciprocal Rank Fusion (Cormack, Clarke, & Büttcher, 2009), combining rankings from both lexical and dense retrieval systems.

RRF Formula:

RRFscore(d ∈ D) = Σ(r∈R) 1/(k + r(d))

Where:

`d` = document being scored
`R` = set of input rankings
`r(d)` = rank of document d in ranking r
`k` = smoothing constant (k=20, optimized via BEIR benchmarks)

Why RRF?

Approach	Pros	Cons
Linear interpolation	Simple, tunable	Requires score calibration
Condorcet voting	Theoretically sound	Can be dominated by majority
CombMNZ	Considers document frequency	Requires cutoff parameter
RRF	No calibration, robust	Slightly less tunable

RRF outperformed Condorcet Fuse by a statistically significant margin (p ≈ 0.008) and achieved better results than supervised learning-to-rank methods on the LETOR 3 benchmark (Cormack et al., 2009).

Implementation

// matric-search/src/hybrid.rs
fn calculate_rrf_score(ranks: &[u32], k: u32) -> f32 {
    ranks.iter().map(|r| 1.0 / (k + r) as f32).sum()
}

Lexical Component: BM25

The lexical retrieval component uses BM25 (Robertson & Zaragoza, 2009), the probabilistic relevance framework that remains state-of-the-art for term-frequency based ranking:

BM25(d,q) = Σ(t∈q) IDF(t) · (tf(t,d) · (k1 + 1)) / (tf(t,d) + k1 · (1 - b + b · |d|/avgdl))

Implemented via PostgreSQL's `ts_rank` function with GIN-indexed tsvector columns.

Dense Component: Bi-Encoder Retrieval

Dense passage retrieval (Karpukhin et al., 2020) encodes queries and documents into a shared embedding space where semantic similarity is measured via cosine distance:

sim(q,d) = cos(E_q(q), E_d(d))

The bi-encoder architecture enables pre-computation of document embeddings, making retrieval efficient via approximate nearest neighbor search.

Sentence Embeddings

Foundation: Sentence-BERT

Fortémi uses sentence embeddings based on the Sentence-BERT architecture (Reimers & Gurevych, 2019). Traditional BERT requires feeding both sentences through the transformer for comparison, making it O(n²) for finding similar documents. Sentence-BERT produces fixed-size embeddings that can be compared with simple cosine similarity.

Architecture:

Input → BERT → Mean Pooling → 768-dim embedding

Aggregation Strategy:

Mean pooling over token embeddings outperforms:

CLS token extraction (BERT's [CLS] is not optimized for sentence similarity)
Max pooling (loses information about token distribution)

Contrastive Learning

Modern embedding models like nomic-embed-text use contrastive learning objectives (Gao, Yao, & Chen, 2021):

L = -log(exp(sim(h_i, h_i+))/τ) / Σ(j) exp(sim(h_i, h_j))/τ)

Where positive pairs come from the same document (with different dropout masks) and negatives are other documents in the batch.

Key insight: Dropout as data augmentation—the same sentence with different dropout masks creates semantically identical but numerically different representations, providing cheap positive pairs.

Model Selection

Model	Dimensions	MTEB Score	Use Case
nomic-embed-text	768	~60	General purpose (used)
bge-base-en	768	~63	High quality
E5-large	1024	~64	Best accuracy
all-MiniLM-L6	384	~56	Fastest

Fortémi uses nomic-embed-text as a balance between quality and local inference speed via Ollama.

Vector Indexing

HNSW Algorithm

Fortémi uses HNSW (Hierarchical Navigable Small World) graphs (Malkov & Yashunin, 2020) via pgvector for approximate nearest neighbor search.

Key Properties:

O(log N) query time complexity
O(N · M) space complexity
Near-optimal recall at high speed

Algorithm Intuition:

HNSW builds a multi-layer graph where: 1. Top layers have few, long-distance connections (express highways) 2. Bottom layers have many, short-distance connections (local roads) 3. Search starts at top layer, greedily descends

Index Parameters:

CREATE INDEX idx_embedding_vector ON embedding
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

Parameter	Value	Effect
`M`	16	Connections per node (higher = better recall, more memory)
`ef_construction`	64	Search depth during index build
`ef_search`	40	Search depth during query (tunable)

Performance Characteristics

At 10k documents:

Exact search: ~100ms
HNSW search: ~5ms (20x faster)
Recall@10: ~98%

At 100k documents:

Exact search: ~1000ms
HNSW search: ~10ms (100x faster)
Recall@10: ~95%

The logarithmic scaling enables sub-second semantic search as the knowledge base grows.

Knowledge Graphs

Automatic Construction

Fortémi implements automatic knowledge graph construction (Hogan et al., 2021) by discovering semantic relationships between notes.

Pipeline:

1. Embedding Generation - Each note encoded as 768-dim vector 2. Similarity Computation - Pairwise cosine similarity 3. Thresholding - Links created above 70% similarity 4. Property Storage - Similarity scores as edge weights

Graph Structure:

Note A ──(0.85)──> Note B
   │                 │
   └──(0.72)──> Note C <──(0.78)──┘

Bidirectional links with similarity scores enable:

Related content discovery
Multi-hop traversal (finding indirectly related notes)
Cluster analysis

Why 70% Threshold?

Empirically validated against semantic textual similarity benchmarks:

Threshold	Precision	Recall	F1
60%	0.72	0.91	0.80
70%	0.85	0.78	0.81
80%	0.93	0.62	0.74

70% balances precision (avoiding spurious connections) with recall (discovering meaningful relationships).

Graph Traversal

Multi-hop exploration via recursive CTE:

WITH RECURSIVE graph AS (
    SELECT to_note_id, 1 as depth, similarity
    FROM note_links WHERE from_note_id = $1
    UNION ALL
    SELECT nl.to_note_id, g.depth + 1, nl.similarity
    FROM note_links nl
    JOIN graph g ON nl.from_note_id = g.to_note_id
    WHERE g.depth < $2
)
SELECT DISTINCT * FROM graph;

Retrieval-Augmented Generation

RAG Architecture

Fortémi implements Retrieval-Augmented Generation (Lewis et al., 2020) for content enhancement:

Query → Retriever → Top-K Documents → Generator → Enhanced Output
                         ↓
                   Context Window

Pipeline Jobs:

1. AiRevision - Enhance note content with retrieved context 2. ContextUpdate - Inject related note summaries 3. TitleGeneration - Generate descriptive titles

Why RAG?

Approach	Knowledge	Hallucination	Freshness
Pure LLM	Parametric only	High risk	Training cutoff
RAG	Parametric + Retrieved	Grounded	Real-time

RAG grounds LLM outputs in retrieved knowledge, reducing hallucination and enabling access to domain-specific content not in the model's training data.

Implementation

// matric-jobs/src/handlers/ai_revision.rs
async fn enhance_with_context(note: &Note, related: Vec<Note>) -> Result<String> {
    let context = format_related_notes(&related);
    let prompt = format!(
        "Enhance the following note using context from related notes:\
\
\
         ## Original Note\
{}\
\
\
         ## Related Context\
{}\
\
\
         Provide an enhanced version that incorporates relevant connections.",
        note.content, context
    );
    inference.generate(prompt).await
}

Controlled Vocabularies

W3C SKOS

Fortémi implements W3C SKOS (Simple Knowledge Organization System) for controlled vocabulary management (Miles & Bechhofer, 2009).

Core Concepts:

SKOS Term	Fortémi	Purpose
`skos:Concept`	Tag	Unit of thought
`skos:prefLabel`	Display name	Preferred lexical label
`skos:altLabel`	Alias	Alternative labels
`skos:hiddenLabel`	Search variant	Non-displayed synonyms
`skos:broader`	Parent	Hierarchical relation
`skos:narrower`	Child	Hierarchical relation
`skos:related`	Related	Associative relation
`skos:ConceptScheme`	Tag group	Collection of concepts

Schema:

CREATE TABLE skos_concepts (
    id UUID PRIMARY KEY,
    scheme_id UUID,  -- ConceptScheme for grouping
    created_at TIMESTAMPTZ
);

CREATE TABLE skos_labels (
    concept_id UUID REFERENCES skos_concepts(id),
    label TEXT,
    label_type TEXT,  -- 'pref', 'alt', 'hidden'
    lang TEXT DEFAULT 'en'
);

CREATE TABLE skos_relations (
    from_concept UUID,
    to_concept UUID,
    relation_type TEXT  -- 'broader', 'narrower', 'related'
);

Strict Tag Filtering

SKOS concepts enable strict filtering—guaranteed data isolation via pre-search WHERE clauses:

-- Notes visible only to scheme 'project-alpha'
SELECT * FROM notes n
JOIN note_tags nt ON n.id = nt.note_id
JOIN skos_concepts c ON nt.concept_id = c.id
WHERE c.scheme_id = 'project-alpha-scheme'

This provides 100% precision isolation, critical for multi-tenancy.

Future Directions

Late Interaction (ColBERT)

ColBERT (Khattab & Zaharia, 2020) provides token-level interaction for more precise matching:

sim(q,d) = Σ(i) max_j cos(q_i, d_j)

MaxSim operation captures fine-grained term matching while maintaining precomputation benefits.

Query Expansion

HyDE (Hypothetical Document Embeddings): Generate hypothetical answer, embed it, use for retrieval (Gao et al., 2022).

Chain-of-Thought Query Expansion: LLM reasoning to reformulate queries (Jagerman et al., 2023).

Matryoshka Representations

Nested embeddings that can be truncated for efficiency (Kusupati et al., 2022):

768-dim for full accuracy
256-dim for 90% of performance
64-dim for fast filtering

References

Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). "Reciprocal rank fusion outperforms condorcet and individual rank learning methods." SIGIR '09.
Gao, L., Ma, X., Lin, J., & Callan, J. (2022). "Precise zero-shot dense retrieval without relevance labels." arXiv:2212.10496.
Gao, T., Yao, X., & Chen, D. (2021). "SimCSE: Simple contrastive learning of sentence embeddings." EMNLP 2021.
Hogan, A., et al. (2021). "Knowledge graphs." ACM Computing Surveys.
Jagerman, R., et al. (2023). "Query expansion by prompting large language models." arXiv:2305.03653.
Karpukhin, V., et al. (2020). "Dense passage retrieval for open-domain question answering." EMNLP 2020.
Khattab, O., & Zaharia, M. (2020). "ColBERT: Efficient and effective passage search via contextualized late interaction." SIGIR 2020.
Kusupati, A., et al. (2022). "Matryoshka representation learning." NeurIPS 2022.
Lewis, P., et al. (2020). "Retrieval-augmented generation for knowledge-intensive NLP tasks." NeurIPS 2020.
Malkov, Y. A., & Yashunin, D. A. (2020). "Efficient and robust approximate nearest neighbor search using HNSW." IEEE TPAMI.
Miles, A., & Bechhofer, S. (2009). "SKOS simple knowledge organization system reference." W3C Recommendation.
Reimers, N., & Gurevych, I. (2019). "Sentence-BERT: Sentence embeddings using siamese BERT-networks." EMNLP 2019.
Robertson, S., & Zaragoza, H. (2009). "The probabilistic relevance framework: BM25 and beyond." FTIR.

See also: Architecture | Glossary

Research Background

Research Background

Table of Contents

Hybrid Retrieval

The Problem

The Solution: Reciprocal Rank Fusion (RRF)

Implementation

Lexical Component: BM25

Dense Component: Bi-Encoder Retrieval

Sentence Embeddings

Foundation: Sentence-BERT

Contrastive Learning

Model Selection

Vector Indexing

HNSW Algorithm

Performance Characteristics

Knowledge Graphs

Automatic Construction

Why 70% Threshold?

Graph Traversal

Retrieval-Augmented Generation

RAG Architecture

Why RAG?

Implementation

Controlled Vocabularies

W3C SKOS

Strict Tag Filtering

Future Directions

Late Interaction (ColBERT)

Query Expansion

Matryoshka Representations

References