RagPipelineOptions
Properties
Section titled “Properties”batchSize?
Section titled “batchSize?”optional batchSize?: number;Batch size for embedMany calls. Default: 100.
chunker
Section titled “chunker”chunker: Chunker;Chunker for splitting documents into chunks.
embedder
Section titled “embedder”embedder: Embedder;Embedder for converting text to vectors.
indexName
Section titled “indexName”indexName: string;Index name for this pipeline’s data.
keywordIndex?
Section titled “keywordIndex?”optional keywordIndex?: KeywordIndex;Keyword index kept in sync with ingestion (the keyword tier for
FusionRetriever/HybridRetriever). Chunk ids match the vector
entry ids (${docId}:${chunkId}).
manifest?
Section titled “manifest?”optional manifest?: IngestManifest;Persistent ingest manifest. When provided, the pipeline:
- locks the index to the embedder that built it (Embedder.id
- dimension) and throws on mismatch at ingest AND retrieve time — a different same-dimension model silently corrupts relevance;
- skips unchanged documents on re-ingest (SHA-256 content hash);
- deletes stale chunks of changed documents (when the store supports admin deletes). Without a manifest, every ingest re-embeds everything and no embedder lock is enforced (previous behavior).
metric?
Section titled “metric?”optional metric?: "cosine" | "euclidean" | "dotproduct";Distance metric for index creation. Default: ‘cosine’.
reranker?
Section titled “reranker?”optional reranker?: Reranker;Optional reranker for post-retrieval refinement.
optional topK?: number;Default topK for retrieval. Default: 10.
vectorStore
Section titled “vectorStore”vectorStore: VectorStoreCore;Vector store for persistent storage and similarity search.
ensureIndex() additionally requires VectorStoreIndexAdmin to
provision indexes at runtime; passing a Core-only adapter (edge
stores such as Cloudflare Vectorize or Upstash Vector whose indexes
are provisioned out-of-band) is supported — ensureIndex() becomes
a no-op on those stores (REQ-17).