Build a Local CLI That Searches Your Documents by Keyword and Meaning

## Use one runtime and give each command group a clear job Choose one supported runtime and package manager as the canonical development path. Do not mix substitute runtimes or installation commands merely because they appear compatible. Keep command responsibilities explicit: - collection commands manage indexed roots and their configuration; - context commands attach search guidance to paths; - retrieval commands return known documents; - search commands rank candidate documents; - maintenance commands inspect or refresh the index; - service commands expose the same capabilities to other processes. Preserve the installed command wrapper when it dispatches compiled application code. Build source with the documented compiler workflow; do not replace the wrapper with a single compiled executable when native extensions depend on the existing layout. ## Track each collection's settings and keep excluding separate from removing Represent each collection with a unique name, source root, file mask, inclusion state, and optional pre-update hook. Support operations to add, list, inspect, rename, include, exclude, and remove collections. Distinguish removal from exclusion: removal deletes the collection registration, while exclusion only omits it from unscoped queries. Allow file listing for an entire collection or a path prefix. Accept both plain collection paths and the tool's virtual collection URI when the interface supports it. Treat add, remove, rename, hook changes, inclusion changes, and index refreshes as state mutations. Present the exact command and expected effect before execution when user approval is required. ## Attach search notes to paths, kept apart from document text Store human-authored context separately from document content so ranking guidance can evolve without rewriting source files. Allow context at: - the current directory detected within a collection; - a specific subpath; - a virtual collection path; - the global root for guidance that applies to every collection. Provide commands to list context, detect paths that lack context, and remove obsolete context. Apply the most specific relevant context while preserving global guidance. Do not confuse context metadata with indexed document text. Make scope visible so a local note cannot silently influence unrelated collections. ## Give each document a short ID and cap how much is returned Assign each document a short identifier derived from the first 6 characters of its content hash. Show the identifier in search results and accept it with or without the leading marker. Support retrieval by path or identifier, optional starting line and line count, and batch retrieval from a glob or comma-separated list. Bound retrieval explicitly: - set a maximum line count per file; - set a maximum byte size and document the default, such as 10 KB; - skip oversized files rather than returning partial content without notice; - include line numbers by default and allow callers to disable them; - return full on-disk paths only when requested. ## Pick the search type for the query, with hybrid search as the default Expose three distinct search paths: - keyword search using full-text ranking such as BM25, without a language model; - vector similarity search using local embeddings, without reranking; - expanded hybrid query search with query expansion and reranking for the highest-quality default. Use reciprocal-rank fusion to combine independently ranked result sets. Keep the original scores and mode visible enough to diagnose unexpected ordering. Prefer the hybrid query path for general use. Choose keyword search when exact terms matter, vector search when semantic similarity is the objective, and a no-rerank option when speed matters more than final ordering quality. ## Let callers choose collections and output formats without changing the results Let callers repeat collection filters to search several named collections while excluding all others. Support a result count, all-results mode, minimum score, full-content mode, and an intent description that sharpens ranking. Keep display and data transport separate. Provide a readable command-line format plus structured JSON, CSV, Markdown, XML, and file-list outputs. Preserve legacy format flags only as aliases to one canonical format option. For batch retrieval, expose line limits, byte limits, and line-number controls independently of search ranking. Do not let a presentation option change which documents were selected. ## Build the search index and split documents into predictable chunks Use a durable local database for full-text indexing and a compatible vector extension for semantic similarity. Run embeddings, reranking, and query expansion through configured local models. Use smart document chunking with a target of 900 tokens per chunk and 15% overlap. Prefer Markdown headings as boundaries when possible. Provide an abstract-syntax-tree-aware strategy for supported code files so chunks align with functions, classes, and imports. Select it through an explicit automatic strategy. Keep regular-expression chunking as the default, and use it consistently for Markdown and unknown file types. Keep indexing, embedding generation, and query-time ranking as separate stages so failures can be diagnosed without rebuilding everything. ## Check index and model health before tuning the ranking Provide status output that shows collection and index health without changing state. Add a diagnostic command for configuration, index, model, and device problems. Benchmark search quality with versioned fixture files. Record the query, expected relevant documents, mode, ranking metrics, model configuration, and index version so comparisons remain reproducible. Use diagnostics before changing ranking weights, models, or device settings. A missing model, stale index, incompatible vector extension, or unavailable device is not evidence that the ranking algorithm itself is wrong. ## Run the tool as a local or HTTP service only when asked Offer a standard-input/output service for local process integration and an optional HTTP service for remote clients. Use a documented default port, such as 8181, while allowing an explicit override. Support foreground operation for debugging and a background-daemon mode for long-running use. Provide a matching stop command that targets only the managed daemon. Keep service transport separate from search semantics. The command-line interface and service interface should share collection, context, retrieval, ranking, and output rules. Do not start a background service implicitly. Report the endpoint, process mode, configuration source, and shutdown method when a service is started. ## Leave index changes for the user to run and keep the installed wrapper intact Never add a collection, generate embeddings, or refresh indexes automatically. Never modify the underlying search database directly. When a requested workflow needs one of these mutations, show the exact command for the user to run and explain the expected files, duration, and resource use. Keep read-only search and retrieval available without broadening permission. Do not run a compilation mode that overwrites the installed shell wrapper or breaks native vector extensions. Do not replace a wrapper that intentionally launches compiled JavaScript from a distribution directory. Treat pre-update hooks as executable configuration. Display them during inspection and run them only as part of an explicitly requested index update. ## Add regression tests and release with a changelog Keep automated tests in the designated test tree and run the documented complete suites through the supported runtime. Add narrow regression tests for collection state, retrieval limits, ranking modes, chunk boundaries, and service lifecycle. Use the canonical release workflow rather than assembling a release manually: 1. add changelog entries under the unreleased section as changes are made; 2. convert that section to a dated version during release; 3. credit external contributions using the project's required reference format; 4. aggregate release notes across the intended version series; 5. run release hooks and verification before publishing. Report the behavior changed, index or schema compatibility, tests run, benchmark impact, migration needs, and any state-changing command left for the user.