doora
Search code by shape,
not by text.

01Overview

doora is a command-line tool that parses source files into Abstract Syntax Trees and executes structural pattern queries against them. It is fundamentally different from text search: it understands the grammar of your code, not just its characters.

Every text-based search tool — grep, ripgrep, ack — treats source code as a string. They cannot tell the difference between a function named authenticate and a comment that mentions authenticate. They cannot find "all functions that take exactly two arguments" or "all unwrap() calls outside test modules."

doora answers all of these queries instantly, across millions of lines of code, using Tree-sitter's incremental parsing library at its core.

02Installation

doora is distributed as a single static binary. Install via Cargo, download a pre-built binary, or build from source.

From crates.io

# Requires Rust 1.78+
cargo install doora

Pre-built binaries

Download from the Releases page. Binaries are available for:

From source

git clone https://github.com/backpack-lab/doora
cd doora
cargo build --release
# Binary at ./target/release/doora

Shell completions

# Bash
doora --generate-completions bash >> ~/.bashrc

# Zsh
doora --generate-completions zsh >> ~/.zshrc

# Fish
doora --generate-completions fish > ~/.config/fish/completions/doora.fish

03Quick start

Run your first structural query in under 60 seconds.

Find all function definitions

Find a specific function by name

doora \
  -q '(function_item name: (identifier) @fn (#eq? @fn "connect"))' \
  -p ./src

Search multiple languages at once

doora \
  -q '(function_declaration name: (identifier) @fn_name)' \
  -p ./src
# auto-detects language per file

Multiple queries in one pass

doora \
  -q '(function_item name: (identifier) @fn_name)' \
  -q '(struct_item name: (type_identifier) @struct_name)' \
  -p ./src --no-color

Launch the interactive TUI

doora -q '(function_item)' -p . --tui

Build a search index for faster queries

doora index ./src --verbose
# Subsequent searches use Bloom filter pre-rejection

04Why structural search

Text search has fundamental limitations that no amount of regex cleverness can overcome. Structural search solves them categorically.

05How AST search works

Source code has grammar. Every language defines how tokens nest into expressions, expressions into statements, and statements into functions and modules. doora makes this grammar queryable.

From bytes to tree

Consider this Rust function:

fn authenticate(user: &str, password: &str) -> bool {
    true
}

Tree-sitter parses this into a Concrete Syntax Tree (CST):

An S-expression query navigates this tree structure. The query (function_item name: (identifier) @fn) matches any function_item node that has a name child of type identifier, and captures that identifier as @fn.

CST vs AST

Traditional compilers produce Abstract Syntax Trees (ASTs) that discard whitespace, comments, and punctuation. doora uses Concrete Syntax Trees (CSTs) that retain 100% fidelity to the original source — including exact byte offsets for every token. This is required for:

• Accurately mapping matches back to line and column numbers for display
• Semantic rewriting that preserves the user's original formatting
• Syntax highlighting driven by the tree itself rather than regex

06Query syntax overview

Queries use Tree-sitter's S-expression pattern syntax — a Lisp-like notation that mirrors the shape of the syntax tree.

Basic node match

(node_type)

Matches any node of the given type anywhere in the tree. Node type names are grammar-specific — see the language sections for common types.

Named capture

(function_item name: (identifier) @my_capture)

The @my_capture syntax tags a sub-node for extraction. Captured text appears in the output. Multiple captures per query are supported.

Equality predicate

(function_item
  name: (identifier) @fn
  (#eq? @fn "connect"))

Matches only when the captured node's text equals "connect" exactly.

Regex predicate

(function_item
  name: (identifier) @fn
  (#match? @fn "^(get|set|update)_"))

Matches function names starting with get_, set_, or update_.

Wildcard child

(call_expression
  function: (identifier) @callee
  arguments: (arguments . (_) .))

The (_) wildcard matches any single node. Use . as an anchor.

Multi-query (single pass)

doora \
  -q '(function_item name: (identifier) @fn_name)' \
  -q '(struct_item name: (type_identifier) @struct_name)' \
  -p ./src

Multiple -q flags are compiled into a single-pass automaton. The AST is traversed exactly once regardless of query count.

07CLI — search command

The default command. When no subcommand is given, doora runs a structural search.

Arguments

Output format

Fields: filepath:line:col   [@capture_name]   "matched_text"

• Filepath is colorized cyan (unless --no-color)
• Capture name is colorized yellow
• Matched text is colorized green, always wrapped in literal quotes
• Line numbers are 1-indexed; columns are 0-indexed byte offsets

Stats output

08CLI — index command

Explicitly builds or updates the Bloom filter index for a directory. Once indexed, search queries with string literals run significantly faster.

The index is stored at <PATH>/.doora-index as a binary file serialized with bincode. It is a manifest of per-file Bloom filter entries, each containing:

• Absolute file path
• Last-modified timestamp (seconds since Unix epoch)
• File size in bytes
• 512-byte Bloom filter (4096 bits, two FNV-1a hash functions)
• Detected language string

09Query guide — basics

A complete reference for writing effective S-expression queries.

Node types

Every language has a set of named node types defined by its Tree-sitter grammar. The node type is the first element of an S-expression. Common node types vary by language:

Field names

Named fields constrain which child to match. The syntax is field_name: (child_type):

(function_item
  name: (identifier) @fn
  parameters: (parameters) @params
  return_type: (_) @ret)

Nested patterns

Patterns can nest to arbitrary depth:

(impl_item
  type: (type_identifier) @impl_type
  body: (declaration_list
    (function_item name: (identifier) @method_name)))

10Query guide — predicates

Predicates filter captures based on their text content. They are evaluated using pre-compiled regex objects, cached at query compile time.

11Query examples by language

A practical library of common patterns for each supported language.

Rust

(function_item name: (identifier) @fn_name)

(call_expression
  function: (field_expression
    field: (field_identifier) @m (#eq? @m "unwrap")))

(struct_item name: (type_identifier) @struct_name)

(impl_item
  trait: (type_identifier) @trait
  type: (type_identifier) @impl_type
  (#eq? @impl_type "MyStruct"))

(function_item
  name: (identifier) @fn
  (#match? @fn "^(get|set|update)_"))

Python

(function_definition name: (identifier) @fn_name)

(decorated_definition
  decorator: (decorator) @decorator
  definition: (function_definition name: (identifier) @fn_name))

(class_definition name: (identifier) @class_name)

JavaScript

(function_declaration name: (identifier) @fn_name)

(class_declaration name: (identifier) @class_name)

(method_definition name: (property_identifier) @method_name)

TypeScript

(interface_declaration name: (type_identifier) @interface_name)

(type_alias_declaration name: (type_identifier) @type_name)

Go

(function_declaration name: (identifier) @fn_name)

(type_declaration
  (type_spec name: (type_identifier) @type_name))

C / C++

(function_definition
  declarator: (function_declarator
    declarator: (identifier) @fn_name))

(class_specifier name: (type_identifier) @class_name)

(type_definition declarator: (type_identifier) @type_name)

12Language auto-detection

When --lang auto is used (the default), doora detects the grammar per file from its extension and walks all supported extensions simultaneously.

In auto mode, the query is compiled against every supported language at startup. Languages for which the query fails to compile (because a node type doesn't exist in that grammar) are silently skipped. Only languages where the query compiles successfully are searched. This means a Rust-only query like (function_item @fn) will search only .rs files even in auto mode.

13Bloom filter index

The index is a pre-parse rejection sieve. Files that mathematically cannot contain the search term are skipped entirely, before tree-sitter is invoked.

How it works

Bloom filter parameters

14Semantic rewriting

doora can surgically replace structural patterns without corrupting surrounding syntax. This is fundamentally different from sed — it operates on the AST, not on text.

A unified diff is always printed before --in-place writes, unless --yes is passed. Rewrites are applied in reverse byte order to avoid offset invalidation when multiple matches occur in the same file.

15Interactive TUI

Launch with --tui for a split-pane terminal explorer with live streaming results.

The TUI uses a fully decoupled async event loop (Tokio). Results stream in live as background threads find matches — the terminal never freezes, even on multi-gigabyte repositories.

16MCP server mode

doora exposes a Model Context Protocol server so LLM coding agents can query your codebase structurally, reducing hallucinations and wasted context tokens.

Why LLMs need structural search

LLM coding agents have limited context windows. When an agent tries to understand a large codebase by reading files, it burns context tokens on irrelevant content and still has no structural understanding of the architecture.

With the doora MCP server, an agent can ask: "What is the exact type signature of the function handling user authentication?" and receive a precise, structured answer in milliseconds — consuming a fraction of the context tokens that reading source files would require.

Setup

{
  "mcpServers": {
    "doora": {
      "command": "doora",
      "args": ["serve", "--mcp"],
      "cwd": "/path/to/your/repo"
    }
  }
}

Available MCP tools

17Architecture — pipeline overview

The search pipeline is a linear sequence from CLI arguments to printed results. Each stage is isolated and independently testable.

Key architectural decisions

• Query compiled before walk: Arc<MultiCompiledQuery> is built once and shared across all Rayon threads with zero per-thread recompilation.
• Ephemeral tree lifecycle: The Tree and source bytes are dropped immediately after extract_matches returns. RAM usage stays flat regardless of repository size.
• Single-pass multi-query: Multiple -q flags are merged into one traversal. The AST is walked once per file regardless of query count.
• BitSet pre-filtering: Each node's kind ID is checked against a pre-computed HashSet<u16> before evaluating the full match pattern.
• Bloom filter sieve: Files without the query's required trigrams are skipped entirely before tree-sitter is invoked.

18Architecture — module reference

The codebase is organized into focused, independently testable modules.

19Architecture — concurrency model

The search pipeline is fully parallel using Rayon's work-stealing thread pool. Every design decision was made to maximize throughput while eliminating contention.

Work-stealing thread pool

The directory walker's Iterator is converted to a Rayon parallel iterator via par_bridge(). Rayon distributes file processing across all available cores using a work-stealing scheduler — when a thread exhausts its local queue, it steals work from the end of another thread's queue.

Thread-local parser pool

Tree-sitter Parser initialization is expensive — it allocates C-level state via FFI. Creating and dropping a parser per file would be catastrophic. Instead, each Rayon worker thread owns exactly one Parser via thread_local!:

thread_local! {
    static PARSER: RefCell<Parser> = RefCell::new(Parser::new());
}

The parser is reconfigured for each file via set_language() before parsing — a cheap operation that resets internal state without a new allocation.

Shared state

The following objects are shared across all threads:

• Arc<HashMap<Language, Arc<MultiCompiledQuery>>> — compiled queries, shared by reference
• Arc<Mutex<Vec<MatchResult>>> — result accumulator, locked only for append
• Arc<Mutex<usize>> — file counter
• Arc<IndexManifest> — Bloom filter index, read-only during search

20Architecture — memory model

doora maintains a flat RAM profile regardless of repository size by enforcing an ephemeral tree lifecycle.

Ephemeral lifecycle

When parsing a file with text search, only a small buffer needs to be in memory. With AST-based search, parsing a file produces a large C-allocated structure with potentially tens of thousands of nodes. If all trees were retained simultaneously, a 10,000-file repository would require tens of gigabytes of RAM.

The solution is the ephemeral lifecycle. In the parallel closure, the drop order is explicit and mandatory:

let (tree, source) = parse_file(entry.path(), &lang)?;
let matches = extract_multi_matches(&tree, source.as_bytes(), &multi, path);
drop(tree);    // C-allocated CST freed immediately
drop(source);  // source bytes freed immediately
// Only the small Vec<MatchResult> persists

Memory-mapped files

For files above 1MB, parse_file uses memmap2 instead of fs::read_to_string. The OS pages in only the bytes actually accessed, backed by the file's page cache rather than a heap allocation:

// Small files: heap allocation
FileSource::Heap(String)

// Large files (≥ 1MB): memory-mapped
FileSource::Mapped(memmap2::Mmap)

Maximum RAM usage

RAM usage is bounded by:

• Number of active Rayon worker threads × (one tree + one source buffer)
• The final accumulated Vec<MatchResult> (small owned strings only)
• The pre-computed CompiledQuery objects (one per language)
• The IndexManifest if the index is loaded (512 bytes per file entry)

21Performance

doora is engineered for sub-second query latency on repositories with millions of lines of code.

Benchmark results

Optimization stack

The following optimizations are layered from outermost to innermost:

• Bloom filter sieve: Skip parsing files that cannot contain the search term. Eliminates tree-sitter invocation entirely for filtered files.
• BitSet kind filtering: Pre-compute which node kind IDs could match the query. Skip match_node evaluation for the vast majority of tree nodes.
• Pre-compiled regex predicates: Compile #match? regex patterns exactly once per query, share via Arc<Regex>. Zero per-file regex compilation.
• Single-pass multi-query automaton: Traverse the AST exactly once per file regardless of how many -q flags were passed.
• Thread-local parser pool: Each Rayon thread owns one Parser for its entire lifetime. Zero per-file parser allocation.
• Memory-mapped file reading: Files ≥ 1MB use mmap to avoid heap allocation of the source string.
• Work-stealing parallelism: Rayon's lock-free scheduler saturates all CPU cores with no idle threads.

Profiling

To profile on your hardware:

# Install flamegraph
cargo install flamegraph

# Run profiler against the Rust compiler source
make profile

# Or manually
cargo flamegraph --bin doora -- \
  -q '(function_item name: (identifier) @fn_name)' \
  -p /path/to/large/repo --lang rust --no-color --quiet

See PROFILING.md for detailed instructions and interpretation guidance.

22Building from source

Requires Rust 1.78 or later (MSRV). All grammar crates are compiled from source as part of the Cargo build.

# Debug build
cargo build

# Release build (recommended for performance testing)
cargo build --release

# Check only (fast compilation check)
cargo check --all-features

Dependencies

23Testing

The test suite covers unit tests, integration tests, and statistical Bloom filter correctness tests.

# Run all tests
cargo test

# Run tests for a specific module
cargo test --lib parser
cargo test --lib query
cargo test --lib walker
cargo test --lib bloom

# Run integration tests
cargo test --test integration_test -- --nocapture

# Run Bloom filter statistical tests
cargo test --test bloom_statistics -- --nocapture

# Run binary tests (CLI validation, stats)
cargo test --bin doora

Test inventory

Fixture files

Integration tests use fixture files in tests/fixtures/:

• simple.rs — Rust functions and types
• multi_fn.rs — Multiple functions for ordering tests
• structs.rs — Struct definitions
• nested.rs — Nested module functions
• empty.rs — Empty file (parse error path)
• simple.py — Python functions
• simple.js — JavaScript functions and classes
• simple.ts — TypeScript interfaces and types
• simple.go — Go functions and structs
• simple.c — C functions and typedefs
• simple.cpp — C++ classes and templates

24Benchmarks

Criterion benchmarks in benches/search_benchmarks.rs. Run automatically in CI with regression detection.

# Run all benchmarks
cargo bench --bench search_benchmarks

# Run as tests (fast, no timing)
cargo bench --bench search_benchmarks -- --test

# Save baseline for comparison
cargo bench -- --save-baseline main

# Compare against baseline
cargo bench -- --baseline main

Benchmark suite

CI fails on regressions exceeding 10% for parallel benchmarks and 2% for single-file benchmarks. Results are uploaded as GitHub Actions artifacts with HTML reports from Criterion.

25Contributing

Contributions are welcome. The project is tracked issue-by-issue with milestones for each subsystem.

Before opening a PR

# Format
cargo fmt

# Lint — zero warnings enforced
cargo clippy --all-features -- -D warnings

# Full test suite
cargo test --all-features

All three must pass cleanly. The CI pipeline enforces #![deny(warnings)] and #![warn(clippy::pedantic)] across the entire codebase.

Code style

• No inline comments, no doc comments, no block comments anywhere in source files
• All public items must have doc comments (enforced by cargo doc --no-deps zero warnings)
• Use Result<T> (the crate's type alias for Result<T, AppError>) for all fallible functions
• Never use unwrap() in production code — only in tests with an explanatory comment
• Explicit drop() calls are mandatory after tree and source are no longer needed

Adding a new language

1. Add tree-sitter-<lang> to Cargo.toml
2. Add a variant to Language enum in types.rs
3. Add extension mapping in walker.rs::extensions_for_language
4. Add grammar arm in parser.rs::get_language
5. Add detection arm in parser.rs::detect_language
6. Add variant in parser.rs::get_all_languages
7. Add lang string to main.rs::resolve_lang and validate
8. Add fixture file to tests/fixtures/
9. Add integration tests following the pattern of existing language tests

License

MIT