M2.3: find in files backend
crates/search: parallel ignore walk + grep-searcher with the project's
ignore rules (vendor searched as library, node_modules and .git skipped),
literal or regex, case, whole word, include/exclude globs, binary and
> 8 MiB skips, match cap, cancel token, buffer overlay.
Engine: search.find returns {searchId} at once and runs off the lock;
pump_vfs drains results into searchResults (one per file, at most 500
per pump) and searchDone; search.cancel. Testkit: generated search tree.
Tests: library unit and property tests, engine tests, TCP control test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
a6b2fd1300
commit
172693c8dd
21 changed files with 2231 additions and 5 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Architecture
|
||||
|
||||
> **Status:** M0 decisions recorded, M1 workspace, desktop bridge, tabs and editor, message catalog, engine construction, settings, themes, the recovery journal, notifications, UI commands, the palette, the settings dialog and the app icon documented · **Last updated:** 2026-10-16
|
||||
> **Status:** M0 decisions recorded, M1 workspace, desktop bridge, tabs and editor, message catalog, engine construction, settings, themes, the recovery journal, notifications, UI commands, the palette, the settings dialog, the app icon and find in files (backend) documented · **Last updated:** 2026-10-16
|
||||
|
||||
## Overview
|
||||
|
||||
|
|
@ -79,7 +79,7 @@ Theme tokens are defined in Rust (`lutra_engine::theme`) and rendered as CSS cus
|
|||
|
||||
### Events
|
||||
|
||||
`lutra_engine::Event`, serialised as `{"kind": "<camelCase>", ...}`: `projectOpened {root, name}`, `projectClosed`, `treeChanged {dir}`, `bufferOpened {id, path}`, `bufferFocused {id}` (M1.13), `bufferChanged {id, version}`, `dirtyChanged {id, dirty}`, `bufferClosed {id}`, `fileChangedOnDisk {path, id, removed}`, `watchFailed {message}`, `recoveryAvailable {entries}` (M1.7, see Recovery journal), `uiCommand {id, params}` (M1.8, see UI commands), `recoveryResolved {paths}` (M1.15, see Recovery journal). Events say what changed; clients fetch data with `project.children` and `buffer.get`. `file.open` on a file whose buffer is already open returns that buffer and emits `bufferFocused {id}` instead of `bufferOpened` (M1.13): a separate event, so that `bufferOpened` keeps meaning "a new buffer exists" (the host's `engine_ready` replay and other clients rely on that) while every client, including the editor, still learns that the user asked to see that buffer and activates its tab.
|
||||
`lutra_engine::Event`, serialised as `{"kind": "<camelCase>", ...}`: `projectOpened {root, name}`, `projectClosed`, `treeChanged {dir}`, `bufferOpened {id, path}`, `bufferFocused {id}` (M1.13), `bufferChanged {id, version}`, `dirtyChanged {id, dirty}`, `bufferClosed {id}`, `fileChangedOnDisk {path, id, removed}`, `watchFailed {message}`, `recoveryAvailable {entries}` (M1.7, see Recovery journal), `uiCommand {id, params}` (M1.8, see UI commands), `recoveryResolved {paths}` (M1.15, see Recovery journal), `searchResults {searchId, path, matches}` and `searchDone {searchId, files, matches, truncated, cancelled, elapsedMs}` (M2.3, see Search). Events say what changed; clients fetch data with `project.children` and `buffer.get`. `file.open` on a file whose buffer is already open returns that buffer and emits `bufferFocused {id}` instead of `bufferOpened` (M1.13): a separate event, so that `bufferOpened` keeps meaning "a new buffer exists" (the host's `engine_ready` replay and other clients rely on that) while every client, including the editor, still learns that the user asked to see that buffer and activates its tab.
|
||||
|
||||
Changes on disk reach the engine through `Engine::pump_vfs()`, which hosts call regularly (the desktop's pump thread every 100 ms; tests call it directly). It maps VFS events to `treeChanged` for the parent directories, reloads a clean buffer whose file changed (unless the file already matches, as after our own save) with `bufferChanged`, and reports `fileChangedOnDisk` for a dirty buffer, which keeps its text, or for a removed file. At most 10,000 VFS events are handled per pump.
|
||||
|
||||
|
|
@ -195,6 +195,17 @@ Watched directories: the VFS watcher is non-recursive, so `project.children` wat
|
|||
- **Keyboard:** Escape or a click outside closes; Tab and Shift+Tab wrap inside the dialog, focus that leaves it is pulled back to the search field, and focus returns to where it was on close.
|
||||
- No OK/Apply/Cancel: every change applies at once, as agents change settings through the same commands.
|
||||
|
||||
### Search (M2.3)
|
||||
|
||||
- `crates/search` (L2) is the library: `Search::new(&Query)` compiles a literal or regex (`grep-regex`; empty, invalid, oversized or newline-matching patterns and bad globs fail up front), `run` walks the project with the `ignore` crate's parallel walker (at most 8 threads) and searches each file with `grep-searcher`; `spawn` runs it on a thread of its own and delivers `SearchEvent::File` per file with matches, then one `SearchEvent::Done`. A `CancelToken` stops it between files and lines.
|
||||
- Which files: the same rules as `lutra-vfs` (`.gitignore`, `.ignore`, `.git/info/exclude`, global excludes, applied without a git repository; ignore files above the root are not read). `.git` and `node_modules` are skipped, the root's `vendor/` is searched as library code even when an ignore file lists it (it is walked as a second root, so its own nested ignore files still apply), dot files are searched, symbolic links are not followed. Include globs keep matching files, exclude globs drop files and whole directories; a glob without `/` matches the name, one with `/` the project-relative path. Files over 8 MiB, binary files (a NUL byte) and unreadable files are skipped silently. The library does not depend on `lutra-vfs`: the walker is configured to the same rules instead, because the VFS reads rules per directory listing and the walker needs them per walk.
|
||||
- Results: positions are 0-based, columns and lengths in UTF-16 code units, lines end at `
|
||||
` (a preceding `
|
||||
` is not part of the line; a lone `
|
||||
` is not a line break here, unlike in `lutra-text`). Invalid UTF-8 is replaced before matching. The preview is the line, or a window of 300 characters starting up to 80 characters before the match. Files arrive in no fixed order. The cap (`maxResults`, default 10,000, at most 100,000) counts matches; `truncated` is set only when a match was left out.
|
||||
- Engine: `search.find {query, regex?, caseSensitive?, wholeWord?, include?, exclude?, maxResults?}` (disabled without a project) snapshots the text of every dirty buffer as an overlay (searched instead of the file; only for files the walk visits), starts the search and returns `{searchId}` at once. Searches run outside the engine lock. Each search's results wait in its own channel in `State::searches`; `Engine::pump_vfs` drains them (`cmd::search::pump`) into `searchResults` events (one per file) and a final `searchDone`. Hosts already publish the pump's events, so the desktop (pump every 100 ms) and the control channel get them without new plumbing; there are at most 10 batches per second and at most 500 `searchResults` per pump (`MAX_RESULTS_PER_PUMP`), which keeps a control subscriber's queue of 4096 events from overflowing; the rest wait for the next pump. `search.cancel {searchId}` returns `{cancelled}` (false once the search ended) and the search still ends with `searchDone {cancelled: true}`. Starting a search does not cancel older ones. A search whose project is no longer open is cancelled by the next pump, its remaining results are dropped and it ends with `cancelled: true`.
|
||||
- Measured (release, Windows 11, 16 threads, generated 20,006-file tree from `lutra_testkit::generate_search_tree`, other builds running): common literal `function` first result 10–43 ms, capped at 10,000 matches after 112–209 ms; a rare literal (full walk, 376 files) first result 17–22 ms, total 0.66–1.43 s (the first round cold); `function` with the 100,000 cap 0.53 s. Through the engine add up to one pump interval (100 ms) before results reach clients. Live check over the control channel against the running app (debug build, 5,900-file project with a dirty buffer): `search.find` answered in 4.5 ms, first `searchResults` line after 55 ms, `searchDone` after 386 ms; vendor hits present, `var/`, `node_modules` and the binary file absent; the dirty buffer's text was found; `search.cancel` ended a search with `cancelled: true`.
|
||||
|
||||
## Decisions (M0)
|
||||
|
||||
| Question | Decision | Why |
|
||||
|
|
@ -212,6 +223,7 @@ Watched directories: the VFS watcher is non-recursive, so `project.children` wat
|
|||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-16 | minor | M2.3: find in files backend (`crates/search`, `search.find`/`search.cancel`, `searchResults`/`searchDone` via the pump) |
|
||||
| 2026-10-16 | minor | M1.15: modal rule (one modal at a time, restore dialog queued), `recoveryResolved` event |
|
||||
| 2026-10-16 | minor | M1.9b: settings dialog (store, search, controls by type, reset, live updates, font-size token override) |
|
||||
| 2026-10-16 | minor | M1.14: app icon from the owner's master, multi-size ICO, placeholder as fallback |
|
||||
|
|
|
|||
|
|
@ -29,6 +29,8 @@ G11 progress: M1.7 added the crash-recovery journal: unsaved buffers are written
|
|||
|
||||
G14 progress: M1.11 moved every UI string in `ui/src` into the English catalog `ui/src/i18n/en.json` behind `t(key, params?)`, enforced by `vue/no-bare-strings-in-template` and a catalog test. Messages the engine sends (`EngineError.message`, the slogan from `app.about`) are still English text; they need message keys with params before a second language can cover them.
|
||||
|
||||
G5 progress: M2.3 landed the find-in-files backend: `crates/search` (parallel walk with the project's ignore rules, literal or regex, case, whole word, globs, binary and size skips, match cap, cancel) and the engine commands `search.find` / `search.cancel` with streamed `searchResults` and `searchDone` events; dirty buffers are searched in their buffer text. First result on a generated 20k-file tree 10–43 ms (release, measured). Open in G5: the Find tool window (M2.4), replace in files, the symbol index and go-to (M2.5, M2.6).
|
||||
|
||||
Known smaller items from M0:
|
||||
|
||||
- The CI workflow has never run: the repository has no remote until the owner decides on hosting.
|
||||
|
|
@ -39,6 +41,7 @@ Known smaller items from M0:
|
|||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-16 | minor | G5: find-in-files backend (M2.3) |
|
||||
| 2026-10-16 | minor | G3: settings dialog, theme picker (M1.9b) |
|
||||
| 2026-10-16 | minor | G3: command palette, global keymap, UI commands (M1.8) |
|
||||
| 2026-10-16 | minor | G2/G3: re-open focuses the tab, open buffers' folders watched, truncated listings shown (M1.13) |
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ Hours are the estimated agent wall-clock hours for one agent to reach behaviour
|
|||
| Editor core | 12 | 9% (est.) | syntax highlighting (M2), code folding, soft wrap and other editor settings, preview tab, pinned/split/reordered tabs, close others/all, reopen closed tab, tab limit, conflict UI for files changed on disk, autosave, local history, encodings other than UTF-8, large-file mode, column selection and multi-caret checked against the docs; recovery journal not yet checked in the running app (gaming mode) | ~106 (est.) | M1.13: opening a file that is already open switches to its tab (`bufferFocused`; engine test, Vitest, control-channel test; live check pending (gaming mode)); files in folders the tree never expanded still reload or report changes on disk, because the parent directory of every open buffer stays watched (engine tests). M1.7: crash-recovery journal (`crates/engine/src/journal.rs`, `cmd/recovery.rs`): dirty buffers are written atomically to `<data_dir>/recovery/<project-id>/` about 1 s after the last edit (at least every 2 s while typing), never per keystroke; removed on save, close and discard; `recoveryAvailable` after `project.open`, `recovery.list/restore/discard`; restored buffers are dirty and keep line ending and BOM; corrupt entries are moved aside with a warning. Restore dialog (`ui/src/components/recovery/`: Restore all / selected / Discard, time, changed-on-disk hint). Tests: engine dropped mid-edit and a new engine restores the exact CRLF + BOM text, corrupt entries, removal paths, a property test of random edit sequences → crash → restore equals the buffer (`crates/engine/tests/recovery.rs`, 10 tests), the same flow headless over the control channel (`crates/control/tests/recovery.rs`), 10 Vitest tests for the dialog. 1 MiB entry write: 8.5 ms median (release, measured). Graded against `plan/phpstorm/04-recovery.md`. M1.6: tabs and a CodeMirror 6 editor in the app (`ui/src/components/editor/`, `ui/src/stores/editor.ts`): open from the project tree or engine events, activate, close by button, middle click or `Mod+W`, dirty dot, Save / Don't Save / Cancel on closing a dirty tab, `Mod+S` / `Mod+Alt+S` from engine command specs, non-blocking notice for a dirty file changed or deleted on disk; local edits go to `buffer.edit` with version-based resync, CRLF files keep `\r\n`; undo/redo, multiple selections, search and bracket matching from CodeMirror's basic setup. Vitest: a burst of 400 keystrokes with stale versions and remote edits loses none, a CRLF burst with deletions converges, store and component tests against a fake engine (`ui/tests/editor/`, 22 tests). Checked in the app on Windows 11 with `--control` and `ui.screenshot`: typing via the UI and via `buffer.edit`, save to disk, close question, middle click, disk-change notice. 1 MB file (16,385 lines): engine `file.open` 80 ms over the control channel (debug build, measured), editor state 1–6 ms and first rendered view 4–11 ms in Chromium (measured), ≈ 90 ms in total (estimated, budget 150 ms). Graded against `plan/phpstorm/03-editor-tabs.md`. `crates/text`: rope buffer, positions, edits (M1.1, unit tests + proptests). M1.3: engine buffers with versions and dirty state, `file.open`, `buffer.get`, `buffer.edit` (UTF-16 change sets, `stale_version`), `file.save`, `file.saveAll`, `buffer.close` with unsaved-changes guard, line ending and BOM kept on save, clean buffers reload on external change; 14 integration tests on temp dirs (`crates/engine/tests/workspace.rs`) |
|
||||
| Project view, VFS | 6 | 8% (est.) | views list (project / scopes), toolbar (Select Opened File, Expand/Collapse All), sort and display options, show/hide excluded, distinct icons for library and excluded folders, file colours, speed search, preview, file operations | ~37 (est.) | `lutra-vfs` (M1.2): lazy sorted listing, ignore rules (.gitignore, .ignore, info/exclude, global excludes, Lutra defaults), native file watching via `notify` with a polling fallback, capped UTF-8 reads, atomic writes; 34 tests on temp dirs, watcher tests run on both backends (`crates/vfs/tests`). M1.5: project tree UI (lazy listing, keyboard navigation, dimmed ignored entries, event refresh) in `ui/src/components/project/`, Vitest against a mocked engine. M1.3: `project.open`, `project.close`, `project.children` (watches listed directories, capped at 256 native / 32 polling, least recently listed dropped), `treeChanged` events from disk, `events.subscribe` on the control channel with a TCP test (`crates/control/tests/events.rs`). M1.4: the tree runs against the engine in the app: folder picker (`project.openDialog`) and `lutra-desktop --control PORT PATH` open a project, the tree lists it lazily with ignored (`var/` from `.gitignore`) and excluded (`node_modules`) entries dimmed, a file created on disk appears in an expanded folder within ~100 ms; verified with `ui.screenshot` on Windows 11 and Vitest (`ui/tests/project/`, `ui/tests/ipcEvents.test.ts`). M1.13: a directory listed only partly (more than 100,000 entries) ends with a "Listing truncated" note row (Vitest). M1.12: a folder that cannot be opened from the picker or the command line (including `unsaved_changes` when switching projects) and stopped file watching show as toasts instead of stderr (`apps/lutra-desktop/src/host.rs`, `ui/src/stores/notifications.ts`, `ui/tests/notifications/`, 19 Vitest tests; live check pending, gaming mode); score unchanged. Graded against `plan/phpstorm/02-project-view.md`: tool window, folder-first tree, dimmed ignored/excluded, keyboard navigation and live refresh exist; most toolbar, view and display options do not |
|
||||
| Navigation | 10 | 0% | everything (M1.8: a command palette over commands only, no go-to-file/class/symbol yet; score unchanged) | ~120 (est.) | M1.8: `ui/src/components/palette/`, `ui/src/commands/` (fuzzy ranking, recent commands, global keymap), Vitest `ui/tests/palette/`, `ui/tests/commands/` |
|
||||
| Search and replace | 6 | 0% | everything | ~80 (est.) | — |
|
||||
| Search and replace | 6 | 0% (est.) | everything a user sees: Find in Files tool window (M2.4), replace in files, scopes, file masks UI, search history, find in path from the tree, structural search; the backend below has no UI yet, so the score stays 0% | ~76 (est.) | M2.3 backend: `crates/search` (parallel `ignore` walk + `grep-searcher`: literal/regex, case, whole word, include/exclude globs, project ignore rules with `vendor/` searched and `node_modules` skipped, binary and > 8 MiB files skipped, match cap, cancel) and engine `search.find` / `search.cancel` with `searchResults` / `searchDone` events through the pump; dirty buffers are searched in their buffer text. Tests: 9 library tests incl. a property test of random queries and file bytes (`crates/search/tests/search.rs`), 3 preview/column unit tests, 4 engine tests (`crates/engine/tests/search.rs`), a TCP control-channel test (find → results → done, invalid regex, cancel; `crates/control/tests/search.rs`). 20k-file generated tree, release: first result 10–43 ms, full walk 0.66–1.43 s (measured); live over the control channel against the running app: first result 55 ms, done 386 ms on a 5,900-file project (debug build, measured) |
|
||||
| Code completion | 10 | 0% | everything | ~250 (est.) | — |
|
||||
| Inspections and quick-fixes | 8 | 0% | everything | ~300 (est.) | — |
|
||||
| Refactoring | 7 | 0% | everything | ~250 (est.) | — |
|
||||
|
|
@ -63,6 +63,7 @@ The project tree is the first user-visible feature (Project view 8%, estimated);
|
|||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-16 | minor | M2.3: Search and replace evidence (find-in-files backend, no UI), score unchanged at 0%, hours 80 → 76 |
|
||||
| 2026-10-16 | minor | M1.9b: Settings row 4% → 7% (est.), settings dialog; M1 done |
|
||||
| 2026-10-16 | minor | M1.8: Navigation and Settings/keymaps evidence (palette, global keymap), scores unchanged |
|
||||
| 2026-10-16 | minor | M1.12: Project view evidence (picker errors as toasts), scores unchanged |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue