M3.2: LSP client core (framing, supervision, requests, document sync)

crates/lsp is now a library: Content-Length JSON-RPC framing over stdio
(32 MiB cap, malformed frames dropped and reported, fuzzed), Server::spawn
with initialize/initialized (utf-16, incremental sync, completion, hover,
signatureHelp, definition, references, rename, publishDiagnostics),
typed requests with timeouts and $/cancelRequest, answers to server
requests (configuration nulls, MethodNotFound), events on a bounded
channel, supervision with backoff restarts that reopen documents and
give up after 5 crashes in 2 min, shutdown/exit, and incremental
document sync from lutra_text edits. Tested against a scripted fake
server binary; docs: architecture, G6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
danny 2026-10-18 00:57:10 +02:00
parent 9ccfc1bf6a
commit 81831c90e0
15 changed files with 4089 additions and 2 deletions

View file

@ -1,6 +1,6 @@
# Architecture
> **Status:** M0 decisions recorded, M1 workspace, desktop bridge, tabs and editor, message catalog, engine construction, settings, themes, the recovery journal, notifications, UI commands, the palette, the settings dialog, the app icon and the syntax layer documented; M2 highlighting and `editor.reveal`, plus the search backend, the Find tool window, the symbol index, the Structure tool window and the go-to popup · **Last updated:** 2026-10-17
> **Status:** M0 decisions recorded, M1 workspace, desktop bridge, tabs and editor, message catalog, engine construction, settings, themes, the recovery journal, notifications, UI commands, the palette, the settings dialog, the app icon and the syntax layer documented; M2 highlighting and `editor.reveal`, plus the search backend, the Find tool window, the symbol index, the Structure tool window and the go-to popup; M3 LSP client library · **Last updated:** 2026-10-18
## Overview
@ -273,6 +273,20 @@ Watched directories: the VFS watcher is non-recursive, so `project.children` wat
- Opening: Enter or a click closes the popup (focus goes back) and runs `editor.reveal` with `{path}` for files (the file's cursor stays) or `{path, line, column}` for symbols. The popup is an exclusive modal (`ModalId` `goto` in `commands/modals.ts`). Like the palette it is a launcher: running any command other than the three open commands closes it (`commands/store.ts` › `execute`), so an agent's `editor.reveal` never leaves it open.
- Live check (debug build, bundled UI, port 7826, temp project with 34 files and a `vendor/` folder): `goto.fileOpen {query: "user"}` listed the two project files before the vendor one with its `library` label; `goto.classOpen {query: "usr"}` and `goto.symbolOpen {query: "User::find"}` switched the tab with marks shown; `palette.open` then `goto.fileOpen` left only the popup; an `editor.reveal` of the symbol item closed the popup and put the cursor on `find`. `goto.file` answered in < 1 ms over the control channel. Enter and clicks cannot be sent over the control channel; Vitest covers them.
### LSP client (M3.2)
- `crates/lsp` (L3, depends on `lutra-text` and `serde_json` only) is a library; the engine wiring is M3.3. `Server::spawn(command, args, root, ServerOptions)` starts the process in `root` with piped stdio (on Windows with `CREATE_NO_WINDOW`) and returns at once with the server and a bounded `Receiver<ServerEvent>` (capacity 4096; overflow drops and counts, `dropped_events()`). A spawn error (missing executable, relative root) is returned synchronously.
- Threads, no async runtime: per process a writer (the only one touching stdin, fed by a channel), a reader (frames from stdout), a stderr drain; per server a supervisor that watches the process, expires timeouts every 20 ms, restarts and shuts down. A server is a few blocking pipes and the engine pump is synchronous, so threads with channels are simpler than pulling an executor into L3. All writes happen under the client's one mutex, so messages leave in the order the state changed; callbacks always run outside it (a panicking callback is caught).
- Framing (`jsonrpc`): `Content-Length` headers (case-insensitive, other headers ignored, header lines capped at 1 KiB and 16 per frame), bodies capped at 32 MiB (`MAX_MESSAGE_BYTES`; an oversized body is read and discarded without buffering). A malformed header block, invalid JSON or a message that is neither request, response nor notification becomes `ServerEvent::ProtocolError(reason)` (the log line) and the reader carries on with the next frame. serde_json's recursion limit bounds nesting. Proptests: message round trips, random bytes, header-like noise, recovery after bad frames, random JSON bodies.
- Initialize: `initialize` (processId, clientInfo, rootUri/rootPath, one workspace folder, `initializationOptions` from the options) with the capabilities of `client_capabilities()`: incremental sync with `didSave`, completion (no snippets, markdown/plain documentation, context), hover, signatureHelp, definition (links), references, rename with prepare, `publishDiagnostics` with versions, `workspace/configuration`, workspace folders, work-done progress, `positionEncodings: ["utf-16"]`; no dynamic registration and no `applyEdit`. Then `initialized`, every open document is (re)opened, queued requests are sent, and the status becomes `Ready`. A server choosing another encoding is logged. `initialize` times out after 30 s; a failed or late `initialize` counts as a crash.
- Requests: `request`/`request_timeout` return a `RequestHandle` (`wait`, `wait_timeout`, `try_result`, `cancel`); `request_with` takes a callback instead. Typed helpers: `completion(path, position, trigger)`, `hover`, `signature_help`, `definition`, `references(..., include_declaration)`, `prepare_rename`, `rename(..., new_name)`; results are raw JSON, with parsers in `types` for `Location::list_from_json`, `WorkspaceEdit::from_json` (documentChanges or changes; resource operations flagged, malformed edits mark it `incomplete`) and `PublishDiagnostics`. Requests issued before `Ready` are queued (the timeout runs from the call). Every request resolves exactly once: result, `Response` error, `Timeout` (default 10 s; `$/cancelRequest` is sent), `Cancelled` (also sends `$/cancelRequest`), `ServerCrashed`, or `Stopped`. At most 1024 requests are pending.
- Server requests: `workspace/configuration` gets one `null` per item (at most 1000); `window/workDoneProgress/create` and capability (un)registration are acknowledged with `null`; `window/showMessageRequest` is shown as a `ShowMessage` event and answered `null`; `workspace/workspaceFolders` returns the root; everything else gets `MethodNotFound`. Notifications become `Diagnostics`, `LogMessage`, `ShowMessage`, `Progress` or the generic `Notification`; stderr lines become `Stderr` (4 KiB cap).
- Supervision: a process exit, a failed `initialize` or a broken watch is a crash. Requests sent to that process fail with `ServerCrashed`, queued ones wait. The status goes to `Restarting {crashes, delay, reason}` with delays 1 s, 2 s, 4 s ... capped at 30 s (doubling per crash in the window); the fifth crash within 2 min ends in `Failed {message}` and fails everything with `Stopped`. All of it is `RestartPolicy` in the options (the tests use milliseconds). After a restart every open document is reopened with its current text and version.
- Document sync: the client keeps each open document in a `lutra_text::Buffer`. `did_open(path, language_id, text)` (version 1), `did_change(path, &[Edit])` with byte-range edits as in `Buffer::apply`, `did_change_full(path, text)`, `did_save(path)` (with the text if the server asks), `did_close(path)`; each change bumps the version and returns it. Edits become UTF-16 content changes (`content_changes`): touching edits are merged and the changes are sent from the end of the text to the start, so sequential application on the server equals the atomic batch, also across CRLF and surrogate pairs (property-tested). The server's `textDocumentSync` decides incremental, full or no `didChange`. Edits that do not fit the client's copy fail with `LspError::Text` and leave it unchanged. Document operations never fail because of the server's state: before `Ready` or while restarting they only update the copy.
- Shutdown: `shutdown(self)` sends `shutdown` (waits up to 2 s), `exit`, kills after 1 s, fails what is pending with `Stopped` and reports `Stopped`; it blocks for up to about 3 s, so callers run it off the engine lock. Dropping the `Server` requests the same stop in the background.
- URIs (`uri`): `path_to_uri` percent-encodes everything except unreserved characters, `/` and a drive colon (`file:///C:/a%20b.php`, UNC as `file://host/share`); `uri_to_path` accepts escapes in any case, an encoded drive colon and `localhost`, and upper-cases the drive letter; bad escapes, NUL and other schemes give `None`.
- Tests: the fake server `src/bin/lutra-lsp-fake.rs` (a test fixture, started via `CARGO_BIN_EXE_lutra-lsp-fake`) covers initialize, server requests, log and stderr, typed requests and errors, queueing before ready, timeout and cancel with `$/cancelRequest`, incremental and full sync (the fake applies the changes and publishes the text as a diagnostic), malformed output, crash, restart and reopen, initialize errors, giving up with the backoff sequence, shutdown and spawn errors (13 tests, about 0.5 s). `real_phpactor` is ignored and runs with `LUTRA_PHPACTOR` (path to `phpactor.phar`, `LUTRA_PHP` for PHP); it was not run in M3.2 (no Phpactor on the dev machine yet; M3.1 downloads it).
## Decisions (M0)
| Question | Decision | Why |
@ -290,6 +304,7 @@ Watched directories: the VFS watcher is non-recursive, so `project.children` wat
| Date | Change | Summary |
|---|---|---|
| 2026-10-18 | minor | M3.2: LSP client library (`crates/lsp`: framing, supervision and restart, initialize, requests with timeout and cancel, server requests, document sync, fake server tests) |
| 2026-10-17 | minor | M2.6: go-to popup (store, debounce and stale drop, modes, `editor.reveal`), `library` on `goto.file` items |
| 2026-10-17 | minor | M2.7: per-buffer syntax tree off the engine lock, `syntax.outline`, `outlineChanged`, left stripe, Structure tool window |
| 2026-10-17 | minor | M2.5: symbol index (`crates/index`: parallel build, updater, fuzzy queries), `index.status`, `goto.file`/`goto.class`/`goto.symbol`, `indexProgress`/`indexReady` via the pump, timing |

View file

@ -1,6 +1,6 @@
# Gaps
> **Status:** M0 closed G1; G2 under way (text buffer, engine buffers, tabs and editor in the app, recovery journal); G3 M1 part done (project tree, palette, settings dialog, themes); G4 under way (tree-sitter outline and folding library, Lezer highlighting, Structure tool window); G5 under way (search backend, Find tool window, symbol index, go-to popup); G14 English UI catalog in place · **Last updated:** 2026-10-17
> **Status:** M0 closed G1; G2 under way (text buffer, engine buffers, tabs and editor in the app, recovery journal); G3 M1 part done (project tree, palette, settings dialog, themes); G4 under way (tree-sitter outline and folding library, Lezer highlighting, Structure tool window); G5 under way (search backend, Find tool window, symbol index, go-to popup); G6 under way (LSP client library); G14 English UI catalog in place · **Last updated:** 2026-10-18
Every known shortfall, ranked. This is the work list: unless the owner gives a task, take the highest open gap. Hours are agent wall-clock hours for one agent, all estimated.
@ -39,10 +39,13 @@ Known smaller items from M0:
- `ui.screenshot` captures the screen region of the window, so covering windows appear in the image; a webview-native capture would avoid that.
- `cargo xtask fixtures` needs Composer, which the M0 machine does not have.
G6 progress: M3.2 landed the LSP client library in `crates/lsp`: JSON-RPC framing over stdio (32 MiB cap, malformed messages dropped and logged, fuzzed), `initialize` with UTF-16 positions, typed requests with timeouts and `$/cancelRequest`, answers to server requests, supervision with backoff restarts (give up after 5 crashes in 2 min) and reopening of documents, incremental document sync from `lutra_text` edits, all tested against a fake server. Not yet wired into the engine (M3.3) and not yet run against a real Phpactor; diagnostics, completion, hover, navigation and rename are still open (M3.3 to M3.5).
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-18 | minor | G6: LSP client library (M3.2) |
| 2026-10-17 | minor | G5: go-to popup (M2.6) |
| 2026-10-17 | minor | G4: Structure tool window (M2.7); G12: raw-JSON command results |
| 2026-10-17 | minor | G5: symbol index and go-to commands (M2.5) |