M1.2: native file watching via notify

- notify 8.2.0 (CC0-1.0) behind the existing Watcher API, allowed by a
  crate-scoped [[licenses.exceptions]] entry in deny.toml (owner
  decision 2026-10-14, noted in AGENTS.md and NOTICE).
- Native notifications only mark directories dirty; after 100 ms of
  quiet (at most 10 intervals) dirty dirs are rescanned and diffed, so
  events, debouncing and rename pairing match the polling backend.
- Polling stays as fallback when the native watcher cannot start, and
  per directory when a native watch is refused. Lost events or watcher
  errors trigger a full rescan; root loss is checked on every tick.
- Additive API: WatchOptions::backend (Auto, Polling) and
  Watcher::is_native.
- Watch tests run on both backends (34 vfs tests on Windows).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
danny 2026-10-14 19:43:15 +02:00
parent 5d49b57b9c
commit 3f190a635e
11 changed files with 500 additions and 86 deletions

View file

@ -33,7 +33,8 @@
//!
//! # Watching
//! [`Project::watch`] returns a [`Watcher`] and a channel of [`VfsEvent`]s;
//! see the `watch` module for the polling design and its limits.
//! see the `watch` module for the native backend (`notify`), its polling
//! fallback and their limits.
//!
//! # File IO
//! [`read_text`] reads up to [`DEFAULT_READ_LIMIT`] bytes and decodes UTF-8
@ -54,4 +55,4 @@ pub use io::{
};
pub use project::{Entry, EntryKind, Listing, MAX_CHILDREN, PathStatus, Project, ProjectOptions};
pub use rules::{EntryRole, GlobalExcludes};
pub use watch::{VfsEvent, WatchOptions, Watcher};
pub use watch::{VfsEvent, WatchBackend, WatchOptions, Watcher};

View file

@ -1,19 +1,27 @@
//! Watching the project for changes on disk.
//!
//! The watcher polls: a background thread re-reads the watched directories
//! every [`WatchOptions::interval`] (100 ms by default) and compares kind,
//! size and modification time of each entry with the previous scan. It needs
//! no dependency; native notifications (the `notify` crate) wait on a licence
//! decision and would sit behind the same API.
//! Two backends sit behind one API:
//! - **Native** (default): operating-system notifications through the
//! `notify` crate (ReadDirectoryChangesW, inotify, FSEvents). A
//! notification only marks its directory as dirty; once no notification
//! has arrived for [`WatchOptions::interval`] (100 ms by default, at most
//! ten intervals after the first one) the dirty directories are rescanned.
//! - **Polling** (fallback): every interval all watched directories are
//! rescanned. Used when the native watcher cannot start for the root
//! (network drives, exhausted inotify limits) or when
//! [`WatchBackend::Polling`] is requested, and for single directories the
//! native watcher refuses.
//!
//! Behaviour:
//! Either way, events come from comparing kind, size and modification time
//! of each entry with the previous scan, so both backends report the same
//! events. Behaviour:
//! - Only watched directories are scanned, non-recursively. The root is
//! watched from the start; callers add the directories the user expanded
//! with [`Watcher::watch_dir`]. Directories that are hidden (`.git`),
//! ignored or excluded (`node_modules`) are refused, so no events come
//! from inside them, and `.git` entries are never reported.
//! - Changes within one interval are coalesced (debounced): several writes
//! give one [`VfsEvent::Modified`], a file created and deleted again gives
//! - Changes within one debounce window are coalesced: several writes give
//! one [`VfsEvent::Modified`], a file created and deleted again gives
//! nothing. An entry that disappears while one with the same kind, size
//! and modification time appears in the same scan is reported as
//! [`VfsEvent::Renamed`].
@ -22,36 +30,55 @@
//! - A watched directory that disappears stops being watched (its parent
//! reports the removal if watched). If the root cannot be read the watcher
//! sends [`VfsEvent::Failed`] and stops: the project keeps working without
//! watching.
//! watching. If the native watcher reports an error or lost events, all
//! watched directories are rescanned.
//! - A change that keeps size and modification time is not seen; modern file
//! systems record modification times finely enough for this to be rare.
use std::collections::{BTreeMap, HashMap};
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::ffi::OsString;
use std::fs;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{self, Receiver, Sender};
use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender};
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use std::thread::{self, JoinHandle};
use std::time::{Duration, SystemTime};
use std::time::{Duration, Instant, SystemTime};
use notify::{RecommendedWatcher, RecursiveMode, Watcher as _};
use crate::VfsError;
use crate::path::check_relative;
use crate::project::{EntryKind, MAX_CHILDREN, Project, kind_of};
use crate::rules::{EntryRole, is_hidden};
/// Longest debounce, in intervals, while notifications keep arriving.
const MAX_DEBOUNCE_INTERVALS: u32 = 10;
/// Which backend [`Project::watch_with`] uses.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum WatchBackend {
/// Native notifications, falling back to polling if they fail to start.
#[default]
Auto,
/// Polling only.
Polling,
}
/// Options for [`Project::watch_with`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct WatchOptions {
/// Time between scans; also the debounce window.
/// Debounce window, and the time between scans when polling.
pub interval: Duration,
/// The backend to use.
pub backend: WatchBackend,
}
impl Default for WatchOptions {
fn default() -> Self {
WatchOptions {
interval: Duration::from_millis(100),
backend: WatchBackend::Auto,
}
}
}
@ -92,7 +119,9 @@ pub enum VfsEvent {
pub struct Watcher {
/// State shared with the thread.
shared: Arc<Shared>,
/// The polling thread, joined on drop.
/// Wakes the thread when the watcher is dropped.
signals: Sender<Signal>,
/// The background thread, joined on drop.
thread: Option<JoinHandle<()>>,
}
@ -103,8 +132,39 @@ struct Shared {
project: Project,
/// Set to ask the thread to stop.
stop: AtomicBool,
/// Last scan of every watched directory, keyed by project-relative path.
dirs: Mutex<BTreeMap<PathBuf, Snapshot>>,
/// Every watched directory, keyed by project-relative path.
dirs: Mutex<BTreeMap<PathBuf, Watched>>,
/// The native watcher, if it started.
native: Mutex<Option<RecommendedWatcher>>,
}
/// One watched directory.
#[derive(Debug)]
struct Watched {
/// Its last scan.
snapshot: Snapshot,
/// The native watcher covers it; otherwise it is polled.
native: bool,
}
/// Messages to the background thread.
#[derive(Debug)]
enum Signal {
/// Something changed at these absolute paths.
Paths(Vec<PathBuf>),
/// The native watcher lost events or failed; rescan everything.
Rescan,
/// The watcher was dropped.
Stop,
}
/// What the next scan has to look at.
#[derive(Default)]
struct Work {
/// Natively watched directories with notifications, project-relative.
dirty: BTreeSet<PathBuf>,
/// Rescan every directory.
full: bool,
}
/// Entries of one directory by name.
@ -142,7 +202,9 @@ impl Project {
}
/// Starts watching the project root; events arrive on the returned
/// channel. Changes made after this returns are reported.
/// channel. Changes made after this returns are reported. A native
/// watcher that fails to start is not an error: the watcher polls
/// instead (see [`Watcher::is_native`]).
///
/// # Errors
/// [`VfsError::WatchUnavailable`] if the root cannot be scanned or the
@ -152,27 +214,40 @@ impl Project {
&self,
options: WatchOptions,
) -> Result<(Watcher, Receiver<VfsEvent>), VfsError> {
let root = scan(self.root()).map_err(|err| VfsError::WatchUnavailable {
let (signals, signal_receiver) = mpsc::channel();
let native = match options.backend {
WatchBackend::Auto => start_native(self.root(), signals.clone()),
WatchBackend::Polling => None,
};
let snapshot = scan(self.root()).map_err(|err| VfsError::WatchUnavailable {
message: format!("cannot read {}: {err}", self.root().display()),
})?;
let mut dirs = BTreeMap::new();
dirs.insert(PathBuf::new(), root);
dirs.insert(
PathBuf::new(),
Watched {
snapshot,
native: native.is_some(),
},
);
let shared = Arc::new(Shared {
project: self.clone(),
stop: AtomicBool::new(false),
dirs: Mutex::new(dirs),
native: Mutex::new(native),
});
let (sender, receiver) = mpsc::channel();
let thread_shared = Arc::clone(&shared);
let thread = thread::Builder::new()
.name("lutra-vfs-watch".into())
.spawn(move || run(&thread_shared, &sender, options.interval))
.spawn(move || run(&thread_shared, &signal_receiver, &sender, options.interval))
.map_err(|err| VfsError::WatchUnavailable {
message: format!("cannot start the watcher thread: {err}"),
})?;
Ok((
Watcher {
shared,
signals,
thread: Some(thread),
},
receiver,
@ -180,10 +255,28 @@ impl Project {
}
}
/// Starts a native watcher on `root` that forwards notifications as
/// [`Signal`]s; `None` if the platform refuses. A send to a stopped thread
/// fails and is ignored: nobody is left to care about the notification.
fn start_native(root: &Path, signals: Sender<Signal>) -> Option<RecommendedWatcher> {
let handler = move |result: notify::Result<notify::Event>| {
let signal = match result {
Ok(event) if event.need_rescan() => Signal::Rescan,
Ok(event) => Signal::Paths(event.paths),
Err(_) => Signal::Rescan,
};
let _thread_gone = signals.send(signal);
};
let mut watcher = notify::recommended_watcher(handler).ok()?;
watcher.watch(root, RecursiveMode::NonRecursive).ok()?;
Some(watcher)
}
impl Watcher {
/// Starts watching the directory `dir` (project-relative). Returns
/// `false` without watching if the directory is hidden, ignored or
/// excluded; watching an already watched directory is a no-op.
/// excluded; watching an already watched directory is a no-op. If the
/// native watcher refuses the directory, it is polled instead.
///
/// # Errors
/// [`VfsError::OutsideProject`] for an escaping path, [`VfsError::NotFound`]
@ -198,16 +291,35 @@ impl Watcher {
return Ok(true);
}
let abs = self.shared.project.root().join(&dir);
let snapshot = scan(&abs).map_err(|err| VfsError::io(&abs, err))?;
self.shared.lock().entry(dir).or_insert(snapshot);
let native = self
.shared
.lock_native()
.as_mut()
.is_some_and(|watcher| watcher.watch(&abs, RecursiveMode::NonRecursive).is_ok());
let snapshot = match scan(&abs) {
Ok(snapshot) => snapshot,
Err(err) => {
self.shared.unwatch_native(&abs);
return Err(VfsError::io(&abs, err));
}
};
self.shared
.lock()
.entry(dir)
.or_insert(Watched { snapshot, native });
Ok(true)
}
/// Stops watching the directory `dir` (project-relative). Unknown
/// directories are ignored.
pub fn unwatch_dir(&self, dir: impl AsRef<Path>) {
if let Ok(dir) = check_relative(dir.as_ref()) {
self.shared.lock().remove(&dir);
let Ok(dir) = check_relative(dir.as_ref()) else {
return;
};
let removed = self.shared.lock().remove(&dir);
if removed.is_some_and(|watched| watched.native) {
self.shared
.unwatch_native(&self.shared.project.root().join(&dir));
}
}
@ -215,37 +327,101 @@ impl Watcher {
pub fn watched_dirs(&self) -> Vec<PathBuf> {
self.shared.lock().keys().cloned().collect()
}
/// Whether the project root is watched natively (`false`: polling).
pub fn is_native(&self) -> bool {
self.shared
.lock()
.get(Path::new(""))
.is_some_and(|watched| watched.native)
}
}
impl Drop for Watcher {
/// Stops and joins the polling thread. A panic inside the thread is not
/// propagated: the watcher is going away and has nothing left to report.
/// Stops the native watcher, then stops and joins the thread. A failed
/// wake-up send means the thread already ended, and a panic inside the
/// thread is not propagated: the watcher is going away and has nothing
/// left to report.
fn drop(&mut self) {
self.shared.stop.store(true, Ordering::SeqCst);
let native = self.shared.lock_native().take();
drop(native);
let _thread_gone = self.signals.send(Signal::Stop);
if let Some(thread) = self.thread.take() {
thread.thread().unpark();
let _thread_result = thread.join();
}
}
}
impl Shared {
/// Locks the snapshots. A poisoned lock is recovered: snapshots are only
/// replaced whole, so they are consistent even after a panic.
fn lock(&self) -> MutexGuard<'_, BTreeMap<PathBuf, Snapshot>> {
/// Locks the watched directories. A poisoned lock is recovered:
/// entries are only replaced whole, so they are consistent even after a
/// panic.
fn lock(&self) -> MutexGuard<'_, BTreeMap<PathBuf, Watched>> {
self.dirs.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Locks the native watcher, recovering a poisoned lock like
/// [`Shared::lock`].
fn lock_native(&self) -> MutexGuard<'_, Option<RecommendedWatcher>> {
self.native.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Removes the native watch on the absolute directory `abs`. Failure is
/// ignored: the directory may already be gone, which ends its watch
/// anyway.
fn unwatch_native(&self, abs: &Path) {
if let Some(watcher) = self.lock_native().as_mut() {
let _already_gone = watcher.unwatch(abs);
}
}
}
/// Body of the polling thread: scan, diff, send, until stopped, until the
/// receiver is dropped or until the root fails.
fn run(shared: &Shared, sender: &Sender<VfsEvent>, interval: Duration) {
impl Work {
/// Records a signal; `false` for [`Signal::Stop`].
fn add(&mut self, signal: Signal, project: &Project) -> bool {
match signal {
Signal::Paths(paths) => {
for path in paths {
if let Some(rel) = project.relative(&path) {
if let Some(parent) = rel.parent() {
self.dirty.insert(parent.to_path_buf());
}
self.dirty.insert(rel);
}
}
true
}
Signal::Rescan => {
self.full = true;
true
}
Signal::Stop => false,
}
}
}
/// Body of the background thread: wait for notifications (debounced) or
/// the polling interval, scan, send; until stopped, until the receiver is
/// dropped or until the root fails.
fn run(shared: &Shared, signals: &Receiver<Signal>, sender: &Sender<VfsEvent>, interval: Duration) {
loop {
thread::park_timeout(interval);
let mut work = Work::default();
match signals.recv_timeout(interval) {
Ok(signal) => {
if !work.add(signal, &shared.project)
|| !debounce(shared, signals, interval, &mut work)
{
return;
}
}
Err(RecvTimeoutError::Timeout) => {}
Err(RecvTimeoutError::Disconnected) => return,
}
if shared.stop.load(Ordering::SeqCst) {
return;
}
let (events, failed) = poll(shared);
let (events, failed) = poll(shared, &work);
for event in events {
if sender.send(event).is_err() {
return;
@ -257,36 +433,83 @@ fn run(shared: &Shared, sender: &Sender<VfsEvent>, interval: Duration) {
}
}
/// Rescans every watched directory and returns the events, plus whether
/// watching failed for good.
fn poll(shared: &Shared) -> (Vec<VfsEvent>, bool) {
/// Collects further signals until none arrives for `interval`, or for at
/// most [`MAX_DEBOUNCE_INTERVALS`] intervals. Returns `false` if the thread
/// must stop.
fn debounce(
shared: &Shared,
signals: &Receiver<Signal>,
interval: Duration,
work: &mut Work,
) -> bool {
let deadline = Instant::now() + interval * MAX_DEBOUNCE_INTERVALS;
loop {
let left = deadline.saturating_duration_since(Instant::now());
if left.is_zero() {
return true;
}
match signals.recv_timeout(interval.min(left)) {
Ok(signal) => {
if !work.add(signal, &shared.project) {
return false;
}
}
Err(RecvTimeoutError::Timeout) => return true,
Err(RecvTimeoutError::Disconnected) => return false,
}
}
}
/// Rescans the polled directories and the dirty native ones (all of them
/// for a full rescan) and returns the events, plus whether watching failed
/// for good. The root is checked on every call, so its loss is noticed
/// even when no notification reports it.
fn poll(shared: &Shared, work: &Work) -> (Vec<VfsEvent>, bool) {
let root = shared.project.root();
let mut dirs = shared.lock();
let mut root_error = match fs::metadata(root) {
Ok(meta) if meta.is_dir() => None,
Ok(_) => Some("not a directory".to_string()),
Err(err) => Some(err.to_string()),
};
let mut changes = Changes::default();
let mut gone = Vec::new();
let mut root_error = None;
for (dir, old) in dirs.iter_mut() {
match scan(&root.join(dir)) {
Ok(new) => {
diff(dir, old, &new, &mut changes);
*old = new;
if root_error.is_none() {
for (dir, watched) in dirs.iter_mut() {
if watched.native && !work.full && !work.dirty.contains(dir) {
continue;
}
Err(err) if dir.as_os_str().is_empty() => {
root_error = Some(err);
break;
match scan(&root.join(dir)) {
Ok(new) => {
diff(dir, &watched.snapshot, &new, &mut changes);
watched.snapshot = new;
}
Err(err) if dir.as_os_str().is_empty() => {
root_error = Some(err.to_string());
break;
}
Err(_) => gone.push((dir.clone(), watched.native)),
}
Err(_) => gone.push(dir.clone()),
}
}
if let Some(err) = root_error {
dirs.clear();
drop(dirs);
let native = shared.lock_native().take();
drop(native);
let failure = VfsError::WatchUnavailable {
message: format!("cannot read {}: {err}", root.display()),
};
return (vec![VfsEvent::Failed(failure)], true);
}
for dir in gone {
dirs.remove(&dir);
for (dir, _) in &gone {
dirs.remove(dir);
}
drop(dirs);
for (dir, native) in gone {
if native {
shared.unwatch_native(&root.join(dir));
}
}
(pair_renames(changes), false)
}