M1.2: native file watching via notify
- notify 8.2.0 (CC0-1.0) behind the existing Watcher API, allowed by a crate-scoped [[licenses.exceptions]] entry in deny.toml (owner decision 2026-10-14, noted in AGENTS.md and NOTICE). - Native notifications only mark directories dirty; after 100 ms of quiet (at most 10 intervals) dirty dirs are rescanned and diffed, so events, debouncing and rename pairing match the polling backend. - Polling stays as fallback when the native watcher cannot start, and per directory when a native watch is refused. Lost events or watcher errors trigger a full rescan; root loss is checked on every tick. - Additive API: WatchOptions::backend (Auto, Polling) and Watcher::is_native. - Watch tests run on both backends (34 vfs tests on Windows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
5d49b57b9c
commit
3f190a635e
11 changed files with 500 additions and 86 deletions
|
|
@ -1,19 +1,27 @@
|
|||
//! Watching the project for changes on disk.
|
||||
//!
|
||||
//! The watcher polls: a background thread re-reads the watched directories
|
||||
//! every [`WatchOptions::interval`] (100 ms by default) and compares kind,
|
||||
//! size and modification time of each entry with the previous scan. It needs
|
||||
//! no dependency; native notifications (the `notify` crate) wait on a licence
|
||||
//! decision and would sit behind the same API.
|
||||
//! Two backends sit behind one API:
|
||||
//! - **Native** (default): operating-system notifications through the
|
||||
//! `notify` crate (ReadDirectoryChangesW, inotify, FSEvents). A
|
||||
//! notification only marks its directory as dirty; once no notification
|
||||
//! has arrived for [`WatchOptions::interval`] (100 ms by default, at most
|
||||
//! ten intervals after the first one) the dirty directories are rescanned.
|
||||
//! - **Polling** (fallback): every interval all watched directories are
|
||||
//! rescanned. Used when the native watcher cannot start for the root
|
||||
//! (network drives, exhausted inotify limits) or when
|
||||
//! [`WatchBackend::Polling`] is requested, and for single directories the
|
||||
//! native watcher refuses.
|
||||
//!
|
||||
//! Behaviour:
|
||||
//! Either way, events come from comparing kind, size and modification time
|
||||
//! of each entry with the previous scan, so both backends report the same
|
||||
//! events. Behaviour:
|
||||
//! - Only watched directories are scanned, non-recursively. The root is
|
||||
//! watched from the start; callers add the directories the user expanded
|
||||
//! with [`Watcher::watch_dir`]. Directories that are hidden (`.git`),
|
||||
//! ignored or excluded (`node_modules`) are refused, so no events come
|
||||
//! from inside them, and `.git` entries are never reported.
|
||||
//! - Changes within one interval are coalesced (debounced): several writes
|
||||
//! give one [`VfsEvent::Modified`], a file created and deleted again gives
|
||||
//! - Changes within one debounce window are coalesced: several writes give
|
||||
//! one [`VfsEvent::Modified`], a file created and deleted again gives
|
||||
//! nothing. An entry that disappears while one with the same kind, size
|
||||
//! and modification time appears in the same scan is reported as
|
||||
//! [`VfsEvent::Renamed`].
|
||||
|
|
@ -22,36 +30,55 @@
|
|||
//! - A watched directory that disappears stops being watched (its parent
|
||||
//! reports the removal if watched). If the root cannot be read the watcher
|
||||
//! sends [`VfsEvent::Failed`] and stops: the project keeps working without
|
||||
//! watching.
|
||||
//! watching. If the native watcher reports an error or lost events, all
|
||||
//! watched directories are rescanned.
|
||||
//! - A change that keeps size and modification time is not seen; modern file
|
||||
//! systems record modification times finely enough for this to be rare.
|
||||
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use std::collections::{BTreeMap, BTreeSet, HashMap};
|
||||
use std::ffi::OsString;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::mpsc::{self, Receiver, Sender};
|
||||
use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender};
|
||||
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
|
||||
use std::thread::{self, JoinHandle};
|
||||
use std::time::{Duration, SystemTime};
|
||||
use std::time::{Duration, Instant, SystemTime};
|
||||
|
||||
use notify::{RecommendedWatcher, RecursiveMode, Watcher as _};
|
||||
|
||||
use crate::VfsError;
|
||||
use crate::path::check_relative;
|
||||
use crate::project::{EntryKind, MAX_CHILDREN, Project, kind_of};
|
||||
use crate::rules::{EntryRole, is_hidden};
|
||||
|
||||
/// Longest debounce, in intervals, while notifications keep arriving.
|
||||
const MAX_DEBOUNCE_INTERVALS: u32 = 10;
|
||||
|
||||
/// Which backend [`Project::watch_with`] uses.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum WatchBackend {
|
||||
/// Native notifications, falling back to polling if they fail to start.
|
||||
#[default]
|
||||
Auto,
|
||||
/// Polling only.
|
||||
Polling,
|
||||
}
|
||||
|
||||
/// Options for [`Project::watch_with`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct WatchOptions {
|
||||
/// Time between scans; also the debounce window.
|
||||
/// Debounce window, and the time between scans when polling.
|
||||
pub interval: Duration,
|
||||
/// The backend to use.
|
||||
pub backend: WatchBackend,
|
||||
}
|
||||
|
||||
impl Default for WatchOptions {
|
||||
fn default() -> Self {
|
||||
WatchOptions {
|
||||
interval: Duration::from_millis(100),
|
||||
backend: WatchBackend::Auto,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -92,7 +119,9 @@ pub enum VfsEvent {
|
|||
pub struct Watcher {
|
||||
/// State shared with the thread.
|
||||
shared: Arc<Shared>,
|
||||
/// The polling thread, joined on drop.
|
||||
/// Wakes the thread when the watcher is dropped.
|
||||
signals: Sender<Signal>,
|
||||
/// The background thread, joined on drop.
|
||||
thread: Option<JoinHandle<()>>,
|
||||
}
|
||||
|
||||
|
|
@ -103,8 +132,39 @@ struct Shared {
|
|||
project: Project,
|
||||
/// Set to ask the thread to stop.
|
||||
stop: AtomicBool,
|
||||
/// Last scan of every watched directory, keyed by project-relative path.
|
||||
dirs: Mutex<BTreeMap<PathBuf, Snapshot>>,
|
||||
/// Every watched directory, keyed by project-relative path.
|
||||
dirs: Mutex<BTreeMap<PathBuf, Watched>>,
|
||||
/// The native watcher, if it started.
|
||||
native: Mutex<Option<RecommendedWatcher>>,
|
||||
}
|
||||
|
||||
/// One watched directory.
|
||||
#[derive(Debug)]
|
||||
struct Watched {
|
||||
/// Its last scan.
|
||||
snapshot: Snapshot,
|
||||
/// The native watcher covers it; otherwise it is polled.
|
||||
native: bool,
|
||||
}
|
||||
|
||||
/// Messages to the background thread.
|
||||
#[derive(Debug)]
|
||||
enum Signal {
|
||||
/// Something changed at these absolute paths.
|
||||
Paths(Vec<PathBuf>),
|
||||
/// The native watcher lost events or failed; rescan everything.
|
||||
Rescan,
|
||||
/// The watcher was dropped.
|
||||
Stop,
|
||||
}
|
||||
|
||||
/// What the next scan has to look at.
|
||||
#[derive(Default)]
|
||||
struct Work {
|
||||
/// Natively watched directories with notifications, project-relative.
|
||||
dirty: BTreeSet<PathBuf>,
|
||||
/// Rescan every directory.
|
||||
full: bool,
|
||||
}
|
||||
|
||||
/// Entries of one directory by name.
|
||||
|
|
@ -142,7 +202,9 @@ impl Project {
|
|||
}
|
||||
|
||||
/// Starts watching the project root; events arrive on the returned
|
||||
/// channel. Changes made after this returns are reported.
|
||||
/// channel. Changes made after this returns are reported. A native
|
||||
/// watcher that fails to start is not an error: the watcher polls
|
||||
/// instead (see [`Watcher::is_native`]).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VfsError::WatchUnavailable`] if the root cannot be scanned or the
|
||||
|
|
@ -152,27 +214,40 @@ impl Project {
|
|||
&self,
|
||||
options: WatchOptions,
|
||||
) -> Result<(Watcher, Receiver<VfsEvent>), VfsError> {
|
||||
let root = scan(self.root()).map_err(|err| VfsError::WatchUnavailable {
|
||||
let (signals, signal_receiver) = mpsc::channel();
|
||||
let native = match options.backend {
|
||||
WatchBackend::Auto => start_native(self.root(), signals.clone()),
|
||||
WatchBackend::Polling => None,
|
||||
};
|
||||
let snapshot = scan(self.root()).map_err(|err| VfsError::WatchUnavailable {
|
||||
message: format!("cannot read {}: {err}", self.root().display()),
|
||||
})?;
|
||||
let mut dirs = BTreeMap::new();
|
||||
dirs.insert(PathBuf::new(), root);
|
||||
dirs.insert(
|
||||
PathBuf::new(),
|
||||
Watched {
|
||||
snapshot,
|
||||
native: native.is_some(),
|
||||
},
|
||||
);
|
||||
let shared = Arc::new(Shared {
|
||||
project: self.clone(),
|
||||
stop: AtomicBool::new(false),
|
||||
dirs: Mutex::new(dirs),
|
||||
native: Mutex::new(native),
|
||||
});
|
||||
let (sender, receiver) = mpsc::channel();
|
||||
let thread_shared = Arc::clone(&shared);
|
||||
let thread = thread::Builder::new()
|
||||
.name("lutra-vfs-watch".into())
|
||||
.spawn(move || run(&thread_shared, &sender, options.interval))
|
||||
.spawn(move || run(&thread_shared, &signal_receiver, &sender, options.interval))
|
||||
.map_err(|err| VfsError::WatchUnavailable {
|
||||
message: format!("cannot start the watcher thread: {err}"),
|
||||
})?;
|
||||
Ok((
|
||||
Watcher {
|
||||
shared,
|
||||
signals,
|
||||
thread: Some(thread),
|
||||
},
|
||||
receiver,
|
||||
|
|
@ -180,10 +255,28 @@ impl Project {
|
|||
}
|
||||
}
|
||||
|
||||
/// Starts a native watcher on `root` that forwards notifications as
|
||||
/// [`Signal`]s; `None` if the platform refuses. A send to a stopped thread
|
||||
/// fails and is ignored: nobody is left to care about the notification.
|
||||
fn start_native(root: &Path, signals: Sender<Signal>) -> Option<RecommendedWatcher> {
|
||||
let handler = move |result: notify::Result<notify::Event>| {
|
||||
let signal = match result {
|
||||
Ok(event) if event.need_rescan() => Signal::Rescan,
|
||||
Ok(event) => Signal::Paths(event.paths),
|
||||
Err(_) => Signal::Rescan,
|
||||
};
|
||||
let _thread_gone = signals.send(signal);
|
||||
};
|
||||
let mut watcher = notify::recommended_watcher(handler).ok()?;
|
||||
watcher.watch(root, RecursiveMode::NonRecursive).ok()?;
|
||||
Some(watcher)
|
||||
}
|
||||
|
||||
impl Watcher {
|
||||
/// Starts watching the directory `dir` (project-relative). Returns
|
||||
/// `false` without watching if the directory is hidden, ignored or
|
||||
/// excluded; watching an already watched directory is a no-op.
|
||||
/// excluded; watching an already watched directory is a no-op. If the
|
||||
/// native watcher refuses the directory, it is polled instead.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VfsError::OutsideProject`] for an escaping path, [`VfsError::NotFound`]
|
||||
|
|
@ -198,16 +291,35 @@ impl Watcher {
|
|||
return Ok(true);
|
||||
}
|
||||
let abs = self.shared.project.root().join(&dir);
|
||||
let snapshot = scan(&abs).map_err(|err| VfsError::io(&abs, err))?;
|
||||
self.shared.lock().entry(dir).or_insert(snapshot);
|
||||
let native = self
|
||||
.shared
|
||||
.lock_native()
|
||||
.as_mut()
|
||||
.is_some_and(|watcher| watcher.watch(&abs, RecursiveMode::NonRecursive).is_ok());
|
||||
let snapshot = match scan(&abs) {
|
||||
Ok(snapshot) => snapshot,
|
||||
Err(err) => {
|
||||
self.shared.unwatch_native(&abs);
|
||||
return Err(VfsError::io(&abs, err));
|
||||
}
|
||||
};
|
||||
self.shared
|
||||
.lock()
|
||||
.entry(dir)
|
||||
.or_insert(Watched { snapshot, native });
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Stops watching the directory `dir` (project-relative). Unknown
|
||||
/// directories are ignored.
|
||||
pub fn unwatch_dir(&self, dir: impl AsRef<Path>) {
|
||||
if let Ok(dir) = check_relative(dir.as_ref()) {
|
||||
self.shared.lock().remove(&dir);
|
||||
let Ok(dir) = check_relative(dir.as_ref()) else {
|
||||
return;
|
||||
};
|
||||
let removed = self.shared.lock().remove(&dir);
|
||||
if removed.is_some_and(|watched| watched.native) {
|
||||
self.shared
|
||||
.unwatch_native(&self.shared.project.root().join(&dir));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -215,37 +327,101 @@ impl Watcher {
|
|||
pub fn watched_dirs(&self) -> Vec<PathBuf> {
|
||||
self.shared.lock().keys().cloned().collect()
|
||||
}
|
||||
|
||||
/// Whether the project root is watched natively (`false`: polling).
|
||||
pub fn is_native(&self) -> bool {
|
||||
self.shared
|
||||
.lock()
|
||||
.get(Path::new(""))
|
||||
.is_some_and(|watched| watched.native)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Watcher {
|
||||
/// Stops and joins the polling thread. A panic inside the thread is not
|
||||
/// propagated: the watcher is going away and has nothing left to report.
|
||||
/// Stops the native watcher, then stops and joins the thread. A failed
|
||||
/// wake-up send means the thread already ended, and a panic inside the
|
||||
/// thread is not propagated: the watcher is going away and has nothing
|
||||
/// left to report.
|
||||
fn drop(&mut self) {
|
||||
self.shared.stop.store(true, Ordering::SeqCst);
|
||||
let native = self.shared.lock_native().take();
|
||||
drop(native);
|
||||
let _thread_gone = self.signals.send(Signal::Stop);
|
||||
if let Some(thread) = self.thread.take() {
|
||||
thread.thread().unpark();
|
||||
let _thread_result = thread.join();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Shared {
|
||||
/// Locks the snapshots. A poisoned lock is recovered: snapshots are only
|
||||
/// replaced whole, so they are consistent even after a panic.
|
||||
fn lock(&self) -> MutexGuard<'_, BTreeMap<PathBuf, Snapshot>> {
|
||||
/// Locks the watched directories. A poisoned lock is recovered:
|
||||
/// entries are only replaced whole, so they are consistent even after a
|
||||
/// panic.
|
||||
fn lock(&self) -> MutexGuard<'_, BTreeMap<PathBuf, Watched>> {
|
||||
self.dirs.lock().unwrap_or_else(PoisonError::into_inner)
|
||||
}
|
||||
|
||||
/// Locks the native watcher, recovering a poisoned lock like
|
||||
/// [`Shared::lock`].
|
||||
fn lock_native(&self) -> MutexGuard<'_, Option<RecommendedWatcher>> {
|
||||
self.native.lock().unwrap_or_else(PoisonError::into_inner)
|
||||
}
|
||||
|
||||
/// Removes the native watch on the absolute directory `abs`. Failure is
|
||||
/// ignored: the directory may already be gone, which ends its watch
|
||||
/// anyway.
|
||||
fn unwatch_native(&self, abs: &Path) {
|
||||
if let Some(watcher) = self.lock_native().as_mut() {
|
||||
let _already_gone = watcher.unwatch(abs);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Body of the polling thread: scan, diff, send, until stopped, until the
|
||||
/// receiver is dropped or until the root fails.
|
||||
fn run(shared: &Shared, sender: &Sender<VfsEvent>, interval: Duration) {
|
||||
impl Work {
|
||||
/// Records a signal; `false` for [`Signal::Stop`].
|
||||
fn add(&mut self, signal: Signal, project: &Project) -> bool {
|
||||
match signal {
|
||||
Signal::Paths(paths) => {
|
||||
for path in paths {
|
||||
if let Some(rel) = project.relative(&path) {
|
||||
if let Some(parent) = rel.parent() {
|
||||
self.dirty.insert(parent.to_path_buf());
|
||||
}
|
||||
self.dirty.insert(rel);
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
Signal::Rescan => {
|
||||
self.full = true;
|
||||
true
|
||||
}
|
||||
Signal::Stop => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Body of the background thread: wait for notifications (debounced) or
|
||||
/// the polling interval, scan, send; until stopped, until the receiver is
|
||||
/// dropped or until the root fails.
|
||||
fn run(shared: &Shared, signals: &Receiver<Signal>, sender: &Sender<VfsEvent>, interval: Duration) {
|
||||
loop {
|
||||
thread::park_timeout(interval);
|
||||
let mut work = Work::default();
|
||||
match signals.recv_timeout(interval) {
|
||||
Ok(signal) => {
|
||||
if !work.add(signal, &shared.project)
|
||||
|| !debounce(shared, signals, interval, &mut work)
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
Err(RecvTimeoutError::Timeout) => {}
|
||||
Err(RecvTimeoutError::Disconnected) => return,
|
||||
}
|
||||
if shared.stop.load(Ordering::SeqCst) {
|
||||
return;
|
||||
}
|
||||
let (events, failed) = poll(shared);
|
||||
let (events, failed) = poll(shared, &work);
|
||||
for event in events {
|
||||
if sender.send(event).is_err() {
|
||||
return;
|
||||
|
|
@ -257,36 +433,83 @@ fn run(shared: &Shared, sender: &Sender<VfsEvent>, interval: Duration) {
|
|||
}
|
||||
}
|
||||
|
||||
/// Rescans every watched directory and returns the events, plus whether
|
||||
/// watching failed for good.
|
||||
fn poll(shared: &Shared) -> (Vec<VfsEvent>, bool) {
|
||||
/// Collects further signals until none arrives for `interval`, or for at
|
||||
/// most [`MAX_DEBOUNCE_INTERVALS`] intervals. Returns `false` if the thread
|
||||
/// must stop.
|
||||
fn debounce(
|
||||
shared: &Shared,
|
||||
signals: &Receiver<Signal>,
|
||||
interval: Duration,
|
||||
work: &mut Work,
|
||||
) -> bool {
|
||||
let deadline = Instant::now() + interval * MAX_DEBOUNCE_INTERVALS;
|
||||
loop {
|
||||
let left = deadline.saturating_duration_since(Instant::now());
|
||||
if left.is_zero() {
|
||||
return true;
|
||||
}
|
||||
match signals.recv_timeout(interval.min(left)) {
|
||||
Ok(signal) => {
|
||||
if !work.add(signal, &shared.project) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
Err(RecvTimeoutError::Timeout) => return true,
|
||||
Err(RecvTimeoutError::Disconnected) => return false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rescans the polled directories and the dirty native ones (all of them
|
||||
/// for a full rescan) and returns the events, plus whether watching failed
|
||||
/// for good. The root is checked on every call, so its loss is noticed
|
||||
/// even when no notification reports it.
|
||||
fn poll(shared: &Shared, work: &Work) -> (Vec<VfsEvent>, bool) {
|
||||
let root = shared.project.root();
|
||||
let mut dirs = shared.lock();
|
||||
let mut root_error = match fs::metadata(root) {
|
||||
Ok(meta) if meta.is_dir() => None,
|
||||
Ok(_) => Some("not a directory".to_string()),
|
||||
Err(err) => Some(err.to_string()),
|
||||
};
|
||||
let mut changes = Changes::default();
|
||||
let mut gone = Vec::new();
|
||||
let mut root_error = None;
|
||||
for (dir, old) in dirs.iter_mut() {
|
||||
match scan(&root.join(dir)) {
|
||||
Ok(new) => {
|
||||
diff(dir, old, &new, &mut changes);
|
||||
*old = new;
|
||||
if root_error.is_none() {
|
||||
for (dir, watched) in dirs.iter_mut() {
|
||||
if watched.native && !work.full && !work.dirty.contains(dir) {
|
||||
continue;
|
||||
}
|
||||
Err(err) if dir.as_os_str().is_empty() => {
|
||||
root_error = Some(err);
|
||||
break;
|
||||
match scan(&root.join(dir)) {
|
||||
Ok(new) => {
|
||||
diff(dir, &watched.snapshot, &new, &mut changes);
|
||||
watched.snapshot = new;
|
||||
}
|
||||
Err(err) if dir.as_os_str().is_empty() => {
|
||||
root_error = Some(err.to_string());
|
||||
break;
|
||||
}
|
||||
Err(_) => gone.push((dir.clone(), watched.native)),
|
||||
}
|
||||
Err(_) => gone.push(dir.clone()),
|
||||
}
|
||||
}
|
||||
if let Some(err) = root_error {
|
||||
dirs.clear();
|
||||
drop(dirs);
|
||||
let native = shared.lock_native().take();
|
||||
drop(native);
|
||||
let failure = VfsError::WatchUnavailable {
|
||||
message: format!("cannot read {}: {err}", root.display()),
|
||||
};
|
||||
return (vec![VfsEvent::Failed(failure)], true);
|
||||
}
|
||||
for dir in gone {
|
||||
dirs.remove(&dir);
|
||||
for (dir, _) in &gone {
|
||||
dirs.remove(dir);
|
||||
}
|
||||
drop(dirs);
|
||||
for (dir, native) in gone {
|
||||
if native {
|
||||
shared.unwatch_native(&root.join(dir));
|
||||
}
|
||||
}
|
||||
(pair_renames(changes), false)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue