M4.10: remotes: fetch, pull, push, clone and manage remotes
Owner decision D23: Lutra runs fetch, pull (merge or rebase per the new git.pull.mode setting), push (upstream set on the first push) and clone through the user's git. Any HTTPS or SSH server works; sign-in stays with git's credential helper (its own window is allowed for network calls only) and the ssh agent. Terminal prompts stay off, ext:: is refused, and URL userinfo is redacted in every message, progress line and remote listing. - vcs: Call::network with a 30 min timeout, a stderr progress sink and a Cancel flag that kills git's process group; remote.rs with clone_repo, fetch, pull, push (--porcelain rejections), remotes and add/set-url/ remove; explained refusals (rejected push, sign-in, host key, not found, unreachable, no upstream, local changes, conflicts). - engine: git.fetch/pull/push/clone/cancel/remotes/remoteAdd/remoteSetUrl/ remoteRemove, one network call at a time, gitProgress events, gitChanged after changes. - ui: status-bar progress with Cancel and ahead/behind, vcs.fetch/pull/ push/clone/manageRemotes (Git menu, no default shortcuts), Clone Repository and Manage Remotes dialogs; desktop pick_dir IPC. - Tests against local bare repositories only (vcs 8, engine 3, Vitest 11) plus unit and property tests; docs updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
48b892a6d5
commit
d097558ba5
36 changed files with 4426 additions and 45 deletions
|
|
@ -46,6 +46,12 @@ pub enum VcsError {
|
|||
/// A commit message is empty.
|
||||
#[error("the commit message is empty")]
|
||||
EmptyMessage,
|
||||
/// The call was cancelled and git was stopped.
|
||||
#[error("the Git operation was cancelled")]
|
||||
Cancelled,
|
||||
/// A remote URL or clone target from the caller is not valid.
|
||||
#[error("{0}")]
|
||||
InvalidInput(String),
|
||||
}
|
||||
|
||||
impl VcsError {
|
||||
|
|
@ -54,7 +60,10 @@ impl VcsError {
|
|||
pub fn is_invalid_input(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
Self::InvalidPath(_) | Self::InvalidName(_) | Self::EmptyMessage
|
||||
Self::InvalidPath(_)
|
||||
| Self::InvalidName(_)
|
||||
| Self::EmptyMessage
|
||||
| Self::InvalidInput(_)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -7,12 +7,16 @@
|
|||
//! - [`Repo`] runs status, stage, unstage, commit, show, branches,
|
||||
//! checkout and branch creation. Calls that change the repository are
|
||||
//! serialised per repository.
|
||||
//! - Remotes (M4.10, owner decision D23): fetch, pull, push, clone and
|
||||
//! remote management, with progress and cancel; credentials stay with
|
||||
//! git's credential helper and the ssh agent (see `remote.rs`).
|
||||
//! - [`RepoWatch`] notices changes inside `.git` (HEAD, index, refs).
|
||||
//!
|
||||
//! Every call goes through [`runner::run`]: arguments as an array, never a
|
||||
//! shell; no terminal or credential prompts; a timeout; capped output;
|
||||
//! machine-readable output only (porcelain v2, `-z`, NUL-separated
|
||||
//! formats). No network operation exists here.
|
||||
//! shell; no terminal prompts (network calls allow the credential helper's
|
||||
//! own window); a timeout; capped output; machine-readable output only
|
||||
//! (porcelain v2, `-z`, NUL-separated formats); URL userinfo redacted in
|
||||
//! messages.
|
||||
//!
|
||||
//! Calls block for as long as git runs, so the engine runs them on their
|
||||
//! own threads, off its lock.
|
||||
|
|
@ -20,6 +24,7 @@
|
|||
mod error;
|
||||
pub mod log;
|
||||
pub mod porcelain;
|
||||
mod remote;
|
||||
mod repo;
|
||||
pub mod runner;
|
||||
mod watch;
|
||||
|
|
@ -27,8 +32,13 @@ mod watch;
|
|||
pub use error::VcsError;
|
||||
pub use log::{CommitDetails, CommitFile, LOG_LIMIT, LogCommit, LogQuery};
|
||||
pub use porcelain::{FileState, FileStatus, Head, Status, Version};
|
||||
pub use remote::{
|
||||
MAX_URL_LEN, Network, PullMode, Pushed, Remote, check_remote_name, check_url, explain,
|
||||
parse_progress,
|
||||
};
|
||||
pub use repo::{
|
||||
Branch, Branches, CommitInfo, Contents, FILE_CAP, Git, MIN_VERSION, Repo, Rev, STATUS_CAP,
|
||||
decode,
|
||||
};
|
||||
pub use runner::{Cancel, ProgressSink, redact_urls};
|
||||
pub use watch::{RepoWatch, fingerprint};
|
||||
|
|
|
|||
719
crates/vcs/src/remote.rs
Normal file
719
crates/vcs/src/remote.rs
Normal file
|
|
@ -0,0 +1,719 @@
|
|||
//! Remotes (M4.10, owner decision D23): fetch, pull, push, clone and the
|
||||
//! list of configured remotes, through the user's git.
|
||||
//!
|
||||
//! - **Any server.** HTTPS and SSH remotes on any host (GitHub, GitLab,
|
||||
//! Forgejo, Bitbucket, a self-hosted server) and local paths work the
|
||||
//! same way, because git does the transport. `ext::` URLs are refused
|
||||
//! (`protocol.ext.allow=never`), so a URL never runs a command.
|
||||
//! - **Credentials stay with git.** HTTPS sign-in goes through git's
|
||||
//! credential helper, which may show its own window (Git Credential
|
||||
//! Manager); SSH uses the user's ssh, its agent and keys. Lutra never
|
||||
//! reads, stores or logs a credential or key, and never answers a
|
||||
//! prompt: terminal prompts are off ([`runner::Call::network`]), so a
|
||||
//! remote that needs a password nobody supplies fails with a message
|
||||
//! instead of hanging.
|
||||
//! - **Redaction.** Remote URLs ([`Remote`]) and every message have the
|
||||
//! userinfo of URLs replaced ([`runner::redact_urls`]).
|
||||
//! - **Clear messages.** A refusal keeps git's text and gets a short
|
||||
//! explanation first when its cause is known ([`explain`]): a rejected
|
||||
//! push, a sign-in that failed, an unknown SSH host key, an unreachable
|
||||
//! or missing repository, no upstream, conflicts after a pull.
|
||||
//! - **Progress and cancel.** Network calls run with `--progress`; each
|
||||
//! stderr line reaches the [`Network`] sink, and its cancel flag kills
|
||||
//! git and every process it started.
|
||||
|
||||
use std::path::Path;
|
||||
use std::time::Duration;
|
||||
|
||||
use crate::VcsError;
|
||||
use crate::repo::{Git, Repo};
|
||||
use crate::runner::{self, Call, Cancel, ProgressSink, redact_urls};
|
||||
|
||||
/// Longest URL accepted.
|
||||
pub const MAX_URL_LEN: usize = 4096;
|
||||
|
||||
/// Cap on config and push output read.
|
||||
const SMALL_CAP: usize = 1024 * 1024;
|
||||
|
||||
/// Global options of every network call: no `ext::` transport.
|
||||
const NETWORK_OPTIONS: [&str; 2] = ["-c", "protocol.ext.allow=never"];
|
||||
|
||||
/// Progress and cancel of one network call.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Network {
|
||||
/// Receives git's progress lines.
|
||||
pub progress: Option<ProgressSink>,
|
||||
/// Stops the call when set.
|
||||
pub cancel: Option<Cancel>,
|
||||
}
|
||||
|
||||
/// A configured remote.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Remote {
|
||||
/// Its name, e.g. `origin`.
|
||||
pub name: String,
|
||||
/// `remote.<name>.url`, userinfo redacted.
|
||||
pub fetch_url: Option<String>,
|
||||
/// `remote.<name>.pushurl`, userinfo redacted; `None` when pushes go
|
||||
/// to the fetch URL.
|
||||
pub push_url: Option<String>,
|
||||
}
|
||||
|
||||
/// How `git pull` joins the fetched commits.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PullMode {
|
||||
/// `git pull --no-rebase`: a merge (fast-forward when possible).
|
||||
Merge,
|
||||
/// `git pull --rebase`: local commits are replayed on top.
|
||||
Rebase,
|
||||
}
|
||||
|
||||
/// What [`Repo::push`] pushed.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Pushed {
|
||||
/// The remote pushed to.
|
||||
pub remote: String,
|
||||
/// The local branch pushed.
|
||||
pub branch: String,
|
||||
/// The branch had no upstream and got one with this push.
|
||||
pub upstream_set: bool,
|
||||
}
|
||||
|
||||
/// Checks a remote URL or path from the caller.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidInput`] when it is blank, longer than
|
||||
/// [`MAX_URL_LEN`], starts with `-` (an option), holds a control
|
||||
/// character, or uses the `ext::` transport.
|
||||
pub fn check_url(url: &str) -> Result<(), VcsError> {
|
||||
let invalid = |why: &str| VcsError::InvalidInput(format!("invalid remote URL: {why}"));
|
||||
if url.trim().is_empty() {
|
||||
return Err(invalid("it is empty"));
|
||||
}
|
||||
if url.len() > MAX_URL_LEN {
|
||||
return Err(invalid("it is too long"));
|
||||
}
|
||||
if url.starts_with('-') {
|
||||
return Err(invalid("it starts with '-'"));
|
||||
}
|
||||
if url.chars().any(char::is_control) {
|
||||
return Err(invalid("it holds a control character"));
|
||||
}
|
||||
if url.trim().to_ascii_lowercase().starts_with("ext::") {
|
||||
return Err(invalid("the ext:: transport is not allowed"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Checks a remote name from the caller; git checks the rest
|
||||
/// (`valid_remote_name`).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidName`] when it is empty, starts with `-`, or holds
|
||||
/// whitespace or a control character.
|
||||
pub fn check_remote_name(name: &str) -> Result<(), VcsError> {
|
||||
if name.is_empty()
|
||||
|| name.len() > 255
|
||||
|| name.starts_with('-')
|
||||
|| name.chars().any(|c| c.is_control() || c.is_whitespace())
|
||||
{
|
||||
return Err(VcsError::InvalidName(name.to_string()));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Known causes of a refusal, each with the text Lutra puts before git's
|
||||
/// message. Checked in order; the first that matches wins.
|
||||
const EXPLANATIONS: &[(&[&str], &str)] = &[
|
||||
(
|
||||
&[
|
||||
"[rejected]",
|
||||
"non-fast-forward",
|
||||
"fetch first",
|
||||
"Updates were rejected",
|
||||
],
|
||||
"The push was rejected: the remote branch has commits that your branch does not have. Pull first, then push again.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"[remote rejected]",
|
||||
"pre-receive hook declined",
|
||||
"protected branch",
|
||||
],
|
||||
"The server refused the push (for example a protected branch or a server-side hook).",
|
||||
),
|
||||
(
|
||||
&["Host key verification failed"],
|
||||
"The server's SSH host key is not known yet. Connect to it once from a terminal (for example ssh -T git@host) to check and accept the key, then try again.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"Authentication failed",
|
||||
"could not read Username",
|
||||
"could not read Password",
|
||||
"terminal prompts disabled",
|
||||
"Permission denied (publickey",
|
||||
"Invalid username or password",
|
||||
"returned error: 401",
|
||||
"returned error: 403",
|
||||
"The requested URL returned error: 403",
|
||||
],
|
||||
"Git could not sign in to the remote. Lutra leaves sign-in to git: set up its credential helper (for example sign in once with Git Credential Manager) or add your SSH key to the ssh agent, then try again.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"Repository not found",
|
||||
"does not appear to be a git repository",
|
||||
"returned error: 404",
|
||||
"not found",
|
||||
],
|
||||
"The remote repository was not found. Check the URL and that you have access to it.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"Could not resolve host",
|
||||
"Failed to connect",
|
||||
"Connection timed out",
|
||||
"Connection refused",
|
||||
"Network is unreachable",
|
||||
"Could not read from remote repository",
|
||||
],
|
||||
"The remote could not be reached. Check the URL and your network connection.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"There is no tracking information",
|
||||
"no upstream",
|
||||
"has no upstream branch",
|
||||
],
|
||||
"The current branch has no upstream branch. Push it once to set one.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"would be overwritten",
|
||||
"You have unstaged changes",
|
||||
"Please commit or stash them",
|
||||
"cannot pull with rebase",
|
||||
],
|
||||
"Local changes are in the way. Commit or stash them, then pull again.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"CONFLICT",
|
||||
"Automatic merge failed",
|
||||
"could not apply",
|
||||
"Resolve all conflicts",
|
||||
],
|
||||
"The pull stopped with conflicts. Resolve them, then commit the merge or continue the rebase.",
|
||||
),
|
||||
(
|
||||
&[
|
||||
"Not possible to fast-forward",
|
||||
"Need to specify how to reconcile divergent branches",
|
||||
],
|
||||
"Your branch and the remote branch have diverged. Choose merge or rebase for pulls in Settings › Git, then pull again.",
|
||||
),
|
||||
(
|
||||
&["already exists and is not an empty directory"],
|
||||
"The target folder exists and is not empty. Choose an empty or new folder.",
|
||||
),
|
||||
];
|
||||
|
||||
/// git's message with a short explanation first when its cause is known
|
||||
/// (see [`EXPLANATIONS`]); URLs are redacted.
|
||||
pub fn explain(message: &str) -> String {
|
||||
let message = redact_urls(message.trim());
|
||||
let lower = message.to_lowercase();
|
||||
let found = EXPLANATIONS.iter().find(|(needles, _)| {
|
||||
needles
|
||||
.iter()
|
||||
.any(|needle| lower.contains(&needle.to_lowercase()))
|
||||
});
|
||||
match found {
|
||||
Some((_, why)) if message.is_empty() => (*why).to_string(),
|
||||
Some((_, why)) => format!("{why}\n\n{message}"),
|
||||
None => message,
|
||||
}
|
||||
}
|
||||
|
||||
/// A progress line split into its phase and percentage: `Receiving
|
||||
/// objects: 45% (450/1000), 1.2 MiB` → (`Receiving objects`, 45). A
|
||||
/// `remote: ` prefix is dropped. Lines without a percentage give the
|
||||
/// whole line and `None`.
|
||||
pub fn parse_progress(line: &str) -> (String, Option<u8>) {
|
||||
let line = line.trim();
|
||||
let line = line.strip_prefix("remote:").map_or(line, str::trim);
|
||||
if let Some((phase, rest)) = line.split_once(':')
|
||||
&& let Some((number, _)) = rest.trim_start().split_once('%')
|
||||
&& let Ok(percent) = number.trim().parse::<u8>()
|
||||
{
|
||||
return (phase.trim().to_string(), Some(percent.min(100)));
|
||||
}
|
||||
(line.to_string(), None)
|
||||
}
|
||||
|
||||
/// Turns a failed network call's error into one with [`explain`]'s text.
|
||||
fn explained(error: VcsError) -> VcsError {
|
||||
match error {
|
||||
VcsError::Git(message) => VcsError::Git(explain(&message)),
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
/// A network call in `cwd` with [`NETWORK_OPTIONS`] first and the
|
||||
/// progress sink and cancel flag of `net`.
|
||||
fn network_call<'a>(cwd: &'a Path, args: &[&str], net: &Network) -> Call<'a> {
|
||||
let mut all: Vec<&str> = NETWORK_OPTIONS.to_vec();
|
||||
all.extend_from_slice(args);
|
||||
Call::network(cwd, &all)
|
||||
.progress(net.progress.clone())
|
||||
.cancel(net.cancel.clone())
|
||||
}
|
||||
|
||||
/// Checks a network call's output, with explained errors.
|
||||
fn checked(output: Result<runner::Output, VcsError>) -> Result<runner::Output, VcsError> {
|
||||
output?.checked().map_err(explained)
|
||||
}
|
||||
|
||||
impl Git {
|
||||
/// Clones `url` into `dir` (`git clone --progress`), with progress and
|
||||
/// cancel.
|
||||
///
|
||||
/// # Parameters
|
||||
/// - `url`: any URL or path git can clone from; checked with
|
||||
/// [`check_url`].
|
||||
/// - `dir`: the new work tree; absolute, its parent must exist, and it
|
||||
/// must be missing or an empty folder.
|
||||
///
|
||||
/// When the clone fails or is cancelled, what it left in `dir` is
|
||||
/// removed again (the folder itself too, if the clone created it).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidInput`] for a bad URL or folder,
|
||||
/// [`VcsError::Cancelled`], or git's refusal with [`explain`]'s text.
|
||||
pub fn clone_repo(&self, url: &str, dir: &Path, net: &Network) -> Result<(), VcsError> {
|
||||
check_url(url)?;
|
||||
let invalid = |why: &str| VcsError::InvalidInput(why.to_string());
|
||||
if !dir.is_absolute() {
|
||||
return Err(invalid("the target folder must be an absolute path"));
|
||||
}
|
||||
let Some(parent) = dir.parent().filter(|p| p.is_dir()) else {
|
||||
return Err(invalid("the parent of the target folder does not exist"));
|
||||
};
|
||||
let existed = dir.exists();
|
||||
if existed {
|
||||
let empty = std::fs::read_dir(dir)
|
||||
.map(|mut entries| entries.next().is_none())
|
||||
.unwrap_or(false);
|
||||
if !dir.is_dir() || !empty {
|
||||
return Err(invalid(
|
||||
"the target folder exists and is not empty; choose an empty or new folder",
|
||||
));
|
||||
}
|
||||
}
|
||||
let Some(target) = dir.to_str() else {
|
||||
return Err(invalid("the target folder's path is not valid Unicode"));
|
||||
};
|
||||
let result = checked(runner::run(
|
||||
self.executable(),
|
||||
network_call(
|
||||
parent,
|
||||
&["clone", "--progress", "--end-of-options", url, target],
|
||||
net,
|
||||
),
|
||||
));
|
||||
if result.is_err() {
|
||||
clean_up(dir, existed);
|
||||
}
|
||||
result.map(|_| ())
|
||||
}
|
||||
}
|
||||
|
||||
/// Removes what a failed clone left: the folder, or only its contents when
|
||||
/// it existed (empty) before. Failures are ignored after a few tries: the
|
||||
/// clone's error is what the caller reports, and a leftover folder is
|
||||
/// visible to the user.
|
||||
fn clean_up(dir: &Path, existed: bool) {
|
||||
for _ in 0..5 {
|
||||
let removed = if existed {
|
||||
std::fs::read_dir(dir).is_ok_and(|entries| {
|
||||
entries.flatten().all(|entry| {
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
std::fs::remove_dir_all(&path).is_ok()
|
||||
} else {
|
||||
std::fs::remove_file(&path).is_ok()
|
||||
}
|
||||
})
|
||||
})
|
||||
} else {
|
||||
std::fs::remove_dir_all(dir).is_ok() || !dir.exists()
|
||||
};
|
||||
if removed {
|
||||
return;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(100));
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses `git config -z --get-regexp '^remote\..*\.(url|pushurl)$'`
|
||||
/// output into remotes, in the order they are configured; the last value
|
||||
/// of a key wins, as in git. URLs are redacted.
|
||||
pub fn parse_remotes(bytes: &[u8]) -> Vec<Remote> {
|
||||
let mut remotes: Vec<Remote> = Vec::new();
|
||||
for entry in bytes.split(|b| *b == 0) {
|
||||
let text = String::from_utf8_lossy(entry);
|
||||
let Some((key, value)) = text.split_once('\n') else {
|
||||
continue;
|
||||
};
|
||||
let Some(rest) = key.strip_prefix("remote.") else {
|
||||
continue;
|
||||
};
|
||||
let (name, push) = if let Some(name) = rest.strip_suffix(".pushurl") {
|
||||
(name, true)
|
||||
} else if let Some(name) = rest.strip_suffix(".url") {
|
||||
(name, false)
|
||||
} else {
|
||||
continue;
|
||||
};
|
||||
if name.is_empty() || value.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let index = match remotes.iter().position(|r| r.name == name) {
|
||||
Some(index) => index,
|
||||
None => {
|
||||
remotes.push(Remote {
|
||||
name: name.to_string(),
|
||||
fetch_url: None,
|
||||
push_url: None,
|
||||
});
|
||||
remotes.len() - 1
|
||||
}
|
||||
};
|
||||
let Some(remote) = remotes.get_mut(index) else {
|
||||
continue;
|
||||
};
|
||||
let value = Some(redact_urls(value));
|
||||
if push {
|
||||
remote.push_url = value;
|
||||
} else {
|
||||
remote.fetch_url = value;
|
||||
}
|
||||
}
|
||||
remotes
|
||||
}
|
||||
|
||||
/// The summaries of the refs `git push --porcelain` reports as rejected
|
||||
/// (lines starting with `!`), e.g. `[rejected] (fetch first)`.
|
||||
pub fn push_rejections(stdout: &[u8]) -> Vec<String> {
|
||||
String::from_utf8_lossy(stdout)
|
||||
.lines()
|
||||
.filter_map(|line| {
|
||||
let rest = line.strip_prefix('!')?;
|
||||
let summary = rest.rsplit('\t').next().unwrap_or(rest).trim();
|
||||
Some(summary.to_string())
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
impl Repo {
|
||||
/// Runs a read in the work tree root.
|
||||
fn read(&self, args: &[&str]) -> Result<runner::Output, VcsError> {
|
||||
self.run(Call::read(self.root(), args, SMALL_CAP))
|
||||
}
|
||||
|
||||
/// One config value; `None` when it is not set.
|
||||
fn config(&self, key: &str) -> Result<Option<String>, VcsError> {
|
||||
let output = self.read(&["config", "--get", "--end-of-options", key])?;
|
||||
if !output.success() {
|
||||
return Ok(None);
|
||||
}
|
||||
let value = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
Ok((!value.is_empty()).then_some(value))
|
||||
}
|
||||
|
||||
/// The configured remotes, in config order, URLs redacted.
|
||||
///
|
||||
/// # Errors
|
||||
/// The runner's errors; no remote is an empty list.
|
||||
pub fn remotes(&self) -> Result<Vec<Remote>, VcsError> {
|
||||
let output = self.read(&[
|
||||
"config",
|
||||
"-z",
|
||||
"--get-regexp",
|
||||
r"^remote\..*\.(url|pushurl)$",
|
||||
])?;
|
||||
if !output.success() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
Ok(parse_remotes(&output.stdout))
|
||||
}
|
||||
|
||||
/// Runs a remote-management call (`git remote …`), serialised with
|
||||
/// the other changes.
|
||||
fn manage(&self, args: &[&str]) -> Result<(), VcsError> {
|
||||
let _guard = self.mutation_guard();
|
||||
self.run(Call::write(self.root(), args))?.checked()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Adds a remote (`git remote add`), without fetching.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidName`], [`VcsError::InvalidInput`], or git's
|
||||
/// refusal (the remote exists, the name is not valid).
|
||||
pub fn remote_add(&self, name: &str, url: &str) -> Result<(), VcsError> {
|
||||
check_remote_name(name)?;
|
||||
check_url(url)?;
|
||||
self.manage(&["remote", "add", "--end-of-options", name, url])
|
||||
}
|
||||
|
||||
/// Changes a remote's URL (`git remote set-url`).
|
||||
///
|
||||
/// # Errors
|
||||
/// As [`Repo::remote_add`]; an unknown remote is git's refusal.
|
||||
pub fn remote_set_url(&self, name: &str, url: &str) -> Result<(), VcsError> {
|
||||
check_remote_name(name)?;
|
||||
check_url(url)?;
|
||||
self.manage(&["remote", "set-url", "--end-of-options", name, url])
|
||||
}
|
||||
|
||||
/// Removes a remote with its remote-tracking branches and the
|
||||
/// upstream settings that named it (`git remote remove`).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidName`], or git's refusal (no such remote).
|
||||
pub fn remote_remove(&self, name: &str) -> Result<(), VcsError> {
|
||||
check_remote_name(name)?;
|
||||
self.manage(&["remote", "remove", "--end-of-options", name])
|
||||
}
|
||||
|
||||
/// Fetches one remote, or every remote when `remote` is `None`
|
||||
/// (`git fetch --all`).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidName`], [`VcsError::Cancelled`], or git's
|
||||
/// refusal with [`explain`]'s text.
|
||||
pub fn fetch(&self, remote: Option<&str>, net: &Network) -> Result<(), VcsError> {
|
||||
let args: Vec<&str> = match remote {
|
||||
Some(name) => {
|
||||
check_remote_name(name)?;
|
||||
vec!["fetch", "--progress", "--end-of-options", name]
|
||||
}
|
||||
None => vec!["fetch", "--progress", "--all"],
|
||||
};
|
||||
let _guard = self.mutation_guard();
|
||||
checked(self.run(network_call(self.root(), &args, net))).map(|_| ())
|
||||
}
|
||||
|
||||
/// Pulls the current branch's upstream. A merge takes git's default
|
||||
/// message without an editor (`--no-edit`).
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::Cancelled`], or git's refusal with [`explain`]'s text
|
||||
/// (no upstream, local changes in the way, conflicts).
|
||||
pub fn pull(&self, mode: PullMode, net: &Network) -> Result<(), VcsError> {
|
||||
let args: &[&str] = match mode {
|
||||
PullMode::Merge => &["pull", "--progress", "--no-rebase", "--no-edit"],
|
||||
PullMode::Rebase => &["pull", "--progress", "--rebase"],
|
||||
};
|
||||
let _guard = self.mutation_guard();
|
||||
checked(self.run(network_call(self.root(), args, net))).map(|_| ())
|
||||
}
|
||||
|
||||
/// Pushes the current branch. With an upstream it goes there
|
||||
/// (`branch.<name>.remote` and `.merge`); without one it goes to a
|
||||
/// branch of the same name on `remote` and becomes the upstream
|
||||
/// (`--set-upstream`). Without `remote` that is the only remote, else
|
||||
/// `origin`. Hooks (`pre-push`) run.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::InvalidName`]; [`VcsError::Git`] when HEAD is detached,
|
||||
/// no remote fits, or git or the server refuses (with [`explain`]'s
|
||||
/// text, also when git's exit status hides a rejected ref);
|
||||
/// [`VcsError::Cancelled`].
|
||||
pub fn push(&self, remote: Option<&str>, net: &Network) -> Result<Pushed, VcsError> {
|
||||
if let Some(name) = remote {
|
||||
check_remote_name(name)?;
|
||||
}
|
||||
let head = self.head()?;
|
||||
let Some(branch) = head.branch.filter(|_| !head.detached) else {
|
||||
return Err(VcsError::Git(
|
||||
"HEAD is detached; check out a branch to push it.".into(),
|
||||
));
|
||||
};
|
||||
let upstream_remote = self
|
||||
.config(&format!("branch.{branch}.remote"))?
|
||||
.filter(|r| r != ".");
|
||||
let merge = self.config(&format!("branch.{branch}.merge"))?;
|
||||
let local_ref = format!("refs/heads/{branch}");
|
||||
let (target, refspec, set_upstream) = match (upstream_remote, merge) {
|
||||
(Some(up), Some(merge)) if remote.is_none_or(|r| r == up) => {
|
||||
(up, format!("{local_ref}:{merge}"), false)
|
||||
}
|
||||
_ => {
|
||||
let target = match remote {
|
||||
Some(name) => name.to_string(),
|
||||
None => self.default_remote()?,
|
||||
};
|
||||
(target, format!("{local_ref}:{local_ref}"), true)
|
||||
}
|
||||
};
|
||||
let mut args = vec!["push", "--progress", "--porcelain"];
|
||||
if set_upstream {
|
||||
args.push("--set-upstream");
|
||||
}
|
||||
args.extend(["--end-of-options", &target, &refspec]);
|
||||
let _guard = self.mutation_guard();
|
||||
let output = self.run(network_call(self.root(), &args, net))?;
|
||||
let rejected = push_rejections(&output.stdout);
|
||||
if !output.success() || !rejected.is_empty() {
|
||||
let mut message = output.message();
|
||||
for line in rejected {
|
||||
if !message.contains(&line) {
|
||||
message.push('\n');
|
||||
message.push_str(&line);
|
||||
}
|
||||
}
|
||||
return Err(VcsError::Git(explain(&message)));
|
||||
}
|
||||
Ok(Pushed {
|
||||
remote: target,
|
||||
branch,
|
||||
upstream_set: set_upstream,
|
||||
})
|
||||
}
|
||||
|
||||
/// The remote a branch without upstream is pushed to: the only remote,
|
||||
/// else `origin`.
|
||||
fn default_remote(&self) -> Result<String, VcsError> {
|
||||
let remotes = self.remotes()?;
|
||||
match remotes.as_slice() {
|
||||
[only] => Ok(only.name.clone()),
|
||||
[] => Err(VcsError::Git(
|
||||
"This repository has no remote to push to. Add one in Git › Manage Remotes…".into(),
|
||||
)),
|
||||
many if many.iter().any(|r| r.name == "origin") => Ok("origin".into()),
|
||||
_ => Err(VcsError::Git(
|
||||
"This repository has several remotes and none is named origin; choose the remote to push to.".into(),
|
||||
)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used)]
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn urls_are_checked() {
|
||||
assert!(check_url("https://github.com/a/b.git").is_ok());
|
||||
assert!(check_url("git@github.com:a/b.git").is_ok());
|
||||
assert!(check_url("C:\\repos\\x.git").is_ok());
|
||||
assert!(check_url("").is_err());
|
||||
assert!(check_url(" ").is_err());
|
||||
assert!(check_url("--upload-pack=evil").is_err());
|
||||
assert!(check_url("https://a\nb").is_err());
|
||||
assert!(check_url("ext::sh -c touch% /tmp/x").is_err());
|
||||
assert!(check_url(&"a".repeat(MAX_URL_LEN + 1)).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remote_names_are_checked() {
|
||||
assert!(check_remote_name("origin").is_ok());
|
||||
assert!(check_remote_name("my.fork/x").is_ok());
|
||||
assert!(check_remote_name("").is_err());
|
||||
assert!(check_remote_name("-x").is_err());
|
||||
assert!(check_remote_name("a b").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn explanations_come_before_git_message() {
|
||||
let text = explain(
|
||||
" ! [rejected] main -> main (fetch first)\nerror: failed to push some refs to 'https://u:p@h/r'",
|
||||
);
|
||||
assert!(text.starts_with("The push was rejected"));
|
||||
assert!(text.contains("https://***@h/r"));
|
||||
assert!(!text.contains("u:p"));
|
||||
assert!(
|
||||
explain(
|
||||
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
|
||||
)
|
||||
.starts_with("Git could not sign in")
|
||||
);
|
||||
assert!(
|
||||
explain("Host key verification failed.\nfatal: Could not read from remote repository.")
|
||||
.starts_with("The server's SSH host key")
|
||||
);
|
||||
assert!(
|
||||
explain("fatal: '/x' does not appear to be a git repository")
|
||||
.starts_with("The remote repository was not found")
|
||||
);
|
||||
assert!(
|
||||
explain("There is no tracking information for the current branch.")
|
||||
.starts_with("The current branch has no upstream")
|
||||
);
|
||||
assert_eq!(explain("something else"), "something else");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn progress_lines_give_phase_and_percent() {
|
||||
assert_eq!(
|
||||
parse_progress("Receiving objects: 45% (450/1000), 1.20 MiB | 2.00 MiB/s"),
|
||||
("Receiving objects".into(), Some(45))
|
||||
);
|
||||
assert_eq!(
|
||||
parse_progress("remote: Counting objects: 100% (3/3), done."),
|
||||
("Counting objects".into(), Some(100))
|
||||
);
|
||||
assert_eq!(
|
||||
parse_progress("Cloning into 'x'..."),
|
||||
("Cloning into 'x'...".into(), None)
|
||||
);
|
||||
assert_eq!(parse_progress("a: 999% b"), ("a: 999% b".into(), None));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remotes_parse_in_order_with_redacted_urls() {
|
||||
let bytes = b"remote.origin.url\nhttps://me:tok@github.com/a/b.git\0remote.my.fork.url\ngit@h:x.git\0remote.origin.pushurl\nssh://git@h/b\0remote.bad.fetch\nx\0junk\0";
|
||||
let remotes = parse_remotes(bytes);
|
||||
assert_eq!(
|
||||
remotes,
|
||||
[
|
||||
Remote {
|
||||
name: "origin".into(),
|
||||
fetch_url: Some("https://***@github.com/a/b.git".into()),
|
||||
push_url: Some("ssh://***@h/b".into()),
|
||||
},
|
||||
Remote {
|
||||
name: "my.fork".into(),
|
||||
fetch_url: Some("git@h:x.git".into()),
|
||||
push_url: None,
|
||||
},
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn push_rejections_are_read_from_porcelain() {
|
||||
let out =
|
||||
b"To /tmp/r.git\n!\trefs/heads/main:refs/heads/main\t[rejected] (fetch first)\nDone\n";
|
||||
assert_eq!(push_rejections(out), ["[rejected] (fetch first)"]);
|
||||
assert!(
|
||||
push_rejections(b"To x\n=\trefs/heads/a:refs/heads/a\t[up to date]\nDone\n").is_empty()
|
||||
);
|
||||
}
|
||||
|
||||
proptest::proptest! {
|
||||
#[test]
|
||||
fn parsers_take_any_bytes(bytes in proptest::collection::vec(proptest::num::u8::ANY, 0..300)) {
|
||||
let _ = parse_remotes(&bytes);
|
||||
let _ = push_rejections(&bytes);
|
||||
let text = String::from_utf8_lossy(&bytes);
|
||||
let _ = parse_progress(&text);
|
||||
let _ = explain(&text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -82,6 +82,11 @@ impl Git {
|
|||
self.version
|
||||
}
|
||||
|
||||
/// The program that runs (for the remote calls in `remote.rs`).
|
||||
pub(crate) fn executable(&self) -> &Path {
|
||||
&self.executable
|
||||
}
|
||||
|
||||
/// Opens the repository that contains `dir` (a nested repository wins
|
||||
/// over a parent one, as git decides).
|
||||
///
|
||||
|
|
@ -231,6 +236,12 @@ impl Repo {
|
|||
&self.prefix
|
||||
}
|
||||
|
||||
/// Serialises a change made outside this file (`remote.rs`) with the
|
||||
/// changes made here.
|
||||
pub(crate) fn mutation_guard(&self) -> MutexGuard<'_, ()> {
|
||||
self.lock()
|
||||
}
|
||||
|
||||
/// Serialises changes: the guard of [`Repo::mutations`]. A poisoned
|
||||
/// lock is taken over: the lock guards no data.
|
||||
fn lock(&self) -> MutexGuard<'_, ()> {
|
||||
|
|
|
|||
|
|
@ -4,11 +4,21 @@
|
|||
//! Every call goes through [`run`], and every process starts in [`spawn`],
|
||||
//! which puts git (and the hooks it runs) into a kill-on-close group
|
||||
//! (`lutra-proc`), so no git outlives Lutra.
|
||||
//!
|
||||
//! Network calls (M4.10, owner decision D23) are built with
|
||||
//! [`Call::network`]: terminal prompts stay off, but the credential
|
||||
//! helper may show its own window (Git Credential Manager), the timeout is
|
||||
//! [`NETWORK_TIMEOUT`], stderr progress lines reach a [`ProgressSink`],
|
||||
//! and a [`Cancel`] flag stops git. Every message that leaves the runner
|
||||
//! has the userinfo of URLs replaced ([`redact_urls`]), so a token in a
|
||||
//! remote URL never reaches an error or a log.
|
||||
|
||||
use std::ffi::OsString;
|
||||
use std::io::{Read, Write};
|
||||
use std::path::Path;
|
||||
use std::process::{Command, ExitStatus, Stdio};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
use lutra_proc::Child;
|
||||
use std::sync::mpsc;
|
||||
|
|
@ -23,9 +33,54 @@ pub const TIMEOUT: Duration = Duration::from_secs(30);
|
|||
/// Timeout of calls that run the user's hooks (commit, checkout).
|
||||
pub const HOOK_TIMEOUT: Duration = Duration::from_secs(120);
|
||||
|
||||
/// Timeout of network calls (fetch, pull, push, clone); [`Cancel`] stops
|
||||
/// them sooner.
|
||||
pub const NETWORK_TIMEOUT: Duration = Duration::from_secs(30 * 60);
|
||||
|
||||
/// Cap on stderr kept for error messages; the rest is read and dropped.
|
||||
pub const STDERR_CAP: usize = 16 * 1024;
|
||||
|
||||
/// Longest progress line passed to a [`ProgressSink`]; longer ones are cut.
|
||||
const PROGRESS_LINE_CAP: usize = 1024;
|
||||
|
||||
/// How often a waiting call checks its [`Cancel`] flag.
|
||||
const CANCEL_POLL: Duration = Duration::from_millis(50);
|
||||
|
||||
/// Receives git's progress lines (stderr split at `\r` and `\n`, blank
|
||||
/// lines dropped, URLs redacted, at most 1 KiB each), on the thread that
|
||||
/// reads stderr.
|
||||
#[derive(Clone)]
|
||||
pub struct ProgressSink(pub Arc<dyn Fn(&str) + Send + Sync>);
|
||||
|
||||
impl std::fmt::Debug for ProgressSink {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("ProgressSink")
|
||||
}
|
||||
}
|
||||
|
||||
/// A flag that stops a running call: once it is set, git (with every
|
||||
/// process it started) is killed within [`CANCEL_POLL`] and the call ends
|
||||
/// with [`VcsError::Cancelled`].
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Cancel(Arc<AtomicBool>);
|
||||
|
||||
impl Cancel {
|
||||
/// A flag that is not set.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Sets the flag.
|
||||
pub fn cancel(&self) {
|
||||
self.0.store(true, Ordering::SeqCst);
|
||||
}
|
||||
|
||||
/// Whether the flag is set.
|
||||
pub fn is_cancelled(&self) -> bool {
|
||||
self.0.load(Ordering::SeqCst)
|
||||
}
|
||||
}
|
||||
|
||||
/// Environment variables that would point git at another repository than
|
||||
/// the one Lutra asked for, removed from every call.
|
||||
const SCRUBBED_ENV: &[&str] = &[
|
||||
|
|
@ -57,6 +112,14 @@ pub struct Call<'a> {
|
|||
/// A read that must not take optional locks (`GIT_OPTIONAL_LOCKS=0`),
|
||||
/// so it never competes with the user's own git for `index.lock`.
|
||||
pub read_only: bool,
|
||||
/// A network call: the credential helper may show its own window
|
||||
/// (`GCM_INTERACTIVE` is left as the user has it); terminal prompts
|
||||
/// stay off.
|
||||
pub network: bool,
|
||||
/// Receives stderr progress lines while git runs.
|
||||
pub progress: Option<ProgressSink>,
|
||||
/// Stops the call when set.
|
||||
pub cancel: Option<Cancel>,
|
||||
}
|
||||
|
||||
impl<'a> Call<'a> {
|
||||
|
|
@ -69,6 +132,9 @@ impl<'a> Call<'a> {
|
|||
timeout: TIMEOUT,
|
||||
stdout_cap,
|
||||
read_only: true,
|
||||
network: false,
|
||||
progress: None,
|
||||
cancel: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -81,6 +147,30 @@ impl<'a> Call<'a> {
|
|||
}
|
||||
}
|
||||
|
||||
/// A network call (fetch, pull, push, clone; D23): it may change the
|
||||
/// repository, has [`NETWORK_TIMEOUT`] and a 1 MiB output cap, and its
|
||||
/// credential helper may show its own window. Terminal prompts stay
|
||||
/// off (`GIT_TERMINAL_PROMPT=0`, no console, stdin closed).
|
||||
pub fn network(cwd: &'a Path, args: &[&str]) -> Self {
|
||||
Self {
|
||||
network: true,
|
||||
timeout: NETWORK_TIMEOUT,
|
||||
..Self::write(cwd, args)
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets the progress sink.
|
||||
pub fn progress(mut self, sink: Option<ProgressSink>) -> Self {
|
||||
self.progress = sink;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the cancel flag.
|
||||
pub fn cancel(mut self, cancel: Option<Cancel>) -> Self {
|
||||
self.cancel = cancel;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets stdin.
|
||||
pub fn stdin(mut self, bytes: Vec<u8>) -> Self {
|
||||
self.stdin = Some(bytes);
|
||||
|
|
@ -112,15 +202,16 @@ impl Output {
|
|||
}
|
||||
|
||||
/// git's message for a failure: stderr, or stdout when stderr is
|
||||
/// empty (hooks often print to stdout), trimmed and capped at
|
||||
/// [`STDERR_CAP`] bytes on a character boundary.
|
||||
/// empty (hooks often print to stdout), trimmed, with URL userinfo
|
||||
/// redacted ([`redact_urls`]) and capped at [`STDERR_CAP`] bytes on a
|
||||
/// character boundary.
|
||||
pub fn message(&self) -> String {
|
||||
let source = if self.stderr.iter().all(u8::is_ascii_whitespace) {
|
||||
&self.stdout
|
||||
} else {
|
||||
&self.stderr
|
||||
};
|
||||
let text = String::from_utf8_lossy(source);
|
||||
let text = redact_urls(&String::from_utf8_lossy(source));
|
||||
let text = text.trim();
|
||||
let mut end = text.len().min(STDERR_CAP);
|
||||
while !text.is_char_boundary(end) {
|
||||
|
|
@ -164,12 +255,12 @@ pub fn spawn(command: Command) -> std::io::Result<Child> {
|
|||
struct Reaper(Option<Child>);
|
||||
|
||||
impl Reaper {
|
||||
/// Waits until the child exits or `deadline` passes.
|
||||
/// Waits until the child exits, `deadline` passes or `cancel` is set.
|
||||
///
|
||||
/// # Returns
|
||||
/// The exit status, or `None` at the deadline (the child is killed
|
||||
/// when the reaper drops).
|
||||
fn wait_until(&mut self, deadline: Instant) -> Option<ExitStatus> {
|
||||
/// The exit status, or `None` at the deadline or on cancel (the child
|
||||
/// is killed when the reaper drops).
|
||||
fn wait_until(&mut self, deadline: Instant, cancel: Option<&Cancel>) -> Option<ExitStatus> {
|
||||
let child = self.0.as_mut()?;
|
||||
let mut pause = Duration::from_millis(1);
|
||||
loop {
|
||||
|
|
@ -182,7 +273,7 @@ impl Reaper {
|
|||
Err(_) => return None,
|
||||
}
|
||||
let now = Instant::now();
|
||||
if now >= deadline {
|
||||
if now >= deadline || cancel.is_some_and(Cancel::is_cancelled) {
|
||||
return None;
|
||||
}
|
||||
thread::sleep(pause.min(deadline - now));
|
||||
|
|
@ -203,25 +294,70 @@ impl Drop for Reaper {
|
|||
}
|
||||
}
|
||||
|
||||
/// Splits progress output into lines at `\r` and `\n` and passes each
|
||||
/// non-blank one to the sink; the unfinished tail waits for the next
|
||||
/// chunk (at most [`PROGRESS_LINE_CAP`] bytes of it are kept).
|
||||
struct LineSplitter {
|
||||
/// Receives the lines.
|
||||
sink: ProgressSink,
|
||||
/// The start of the current line.
|
||||
pending: Vec<u8>,
|
||||
}
|
||||
|
||||
impl LineSplitter {
|
||||
/// Feeds bytes; `end` passes the tail on as a last line.
|
||||
fn feed(&mut self, bytes: &[u8], end: bool) {
|
||||
for &byte in bytes {
|
||||
if byte == b'\r' || byte == b'\n' {
|
||||
self.flush();
|
||||
} else if self.pending.len() < PROGRESS_LINE_CAP {
|
||||
self.pending.push(byte);
|
||||
}
|
||||
}
|
||||
if end {
|
||||
self.flush();
|
||||
}
|
||||
}
|
||||
|
||||
/// Passes the current line on, unless it is blank.
|
||||
fn flush(&mut self) {
|
||||
let line = redact_urls(&String::from_utf8_lossy(&self.pending));
|
||||
self.pending.clear();
|
||||
let line = line.trim();
|
||||
if !line.is_empty() {
|
||||
(self.sink.0)(line);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads `reader` to its end on a new thread, keeping at most `cap` bytes.
|
||||
///
|
||||
/// # Parameters
|
||||
/// - `fail_over_cap`: stop at once and report `Err(())` when more than
|
||||
/// `cap` bytes arrive (stdout), instead of reading the rest and
|
||||
/// dropping it (stderr).
|
||||
/// - `sink`: also receives every line read (stderr progress).
|
||||
fn collect(
|
||||
mut reader: impl Read + Send + 'static,
|
||||
cap: usize,
|
||||
fail_over_cap: bool,
|
||||
sink: Option<ProgressSink>,
|
||||
) -> mpsc::Receiver<Result<Vec<u8>, ()>> {
|
||||
let (tx, rx) = mpsc::channel();
|
||||
thread::spawn(move || {
|
||||
let mut kept = Vec::new();
|
||||
let mut chunk = [0u8; 64 * 1024];
|
||||
let mut lines = sink.map(|sink| LineSplitter {
|
||||
sink,
|
||||
pending: Vec::new(),
|
||||
});
|
||||
let result = loop {
|
||||
match reader.read(&mut chunk) {
|
||||
Ok(0) => break Ok(kept),
|
||||
Ok(n) => {
|
||||
if let Some(lines) = lines.as_mut() {
|
||||
lines.feed(chunk.get(..n).unwrap_or_default(), false);
|
||||
}
|
||||
let room = cap.saturating_sub(kept.len());
|
||||
if n > room && fail_over_cap {
|
||||
break Err(());
|
||||
|
|
@ -232,6 +368,9 @@ fn collect(
|
|||
Err(_) => break Ok(kept),
|
||||
}
|
||||
};
|
||||
if let Some(lines) = lines.as_mut() {
|
||||
lines.feed(&[], true);
|
||||
}
|
||||
let _ = tx.send(result);
|
||||
});
|
||||
rx
|
||||
|
|
@ -246,13 +385,15 @@ fn collect(
|
|||
/// Global options on every call: `--no-pager`, `--literal-pathspecs`
|
||||
/// (paths are never patterns), `-c core.quotepath=false`,
|
||||
/// `-c color.ui=false`. Environment: `GIT_TERMINAL_PROMPT=0`,
|
||||
/// `GCM_INTERACTIVE=never`, `GIT_OPTIONAL_LOCKS=0` for reads, and the
|
||||
/// variables that redirect git to another repository removed.
|
||||
/// `GCM_INTERACTIVE=never` (except for network calls, whose credential
|
||||
/// helper may show its own window), `GIT_OPTIONAL_LOCKS=0` for reads, and
|
||||
/// the variables that redirect git to another repository removed.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`VcsError::Spawn`], [`VcsError::Timeout`] (git is killed),
|
||||
/// [`VcsError::TooLarge`] (git is killed). A non-zero exit status is not an
|
||||
/// error here; see [`Output::checked`].
|
||||
/// [`VcsError::TooLarge`] (git is killed), [`VcsError::Cancelled`] (git
|
||||
/// is killed). A non-zero exit status is not an error here; see
|
||||
/// [`Output::checked`].
|
||||
pub fn run(executable: &Path, call: Call<'_>) -> Result<Output, VcsError> {
|
||||
let mut command = Command::new(executable);
|
||||
command
|
||||
|
|
@ -267,7 +408,6 @@ pub fn run(executable: &Path, call: Call<'_>) -> Result<Output, VcsError> {
|
|||
.args(&call.args)
|
||||
.current_dir(call.cwd)
|
||||
.env("GIT_TERMINAL_PROMPT", "0")
|
||||
.env("GCM_INTERACTIVE", "never")
|
||||
.stdin(if call.stdin.is_some() {
|
||||
Stdio::piped()
|
||||
} else {
|
||||
|
|
@ -278,6 +418,9 @@ pub fn run(executable: &Path, call: Call<'_>) -> Result<Output, VcsError> {
|
|||
for name in SCRUBBED_ENV {
|
||||
command.env_remove(name);
|
||||
}
|
||||
if !call.network {
|
||||
command.env("GCM_INTERACTIVE", "never");
|
||||
}
|
||||
if call.read_only {
|
||||
command.env("GIT_OPTIONAL_LOCKS", "0");
|
||||
}
|
||||
|
|
@ -296,22 +439,41 @@ pub fn run(executable: &Path, call: Call<'_>) -> Result<Output, VcsError> {
|
|||
let (Some(stdout), Some(stderr)) = (stdout, stderr) else {
|
||||
return Err(VcsError::Spawn("no pipes".into()));
|
||||
};
|
||||
let out_rx = collect(stdout, call.stdout_cap, true);
|
||||
let err_rx = collect(stderr, STDERR_CAP, false);
|
||||
let out_rx = collect(stdout, call.stdout_cap, true, None);
|
||||
let err_rx = collect(stderr, STDERR_CAP, false, call.progress);
|
||||
let cancel = call.cancel;
|
||||
let cancelled = || cancel.as_ref().is_some_and(Cancel::is_cancelled);
|
||||
let timeout = || VcsError::Timeout {
|
||||
seconds: call.timeout.as_secs(),
|
||||
};
|
||||
let remaining = || deadline.saturating_duration_since(Instant::now());
|
||||
let stdout = match out_rx.recv_timeout(remaining()) {
|
||||
Ok(Ok(bytes)) => bytes,
|
||||
Ok(Err(())) => {
|
||||
return Err(VcsError::TooLarge {
|
||||
limit: call.stdout_cap,
|
||||
});
|
||||
let stdout = loop {
|
||||
if cancelled() {
|
||||
return Err(VcsError::Cancelled);
|
||||
}
|
||||
let slice = if cancel.is_some() {
|
||||
remaining().min(CANCEL_POLL)
|
||||
} else {
|
||||
remaining()
|
||||
};
|
||||
match out_rx.recv_timeout(slice) {
|
||||
Ok(Ok(bytes)) => break bytes,
|
||||
Ok(Err(())) => {
|
||||
return Err(VcsError::TooLarge {
|
||||
limit: call.stdout_cap,
|
||||
});
|
||||
}
|
||||
Err(mpsc::RecvTimeoutError::Timeout) if !remaining().is_zero() => {}
|
||||
Err(_) => return Err(timeout()),
|
||||
}
|
||||
Err(_) => return Err(timeout()),
|
||||
};
|
||||
let status = reaper.wait_until(deadline).ok_or_else(timeout)?;
|
||||
let Some(status) = reaper.wait_until(deadline, cancel.as_ref()) else {
|
||||
return Err(if cancelled() {
|
||||
VcsError::Cancelled
|
||||
} else {
|
||||
timeout()
|
||||
});
|
||||
};
|
||||
let stderr = err_rx
|
||||
.recv_timeout(remaining().max(Duration::from_millis(200)))
|
||||
.ok()
|
||||
|
|
@ -324,6 +486,42 @@ pub fn run(executable: &Path, call: Call<'_>) -> Result<Output, VcsError> {
|
|||
})
|
||||
}
|
||||
|
||||
/// Replaces the userinfo of every URL in `text` (`https://user:token@host`
|
||||
/// becomes `https://***@host`), so credentials in a remote URL never reach
|
||||
/// an error message, a progress line or a log. A URL is `://` after a
|
||||
/// scheme character (letter, digit, `+`, `-`, `.`); its authority ends at
|
||||
/// `/`, `?`, `#`, whitespace, a quote or an angle bracket, and its
|
||||
/// userinfo is everything before the last `@` in it. scp-like addresses
|
||||
/// (`git@host:path`) carry a user name only and stay as they are.
|
||||
pub fn redact_urls(text: &str) -> String {
|
||||
let mut out = String::with_capacity(text.len());
|
||||
let mut rest = text;
|
||||
while let Some(at) = rest.find("://") {
|
||||
let (before, after) = rest.split_at(at + 3);
|
||||
let scheme_ok = before
|
||||
.get(..at)
|
||||
.and_then(|s| s.chars().next_back())
|
||||
.is_some_and(|c| c.is_ascii_alphanumeric() || matches!(c, '+' | '-' | '.'));
|
||||
out.push_str(before);
|
||||
let end = after
|
||||
.find(|c: char| {
|
||||
matches!(c, '/' | '?' | '#' | '"' | '\'' | '<' | '>' | '`') || c.is_whitespace()
|
||||
})
|
||||
.unwrap_or(after.len());
|
||||
let (authority, tail) = after.split_at(end);
|
||||
match authority.rfind('@') {
|
||||
Some(user_end) if scheme_ok => {
|
||||
out.push_str("***");
|
||||
out.push_str(authority.get(user_end..).unwrap_or_default());
|
||||
}
|
||||
_ => out.push_str(authority),
|
||||
}
|
||||
rest = tail;
|
||||
}
|
||||
out.push_str(rest);
|
||||
out
|
||||
}
|
||||
|
||||
/// Keeps git from opening a console window on Windows.
|
||||
#[cfg(windows)]
|
||||
fn hide_console(command: &mut Command) {
|
||||
|
|
@ -364,6 +562,69 @@ mod tests {
|
|||
assert!(output(b"", b"").message().starts_with("git failed"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn urls_lose_their_userinfo() {
|
||||
assert_eq!(
|
||||
redact_urls("fatal: unable to access 'https://me:ghp_secret@github.com/a/b.git/': 403"),
|
||||
"fatal: unable to access 'https://***@github.com/a/b.git/': 403"
|
||||
);
|
||||
assert_eq!(redact_urls("http://tok@h:8080"), "http://***@h:8080");
|
||||
assert_eq!(
|
||||
redact_urls("ssh://git@host/x and https://a:b@c/d"),
|
||||
"ssh://***@host/x and https://***@c/d"
|
||||
);
|
||||
assert_eq!(
|
||||
redact_urls("git@github.com:a/b.git"),
|
||||
"git@github.com:a/b.git"
|
||||
);
|
||||
assert_eq!(
|
||||
redact_urls("https://github.com/a@b"),
|
||||
"https://github.com/a@b"
|
||||
);
|
||||
assert_eq!(redact_urls("file:///tmp/x"), "file:///tmp/x");
|
||||
assert_eq!(redact_urls("://@ é ://x@y"), "://@ é ://x@y");
|
||||
assert_eq!(
|
||||
output(b"", b"error: https://u:p@h/r").message(),
|
||||
"error: https://***@h/r"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn progress_lines_split_at_carriage_returns() {
|
||||
let seen = Arc::new(std::sync::Mutex::new(Vec::<String>::new()));
|
||||
let copy = Arc::clone(&seen);
|
||||
let mut lines = LineSplitter {
|
||||
sink: ProgressSink(Arc::new(move |l: &str| {
|
||||
copy.lock().unwrap().push(l.to_string())
|
||||
})),
|
||||
pending: Vec::new(),
|
||||
};
|
||||
lines.feed(b"Receiving objects: 10% (1/10)\rReceiving obj", false);
|
||||
lines.feed(
|
||||
b"ects: 100% (10/10), done.\n\nremote: https://a:b@h/x",
|
||||
false,
|
||||
);
|
||||
lines.feed(b"", true);
|
||||
assert_eq!(
|
||||
*seen.lock().unwrap(),
|
||||
[
|
||||
"Receiving objects: 10% (1/10)",
|
||||
"Receiving objects: 100% (10/10), done.",
|
||||
"remote: https://***@h/x",
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
proptest::proptest! {
|
||||
#[test]
|
||||
fn redaction_never_panics_and_keeps_no_userinfo(text in ".{0,200}") {
|
||||
let out = redact_urls(&text);
|
||||
if !text.contains("://") {
|
||||
proptest::prop_assert_eq!(out, text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn message_is_capped_on_a_char_boundary() {
|
||||
let long = "é".repeat(STDERR_CAP);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue