-
Notifications
You must be signed in to change notification settings - Fork 5
Prevent hanging in svn+ssh #1230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,14 @@ | ||
| """Svn repository.""" | ||
|
|
||
| import contextlib | ||
| import functools | ||
| import os | ||
| import pathlib | ||
| import re | ||
| from collections.abc import Callable, Generator, Sequence | ||
| from pathlib import Path | ||
| from typing import NamedTuple | ||
| from urllib.parse import urlparse | ||
|
|
||
| from dfetch.log import get_logger | ||
| from dfetch.util.cmdline import SubprocessCommandError, run_on_cmdline | ||
|
|
@@ -15,6 +17,43 @@ | |
|
|
||
| logger = get_logger(__name__) | ||
|
|
||
| _SSH_HOST_KEY_MSGS = ("host key verification failed", "authenticity of host") | ||
|
|
||
|
|
||
| # As a cli tool, we can safely assume this remains stable during the runtime, caching for speed is better | ||
| @functools.lru_cache | ||
| def _extend_env_for_non_interactive_mode() -> dict[str, str]: | ||
| """Extend the environment vars for svn running in non-interactive mode.""" | ||
| env = os.environ.copy() | ||
| ssh_cmd = env.get("SVN_SSH", "ssh") | ||
| if "BatchMode=" not in ssh_cmd: | ||
| ssh_cmd += " -o BatchMode=yes" | ||
| else: | ||
| logger.debug('BatchMode already configured in SVN_SSH: "%s"', ssh_cmd) | ||
| env["SVN_SSH"] = ssh_cmd | ||
| return env | ||
|
|
||
|
|
||
| def _ssh_target_from_url(url: str) -> str: | ||
| """Return the ``[user@]host`` portion of a svn+ssh URL, or the URL itself.""" | ||
| parsed = urlparse(url) | ||
| host = parsed.hostname or url | ||
| return f"{parsed.username}@{host}" if parsed.username else host | ||
|
|
||
|
|
||
| def _raise_if_ssh_host_key_error(url: str, exc: SubprocessCommandError) -> None: | ||
| """Raise a helpful RuntimeError if *exc* looks like an SSH host-key failure.""" | ||
| stderr_lower = exc.stderr.lower() | ||
| if any(msg in stderr_lower for msg in _SSH_HOST_KEY_MSGS): | ||
| target = _ssh_target_from_url(url) | ||
| raise RuntimeError( | ||
| f"SSH host key verification failed while connecting to '{url}'.\n" | ||
| "Add the host to your known hosts file, for example by running:\n" | ||
| f" ssh-keyscan {target} >> ~/.ssh/known_hosts\n" | ||
| "Or test the SSH connection manually:\n" | ||
| f" ssh -T {target}" | ||
| ) from exc | ||
|
Comment on lines
+37
to
+55
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🌐 Web query:
💡 Result: The ssh-keyscan utility does not accept targets in the user@host format [1][2][3]. It is designed specifically to gather the public SSH host keys of servers, and its arguments expect only hostnames, IP addresses, or CIDR network ranges [1][4][5]. According to the official documentation, the synopsis for ssh-keyscan is [1][3]: ssh-keyscan [-46cDHqv] [-f file] [-O option] [-p port] [-T timeout] [-t type] [host | addrlist namelist] If you provide a string in the user@host format, ssh-keyscan will attempt to resolve that entire string as a hostname, which will fail if the string is not a valid, resolvable host identifier [1][4]. The tool's purpose is to facilitate the creation and verification of ssh_known_hosts files by connecting to the server's public key infrastructure, not to authenticate as a specific user [5][6]. If you need to manage keys for a target accessed as user@host, you should pass only the host portion (the hostname or IP address) to ssh-keyscan [1][7]. Top results: [1][4][3] Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
ls -la
sed -n '1,220p' dfetch/vcs/svn.py | nl -ba | sed -n '1,220p'Repository: dfetch-org/dfetch Length of output: 1911 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '1,140p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '25,90p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '25,90p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '25,90p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '90,190p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
nl -ba dfetch/vcs/svn.py | sed -n '30,80p'Repository: dfetch-org/dfetch Length of output: 104 🏁 Script executed: #!/bin/bash
set -euo pipefail
wc -l dfetch/vcs/svn.py
echo "----"
cat -n dfetch/vcs/svn.py | sed -n '1,120p'
echo "----"
cat -n dfetch/vcs/svn.py | sed -n '120,220p'Repository: dfetch-org/dfetch Length of output: 9233 Fix SSH host-key remediation guidance to use host-only for 🤖 Prompt for AI Agents |
||
|
|
||
|
|
||
| def get_svn_version() -> tuple[str, str]: | ||
| """Get the name and version of svn.""" | ||
|
|
@@ -49,9 +88,14 @@ def __init__(self, remote: str) -> None: | |
| def is_svn(self) -> bool: | ||
| """Check if is SVN.""" | ||
| try: | ||
| run_on_cmdline(logger, ["svn", "info", self._remote, "--non-interactive"]) | ||
| run_on_cmdline( | ||
| logger, | ||
| ["svn", "info", self._remote, "--non-interactive"], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| return True | ||
| except SubprocessCommandError as exc: | ||
| _raise_if_ssh_host_key_error(self._remote, exc) | ||
| if exc.stderr.startswith("svn: E170013"): | ||
| raise RuntimeError( | ||
| f">>>{exc.cmd}<<< failed!\n" | ||
|
|
@@ -67,20 +111,30 @@ def list_of_branches(self) -> list[str]: | |
| result = run_on_cmdline( | ||
| logger, | ||
| ["svn", "ls", "--non-interactive", f"{self._remote}/branches"], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| return [ | ||
| line.strip("/\r") | ||
| for line in result.stdout.decode().splitlines() | ||
| if line.strip("/\r") | ||
| ] | ||
| except (SubprocessCommandError, RuntimeError): | ||
| except SubprocessCommandError as exc: | ||
| _raise_if_ssh_host_key_error(self._remote, exc) | ||
| return [] | ||
| except RuntimeError: | ||
| return [] | ||
|
|
||
| def list_of_tags(self) -> list[str]: | ||
| """Get list of all available tags.""" | ||
| result = run_on_cmdline( | ||
| logger, ["svn", "ls", "--non-interactive", f"{self._remote}/tags"] | ||
| ) | ||
| try: | ||
| result = run_on_cmdline( | ||
| logger, | ||
| ["svn", "ls", "--non-interactive", f"{self._remote}/tags"], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| except SubprocessCommandError as exc: | ||
| _raise_if_ssh_host_key_error(self._remote, exc) | ||
| raise | ||
| return [ | ||
| str(tag).strip("/\r") for tag in result.stdout.decode().split("\n") if tag | ||
| ] | ||
|
|
@@ -116,7 +170,9 @@ def ls_tree(self, url_path: str) -> list[tuple[str, bool]]: | |
| """List immediate children of *url_path* as ``(name, is_dir)`` pairs.""" | ||
| try: | ||
| result = run_on_cmdline( | ||
| logger, ["svn", "ls", "--non-interactive", url_path] | ||
| logger, | ||
| ["svn", "ls", "--non-interactive", url_path], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| entries: list[tuple[str, bool]] = [] | ||
| for line in result.stdout.decode().splitlines(): | ||
|
|
@@ -126,7 +182,10 @@ def ls_tree(self, url_path: str) -> list[tuple[str, bool]]: | |
| is_dir = line.endswith("/") | ||
| entries.append((line.rstrip("/"), is_dir)) | ||
| return entries | ||
| except (SubprocessCommandError, RuntimeError): | ||
| except SubprocessCommandError as exc: | ||
| _raise_if_ssh_host_key_error(url_path, exc) | ||
| return [] | ||
| except RuntimeError: | ||
| return [] | ||
|
|
||
|
|
||
|
|
@@ -176,7 +235,7 @@ def externals_from_url(url: str, revision: str = "") -> list[External]: | |
| if revision: | ||
| cmd += ["--revision", revision] | ||
| cmd += [url] | ||
| result = run_on_cmdline(logger, cmd) | ||
| result = run_on_cmdline(logger, cmd, env=_extend_env_for_non_interactive_mode()) | ||
| repo_root = SvnRepo.get_info_from_target(url)["Repository Root"] | ||
| normalized = SvnRepo._normalize_url_prefix(result.stdout.decode(), url) | ||
| return SvnRepo._parse_externals(normalized, repo_root) | ||
|
|
@@ -292,9 +351,12 @@ def get_info_from_target(target: str = "") -> dict[str, str]: | |
| """Get the info of the given target.""" | ||
| try: | ||
| result = run_on_cmdline( | ||
| logger, ["svn", "info", "--non-interactive", target.strip()] | ||
| logger, | ||
| ["svn", "info", "--non-interactive", target.strip()], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ).stdout.decode() | ||
| except SubprocessCommandError as exc: | ||
| _raise_if_ssh_host_key_error(target, exc) | ||
| if exc.stderr.startswith("svn: E170013"): | ||
| raise RuntimeError( | ||
| f">>>{exc.cmd}<<< failed!\n" | ||
|
|
@@ -335,6 +397,7 @@ def get_last_changed_revision(target: str | Path) -> str: | |
| "last-changed-revision", | ||
| target_str, | ||
| ], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| .stdout.decode() | ||
| .strip() | ||
|
|
@@ -382,6 +445,7 @@ def export(url: str, rev: str = "", dst: str = ".") -> None: | |
| ["svn", "export", "--non-interactive", "--force"] | ||
| + (["--revision", rev] if rev else []) | ||
| + [url, dst], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
|
|
||
| @staticmethod | ||
|
|
@@ -390,7 +454,9 @@ def files_in_path(url_path: str) -> list[str]: | |
| return [ | ||
| str(line) | ||
| for line in run_on_cmdline( | ||
| logger, ["svn", "list", "--non-interactive", url_path] | ||
| logger, | ||
| ["svn", "list", "--non-interactive", url_path], | ||
| env=_extend_env_for_non_interactive_mode(), | ||
| ) | ||
| .stdout.decode() | ||
| .splitlines() | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,7 +7,7 @@ manifest: | |
| default: true # Set it as default | ||
|
|
||
| - name: sourceforge | ||
| url-base: svn://svn.code.sf.net/p/ | ||
| url-base: svn+ssh://svn.code.sf.net/p/ | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial | 💤 Low value 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Description: Verify the example SVN+SSH URL is accessible (read-only check)
# Note: This will fail if SSH keys are not configured for svn.code.sf.net
svn info --non-interactive svn+ssh://svn.code.sf.net/p/tortoisesvn/code 2>&1 | head -20Repository: dfetch-org/dfetch Length of output: 103 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the referenced example config to confirm the final URL and any dependent references
sed -n '1,120p' example/dfetch.yaml | cat -nRepository: dfetch-org/dfetch Length of output: 2527 🌐 Web query:
💡 Result: To achieve non-interactive authentication for svn+ssh on SourceForge, you must rely on SSH key-based authentication rather than Subversion-level credential passing [1][2][3]. Because svn+ssh uses an external SSH client to tunnel the connection, standard Subversion --username/--password flags do not pass credentials to the underlying SSH process [4][5]. Follow these steps to configure your environment for non-interactive access: 1. Register your SSH public key with your SourceForge account via the Account SSH page (accessible under your User Account settings) [2][3][6]. 2. Ensure your local SSH client is configured to use this key automatically [2][3]. - On Linux/macOS (OpenSSH), verify your key is added to your ssh-agent [6]. You can also use ~/.ssh/config to explicitly map the host svn.code.sf.net to your identity file: Host svn.code.sf.net IdentityFile ~/.ssh/your_private_key IdentitiesOnly yes - On Windows (PuTTY/Plink), load your private key (.ppk) into Pageant [2][3]. Alternatively, you can point your Subversion configuration to Plink using the SVN_SSH environment variable [2][3]: set SVN_SSH="C:\path\to\plink.exe -batch -i C:\path\to\your\key.ppk" 3. Test your configuration with a simple command like Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n "svn\+ssh|sourceforge|tortoisesvn|SSH key|SSH keys|svn\.code\.sf\.net" -S . || trueRepository: dfetch-org/dfetch Length of output: 5102 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Show the README section mentioning svn:// vs svn+ssh, plus the changelog entry.
sed -n '60,130p' README.md | cat -n
echo "----"
sed -n '1,80p' CHANGELOG.rst | cat -n
# Show the specific example YAML lines already referenced.
sed -n '1,60p' example/dfetch.yaml | cat -n
# Inspect the svn+ssh-related tests referenced by rg output.
sed -n '480,620p' tests/test_svn.py | cat -nRepository: dfetch-org/dfetch Length of output: 14793 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Also inspect the purl/security references around svn+ssh to understand stated intent.
sed -n '330,430p' security/tm_usage.py | cat -n
sed -n '1,120p' dfetch/vcs/svn.py | cat -nRepository: dfetch-org/dfetch Length of output: 10431 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Locate where manifest 'remotes' url-base + project repo-path are combined (especially for 'vcs: svn').
rg -n "url-base|repo-path|remote:|vcs:\s*svn|svn\+" dfetch example doc tests -SRepository: dfetch-org/dfetch Length of output: 20485 Clarify SSH key/known-host setup for the SourceForge
🤖 Prompt for AI Agents |
||
|
|
||
| projects: | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update placeholder issue reference.
The changelog entry references
(#0), which appears to be a placeholder. Based on the PR objectives, this should reference issue or PR#1230.📝 Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents