Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu
- Fix: an unresolved bare npm import no longer aliases onto an unrelated same-named local file (#1638, thanks @EveX1). `import colors from "tailwindcss/colors"` in a `.tsx` file emitted an `imports_from` edge to the bare id `colors`, and build.py's pre-migration alias index (which registers every local file's bare stem) then remapped it onto an unrelated `backend/utils/colors.py` — a confident (`EXTRACTED`) cross-language phantom edge, and one per `.tsx` file sharing the import. In a real monorepo eight unrelated `.tsx` files all landed on a single Python module. Common package subpaths (`colors`, `utils`, `types`, `config`, `client`) collide this way constantly. The external-import fallback now namespaces its target with the `ref` prefix (the same J-4 convention used for tsconfig `extends`/`$ref` externals), so it can never collapse to a local file/symbol id; the ref-namespaced target has no node, so build drops it as an external reference — the correct outcome for a third-party import.
- Fix: `graph.json` node/edge ordering is now stable run-to-run for document/semantic corpora (#1632, thanks @umeshpsatwe). With a parallel LLM backend, `extract_corpus_parallel` merged chunk results in completion order, so which network call happened to return first reordered the nodes and edges even when the model returned identical content — churning `graph.json` between otherwise-identical runs. Chunks are now merged in deterministic submission order after the pool drains (matching the serial path); the progress callback still fires in completion order so long local runs aren't silent. Note: the semantic content the LLM extracts is itself nondeterministic run-to-run — this fix removes the pipeline's own ordering churn, not the model's variance.

- Feat: Haxe (`.hx`) language support via `tree-sitter-haxe`. Extracts classes, interfaces, enums, enum abstracts, typedefs, and functions. Includes a fallback pass for files where the grammar emits scattered tokens instead of declaration nodes. `tree-sitter-haxe` has no PyPI release, so there is no `graphifyy` extra for it — install with `pip install git+https://github.com/masquepublishing/tree-sitter-haxe.git`.
- Fix: `graphify export obsidian` no longer crashes in `to_canvas` on a dangling community member (#1236 follow-up, thanks @swells808). The original #1236 fix guarded `to_obsidian` but not `to_canvas`, so a community member id with no backing node in the graph still raised `KeyError` while writing `graph.canvas` — after the notes had exported, leaving a partial mirror. `to_canvas` now applies the same dangling-member filter (`m in G and m in node_filenames`) in both the box-sizing and card-layout loops.

- Feat: TS/JS member calls on a local `new` binding or a type-annotated parameter now resolve (#1630, thanks @DanielC000). `const s = new Svc(); s.doThing()` and a call on a typed param — including inside a returned closure (`(svc: Svc) => () => svc.doThing()`) — now emit `calls` edges to the receiver type's method, so `affected` no longer silently under-reports. Extends the #1316 `this.field` resolver: the per-file type table now also learns local `new` bindings and bare-typed parameters, and `walk_calls` descends into inline/returned closures (attributing their calls to the enclosing function) instead of stopping at the arrow boundary. Resolution keeps the single-definition guard; an untyped or non-bare-typed (array/union/generic) receiver produces no edge.
Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,16 @@ Codex users also need `multi_agent = true` under `[features]` in `~/.codex/confi
| `chinese` | Chinese query segmentation (jieba) | `uv tool install "graphifyy[chinese]"` |
| `all` | Everything above | `uv tool install "graphifyy[all]"` |

Haxe is not in this table: `tree-sitter-haxe` has no PyPI release (upstream
`vantreeseba/tree-sitter-haxe` hasn't cut one), and PyPI rejects any package
upload whose metadata contains a direct URL/VCS dependency — so it can't be
declared as a `graphifyy` extra without blocking every future release of this
package. Install the patched fork manually to enable `.hx` support:

```bash
pip install git+https://github.com/masquepublishing/tree-sitter-haxe.git
```

</details>

---
Expand Down Expand Up @@ -327,7 +337,8 @@ To remove graphify from all platforms at once: `graphify uninstall` (add `--purg

| Type | Extensions |
|------|-----------|
| Code (36 tree-sitter grammars) | `.py .ts .mts .cts .js .jsx .tsx .mjs .go .rs .java .c .cpp .cc .cxx .h .hpp .cu .cuh .metal .rb .cs .kt .kts .scala .php .swift .lua .luau .toc .zig .ps1 .psm1 .psd1 .ex .exs .m .mm .jl .vue .svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json .dm .dme .dmi .dmm .dmf .sln .slnx .csproj .fsproj .vbproj .xaml .razor .cshtml` (`.dm`/`.dme` requires `uv tool install graphifyy[dm]`; `.mts`/`.cts` reuse the TypeScript grammar, `.cc`/`.cxx` and CUDA `.cu`/`.cuh` and Metal `.metal` reuse the C++ grammar) |
| Code (37 tree-sitter grammars) | `.py .ts .mts .cts .js .jsx .tsx .mjs .go .rs .java .c .cpp .cc .cxx .h .hpp .cu .cuh .metal .rb .cs .kt .kts .scala .php .swift .lua .luau .toc .zig .ps1 .psm1 .psd1 .ex .exs .m .mm .jl .vue .svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json .dm .dme .dmi .dmm .dmf .sln .slnx .csproj .fsproj .vbproj .xaml .razor .cshtml` (`.dm`/`.dme` requires `uv tool install graphifyy[dm]`; `.mts`/`.cts` reuse the TypeScript grammar, `.cc`/`.cxx` and CUDA `.cu`/`.cuh` and Metal `.metal` reuse the C++ grammar) |
| Haxe | `.hx` (requires `pip install git+https://github.com/masquepublishing/tree-sitter-haxe.git`; not a PyPI package, so no `graphifyy` extra exists for it — see below; classes, interfaces, enums, enum abstracts, typedefs, functions) |
| Salesforce Apex | `.cls .trigger` (regex-based; classes, interfaces, enums, methods, triggers, SOQL/DML edges) |
| Terraform / HCL | `.tf .tfvars .hcl` (requires `uv tool install graphifyy[terraform]`) |
| MCP configs | `.mcp.json` `mcp.json` `mcp_servers.json` `claude_desktop_config.json` — extracts server nodes, package refs, env var requirements |
Expand Down
21 changes: 19 additions & 2 deletions graphify/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3265,7 +3265,7 @@ def main() -> None:
if len(sys.argv) < 3:
print('Usage: graphify explain "<node>" [--graph path]', file=sys.stderr)
sys.exit(1)
from graphify.serve import _find_node
from graphify.serve import _find_node_tiers
from networkx.readwrite import json_graph

label = sys.argv[2]
Expand All @@ -3288,10 +3288,27 @@ def main() -> None:
G = json_graph.node_link_graph(_raw, edges="links")
except TypeError:
G = json_graph.node_link_graph(_raw)
matches = _find_node(G, label)
tiers = _find_node_tiers(G, label)
matches = [nid for tier in tiers for nid in tier]
if not matches:
print(f"No node matching '{label}' found.")
sys.exit(0)
top_tier = next(t for t in tiers if t)
# A class and its own same-named constructor method (e.g. "TableRules"
# the class and "TableRules()" the method) legitimately share a bare
# label and land in the same tier — that's not a meaningful collision,
# just Haxe/C++ constructor-naming convention, and warning on it would
# fire for nearly every class lookup depot-wide. Only warn when the
# tied candidates actually come from different source files.
distinct_sources = {G.nodes[n].get("source_file", "") for n in top_tier}
if len(top_tier) >= 2 and len(distinct_sources) >= 2:
print(
f"warning: '{label}' match was ambiguous ({len(top_tier)} "
f"equally-ranked matches across {len(distinct_sources)} files) — "
f"showing '{G.nodes[matches[0]].get('label', matches[0])}' from "
f"{G.nodes[matches[0]].get('source_file', '?') or '(no source)'}",
file=sys.stderr,
)
nid = matches[0]
d = G.nodes[nid]
print(f"Node: {d.get('label', nid)}")
Expand Down
41 changes: 36 additions & 5 deletions graphify/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -507,7 +507,32 @@ def build_from_json(extraction: dict, *, directed: bool = False, root: str | Pat
# fragment (e.g. an incremental update whose fragment references a symbol in a
# file that was NOT re-extracted) still resolves to the migrated node instead
# of dangling. Only fills gaps — never overrides a real node id.
#
# The old-stem form drops the extension and (for the file node itself) every
# directory but the immediate parent, so it collapses easily: "ping.h" and
# "ping.php" in different directories both alias to bare "ping". Collecting
# every candidate for an alias BEFORE committing any of them — and only
# committing when exactly one candidate claims it — keeps this a precise
# re-keying aid instead of a silent cross-file (and cross-language) merge.
# Without this, a dangling edge to a bare, deliberately-unscoped fallback id
# (e.g. the C/C++ extractor's last-resort target for an #include it couldn't
# resolve to a real path) could ride this alias onto whichever unrelated
# same-stem file happened to be inserted first into ``node_set`` — a Python
# set, so "first" is hash-order, not anything meaningful.
#
# A file node's OWN id is not always a clean ``new_stem`` prefix: when a
# same-directory ``.h``/``.cpp`` pair collides on their shared pre-extension
# id, _disambiguate_colliding_node_ids salts both apart into ids like
# ``tools_aolserver_utility_h_tools_aolserver_utility`` — which no longer
# string-prefixes cleanly for the suffix math below. Detecting "this IS the
# file node" by label (every file node's label is its own basename,
# regardless of id mangling) instead of by id shape keeps a salted file node
# in the alias competition, so a genuine collision (a C header AND an
# unrelated same-named PHP script) is still caught as ambiguous instead of
# the header silently dropping out of the race and leaving the PHP file as
# the lone (wrong) "unambiguous" winner.
from graphify.extractors.base import _file_stem as _fs
_alias_candidates: dict[str, set[str]] = {}
for nid in node_set:
attrs = G.nodes[nid]
sf = attrs.get("source_file")
Expand All @@ -517,15 +542,21 @@ def build_from_json(extraction: dict, *, directed: bool = False, root: str | Pat
if rel.is_absolute():
continue
new_stem = make_id(_fs(rel))
suffix = ""
if _normalize_id(nid).startswith(new_stem):
suffix = _normalize_id(nid)[len(new_stem):] # leading "_entity" or ""
if str(attrs.get("label", "")) == rel.name:
suffix = "" # this node IS the file, whatever its (possibly salted) id
else:
suffix = ""
if _normalize_id(nid).startswith(new_stem):
suffix = _normalize_id(nid)[len(new_stem):] # leading "_entity" or ""
for old_stem in _old_file_stems(rel):
if old_stem == new_stem:
continue
alias = old_stem + suffix
norm_to_id.setdefault(_normalize_id(alias), nid)
norm_to_id.setdefault(alias, nid)
_alias_candidates.setdefault(_normalize_id(alias), set()).add(nid)
_alias_candidates.setdefault(alias, set()).add(nid)
for alias_key, candidates in _alias_candidates.items():
if len(candidates) == 1:
norm_to_id.setdefault(alias_key, next(iter(candidates)))
# Iterate edges in a deterministic order. The graph is undirected and stores
# direction in _src/_tgt; when two edges collapse onto the same node pair the
# last write wins, so an unstable iteration order flips _src/_tgt run-to-run
Expand Down
45 changes: 40 additions & 5 deletions graphify/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,47 @@
# use. The semantic cache is deliberately NOT versioned — its entries are
# produced by the LLM from file contents, and invalidating them on every
# release would re-bill extraction for unchanged files.
try:
from importlib.metadata import version as _pkg_version
def _detect_extractor_version() -> str:
"""Best-effort version string for AST cache namespacing.

Installed-package metadata is tried first, so this keeps working exactly
as designed if graphify is ever `pip install`ed. But a checkout used
purely via PYTHONPATH (this repo's own documented workflow — see
code-map/CLAUDE.md's Setup section, which never runs `pip install` for
graphify itself) never registers package metadata, so that lookup always
raises and this used to collapse to a constant "unknown" bucket that
never invalidated across extractor code changes, silently serving
arbitrarily stale per-file results forever. Falling back to this
checkout's git commit gives PYTHONPATH-only use the same invalidate-on
-change behavior pip installs get for free.
"""
try:
from importlib.metadata import version as _pkg_version

return _pkg_version("graphifyy")
except Exception:
pass
try:
import subprocess

repo_root = Path(__file__).resolve().parent.parent
result = subprocess.run(
["git", "rev-parse", "--short", "HEAD"],
cwd=repo_root,
capture_output=True,
text=True,
timeout=5,
check=True,
)
commit = result.stdout.strip()
if commit:
return f"git-{commit}"
except Exception:
pass
return "unknown"


_EXTRACTOR_VERSION = _pkg_version("graphifyy")
except Exception:
_EXTRACTOR_VERSION = "unknown"
_EXTRACTOR_VERSION = _detect_extractor_version()

# Version dirs already swept this process — cleanup runs once per (base, version).
_cleaned_ast_dirs: set[str] = set()
Expand Down
2 changes: 1 addition & 1 deletion graphify/detect.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class FileType(str, Enum):

_MANIFEST_PATH = str(out_path("manifest.json"))

CODE_EXTENSIONS = {'.py', '.ts', '.tsx', '.mts', '.cts', '.js', '.jsx', '.mjs', '.ejs', '.ets', '.go', '.rs', '.java', '.groovy', '.gradle', '.cpp', '.cc', '.cxx', '.c', '.h', '.hpp', '.cu', '.cuh', '.metal', '.rb', '.swift', '.kt', '.kts', '.cs', '.scala', '.php', '.lua', '.luau', '.toc', '.zig', '.ps1', '.psm1', '.psd1', '.ex', '.exs', '.m', '.mm', '.jl', '.vue', '.svelte', '.astro', '.dart', '.v', '.sv', '.svh', '.sql', '.r', '.f', '.F', '.f90', '.F90', '.f95', '.F95', '.f03', '.F03', '.f08', '.F08', '.pas', '.pp', '.dpr', '.dpk', '.lpr', '.inc', '.dfm', '.lfm', '.lpk', '.sh', '.bash', '.json', '.tf', '.tfvars', '.hcl', '.dm', '.dme', '.dmi', '.dmm', '.dmf', '.sln', '.slnx', '.csproj', '.fsproj', '.vbproj', '.xaml', '.razor', '.cshtml', '.cls', '.trigger'}
CODE_EXTENSIONS = {'.py', '.ts', '.tsx', '.mts', '.cts', '.js', '.jsx', '.mjs', '.ejs', '.ets', '.go', '.rs', '.java', '.groovy', '.gradle', '.cpp', '.cc', '.cxx', '.c', '.h', '.hpp', '.cu', '.cuh', '.metal', '.rb', '.swift', '.kt', '.kts', '.cs', '.scala', '.php', '.lua', '.luau', '.toc', '.zig', '.ps1', '.psm1', '.psd1', '.ex', '.exs', '.m', '.mm', '.jl', '.vue', '.svelte', '.astro', '.dart', '.v', '.sv', '.svh', '.sql', '.r', '.f', '.F', '.f90', '.F90', '.f95', '.F95', '.f03', '.F03', '.f08', '.F08', '.pas', '.pp', '.dpr', '.dpk', '.lpr', '.inc', '.dfm', '.lfm', '.lpk', '.sh', '.bash', '.json', '.tf', '.tfvars', '.hcl', '.dm', '.dme', '.dmi', '.dmm', '.dmf', '.sln', '.slnx', '.csproj', '.fsproj', '.vbproj', '.xaml', '.razor', '.cshtml', '.cls', '.trigger', '.hx'}
DOC_EXTENSIONS = {'.md', '.mdx', '.qmd', '.txt', '.rst', '.html', '.yaml', '.yml'}
PAPER_EXTENSIONS = {'.pdf'}
IMAGE_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.gif', '.webp', '.svg'}
Expand Down
Loading