Fix false-alarm community hierarchy check (issue #33)#45
Merged
Conversation
detect_communities asserted Q(fine) > Q(coarse) and logged a "sanity check failed" warning on every build where it didn't hold. That assertion is mathematically unsound: Newman modularity is resolution-dependent and maximised at a single scale, so a finer partition scores LOWER at the implicit gamma=1 by construction. Measured the generalized modularity Q_gamma of the actual coarse and fine partitions across gamma: coarse wins for all gamma < 2, fine overtakes at gamma ~ 2.0 == BETA_FINE. So a healthy refinement ALWAYS "failed" the check. Replace with _hierarchy_health_warning(), which tests the real failure mode issue #33 is about — the fine level collapsing into FEWER communities than coarse (n_fine < n_coarse) — plus a MIN_HEALTHY_Q floor catching near-random partitions. Returns None when healthy. Unit-tested in isolation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
detect_communitiesassertedQ(fine) > Q(coarse)and logged "Community hierarchy sanity check failed" on every build where it didn't hold — including the current healthy state (Qc=0.339, Qf=0.281, 7 coarse / 11 fine).Why the assertion is unsound
Newman modularity is resolution-dependent and maximised at a single scale, so a finer partition scores lower at the implicit γ=1 by construction. A healthy refinement can therefore never satisfy
Q(fine) > Q(coarse)at γ=1 — the check fired on every good build.Measured generalized modularity
Q_γof the actual coarse/fine partitions across γ on the live (post-threshold) matrix:The fine partition only overtakes coarse at γ ≈ 2.0 =
BETA_FINE— exactly its design resolution. So the partition is fine; the check was comparing at the wrong scale.Fix
Replace the assertion with
_hierarchy_health_warning(n_coarse, n_fine, q_coarse, q_fine), which tests what issue #33 actually cares about:n_fine < n_coarse(the fine level collapsing into fewer basins, the original Bug: community hierarchy is inverted (fine partition has fewer clusters than coarse) #33 bug).MIN_HEALTHY_Q(0.05), i.e. barely better than random.Returns
Nonewhen healthy. Pure function, unit-tested in isolation.Tests
New
TestHierarchyHealthWarning: healthy hierarchy is silent, inverted counts warn, weak structure warns, equal counts OK. Full suite: 563 passed.