Skip to content

Commit 8790736

Browse files
brendancolbrendancol
committed
geotiff tests: consolidate writer cluster (cog, bigtiff, overview) (#2400)
Folds 26 writer/COG/BigTIFF/overview test files into a four-file write/ subpackage per epic #2390 PR 7: - write/test_basic.py: generic writer paths (compression, tiling, kwarg order, return path, layout monkeypatch, VRT writer surface). - write/test_cog.py: COG writer compliance and invalid-input errors. - write/test_bigtiff.py: BigTIFF + COG compliance. - write/test_overview.py: overview-level and nodata-aware overview tests. Tests-only restructure. No source changes. The fold preserves 497 test ids (collect-only counts match before and after). Module-level dedup removed identical-body shadowed helpers (`_build_source_tif`, `_gpu_available`, `_HAS_GPU`) and renamed the one private-import `write_vrt` so it does not shadow the public re-export used by the kwarg-order signature tests in the same file. The release gate checklist (docs/source/reference/release_gate_geotiff.rst) and the user-facing GeoTIFF docs (docs/source/reference/geotiff.rst) had their citations re-pointed at the new file paths so the rst-parity gate (test_release_gate_2321.py) keeps passing. CLUSTER_AUDIT_PR7.md is in this commit and is deleted in a follow-up commit on the same branch before merge per the epic contract. Closes #2400 Part of #2390
1 parent 04514ad commit 8790736

33 files changed

Lines changed: 5494 additions & 5913 deletions

‎docs/source/reference/geotiff.rst‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ COG validator CI gate
208208
on every PR. A dedicated Linux job (``pytest-cog-validator``)
209209
installs rio-cogeo and the GDAL Python bindings from conda-forge,
210210
sets ``XRSPATIAL_REQUIRE_COG_VALIDATOR=1``, and runs the compliance
211-
suite in ``xrspatial/geotiff/tests/test_cog_writer_compliance.py``.
211+
suite in ``xrspatial/geotiff/tests/write/test_cog.py``.
212212
With the env var set, a missing validator dependency is a hard
213213
failure instead of a silent skip, so a misconfigured install step
214214
cannot quietly let the gate pass. Contributors without rio-cogeo
@@ -494,7 +494,7 @@ is a plain GeoTIFF or a COG.
494494

495495
``SUPPORTED_FEATURES['writer.bigtiff_cog']`` is currently ``advanced``.
496496
The external-interop gate lives in
497-
``xrspatial/geotiff/tests/test_bigtiff_cog_compliance_2286.py`` and
497+
``xrspatial/geotiff/tests/write/test_bigtiff.py`` and
498498
covers the BigTIFF-specific layout (header, IFDs, tile and overview
499499
offset tables), one lossless integer codec, one lossless float codec,
500500
single-band and 3-band, one overview level, plus an auto-promotion row
@@ -516,9 +516,9 @@ regression test that locks the behaviour.
516516
* - Combination
517517
- Regression test
518518
* - ``to_geotiff(cog=True, tiled=False)``
519-
- ``xrspatial/geotiff/tests/test_cog_requires_tiled_2312.py``
519+
- ``xrspatial/geotiff/tests/write/test_cog.py``
520520
* - ``to_geotiff(cog=True, tile_size <= 0)``
521-
- ``xrspatial/geotiff/tests/test_cog_tile_size_hang_2311.py``
521+
- ``xrspatial/geotiff/tests/write/test_cog.py``
522522
* - Warped VRT
523523
(``<VRTDataset subClass="VRTWarpedDataset">`` or
524524
``<VRTRasterBand subClass="VRTWarpedRasterBand">``)

‎docs/source/reference/release_gate_geotiff.rst‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -200,14 +200,14 @@ Local GeoTIFF read and write
200200
- stable
201201
- ``to_geotiff`` writes a file that ``open_geotiff`` reads back
202202
bit-exact for every stable codec.
203-
- ``xrspatial/geotiff/tests/test_cog_writer_compliance.py``,
203+
- ``xrspatial/geotiff/tests/write/test_cog.py``,
204204
``xrspatial/geotiff/tests/test_attrs_finalization_parity_2211.py``
205205
- `#2341`_
206206
* - ``writer.overviews``
207207
- advanced
208208
- Internal overview IFDs round-trip; the reader can pick a level.
209209
- ``xrspatial/geotiff/tests/test_dask_overview_level.py``,
210-
``xrspatial/geotiff/tests/test_cog_overview_nodata_1613.py``
210+
``xrspatial/geotiff/tests/write/test_overview.py``
211211
- `#2286`_
212212
* - ``writer.bigtiff``
213213
- advanced
@@ -274,14 +274,13 @@ Cloud-optimized GeoTIFF (COG)
274274
- ``to_geotiff(cog=True)`` writes an IFD-first tiled file with internal
275275
overviews that ``rio-cogeo`` accepts (CI-gated by
276276
``XRSPATIAL_REQUIRE_COG_VALIDATOR=1``).
277-
- ``xrspatial/geotiff/tests/test_cog_writer_compliance.py``,
278-
``xrspatial/geotiff/tests/test_cog_parity_2286.py``
277+
- ``xrspatial/geotiff/tests/write/test_cog.py``
279278
- `#2286`_
280279
* - ``reader.local_cog``
281280
- stable
282281
- Local COG with overview IFDs decodes byte-for-byte through eager and
283282
dask paths.
284-
- ``xrspatial/geotiff/tests/test_cog.py``,
283+
- ``xrspatial/geotiff/tests/write/test_cog.py``,
285284
``xrspatial/geotiff/tests/test_golden_corpus_overview_cog_1930.py``
286285
- `#2286`_
287286
* - ``reader.http_cog``
@@ -296,19 +295,19 @@ Cloud-optimized GeoTIFF (COG)
296295
- advanced
297296
- BigTIFF + COG combination passes the dedicated compliance suite
298297
(header magic, IFDs, tile and overview offset tables).
299-
- ``xrspatial/geotiff/tests/test_bigtiff_cog_compliance_2286.py``
298+
- ``xrspatial/geotiff/tests/write/test_bigtiff.py``
300299
- `#2286`_
301300
* - ``writer.cog`` -- tile-layout pre-flight (``cog=True, tiled=False``)
302301
- stable
303302
- Raises ``ValueError`` at the writer entry point regardless of dtype
304303
or codec.
305-
- ``xrspatial/geotiff/tests/test_cog_requires_tiled_2312.py``
304+
- ``xrspatial/geotiff/tests/write/test_cog.py``
306305
- `#2286`_
307306
* - ``writer.cog`` -- tile-size pre-flight (non-positive ``tile_size``)
308307
- stable
309308
- Non-positive tile sizes raise ``ValueError`` regardless of the
310309
``tiled`` flag.
311-
- ``xrspatial/geotiff/tests/test_cog_tile_size_hang_2311.py``
310+
- ``xrspatial/geotiff/tests/write/test_cog.py``
312311
- `#2286`_
313312

314313
HTTP / fsspec reads

‎xrspatial/geotiff/tests/CLUSTER_AUDIT_PR7.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# CLUSTER_AUDIT_PR7.md — Writer / COG / BigTIFF cluster (epic #2390 PR 7)
2+
3+
Temporary audit table tracking the 26 deleted writer-cluster files and
4+
where their tests land in the consolidated layout. **This file is
5+
deleted on a follow-up commit on the same branch before the PR is
6+
approved per the epic contract.**
7+
8+
Result: 497 collected tests in `write/` mirror 497 collected tests
9+
across the 26 source files (counts captured before and after the move
10+
with `pytest --collect-only -q`). All pass on the run captured below
11+
the table.
12+
13+
## Counts
14+
15+
| Bucket | Source files | New file | Tests collected |
16+
|-------------------|--------------|-----------------------------------|-----------------|
17+
| Generic writer | 14 | `write/test_basic.py` | (see below) |
18+
| COG writer | 6 | `write/test_cog.py` | (see below) |
19+
| BigTIFF + COG | 1 | `write/test_bigtiff.py` | (see below) |
20+
| Overview policy | 5 | `write/test_overview.py` | (see below) |
21+
| **Total** | **26** | **4** | **497** |
22+
23+
`pytest xrspatial/geotiff/tests/write/ --collect-only -q | tail -1`
24+
reports `497 tests collected`. The same count was captured against the
25+
26 source files immediately before the move (see PR description).
26+
27+
## File map
28+
29+
### `write/test_basic.py` — generic writer paths
30+
31+
| Old file | New module section | Notes |
32+
|-----------------------------------------------------------|------------------------------------------------------|-------|
33+
| `test_writer.py` | `# Folded from: test_writer.py` | verbatim |
34+
| `test_writer_matrix.py` | `# Folded from: test_writer_matrix.py` | verbatim |
35+
| `test_writer_kwarg_order_1922.py` | `# Folded from: test_writer_kwarg_order_1922.py` | verbatim |
36+
| `test_writer_return_path_1938.py` | `# Folded from: test_writer_return_path_1938.py` | verbatim |
37+
| `test_writer_uncompressed_tiled_no_dead_alloc_1736.py` | section banner | verbatim |
38+
| `test_write_layout_monkeypatch_contract_2248.py` | section banner | verbatim |
39+
| `test_write_vrt_path_kwarg_1946.py` | section banner | verbatim |
40+
| `test_write_vrt_crs_1715.py` | section banner | verbatim |
41+
| `test_write_vrt_bool_nodata_1921.py` | section banner | verbatim |
42+
| `test_write_vrt_int_nodata_1684.py` | section banner | verbatim |
43+
| `test_vrt_writer_int64_1833.py` | section banner | verbatim |
44+
| `test_vrt_writer_photometric_1861.py` | section banner | verbatim |
45+
| `test_vrt_writer_source_compat_1733.py` | section banner | `write_vrt` import aliased to `_priv_write_vrt` to avoid shadowing the public re-export used by the kwarg-order tests above |
46+
| `test_vrt_write.py` | section banner | verbatim |
47+
48+
Module-level dedup: the identical `_build_source_tif(tmp_path, name='src.tif')` helper that the `test_write_vrt_path_kwarg_1946.py` and `test_write_vrt_crs_1715.py` sections both defined was reduced to one copy (the first), since the two source bodies were byte-for-byte identical.
49+
50+
### `write/test_cog.py` — COG writer compliance + overview/nodata
51+
52+
| Old file | New module section | Notes |
53+
|-------------------------------------------|-------------------------------|-------|
54+
| `test_cog.py` | section banner | verbatim; `from .conftest import gpu_available` rewritten to `from ..conftest import gpu_available` since `write/` is one level deeper |
55+
| `test_cog_writer_compliance.py` | section banner | verbatim |
56+
| `test_cog_invalid_input_errors_2286.py` | section banner | the local `_float_da` helper (shape default `(8, 8)`) was renamed to `_float_da_small` for this section so the later sections' `_float_da` (`(64, 64)`) does not shadow it; all in-section call sites updated |
57+
| `test_cog_parity_2286.py` | section banner | verbatim |
58+
| `test_cog_requires_tiled_2312.py` | section banner | verbatim |
59+
| `test_cog_tile_size_hang_2311.py` | section banner | verbatim |
60+
61+
Module-level dedup: the docstring-only-differing duplicate `_float_da(shape=(64, 64))` from the third section was removed (the second section's definition is reused).
62+
63+
### `write/test_bigtiff.py` — BigTIFF threshold + COG compliance
64+
65+
| Old file | New module section | Notes |
66+
|-------------------------------------------|-------------------------------|-------|
67+
| `test_bigtiff_cog_compliance_2286.py` | section banner | verbatim |
68+
69+
No other current files match the `bigtiff` writer-side criterion. The eager-overhead test (`test_eager_bigtiff_overhead_exact_1905.py`) stays in the read/streaming cluster (PR 8) per the epic, since it exercises the threshold from the *read* path.
70+
71+
### `write/test_overview.py` — overview-level + nodata-aware overviews
72+
73+
| Old file | New module section | Notes |
74+
|------------------------------------------------|----------------------|-------|
75+
| `test_cog_overview_ceil_2105.py` | section banner | verbatim |
76+
| `test_cog_overview_nodata_1613.py` | section banner | verbatim |
77+
| `test_cog_cubic_overview_nodata_1623.py` | section banner | verbatim |
78+
| `test_cog_cubic_int_overview_nodata_1975.py` | section banner | verbatim |
79+
| `test_cog_int_overview_nodata_2026_05_12.py` | section banner | verbatim |
80+
81+
Module-level dedup: the four sections each defined an identical-body `_gpu_available() -> bool` helper and a `_HAS_GPU = _gpu_available()` module assignment. The first definition is kept and the three later identical copies (and their reassignments) were dropped.
82+
83+
## Cross-suite reference updates
84+
85+
Several files outside the writer cluster cited the old filenames; the
86+
release-gate test (`test_release_gate_2321.py`) fails if a cited file
87+
is gone, so the following docs were updated:
88+
89+
| File | Change |
90+
|----------------------------------------------------------|--------|
91+
| `docs/source/reference/release_gate_geotiff.rst` | `writer.local_file`, `writer.cog`, `writer.overviews`, `writer.bigtiff_cog`, `writer.cog (tile-layout / tile-size pre-flight)`, and `reader.local_cog` rows now point at `write/test_cog.py`, `write/test_overview.py`, and `write/test_bigtiff.py` instead of the deleted files. |
92+
| `docs/source/reference/geotiff.rst` | Prose references and the "Known unsupported combinations" table updated to the new paths. |
93+
94+
Comment-only references in `xrspatial/geotiff/_writer.py`,
95+
`xrspatial/geotiff/_attrs.py`,
96+
`xrspatial/geotiff/tests/test_overview_block_order_2308.py`, and
97+
`xrspatial/geotiff/tests/test_release_gate_cog.py` were left in place:
98+
they read as historical notes ("see test_cog_writer_compliance for the
99+
skip semantics") and the test gate only checks the rst checklist, not
100+
Python comments. PR 11's lint pass can refresh these in bulk.
101+
102+
## Out of scope (per PR 7's "MUST NOT touch" list)
103+
104+
- HTTP-side COG tests: `test_cog_http_close_on_error_1816.py`,
105+
`test_cog_http_concurrent.py`,
106+
`test_cog_http_parallel_decode_2026_05_15.py`. Integration cluster
107+
for PR 9.
108+
- Reader-side files: PR 8.
109+
- VRT validation / metadata / window / dtype / missing-sources /
110+
parity files: PRs 1, 2, 4, 6.
111+
112+
## Verification
113+
114+
```
115+
pytest xrspatial/geotiff/tests/write/ -v # 497 passed
116+
pytest xrspatial/geotiff/tests/ -x -q # 5706 passed, 68 skipped, 6 xfailed, 1 xpassed
117+
```
118+
119+
Captured locally on Python 3.14, branch `issue-2400`.

0 commit comments

Comments
 (0)