Skip to content

Commit 30d09e1

Browse files
ElleNajtclaude
andcommitted
Add CLAUDE.md and CI debugging notes
Document the known CI test failures due to macOS vs Linux differences in org-table-align behavior. Tests pass locally on macOS but fail on Linux CI due to extra columns being created. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent ab9873f commit 30d09e1

2 files changed

Lines changed: 125 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# ob-python-extras
2+
3+
Emacs package for enhanced Python org-babel support.
4+
5+
## Known Issues
6+
7+
### CI Test Failures (macOS vs Linux)
8+
9+
The GitHub CI tests are failing due to platform differences between macOS (where golden files are generated) and Linux (where CI runs).
10+
11+
**Root cause:** On macOS, `org-table-align` creates an extra empty column in tables. This appears to be caused by carriage return (`\r`) characters being inserted somewhere in the Emacs processing pipeline. The `\r` shows up as content in what becomes an extra column.
12+
13+
**Symptoms:**
14+
- Golden files (macOS) have tables with 6 columns, ending with `| \r |`
15+
- Linux output has tables with 5 columns, no trailing empty column
16+
- Separator rows differ: `|---+---------|----|` (macOS) vs `|---+---------|` (Linux)
17+
18+
**Attempted fixes:**
19+
- Added tabulate dependency for cleaner DataFrame output
20+
- Added normalization in `tests/run_expect_tests.sh` to strip trailing empty columns and CR characters
21+
- The normalization partially works but doesn't fully resolve the issue
22+
23+
**Status:** Tests pass locally on macOS but fail on Linux CI. Need to either:
24+
1. Generate golden files on Linux
25+
2. Find and fix the source of the `\r` characters in the Emacs processing
26+
3. Make normalization more robust
27+
28+
See debugging notes in `tests/CI_DEBUG.md`.
29+
30+
## Running Tests
31+
32+
```bash
33+
cd tests
34+
./run_expect_tests.sh # Run tests
35+
./run_expect_tests.sh --update-goldens # Regenerate golden files
36+
```
37+
38+
## Dependencies
39+
40+
Uses uv for Python dependency management. Run `uv sync` to install dependencies.

tests/CI_DEBUG.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# CI Test Failure Debugging Notes
2+
3+
## Problem Summary
4+
5+
Tests pass on macOS but fail on Linux CI. The issue is extra columns appearing in org tables on macOS.
6+
7+
## Evidence
8+
9+
### macOS output (golden files):
10+
```
11+
| idx | Name | Age | Score | \r |
12+
|-----+------+-----+-------|----|
13+
| 0 | Joe | 44 | 92.5 | \r |
14+
```
15+
16+
### Linux output (CI):
17+
```
18+
| idx | Name | Age | Score |
19+
|-----+------+-----+-------|
20+
| 0 | Joe | 44 | 92.5 |
21+
```
22+
23+
## Where the `\r` comes from
24+
25+
The carriage return is NOT in the Python output. Verified by checking:
26+
```python
27+
.venv/bin/python3 -c "
28+
import pandas as pd
29+
df = pd.DataFrame({'Name': ['Joe'], 'Age': [44]})
30+
print(repr(df.to_markdown()))
31+
"
32+
# Output is clean, no \r
33+
```
34+
35+
The `\r` appears after Emacs processes the output. It's present in the staging org files:
36+
```bash
37+
grep -A 3 "RESULTS: print_table" tests/staging/babel-formatting.org | od -c
38+
# Shows literal \r byte before the final |
39+
```
40+
41+
## Suspects
42+
43+
1. **org-table-align** - Called in `ob-python-extras/adjust-org-babel-results`. May interpret trailing content differently on macOS.
44+
45+
2. **Process output handling** - How Emacs captures Python process stdout may differ between macOS and Linux.
46+
47+
3. **Line ending handling** - macOS may be adding CR characters somewhere in the pipeline.
48+
49+
## Attempted Fixes
50+
51+
### 1. Added tabulate dependency
52+
Tabulate produces cleaner markdown output than the CSV fallback. Didn't fix the issue - the `\r` is added after Python outputs the data.
53+
54+
### 2. Normalization in run_expect_tests.sh
55+
Added jq filters to strip:
56+
- `| \r |` at end of lines
57+
- `| |` (whitespace) at end of lines
58+
- `|----|` (separator) at end of lines
59+
- Bare `\r` characters
60+
61+
Partially works but doesn't catch all cases because the separator column can have variable numbers of dashes.
62+
63+
## Potential Solutions
64+
65+
### Option A: Generate golden files on Linux
66+
Run `--update-goldens` in CI or on a Linux machine, then commit those. This would make the "expected" output match what Linux produces.
67+
68+
### Option B: Fix the source of \r
69+
Find where in Emacs/ob-python-extras the carriage returns are being inserted and prevent it. Likely somewhere in:
70+
- `ob-python-extras.el` around line 464 (`ob-python-extras/my-align-advice-function`)
71+
- Or in how babel captures process output
72+
73+
### Option C: More aggressive normalization
74+
Strip ALL trailing table columns that match `| .* |$` pattern, not just specific ones. Risk: might strip legitimate content.
75+
76+
### Option D: Skip table comparison
77+
Only compare non-table lines, or use a fuzzy comparison for tables that ignores trailing columns.
78+
79+
## Files Involved
80+
81+
- `tests/run_expect_tests.sh` - Test runner with normalization logic
82+
- `tests/extract-results.el` - Extracts results from org files to JSON
83+
- `tests/golden/*.json` - Expected outputs (generated on macOS)
84+
- `ob-python-extras.el` - Main package, has org-table-align advice
85+
- `python/print_org_df.py` - DataFrame to org-table conversion

0 commit comments

Comments
 (0)