Skip to content

Fix: Normalize All Line Endings to LF (CRLF → LF Conversion)#671

Open
vedantvakharia wants to merge 1 commit intoopenclimatefix:mainfrom
vedantvakharia:fix/normalize-line-endings-to-lf
Open

Fix: Normalize All Line Endings to LF (CRLF → LF Conversion)#671
vedantvakharia wants to merge 1 commit intoopenclimatefix:mainfrom
vedantvakharia:fix/normalize-line-endings-to-lf

Conversation

@vedantvakharia
Copy link
Copy Markdown

Problem

The repository has inconsistent line endings (mix of CRLF and LF), causing CI failures for contributors using Windows. Prettier on Linux CI expects LF but finds CRLF, producing thousands of "Delete " errors that block pull requests.

Root Cause

  1. Original development on Windows → Files saved with CRLF line endings (\r\n)
  2. No .gitattributes file → Git didn't auto-normalize line endings
  3. No endOfLine in Prettier config → Platform-dependent behavior
  4. Windows contributors commit CRLF → Linux CI fails with Prettier errors

Example CI Error

./components/tooltip.tsx
1:33  Error: Delete `␍`  prettier/prettier
2:30  Error: Delete `␍`  prettier/prettier
3:30  Error: Delete `␍`  prettier/prettier
...
(thousands more errors)

Solution

This PR performs a one-time normalization of all line endings to LF (Linux/Mac standard):

Changes Made

1. Added .gitattributes

  • Forces Git to convert all text files to LF on commit
  • Works automatically for contributors on all platforms (Windows/Mac/Linux)
  • Explicit file type declarations ensure consistent handling
* text=auto eol=lf
*.ts text eol=lf
*.tsx text eol=lf
*.js text eol=lf
# ... more file types

2. Updated Prettier Configuration

  • Set endOfLine: "lf" explicitly in both nowcasting-app and quartz-app
  • Ensures consistent behavior across all platforms
  • Located in apps/*/package.json files

Before:

"prettier": {
  "trailingComma": "none",
  "tabWidth": 2,
  "singleQuote": false,
  "printWidth": 100
}

After:

"prettier": {
  "endOfLine": "lf",
  "trailingComma": "none",
  "tabWidth": 2,
  "singleQuote": false,
  "printWidth": 100
}

3. Added .editorconfig

  • Modern editors (VS Code, WebStorm, etc.) respect this automatically
  • Prevents accidental CRLF in new files
  • Provides consistent settings for all contributors
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
indent_style = space
indent_size = 2

4. Added .prettierignore

  • Excludes large data files from formatting
  • Keeps GeoJSON files minified for performance
  • Prevents unnecessary repository bloat
# Ignore large data files
apps/nowcasting-app/data/
apps/quartz-app/data/

5. Converted All Text Files

  • Method: Prettier with --write flag
  • Files affected: 58 text files
  • File types: .ts, .tsx, .js, .jsx, .json, .md, .css, .yaml, .yml
  • Data files: Kept minified (excluded from conversion)
  • Verification: Zero actual code changes (verified with git diff -w)

Large Data Files Excluded

Intentionally NOT formatted to keep them minified for performance:

  1. apps/nowcasting-app/data/GSP_regions_4326_20250109.json (~14MB GeoJSON)
  2. apps/nowcasting-app/data/dno_license_areas_20200506.json (DNO license boundaries)
  3. apps/nowcasting-app/data/ng_zones.json (National Grid zones)

Rationale:

  • These are large geographic data files (GeoJSON) with 355,000+ lines of coordinates
  • They are programmatically generated, never manually edited
  • Formatting them would add ~480,000 unnecessary line changes
  • Minified format is faster to load and reduces repository size
  • .prettierignore now excludes apps/*/data/ to prevent future formatting

Impact

Fixes

  • CI no longer fails with Prettier CRLF errors
  • Windows contributors can now contribute without line ending issues
  • All developers have consistent experience regardless of OS
  • Future line ending issues prevented automatically

Changes

  • Modified files: 58 text files
  • Line changes: 91,317 insertions(+), 364,694 deletions(-)
  • Actual code changes: None (only invisible line ending characters)
  • Repository optimization: Large data files kept minified

Performance Benefits

  • Smaller repo: Data files remain minified (~480,000 line changes avoided)
  • Faster operations: Git operations not slowed by large formatted files
  • Standard compliance: Follows Unix/Linux standard (LF)

Review Guide

This PR is large but simple:

  • Every line of code files shows as "changed" (but only the invisible line ending character)
  • Use GitHub's "Hide whitespace" option to review: Add ?w=1 to the PR URL
  • With whitespace hidden, diff should show only the 4 new/modified config files:
    • .gitattributes (new)
    • .editorconfig (new)
    • .prettierignore (new)
    • apps/*/package.json (modified - added endOfLine)

Breaking Changes

None. This is purely a whitespace change. Code functionality is completely unchanged.

  • No API changes
  • No behavior changes
  • No dependency changes
  • Only invisible line ending characters modified

Note: This is a critical infrastructure improvement that will benefit all current and future contributors by eliminating a major source of CI failures.

- Add .gitattributes to enforce LF line endings in Git
- Update Prettier config to enforce LF endings (endOfLine: 'lf')
- Add .editorconfig for editor consistency
- Add .prettierignore to exclude large data files
- Convert all text files from CRLF to LF using Prettier
- Fixes CI Prettier failures on Linux runners

This is a one-time infrastructure fix that:
- Enables Windows developers to contribute without CI failures
- Reduces repository size by excluding data files from formatting
- Follows Unix/Linux standard (LF)
- Prevents future line ending inconsistencies

Technical details:
- Changed files: 58 text files
- Total line changes: ~91,000 insertions, ~365,000 deletions
- Git diff with -w flag shows no actual code changes
- Used Prettier for safe, consistent conversion
- Large data files (GeoJSON) kept minified for performance

Breaking changes: None (purely whitespace)
@vercel
Copy link
Copy Markdown

vercel bot commented Dec 31, 2025

@vedantvakharia is attempting to deploy a commit to the Open Climate Fix Team on Vercel.

A member of the Team first needs to authorize it.

@vedantvakharia
Copy link
Copy Markdown
Author

Known CI Issues (Pre-existing)

Note: Cypress tests are currently failing due to missing Cypress Cloud configuration. This is a pre-existing issue unrelated to this PR.

The CRLF fix has been verified to work correctly:

  • Prettier checks pass locally
  • Linting passes
  • Build passes
  • Unit tests pass (89/89)
  • Cypress Cloud recording (see #XXX for fix)

The Cypress issue is being addressed separately in PR #672 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant