Sporadic `make execute` error: `ERROR: No such file or directory: '<REPO_ROOT>/index.html'`

[lanery]

This error (traceback below) has happened sporadically a number of times now upon running make execute. I haven't done a great job tracking the stats, but I'd say it happens about 25% of the time I run make execute now that my notebook is pretty hefty. I don't recall it happening a few weeks ago when the notebook was a lot slimmer.

Usually (sorry for not having stats here either), it successfully executes on the next run. When it does execute successfully, the traceback is exactly the same up until the error. The reason I'm posting the issue here is that it's odd/suspicious that it's looking for the index.html in the root directory of the repo as opposed to in _site/ where it belongs.

To put some numbers on hefty, index.ipynb and _site/index.html are both ~18 MB.

Full traceback:

(2025-raman-spectrometer-comparison) ryanlane@Ryans-MacBook-Pro-2 2025-raman-spectrometer-comparison % make execute
# Project-wide render without executing code cells. Instead, rely on
# pre-computed results present in _freeze/
quarto render
[1/4] CONTRIBUTING.qmd
[2/4] FAQ.qmd
[3/4] SETUP.qmd
[4/4] index.ipynb

Output created: _site/index.html

# Now, render `index.ipynb`, with code execution. This will populate
# _freeze/index/ with pre-computed results.
quarto render index.ipynb --execute

Starting python3 kernel...Done

Executing 'index.ipynb'
  Cell 1/11: ''.........................Done
  Cell 2/11: 'fig-acetonitrile'.........Done
  Cell 3/11: 'tbl-acetonitrile'.........Done
  Cell 4/11: ''.........................Done
  Cell 5/11: 'fig-cc124'................Done
  Cell 6/11: 'tbl-chlamy'...............Done
  Cell 7/11: 'fig-batchpreprocessing'...Done
  Cell 8/11: ''.........................Done
  Cell 9/11: 'fig-lda'..................Done
  Cell 10/11: ''.........................Done
  Cell 11/11: 'fig-classification'.......Done

pandoc
  to: html
  output-file: index.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png
  reference-location: margin
  css:
    - assets/css/main.css
  toc: true

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  comments:
    giscus:
      repo: Arcadia-Science/notebook-pub-template
      input-position: top
  bibliography:
    - ref.bib
  csl: assets/arcadia.csl
  code-annotations: hover
  page-layout: article
  title-block-banner: true
  toc-expand: 2
  title: '{{< var pub.title >}} --'
  date: March 4, 2025
  abstract-title: Summary

ERROR: NotFound: No such file or directory (os error 2): readfile '/Users/ryanlane/Repositories/2025-raman-spectrometer-comparison/index.html'
Path: /Users/ryanlane/Repositories/2025-raman-spectrometer-comparison/index.html

Stack trace:
Path: /Users/ryanlane/Repositories/2025-raman-spectrometer-comparison/index.html
    at readTextFileSync (ext:deno_fs/30_fs.js:864:10)
    at Object.Deno.readTextFileSync (file:///Applications/quarto/bin/quarto.js:5149:25)
    at file:///Applications/quarto/bin/quarto.js:78454:36
    at withTimingAsync (file:///Applications/quarto/bin/quarto.js:16879:31)
    at runHtmlPostprocessors (file:///Applications/quarto/bin/quarto.js:78452:15)
    at Object.complete (file:///Applications/quarto/bin/quarto.js:78290:49)
    at Object.onPostProcess (file:///Applications/quarto/bin/quarto.js:85808:53)
    at renderFileInternal (file:///Applications/quarto/bin/quarto.js:85792:26)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async renderFiles (file:///Applications/quarto/bin/quarto.js:85604:17)
make: *** [execute] Error 1

[ekiefl]

Since it happens somewhat reliably, can you experiment with running rm -rf _site; make execute instead of make execute and see if continues to happen? It's a bit of a shot in the dark, but my intuition is telling me this might fix it.

[ekiefl]

Any update on this bug?

[lanery]

I've run rm -rf _site; make execute 5 or 6 times now without error. So that's good! At the same time, I also changed my deadliest plotly figure to render with about 1/3 the number of traces. So while good, it's maybe a bit inconclusive.

Can't really make it a priority now, but I thought of making a test branch for my repo and running these commands every 15 min or so in a bash script overnight.