Summary
In scitex/io/_save.py, _determine_save_path deliberately rewrites relative paths based on calling context (script / Jupyter / IPython / interactive). The user-supplied path is never honoured, but no warning identifies the redirect and the verbose log obfuscates the actual target.
Mapping (from _save.py:_determine_save_path)
| Context |
Relative "foo/bar.pkl" resolves to |
| Jupyter |
<notebook>_out/foo/bar.pkl |
Script (python my_script.py) |
<script_name>_out/foo/bar.pkl |
IPython / python -c / interactive |
/tmp/$USER/foo/bar.pkl |
| Other |
<cwd>/output/foo/bar.pkl |
User-facing problem
In standalone scripts (slurm jobs, ad-hoc pipelines, ones outside @stx.session), users pass relative paths expecting pandas.to_csv-like behaviour. The redirect is invisible:
```
$ python my_script.py --out features/p01.pkl
SUCC: Saved to: ./../../../../../../some/scripts/dir/my_script_out/features/p01.pkl (920 KB)
$ ls features/
ls: cannot access 'features/': No such file or directory
```
The user thinks features/p01.pkl was saved at the working directory. It was not.
Three concrete improvements
- Print the absolute path in the SUCC message, not a relative-traversal string:
```
SUCC: Saved to: /scripts/dir/my_script_out/features/p01.pkl (920 KB)
```
- Note the redirect when it happens — e.g.:
```
INFO: relative path "features/p01.pkl" redirected to "" (script-context auto-output)
```
- Document the mapping in
stx.io.save's docstring — currently the docstring says nothing about path rewriting.
A fourth option (more invasive): add an absolute_path: bool = False kwarg or honour absolute paths verbatim (it already does this, but the docstring should make that explicit).
Why this matters during the migration
The redirect is intentional and pairs nicely with @stx.session reproducibility tracking. But standalone consumers of scitex-io (and ad-hoc scripts that just want pandas.to_csv-like behaviour) hit silent data loss patterns because the path was redirected and the consumer cannot find their file.
Cross-ref
Originally filed at scitex-io#26 — closed as misdirected since save() lives in scitex-python umbrella.
Summary
In
scitex/io/_save.py,_determine_save_pathdeliberately rewrites relative paths based on calling context (script / Jupyter / IPython / interactive). The user-supplied path is never honoured, but no warning identifies the redirect and the verbose log obfuscates the actual target.Mapping (from
_save.py:_determine_save_path)"foo/bar.pkl"resolves to<notebook>_out/foo/bar.pklpython my_script.py)<script_name>_out/foo/bar.pklpython -c/ interactive/tmp/$USER/foo/bar.pkl<cwd>/output/foo/bar.pklUser-facing problem
In standalone scripts (slurm jobs, ad-hoc pipelines, ones outside
@stx.session), users pass relative paths expectingpandas.to_csv-like behaviour. The redirect is invisible:```
$ python my_script.py --out features/p01.pkl
SUCC: Saved to: ./../../../../../../some/scripts/dir/my_script_out/features/p01.pkl (920 KB)
$ ls features/
ls: cannot access 'features/': No such file or directory
```
The user thinks
features/p01.pklwas saved at the working directory. It was not.Three concrete improvements
```
SUCC: Saved to: /scripts/dir/my_script_out/features/p01.pkl (920 KB)
```
```
INFO: relative path "features/p01.pkl" redirected to "" (script-context auto-output)
```
stx.io.save's docstring — currently the docstring says nothing about path rewriting.A fourth option (more invasive): add an
absolute_path: bool = Falsekwarg or honour absolute paths verbatim (it already does this, but the docstring should make that explicit).Why this matters during the migration
The redirect is intentional and pairs nicely with
@stx.sessionreproducibility tracking. But standalone consumers ofscitex-io(and ad-hoc scripts that just wantpandas.to_csv-like behaviour) hit silent data loss patterns because the path was redirected and the consumer cannot find their file.Cross-ref
Originally filed at scitex-io#26 — closed as misdirected since
save()lives in scitex-python umbrella.