Skip to content

UX: scitex.io.save() silently rewrites relative paths based on calling context #277

@ywatanabe1989

Description

@ywatanabe1989

Summary

In scitex/io/_save.py, _determine_save_path deliberately rewrites relative paths based on calling context (script / Jupyter / IPython / interactive). The user-supplied path is never honoured, but no warning identifies the redirect and the verbose log obfuscates the actual target.

Mapping (from _save.py:_determine_save_path)

Context Relative "foo/bar.pkl" resolves to
Jupyter <notebook>_out/foo/bar.pkl
Script (python my_script.py) <script_name>_out/foo/bar.pkl
IPython / python -c / interactive /tmp/$USER/foo/bar.pkl
Other <cwd>/output/foo/bar.pkl

User-facing problem

In standalone scripts (slurm jobs, ad-hoc pipelines, ones outside @stx.session), users pass relative paths expecting pandas.to_csv-like behaviour. The redirect is invisible:

```
$ python my_script.py --out features/p01.pkl
SUCC: Saved to: ./../../../../../../some/scripts/dir/my_script_out/features/p01.pkl (920 KB)
$ ls features/
ls: cannot access 'features/': No such file or directory
```

The user thinks features/p01.pkl was saved at the working directory. It was not.

Three concrete improvements

  1. Print the absolute path in the SUCC message, not a relative-traversal string:
    ```
    SUCC: Saved to: /scripts/dir/my_script_out/features/p01.pkl (920 KB)
    ```
  2. Note the redirect when it happens — e.g.:
    ```
    INFO: relative path "features/p01.pkl" redirected to "" (script-context auto-output)
    ```
  3. Document the mapping in stx.io.save's docstring — currently the docstring says nothing about path rewriting.

A fourth option (more invasive): add an absolute_path: bool = False kwarg or honour absolute paths verbatim (it already does this, but the docstring should make that explicit).

Why this matters during the migration

The redirect is intentional and pairs nicely with @stx.session reproducibility tracking. But standalone consumers of scitex-io (and ad-hoc scripts that just want pandas.to_csv-like behaviour) hit silent data loss patterns because the path was redirected and the consumer cannot find their file.

Cross-ref

Originally filed at scitex-io#26 — closed as misdirected since save() lives in scitex-python umbrella.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions