From 2560fccd40153f9a35e7232e865b3dcd5affaac7 Mon Sep 17 00:00:00 2001 From: Joel Adams Date: Wed, 18 Feb 2026 16:03:15 +0000 Subject: [PATCH 1/2] refactor docs `.rst` files to `.md` --- docs/animation.md | 248 +++++++++++++++++++++++++++ docs/animation.rst | 261 ----------------------------- docs/{api.rst => api.md} | 6 +- docs/conf.py | 5 +- docs/contributing.md | 2 + docs/contributing.rst | 4 - docs/getting_started.md | 61 +++++++ docs/getting_started.rst | 75 --------- docs/index.md | 32 ++++ docs/index.rst | 33 ---- docs/key_functionality.md | 313 ++++++++++++++++++++++++++++++++++ docs/key_functionality.rst | 335 ------------------------------------- docs/known_issues.md | 11 ++ docs/known_issues.rst | 9 - docs/unit_conversion.md | 201 ++++++++++++++++++++++ docs/unit_conversion.rst | 217 ------------------------ 16 files changed, 875 insertions(+), 938 deletions(-) create mode 100644 docs/animation.md delete mode 100644 docs/animation.rst rename docs/{api.rst => api.md} (70%) create mode 100644 docs/contributing.md delete mode 100644 docs/contributing.rst create mode 100644 docs/getting_started.md delete mode 100644 docs/getting_started.rst create mode 100644 docs/index.md delete mode 100644 docs/index.rst create mode 100644 docs/key_functionality.md delete mode 100644 docs/key_functionality.rst create mode 100644 docs/known_issues.md delete mode 100644 docs/known_issues.rst create mode 100644 docs/unit_conversion.md delete mode 100644 docs/unit_conversion.rst diff --git a/docs/animation.md b/docs/animation.md new file mode 100644 index 0000000..6236923 --- /dev/null +++ b/docs/animation.md @@ -0,0 +1,248 @@ +# Animations + +[`xarray.DataArray.epoch.animate`](project:#sdf_xarray.plotting.animate) +creates a ; it is designed to +mimic . + +```{jupyter-execute} +import sdf_xarray as sdfxr +import xarray as xr +import matplotlib.pyplot as plt +from matplotlib.animation import FuncAnimation +from IPython.display import HTML +``` + +## Basic usage + +The type of plot that is animated is determined by the dimensionality of the + object. + +```{note} +`time` is considered a dimension in the same way as spatial co-ordinates, so 1D time +resolved data has 2 dimensions. +``` + +```{csv-table} +:header: > +: "Dimensions", "Plotting function", "Notes" +:widths: "auto" + +`2`, , "" +`3`, , "" +`>3`, , "Not fully implemented" +``` + +### 1D simulation + +We can animate a variable of a 1D simulation in the following way. +It is important to note that since the dataset is time resolved, it has +2 dimensions. + +```{warning} +`anim.show()` will only show the animation in a Jupyter notebook. +``` + +```{jupyter-execute} +# Open the SDF files +ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") + +# Access a DataArray within the Dataset +da = ds["Derived_Number_Density_Electron"] + +# Create the FuncAnimation object +anim = da.epoch.animate() + +# Display animation as jshtml +anim.show() +``` + +````{tip} +The animations can be saved with +```bash +anim.save("path/to/save/animation.gif") +``` + +where `.gif` can be replaced with any supported file format. + +It can also be viewed from a Python interpreter with: +```bash +fig, ax = plt.subplots() +anim = da.epoch.animate(ax=ax) +plt.show() +``` +```` + +### 2D simulation + +Plotting a 2D simulation can be done in exactly the same way. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") +da = ds["Derived_Number_Density_Electron"] +anim = da.epoch.animate() +anim.show() +``` + +We can also take a lineout of a 2D simulation to create 2D data and +plot it as a . + +```{jupyter-execute} +da = ds["Derived_Number_Density_Electron"] +da_lineout = da.sel(Y_Grid_mid = 1e-6, method = "nearest") +anim = da_lineout.epoch.animate(title = "Y = 1e-6 [m]") +anim.show() +``` + +### 3D simulation + +Opening a 3D simulation as a multi-file dataset and plotting it will +return a . However, this may not be +desirable. We can plot a 3D simulation along a certain plane in the +same way a 2D simulation can be plotted along a line. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_3d/*.sdf") + +da = ds["Derived_Number_Density"] +da_lineout = da.sel(Y_Grid_mid = 0, method="nearest") +anim = da_lineout.epoch.animate(title = "Y = 0 [m]", fps = 2) +anim.show() +``` + +A single SDF file can be animated by changing the time coordinate of +the animation. + +```{jupyter-execute} +ds = sdfxr.open_dataset("tutorial_dataset_3d/0005.sdf") +da = ds["Derived_Number_Density"] +anim = da.epoch.animate(t = "X_Grid_mid") +anim.show() +``` + +## Moving window + +EPOCH allows for simulations that have a moving simulation window +(changing x-axis over time). [`xarray.DataArray.epoch.animate`](project:#sdf_xarray.plotting.animate) can accept the boolean parameter +`move_window` and change the x-axis limits accordingly. + +```{warning} + does not currently function with moving window data. +You must use and specify arguments in the following way. +``` + +```{jupyter-execute} +ds = xr.open_mfdataset( + "tutorial_dataset_2d_moving_window/*.sdf", + preprocess = sdfxr.SDFPreprocess(), + combine = "nested", + join = "outer", + compat="no_conflicts", + concat_dim="time", + ) + +da = ds["Derived_Number_Density_Beam_Electrons"] +anim = da.epoch.animate(move_window=True, fps = 5) +anim.show() +``` + +```{warning} +Importing some datasets with moving windows can cause vertical banding +in the , which will affect the animation. The cause for +this is unknown but can be circumvented by setting `join = "override"`. +``` + +## Customisation + +The animation can be customised in much the same way as , +see [`xarray.DataArray.epoch.animate`](project:#sdf_xarray.plotting.animate) for more details. +The coordinate units can be converted before plotting as in [](./unit_conversion.md#unit-conversion). +Some functionality such as `aspect` and `size` are not fully implemented yet. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") + +# Change the units of the coordinates +ds = ds.epoch.rescale_coords(1e6, "µm", ["X_Grid_mid", "Y_Grid_mid"]) +ds = ds.epoch.rescale_coords(1e15, "fs", ["time"]) +ds["time"].attrs["long_name"] = "t" + +# Change units and name of the variable +da = ds["Derived_Number_Density_Electron"] +da.data = da.values * 1e-6 +da.attrs["units"] = "cm$^{-3}$" +da.attrs["long_name"] = "$n_e$" + +anim = da.epoch.animate( + fps = 2, + max_percentile = 95, + title = "Target A", + cmap = "plasma", + ) +anim.show() +``` + +## Combining multiple animations + +[`xarray.Dataset.epoch.animate_multiple`](project:#sdf_xarray.plotting.animate_multiple) creates a +that contains multiple plots layered on top of each other. + +### 1D simulation + +What follows is an example of how to combine multiple animations on the +same axis. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") + +anim = ds.epoch.animate_multiple( + ds["Derived_Number_Density_Electron"], + ds["Derived_Number_Density_Ion"], + datasets_kwargs=[{"label": "Electron"}, {"label": "Ion"}], + ylim=(0e27,4e27), + ylabel="Derived Number Density [1/m$^3$]" +) + +anim.show() +``` + +### 2D simulation + +```{tip} +To correctly display 2D data on top of one another you need to specify +the `alpha` value which sets the opacity of the plot. +``` + +This also works with 2 dimensional data. + +```{jupyter-execute} +import numpy as np +from matplotlib.colors import LogNorm + +ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") + +flux_magnitude = np.sqrt( + ds["Derived_Poynting_Flux_x"]**2 + + ds["Derived_Poynting_Flux_y"]**2 + + ds["Derived_Poynting_Flux_z"]**2 +) +flux_magnitude.attrs["long_name"] = "Poynting Flux Magnitude" +flux_magnitude.attrs["units"] = "W/m$^2$" + +# Cut-off low energy values so that they will be rendered as transparent +# in the plot as they've been set to NaN +flux_masked = flux_magnitude.where(flux_magnitude > 0.2e23) +flux_norm = LogNorm( + vmin=float(flux_masked.min()), + vmax=float(flux_masked.max()) +) + +anim = ds.epoch.animate_multiple( + ds["Derived_Number_Density_Electron"], + flux_masked, + datasets_kwargs=[ + {"alpha": 1.0}, + {"cmap": "hot", "norm": flux_norm, "alpha": 0.9}, + ], +) +anim.show() +``` diff --git a/docs/animation.rst b/docs/animation.rst deleted file mode 100644 index 92dc31c..0000000 --- a/docs/animation.rst +++ /dev/null @@ -1,261 +0,0 @@ -.. _sec-animation: - -.. |animate_accessor| replace:: `xarray.DataArray.epoch.animate - ` - -.. |animate_multiple_accessor| replace:: `xarray.Dataset.epoch.animate_multiple - ` - -========== -Animations -========== - -|animate_accessor| creates a `matplotlib.animation.FuncAnimation`; it is -designed to mimic `xarray.DataArray.plot`. - -.. jupyter-execute:: - - import sdf_xarray as sdfxr - import xarray as xr - import matplotlib.pyplot as plt - from matplotlib.animation import FuncAnimation - from IPython.display import HTML - -Basic usage ------------ - -The type of plot that is animated is determined by the dimensionality of the `xarray.DataArray` object. - -.. note:: - ``time`` is considered a dimension in the same way as spatial co-ordinates, so 1D time - resolved data has 2 dimensions. - -.. csv-table:: - :header: "Dimensions", "Plotting function", "Notes" - :widths: auto - :align: center - - "2", "`xarray.plot.line`", "" - "3", "`xarray.plot.pcolormesh`", "" - ">3", "`xarray.plot.hist`", "Not fully implemented" - - -1D simulation -~~~~~~~~~~~~~ - -We can animate a variable of a 1D simulation in the following way. -It is important to note that since the dataset is time resolved, it has -2 dimensions. - -.. warning:: - ``anim.show()`` will only show the animation in a Jupyter notebook. - -.. jupyter-execute:: - - # Open the SDF files - ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") - - # Access a DataArray within the Dataset - da = ds["Derived_Number_Density_Electron"] - - # Create the FuncAnimation object - anim = da.epoch.animate() - - # Display animation as jshtml - anim.show() - -.. tip:: - The animations can be saved with - - .. code-block:: bash - - anim.save("path/to/save/animation.gif") - - where ``.gif`` can be replaced with any supported file format. - - It can also be viewed from a Python interpreter with: - - .. code-block:: bash - - fig, ax = plt.subplots() - anim = da.epoch.animate(ax=ax) - plt.show() - -2D simulation -~~~~~~~~~~~~~ - -Plotting a 2D simulation can be done in exactly the same way. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") - da = ds["Derived_Number_Density_Electron"] - anim = da.epoch.animate() - anim.show() - -We can also take a lineout of a 2D simulation to create 2D data and -plot it as a `xarray.plot.line`. - -.. jupyter-execute:: - - da = ds["Derived_Number_Density_Electron"] - da_lineout = da.sel(Y_Grid_mid = 1e-6, method = "nearest") - anim = da_lineout.epoch.animate(title = "Y = 1e-6 [m]") - anim.show() - -3D simulation -~~~~~~~~~~~~~ - -Opening a 3D simulation as a multi-file dataset and plotting it will -return a `xarray.plot.hist`. However, this may not be -desirable. We can plot a 3D simulation along a certain plane in the -same way a 2D simulation can be plotted along a line. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_3d/*.sdf") - - da = ds["Derived_Number_Density"] - da_lineout = da.sel(Y_Grid_mid = 0, method="nearest") - anim = da_lineout.epoch.animate(title = "Y = 0 [m]", fps = 2) - anim.show() - -A single SDF file can be animated by changing the time coordinate of -the animation. - -.. jupyter-execute:: - - ds = sdfxr.open_dataset("tutorial_dataset_3d/0005.sdf") - da = ds["Derived_Number_Density"] - anim = da.epoch.animate(t = "X_Grid_mid") - anim.show() - -Moving window -------------- - -EPOCH allows for simulations that have a moving simulation window -(changing x-axis over time). |animate_accessor| can accept the boolean parameter -``move_window`` and change the x-axis limits accordingly. - -.. warning:: - `sdf_xarray.open_mfdataset` does not currently function with moving window data. - You must use `xarray.open_mfdataset` and specify arguments in the following way. - -.. jupyter-execute:: - - ds = xr.open_mfdataset( - "tutorial_dataset_2d_moving_window/*.sdf", - preprocess = sdfxr.SDFPreprocess(), - combine = "nested", - join = "outer", - compat="no_conflicts", - concat_dim="time", - ) - - da = ds["Derived_Number_Density_Beam_Electrons"] - anim = da.epoch.animate(move_window=True, fps = 5) - anim.show() - -.. warning:: - Importing some datasets with moving windows can cause vertical banding - in the `xarray.Dataset`, which will affect the animation. The cause for - this is unknown but can be circumvented by setting ``join = "override"``. - -Customisation -------------- - -The animation can be customised in much the same way as `xarray.DataArray.plot`, -see |animate_accessor| for more details. The coordinate units can be converted -before plotting as in :ref:`sec-unit-conversion`. Some functionality such as -``aspect`` and ``size`` are not fully implemented yet. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") - - # Change the units of the coordinates - ds = ds.epoch.rescale_coords(1e6, "µm", ["X_Grid_mid", "Y_Grid_mid"]) - ds = ds.epoch.rescale_coords(1e15, "fs", ["time"]) - ds["time"].attrs["long_name"] = "t" - - # Change units and name of the variable - da = ds["Derived_Number_Density_Electron"] - da.data = da.values * 1e-6 - da.attrs["units"] = "cm$^{-3}$" - da.attrs["long_name"] = "$n_e$" - - anim = da.epoch.animate( - fps = 2, - max_percentile = 95, - title = "Target A", - cmap = "plasma", - ) - anim.show() - -Combining multiple animations ------------------------------ - -|animate_multiple_accessor| creates a `matplotlib.animation.FuncAnimation` -that contains multiple plots layered on top of each other. - -1D simulation -~~~~~~~~~~~~~ - -What follows is an example of how to combine multiple animations on the -same axis. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") - - anim = ds.epoch.animate_multiple( - ds["Derived_Number_Density_Electron"], - ds["Derived_Number_Density_Ion"], - datasets_kwargs=[{"label": "Electron"}, {"label": "Ion"}], - ylim=(0e27,4e27), - ylabel="Derived Number Density [1/m$^3$]" - ) - - anim.show() - -2D simulation -~~~~~~~~~~~~~ - -.. tip:: - To correctly display 2D data on top of one another you need to specify - the ``alpha`` value which sets the opacity of the plot. - -This also works with 2 dimensional data. - -.. jupyter-execute:: - - import numpy as np - from matplotlib.colors import LogNorm - - ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") - - flux_magnitude = np.sqrt( - ds["Derived_Poynting_Flux_x"]**2 + - ds["Derived_Poynting_Flux_y"]**2 + - ds["Derived_Poynting_Flux_z"]**2 - ) - flux_magnitude.attrs["long_name"] = "Poynting Flux Magnitude" - flux_magnitude.attrs["units"] = "W/m$^2$" - - # Cut-off low energy values so that they will be rendered as transparent - # in the plot as they've been set to NaN - flux_masked = flux_magnitude.where(flux_magnitude > 0.2e23) - flux_norm = LogNorm( - vmin=float(flux_masked.min()), - vmax=float(flux_masked.max()) - ) - - anim = ds.epoch.animate_multiple( - ds["Derived_Number_Density_Electron"], - flux_masked, - datasets_kwargs=[ - {"alpha": 1.0}, - {"cmap": "hot", "norm": flux_norm, "alpha": 0.9}, - ], - ) - anim.show() \ No newline at end of file diff --git a/docs/api.rst b/docs/api.md similarity index 70% rename from docs/api.rst rename to docs/api.md index 2fc966d..e4c8462 100644 --- a/docs/api.rst +++ b/docs/api.md @@ -1,10 +1,10 @@ -=============== - API Reference -=============== +# API Reference +```{eval-rst} .. autosummary:: :toctree: generated :template: custom-module-template.rst :recursive: sdf_xarray +``` diff --git a/docs/conf.py b/docs/conf.py index f4adb55..1131b56 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,6 +53,9 @@ "sphinx_togglebutton", ] +source_suffix = {".rst": "restructuredtext", ".md": "markdown"} +myst_heading_anchors = 3 + autosummary_generate = True # Add any paths that contain templates here, relative to this directory. @@ -126,7 +129,7 @@ "scipy": ("https://docs.scipy.org/doc/scipy", None), "xarray": ("https://docs.xarray.dev/en/latest", None), "pint": ("https://pint.readthedocs.io/en/stable", None), - "pint-xarray": ("https://pint-xarray.readthedocs.io/en/stable", None), + "pint_xarray": ("https://pint-xarray.readthedocs.io/en/stable", None), "pooch": ("https://www.fatiando.org/pooch/latest", None), "matplotlib": ("https://matplotlib.org/stable", None), } diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..78caf34 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,2 @@ +```{include} ../CONTRIBUTING.md +``` diff --git a/docs/contributing.rst b/docs/contributing.rst deleted file mode 100644 index f3de7ba..0000000 --- a/docs/contributing.rst +++ /dev/null @@ -1,4 +0,0 @@ -.. _sec-contributing: - -.. include:: ../CONTRIBUTING.md - :parser: myst_parser.sphinx_ diff --git a/docs/getting_started.md b/docs/getting_started.md new file mode 100644 index 0000000..177d9b1 --- /dev/null +++ b/docs/getting_started.md @@ -0,0 +1,61 @@ +# Getting Started + +## Installation + +```{important} +To install this package, ensure that you are using one of the supported Python +versions [![Supported Python versions](https://img.shields.io/pypi/pyversions/sdf-xarray.svg)](https://github.com/epochpic/sdf-xarray) +``` + +Install sdf-xarray from PyPI with: + +```bash +pip install sdf-xarray +``` + +or download this code locally: + +```bash +git clone --recursive https://github.com/epochpic/sdf-xarray.git +cd sdf-xarray +pip install . +``` + +## Interaction + +There are two main ways to load EPOCH SDF files into xarray objects: using the dedicated + functions or using the standard interface with our custom engine. +For examples of how to use these functions see [](./key_functionality.md#loading-sdf-files). + +All code examples throughout this documentation are visualised using Jupyter notebooks +so that you can interactively explore the datasets. To do this on your machine make +sure that you have the necessary dependencies installed: + +```bash +pip install "sdf-xarray[jupyter]" +``` + +```{important} +When loading SDF files, variables related to `boundaries`, `cpu` and `output file` +are excluded as they are problematic. If you wish to load these variables in see +[](./key_functionality.md#loading-raw-files). +``` + +### Using sdf_xarray (Recommended) + +These functions are wrappers designed specifically for SDF data, providing the most +straightforward experience: + +- **Single files**: Use or +- **Multiple files**: Use or +- **Raw files**: use + +### Using xarray + +If you prefer using the native functions, you can use the , + and . Strangely there is no function in + for `xarray.open_mfdatatree`. + +These functions should all work out of the box as long as is installed on your +system, if you are having issues with it reading files, you might need to pass the parameter +`engine=sdf_engine` when calling any of the above xarray functions. diff --git a/docs/getting_started.rst b/docs/getting_started.rst deleted file mode 100644 index 05b244b..0000000 --- a/docs/getting_started.rst +++ /dev/null @@ -1,75 +0,0 @@ -.. _sec-getting-started: - -================= - Getting Started -================= - -Installation ------------- - -.. |python_versions_pypi| image:: https://img.shields.io/pypi/pyversions/sdf-xarray.svg - :alt: Supported Python versions - :target: https://pypi.org/project/sdf-xarray/ - -.. important:: - - To install this package, ensure that you are using one of the supported Python - versions |python_versions_pypi| - -Install sdf-xarray from PyPI with: - -.. code-block:: bash - - pip install sdf-xarray - -or download this code locally: - -.. code-block:: bash - - git clone --recursive https://github.com/epochpic/sdf-xarray.git - cd sdf-xarray - pip install . - - -Interaction ------------ - -There are two main ways to load EPOCH SDF files into xarray objects: using the dedicated -`sdf_xarray` functions or using the standard `xarray` interface with our custom engine. -For examples of how to use these functions see :ref:`loading-sdf-files`. - -All code examples throughout this documentation are visualised using Jupyter notebooks -so that you can interactively explore the datasets. To do this on your machine make -sure that you have the necessary dependencies installed: - -.. code-block:: bash - - pip install "sdf-xarray[jupyter]" - -.. important:: - - When loading SDF files, variables related to ``boundaries``, ``cpu`` and ``output file`` - are excluded as they are problematic. If you wish to load these variables in see - :ref:`loading-raw-files`. - - -Using sdf_xarray (Recommended) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -These functions are wrappers designed specifically for SDF data, providing the most -straightforward experience: - -- **Single files**: Use `sdf_xarray.open_dataset` or `sdf_xarray.open_datatree` -- **Multiple files**: Use `sdf_xarray.open_mfdataset` or `sdf_xarray.open_mfdatatree` -- **Raw files**: use `sdf_xarray.sdf_interface.SDFFile` - -Using xarray -~~~~~~~~~~~~ - -If you prefer using the native `xarray` functions, you can use the `xarray.open_dataset`, -`xarray.open_datatree` and `xarray.open_mfdataset`. Strangely there is no function in -`xarray` for ``xarray.open_mfdatatree``. - -These functions should all work out of the box as long as `sdf_xarray` is installed on your -system, if you are having issues with it reading files, you might need to pass the parameter -``engine=sdf_engine`` when calling any of the above xarray functions. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..4e4ac95 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,32 @@ +# sdf-xarray + +`sdf-xarray` provides a backend for [xarray](https://xarray.dev) to +read SDF files as created by the [EPOCH](https://epochpic.github.io) +plasma PIC code. + +`sdf-xarray` uses the [SDF-C](https://github.com/epochpic/SDF_C) library. + +```{toctree} +:caption: 'Contents' +:maxdepth: 1 + +getting_started.md +key_functionality.md +animation.md +unit_conversion.md +known_issues.md +contributing.md +``` + +````{toctree} +:caption: Reference +:maxdepth: 2 + +api.md +```` + +# Indices and tables + +- {ref}`genindex` +- {ref}`modindex` +- {ref}`search` diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 820a349..0000000 --- a/docs/index.rst +++ /dev/null @@ -1,33 +0,0 @@ -sdf-xarray -=========== - -``sdf-xarray`` provides a backend for `xarray `_ to -read SDF files as created by the `EPOCH `_ -plasma PIC code. - -``sdf-xarray`` uses the `SDF-C `_ library. - - -.. toctree:: - :maxdepth: 1 - :caption: Contents: - - Getting Started - Key Functionality - Animations - Unit Conversion - Known Issues - Contributing - -.. toctree:: - :maxdepth: 2 - :caption: Reference - - API Reference - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/key_functionality.md b/docs/key_functionality.md new file mode 100644 index 0000000..a64167d --- /dev/null +++ b/docs/key_functionality.md @@ -0,0 +1,313 @@ +# Key Functionality + +```{jupyter-execute} +import xarray as xr +import sdf_xarray as sdfxr +import matplotlib.pyplot as plt +``` + +## Loading SDF files + +### Loading single files + +```{jupyter-execute} +sdfxr.open_dataset("tutorial_dataset_1d/0010.sdf") +``` + +You can also load the data in as a , which organises the data +hierarchically into `groups` (for example grouping related quantities such as the individual +components of the electric and magnetic fields) while keeping each item as a . + +```{jupyter-execute} +sdfxr.open_datatree("tutorial_dataset_1d/0010.sdf") +``` + +(loading-raw-files)= + +### Loading raw files + +If you wish to load data directly from the `SDF.C` library and ignore +the interface layer. + +```{jupyter-execute} +raw_ds = sdfxr.SDFFile("tutorial_dataset_1d/0010.sdf") +raw_ds.variables.keys() +``` + +### Loading multiple files + +Multiple files can be loaded using one of two methods. The first of which +is by using the . + +```{tip} +If your simulation includes multiple `output` blocks that specify different variables +for output at various time steps, variables not present at a specific step will default +to a nan value. To remove these nan values we suggest using the +function or following our implmentation in [](#loading-sparse-data). +``` + +```{jupyter-execute} +sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") +``` + +Alternatively, files can be loaded using however when loading in +all the files we have do some processing of the data so that we can correctly align it along +the time dimension; This is done via the `preprocess` parameter utilising the + function. + +```{jupyter-execute} +xr.open_mfdataset( + "tutorial_dataset_1d/*.sdf", + join="outer", + compat="no_conflicts", + preprocess=sdfxr.SDFPreprocess() +) +``` + +You can also load the data in as a , which organises the data +hierarchically into `groups` (for example grouping related quantities such as the individual +components of the electric and magnetic fields) while keeping each item as a . + +```{jupyter-execute} +sdfxr.open_mfdatatree("tutorial_dataset_1d/*.sdf") +``` + +### Loading sparse data + +When dealing with sparse data (where different variables are saved at different, +non-overlapping time steps) you can optimize memory usage by loading the data with + using the parameter `separate_times=True`. This +approach creates a distinct time dimension for each output block, avoiding the +need for a single, large time dimension that would be filled with nan values. This +significantly reduces memory consumption, though it requires more deliberate handling +if you need to compare variables that exist on these different time coordinates. + +```{jupyter-execute} +sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf", separate_times=True) +``` + +### Loading particle data + +```{warning} +It is **not recommended** to use or + to load particle data from multiple +SDF outputs. The number of particles often varies between outputs, +which can lead to inconsistent array shapes that these functions +cannot handle. Instead, consider loading each file individually and +then concatenating them manually. +``` + +```{note} +When loading multiple probes from a single SDF file, you **must** use the +`probe_names` parameter to assign a unique name to each. For example, +use `probe_names=["Front_Electron_Probe", "Back_Electron_Probe"]`. +Failing to do so will result in dimension name conflicts. +``` + +By default, particle data isn't kept as it takes up a lot of space. +Pass `keep_particles=True` as a keyword argument to + (for single files) or (for +multiple files). + +```{jupyter-execute} +sdfxr.open_dataset("tutorial_dataset_1d/0010.sdf", keep_particles=True) +``` + +### Loading specific variables + +When loading datasets containing several (`>10`) coordinates/dimensions +using , may struggle to locate +the necessary RAM to concatenate all of the data (as seen in +[Issue #57](https://github.com/epochpic/sdf-xarray/issues/57)). +In this instance, you can optimize memory usage by loading only the data +you need using the keyword argument `data_vars` and passing one or more +variables. This creates a dataset consisting only of the given variable(s) +and the relevant coordinates/dimensions, significantly reducing memory +consumption. + +```{jupyter-execute} +sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf", data_vars=["Electric_Field_Ex"]) +``` + +(loading-input-deck)= + +### Loading the input.deck + +When loading SDF files, will attempt to automatically load +the `input.deck` file used to initialise the simulation from the same +directory as the SDF file. If the file is not found, it will silently fail +and continue loading the SDF file as normal. This file contains the initial +simulation setup information which is not present in SDF outputs. By loading +this file, you can access these parameters as part of your dataset's metadata. +To do this, use the `deck_path` parameter when loading an SDF file with +, , , +, or . + +There are a few ways you can load an input deck: + +- **Default behaviour**: The input deck is loaded from the same directory + as the SDF file if it exists. If it does not exist, it will silently fail. +- **Relative path**: (e.g. `"template.deck"`) Searches for that specific filename + within the same directory as the SDF file. +- **Absolute path**: (e.g. `"/path/to/input.deck"`) Uses the full, specified path + to locate the file. + +An example of loading a deck can be seen below + +````{toggle} +```{jupyter-execute} +import json +from IPython.display import Code + +ds = xr.open_dataset("tutorial_dataset_1d/0010.sdf") +# The results are accessible by calling +deck = ds.attrs["deck"] + +# Some prettification to make it looks nice in jupyter notebooks +json_str = json.dumps(deck, indent=4) +Code(json_str, language='json') +``` +```` + +## Data interaction examples + +When loading in either a single dataset or a group of datasets you +can access the following methods to explore the dataset: + +- `ds.variables` to list variables. (e.g. Electric Field, Magnetic + Field, Particle Count) +- `ds.coords` for accessing coordinates/dimensions. (e.g. x-axis, + y-axis, time) +- `ds.attrs` for metadata attached to the dataset. (e.g. filename, + step, time) + +It is important to note here that lazily loads the data +meaning that it only explicitly loads the results your currently +looking at when you call `.values` + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") + +ds["Electric_Field_Ex"] +``` + +On top of accessing variables you can plot these +using the built-in function (see +) which is +a simple call to `matplotlib`. This also means that you can access +all the methods from `matplotlib` to manipulate your plot. + +```{jupyter-execute} +# This is discretized in both space and time +ds["Electric_Field_Ex"].plot() +plt.title("Electric field along the x-axis") +plt.show() +``` + +When loading a multi-file dataset using , a +time dimension is automatically added to the resulting . +This dimension represents all the recorded simulation steps and allows +for easy indexing. To quickly determine the number of time steps available, +you can check the size of the time dimension. + +```{jupyter-execute} +# This corresponds to the number of individual SDF files loaded +print(f"There are a total of {ds['time'].size} time steps") + +# You can look up the actual simulation time for any given index +sim_time = ds['time'].values[20] +print(f"The time at the 20th simulation step is {sim_time:.2e} s") +``` + +You can select and extract a single simulation snapshot using the integer +index of the time step with the function. This can be +done by passsing the index to the `time` parameter (e.g., `time=0` for +the first snapshot). + +```{jupyter-execute} +# We can plot the variable at a given time index +ds["Electric_Field_Ex"].isel(time=20) +``` + +We can also use the function if you wish to pass a +value intead of an index. + +```{tip} +If you know roughly what time you wish to select but not the exact value +you can use the parameter `method="nearest"`. +``` + +```{jupyter-execute} +ds["Electric_Field_Ex"].sel(time=sim_time) +``` + +## Visualisation on HPCs + +In many cases you will be running EPOCH simulations via a HPC cluster and your +subsequent SDF files will probably be rather large and cumbersome to interact with +via traditional Jupyter notebooks. In some cases your HPC may outright block the +use of Jupyter notebooks entirely. To circumvent this issue you can use a Terminal +User Interface (TUI) which renders the contents of SDF files directly in a Terminal +and allows for you to do some simple data analysis and visualisation. To do this we +shall leverage the [xr-tui](https://github.com/samueljackson92/xr-tui) package +which can be installed to either a venv or globally using: + +```bash +pipx install xr-tui sdf-xarray +``` + +or if you are using `uv` + +```bash +uv tool install xr-tui --with sdf-xarray +``` + +Once installed you can visualise SDF files by simply writing in the command line + +```bash +xr path/to/simulation/0000.sdf +# OR +xr path/to/simulation/*.sdf +``` + +Below is an example gif of how this interfacing looks as seen on +[xr-tui](https://github.com/samueljackson92/xr-tui) `README.md`: + +![xr-tui interfacing gif](https://raw.githubusercontent.com/samueljackson92/xr-tui/main/demo.gif) + +## Manipulating data + +These datasets can also be easily manipulated the same way as you +would with `numpy` arrays. + +```{jupyter-execute} +ds["Laser_Absorption_Fraction_in_Simulation"] = ( + (ds["Total_Particle_Energy_in_Simulation"] - ds["Total_Particle_Energy_in_Simulation"][0]) + / ds["Absorption_Total_Laser_Energy_Injected"] +) * 100 + +# We can also manipulate the units and other attributes +ds["Laser_Absorption_Fraction_in_Simulation"].attrs["units"] = "%" +ds["Laser_Absorption_Fraction_in_Simulation"].attrs["long_name"] = "Laser Absorption Fraction" + +ds["Laser_Absorption_Fraction_in_Simulation"].plot() +plt.title("Laser absorption fraction in simulation") +plt.show() +``` + +You can also call the `plot()` function on several variables with +labels by delaying the call to `plt.show()`. + +```{jupyter-execute} +ds["Total_Particle_Energy_Electron"].plot(label="Electron") +ds["Total_Particle_Energy_Ion"].plot(label="Ion") +plt.title("Particle Energy in Simulation per Species") +plt.legend() +plt.show() +``` + +```{jupyter-execute} +print(f"Total laser energy injected: {ds["Absorption_Total_Laser_Energy_Injected"][-1].values:.1e} J") +print(f"Total particle energy absorbed: {ds["Total_Particle_Energy_in_Simulation"][-1].values:.1e} J") +print(f"The laser absorption fraction: {ds["Laser_Absorption_Fraction_in_Simulation"][-1].values:.1f} %") +``` diff --git a/docs/key_functionality.rst b/docs/key_functionality.rst deleted file mode 100644 index b87b6fd..0000000 --- a/docs/key_functionality.rst +++ /dev/null @@ -1,335 +0,0 @@ -.. _sec-key-functionality: - -================== -Key Functionality -================== - -.. jupyter-execute:: - - import xarray as xr - import sdf_xarray as sdfxr - import matplotlib.pyplot as plt - %matplotlib inline - -.. _loading-sdf-files: - -Loading SDF files ------------------ - -Loading single files -~~~~~~~~~~~~~~~~~~~~ - -.. jupyter-execute:: - - sdfxr.open_dataset("tutorial_dataset_1d/0010.sdf") - -You can also load the data in as a `xarray.DataTree`, which organises the data -hierarchically into ``groups`` (for example grouping related quantities such as the individual -components of the electric and magnetic fields) while keeping each item as a `xarray.Dataset`. - -.. jupyter-execute:: - - sdfxr.open_datatree("tutorial_dataset_1d/0010.sdf") - -.. _loading-raw-files: - -Loading raw files -~~~~~~~~~~~~~~~~~ - -If you wish to load data directly from the ``SDF.C`` library and ignore -the `xarray` interface layer. - -.. jupyter-execute:: - - raw_ds = sdfxr.SDFFile("tutorial_dataset_1d/0010.sdf") - raw_ds.variables.keys() - -Loading multiple files -~~~~~~~~~~~~~~~~~~~~~~ - -Multiple files can be loaded using one of two methods. The first of which -is by using the `sdf_xarray.open_mfdataset`. - -.. tip:: - - If your simulation includes multiple ``output`` blocks that specify different variables - for output at various time steps, variables not present at a specific step will default - to a nan value. To remove these nan values we suggest using the `xarray.DataArray.dropna` - function or following our implmentation in :ref:`loading-sparse-data`. - -.. jupyter-execute:: - - sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") - -Alternatively, files can be loaded using `xarray.open_mfdataset` however when loading in -all the files we have do some processing of the data so that we can correctly align it along -the time dimension; This is done via the ``preprocess`` parameter utilising the -`sdf_xarray.SDFPreprocess` function. - -.. jupyter-execute:: - - xr.open_mfdataset( - "tutorial_dataset_1d/*.sdf", - join="outer", - compat="no_conflicts", - preprocess=sdfxr.SDFPreprocess() - ) - -You can also load the data in as a `xarray.DataTree`, which organises the data -hierarchically into ``groups`` (for example grouping related quantities such as the individual -components of the electric and magnetic fields) while keeping each item as a `xarray.Dataset`. - -.. jupyter-execute:: - - sdfxr.open_mfdatatree("tutorial_dataset_1d/*.sdf") - -.. _loading-sparse-data: - -Loading sparse data -~~~~~~~~~~~~~~~~~~~ - -When dealing with sparse data (where different variables are saved at different, -non-overlapping time steps) you can optimize memory usage by loading the data with -`sdf_xarray.open_mfdataset` using the parameter ``separate_times=True``. This -approach creates a distinct time dimension for each output block, avoiding the -need for a single, large time dimension that would be filled with nan values. This -significantly reduces memory consumption, though it requires more deliberate handling -if you need to compare variables that exist on these different time coordinates. - -.. jupyter-execute:: - - sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf", separate_times=True) - -Loading particle data -~~~~~~~~~~~~~~~~~~~~~ - -.. warning:: - It is **not recommended** to use `xarray.open_mfdataset` or - `sdf_xarray.open_mfdataset` to load particle data from multiple - SDF outputs. The number of particles often varies between outputs, - which can lead to inconsistent array shapes that these functions - cannot handle. Instead, consider loading each file individually and - then concatenating them manually. - -.. note:: - When loading multiple probes from a single SDF file, you **must** use the - ``probe_names`` parameter to assign a unique name to each. For example, - use ``probe_names=["Front_Electron_Probe", "Back_Electron_Probe"]``. - Failing to do so will result in dimension name conflicts. - -By default, particle data isn't kept as it takes up a lot of space. -Pass ``keep_particles=True`` as a keyword argument to -`xarray.open_dataset` (for single files) or `xarray.open_mfdataset` (for -multiple files). - -.. jupyter-execute:: - - sdfxr.open_dataset("tutorial_dataset_1d/0010.sdf", keep_particles=True) - -Loading specific variables -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When loading datasets containing several (``>10``) coordinates/dimensions -using `sdf_xarray.open_mfdataset`, ``xarray`` may struggle to locate -the necessary RAM to concatenate all of the data (as seen in -`Issue #57 `_). -In this instance, you can optimize memory usage by loading only the data -you need using the keyword argument ``data_vars`` and passing one or more -variables. This creates a dataset consisting only of the given variable(s) -and the relevant coordinates/dimensions, significantly reducing memory -consumption. - -.. jupyter-execute:: - - sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf", data_vars=["Electric_Field_Ex"]) - -.. _loading-input-deck: - -Loading the input.deck -~~~~~~~~~~~~~~~~~~~~~~ - -When loading SDF files, `sdf_xarray` will attempt to automatically load -the ``input.deck`` file used to initialise the simulation from the same -directory as the SDF file. If the file is not found, it will silently fail -and continue loading the SDF file as normal. This file contains the initial -simulation setup information which is not present in SDF outputs. By loading -this file, you can access these parameters as part of your dataset's metadata. -To do this, use the ``deck_path`` parameter when loading an SDF file with -`sdf_xarray.open_dataset`, `xarray.open_dataset`, `sdf_xarray.open_datatree`, -`xarray.open_datatree`, `sdf_xarray.open_mfdataset` or `sdf_xarray.open_mfdatatree`. - -There are a few ways you can load an input deck: - -- **Default behaviour**: The input deck is loaded from the same directory - as the SDF file if it exists. If it does not exist, it will silently fail. -- **Relative path**: (e.g. ``"template.deck"``) Searches for that specific filename - within the same directory as the SDF file. -- **Absolute path**: (e.g. ``"/path/to/input.deck"``) Uses the full, specified path - to locate the file. - -An example of loading a deck can be seen below - -.. toggle:: - - .. jupyter-execute:: - - import json - from IPython.display import Code - - ds = xr.open_dataset("tutorial_dataset_1d/0010.sdf") - # The results are accessible by calling - deck = ds.attrs["deck"] - - # Some prettification to make it looks nice in jupyter notebooks - json_str = json.dumps(deck, indent=4) - Code(json_str, language='json') - -Data interaction examples -------------------------- - -When loading in either a single dataset or a group of datasets you -can access the following methods to explore the dataset: - -- ``ds.variables`` to list variables. (e.g. Electric Field, Magnetic - Field, Particle Count) -- ``ds.coords`` for accessing coordinates/dimensions. (e.g. x-axis, - y-axis, time) -- ``ds.attrs`` for metadata attached to the dataset. (e.g. filename, - step, time) - -It is important to note here that ``xarray`` lazily loads the data -meaning that it only explicitly loads the results your currently -looking at when you call ``.values`` - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") - - ds["Electric_Field_Ex"] - -On top of accessing variables you can plot these `xarray.Dataset` -using the built-in `xarray.DataArray.plot` function (see -https://docs.xarray.dev/en/stable/user-guide/plotting.html) which is -a simple call to ``matplotlib``. This also means that you can access -all the methods from ``matplotlib`` to manipulate your plot. - -.. jupyter-execute:: - - # This is discretized in both space and time - ds["Electric_Field_Ex"].plot() - plt.title("Electric field along the x-axis") - plt.show() - -When loading a multi-file dataset using `sdf_xarray.open_mfdataset`, a -time dimension is automatically added to the resulting `xarray.Dataset`. -This dimension represents all the recorded simulation steps and allows -for easy indexing. To quickly determine the number of time steps available, -you can check the size of the time dimension. - -.. jupyter-execute:: - - # This corresponds to the number of individual SDF files loaded - print(f"There are a total of {ds['time'].size} time steps") - - # You can look up the actual simulation time for any given index - sim_time = ds['time'].values[20] - print(f"The time at the 20th simulation step is {sim_time:.2e} s") - -You can select and extract a single simulation snapshot using the integer -index of the time step with the `xarray.Dataset.isel` function. This can be -done by passsing the index to the ``time`` parameter (e.g., ``time=0`` for -the first snapshot). - -.. jupyter-execute:: - - # We can plot the variable at a given time index - ds["Electric_Field_Ex"].isel(time=20) - -We can also use the `xarray.Dataset.sel` function if you wish to pass a -value intead of an index. - -.. tip:: - - If you know roughly what time you wish to select but not the exact value - you can use the parameter ``method="nearest"``. - -.. jupyter-execute:: - - ds["Electric_Field_Ex"].sel(time=sim_time) - -Visualisation on HPCs ---------------------- - -In many cases you will be running EPOCH simulations via a HPC cluster and your -subsequent SDF files will probably be rather large and cumbersome to interact with -via traditional Jupyter notebooks. In some cases your HPC may outright block the -use of Jupyter notebooks entirely. To circumvent this issue you can use a Terminal -User Interface (TUI) which renders the contents of SDF files directly in a Terminal -and allows for you to do some simple data analysis and visualisation. To do this we -shall leverage the `xr-tui `_ package -which can be installed to either a venv or globally using: - -.. code-block:: bash - - pip install xr-tui sdf-xarray - -or if you are using ``uv`` - -.. code-block:: bash - - uv tool install xr-tui --with sdf-xarray - -Once installed you can visualise SDF files by simply writing in the command line - -.. code-block:: bash - - xr path/to/simulation/0000.sdf - # OR - xr path/to/simulation/*.sdf - - -Below is an example gif of how this interfacing looks as seen on -`xr-tui `_ ``README.md``: - -.. image:: https://raw.githubusercontent.com/samueljackson92/xr-tui/main/demo.gif - :alt: xr-tui interfacing gif - :align: center - -Manipulating data ------------------ - -These datasets can also be easily manipulated the same way as you -would with ``numpy`` arrays. - -.. jupyter-execute:: - - ds["Laser_Absorption_Fraction_in_Simulation"] = ( - (ds["Total_Particle_Energy_in_Simulation"] - ds["Total_Particle_Energy_in_Simulation"][0]) - / ds["Absorption_Total_Laser_Energy_Injected"] - ) * 100 - - # We can also manipulate the units and other attributes - ds["Laser_Absorption_Fraction_in_Simulation"].attrs["units"] = "%" - ds["Laser_Absorption_Fraction_in_Simulation"].attrs["long_name"] = "Laser Absorption Fraction" - - ds["Laser_Absorption_Fraction_in_Simulation"].plot() - plt.title("Laser absorption fraction in simulation") - plt.show() - -You can also call the ``plot()`` function on several variables with -labels by delaying the call to ``plt.show()``. - -.. jupyter-execute:: - - ds["Total_Particle_Energy_Electron"].plot(label="Electron") - ds["Total_Particle_Energy_Ion"].plot(label="Ion") - plt.title("Particle Energy in Simulation per Species") - plt.legend() - plt.show() - - -.. jupyter-execute:: - - print(f"Total laser energy injected: {ds["Absorption_Total_Laser_Energy_Injected"][-1].values:.1e} J") - print(f"Total particle energy absorbed: {ds["Total_Particle_Energy_in_Simulation"][-1].values:.1e} J") - print(f"The laser absorption fraction: {ds["Laser_Absorption_Fraction_in_Simulation"][-1].values:.1f} %") diff --git a/docs/known_issues.md b/docs/known_issues.md new file mode 100644 index 0000000..13ad761 --- /dev/null +++ b/docs/known_issues.md @@ -0,0 +1,11 @@ +# Known Issues + +There are a couple of known 'quirks' in `sdf_xarray`: + +- [Issue #57](https://github.com/epochpic/sdf-xarray/issues/57) Loading multiple +SDF files with can lead to out-of-memory +errors. The issue is believed to stem from how the underlying library +handles coordinates, causing it to infer an excessively large array shape that +requests far more memory than is needed. Due to the significant architectural +changes required for a fix, the maintainers do not plan to resolve this. The +recommended solution is to load the files individually or in smaller batches. diff --git a/docs/known_issues.rst b/docs/known_issues.rst deleted file mode 100644 index f2287d7..0000000 --- a/docs/known_issues.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _sec-known-issues: - -============ -Known Issues -============ - -There are a couple of known 'quirks' in `sdf_xarray`: - -- `Issue #57 `_ Loading multiple SDF files with `sdf_xarray.open_mfdataset` can lead to out-of-memory errors. The issue is believed to stem from how the underlying `xarray` library handles coordinates, causing it to infer an excessively large array shape that requests far more memory than is needed. Due to the significant architectural changes required for a fix, the maintainers do not plan to resolve this. The recommended solution is to load the files individually or in smaller batches. \ No newline at end of file diff --git a/docs/unit_conversion.md b/docs/unit_conversion.md new file mode 100644 index 0000000..983fb15 --- /dev/null +++ b/docs/unit_conversion.md @@ -0,0 +1,201 @@ +# Unit Conversion + +The package automatically extracts the units for each +coordinate/variable/constant from an SDF file and stores them as an +attribute called `"units"`. Sometimes we want to convert our data from one format to +another, e.g. converting the grid coordinates from meters to microns, time from seconds +to femto-seconds or particle energy from Joules to electron-volts. + +```{jupyter-execute} +import sdf_xarray as sdfxr +import matplotlib.pyplot as plt +%matplotlib inline + +plt.rcParams.update({ + "axes.labelsize": 16, + "xtick.labelsize": 14, + "ytick.labelsize": 14, + "axes.titlesize": 16, + "figure.titlesize": 18, +}) +``` + +## Rescaling coordinates + +For simple scaling and unit relabelling of coordinates (e.g., converting meters to microns), +the most straightforward approach is to use the [`xarray.Dataset.epoch.rescale_coords`](project:#sdf_xarray.dataset_accessor.EpochAccessor.rescale_coords) dataset accessor. +This function scales the coordinate values by a given multiplier and updates the +`"units"` attribute in one step. + +### Rescaling grid coordinates + +We can use the [`xarray.Dataset.epoch.rescale_coords`](project:#sdf_xarray.dataset_accessor.EpochAccessor.rescale_coords) method to convert X, Y, and Z coordinates from meters +(`m`) to microns (`µm`) by applying a multiplier of `1e6`. + +```{jupyter-execute} +fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6)) + +ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") +ds_in_microns = ds.epoch.rescale_coords(1e6, "µm", ["X_Grid_mid", "Y_Grid_mid"]) + +ds["Derived_Number_Density_Electron"].isel(time=0).plot(ax=ax1, x="X_Grid_mid", y="Y_Grid_mid") +ax1.set_title("Original X Coordinate (m)") + +ds_in_microns["Derived_Number_Density_Electron"].isel(time=0).plot(ax=ax2, x="X_Grid_mid", y="Y_Grid_mid") +ax2.set_title("Rescaled X Coordinate (µm)") + +fig.tight_layout() +``` + +### Rescaling time coordinate + +We can also use the [`xarray.Dataset.epoch.rescale_coords`](project:#sdf_xarray.dataset_accessor.EpochAccessor.rescale_coords) method to convert the time coordinate from +seconds (`s`) to femto-seconds (`fs`) by applying a multiplier of `1e15`. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") +ds["time"] +``` + +```{jupyter-execute} +ds = ds.epoch.rescale_coords(1e15, "fs", "time") +ds["time"] +``` + +## Unit conversion with pint-xarray + +While this is sufficient for most use cases, we can enhance this functionality +using the [pint](https://pint.readthedocs.io/en/stable) library. +Pint allows us to specify the units of a given array and convert them +to another, which is incredibly handy. We can take this a step further, +however, and utilize the [pint-xarray](https://pint-xarray.readthedocs.io/en/stable) +library. This library allows us to infer units directly from an + while retaining all the information about the +. This works very similarly to taking a NumPy array and +multiplying it by a constant or another array, which returns a new array; +however, this library will also retain the unit logic (specifically the +`"units"` information). + +```{note} +Unit conversion is not supported on coordinates in `pint_xarray` which is due to an +underlying issue with how implements indexes. +``` + +### Installation + +To install the pint libraries you can simply run the following optional +dependency pip command which will install both the `pint` and `pint_xarray` +libraries. Once installed the +[`xarray.Dataset.pint`](https://pint-xarray.readthedocs.io/en/stable/api.html#dataset) +accessor should become accessible. You can install these optional dependencies via pip: + +```bash +pip install "sdf_xarray[pint]" +``` + +### Quantifying DataArrays + +When using `pint_xarray`, the library attempts to infer units from the +`"units"` attribute on each . In the following example we will +extract the time-resolved total particle energy of electrons which is measured in +Joules and convert it to electron volts. + +```{jupyter-execute} +ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") +ds["Total_Particle_Energy_Electron"] +``` + +Once you call the type is inferred the original + `"units"` attribute which is then removed and the data is +converted to a . + +```{note} +You can also specify the units yourself by passing it as a string +(e.g. `"J"`) into the function call. +``` + +```{jupyter-execute} +total_particle_energy = ds["Total_Particle_Energy_Electron"].pint.quantify() +total_particle_energy +``` + +Now that this dataset has been converted a , we can check +it's units and dimensionality + +```{jupyter-execute} +print(total_particle_energy.pint.units) +print(total_particle_energy.pint.dimensionality) +``` + +### Converting units + +We can now convert it to electron volts utilising the +function + +```{jupyter-execute} +total_particle_energy_ev = total_particle_energy.pint.to("eV") +total_particle_energy_ev +``` + +### Unit propagation + +Suppose instead of converting to `"eV"`, we want to convert to `"W"` +(watts). To do this, we divide the total particle energy by time. However, +since coordinates in cannot be directly converted to +, we must first extract the coordinate values manually +and create a new Pint quantity for time. + +Once both arrays are quantified, Pint will automatically handle the unit +propagation when we perform arithmetic operations like division. + +```{note} +Pint does not automatically simplify `"J/s"` to `"W"`, so we use + to convert the unit string. Since +these units are the same it will not change the underlying data, only the +units. This is only a small formatting choice and is not required. +``` + +```{jupyter-execute} +import pint + +time_values = total_particle_energy.coords["time"].data +time = pint.Quantity(time_values, "s") +total_particle_energy_w = total_particle_energy / time # units: joule / second +total_particle_energy_w = total_particle_energy_w.pint.to("W") # units: watt +``` + +### Dequantifying and restoring units + +```{note} +If this function is not called prior to plotting then the `units` will be +inferred from the array which will return the long +name of the units. i.e. instead of returning `"eV"` it will return +`"electron_volt"`. +``` + +The function converts the data from + back to the original and adds +the `"units"` attribute back in. It also has an optional `format` parameter +that allows you to specify the formatting type of `"units"` attribute. We +have used the `format="~P"` option as it shortens the unit to its +"short pretty" format (`"eV"`). For more options, see the +[Pint formatting documentation](https://pint.readthedocs.io/en/stable/user/formatting.html). + +```{jupyter-execute} +total_particle_energy_ev = total_particle_energy_ev.pint.dequantify(format="~P") +total_particle_energy_w = total_particle_energy_w.pint.dequantify(format="~P") +total_particle_energy_ev +``` + +To confirm the conversion has worked correctly, we can plot the original and +converted side by side: + +```{jupyter-execute} +fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(16,8)) +ds["Total_Particle_Energy_Electron"].plot(ax=ax1) +total_particle_energy_ev.plot(ax=ax2) +total_particle_energy_w.plot(ax=ax3) +ax4.set_visible(False) +fig.suptitle("Comparison of conversion from Joules to electron volts and watts") +fig.tight_layout() +``` diff --git a/docs/unit_conversion.rst b/docs/unit_conversion.rst deleted file mode 100644 index aeae0b6..0000000 --- a/docs/unit_conversion.rst +++ /dev/null @@ -1,217 +0,0 @@ -.. |rescale_coords_accessor| replace:: `xarray.Dataset.epoch.rescale_coords - ` - -.. _sec-unit-conversion: - -=============== -Unit Conversion -=============== - -The ``sdf-xarray`` package automatically extracts the units for each -coordinate/variable/constant from an SDF file and stores them as an `xarray.Dataset` -attribute called ``"units"``. Sometimes we want to convert our data from one format to -another, e.g. converting the grid coordinates from meters to microns, time from seconds -to femto-seconds or particle energy from Joules to electron-volts. - -.. jupyter-execute:: - - import sdf_xarray as sdfxr - import matplotlib.pyplot as plt - %matplotlib inline - - plt.rcParams.update({ - "axes.labelsize": 16, - "xtick.labelsize": 14, - "ytick.labelsize": 14, - "axes.titlesize": 16, - "figure.titlesize": 18, - }) - - -Rescaling coordinates ---------------------- - -For simple scaling and unit relabelling of coordinates (e.g., converting meters to microns), -the most straightforward approach is to use the |rescale_coords_accessor| dataset accessor. -This function scales the coordinate values by a given multiplier and updates the -``"units"`` attribute in one step. - -Rescaling grid coordinates -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We can use the |rescale_coords_accessor| method to convert X, Y, and Z coordinates from meters -(``m``) to microns (``µm``) by applying a multiplier of ``1e6``. - -.. jupyter-execute:: - - fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6)) - - ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") - ds_in_microns = ds.epoch.rescale_coords(1e6, "µm", ["X_Grid_mid", "Y_Grid_mid"]) - - ds["Derived_Number_Density_Electron"].isel(time=0).plot(ax=ax1, x="X_Grid_mid", y="Y_Grid_mid") - ax1.set_title("Original X Coordinate (m)") - - ds_in_microns["Derived_Number_Density_Electron"].isel(time=0).plot(ax=ax2, x="X_Grid_mid", y="Y_Grid_mid") - ax2.set_title("Rescaled X Coordinate (µm)") - - fig.tight_layout() - - -Rescaling time coordinate -~~~~~~~~~~~~~~~~~~~~~~~~~ - -We can also use the |rescale_coords_accessor| method to convert the time coordinate from -seconds (``s``) to femto-seconds (``fs``) by applying a multiplier of ``1e15``. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_2d/*.sdf") - ds["time"] - -.. jupyter-execute:: - - ds = ds.epoch.rescale_coords(1e15, "fs", "time") - ds["time"] - -Unit conversion with pint-xarray --------------------------------- - -While this is sufficient for most use cases, we can enhance this functionality -using the `pint `_ library. -Pint allows us to specify the units of a given array and convert them -to another, which is incredibly handy. We can take this a step further, -however, and utilize the `pint-xarray -`_ library. This library -allows us to infer units directly from an `xarray.Dataset.attrs` while -retaining all the information about the `xarray.Dataset`. This works -very similarly to taking a NumPy array and multiplying it by a constant or -another array, which returns a new array; however, this library will also -retain the unit logic (specifically the ``"units"`` information). - -.. note:: - Unit conversion is not supported on coordinates in ``pint-xarray`` which is due to an - underlying issue with how ``xarray`` implements indexes. - -Installation -~~~~~~~~~~~~ - -To install the pint libraries you can simply run the following optional -dependency pip command which will install both the ``pint`` and ``pint-xarray`` -libraries. Once installed the ``xarray.Dataset.pint`` accessor should become -accessible. You can install these optional dependencies via pip: - -.. code:: console - - $ pip install "sdf_xarray[pint]" - - -Quantifying DataArrays -~~~~~~~~~~~~~~~~~~~~~~ - -When using ``pint-xarray``, the library attempts to infer units from the -``"units"`` attribute on each `xarray.DataArray`. In the following example we will -extract the time-resolved total particle energy of electrons which is measured in -Joules and convert it to electron volts. - -.. jupyter-execute:: - - ds = sdfxr.open_mfdataset("tutorial_dataset_1d/*.sdf") - ds["Total_Particle_Energy_Electron"] - -Once you call `xarray.DataArray.pint.quantify` the type is inferred the original -`xarray.DataArray` ``"units"`` attribute which is then removed and the data is -converted to a `pint.Quantity`. - -.. note:: - You can also specify the units yourself by passing it as a string - (e.g. ``"J"``) into the `xarray.DataArray.pint.quantify` function call. - -.. jupyter-execute:: - - total_particle_energy = ds["Total_Particle_Energy_Electron"].pint.quantify() - total_particle_energy - - -Now that this dataset has been converted a `pint.Quantity`, we can check -it's units and dimensionality - -.. jupyter-execute:: - - print(total_particle_energy.pint.units) - print(total_particle_energy.pint.dimensionality) - - -Converting units -~~~~~~~~~~~~~~~~ - -We can now convert it to electron volts utilising the `xarray.DataArray.pint.to` -function - -.. jupyter-execute:: - - total_particle_energy_ev = total_particle_energy.pint.to("eV") - total_particle_energy_ev - -Unit propagation -~~~~~~~~~~~~~~~~ - -Suppose instead of converting to ``"eV"``, we want to convert to ``"W"`` -(watts). To do this, we divide the total particle energy by time. However, -since coordinates in `xarray.Dataset` cannot be directly converted to -`pint.Quantity`, we must first extract the coordinate values manually -and create a new Pint quantity for time. - -Once both arrays are quantified, Pint will automatically handle the unit -propagation when we perform arithmetic operations like division. - -.. note:: - Pint does not automatically simplify ``"J/s"`` to ``"W"``, so we use - `xarray.DataArray.pint.to` to convert the unit string. Since these units are - the same it will not change the underlying data, only the units. This is - only a small formatting choice and is not required. - -.. jupyter-execute:: - - import pint - - time_values = total_particle_energy.coords["time"].data - time = pint.Quantity(time_values, "s") - total_particle_energy_w = total_particle_energy / time # units: joule / second - total_particle_energy_w = total_particle_energy_w.pint.to("W") # units: watt - -Dequantifying and restoring units -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: - If this function is not called prior to plotting then the ``units`` will be - inferred from the `pint.Quantity` array which will return the long - name of the units. i.e. instead of returning ``"eV"`` it will return - ``"electron_volt"``. - -The `xarray.DataArray.pint.dequantify` function converts the data from -`pint.Quantity` back to the original `xarray.DataArray` and adds -the ``"units"`` attribute back in. It also has an optional ``format`` parameter -that allows you to specify the formatting type of ``"units"`` attribute. We -have used the ``format="~P"`` option as it shortens the unit to its -"short pretty" format (``"eV"``). For more options, see the `Pint formatting -documentation `_. - -.. jupyter-execute:: - - total_particle_energy_ev = total_particle_energy_ev.pint.dequantify(format="~P") - total_particle_energy_w = total_particle_energy_w.pint.dequantify(format="~P") - total_particle_energy_ev - -To confirm the conversion has worked correctly, we can plot the original and -converted `xarray.Dataset` side by side: - -.. jupyter-execute:: - - fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(16,8)) - ds["Total_Particle_Energy_Electron"].plot(ax=ax1) - total_particle_energy_ev.plot(ax=ax2) - total_particle_energy_w.plot(ax=ax3) - ax4.set_visible(False) - fig.suptitle("Comparison of conversion from Joules to electron volts and watts") - fig.tight_layout() From 1442d4cae88a67dfa17068ea0762601fbbde84aa Mon Sep 17 00:00:00 2001 From: Joel Adams Date: Wed, 18 Feb 2026 16:06:58 +0000 Subject: [PATCH 2/2] README format line length --- README.md | 54 ++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index c1ebe44..80b1378 100644 --- a/README.md +++ b/README.md @@ -8,10 +8,10 @@ [![Read the Docs](https://img.shields.io/readthedocs/sdf-xarray?logo=readthedocs&link=https%3A%2F%2Fsdf-xarray.readthedocs.io%2F)](https://sdf-xarray.readthedocs.io) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) - -sdf-xarray provides a backend for [xarray](https://xarray.dev) to read SDF files as created by -[EPOCH](https://epochpic.github.io) using the [SDF-C](https://github.com/epochpic/SDF_C) library. -Part of [BEAM](#broad-epoch-analysis-modules-beam) (Broad EPOCH Analysis Modules). +sdf-xarray provides a backend for [xarray](https://xarray.dev) to read SDF files +as created by [EPOCH](https://epochpic.github.io) using the +[SDF-C](https://github.com/epochpic/SDF_C) library. Part of +[BEAM](#broad-epoch-analysis-modules-beam) (Broad EPOCH Analysis Modules). ## Installation @@ -59,9 +59,9 @@ print(df["Electric_Field_Ex"]) ### Multi-file loading -You can open all the SDF files for a given simulation by calling the `open_mfdataset` -function from `sdf_xarray`. This will additionally add a time dimension using the `"time"` -value stored in each files attributes. +You can open all the SDF files for a given simulation by calling the +`open_mfdataset` function from `sdf_xarray`. This will additionally add +a time dimension using the `"time"` value stored in each files attributes. > [!IMPORTANT] > If your simulation has multiple `output` blocks so that not all variables are @@ -86,34 +86,52 @@ print(ds) ## Citing -If sdf-xarray contributes to a project that leads to publication, please acknowledge this by citing sdf-xarray. This can be done by clicking the "cite this repository" button located near the top right of this page. +If sdf-xarray contributes to a project that leads to publication, please acknowledge +this by citing sdf-xarray. This can be done by clicking the "cite this repository" +button located near the top right of this page. ## Contributing -We welcome contributions to the BEAM ecosystem! Whether it's reporting issues, suggesting features, or submitting pull requests, your input helps improve these tools for the community. +We welcome contributions to the BEAM ecosystem! Whether it's reporting issues, +suggesting features, or submitting pull requests, your input helps improve these +tools for the community. ### How to Contribute There are many ways to get involved: -- **Report bugs**: Found something not working as expected? Open an issue with as much detail as possible. -- **Request a feature**: Got an idea for a new feature or enhancement? Open a feature request on [GitHub Issues](https://github.com/epochpic/sdf-xarray/issues)! -- **Improve the documentation**: We aim to keep our docs clear and helpful—if something's missing or unclear, feel free to suggest edits. -- **Submit code changes**: Bug fixes, refactoring, or new features are welcome. +- **Report bugs**: Found something not working as expected? Open an issue with as +much detail as possible. +- **Request a feature**: Got an idea for a new feature or enhancement? Open a feature +request on [GitHub Issues](https://github.com/epochpic/sdf-xarray/issues)! +- **Improve the documentation**: We aim to keep our docs clear and helpful—if +something's missing or unclear, feel free to suggest edits. +- **Submit code changes**: Bug fixes, refactoring, or new features are welcome. All code is automatically linted, formatted, and tested via GitHub Actions. -To run checks locally before opening a pull request, see [CONTRIBUTING.md](CONTRIBUTING.md) or [readthedocs documentation](https://sdf-xarray.readthedocs.io/en/latest/contributing.html) +To run checks locally before opening a pull request, see +[CONTRIBUTING.md](CONTRIBUTING.md) or [readthedocs documentation](https://sdf-xarray.readthedocs.io/en/latest/contributing.html) ## Broad EPOCH Analysis Modules (BEAM) ![BEAM logo](./BEAM.png) -**BEAM** is a collection of independent yet complementary open-source tools for analysing EPOCH simulations, designed to be modular so researchers can adopt only the components they require without being constrained by a rigid framework. In line with the **FAIR principles — Findable**, **Accessible**, **Interoperable**, and **Reusable** — each package is openly published with clear documentation and versioning (Findable), distributed via public repositories (Accessible), designed to follow common standards for data structures and interfaces (Interoperable), and includes licensing and metadata to support long-term use and adaptation (Reusable). The packages are as follows: - -- [sdf-xarray](https://github.com/epochpic/sdf-xarray): Reading and processing SDF files and converting them to [xarray](https://docs.xarray.dev/en/stable/). +**BEAM** is a collection of independent yet complementary open-source tools for +analysing EPOCH simulations, designed to be modular so researchers can adopt only +the components they require without being constrained by a rigid framework. In +line with the **FAIR principles — Findable**, **Accessible**, **Interoperable**, +and **Reusable** — each package is openly published with clear documentation and +versioning (Findable), distributed via public repositories (Accessible), designed +to follow common standards for data structures and interfaces (Interoperable), and +includes licensing and metadata to support long-term use and adaptation (Reusable). +The packages are as follows: + +- [sdf-xarray](https://github.com/epochpic/sdf-xarray): Reading and processing SDF +files and converting them to [xarray](https://docs.xarray.dev/en/stable/). - [epydeck](https://github.com/epochpic/epydeck): Input deck reader and writer. -- [epyscan](https://github.com/epochpic/epyscan): Create campaigns over a given parameter space using various sampling methods. +- [epyscan](https://github.com/epochpic/epyscan): Create campaigns over a given +parameter space using various sampling methods. ## PlasmaFAIR