Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions content/scientific-data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,55 +140,55 @@
- | Long term
| storage/sharing:

* - :ref:`Pickle <pickle>`

Check warning on line 143 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'pickle'

Check warning on line 143 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'pickle'

Check warning on line 143 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'pickle'
- ❌

Check warning on line 144 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'pickle'

Check warning on line 144 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'pickle'
- 🟨
- ✅
- 🟨
- 🟨
- ❌

* - :ref:`CSV <csv>`

Check warning on line 151 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'csv'

Check warning on line 151 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'csv'

Check warning on line 151 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'csv'
- ✅

Check warning on line 152 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'csv'

Check warning on line 152 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'csv'
- ❌
- ❌
- ✅
- 🟨
- ✅

* - :ref:`Feather <feather>`

Check warning on line 159 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'feather'

Check warning on line 159 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'feather'

Check warning on line 159 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'feather'
- ❌

Check warning on line 160 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'feather'

Check warning on line 160 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'feather'
- ✅
- ❌
- ✅
- ❌
- ❌

* - :ref:`Parquet <parquet>`

Check warning on line 167 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'parquet'

Check warning on line 167 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'parquet'

Check warning on line 167 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'parquet'
- ❌

Check warning on line 168 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'parquet'

Check warning on line 168 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'parquet'
- ✅
- 🟨
- ✅
- 🟨
- ✅

* - :ref:`npy <npy>`

Check warning on line 175 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'npy'

Check warning on line 175 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'npy'

Check warning on line 175 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'npy'
- ❌

Check warning on line 176 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'npy'

Check warning on line 176 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'npy'
- 🟨
- ❌
- ❌
- ✅
- ❌

* - :ref:`HDF5 <hdf5>`

Check warning on line 183 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'hdf5'

Check warning on line 183 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'hdf5'

Check warning on line 183 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'hdf5'
- ❌

Check warning on line 184 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'hdf5'

Check warning on line 184 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'hdf5'
- ✅
- ❌
- ❌
- ✅
- ✅

* - :ref:`NetCDF4 <netcdf4>`

Check warning on line 191 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'netcdf4'

Check warning on line 191 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'netcdf4'

Check warning on line 191 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'netcdf4'
- ❌
- ✅
- ❌
Expand All @@ -196,7 +196,7 @@
- ✅
- ✅

* - :ref:`JSON <json>`

Check warning on line 199 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'json'

Check warning on line 199 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'json'

Check warning on line 199 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'json'
- ✅
- ❌
- 🟨
Expand All @@ -204,7 +204,7 @@
- ❌
- ✅

* - :ref:`Excel <excel>`

Check warning on line 207 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'excel'

Check warning on line 207 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'excel'
- ❌
- ❌
- ❌
Expand All @@ -212,7 +212,7 @@
- ❌
- 🟨

* - :ref:`Graph formats <graph>`

Check warning on line 215 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'graph'

Check warning on line 215 in content/scientific-data.rst

View workflow job for this annotation

GitHub Actions / Build

undefined label: 'graph'
- 🟨
- 🟨
- ❌
Expand All @@ -226,6 +226,8 @@
- 🟨 : Ok / depends on a case
- ❌ : Bad

Adapted from Aalto university's `Python for scientific computing <https://aaltoscicomp.github.io/python-for-scicomp/work-with-data/#what-is-a-data-format>`__.

Some of these formats (e.g. JSON and CSV) are saved as text files (ASCII), thus they are
human-readable. This makes them easier to visually check them (e.g. for format errors) and
are supported out of the box by many tools. However, they tend to be slower during I/O and
Expand Down
46 changes: 38 additions & 8 deletions content/stack.rst
Original file line number Diff line number Diff line change
Expand Up @@ -359,14 +359,44 @@ Views and copies of arrays
I/O with NumPy
^^^^^^^^^^^^^^

- Numpy provides functions for reading data from file and for writing data
into the files
- Simple text files

- :meth:`numpy.loadtxt`
- :meth:`numpy.savetxt`
- Data in regular column layout
- Can deal with comments and different column delimiters
Numpy provides functions for reading from/writing to files. Both ASCII and binary
formats are supported with the CSV and npy/npz formats:

.. tabs::

.. tab:: CSV

The ``numpy.loadtxt()`` and ``numpy.savetxt()`` functions can be used. They
save in a regular column layout and can deal with different delimiters,
column titles and numerical representations.

.. code-block:: python

a = np.array([1, 2, 3, 4])
np.savetxt("my_array.csv", a)
b = np.loadtxt("my_array.csv")
a == b
# True

.. tab:: Binary

The npy format is a binary format used to dump arrays of any
shape. Several arrays can be saved into a single npz file, which is
simply a zipped collection of different npy files. All the arrays to
be saved into a npz file can be passed as kwargs to the ``numpy.savez()``
function. The data can then be recovered using the ``numpy.load()`` method,
which returns a dictionary-like object in which each key points to one of the arrays:

.. code-block:: python

a = np.array([1, 2, 3, 4])
b = np.array([5, 6, 7, 8])

np.savez("my_arrays.npz", array_1=a, array_2=b)
data = np.load("my_arrays.npz")
data['array_1'] == a
data['array_2'] == b
# Both are true


Random numbers
Expand Down
Loading