Skip to content

Conversation

@cscjlan
Copy link
Contributor

@cscjlan cscjlan commented Feb 17, 2025

Refactor FsGrid

FsGrid is now a "singleton" in the sense that vlasiator uses only a single instance of it. It's a structure that has the coordinates for different spaces (global, rank local, physical), maps ranks to a 3D neighbourhood and stores MPI_Datatype information (offsets to arrays etc.) per neighbour.

Data storage is now handled by a separate structure, FsData. It owns the data, storing the allocated pointer in a std::unique_ptr. Currently it only allocates memory for the CPU (using std::malloc), but it can be easily extended to allocate memory using Cuda, Hip, Sycl or any other API.

Data is accessed with the help of FsStencil that computes 1D indices from 3D indices.

FsGrid has a parallel_for method, which can be used to run some code (defined by a lambda) for each local cell of the fsgrid. Currently it's only implemented for the CPU.

There are over 50 tests for FsGrid and related structs that assert the correct behaviour.

Other changes:

  • Code was moved from fsgrid.hpp at root to src/grid.hpp, while fsgrid.hpp still exists for single file include purpose
  • FsGridTools was changed from a struct with only static functions to a namespace with static functions
  • FsGrid doesn't inherit from FsGridTools anymore
  • Many tests have been added
  • All member variables of FsGrid are private
  • Most member variables are const
  • FsGrid constructor body does nothing, variables are initialized in the member initializer list by separate (static) functions
  • Unused and unnecessary variables have been removed
  • Code has been split to smaller pieces that are easier to understand and test
  • Coordinate related functionality has been moved to a separate structure

Removed unnecessary inheritance of the pointless struct
…function; Add test for comparing fsgrid display string to a reference string; Add testdata directory that's symlinked to build dir during build
@ykempf
Copy link
Contributor

ykempf commented Jul 11, 2025

Turns out that trying to call fsgrid member functions from the lambda's loop bodies in the way I was attempting that was breaking things badly in some cases. So I now resolved my conundrum from 2 months ago – how to obtain i,j,k / global i,j,k / cellid / physical coordinates inside the loop body – by providing it to the loop body from various fsgrid.parallel_for_* versions. This leads to some code duplication but in many cases we don't need any of these, or only a subset, so to avoid recomputing and passing them all the time I went for this way of doing. Feel free to point me to more elegant ways of handling these cases now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants