mtl/ofi: add FI_MR_ALLOCATED to mr_mode for cuda support

For cuda support, mtl/ofi need to explicitly register memory.
Therefore, it need to specify FI_MR_ALLOCATED to its mr mode,
to indicate that mtl/ofi will only register memory that
is "allocated" (backed by physical memory pages), because most
libfabric providers have this requirement.

Signed-off-by: Wei Zhang <wzam@amazon.com>

Author: wzamazon

ompi/mca/mtl/ofi/mtl_ofi_component.c

@@ -653,7 +653,7 @@ ompi_mtl_ofi_component_init(bool enable_progress_threads,
     /** If Open MPI is built with CUDA, request device transfer
      *  capabilities */
     hints->caps |= FI_HMEM;
-    hints->domain_attr->mr_mode |= FI_MR_HMEM;
+    hints->domain_attr->mr_mode |= FI_MR_HMEM | FI_MR_ALLOCATED;
     /**
      * Note: API version 1.9 is the first version that supports FI_HMEM
      */