Typed objects block model (#216)

This PR brings block models under the same typed object envolope asrecently added objects.

Key parts of this pull requets:
* BLock models are now an OPTIONAL dependency for objects. **We want to
have a single point of entry** and block models on evo are not just
references from the perspective of the user - they are actual block
models that the user expects to interact with. If the user does not
install block models, they can interact with block model metadata
handling the current dependency in a similar way to pandas and numpy.
Evo-sdk user will get the whole package, sneaky developer who only cares
about very thin set of changes - can get evo-objects with cut down
dependencies.
* Block model dependencies were adjusted so that by default it can be
used just to parse metadata. Interaction with data is now an optional
dependency, this way for basic services that don't care about
interacting with evo-data and that user evo-objects there is no extra
dependency added.
* Block models support dataframe conversions, pretty printing via
widgets, report creation, and automated mapping/creation from
`object_from_reference/path`.
* I have shared typed definitions in Common, without going as far as
adding shared functionality that relies on numpy to avoid new depenency
on common.
* Block model in objects acts as a wrapper for a fully featured typed
objects in evo-block models that can exist by itself.

* I have not added any converters to and from block model and 3D grid...
but it is not that hard as interfaces are similar though not identical
and because the objects are different enough - we do not want to have
more shared interfaces at this stage other than common base blocks.

* OCTREE and FULLY SUBBLOCKED modes support was left out for now (via
typed Objects) - PR is large already

Author: denis-simo

code-samples/blockmodels/reports.ipynb

@@ -539,20 +539,33 @@
    "source": [
     "---\n",
     "\n",
-    "## WIP: Creating Kriging and Compute\n",
+    "## 8. Create Target Block Model\n",
     "\n",
-    "The following sections demonstrate how to run kriging estimation using Evo Compute.\n",
-    "\n",
-    "**Note:** This functionality is under development. The code below shows the expected API pattern."
+    "Create a Block Model to hold the kriging results. The block model defines the estimation grid."
    ]
   },
   {
-   "cell_type": "markdown",
+   "cell_type": "code",
+   "execution_count": null,
    "metadata": {},
+   "outputs": [],
    "source": [
-    "### WIP: Create Target Block Model\n",
+    "from evo.blockmodels import Point3, RegularBlockModel, RegularBlockModelData, Size3d, Size3i, Units\n",
+    "\n",
+    "# Define block model covering the drill hole extent\n",
+    "bm_data = RegularBlockModelData(\n",
+    "    name=\"CU Kriging Estimate\",\n",
+    "    description=\"Block model with kriged copper grades\",\n",
+    "    origin=Point3(x=444750, y=492850, z=2350),\n",
+    "    n_blocks=Size3i(nx=50, ny=75, nz=45),  # 50x75x45 blocks\n",
+    "    block_size=Size3d(dx=20.0, dy=20.0, dz=20.0),  # 20m blocks\n",
+    "    coordinate_reference_system=\"EPSG:32650\",\n",
+    "    size_unit_id=Units.METRES,\n",
+    ")\n",
     "\n",
-    "Create a Block Model to hold the kriging results. The block model defines the estimation grid."
+    "block_model = await RegularBlockModel.create(manager, bm_data)\n",
+    "print(f\"Created Block Model: {block_model.name}\")\n",
+    "print(f\"Block Model ID: {block_model.id}\")"
    ]
   },
   {
@@ -561,32 +574,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: Create target block model for kriging estimation\n",
-    "#\n",
-    "# from evo.objects.typed import BlockModel, RegularBlockModelData, Point3, Size3i, Size3d\n",
-    "# from evo.blockmodels import Units\n",
-    "#\n",
-    "# # Define block model covering the downhole extent\n",
-    "# bm_data = RegularBlockModelData(\n",
-    "#     name=\"CU Kriging Estimate\",\n",
-    "#     description=\"Block model with kriged copper grades\",\n",
-    "#     origin=Point3(x=444750, y=492850, z=2350),\n",
-    "#     n_blocks=Size3i(nx=50, ny=75, nz=45),  # 50x75x45 blocks\n",
-    "#     block_size=Size3d(dx=20.0, dy=20.0, dz=20.0),  # 20m blocks\n",
-    "#     crs=\"EPSG:32650\",\n",
-    "#     size_unit_id=Units.METRES,\n",
-    "# )\n",
-    "#\n",
-    "# block_model = await BlockModel.create_regular(manager, bm_data)\n",
-    "# print(f\"Created Block Model: {block_model.name}\")\n",
-    "# print(f\"Bounding Box: {block_model.bounding_box}\")"
+    "# Display the block model metadata\n",
+    "block_model.version"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### WIP: Define Kriging Parameters\n",
+    "## WIP. Define Kriging Parameters\n",
     "\n",
     "Configure the kriging search neighborhood and estimation parameters."
    ]
@@ -597,15 +593,13 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: Define kriging parameters\n",
-    "#\n",
     "# from evo.compute.tasks import SearchNeighborhood\n",
     "# from evo.compute.tasks.kriging import KrigingParameters\n",
     "#\n",
     "# # Use the search ellipsoid we created earlier (2x variogram range)\n",
     "# params = KrigingParameters(\n",
     "#     source=pointset.attributes[\"CU_pct\"],  # Source attribute\n",
-    "#     target=block_model.attributes[\"CU_estimate\"],  # Target attribute\n",
+    "#     target=block_model.attributes[f\"CU_samples_{max_samples}\"]\n",
     "#     variogram=variogram,\n",
     "#     search=SearchNeighborhood(\n",
     "#         ellipsoid=search_ellipsoid,\n",
@@ -614,15 +608,15 @@
     "#     ),\n",
     "# )\n",
     "#\n",
-    "# print(f\"Kriging source: {params.source}\")\n",
+    "# print(f\"Kriging source: CU_pct from pointset\")\n",
     "# print(f\"Search ellipsoid: major={search_ellipsoid.ranges.major}m\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### WIP: Run Kriging Task\n",
+    "## WIP. Run Kriging Task\n",
     "\n",
     "Submit and run the kriging task using Evo Compute."
    ]
@@ -633,23 +627,21 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: Run kriging task\n",
-    "#\n",
     "# from evo.compute.tasks import run\n",
     "#\n",
     "# # Submit kriging task (progress feedback is shown by default)\n",
     "# print(\"Submitting kriging task...\")\n",
     "# results = await run(manager, [params])\n",
     "#\n",
     "# print(f\"Kriging complete!\")\n",
-    "# print(f\"Result: {results[0].message}\")"
+    "# print(f\"Result: {results[0].status}\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### WIP: View Kriging Results\n",
+    "## WIP. View Kriging Results\n",
     "\n",
     "Refresh the block model and view the estimated grades."
    ]
@@ -660,13 +652,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: View kriging results\n",
-    "#\n",
-    "# # Refresh block model to see new attributes\n",
-    "# block_model = await block_model.refresh()\n",
-    "#\n",
-    "# # Display the block model (pretty-printed with Portal/Viewer links)\n",
-    "# block_model"
+    "# Refresh block model to see new attributes\n",
+    "await block_model.refresh()\n",
+    "\n",
+    "# Display the block model version (shows updated columns)\n",
+    "block_model.version"
    ]
   },
   {
@@ -675,21 +665,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: Query estimated values\n",
-    "#\n",
-    "# # Get the kriged values as a DataFrame\n",
-    "# results_df = await block_model.to_dataframe(columns=[\"CU_estimate\"])\n",
-    "#\n",
-    "# print(f\"Estimated {len(results_df)} blocks\")\n",
-    "# print(f\"\\nStatistics:\")\n",
-    "# print(results_df[\"CU_estimate\"].describe())"
+    "# Get the kriged values as a DataFrame\n",
+    "results_df = block_model.cell_data\n",
+    "\n",
+    "print(f\"Estimated {len(results_df)} blocks\")\n",
+    "print(\"\\nStatistics for CU_estimate:\")\n",
+    "print(results_df[\"CU_estimate\"].describe())"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### WIP: Running Multiple Kriging Scenarios\n",
+    "## WIP. Running Multiple Kriging Scenarios\n",
     "\n",
     "Run multiple kriging tasks concurrently to compare different parameters."
    ]
@@ -700,8 +688,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# WIP: Multiple kriging scenarios with different max_samples\n",
-    "#\n",
     "# max_samples_values = [5, 10, 15, 20]\n",
     "#\n",
     "# # Create parameter sets for each scenario\n",
@@ -721,7 +707,7 @@
     "# # Run all scenarios in parallel\n",
     "# print(f\"Submitting {len(parameter_sets)} kriging tasks...\")\n",
     "# results = await run(manager, parameter_sets)\n",
-    "# print(f\"All {len(results)} scenarios completed!\")"
+    "# print(f\"All {len(results)} scenarios completed!\")\n"
    ]
   }
  ],

code-samples/geoscience-objects/running-kriging-compute/running-kriging-compute.ipynb

@@ -1,7 +1,7 @@
 [project]
 name = "evo-blockmodels"
 description = "Python SDK for using the Seequent Evo Geoscience Block Model API"
-version = "0.1.1"
+version = "0.2.0"
 requires-python = ">=3.10"
 license-files = ["LICENSE.md"]
 dynamic = ["readme"]
@@ -21,14 +21,14 @@ Homepage = "https://www.seequent.com/"
 Documentation = "https://developer.seequent.com/"
 
 [project.optional-dependencies]
-pyarrow = ["pyarrow>=19.0.0"]
+utils = ["pyarrow>=19.0.0", "pandas>=2.0.0"]
 aiohttp = ["evo-sdk-common[aiohttp]>=0.1.0"]
 notebooks = ["evo-sdk-common[notebooks]>=0.1.0"]
 
 [dependency-groups]
 # Dev dependencies. The version is left unspecified so the latest is installed.
 test = [
-    "evo-blockmodels[aiohttp,pyarrow]",
+    "evo-blockmodels[aiohttp,utils]",
     "parameterized==0.9.0",
     "pytest",
 ]
@@ -40,7 +40,7 @@ dev = [
     {include-group = "test"}
 ]
 notebooks = [
-    "evo-blockmodels[aiohttp,notebooks,pyarrow]",
+    "evo-blockmodels[aiohttp,notebooks,utils]",
     "jupyter",
 ]
 

packages/evo-blockmodels/pyproject.toml

@@ -33,7 +33,7 @@
     Version,
 )
 from .endpoints import models
-from .endpoints.api import ColumnOperationsApi, JobsApi, MetadataApi, OperationsApi, VersionsApi
+from .endpoints.api import ColumnOperationsApi, JobsApi, MetadataApi, OperationsApi, ReportsApi, VersionsApi
 from .endpoints.models import (
     AnyUrl,
     BBox,
@@ -127,6 +127,7 @@ def __init__(self, environment: Environment, connector: APIConnector, cache: ICa
         self._operations_api = OperationsApi(connector)
         self._column_operations_api = ColumnOperationsApi(connector)
         self._metadata_api = MetadataApi(connector)
+        self._reports_api = ReportsApi(connector)
         self._cache = cache
 
     @classmethod
@@ -311,7 +312,6 @@ async def _upload_data(self, bm_id: uuid.UUID, job_id: uuid.UUID, upload_url: st
 
         cache_location = get_cache_location_for_upload(self._cache, self._environment, job_id)
         pyarrow.parquet.write_table(data, cache_location)
-
         # Upload the data
         upload = BlockModelUpload(self._connector, self._environment, bm_id, job_id, upload_url)
         await upload.upload_from_path(cache_location, self._connector.transport)
@@ -368,6 +368,19 @@ async def list_block_models(self) -> list[BlockModel]:
 
         return [self._bm_from_model(m) for m in response.results]
 
+    async def get_block_model(self, bm_id: UUID) -> BlockModel:
+        """Get a block model by ID.
+
+        :param bm_id: The ID of the block model to retrieve.
+        :return: The BlockModel metadata.
+        """
+        response = await self._metadata_api.retrieve_block_model(
+            bm_id=str(bm_id),
+            workspace_id=str(self._environment.workspace_id),
+            org_id=str(self._environment.org_id),
+        )
+        return self._bm_from_model(response)
+
     async def list_all_block_models(self, page_limit: int | None = 100) -> list[BlockModel]:
         """Return all block models for the current workspace, following paginated responses.
 
@@ -527,6 +540,26 @@ async def create_block_model(
             version = await self._add_new_columns(create_result.bm_uuid, initial_data, units, geometry_change)
         return self._bm_from_model(create_result), version
 
+    async def add_new_subblocked_columns(
+        self,
+        bm_id: UUID,
+        data: Table,
+        units: dict[str, str] | None = None,
+    ):
+        """Add new columns to an existing sub-blocked block model. This will not change the sub-blocking structure, thus the provided data must match existing sub-blocks in the model.
+
+        Units for the columns can be provided in the `units` dictionary.
+
+        This method requires the `pyarrow` package to be installed, and the 'cache' parameter to be set in the constructor.
+
+        :param bm_id: The ID of the block model to add columns to.
+        :param data: The data containing the new columns to add.
+        :param units: A dictionary mapping column names within `data` to units.
+        :raises CacheNotConfiguredException: If the cache is not configured.
+        :return: The new version of the block model with the added columns.
+        """
+        return await self._add_new_columns(bm_id, data, units, geometry_change=False)
+
     async def _add_new_columns(
         self,
         bm_id: UUID,
@@ -580,26 +613,6 @@ async def _add_new_columns(
         version = await self._upload_data(bm_id, update_response.job_uuid, str(update_response.upload_url), data)
         return _version_from_model(version)
 
-    async def add_new_subblocked_columns(
-        self,
-        bm_id: UUID,
-        data: Table,
-        units: dict[str, str] | None = None,
-    ):
-        """Add new columns to an existing sub-blocked block model. This will not change the sub-blocking structure, thus the provided data must match existing sub-blocks in the model.
-
-        Units for the columns can be provided in the `units` dictionary.
-
-        This method requires the `pyarrow` package to be installed, and the 'cache' parameter to be set in the constructor.
-
-        :param bm_id: The ID of the block model to add columns to.
-        :param data: The data containing the new columns to add.
-        :param units: A dictionary mapping column names within `data` to units.
-        :raises CacheNotConfiguredException: If the cache is not configured.
-        :return: The new version of the block model with the added columns.
-        """
-        return await self._add_new_columns(bm_id, data, units, geometry_change=False)
-
     async def add_new_columns(
         self,
         bm_id: UUID,

packages/evo-blockmodels/src/evo/blockmodels/client.py

@@ -230,3 +230,16 @@ class Version:
     """
     Columns within this version
     """
+
+    def __repr__(self) -> str:
+        """Return a concise string representation of the version."""
+        col_names = [c.title for c in self.columns]
+        bbox_str = ""
+        if self.bbox:
+            bbox_str = f", bbox=i[{self.bbox.i_minmax.min}-{self.bbox.i_minmax.max}] j[{self.bbox.j_minmax.min}-{self.bbox.j_minmax.max}] k[{self.bbox.k_minmax.min}-{self.bbox.k_minmax.max}]"
+        return (
+            f"Version(id={self.version_id}, "
+            f"created={self.created_at.strftime('%Y-%m-%d %H:%M:%S')}, "
+            f"by={self.created_by.name or self.created_by.email}{bbox_str}, "
+            f"columns={col_names})"
+        )