[Feat] Support using same monitor instance and regist python stats

Author: flesher0813

docs/source/developer-guide/add_metrics.md

@@ -0,0 +1,113 @@
+# Custom Metrics
+UCM supports custom metrics with bidirectional updates from both Python and C++ runtimes. The unified monitoring interface provides the ability to mutate stats across language boundaries through a shared metrics registry.
+
+## Architecture Overview
+The metrics consists of these components below:
+- **metrics** : Central stats registry that manages all metric lifecycle operations (registration, creation, updates, queries)
+- **observability.py** : Prometheus integration layer that handles metric exposition and multiprocess collection
+- **metrics_config.yaml** : Declarative configuration that defines which custom metrics to register and their properties
+
+## Getting Started
+### Define Metrics in YAML Configuration
+Prometheus provides three fundamental metric types: Counter, Gauge, and Histogram. UCM implements corresponding wrappers for each type. The method for adding new metrics is as follows; please refer to the [example YAML](https://github.com/ModelEngine-Group/unified-cache-management/blob/develop/examples/metrics/metrics_configs.yaml) for more detailed information. 
+```yaml
+log_interval: 5  # Interval in seconds for logging metrics
+
+prometheus:
+  multiproc_dir: "/vllm-workspace"  # Directory for Prometheus multiprocess mode
+
+  metric_prefix: "ucm:"
+  
+  # Enable/disable metrics by category
+  enabled_metrics:
+    counters: true
+    gauges: true
+    histograms: true
+  
+  # Counter metrics configuration
+  counters:
+    - name: "received_requests"
+      documentation: "Total number of requests sent to ucm"
+  
+  # Gauge metrics configuration
+  gauges:
+    - name: "lookup_hit_rate"
+      documentation: "Hit rate of ucm lookup requests"
+      multiprocess_mode: "livemostrecent"
+  
+  # Histogram metrics configuration
+  histograms:
+    - name: "load_requests_num"
+      documentation: "Number of requests loaded from ucm"
+      buckets: [1, 5, 10, 20, 50, 100, 200, 500, 1000]
+```
+
+### Use Monitor APIs to Update Stats
+The monitor provides a unified interface for metric operations. Note that the workflow requires registering a stats class before creating an instance.
+:::::{tab-set}
+:sync-group: install
+
+::::{tab-item} Python side interfaces
+:selected:
+:sync: py
+**Lifecycle Methods**
+- `register_istats(name, py::object)`: Register a new stats class implementation.
+- `create_stats(name)`: Create and initialize a registered stats object.
+
+**Operation Methods**
+- `update_stats(name, dict)`: Update specific fields of a specific stats object.
+- `get_stats(name)`: Retrieve current values of a specific stats object.
+- `get_stats_and_clear(name)`:  Retrieve and reset a specific stats object.
+- `get_all_stats_and_clear()`: Retrieve and reset all stats objects.
+- `reset_stats(name)`: Reset a specific stats object to initial state.
+- `reset_all()`: Reset all stats registered in monitor.
+
+**Example:** Using built-in ConnStats
+```python
+from ucm.integration.vllm.conn_stats import ConnStats
+from ucm.shared.metrics import ucmmonitor
+
+conn_stats = ConnStats(name="ConnStats")
+ucmmonitor.register_stats("ConnStats", conn_stats) # Register stats
+ucmmonitor.create_stats("ConnStats") # Create a stats obj
+
+# Update stats
+ucmmonitor.update_stats(
+    "ConnStats",
+    {"interval_lookup_hit_rates": external_hit_blocks / len(ucm_block_ids)},
+)
+
+```
+See more detailed example in [test case](https://github.com/ModelEngine-Group/unified-cache-management/tree/develop/ucm/shared/test/example).
+
+::::
+
+::::{tab-item} C++ side interfaces
+:sync: cc
+**Lifecycle Methods**
+- `RegistStats(std::string name, Creator creator)`: Register a new stats class implementation.
+- `CreateStats(const std::string& name)`: Create and initialize a registered stats object.
+
+**Operation Methods**
+- `UpdateStats(const std::string& name, const std::unordered_map<std::string, double>& params)`: Update specific fields of a specific stats object.
+- `ResetStats(const std::string& name)`: Retrieve current values of a specific stats object.
+- `ResetAllStats()`: Retrieve and reset a specific stats object.
+- `GetStats(const std::string& name)`: Retrieve and reset all stats objects.
+- `GetStatsAndClear(const std::string& name)`: Reset a specific stats object to initial state.
+- `GetAllStatsAndClear()`: Reset all stats registered in monitor.
+
+**Example:** Implementing custom stats in C++
+UCM supports custom metrics by following steps:
+- Step 1: linking the static library monitor_static
+   ```c++
+    target_link_libraries(xxxstore PUBLIC storeinfra monitor_static)
+    ```
+- Step 2: Inheriting from the IStats class to implement custom stats classes
+- Step 3: Register stats class to monitor
+- Step 4: Create stats object in monitor
+- Step 5: Update or get stats info using operation methods
+
+See more detailed example in [test case](https://github.com/ModelEngine-Group/unified-cache-management/tree/develop/ucm/shared/test/case).
+
+::::
+:::::

docs/source/index.md

@@ -63,6 +63,7 @@ user-guide/metrics/metrics
 :caption: Developer Guide
 :maxdepth: 1
 developer-guide/contribute
+developer-guide/add_metrics
 :::
 
 :::{toctree}

ucm/integration/vllm/conn_stats.py

@@ -0,0 +1,20 @@
+from ucm.shared.metrics import ucmmonitor
+
+
+class ConnStats:
+    def __init__(self, name: str = "PyStats1"):
+        self._name = name
+        self._data = {}
+
+    def Name(self) -> str:
+        return self._name
+
+    def Update(self, params):
+        for k, v in params.items():
+            self._data.setdefault(k, []).append(v)
+
+    def Reset(self):
+        self._data.clear()
+
+    def Data(self):
+        return self._data.copy()

ucm/integration/vllm/ucm_connector.py

@@ -17,9 +17,10 @@
 from vllm.platforms import current_platform
 from vllm.v1.core.sched.output import SchedulerOutput
 
+from ucm.integration.vllm.conn_stats import ConnStats
 from ucm.logger import init_logger
+from ucm.observability import UCMStatsLogger
 from ucm.shared.metrics import ucmmonitor
-from ucm.shared.metrics.observability import UCMStatsLogger
 from ucm.store.factory import UcmConnectorFactory
 from ucm.store.ucmstore import Task, UcmKVStoreBase
 from ucm.utils import Config
@@ -172,12 +173,14 @@ def __init__(self, vllm_config: "VllmConfig", role: KVConnectorRole):
 
         self.metrics_config = self.launch_config.get("metrics_config_path", "")
         if self.metrics_config:
+            conn_stats = ConnStats(name="ConnStats")
+            ucmmonitor.register_stats("ConnStats", conn_stats)
+            ucmmonitor.create_stats("ConnStats")
             self.stats_logger = UCMStatsLogger(
                 vllm_config.model_config.served_model_name,
                 self.global_rank,
                 self.metrics_config,
             )
-            self.monitor = ucmmonitor.StatsMonitor.get_instance()
 
         self.synchronize = (
             torch.cuda.synchronize
@@ -236,7 +239,7 @@ def get_num_new_matched_tokens(
             f"hit external: {external_hit_blocks}"
         )
         if self.metrics_config:
-            self.monitor.update_stats(
+            ucmmonitor.update_stats(
                 "ConnStats",
                 {"interval_lookup_hit_rates": external_hit_blocks / len(ucm_block_ids)},
             )
@@ -532,7 +535,7 @@ def start_load_kv(self, forward_context: "ForwardContext", **kwargs) -> None:
             / 1024
         )  # GB/s
         if self.metrics_config and is_load:
-            self.monitor.update_stats(
+            ucmmonitor.update_stats(
                 "ConnStats",
                 {
                     "load_requests_num": num_loaded_request,
@@ -622,7 +625,7 @@ def wait_for_save(self) -> None:
             / 1024
         )  # GB/s
         if self.metrics_config and is_save:
-            self.monitor.update_stats(
+            ucmmonitor.update_stats(
                 "ConnStats",
                 {
                     "save_requests_num": num_saved_request,

ucm/shared/metrics/observability.py ucm/observability.pyucm/shared/metrics/observability.py renamed to ucm/observability.py

@@ -195,7 +195,7 @@ def log_prometheus(self, stats: Any):
             try:
                 metric_mapped = self.metric_mappings[stat_name]
                 if metric_mapped is None:
-                    logger.warning(f"Stat {stat_name} not initialized.")
+                    logger.debug(f"Stat {stat_name} not initialized.")
                     continue
                 metric_obj = getattr(self, metric_mapped["attr"], None)
                 metric_type = metric_mapped["type"]
@@ -213,8 +213,8 @@ def log_prometheus(self, stats: Any):
                         else:
                             value = []
                     self._log_histogram(metric_obj, value)
-            except Exception as e:
-                logger.warning(f"Failed to log metric {stat_name}: {e}")
+            except Exception:
+                logger.debug(f"Failed to log metric {stat_name}")
 
     @staticmethod
     def _metadata_to_labels(metadata: UCMEngineMetadata):
@@ -267,8 +267,6 @@ def __init__(self, model_name: str, rank: int, config_path: str = ""):
         # Load configuration
         config = self._load_config(config_path)
         self.log_interval = config.get("log_interval", 10)
-
-        self.monitor = ucmmonitor.StatsMonitor.get_instance()
         self.prometheus_logger = PrometheusLogger.GetOrCreate(self.metadata, config)
         self.is_running = True
 
@@ -296,7 +294,7 @@ def _load_config(self, config_path: str) -> Dict[str, Any]:
     def log_worker(self):
         while self.is_running:
             # Use UCMStatsMonitor.get_states_and_clear() from external import
-            stats = self.monitor.get_stats_and_clear("ConnStats")
+            stats = ucmmonitor.get_all_stats_and_clear().data
             self.prometheus_logger.log_prometheus(stats)
             time.sleep(self.log_interval)
 

ucm/shared/metrics/CMakeLists.txt

@@ -1,15 +1,32 @@
-file(GLOB_RECURSE CORE_SRCS CONFIGURE_DEPENDS
-     "${CMAKE_CURRENT_SOURCE_DIR}/cc/stats/*.cc"
-     "${CMAKE_CURRENT_SOURCE_DIR}/cc/*.cc")
-add_library(monitor_static STATIC ${CORE_SRCS})
+file(GLOB_RECURSE DOMAIN_SRCS
+  "${CMAKE_CURRENT_SOURCE_DIR}/cc/domain/*.cc"
+  "${CMAKE_CURRENT_SOURCE_DIR}/cc/domain/stats/*.cc"
+)
+
+file(GLOB_RECURSE API_SRCS
+  "${CMAKE_CURRENT_SOURCE_DIR}/cc/api/*.cc"
+)
+
+add_library(monitor_static STATIC
+  ${DOMAIN_SRCS}
+  ${API_SRCS}
+)
+
 set_property(TARGET monitor_static PROPERTY POSITION_INDEPENDENT_CODE ON)
+
 target_include_directories(monitor_static PUBLIC
-    $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/cc>
-    $<INSTALL_INTERFACE:include>)
+  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/cc/domain>
+  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/cc/api>
+  $<INSTALL_INTERFACE:include>
+)
+
 set_target_properties(monitor_static PROPERTIES OUTPUT_NAME monitor)
 
 file(GLOB_RECURSE BINDINGS_SRCS CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/cpy/*.cc")
 pybind11_add_module(ucmmonitor ${BINDINGS_SRCS})
-target_link_libraries(ucmmonitor PRIVATE -Wl,--whole-archive monitor_static -Wl,--no-whole-archive)
-target_include_directories(ucmmonitor PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cc)
+target_link_libraries(ucmmonitor PRIVATE monitor_static)
+
+target_include_directories(ucmmonitor PRIVATE
+  "${CMAKE_CURRENT_SOURCE_DIR}/cc/api"
+)
 set_target_properties(ucmmonitor PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})

ucm/shared/metrics/cc/api/stats_monitor_api.cc

@@ -0,0 +1,64 @@
+/**
+ * MIT License
+ *
+ * Copyright (c) 2025 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ * */
+#include "stats_monitor_api.h"
+namespace UC::Metrics {
+
+void RegistStats(std::string name, Creator creator)
+{
+    StatsRegistry::GetInstance().RegisterStats(name, creator);
+}
+
+void CreateStats(const std::string& name) { StatsMonitor::GetInstance().CreateStats(name); }
+
+void UpdateStats(const std::string& name, const std::unordered_map<std::string, double>& params)
+{
+    StatsMonitor::GetInstance().UpdateStats(name, params);
+}
+
+void ResetStats(const std::string& name) { StatsMonitor::GetInstance().ResetStats(name); }
+
+void ResetAllStats() { StatsMonitor::GetInstance().ResetAllStats(); }
+
+StatsResult GetStats(const std::string& name)
+{
+    StatsResult result;
+    result.data = StatsMonitor::GetInstance().GetStats(name);
+    return result;
+}
+
+StatsResult GetStatsAndClear(const std::string& name)
+{
+    StatsResult result;
+    result.data = StatsMonitor::GetInstance().GetStatsAndClear(name);
+    return result;
+}
+
+StatsResult GetAllStatsAndClear()
+{
+    StatsResult result;
+    result.data = StatsMonitor::GetInstance().GetAllStatsAndClear();
+    return result;
+}
+
+} // namespace UC::Metrics