[Runtime] Migrate from high-level iree_runtime API to low-level VM/HAL API#160
Open
sjain-stanford wants to merge 22 commits intomainfrom
Open
[Runtime] Migrate from high-level iree_runtime API to low-level VM/HAL API#160sjain-stanford wants to merge 22 commits intomainfrom
sjain-stanford wants to merge 22 commits intomainfrom
Conversation
…L API (#82) Replace the iree_runtime_* wrapper layer (instance, session, call) with direct iree_vm_* and iree_hal_* APIs for precise lifetime control and device management: - iree_runtime_instance_t -> iree_vm_instance_t + global driver registry - iree_runtime_session_t -> iree_vm_context_t with explicit HAL/bytecode module registration - iree_runtime_call_t -> iree_vm_list_t + iree_vm_invoke Cache the resolved iree_vm_function_t in Graph during context creation to avoid repeated lookups on each execute() call. Use std::call_once for HAL driver registration since the global driver registry persists across VM instance lifetimes and does not support re-registration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
# Conflicts: # include/fusilli/backend/runtime.h # include/fusilli/graph/graph.h
iree_vm_invoke accepts NULL outputs when the function has void return. Fusilli's compiled functions write results in-place to buffer views passed as inputs, so no output marshaling is needed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Replace unused iree/base/status.h with iree/base/allocator.h and iree/base/config.h to directly provide iree_allocator_system and iree_device_size_t. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Already transitively included via iree/modules/hal/module.h. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Moves the input count computation from execute() to createVmContext() since the capacity is deterministic after compilation. Also fixes a missing +1 for the workspace buffer slot. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Adds vm prefix to disambiguate Graph's IREE VM members from other context_/function_ members in the codebase. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Unifies the ref ownership pattern across all VM input list pushes to explicitly retain then move, matching the workspace buffer style. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Wrap the iree_vm_list_t in a unique_ptr with a custom deleter to prevent resource leaks on early-return error paths. Follows the existing IreeVmContextDeleter / IreeHalDeviceDeleter pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Move vmContext_ assignment immediately after iree_vm_context_create so the unique_ptr owns cleanup from the start. Previously, failures in HAL module creation or bytecode loading would leak the raw context. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
std::optional default-constructs to empty, making the explicit = std::nullopt unnecessary on member variable declarations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
createAMDGPUDevice() was not releasing the driver after iree_hal_driver_create_device_by_id, leaking it on both success and failure paths. The device internally retains its own reference to the driver, so the caller's reference must be released separately. Mirrors the pattern already used in createCPUDevice(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
Signed-off-by: Sambhav Jain <sambhav@alumni.stanford.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #82.
Fixes #15.
Replace the high-level
iree_runtime_*wrapper layer with directiree_vm_*andiree_hal_*APIs for precise lifetime control and device management.Key type mapping
iree_runtime_instance_tiree_vm_instance_t+ global driver registryiree_runtime_session_tiree_vm_context_twith explicit HAL/bytecode module registrationiree_runtime_call_tiree_vm_list_t+iree_vm_invokeMajor changes
runtime.h,backend.h,handle.h,graph.h): Replace alliree_runtime_*types and calls with their low-level equivalents across instance creation, device creation, context setup, bytecode loading, and function invocation (addresses [Core] Move away from highleveliree_runtime_*API #82).runtime.h): Extract driver registration intoregisterHalDriversOnce()usingstd::call_once, since the global driver registry persists across VM instance lifetimes and does not support re-registration.runtime.h,graph.h): Cache resolvediree_vm_function_tinGraph::vmFunction_duringcreateVmContext()to avoid repeated lookups on eachexecute()call (addresses [Perf] Memoize runtime call initialization and reuse calls betweenGraph::executeinvocations #15).runtime.h,graph.h): Pre-computevmInputListCapacity_duringcreateVmContext()to avoid recomputing on everyexecute()call.runtime.h): Passnullptrfor outputs iniree_vm_invokesince compiled functions write results in-place (void return).Bug fixes
runtime.h): Wrapiree_vm_list_tinIreeVmListUniquePtrTypeto prevent resource leaks on early-return error paths inexecute().runtime.h): AssignvmContext_immediately afteriree_vm_context_createso failures in HAL module creation or bytecode loading don't leak the raw context.runtime.h):createAMDGPUDevice()was not releasing the driver after device creation, leaking it on both success and failure paths. The device internally retains its own reference, so the caller's reference must be released separately (mirrors the pattern increateCPUDevice()).Minor changes
backend.h): AddIreeVmContextDeleter,IreeVmListDeleterand correspondingunique_ptraliases.backend.h,buffer.h,handle.h,runtime.h, test files): Replace<iree/runtime/api.h>with granular<iree/hal/api.h>,<iree/vm/api.h>,<iree/modules/hal/module.h>, etc.tensor_attributes.h,graph.h): Remove= std::nulloptonstd::optionalmembers that default-construct to empty.tensor_attributes.h,graph_import.h,fusilli_plugin.cpp,test_fusilli_plugin_api.cpp,runtime.h): Normalize// C++ 20→// C++20across the codebase.handle.h,graph.h,runtime.h,test_graph.cpp): Update comments to reflect VM/HAL terminology.No changes to CMakeLists.txt, samples, benchmarks, or public API.
🤖 Generated with Claude Code