Graph typing#5
Merged
Merged
Conversation
Convert OnnxNodeGenerator to an IIncrementalGenerator and reorganize generation logic; introduce MarkdownHelper and TextHelper utilities and add AcceptTypeAttribute to support generated [AcceptType<...>] annotations. Update generator to emit accept-type attributes, improved doc/formatting helpers, and a TryFromProto resolver. Add TextHelper, AcceptTypeAttribute, and a sample ReluInputOptionsX in ConsoleTest; adjust OnnxGraph imports/usage accordingly. These changes improve codegen robustness, documentation handling, and type acceptance metadata in generated sources.
Insert generation of [AcceptType<T>] attributes when building node fields and property declarations by calling GetAcceptTypeAttributes in both places. Update GetAttributeComment to accept an OperatorAttribute parameter. Replace direct use of x.Types with types derived from x.Type via FromProto and MapType to compute allowed types. These changes ensure generated properties include AcceptType annotations reflecting the attribute's accepted CLR types for downstream tooling/validation.
Extend type mapping and runtime support for ONNX non-tensor value types (sequence, map, sparse tensor, optional, opaque). - Refactor OnnxNodeGenerator.MapType to normalize types and use helpers (TryUnwrapType, TryMapTypedEdge, TryMapElementType) to map concrete ONNX type strings (e.g. tensor(int32), seq(...), map(...)). - Implement FromProto handling in OnnxValue so sequence, map, sparse tensor, opaque and optional types are deserialized instead of throwing. - Add concrete OnnxValueType subclasses: OnnxSparseTensorType, OnnxSequenceType, OnnxMapType, OnnxOptionalType, OnnxOpaqueType with ToProto(), constructors and ToString() implementations. - Add tests: a round-trip Save/Load test for non-tensor graph value types and additional MapType unit tests (including a coverage check against bundled schema). Also minor test cleanup (System.Text.Json import and enum reference fix). These changes enable mapping, serialization, and deserialization of a wider set of ONNX value types beyond tensors.
Include Assets\gtcrn_simple.onnx as an OnnxModel in Onnxify.ConsoleTest.csproj so the test project includes the GTCRN model for testing and packaging.
Rename and clarify option builder lists (factoryInputOutputOptions, protoInputOutputOptions), switch string generation to explicit blocks for readability, and replace inputs/outputs length checks from .Length to .Count to support collection types. Replace LINQ-based array construction for node inputs/outputs with explicit List<IOnnxGraphEdge?> population using indexed ElementAtOrDefault and graph.GetValue, preserving nulls. Add ArgumentNullException.ThrowIfNull checks to OnnxGraph.GetNode and OnnxGraph.GetValue for safer public API usage.
Use ONNX historical schemas and deterministic ordering, update build includes, and regenerate operator assets. CMake: replace manual include dirs with onnxruntime_add_include_to_target for onnx, onnx_proto, protobuf and nlohmann_json; adjust target_link_options formatting. main.cpp: reorder includes, switch to ONNX_NAMESPACE::OpSchemaRegistry::get_all_schemas_with_history(), add stable sort by domain/Name/SinceVersion, and tweak summary text. Assets: regenerate src/Onnxify/Assets/onnx_operators.json to reflect the collected/ordered schema changes (many operators updated/added, versions, types and docs adjusted).
Introduce helpers to prefer the latest operator schema versions (GetLatestSchemaByKey / GetLatestOperatorSchemas) and update generators to use them. Adjust node generation to handle variable input/output counts (use Math.Max and sized lists), minor API visibility and formatting tweaks (OperatorKey made internal, whitespace/paren fixes), and change Distinct().ToList() usage. Add OnnxGraph.TryGetImportedOpset for opset lookup. Add extensive compatibility/unit tests and update test model opset creation to explicit OnnxModelCreationOptions where needed.
Introduce a source generator (OnnxCompatibilityMetadataGenerator) that produces bundled operator metadata from onnx_operators.json, and add a runtime validation system (OnnxCompatibilityValidation) and repository (OnnxOperatorSchemaRepository/OnnxBundledOperatorSchema) to resolve schemas and check compatibility. Integrate compatibility checks into OnnxModel (capture OpsetImports, enforce structural compatibility before serialization, and expose ValidateCompatibility), extend OnnxModelBaseOptions with OpsetImports and NodeTypeResolutionStrategy, and update OnnxNode.FromProto to honor node type resolution strategy and fall back to generic nodes or throw on incompatible structures. Also move MarkdownHelper/TextHelper into a Helpers folder.
Introduce a new compatibility subsystem and update generators/usage. Adds multiple files under Compatibility: OnnxBundledOperatorSchema, OnnxCompatibilityValidationModels, OnnxCompatibilityValidationResult, OnnxCompatibilityValidator, OnnxGeneratedCompatibilityMetadata, OnnxOperatorSchemaResolver, OnnxOperatorSupport and related types (OperatorKey, resolution enums). The validator supports structural and target-runtime modes, emits detailed diagnostics, and validates arity/attributes against bundled schemas. Replaces the large removed OnnxCompatibilityValidation.cs with these split implementations and migrates the singleton name from Repository to Resolver. Update source generators (OnnxCompatibilityMetadataGenerator, OnnxNodeGenerator) to collect AdditionalText inputs, use mapped file lookup, adjust generated code formatting/named parameters, and change generator method visibilities. Update OnnxModel and OnnxNode to use OnnxOperatorSchemaResolver and refactor FromProto handling for nodes.
Add two shell installers (install-onnxify-cli.sh, install-onnxify-skills.sh) to pack/install the local Onnxify CLI and to copy bundled Codex skills. Update README.md to document NuGet packages (table with links), expand requirements, mark a couple TODOs as done, and provide cross-platform Getting Started instructions and examples for using the new install scripts and installing the Codex skill. Also include guidance to restart Codex after installation.
Add a new ModelGenerator reference (references/model-generator.md) describing purpose, usage, csproj setup, naming overrides, runtime examples, and repo entry points. Expand onnxify-internal SKILL.md with a longer description covering repository-facing packages and add a Code Style section (preferred vertical argument formatting and C++-style constant names). Update api-surface to document the ModelGenerator surface and tests. Reformat examples to follow the new style and fix minor docs formatting/clarity in ml-pipelines, onnxify-graph-examples, safetensors, and torchsharp-export-examples (including a note to dispose TorchInferenceResult and a clarification about the final linear projection).
Expand docs and SKILL surface for Onnxify.ModelGenerator, including usage guidance, recommended csproj setup (OnnxModel item), naming overrides, runtime examples, and notes. Minor update to Onnxify.ProjectGenerator docs and release notes adding an overload to generate from an in-memory OnnxModel. Standardize newline constant naming in agent generators: rename NewLine to NEW_LINE and update NormalizeLineEndings in OperatorSkillGenerator.cs and TorchSharpConverterSkillGenerator.cs.
Standardize constant naming and adjust related code + tests. Key changes: - Rename many constant identifiers to UPPER_SNAKE_CASE (PipelineStageCategories, SafeTensors, generator keys, etc.) and update all usages. - Bump package versions to 0.0.0.12 in multiple project files. - OnnxModelGenerator: rename internal keys, add XML docs, update generated symbol names (e.g. MODEL_PROJECT_RELATIVE_PATH) and option lookups. - OnnxProjectGenerator: expose Generate(OnnxModel, ProjectGeneratorOptions) overload, use GetGeneratedModelFileName, and replace inline threshold constant with INLINE_TENSOR_ELEMENT_THRESHOLD. - ProjectGeneratorOptions: make InputModelPath nullable, add InputModelName, implement GetGeneratedModelFileName and helper GetInputModelStem for robust fallback naming. - SafeTensors: rename constants (MAX_HEADER_SIZE, LENGTH_PREFIX_SIZE, PREVIEW_EDGE_COUNT) and update code paths accordingly. - Fix bug in OnnxCompatibilityMetadataGenerator where MaximumInputs was computed from Outputs; now uses Inputs. - Update examples and tests to match new names/behaviour and add/expand tests for model loading, compatibility, and in-memory project generation. These changes are mostly mechanical renames, API stabilizations, and test updates to reflect the new naming and bug fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.