Skip to content

Feature/minimal mlflow inference examle#47

Open
tibrk wants to merge 3 commits intomainfrom
feature/minimal-mlflow-inference-examle
Open

Feature/minimal mlflow inference examle#47
tibrk wants to merge 3 commits intomainfrom
feature/minimal-mlflow-inference-examle

Conversation

@tibrk
Copy link
Contributor

@tibrk tibrk commented Mar 12, 2026

This pull request introduces a minimal example for deploying and testing an MLflow-tracked model as a KServe InferenceService using both v1 and v2 inference protocols.

Deployment and Protocol Support:

  • Added example InferenceService manifests for both v1 and v2 KServe inference protocols, demonstrating how to deploy an MLflow model using KServe, including placeholders for customization. [1] [2] [3]
  • Added a deep dive into both protocol versions. Explaining differences and providing a notebook to get some hands on experience with both.

Testing and Usage:

  • Added a Python script (test_inference_service.py) to send authenticated inference requests to the deployed service, supporting both v1 and v2 protocols and reading request bodies from JSON files.

Sample Request Bodies:

  • Provided sample JSON request bodies formatted for both v1 and v2 inference protocols, using realistic data from the mobile price classification dataset. [1] [2]

Documentation:

  • Added a comprehensive README.md explaining prerequisites, deployment steps, environment variable configuration, testing instructions, and the request body format for the minimal MLflow model inference example.

@tibrk tibrk requested a review from geier March 12, 2026 12:15
@tibrk tibrk force-pushed the feature/minimal-mlflow-inference-examle branch from a820b8f to 917fc63 Compare March 12, 2026 12:57
@tibrk tibrk force-pushed the feature/minimal-mlflow-inference-examle branch from 917fc63 to a2eceff Compare March 12, 2026 13:31
@tmvfb tmvfb self-requested a review March 12, 2026 18:16
@tmvfb
Copy link
Collaborator

tmvfb commented Mar 12, 2026

@tibrk, thanks for the contribution, testing this! Before testing I have a few nitpicks/suggestions:

  1. Could we please rename the files to look conventionally (.ipynb should be snake case, .yaml should be kebab case, we could also shorten InferenceService to isvc).

  2. If we can remove the duplicated v2-mlflow-inference-body.json, let's go for it.

  3. I'd suggest clearer names for folders, e.g.

    mlflow-kserve-minimal-example
    mlflow-kserve-inference-protocols
    
  4. Maybe we'll need a short README in the /serving folder after this PR, since we'll have 4 subfolders already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants