Add minimal KServe + MinIO model serving example notebook#42
Add minimal KServe + MinIO model serving example notebook#42
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive example notebook demonstrating end-to-end model serving using KServe and MinIO. The notebook provides a practical guide for data scientists to train a model, upload it to MinIO object storage, deploy it via KServe, and test the deployed service.
Changes:
- Added a Jupyter notebook that demonstrates training an sklearn SVM classifier on the Iris dataset, uploading it to MinIO, and deploying it as a KServe InferenceService
- Includes complete workflow from model training through deployment testing with API key authentication
- Provides code examples for interacting with MinIO using s3fs and managing KServe resources via kubectl
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Korel
force-pushed
the
kserve-minimal-s3-example
branch
6 times, most recently
from
3dda804 to
c2004b6
Compare
|
Thanks @Korel for the example. We were missing this one for quite some time. I just added a bunch of opinions. Feel free to close them. Overall i would love to reduce the complexity and remove all "automation" that is not strictly necessary in favor of a clear "easy" example. This is probably the first serving example a customer would encounter. |
@tibrk thanks for the review! However, I do not see any comments on the changes, where have you added your review result? |
| "# The notebook is already setup with minio credentials for the bucket that start with <namespace>-data\n", | ||
| "with open(\"/var/run/secrets/kubernetes.io/serviceaccount/namespace\", \"r\") as namespace_file:\n", | ||
| " namespace = namespace_file.read()\n", | ||
| "s3_bucket = f\"{namespace}-data\"\n", |
There was a problem hiding this comment.
This seems to be a bit "complicated". I would prefer a comment explaining what the user has to define. This way we "educate" them for future projects.
| "manifest_file_name=\"inferenceservice.yaml\"\n", | ||
| "with open(manifest_file_name, \"w\") as manifest_file:\n", | ||
| " manifest_file.write(inference_service_manifest)" | ||
| ] |
There was a problem hiding this comment.
Do we really need to create the yaml file using python? We could just leave it as a yaml file in the repository.
Again maybe don't use templating but educate the user what each part means.
|
@tibrk Ok now I see them 😄 |
|
Yeah, forgot to click the submit review button.... Again Regarding the Minio Upload.
|
|
I can confirm this works! Thanks! I'm torn on the complexity vs automation. |
Korel
force-pushed
the
kserve-minimal-s3-example
branch
from
c2004b6 to
4f19a77
Compare
* Train a simple sklearn SVM classifier on the Iris dataset and serialize it with joblib * Upload the resulting `model.joblib` to a MinIO S3 bucket using `s3fs` * Generate and apply a KServe `InferenceService` manifest pointing to the uploaded model * Wait for the `InferenceService` to become ready and retrieve its URL * Send a test prediction request to the deployed service using an API key
Korel
force-pushed
the
kserve-minimal-s3-example
branch
from
4f19a77 to
2e5275b
Compare
|
@tibrk @geier I tried to create a version without the automation in a folder called minimal-s3-model-manual , used markdown sections to explain what needs to be done. Please take a look and compare both versions and give an opinion about which one we should go with :) I'll use git rebase to remove the one we don't want. I am a fan of the automation because it makes it easier for me to test stuff and I understand what's going on, but that is not a good arguement for the datascientist which lacks the information about the platform |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
I'm also a big fan of automation for using it for testing and just being able to run "run all cells", but I do see Timos point |
Korel
force-pushed
the
kserve-minimal-s3-example
branch
2 times, most recently
from
a9c7b99 to
05dcbab
Compare
Korel
force-pushed
the
kserve-minimal-s3-example
branch
from
05dcbab to
67bb17a
Compare
|
@Korel : Thanks. Looks great in general. Regarding the choice between the manual and the automated version, i'd like to propose a thir options, as discussed. Lets use the automated one, but add a lot of text, explanations and alternative ways to do it in markdown cells like the ones used in the manual example. E.g. "The snippet below waits for the inference service to be up and running, however, you can also have a look at the [ui](link to the ui) in order so see what is going on. |
|
@ALL: While testing this I realized, that this does not work on "older" deployments. Which means we should really come up with a versioning concept for our examples, Ideally before merging this. |
|
I just came back to this PR since #47 is also covering inference. Maybe we should align them? |
| "inference_service_url = \"\" # Enter the URL of the deployed InferenceService, such as https://<cluster-domain>/serving/<namespace>/<model-name>\n", | ||
| "inference_service_name = \"\" # Enter the name of the deployed InferenceService, such as <model-name>\n", | ||
| "if not inference_service_url or not inference_service_name:\n", | ||
| " raise RuntimeError(\"Please provide the URL and name of the deployed InferenceService, such as https://<cluster-domain>/serving/<namespace>/<model-name> and <model-name>\" +\n", |
There was a problem hiding this comment.
This needs to be the inference service name, not the model name.
| "\n", | ||
| "After the InferenceService is ready, you can test it with a sample request. You can use the following command to get the URL of the InferenceService: \n", | ||
| "```bash\n", | ||
| "kubectl get inferenceservice <model-name> -n <namespace> -o jsonpath='{.status.url}'\n", |
There was a problem hiding this comment.
Same here. (inference service name)
I needed to test kserve getting a model from minio and realized we don't have an example for it, so I created one. Any feedback is appreciated :)
The notebook does:
model.joblibto a MinIO S3 bucket usings3fsInferenceServicemanifest pointing to the uploaded modelInferenceServiceto become ready and retrieve its URLI hope it is not using too many jupyter notebook magic that makes it hard to understand from a datascientist perspective.