[QEFF. Finetuning]: rebasing ft_experimental against main#843
Open
smedhe wants to merge 70 commits intoquic:ft_experimentalfrom
Open
[QEFF. Finetuning]: rebasing ft_experimental against main#843smedhe wants to merge 70 commits intoquic:ft_experimentalfrom
smedhe wants to merge 70 commits intoquic:ft_experimentalfrom
Conversation
carry over patch quic#693 Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com>
Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>
Updating README, custom script for 2-layer instruction for Wan Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>
Added step wise instructions for MULTI NODE Finetuning. --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>
Add support for multi-node Distributed Data Parallel (DDP) training to the QEfficient finetuning pipeline. This enables scaling training across multiple nodes while keeping the existing single-node behavior unchanged. Commands for DDP across 2 servers: For the Master Addr or the Primary Machine, use node-rank as 0: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=0 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 For Node 1, use node-rank as 1: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=1 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
QEfficient should ignore providing `-mdp-load-partition-config` when `-mdp-dump-partition-config` is provided in compiler_options of compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>
Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>
Handled the edge case where num samples in a dataset are less than 20. Corrected the dataset link in grammar_dataset.py Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Since CCL is deactivated by default, the value of CCL lists (ccl_prefill and ccl_decode) should be None by default. In infer.py script the value of these lists wasn't None and it caused the problem of ccl activation by default. In this PR we addressed this issue. --------- Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>
In this PR: 1) We have modified the code to support PP+DDP on multi-server setup 2) Added preprocessing file for grammar dataset 3) Modified the naming convention for output dir to include the node rank of the server --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Added default NPI file for Gemma3. 1. Eliminates the need to provide NPI file as an extra argument by user. NPI file added as default, no need to provide it explicitly in the example script --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>
Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com> Co-authored-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Co-authored-by: Amit Raj <amitraj@qti.qualcomm.com>
…WQ and FP8 models. (quic#735) Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B test due to a compiler issue to unblock the CI --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
Updated Qeff version to mainline --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
Reverts quic#741 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
The decode‑only GPT‑OSS model was failing when executing subfunctions due to somehow considering a dynamic dim value during reduced‑sum calculation. This caused incorrect tensor reduction and resulted in compilation errors. The fix replaces the reduction logic with an einsum-based computation, ensuring stable and deterministic summation regardless of dimension shape. --------- Signed-off-by: asmigosw <asmigosw@qti.qualcomm.com>
- updated the random sampling gold text, ids for InternVL2_5-1B Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>
Support to skip export, compilation if qpc already exists - Updated Flux, wan configs, pipelines with qpc_path changes --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>
The SW issue came with prompt + generation length > SW. Fix 1. Cache updated with HybridSlidingWindowCache in cache utils --------- Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>
Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>
Fix gemma3 to support cb with new SW code Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>
This PR fixes subfunction-based export issues for the following models: 1. `bigcode/starcoder` 2. `ibm-granite/granite-20b-code-base-8k` 3. `ibm-granite/granite-20b-code-instruct-8k` 4. `Qwen3-30B-A3B-Instruct-2507` 5. `Mixtral-8x7B` In addition, it updates the Causal LM subfunction test file to make it more robust and resilient across models. --------- Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
Updated the mainline version to 1.22.0.dev0 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
qaic-exec is going to be deprecated. Updated the code to use new qaic-compile for compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>
- skip subfn handling in export utils for diffusers, we handle this in export() of diffuser models --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
…quic#645) - Added functionality to register dataset, model, optimizer, trainer objects in a registry and fetch the class of given object based on configuration provided. - Also, added simple test cases to verify the functionality. --------- Signed-off-by: meetkuma <meetkuma@qti.qualcomm.com>
) Adding a Script for Registering and Retrieving Optimizer Classes The script includes: get_optimizer() Returns the optimizer class and kwargs. Additionally, there is a test_optimizer.py script that validates the functionality of the optimizer registration and retrieval process. --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>
…ong with its test cases. (quic#647) Edited the SFTDataset class to enable custom dataset loading. Updated the dataset.py file to only enable support for SFTDataset type. Created test file to check the functionalities. --------- Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Adding a Script for Registering and Retrieving Callback Classes It has create_callback() function which creates an instance of callback. Additionally, there is a test_callbacks.py script that validates the functionality and retrieval process. --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>
Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>
- Added Base Model class and HF model class. - Base Model class will support FT for any custom model and will be a common skeleton for any model, including any HF model. - Added unit tests for these. --------- Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com>
This PR contains all the changes of PR quic#660 along with all the comments being addressed. The new PR was created due a rebase issue. Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Added Readme file for the parameters used in sample config. --------- Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com> Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com> Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com> Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com> Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com> Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com> Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: asmigosw <asmigosw@qti.qualcomm.com> Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com> Signed-off-by: meetkuma <meetkuma@qti.qualcomm.com> Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Onkar Chougule <168134249+ochougul@users.noreply.github.com> Co-authored-by: Mohit Soni <quic_mohisoni@quicinc.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: vjanfaza <vjanfaza@qti.qualcomm.com> Co-authored-by: Ann Kuruvilla <quic_akuruvil@quicinc.com> Co-authored-by: smedhe <smedhe@qti.qualcomm.com> Co-authored-by: asmigosw <asmigosw@qti.qualcomm.com> Co-authored-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Co-authored-by: Amit Raj <amitraj@qti.qualcomm.com> Co-authored-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com> Co-authored-by: Rishin Raj <rishinr@qti.qualcomm.com> Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Co-authored-by: Dipankar Sarkar <dipankar@qti.qualcomm.com> Co-authored-by: Meet Patel <meetkuma@qti.qualcomm.com> Co-authored-by: Swati Allabadi <quic_sallabad@quicinc.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com>
…l config (quic#747) This PR contain: 1.documentation for new finetune experimental stack. 2. Updates inconfig_manager.py --------- Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com> Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com> Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com> Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com> Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com> Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com> Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: asmigosw <asmigosw@qti.qualcomm.com> Signed-off-by: meetkuma <meetkuma@qti.qualcomm.com> Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Onkar Chougule <168134249+ochougul@users.noreply.github.com> Co-authored-by: vjanfaza <vjanfaza@qti.qualcomm.com> Co-authored-by: Ann Kuruvilla <quic_akuruvil@quicinc.com> Co-authored-by: smedhe <smedhe@qti.qualcomm.com> Co-authored-by: Rishin Raj <rishinr@qti.qualcomm.com> Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Co-authored-by: Dipankar Sarkar <dipankar@qti.qualcomm.com> Co-authored-by: Mohit Soni <quic_mohisoni@quicinc.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: asmigosw <asmigosw@qti.qualcomm.com> Co-authored-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Co-authored-by: Amit Raj <amitraj@qti.qualcomm.com> Co-authored-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com> Co-authored-by: Meet Patel <meetkuma@qti.qualcomm.com> Co-authored-by: Swati Allabadi <quic_sallabad@quicinc.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com>
…t file (quic#787) 1) Adding text field required by TRL's scripts. 2) Passing config_name in the load_dataset_builder 3) Updated test_dataset accordingly. Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com>
….py) and related code (quic#791) 1) Added FinetuningPipeline (finetune_experiemental.py) which integrates all the components added for HF-trainer and enable running fine tuning through it. 2) Added files to handle PEFT and training config. 3) Made changes in the config_manager and callbacks files. 4) Added unit tests for the FinetuningPipeline (test_finetune.py) 5) Updated tests in test_callback and test_config_manager based on above changes. Finetuning on openai/gsm8k for 5 epochs on single SOC gave the following numbers: {"eval_loss":1.0224987268447876,"eval_runtime":484.8933,"eval_samples_per_second":2.72,"eval_steps_per_second":2.72,"eval_entropy":0.9871161538059735,"eval_num_tokens":6525025.0,"eval_mean_token_accuracy":0.7452040632806826,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365} {"train_runtime":32856.1501,"train_samples_per_second":1.137,"train_steps_per_second":1.137,"total_flos":3.8132170931712e+16,"train_loss":1.0178058738101043,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365} Training loss at the start of training :1.5146, Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com>
Signed-off-by: Ann <quic_akuruvil@quicinc.com>
Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>
* Added PP support in HF trainer stack. * Updated the documentation for the same. * Sample command to test PP : QAIC_VISIBLE_DEVICES=0,1 python -m QEfficient.cloud.finetune_experimental QEfficient/finetune/experimental/configs/sample_pp_config.yaml --------- Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm.com> Co-authored-by: Swati Allabadi <sallabad@qti.qualcomm>
To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>
Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>
Modified test_finetune.py Changed optimizer names --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>
…set registration (quic#835) Added example script for registering seq_completion dataset_type and also updated the hf_finetune.md. --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe
requested review from
ochougul,
quic-amitraj,
quic-hemagnih and
quic-rishinr
as code owners
Contributor
quic-akuruvil
left a comment
quic-akuruvil
left a comment
There was a problem hiding this comment.
Resolve the conflicts on UI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rebasing ft_experimental branch against main