Measure compute needed in FLOPs

[jonas-becker]

Is your feature request related to a problem? Please describe.
Currently, we only log the compute times of MALLM.

Describe the solution you'd like
The proper way to compare MALLM and a single-LLM baseline in terms of computing would be by FLOPs.
Thus, we want to track the FLOPs and output them to the results similar to the compute times.

[lkaesberg]

import math

def estimate_flops(model_params, input_length, output_length, flops_per_param=3):
    """
    Estimate the number of FLOPs for a TGI model.
    
    :param model_params: Number of parameters in the model
    :param input_length: Length of the input sequence
    :param output_length: Length of the generated output sequence
    :param flops_per_param: Estimated FLOPs per parameter per token (default: 3)
    :return: Estimated total FLOPs
    """
    total_length = input_length + output_length
    flops_per_token = model_params * flops_per_param
    total_flops = flops_per_token * total_length
    
    return total_flops

def format_flops(flops):
    """
    Format FLOPs into a human-readable string.
    """
    if flops < 1e3:
        return f"{flops:.2f} FLOPs"
    elif flops < 1e6:
        return f"{flops/1e3:.2f} KFLOPs"
    elif flops < 1e9:
        return f"{flops/1e6:.2f} MFLOPs"
    elif flops < 1e12:
        return f"{flops/1e9:.2f} GFLOPs"
    else:
        return f"{flops/1e12:.2f} TFLOPs"

# Example usage
model_params = 7e9  # 7B parameter model (e.g., LLaMA-7B)
input_length = 100
output_length = 50

estimated_flops = estimate_flops(model_params, input_length, output_length)
print(f"Estimated compute cost: {format_flops(estimated_flops)}")

Something like this is possible to estimate it. Difficult to get the actual data for tgi and we need a way to get the model param count

[jpwahle]

I think this is great, but @flowun will start a project mid August to look specifically at that, then I will assign him to the issue :)