Add int8 static quantization workflow

[jcaip]

This PR adds in a new static quant workflow based off of Int8Tensor (#3407).

It introduces a new config, Int8StaticActivationInt8WeightConfig which requires a scale tensor and granularity

static_config = Int8StaticActivationInt8WeightConfig(
    scale=int8_input.scale.detach(), granularity=PerRow
)
quantize_(model_static_quant, static_config)

Currently PerRow and PerTensor symmetric quant is support only.

This scale tensor is stored on the weight Int8Tensor under activation_scale, and is used to create a new activation Int8Tensor for static quantization.

It would be nice to store this scale tensor in QuantizeTensorToInt8Kwargs but unfortunately this breaks dynamo tracing, as we store the quant kwargs as an object for the weight tensor and we are unable to fakeify them properly.

As a result, we need to keep track and pass scale outside of this Kwargs object.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3442

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 8ae921d with merge base c4273fe ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.