PyTorch 2.0+ support for Inf1

[weijihuang]

Describe the feature

From the list of supported container image, it seems that the support for PyTorch inference in inf1 has been dropped since PyTorch 1.x?

Can we build the image for higher version of PyTorch? Or at least, can we serve ourselves to upgrade some key packages, e.g. torch-neuron, for inf1?

Use Case

1. We do not need high end inference type, e.g. inf2, or trn.
2. We want the production environment is using updated key dependencies, so it is more aligned with development environment.
3. A lot of new models and Python packages rely on newer version of PyTorch. Without being supported, we cannot really deploy those models to inf1 any more.

Proposed Solution

AWS team can build a public image for later version of PyTorch on inf1.

Other Information

I see people mentioned in 2023 that the PyTorch 2.0 support is coming, in #743 . But I cannot see it anywhere.

Acknowledgements

• I may be able to implement this feature request

[shrihasr-aws]

Thank you for filing the issue @weijihuang , we will look into this.

[shrihasr-aws]

Thank you for your feedback. PyTorch 2.0+ support for Inf1 is not currently on our roadmap. If this changes, we will update this issue.

[weijihuang]

Thank you for your feedback. PyTorch 2.0+ support for Inf1 is not currently on our roadmap. If this changes, we will update this issue.

@shrihasr-aws Thanks for checking with the team. Is there any way where we can customize our own torch dependencies?

[shrihasr-aws]

Hi @weijihuang, the user cannot customize their own torch dependencies. Currently PyTorch support depends on a Neuron specific version of PyTorch v1.3.1.