[SPIKE] Ponytail or / and Caveman behind a feature flag

[edenreich]

Summary

This is a feature that allows to turn any LLM to be more cost effective, by enforcing them to output less tokens.

This should not be a default behaviour, but rather something the operator enables.

Ponytail: https://github.com/DietrichGebert/ponytail
Cavemen: https://github.com/juliusbrussee/caveman

Acceptance Criteria

• There are clear insights of the implementation road map after explorations
• Follow-ups github issues have been created

[edenreich]

To my understanding these are just skills...will probably close this spike because if it's just skills it's already exists and can be installed easily with:

infer skills install <skill url>