Add OpenAI nano fallback for HF inference outages (silent UI)

[brej-29]

Summary:

• Introduce a silent OpenAI fallback (nano, gpt-5-nano) for Hugging Face Inference failures. HF remains the default path; fallback occurs only when HF request fails (timeout/5xx/network).
• OpenAI fallback uses the official Python SDK (openai) via the Responses API and does not send unsupported params like temperature or top_p for GPT-5 family models.
• UI remains clean: no HF stack traces shown; logs capture errors and a brief fallback status is shown when used.

What changed:

• app/streamlit_app.py

  • Added _get_openai_settings() to read OPENAI_API_KEY and OPENAI_FALLBACK_MODEL from st.secrets or env vars (defaulting to gpt-5-nano).
  • Added _call_openai_fallback(system_prompt, user_prompt, max_tokens) to call the OpenAI Responses API using the configured model, returning the generated text.
  • Enhanced _call_model to keep the Hugging Face path unchanged, but wrap the HF call in a try/except. On exception, log the error and attempt the OpenAI fallback. If fallback succeeds, return its result and display a small status caption; if it fails, show a user-friendly error without a stack trace.
  • Introduced a non-intrusive UI indicator (st.caption) when using the backup provider.

• requirements.txt

  • Added openai>=2.15.0 to ensure the OpenAI SDK is available in local, CI, and Streamlit Cloud environments.

• .streamlit/secrets.toml.example

  • Added placeholders for:
    OPENAI_API_KEY = ""
    OPENAI_FALLBACK_MODEL = "gpt-5-nano"
  • Kept existing HF secret keys intact.

• tests/test_streamlit_openai_fallback.py

  • Added a unit test that mocks the HF client to raise an error and mocks the OpenAI client to return a known output. Verifies that the fallback path is exercised and returns the expected SQL text, while confirming the UI caption is triggered without showing stack traces.

• tests are designed to run offline (no real network calls) by mocking the external clients.

Notes on behavior and usage:

• HF endpoint remains the default behavior. If HF works, there is no UI indication of fallback.
• If HF inference fails (e.g., timeout, 5xx, or network error), the app will log the error, switch to the OpenAI nano fallback (gpt-5-nano), and return the OpenAI-generated output.
• The OpenAI fallback uses the OpenAI Responses API and does not pass temperature/top_p parameters for GPT-5 family models.
• In case of missing OPENAI_API_KEY or misconfiguration, the fallback will raise a runtime error; the UI will show a generic error message without exposing stack traces.

How this meets the task:

• Adds silent OpenAI fallback for HF failures and preserves existing Streamlit behavior when HF works.
• Minimal changes focused on isolated components to avoid large refactors.
• New branch/PR should be created off main with the title reflected above.

---

This pull request was co-created with Cosine Genie

Original Task: analytics-copilot-text2sql/8bx49slql6sg
Author: Brejesh Balakrishnan