Develop#2
Open
oscarcitoz wants to merge 359 commits into
Open
Conversation
add logic fallback antrhopic
add history in memory
add logic direct scrapper
add scrapper dropi
add new services dropi
add logic for create context and brand for store
comment variants.
new logic for alibaba and price
New response schema for product-modeling style:
- modeling_scene_brief: STATIC composition for Gemini Image
- kling_animation_prompt: what Kling animates (3 emotional beats)
- modeling_arc: array of 3 {timing, action, emotion} beats
- viral_hook_first_3_seconds: visual hook
No script_part_a/b (silent video), no cinematic_*, no ugc_*.
Made script_part_a, ends_with_product_name, viral_hook Optional
so modeling payloads parse without requiring script fields.
3 new validators:
- modeling_scene_brief_min_chars:150
- kling_animation_prompt_min_chars:100
- modeling_arc_has_3_beats
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…chema feat(video-studio): product-modeling schema + validators
…nfig
SYSTEM_PROMPT and CTA_DETECTION_INSTRUCTION used to be Python constants in
section_image_service.py — every iteration required a full CI + build +
rolling deploy and risked aborting in-flight Gemini generations at SIGTERM.
This change moves them to the existing agent-config registry (Fluxi's
`agent_configs` table in the `agents` DB, already hitting `get_agent()`
from conversation-engine), so they can be edited at runtime via the
agent-config UI or a simple SQL UPDATE, and propagate to all CE pods
within 60 seconds with zero deploy.
Two new agent_configs rows are seeded by scripts/seed_ai_prompts.sql:
- section_image_system
- section_image_cta_detection
They live next to every other prompt the platform already manages through
agent-config (benefits_section, custom_section_image, objections_image,
etc.), and automatically get free versioning via the existing
prompt_versions table.
Changes:
- app/services/prompt_config_service.py (new):
Class-level cache (60s TTL) on top of the existing get_agent() client.
Fallbacks are registered at import time by each consumer and used on
any fetch failure (network, missing row, empty prompt) so the service
never breaks because agent-config is down.
asyncio.Lock guards against cache stampede on concurrent misses.
- app/services/section_image_service.py:
- SYSTEM_PROMPT -> FALLBACK_SYSTEM_PROMPT. Also drops 3 rules the team
found too restrictive for Gemini: the "photo cutout" wording, the
"NEVER redraw" line, and the "NOT a social media ad" line.
- CTA_DETECTION_INSTRUCTION -> FALLBACK_CTA_DETECTION (leading newlines
removed; the parts join now adds them between blocks).
- EDIT_SYSTEM_PROMPT stays hardcoded (deliberate — other work in
progress depends on its current wording).
- _build_prompt is now async and pulls both from PromptConfigService.
- tests/unit/services/test_prompt_config_service.py (new): fallback path,
agent-config hit, caching, invalidate (one / all), RuntimeError when
nothing is available, plus the _fetch path (happy, exception, empty
prompt, missing attribute).
- tests/unit/services/test_section_image_service.py: _build_prompt tests
converted to async, added an edit_mode case and a "uses agent-config
value when available" case. All 38 tests in this file pass.
Zero new env vars — HOST_AGENT_CONFIG is already set on both environments.
Zero new tables, endpoints, or migrations in ecommerce-service — the
agent-config service already exposes everything needed for read/edit.
Deploy notes:
1. Deploy CE first (safe thanks to fallback even if the seed isn't
applied yet).
2. Run scripts/seed_ai_prompts.sql on the `agents` DB in both dev and
prod. The SQL is idempotent (ON CONFLICT DO NOTHING).
3. From then on, editing the prompt is `UPDATE agent_configs SET prompt
= '...' WHERE agent_id = 'section_image_system';` — takes effect on
all pods within 60 s.
feat(section-image): load SYSTEM_PROMPT + CTA_DETECTION from agent-config
…nfig
SYSTEM_PROMPT and CTA_DETECTION_INSTRUCTION used to be Python constants in
section_image_service.py — every iteration required a full CI + build +
rolling deploy and risked aborting in-flight Gemini generations at SIGTERM.
This change moves them to the existing agent-config registry (Fluxi's
`agent_configs` table in the `agents` DB, already hitting `get_agent()`
from conversation-engine), so they can be edited at runtime via the
agent-config UI or a simple SQL UPDATE, and propagate to all CE pods
within 60 seconds with zero deploy.
Two new agent_configs rows are seeded by scripts/seed_ai_prompts.sql:
- section_image_system
- section_image_cta_detection
They live next to every other prompt the platform already manages through
agent-config (benefits_section, custom_section_image, objections_image,
etc.), and automatically get free versioning via the existing
prompt_versions table.
Changes:
- app/services/prompt_config_service.py (new):
Class-level cache (60s TTL) on top of the existing get_agent() client.
Fallbacks are registered at import time by each consumer and used on
any fetch failure (network, missing row, empty prompt) so the service
never breaks because agent-config is down.
asyncio.Lock guards against cache stampede on concurrent misses.
- app/services/section_image_service.py:
- SYSTEM_PROMPT -> FALLBACK_SYSTEM_PROMPT. Also drops 3 rules the team
found too restrictive for Gemini: the "photo cutout" wording, the
"NEVER redraw" line, and the "NOT a social media ad" line.
- CTA_DETECTION_INSTRUCTION -> FALLBACK_CTA_DETECTION (leading newlines
removed; the parts join now adds them between blocks).
- EDIT_SYSTEM_PROMPT stays hardcoded (deliberate — other work in
progress depends on its current wording).
- _build_prompt is now async and pulls both from PromptConfigService.
- tests/unit/services/test_prompt_config_service.py (new): fallback path,
agent-config hit, caching, invalidate (one / all), RuntimeError when
nothing is available, plus the _fetch path (happy, exception, empty
prompt, missing attribute).
- tests/unit/services/test_section_image_service.py: _build_prompt tests
converted to async, added an edit_mode case and a "uses agent-config
value when available" case. All 38 tests in this file pass.
Zero new env vars — HOST_AGENT_CONFIG is already set on both environments.
Zero new tables, endpoints, or migrations in ecommerce-service — the
agent-config service already exposes everything needed for read/edit.
Deploy notes:
1. Deploy CE first (safe thanks to fallback even if the seed isn't
applied yet).
2. Run scripts/seed_ai_prompts.sql on the `agents` DB in both dev and
prod. The SQL is idempotent (ON CONFLICT DO NOTHING).
3. From then on, editing the prompt is `UPDATE agent_configs SET prompt
= '...' WHERE agent_id = 'section_image_system';` — takes effect on
all pods within 60 s.
…prod hotfix(section-image): load SYSTEM_PROMPT + CTA_DETECTION from agent-config (cherry-pick #155)
Adds the HTML-section pipeline end-to-end: generate, edit via chat, and orchestrate contextual images. Follows the same dynamic-config approach introduced for section-image prompts in #155 so every HTML prompt is editable at runtime in agent-config without a deploy. NEW — HTML sections (Universal Builder) * `app/services/section_html_service.py` - `generate_section_html()` initial section from template + product - `edit_section_html()` chat-driven edits - `generate_template_html()` template studio - `orchestrate_image_prompts()` one image prompt per placehold.co - `_process_new_images_in_edit()` + `_sanitize_image_urls()` — runs the image pipeline on edits (AI-generated new placeholders → orchestrator + sub-image-generator → S3 URLs in final HTML) - `preview_prompt()` — read-only endpoint for the template studio * `app/services/sub_image_service.py` - SubImageService with parallel generation + OpenAI fallback - Exposes SUB_IMAGE_MODEL / FALLBACK / RETRIES constants * `app/prompts/section_html_prompts.py` - Four prompts: GENERATE_SYSTEM / EDIT_SYSTEM / IMAGE_ORCHESTRATOR / TEMPLATE_STUDIO - Registered as fallbacks in PromptConfigService with agent_ids section_html_generate_system, section_html_edit_system, section_html_image_orchestrator, section_html_template_studio → runtime-editable via agent-config, hardcoded values used if agent-config is unreachable * `app/requests/*` + `app/responses/*` — Pydantic DTOs * Audit logging extended to 100 000 chars so full HTML responses are captured in analytics.prompt_logs NEW — Gemini SDK v2 (streaming) * `app/externals/ai_direct/gemini_text_v2.py` - call_gemini_freeform_v2() using google-genai SDK + interactions API + SSE streaming. Removes the ~60s server-side disconnect we were hitting with raw generateContent when thinking tokens inflated the response time. Used by edit_section_html. * `app/externals/ai_direct/gemini_text.py` - Adds thinking_level support + bumps aiohttp timeout 180→600s - generate_section_html still uses v1 (phase 3 migrates it too) CHANGED — section_image preview also uses dynamic prompts * `app/services/section_image_service.py::preview_image_prompt` now resolves the system prompt via `PromptConfigService.get()` (async) so the template studio preview reflects what's actually in the DB, not just the import-time fallback. Consistent with #155's intent. * `app/controllers/handle_controller.py` — matching `await` on the preview callers + endpoints for section-html flows. CORS for local dev * `main.py` adds CORSMiddleware only when ENVIRONMENT != "prod"; regex allows localhost:3000 / :3001 / :31xx / :5173 so the Next.js builder can hit :8000 directly and bypass the 30s proxy timeout on long AI responses. Seed SQL * `scripts/seed_ai_prompts.sql` now includes INSERT statements for the four HTML prompt agent_ids. REPLACE ME placeholders — copy the current FALLBACK_* values when seeding a real environment. Housekeeping * .gitignore: `venv39_backup/` (local Python 3.9 venv leftover) Scripts * `scripts/test-sdk-migration.py` — smoke test for gemini_text_v2 * `scripts/cleanup-broken-image-urls.py` — one-shot to regenerate images for legacy sections with unsplash/pexels/placehold.co URLs that were never resolved (runs the full image pipeline over stored HTML, writes back to website_sections) Deps: requires `google-genai>=1.73.1` (add to requirements.txt). Env vars in K8s: AUDIT_DB_HOST/PORT/USER/PASSWORD/NAME pointing at the analytics database that holds `prompt_logs`. See PR_PLAN.md + UNIVERSAL_BUILDER_STATUS.md in the workspace for the full cross-repo picture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eaming-images feat(section-html): SDK migration + image pipeline + dynamic prompts
Two minimal, surgical logging additions to SectionImageService._do_generate that close the biggest visibility gaps we've hit during incident investigations. Change 1 — persist Gemini raw text response on success Previously `log_prompt()` received no `response_text`, leaving the `prompt_logs.response_text` column NULL for all 11k+ section_image rows. Any debug of "why did this section turn out weird?" required reproducing the call end-to-end against Gemini (~60s + API cost) to see what the model actually replied. Now we capture the first 10KB of `text_response` right before `del`, and pass it through to `log_prompt`. Change 2 — log every failed Gemini attempt, not only the final outcome Previously the 5-retry loop only logged the final outcome (success, fallback or error). Transient failures like "Gemini no image in response. finishReason: STOP, text: BOTONES:" or "empty parts" lived in stdout only and rotated out of CloudWatch within days. With this change each retry emits its own row with `status=attempt_failed`, its own `attempt_number`, `elapsed_ms`, and the truncated `error_message`, so failure-mode analysis (how often, what pattern, when in bursts) becomes a single SQL query against `prompt_logs`. No existing behavior changes: success/fallback/error paths keep their exact prior semantics. The final error log after fallback-also-fails is preserved as-is. All prior fields of the success log are preserved. Tests 11 new unit tests in test_section_image_logging.py cover: - success path: response_text logged and truncated at 10KB - None/empty text_response is coerced to "" (no crash) - retry path: N attempt_failed rows + 1 final success - gemini-fail + openai-fallback-works: 5 + 1 rows - gemini-fail + openai-fail: 5 + 1 (original error raised) - error_message truncated at 1000 chars - backward-compat: every prior log field still present Full suite: 416 passed, 1 skipped, 0 regressions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(logs): persist AI raw response + log each failed retry attempt
Two minimal, surgical logging additions to SectionImageService._do_generate that close the biggest visibility gaps we've hit during incident investigations. Change 1 — persist Gemini raw text response on success Previously `log_prompt()` received no `response_text`, leaving the `prompt_logs.response_text` column NULL for all 11k+ section_image rows. Any debug of "why did this section turn out weird?" required reproducing the call end-to-end against Gemini (~60s + API cost) to see what the model actually replied. Now we capture the first 10KB of `text_response` right before `del`, and pass it through to `log_prompt`. Change 2 — log every failed Gemini attempt, not only the final outcome Previously the 5-retry loop only logged the final outcome (success, fallback or error). Transient failures like "Gemini no image in response. finishReason: STOP, text: BOTONES:" or "empty parts" lived in stdout only and rotated out of CloudWatch within days. With this change each retry emits its own row with `status=attempt_failed`, its own `attempt_number`, `elapsed_ms`, and the truncated `error_message`, so failure-mode analysis (how often, what pattern, when in bursts) becomes a single SQL query against `prompt_logs`. No existing behavior changes: success/fallback/error paths keep their exact prior semantics. The final error log after fallback-also-fails is preserved as-is. All prior fields of the success log are preserved. Tests 11 new unit tests in test_section_image_logging.py cover: - success path: response_text logged and truncated at 10KB - None/empty text_response is coerced to "" (no crash) - retry path: N attempt_failed rows + 1 final success - gemini-fail + openai-fallback-works: 5 + 1 rows - gemini-fail + openai-fail: 5 + 1 (original error raised) - error_message truncated at 1000 chars - backward-compat: every prior log field still present Full suite: 416 passed, 1 skipped, 0 regressions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…master feat(logs): persist AI raw response + log each failed retry attempt (PROD)
Adds POST /analyze-funnel endpoint that analyzes ad funnel metrics and returns a structured action plan. Ported from the n8n workflow "Identificar constraints y prioridades", using Gemini Flash with structured JSON output. - AnalyzeFunnelRequest/Response DTOs with ad context, raw metrics, rates - Semáforo (traffic-light) classifier with dropshipping_prospecting thresholds for hook_rate, thruplay_rate, ctr, cpc, roas, click_to_purchase - Cerebro Estratégico system prompt ported verbatim from n8n - Uses call_gemini_structured with gemini-flash-latest + responseSchema - Audit logging via log_prompt to prompt_logs - @require_api_key auth (server-to-server call from ecommerce-service) - 28 unit tests covering classifier thresholds and service orchestration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: add funnel analysis AI agent ("Cerebro Estratégico")
Drop the interactive checklist section from SYSTEM_PROMPT, RESPONSE_SCHEMA, and AnalyzeFunnelResponse. The action_plan list already covers the "what to do" surface; the separate checklist was redundant and drove unnecessary UI complexity (optimistic updates, rollback, PATCH endpoint). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…list refactor: remove today_checklist from funnel analysis agent
# Conflicts: # app/services/section_image_service.py # scripts/seed_ai_prompts.sql
Add dropi gt key
…hema-sync feat: add UGC voiceover director support
…ture-cleanup refactor: add voiceover service interfaces
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.