AI‑Powered Receipt Indexing & Budget Tracking with Azure Logic Apps and AI Agents

Overview

Beyond archiving receipts, the goal was to:

• Search historical purchases using natural language
• Track spend against a budget
• Use AI agent to simolify the workflow

This repository contains the resulting solution built with Azure Logic Apps, Azure AI Search, and Microsoft Foundry agents.

---

Architecture Diagram (ASCII)

────────────────────────────────────────┐ │ 📧 Incoming Purchase Confirmation │ │ (Office 365 / Outlook Inbox) │ └──────────────────────┬─────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 🔔 Logic App Trigger │ │ When New Email Arrives (V3) │ └──────────────────────┬─────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 📄 Email Retrieval + HTML Extraction │ │ Get Email (V2) → Extract HTML Body │ └──────────────────────┬─────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 🧠 Inline AI Agent (Logic Apps) │ │ - Instruction-driven parsing │ │ - Vendor-agnostic receipt extraction │ │ - No model or endpoint plumbing │ └──────────────────────┬─────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ ✅ Schema Enforcement │ │ Parse JSON (Strict Receipt Schema) │ └───────────────┬──────────────┬────────┘ │ │ │ │ ▼ ▼ ┌──────────────────────┐ ┌──────────────────────────┐ │ 🗄️ Azure Blob │ │ 🔎 Azure AI Search │ │ Receipt JSON Archive │ Semantically Indexed │ │ │ │ Purchase Documents │ └──────────────────────┘ └──────────────┬───────────┘ │ ▼ ┌──────────────────────────────────────────┐ │ 🤖 Microsoft Foundry Agent │ │ - Budget comparison & reasoning │ │ - Computes spend, thresholds, deltas │ │ - Generates grounded explanation │ └──────────────────────┬───────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ ✉️ Budget Notification Email │ │ Outlook Connector (HTML Summary) │ │ "Within Budget" or "Over Budget" │ └────────────────────────────────────────┘

Recommeded way to create the index curl -X PUT
"https://.search.windows.net/indexes/receipts-index?api-version=2025-09-01"
-H "Content-Type: application/json"
-H "api-key: "
-d '{ "name": "receipts-index", "fields": [ { "name": "id", "type": "Edm.String", "key": true, "searchable": false, "filterable": true, "sortable": true, "facetable": false }, { "name": "vendor", "type": "Edm.String", "searchable": true, "filterable": true, "sortable": false, "facetable": true }, { "name": "orderNumber", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": true, "facetable": false }, { "name": "purchaseDate", "type": "Edm.DateTimeOffset", "searchable": false, "filterable": true, "sortable": true, "facetable": false }, { "name": "month", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": true }, { "name": "year", "type": "Edm.Int32", "searchable": false, "filterable": true, "sortable": true, "facetable": true }, { "name": "messageType", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": true }, { "name": "project", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": true }, { "name": "total", "type": "Edm.Double", "searchable": false, "filterable": true, "sortable": true, "facetable": true }, { "name": "tax", "type": "Edm.Double", "searchable": false, "filterable": true, "sortable": true, "facetable": false }, { "name": "discount", "type": "Edm.Double", "searchable": false, "filterable": true, "sortable": true, "facetable": false }, { "name": "currency", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": true }, { "name": "category", "type": "Edm.String", "searchable": true, "filterable": true, "sortable": false, "facetable": true }, { "name": "itemDescription", "type": "Edm.String", "searchable": true, "filterable": false, "sortable": false, "facetable": false }, { "name": "jsonBlobPath", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": false }, { "name": "status", "type": "Edm.String", "searchable": false, "filterable": true, "sortable": false, "facetable": true } ] }'