EdgeFlow: High-Performance Browser-Based Workflow Engine

📅 Project Plan Checklist

💡 Key Optimizations (Refactored)

1. Persistence: Added zustand/persist to save work across reloads.
2. Network: Added "Proxy Mode" to bypass CORS restrictions during demos.
3. Sandbox: Replaced with syntax with new Function(...args) to support React Strict Mode.

---

Day 1: Canvas Core —— React Flow & State Persistence

Goal: Build project skeleton, implement canvas drag & drop, and ensure data survives page reloads.

• 1.1 Environment Setup (1h)

  • Initialize Vite + React + TypeScript.
  • Install dependencies: @xyflow/react, zustand, immer, lucide-react.
  • Configure Tailwind CSS.

• 1.2 State Management & Persistence (2h)

  • Principles: Understand Zustand set/get.
  • Implementation: Create store.ts. Define nodes, edges, onNodesChange, onConnect.
  • ⚠️ [NEW] Persistence Integration:
    • Import persist from zustand/middleware.
    • Wrap store config to save to LocalStorage (key: edgeflow-storage).
    • Test: Refresh page -> Canvas state should remain.

• 1.3 Custom Node UI (3h)

  • Create AgentNode.tsx with "Left-In, Right-Out" Handle layout.
  • Style: Icon, Title, Status Indicator (Loading/Success/Error).
  • Register in nodeTypes.

---

Day 2: Metadata Driven —— Dynamic Form System

Goal: "Configuration is UI". Click node -> Open Drawer -> Generate Form based on JSON Schema.

• 2.1 Sidebar Drawer Logic (2h)

  • Import shadcn/ui Sheet component.
  • Logic: Click node -> Set selectedNodeId -> Open Sheet.

• 2.2 JSON Schema Registry (2h)

  • Create nodeRegistry.ts.
  • Define "HTTP Request" Node (inputs: URL, Method, Body, UseProxy).
  • Define "Code" Node (inputs: Javascript Code).
  • [NEW] Define "AI Agent" Node (inputs: Model, Prompt, API Key).

• 2.3 Dynamic Form Rendering (4h)

  • Use react-hook-form.
  • Create ParameterRender component.
  • Sync form changes back to Store Node Data (updateNodeData).

---

Day 3: Initial Run —— Logic & Network Layer

Goal: Execute logic in Main Thread first, handling real network requests.

• 3.1 Topological Sort (3h)

  • Implement getExecutionSequence(nodes, edges) using BFS/Kahn's Algorithm.

• 3.2 Execution Engine with Proxy Strategy (3h)

  • Implement runWorkflow function.
  • ⚠️ [NEW] HTTP Node Implementation:
    • Add logic: Check node.data.useProxy.
    • If true: Prepend https://cors-anywhere.herokuapp.com/ to target URL.
    • If false: Fetch directly (Direct Mode).
  • Store results in executionResults.

• 3.3 Visual Feedback (2h)

  • Update Node UI based on status (Yellow -> Green/Red).

---

Phase 2: Architecture Refactoring (Day 4 - Day 7)

Goal: Transform into a professional, non-blocking engine.

Day 4: Layered Architecture —— Web Worker & Comlink

Goal: Offload execution to Worker to keep UI at 60FPS.

• 4.1 Web Worker Setup (2h)

  • Create workflow.worker.ts.
  • Test basic Ping/Pong message.

• 4.2 Comlink Integration (2h)

  • Install comlink.
  • Expose Worker functions as RPC Promise objects.

• 4.3 Engine Migration (4h)

  • Move runWorkflow and Topo Sort logic into Worker.
  • Main Thread: Send Graph JSON -> Await Result.

---

Day 5: Intelligent Core —— AST Dependency Analysis

Goal: Support {{ $node["A"].data }} via static analysis.

• 5.1 AST Setup (2h)

  • Install acorn.
  • Test parsing simple JS expressions in console.

• 5.2 Dependency Extraction (3h)

  • Implement extractDependencies(code) using Acorn.
  • Walk the AST tree to find MemberExpression (e.g., $node.Http).

• 5.3 Data Injection Logic (3h)

  • In Worker: Before running a node, parse its code.
  • Find dependencies -> Fetch data from executionResults -> Inject into context.

---

Day 6: Secure Runtime —— Strict Mode Compatible Sandbox

Goal: Execute user code safely without using with (to support React Strict Mode).

• 6.1 Scope Preparation (2h)

  • Define the context object: const context = { $node: ..., utils: ..., console: ... }.
  • Extract keys and values: Object.keys(context), Object.values(context).

• 6.2 Safe Execution Implementation (4h)

  • ⚠️ [NEW] Replace with logic:

    // Create function with explicit arguments
    const safeFn = new Function(...scopeKeys, `return ${userCode}`);
    
    // Execute with actual values
    try {
      const result = safeFn(...scopeValues);
    } catch (e) {
      throw new Error(`Execution failed: ${e.message}`);
    }

  • Verify: Ensure variables defined in context are accessible in user code.

• 6.3 Security Hardening (Optional)

  • Shadow dangerous globals in the Worker scope (e.g., self.fetch = null if needed).

---

Day 7: Final Polish & Portfolio Ready

Goal: High-end UX and Interview Prep.

• 7.1 Monaco Editor Integration (3h)

  • Replace standard Textarea with Monaco Editor for Code Nodes.
  • (Bonus) Add syntax highlighting.

• 7.2 Data Inspector (2h)

  • Hover/Click a node after run to see Input/Output JSON in a nice viewer.

• 7.3 Resume & Demo Assets (3h)

  • Record GIF: Drag Node -> Config (Proxy On) -> Connect -> Run -> Result.
  • Update README with "Off-Main-Thread", "AST", "Strict Mode Sandbox".

---

Day 8: AI Agent Node - Core Implementation

Goal: Implement fully functional AI Agent node with LLM integration, secure storage, and multi-turn conversations.

• 8.1 Type System Extension (30min)

  • Extend ParameterType with 'ai-model' | 'ai-template' | 'skill-source'
  • Define AIAgentNodeData interface:
    • Basic config: model, provider, apiKey, temperature, maxTokens
    • Prompt config: systemMessage, prompt
    • Advanced toggles: enableStream, enableFunctionCalling, enableMultiTurn
    • SKILL config: skillSources array
    • Conversation config: conversationId, maxHistoryLength
  • Define LLMMessage, LLMResponse, FunctionCall, SkillSource interfaces
  • Update WorkflowNodeData union type

• 8.2 Node Registry Configuration (30min)

  • Configure complete ai-agent parameters in nodeRegistry.ts:
    • Node name, provider selector, model selector
    • API Key input (with encryption hint)
    • System Message (code type)
    • User Prompt (code type, support {{ $node["XXX"].data }} syntax)
    • Temperature & Max Tokens parameters
    • Advanced feature toggles
    • Max history length setting
  • Configure multiple handles (main-input/output, context-input, tool-output)

• 8.3 Encryption Service (1h)

  • Create src/workers/services/crypto.service.ts
  • Implement CryptoService class with Web Crypto API:
    • encrypt(plaintext, password?): AES-GCM 256-bit, PBKDF2 key derivation
    • decrypt(ciphertext, password?): Decrypt and return plaintext
    • getDefaultKey(): localStorage-based key persistence
  • Error handling: return null on decryption failure

• 8.4 LLM Service Implementation (2h)

  • Create src/workers/services/llm.service.ts
  • Define LLMChatParams and LLMStreamParams interfaces
  • Implement chat(params): Route to OpenAI or Anthropic
  • Implement chatOpenAI(params):
    • Build request body (model, messages, temperature, max_tokens)
    • Fetch https://api.openai.com/v1/chat/completions
    • Parse response and return LLMResponse
  • Implement chatAnthropic(params):
    • Extract system message
    • Fetch https://api.anthropic.com/v1/messages
    • Parse Claude format response
  • Error handling: unified error format with status code

• 8.5 Worker Execution Engine Integration (1.5h)

  • Import services: LLMService, CryptoService, ConversationService
  • Instantiate services in WorkflowEngine class
  • Implement private method executeAIAgent(...):
    • Decrypt API Key
    • Call injectVariables to inject data into Prompt
    • Build messages array (system + history + user)
    • Call llmService.chat
    • Save to conversation history if multi-turn enabled
    • Return LLMResponse
  • Implement injectVariables(template, contextNodeData):
    • Regex replace {{ $node["XXX"].data }}
    • Inject data as JSON string
  • Add ai-agent node handling in runWorkflow method

• 8.6 Conversation Storage Service (30min)

  • Create src/workers/services/conversation.service.ts
  • Define Conversation interface (id, createdAt, updatedAt, messages)
  • Implement ConversationService class (memory + localStorage):
    • createConversation(): Generate UUID, init empty messages
    • getConversation(id): Retrieve from Map
    • addMessage(conversationId, message): Add and update timestamp
    • getRecentMessages(conversationId, maxCount): Get last N messages
    • deleteConversation(id): Remove conversation
    • loadFromStorage() / saveToStorage(): Persistence

---

Day 9: AI Agent Node - Advanced Features

Goal: Add streaming output, function calling, SKILL plugin system, and upgrade to IndexedDB.

• 9.1 Streaming Output (1.5h)

  • Implement streamChat(params) in llm.service.ts
  • Implement streamOpenAI(params):
    • Add stream: true to request body
    • Use response.body?.getReader() to read stream
    • Parse SSE format (data: {...})
    • Accumulate content and call onChunk(delta) callback
    • Return complete LLMResponse
  • Implement streamAnthropic(params): Parse Claude SSE format
  • Error handling: fallback to non-streaming on failure

• 9.2 Worker Callback Extension (30min)

  • Extend WorkerCallbacks interface:
    • Add onStreamChunk?(nodeId, chunk) optional callback
  • Update executeAIAgent:
    • Pass onChunk callback in streaming mode
    • Call callbacks.onStreamChunk?.(nodeId, chunk)

• 9.3 IndexedDB Storage Upgrade (1h)

  • Install dependency: npm install idb
  • Create src/utils/indexeddb.ts
  • Define EdgeFlowDB schema:
    • conversations store (id index + updatedAt index)
    • apiKeys store (id index)
  • Implement IndexedDBService class:
    • init(): Initialize with openDB
    • saveConversation(conversation)
    • getConversation(id)
    • getAllConversations()
    • deleteConversation(id)
  • Replace localStorage with IndexedDB in ConversationService

• 9.4 Function Calling Implementation (1.5h)

  • Create src/workers/services/function-calling.service.ts
  • Define FunctionDefinition and FunctionExecutionResult interfaces
  • Implement FunctionCallingService class:
    • registerFunction(name, fn, definition): Register tool function
    • getFunctionDefinitions(): Get all definitions for LLM
    • executeFunctionCall(name, arguments_): Execute and return result
    • handleLLMFunctionCall(functionCall): Process LLM tool call
  • Integrate in LLMService:
    • Add functions? optional parameter to chat()
    • Include tools field in request body
    • Detect finish_reason: 'function_call' or tool_calls
    • Return functionCall field in response
  • Update WorkflowEngine:
    • Detect functionCall in response
    • Call FunctionCallingService to execute
    • Append result to messages and call LLM again

• 9.5 SKILL Plugin System (1.5h)

  • Create src/workers/services/skill-loader.service.ts
  • Define Skill interface (name, version, description, execute)
  • Implement SkillLoaderService class:
    • loadFromGitHub(url): Fetch raw code and eval
    • loadFromNPM(packageName): Dynamic import via esm.sh CDN
    • evalSkill(code): Sandbox with new Function, validate interface
    • getSkill(name): Retrieve loaded skill
    • executeSkill(name, input): Execute skill
  • Integrate in WorkflowEngine:
    • Preload configured SKILLS on startup
    • Register SKILLS as Function Calling tools
    • Unified execution entry point

• 9.6 Prompt Template Management (30min)

  • Create src/utils/prompt-templates.ts
  • Define PromptTemplate interface
  • Define PROMPT_TEMPLATES constant array:
    • Text Summarization
    • Code Review
    • Translation
    • JSON Transformation
    • Blog Generation
  • Implement PromptTemplateService class:
    • getAllTemplates()
    • getTemplate(id)
    • addTemplate(template)
    • removeTemplate(id)
  • Add template selector in nodeRegistry.ts

• 9.7 UI Component Updates (Optional)

  • Extend ParameterRender component:
    • ai-template type: Dropdown for template selection
    • skill-source type: SKILL source configuration form
  • Add template auto-fill logic
  • Add streaming output UI display

---

🚀 Git Workflow

# Initialize
git init
git add .
git commit -m "Initial: Project scaffold with detailed plan"

# Create Repo on GitHub called 'edgeflow'
git remote add origin https://github.com/YOUR_USER/edgeflow.git
git push -u origin main