Automated Email Reply on Stock Center Order Labels

[ktun95]

- [x] fix `go-github` version mismatch across packages.
- [x] Define Label Email command.
- [x] Define struct for OrderData.
        - recipient email address
        - order id
- [ ] Fetch issue `github.event.issue.number`.
    - [x] Write function
    - [x] Create mock GET issue data
    - [x] Add route to routeTable
    - [ ] Reevaluate
        - Does this need testing?
    - [ ] Write test(s) for function
        - [ ] Implement route in mock github API server
- [x] Convert Github Issue body markdown to HTML
    - [x] Use `yuin/goldmark` package
- [x] Extract recipient order data from HTML
    - [x] Extract Order ID from second paragraph
    - [x] Extract Billing Address from table with "Billing Information"
    - [x] Write test(s)
- [x] Extract recipient order data from markdown (Regexp)
    - [x] Write function(s)
    - [x] Write test(s)
- [ ] Email CC?
- [ ] Prepare email body from template.
    - [x] Write functions
    - [ ] Write test(s)
    - [ ] Refactor template file path logic.
        - should not be hardcoded
    
- [x] Make API request to Mailgun endpoint.

Description

When a specific label is added to an issue, send an email to the address in listed in the email body.

Requirements

• An email is sent automatically when a label is added to an issue.
• Receiver email address and other information should be retrieved from the issue body.
• Email subject and body differ based on label added.
  • Growing / in Preparation
  • Shipping
  • invalid

Outline

1. Subscribe to github issue label event on Stock-Center-Orders
2. Extract receiver information from the issue body.
   • Email Address
   • Order ID
3. Generate Email subject on body from template using receiver information.
4. Send an email from some address; dictybase@northwestern.edu

sequenceDiagram
    actor Richard
    participant Stock-Center-Orders-repo
    participant gha-runner
    participant dictybase-docker/github-actions
    participant mailgun
    actor Order Recipient

Richard->>Stock-Center-Orders-repo: Adds label to issue
Stock-Center-Orders-repo->>gha-runner: Label event triggers workflow to run on
gha-runner->>dictybase-docker/github-actions: Checks out code
dictybase-docker/github-actions->>mailgun: Sends API request to
mailgun ->> Order Recipient: sends email to

Loading

Testing

There is a Stock-Center-Orders test repository that can be used for testing?

• Yes, orders issues should be generated in dictybase-playground/learn-github-action. It's not currently doing that but contains Order issues to play with.

Approaches

Github Webhooks

Pros:

• Direct, real-time notifications when label events occur
• Full control over the endpoint logic and email sending
• Can process the payload immediately and send emails synchronously
• No dependency on GitHub's infrastructure for execution (runs on your server)
• Can integrate with existing services like your event-messenger
• Payload contains all necessary information (issue body, labels, etc.)

Cons:

• Requires hosting and maintaining a web server/endpoint
• Need to handle webhook authentication and security
• Must manage uptime, scaling, and error handling yourself
• Infrastructure costs for running the service
• More complex deployment and monitoring setup
• Need to handle retries and failures manually

Github Actions

Pros:

• No server infrastructure to maintain (serverless)
• Runs directly in GitHub's infrastructure
• Easy to set up with YAML configuration
• Built-in secret management for email credentials
• Version controlled alongside your code
• Free tier available for public repositories
• Simple to test and debug with workflow runs
• Can use existing actions from the marketplace

Cons:

• Subject to GitHub Actions usage limits and quotas
• Less flexible than custom server code
• Debugging can be slower (need to trigger events or use workflow_dispatch)
• May have cold start delays
• Harder to integrate with existing services like event-messenger
• Limited to GitHub's execution environment

Unknowns

• How will the project be organized? Where should it be kept?

• Is using Github Webhooks viable?

  • Can subscribe to issue label events
  • It sends a payload to an endpoint of our choosing. Can it specify all the necessary information in that payload?

• What is our event-messenger service used for? Can it be used for the project?

• How does email even work?

  • SMTP?

[ktun95]

• event-messenger and github-actions should pull the issue template from a common source? Or there should be some versioning system, assuming the issue.tmpl is subject to change

[ktun95]

• try out different parsers, html, plaintext, markdown
• check template in event-messenger
• try the test domain for mailgun

[ktun95]

Work Report: feat/issue-label-email Branch

Date: 2026-01-22
Branch: feat/issue-label-email
Base Branch: develop
Total Commits: 22

Overview

This branch implements a new CLI command issue-label-email that extracts order information (Order ID and billing email) from GitHub issues. The feature is designed to automate processing of order-related GitHub issues by parsing structured markdown/HTML content and extracting key billing information.

Feature Purpose

The issue-label-email command is intended to:

1. Fetch a specific GitHub issue by number
2. Parse the issue body (which contains order information in markdown/HTML format)
3. Extract the Order ID and billing email address from the parsed content
4. Send an email to the recipient when certain labels are added to the issue (email functionality not yet implemented)

Implementation Summary

New Files Created (22 commits)

1. internal/html/parser.go - HTML parsing utilities

   • MarkdownToHTML() - Converts markdown to HTML using goldmark library
   • ParseTables() - Extracts all tables from HTML content
   • ExtractBillingEmail() - Finds billing email from table with "Billing Address" header
   • ExtractOrderID() - Finds Order ID from paragraph tags
   • Supporting functions for HTML node traversal and text extraction

2. internal/html/parser_test.go - Comprehensive tests for HTML parsing

   • TestParseTables() - Validates 4 tables are extracted correctly
   • TestExtractBillingEmail() - Verifies billing email extraction
   • TestExtractOrderID() - Verifies order ID extraction
   • All tests use testdata/issue.json for realistic data

3. internal/app/issue/extract.go - Data extraction logic

   • OrderData struct - Holds orderID and recipientEmail
   • IssueProcessor struct - Processes issue body to extract order data
   • extractFromTitle() - Extracts order ID and email from issue title
   • extractFromBody() - Extracts order ID and email from markdown body
   • extractOrderIDFromBody() - Regex-based order ID extraction from markdown
   • extractBillingEmailFromBody() - Extracts email from billing address table in markdown
   • Multiple regex-based helper functions

4. internal/app/issue/extract_test.go - Tests for extraction logic

   • TestExtractFromTitle() - Validates title parsing
   • TestExtractFromBody() - Validates markdown body parsing

5. internal/app/issue/issue_test.go - Integration tests

   • TestGetIssue() - Tests fetching issue from GitHub API using fake server

6. internal/app/issue/issue.tmpl - Go template for order reports

   • Markdown template for generating formatted order reports
   • Includes sections for: shipping/billing info, stocks ordered, strain information, strain storage, plasmid information

7. testdata/issue.json - Test fixture

   • Real-world example of a GitHub issue with order data
   • Contains both markdown body and HTML body_html fields
   • Used by all test files

Modified Files

1. cmd/github-actions/main.go

   • Added cmd.IssueLabelEmailCmds() to command list

2. internal/cmd/issue.go

   • Added IssueLabelEmailCmds() function
   • Defines CLI command with flags: label, issueid, domain, apiKey
   • Maps to issue.IssueLabelEmail() action

3. internal/app/issue/issue.go

   • Added IssueLabelEmail() - Main command handler
   • Added getIssue() - Fetches issue from GitHub API
   • Added getIssueBody() - Extracts issue body
   • Added getIssueBodyHTML() - Fetches HTML version of issue body (currently unused)
   • Flow: Fetch issue → Get markdown body → Convert to HTML → Extract order data → Log results

4. internal/fake/http.go

   • Added route for /repos/{owner}/{repo}/issues/{number} endpoint
   • Returns testdata/issue.json for testing

5. go.mod / go.sum

   • Added dependency: github.com/yuin/goldmark v1.7.16 for markdown to HTML conversion

Uncommitted Changes

1. .golangci.yml - Significant refactoring of linter configuration

   • Updated to version 2 format
   • Changed from disable-all + enable to default: none + enable
   • Reorganized linters into logical groups
   • Added settings, exclusions, and formatters sections
   • Moved formatters (gofmt, goimports) to separate section
   • NOTE: This should be reviewed and committed separately as it's infrastructure config

2. README.md - Added comprehensive project structure documentation

   • Documented all directories in /internal/app/ and /internal/
   • Added descriptions of what each module does
   • Listed key features of the application
   • NOTE: This is a significant improvement to documentation

Untracked Files (Should NOT be committed)

1. .golangci.bck.yml - Backup of old golangci config (created Jan 15)
2. api-token.txt - Contains API token (NEVER commit this!)

Architecture & Design Decisions

Two-Path Approach for Data Extraction

The implementation provides two methods for extracting order data:

1. Markdown-based extraction (extract.go)

   • Uses regex to parse markdown text directly
   • Functions: extractFromBody(), extractOrderIDFromBody(), extractBillingEmailFromBody()
   • Parses markdown table structure with pipe delimiters
   • More fragile but doesn't require HTML conversion

2. HTML-based extraction (html/parser.go)

   • Converts markdown to HTML using goldmark
   • Parses HTML DOM structure
   • Functions: MarkdownToHTML(), ParseTables(), ExtractOrderID(), ExtractBillingEmail()
   • More robust and maintainable
   • Currently used in IssueLabelEmail() command

Current Implementation Flow

```
IssueLabelEmail() command
↓
getIssue() - Fetch issue via GitHub API
↓
getIssueBody() - Extract markdown body
↓
MarkdownToHTML() - Convert markdown → HTML
↓
ExtractOrderID() + ExtractBillingEmail() - Parse HTML
↓
Create OrderData struct
↓
Log results (email sending not implemented yet)
```

Data Structure

The issue body contains order information in this format:

```markdown
Order ID: 37500885

Shipping address		Billing address
[shipping info]		[billing info]
		email@example.com

[Additional tables for strain/plasmid information]
```

Testing Status

Test Coverage

All core functionality is thoroughly tested:

• ✅ HTML parsing functions (parser_test.go)
• ✅ Markdown extraction functions (extract_test.go)
• ✅ Issue fetching with fake GitHub server (issue_test.go)
• ✅ All tests use realistic test data from testdata/issue.json
• ✅ Tests verify extraction of specific values: Order ID "37500885", Email "art@vandelayindustries.com"

Test Data

testdata/issue.json contains:

• Issue Pulling refactor/news into develop #193 from dictybase-playground/learn-github-action
• Title: "Order ID:37500885 art@vandelayindustries.com"
• Full markdown body with shipping/billing tables
• HTML-rendered body (body_html field)
• 4 tables: shipping/billing, stocks, strain info, strain storage

What's Working

1. ✅ Fetching GitHub issues by number
2. ✅ Converting markdown to HTML
3. ✅ Parsing HTML tables
4. ✅ Extracting Order ID from paragraphs
5. ✅ Extracting billing email from table columns
6. ✅ Regex-based extraction from markdown
7. ✅ Fake GitHub server for testing
8. ✅ CLI command registration
9. ✅ Comprehensive test coverage

What's NOT Implemented

1. ❌ Email sending functionality - The core purpose of the command

   • CLI flags exist (domain, apiKey) but are unused
   • No Mailgun integration code written
   • No email template rendering
   • No actual email dispatch logic

2. ❌ Label checking logic - Command should trigger on specific labels

   • --label flag exists but is not used in the code
   • No validation that correct label was added

3. ❌ Error handling for email operations

4. ❌ Integration tests for complete end-to-end flow

Technical Debt & Notes

Code Organization

1. Dual extraction methods - Both markdown-based (extract.go) and HTML-based (parser.go) exist

   • The HTML-based approach is currently used
   • The markdown-based approach in extract.go is mostly unused
   • Consider: Remove regex-based markdown extraction or document why both exist

2. Unused functions

   • getIssueBodyHTML() in issue.go (lines 274-314) - Fetches HTML via GitHub API but unused
   • IssueProcessor struct in extract.go - Defined but not used in main flow
   • extractFromTitle() - Not used in current implementation

3. Template file (issue.tmpl)

   • Created but never used in code
   • Appears to be for generating order report markdown
   • Purpose unclear - may be for future email content or issue comments

Dependencies

• github.com/yuin/goldmark v1.7.16 - Markdown parser (upgraded in commit 7aa969c)
• github.com/google/go-github/v62 - GitHub API client
• golang.org/x/net/html - HTML parsing

Potential Issues

1. Hardcoded assumptions about table structure

   • Billing email extraction assumes specific table format
   • May break if issue template changes

2. No validation of extracted data

   • No email format validation
   • No order ID format validation

3. .golangci.yml changes

   • Major refactoring of linter config
   • Should be tested separately to ensure all linters still work
   • May cause CI pipeline changes

Next Steps (Recommended)

High Priority

1. Implement email sending functionality

   • Integrate Mailgun API client
   • Use domain and apiKey flags
   • Create email template (possibly using issue.tmpl)
   • Add email sending logic to IssueLabelEmail()

2. Implement label checking

   • Use the --label flag value
   • Define which labels should trigger emails
   • Add configuration for allowed labels

3. Handle the uncommitted changes

   • Review and commit .golangci.yml changes separately
   • Review and commit README.md documentation
   • Delete .golangci.bck.yml backup file
   • Ensure api-token.txt is in .gitignore

Medium Priority

4. Code cleanup

   • Remove unused getIssueBodyHTML() or document why it's kept
   • Remove or document unused markdown extraction functions
   • Decide on template usage strategy

5. Add data validation

   • Validate email format
   • Validate order ID format
   • Add better error messages

6. Integration testing

   • Test complete flow with real GitHub issues
   • Test email sending (with test mode)
   • Add E2E test

Low Priority

7. Documentation

   • Update CLI docs (docs/README.md)
   • Add code comments explaining dual extraction approach
   • Document issue template requirements

8. Refactoring

   • Consider consolidating extraction approaches
   • Simplify error handling
   • Add structured logging

Command Usage

Current CLI signature (from internal/cmd/issue.go):

```bash
github-actions issue-label-email
--token=
--owner=
--repository=
--issue=
--label=
--domain=
--apiKey=
```

Note: The issueid flag is defined but command actually uses global --issue flag (inconsistency to fix)

Commit History Summary

The work was done incrementally over 22 commits:

1. Initial implementation (commits 43b7f96-48d0635) - Basic structure, HTML parsing, regex extraction
2. Template work (9081772) - Created order report template
3. Refactoring phase (93af551-15865e2) - Restructured code, renamed functions, improved organization
4. Testing improvements (574bce1-033535a) - Added comprehensive tests
5. Final implementation (79c11e2-60a23cc) - Completed IssueLabelEmail command, added MarkdownToHTML

Files Changed

Added (7 files, 892 lines)

• internal/html/parser.go (226 lines)
• internal/html/parser_test.go (82 lines)
• internal/app/issue/extract.go (150 lines)
• internal/app/issue/extract_test.go (54 lines)
• internal/app/issue/issue_test.go (42 lines)
• internal/app/issue/issue.tmpl (63 lines)
• testdata/issue.json (88 lines)

Modified (6 files)

• cmd/github-actions/main.go (+1 line)
• internal/cmd/issue.go (+27 lines)
• internal/app/issue/issue.go (+145 lines)
• internal/fake/http.go (+11 lines)
• go.mod (+1 dependency)
• go.sum (+2 lines)

Uncommitted (2 files)

• .golangci.yml (major refactoring)
• README.md (+130 lines of documentation)

Key References

• Issue parsing uses testdata: testdata/issue.json
• Main command handler: internal/app/issue/issue.go:203-272 (IssueLabelEmail function)
• HTML extraction: internal/html/parser.go:214-226 (MarkdownToHTML), lines 114-149 (ExtractBillingEmail), lines 164-196 (ExtractOrderID)
• Markdown extraction: internal/app/issue/extract.go:138-150 (extractFromBody)

Summary

The branch implements a solid foundation for extracting order information from GitHub issues. The HTML parsing and data extraction logic is complete and well-tested. The main gap is the email sending functionality, which is the core purpose of the command. Once the Mailgun integration is added and label checking is implemented, this feature will be ready for production use.

The uncommitted changes to .golangci.yml and README.md should be reviewed and committed as separate infrastructure/documentation updates.