Tool to explore IEX DEEP .pcap.gz

Usage: iex <file.pcap.gz> [flags]

Flags:
  --scan-limit=N     Total messages to read from file
  --scan-offset=N    Messages to skip at beginning of file
  --match-limit=N    Filtered messages to output
  --match-offset=N   Filtered messages to skip before output
  --human-dates      Format timestamps as readable strings
  --format=csv|jsonl Output format (default: human-readable)
  --keep-type        Keep 'type' field even if only one --include is specified
  --exclude=T1,T2    Exclude specific message types (comma-separated)
  --include=Type(t1) Include specific message type and fields
  --include=*(t1)    Auto-include all types having the specified fields
      Selection rules:
        - field      Include field
        - _field     Exclude field (overrides *)
        - *          Include all remaining fields
        - field>val  Filter and include
        - _field=val Filter and exclude
        - fld:alias  Rename field in output
      Order is determined by first mention.

Available Message Types and Fields:
  - OfficialPrice(price_type, timestamp, symbol, price)
  - OperationalHaltStatus(halt_status, timestamp, symbol)
  - PriceLevelUpdate(event_flags, timestamp, symbol, side, price, size)
  - QuoteUpdate(flags, timestamp, symbol, bid_size, bid_price, ask_price, ask_size)
  - SecurityDirectory(flags, timestamp, symbol, round_lot_size, adjusted_price, luld_tier)
  - ShortSalePriceTestStatus(status, timestamp, symbol, detail)
  - SystemEvent(event_code, timestamp)
  - TradeReport(sale_condition_flags, timestamp, symbol, size, price, trade_id)
  - TradingStatus(status, timestamp, symbol, reason)

On my macbook ±16Gb .pcap.gz file is crunched into 281Mb of trades CSV in about 8 minutes, processing rate is ±35Mb/s in compressed gz, ±1.4M/s in IEXTP messages.

time julia --project -t auto -m iex pcaps/DEEP-2026-03-09.pcap.gz '--include=TradeReport(symbol,size,price,timestamp)' --format=csv --human-dates > trades-2026-03-09.csv

Scanned 676384817 messages. Matched 7773198 total. Printed 7773198.

1991,67s user 159,00s system 468% cpu 7:39,14 total

Examples:

# Full log for any given symbol:
julia --project -t auto -m iex pcaps/DEEP-2026-03-10.pcap.gz '--include=*(_symbol="AAPL", timestamp, size, price, bid_size:size, ask_size:size, bid_price:price, ask_price:price, adjusted_price:price, event_flags:flags, halt_status:status, sale_condition_flags:flags, *)' --human-dates --format=csv > AAPL.csv

# Trades
time julia --project -t auto -m iex pcaps/DEEP-2026-03-10.pcap.gz '--include=TradeReport(symbol,size,price,timestamp)' --format=csv --human-dates > trades-2026-03-10.csv

Made with gemini.