fix: backtest historical date range - paginated getCandlesByRange

[fray-cloud]

Summary

• Fixes No candle data available for the specified date range error when running backtests on historical dates
• Root cause: fetchHistoricalCandles() only fetched latest 200 candles then filtered, missing any data older than ~33 days (4h) or ~8 days (1h)
• Adds getCandlesByRange(symbol, interval, startTime, endTime) to IExchangeRest interface with full pagination in each adapter:
  • Binance: forward walk with startTime/endTime params, 1000 candles/page
  • Bybit: forward walk with start/end params, 1000 candles/page
  • Upbit: backward walk via to param, 200 candles/page
• backtests.service.ts now calls getCandlesByRange directly (no more manual pagination in service layer)

Closes #75

Test plan

• All 76 existing exchange-adapter unit tests pass (npm run test:unit)
• Backtest with a date range > 33 days ago on Binance (e.g., 60-day window on 4h candles) returns candle data
• Backtest with recent date range still works (no regression)
• Redis cache key unchanged — cached results still served correctly

🤖 Generated with Claude Code

Summary by Sourcery

Implement paginated, date-range candle fetching across exchange adapters and wire it into backtesting and UI configuration.

New Features:

• Add getCandlesByRange(symbol, interval, startTime, endTime) to the exchange REST interface and implement it for Binance, Bybit, and Upbit with proper pagination over arbitrary historical ranges.
• Support enumerated parameter options in flow node definitions and render them as select dropdowns in the node inspector UI.

Bug Fixes:

• Fix missing historical candle data in backtests by fetching all candles within the requested date range instead of only the latest page.

Enhancements:

• Normalize symbols to uppercase before fetching historical and backtesting data to ensure consistent cache keys and DB lookups.
• Extend node parameter metadata to carry optional value lists, improving configurability and reducing invalid user input in flow nodes.

[sourcery-ai]

Reviewer's Guide

Implements exchange-specific, fully paginated historical candle fetching via a new getCandlesByRange API on all REST adapters, updates the backtest service to rely on this range-based pagination (preserving Redis cache keys), and enhances the flow-node inspector UI to support select-style param inputs using declarative options metadata.

Sequence diagram for backtest historical candle fetching with getCandlesByRange

sequenceDiagram
  actor Trader
  participant WebApp
  participant WorkerService_BacktestsService as BacktestsService
  participant Redis
  participant ExchangeAdapter as IExchangeRest_Adapter
  participant ExchangeApi as Exchange_HTTP_API

  Trader->>WebApp: Configure backtest (exchange, symbol, interval, date range)
  WebApp->>WorkerService_BacktestsService: Start backtest request

  WorkerService_BacktestsService->>Redis: GET backtest:candles:exchange:SYMBOL:interval:start:end
  alt Cache hit
    Redis-->>WorkerService_BacktestsService: Cached candles
    WorkerService_BacktestsService-->>WebApp: Use cached candles for backtest
  else Cache miss
    Redis-->>WorkerService_BacktestsService: null
    WorkerService_BacktestsService->>ExchangeAdapter: getCandlesByRange(symbol, interval, startTime, endTime)
    activate ExchangeAdapter
    loop Paginated candle fetch
      ExchangeAdapter->>ExchangeApi: GET /candles with pagination params
      ExchangeApi-->>ExchangeAdapter: Page of candles (newest first or oldest first)
      ExchangeAdapter->>ExchangeAdapter: Normalize order and filter by startTime/endTime
      ExchangeAdapter->>ExchangeAdapter: Append or prepend to allCandles
    end
    ExchangeAdapter-->>WorkerService_BacktestsService: allCandles
    deactivate ExchangeAdapter

    alt Candles found
      WorkerService_BacktestsService->>Redis: SET cacheKey allCandles EX CANDLE_CACHE_TTL
    else No candles
      WorkerService_BacktestsService->>Redis: (no write)
    end

    WorkerService_BacktestsService-->>WebApp: Candles for backtest
  end

  WebApp-->>Trader: Show backtest results

Loading

Sequence diagram for NodeInspector param editing with options-based select inputs

sequenceDiagram
  actor User
  participant WebApp
  participant FlowEditor
  participant NodeInspector
  participant NODE_TYPE_REGISTRY

  User->>FlowEditor: Select node in strategy flow
  FlowEditor->>NODE_TYPE_REGISTRY: Lookup NodeTypeInfo for node.type
  NODE_TYPE_REGISTRY-->>FlowEditor: NodeTypeInfo (params with options metadata)
  FlowEditor->>NodeInspector: Render inspector with node and NodeTypeInfo

  NodeInspector->>NodeInspector: Split params into requiredParams and optionalParams
  loop For each ParamDefinition
    NodeInspector->>NodeInspector: Determine options for param.key
    alt ParamDefinition.options is defined and non-empty
      NodeInspector->>NodeInspector: Render ParamInput as <select> with options
    else No options
      NodeInspector->>NodeInspector: Render ParamInput as boolean toggle, number, or text input
    end
  end

  User->>NodeInspector: Change param via select or input
  NodeInspector->>FlowEditor: updateNodeConfig(nodeId, updatedConfig)
  FlowEditor-->>User: Updated node configuration reflected in UI

Loading

Class diagram for exchange REST adapters and new getCandlesByRange API

classDiagram
  class Candle {
    +string exchange
    +string symbol
    +string interval
    +string open
    +string high
    +string low
    +string close
    +string volume
    +number timestamp
  }

  class OrderResult {
    +string exchange
    +string symbol
    +string status
    +string side
    +string type
    +string price
    +string amount
    +string filled
  }

  class Market {
    +string symbol
    +string baseAsset
    +string quoteAsset
  }

  class IExchangeRest {
    <<interface>>
    +createOrder(symbol, side, type, amount, price) Promise~OrderResult~
    +getOrder(symbol, orderId) Promise~OrderResult~
    +cancelOrder(symbol, orderId) Promise~OrderResult~
    +getMarkets() Promise~Market[]~
    +getCandles(symbol, interval, limit) Promise~Candle[]~
    +getCandlesByRange(symbol, interval, startTime, endTime) Promise~Candle[]~
  }

  class BinanceRest {
    +string exchangeId
    +getCandles(symbol, interval, limit) Promise~Candle[]~
    +getCandlesByRange(symbol, interval, startTime, endTime) Promise~Candle[]~
    +createOrder(symbol, side, type, amount, price) Promise~OrderResult~
    +getOrder(symbol, orderId) Promise~OrderResult~
    +cancelOrder(symbol, orderId) Promise~OrderResult~
    +getMarkets() Promise~Market[]~
    -mapOrderResult(o)
  }

  class BybitRest {
    +string exchangeId
    +getCandles(symbol, interval, limit) Promise~Candle[]~
    +getCandlesByRange(symbol, interval, startTime, endTime) Promise~Candle[]~
    +createOrder(symbol, side, type, amount, price) Promise~OrderResult~
    +getOrder(symbol, orderId) Promise~OrderResult~
    +cancelOrder(symbol, orderId) Promise~OrderResult~
    +getMarkets() Promise~Market[]~
    -mapOrder(o)
  }

  class UpbitRest {
    +string exchangeId
    +getCandles(symbol, interval, limit) Promise~Candle[]~
    +getCandlesByRange(symbol, interval, startTime, endTime) Promise~Candle[]~
    +createOrder(symbol, side, type, amount, price) Promise~OrderResult~
    +getOrder(symbol, orderId) Promise~OrderResult~
    +cancelOrder(symbol, orderId) Promise~OrderResult~
    +getMarkets() Promise~Market[]~
    -mapOrderResponse(o)
  }

  class BacktestsService {
    -redis
    -fetchHistoricalCandles(exchange, symbol, interval, startDate, endDate) Promise~Candle[]~
  }

  class DataService {
    -loadFromDb(exchange, symbol, interval, startTime, endTime) Promise~OhlcvCandle[]~
    +getHistoricalCandles(exchange, symbol, interval, startTime, endTime) Promise~OhlcvCandle[]~
  }

  class OhlcvCandle {
    +string exchange
    +string symbol
    +string interval
    +string open
    +string high
    +string low
    +string close
    +string volume
    +number timestamp
  }

  IExchangeRest <|.. BinanceRest
  IExchangeRest <|.. BybitRest
  IExchangeRest <|.. UpbitRest

  BacktestsService --> IExchangeRest : uses getCandlesByRange
  DataService --> IExchangeRest : may use exchange adapters externally
  DataService --> OhlcvCandle
  BinanceRest --> Candle
  BybitRest --> Candle
  UpbitRest --> Candle

Loading

Class diagram for flow node param definitions and NodeInspector options support

classDiagram
  class ParamDefinition {
    +string key
    +boolean required
    +string[] options
  }

  class PortDefinition {
    +string name
    +string type
  }

  class NodeTypeInfo {
    +string subtype
    +string label
    +PortDefinition[] inputs
    +PortDefinition[] outputs
    +Record~string, unknown~ defaultConfig
    +ParamDefinition[] params
  }

  class NODE_TYPE_REGISTRY {
    <<map>>
    +Record~string, NodeTypeInfo~
  }

  class NodeConfig {
    +Record~string, unknown~ values
  }

  class Node {
    +string id
    +string type
    +NodeConfig config
  }

  class ParamInput {
    +paramKey string
    +value unknown
    +options string[]
    +onChange(val) void
  }

  class NodeInspector {
    -node Node
    -registry NodeTypeInfo
    -requiredParams ParamDefinition[]
    -optionalParams ParamDefinition[]
    +render()
  }

  NODE_TYPE_REGISTRY --> NodeTypeInfo : values
  NodeTypeInfo --> ParamDefinition : has
  NodeTypeInfo --> PortDefinition : has
  Node --> NodeConfig : has
  NodeInspector --> Node : inspects
  NodeInspector --> NodeTypeInfo : reads metadata
  NodeInspector --> ParamInput : renders
  ParamInput --> ParamDefinition : uses key and options

  class RsiNodeTypeInfo {
    +defaultConfig period, source
    +params period, source
  }

  class MacdConditionNodeTypeInfo {
    +defaultConfig operator, threshold
    +params operator, threshold
  }

  class CrossOverConditionNodeTypeInfo {
    +defaultConfig direction
    +params direction
  }

  class AndOrConditionNodeTypeInfo {
    +defaultConfig operator
    +params operator
  }

  class MarketOrderNodeTypeInfo {
    +defaultConfig side, amount
    +params side, amount
  }

  NODE_TYPE_REGISTRY --> RsiNodeTypeInfo
  NODE_TYPE_REGISTRY --> MacdConditionNodeTypeInfo
  NODE_TYPE_REGISTRY --> CrossOverConditionNodeTypeInfo
  NODE_TYPE_REGISTRY --> AndOrConditionNodeTypeInfo
  NODE_TYPE_REGISTRY --> MarketOrderNodeTypeInfo

  RsiNodeTypeInfo --> ParamDefinition : source.options [close, open, high, low]
  MacdConditionNodeTypeInfo --> ParamDefinition : operator.options [<, >, <=, >=, ==]
  CrossOverConditionNodeTypeInfo --> ParamDefinition : direction.options [above, below]
  AndOrConditionNodeTypeInfo --> ParamDefinition : operator.options [AND, OR]
  MarketOrderNodeTypeInfo --> ParamDefinition : side.options [buy, sell]

Loading

File-Level Changes

Change	Details	Files
Add exchange-specific paginated historical candle range fetching to all REST adapters via a new getCandlesByRange method.	Extend IExchangeRest with getCandlesByRange(symbol, interval, startTime, endTime) returning Candle[]. Implement Binance getCandlesByRange using /api/v3/klines with startTime/endTime, 1000-candle pages, forward pagination, and interval-based pageStart advancement. Implement Bybit getCandlesByRange using /v5/market/kline with category=spot, start/end window parameters, 1000-candle pages, forward pagination, and response validation on retCode. Implement Upbit getCandlesByRange using backward pagination via to ISO timestamp, 200-candle pages, range filtering, and chronological assembly. Introduce per-adapter parseIntervalMs helpers to compute interval duration in ms for page stepping across 1m–1d intervals.	packages/exchange-adapters/src/interfaces/exchange-rest.ts packages/exchange-adapters/src/binance/binance.rest.ts packages/exchange-adapters/src/bybit/bybit.rest.ts packages/exchange-adapters/src/upbit/upbit.rest.ts
Refactor backtest and historical data services to use the new range-based candle fetching while preserving caching behavior and symbol normalization.	Change BacktestsService.fetchHistoricalCandles to delegate to adapter.getCandlesByRange instead of manually paginating getCandles, while keeping the existing Redis cache key format and TTL. Normalize symbols to uppercase in BacktestsService.fetchHistoricalCandles before cache key construction and adapter calls. Normalize symbols to uppercase in DataService.getHistoricalCandles before DB lookup to keep storage and retrieval consistent with adapter expectations.	apps/worker-service/src/backtests/backtests.service.ts apps/worker-service/src/backtesting/data.service.ts
Enhance flow-node param configuration to support constrained options rendered as inputs driven by metadata.	Extend ParamDefinition with an optional options:string[] field and populate it for several node types (e.g., RSI source, comparison operators, crossover direction, logical operators, order side). Update NodeInspector to pass options through to ParamInput for both required and optional params, and to treat options-bearing params as select controls. Update ParamInput to render a styled when options are provided, falling back to the existing boolean/button/text input behavior otherwise, and to use PARAM_VALUE_LABELS for display labels.	packages/types/src/flow.ts apps/web/src/components/flows/node-inspector.tsx

Possibly linked issues

• #fix: 백테스트 과거 날짜 범위 지원 불가 (최근 200캔들만 조회): PR이 fetchHistoricalCandles를 기간 페이지네이션 방식으로 수정해 이슈의 과거 백테스트 캔들 조회 버그를 해결한다.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 2 issues, and left some high level feedback:

• The parseIntervalMs helper is duplicated in each exchange adapter; consider extracting this to a shared utility (or the common types layer) to keep the interval mapping consistent and avoid drift between adapters.
• Each getCandlesByRange implementation hard-codes MAX_PAGES = 100, which can silently truncate very long ranges; it may be safer to derive the max pages from the requested time span or surface an explicit error/Log when the page cap is hit while still within the requested range.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `parseIntervalMs` helper is duplicated in each exchange adapter; consider extracting this to a shared utility (or the common types layer) to keep the interval mapping consistent and avoid drift between adapters.
- Each `getCandlesByRange` implementation hard-codes `MAX_PAGES = 100`, which can silently truncate very long ranges; it may be safer to derive the max pages from the requested time span or surface an explicit error/Log when the page cap is hit while still within the requested range.

## Individual Comments

### Comment 1
<location path="packages/exchange-adapters/src/bybit/bybit.rest.ts" line_range="243-244" />
<code_context>
+      '4h': '240',
+      '1d': 'D',
+    };
+    const bybitInterval = INTERVAL_MAP[interval] || '1';
+    const intervalMs = parseIntervalMs(interval);
+    const PAGE_LIMIT = 1000;
+    const allCandles: Candle[] = [];
</code_context>
<issue_to_address>
**issue (bug_risk):** Bybit candle interval handling has the same silent fallback-to-1m behavior as Upbit.

An unsupported `interval` currently falls back to Bybit `'1'` with `intervalMs` as 1 minute, which can silently distort backtest data. Consider validating `interval` against `INTERVAL_MAP` and throwing for unsupported values instead of defaulting.
</issue_to_address>

### Comment 2
<location path="apps/web/src/components/flows/node-inspector.tsx" line_range="49-64" />
<code_context>
   onChange: (val: unknown) => void;
+  options?: string[];
 }) {
+  if (options && options.length > 0) {
+    return (
+      <select
+        value={String(value)}
+        onChange={(e) => onChange(e.target.value)}
+        className="rounded border border-border bg-background px-2 py-1 text-xs text-foreground outline-none focus:border-primary"
+      >
+        {options.map((opt) => (
+          <option key={opt} value={opt}>
+            {PARAM_VALUE_LABELS[opt] ?? opt}
+          </option>
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Handling values not present in `options` would make the select more robust to future config changes.

If the stored config has a value not in `options`, the `<select>` will get a `value` with no matching `<option>`, which triggers React warnings and can confuse users. Consider checking `!options.includes(String(value))` and either falling back to a safe default or rendering an explicit “unknown value” option to keep the UI consistent.

```suggestion
}) {
  if (options && options.length > 0) {
    const stringValue = String(value);
    const hasMatchingOption = options.includes(stringValue);

    return (
      <select
        value={stringValue}
        onChange={(e) => onChange(e.target.value)}
        className="rounded border border-border bg-background px-2 py-1 text-xs text-foreground outline-none focus:border-primary"
      >
        {!hasMatchingOption && (
          <option value={stringValue} disabled>
            {PARAM_VALUE_LABELS[stringValue] ?? `Unknown value: ${stringValue}`}
          </option>
        )}
        {options.map((opt) => (
          <option key={opt} value={opt}>
            {PARAM_VALUE_LABELS[opt] ?? opt}
          </option>
        ))}
      </select>
    );
  }
```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue (bug_risk): Bybit candle interval handling has the same silent fallback-to-1m behavior as Upbit.

An unsupported interval currently falls back to Bybit '1' with intervalMs as 1 minute, which can silently distort backtest data. Consider validating interval against INTERVAL_MAP and throwing for unsupported values instead of defaulting.

[sourcery-ai]

suggestion (bug_risk): Handling values not present in options would make the select more robust to future config changes.

If the stored config has a value not in options, the <select> will get a value with no matching <option>, which triggers React warnings and can confuse users. Consider checking !options.includes(String(value)) and either falling back to a safe default or rendering an explicit “unknown value” option to keep the UI consistent.

Suggested change

	}) {
	if (options && options.length > 0) {
	return (
	<select
	value={String(value)}
	onChange={(e) => onChange(e.target.value)}
	className="rounded border border-border bg-background px-2 py-1 text-xs text-foreground outline-none focus:border-primary"
	>
	{options.map((opt) => (
	<option key={opt} value={opt}>
	{PARAM_VALUE_LABELS[opt] ?? opt}
	</option>
	))}
	</select>
	);
	}
	}) {
	if (options && options.length > 0) {
	const stringValue = String(value);
	const hasMatchingOption = options.includes(stringValue);
	return (
	<select
	value={stringValue}
	onChange={(e) => onChange(e.target.value)}
	className="rounded border border-border bg-background px-2 py-1 text-xs text-foreground outline-none focus:border-primary"
	>
	{!hasMatchingOption && (
	<option value={stringValue} disabled>
	{PARAM_VALUE_LABELS[stringValue] ?? `Unknown value: ${stringValue}`}
	</option>
	)}
	{options.map((opt) => (
	<option key={opt} value={opt}>
	{PARAM_VALUE_LABELS[opt] ?? opt}
	</option>
	))}
	</select>
	);
	}