MATHEMATICS.md

Mathematical and Machine-Learning Model

1. Candidate representation

Each detected monetary amount is represented as a candidate:

$$ c_i = (p_i, q_i, s_i, t_i, d_i) $$

where:

• $p_i$ is the positive numeric amount;
• $q_i$ is the detected or expected currency;
• $s_i$ is the extraction source;
• $t_i$ is the visible text associated with the amount;
• $d_i$ is the local page context surrounding the candidate.

The parser supports common decimal conventions. When both . and , are present, the rightmost separator is interpreted as the decimal separator. Repeated groups of three digits are interpreted as thousands separators.

2. Local feature vector

The local model maps each candidate and requested item to an eleven-dimensional feature vector:

$$ \mathbf{x}_i = [x_1, x_2, \ldots, x_{11}] $$

The features include:

• extraction-source indicators;
• currency agreement;
• positive and negative price attributes;
• query-token overlap;
• sale-price and list-price terms;
• $\log(1+p_i)$;
• normalized text length.

The query-overlap ratio is defined as:

$$ r_i = \frac{\lvert {w \in Q : w \text{ occurs in } d_i} \rvert}{\max(1, \lvert Q \rvert)} $$

where $Q$ is the set of normalized query tokens.

3. Deterministic local confidence

When no trained model is available, the system computes a linear score followed by a logistic transformation:

$$ z_i = b + \mathbf{w}^{\mathsf{T}}\mathbf{x}_i $$

$$ P_i = \sigma(z_i) = \frac{1}{1 + e^{-z_i}} $$

The weights reward structured product data, matching currencies, price-labelled elements, and query overlap. They penalize list-price, MSRP, and other negative contexts.

This score is a transparent ranking heuristic. It must not be interpreted as a calibrated probability that the extracted amount is the true market price.

4. Optional Random Forest

A labelled candidate dataset can be used to train $T$ classification trees on bootstrap samples. The final confidence is the average positive-class probability across all trees:

$$ \widehat{P}(y=1 \mid \mathbf{x}) = \frac{1}{T}\sum_{t=1}^{T} P_t(y=1 \mid \mathbf{x}) $$

The implementation uses class balancing, bounded tree depth, a minimum leaf size, and a fixed random seed. These choices reduce variance and make training reproducible.

Users must create labels from their own legally collected and appropriately licensed pages.

5. Robust outlier flagging

For multiple prices associated with the same item and currency, define the median price as:

$$ m = \mathrm{median}(p_1, p_2, \ldots, p_n) $$

The median absolute deviation is:

$$ \mathrm{MAD} = \mathrm{median}(|p_i-m|) $$

The modified z-score of candidate $i$ is:

$$ z_i^{*} = 0.6745,\frac{p_i-m}{\mathrm{MAD}} $$

A candidate is marked as an outlier when:

$$ |z_i^{*}| > 3.5 $$

This rule is applied only when at least three observations are available and $\mathrm{MAD} > 0$.

The selected row is the highest-confidence non-outlier. It is not necessarily the lowest observed price.

6. Throughput model

Let:

• $N$ be the number of enabled items;
• $U$ be the maximum number of URLs processed per item;
• $\tau$ be the average request time;
• $W$ be the number of workers.

An idealized lower bound for the total runtime is:

$$ T_{\mathrm{run}} \gtrsim \frac{N U \tau}{W} $$

Actual runtime is generally higher because:

• requests are rate-limited per domain;
• browser fallbacks are more expensive than direct HTTP requests;
• proxy and cloud providers add latency;
• retries, timeouts, and anti-bot protections increase execution time.

Increasing $W$ does not override per-domain delays, provider limits, or website policies.

7. Limitations

• A confidence score measures consistency with the selected features and training labels, not objective truth.
• Displayed prices may exclude taxes, shipping, minimum-order quantities, contract conditions, or regional restrictions.
• Currency matching is not currency conversion.
• A product-page price may be stale, personalized, dynamically generated, or dependent on authentication.
• Statistical outlier rejection may remove legitimate offers in small or multimodal markets.
• A trained model can reproduce biases or annotation errors present in its training dataset.