AI-Powered Price Intelligence: Feature Engineering and Real-Time Serving

Price intelligence used to mean a spreadsheet of competitor prices refreshed weekly. In 2026 it means a continuous feature pipeline, a forecasting model running on a schedule, and a serving layer that feeds pricing decisions into the checkout flow within hundreds of milliseconds. The modeling is well understood; the hard part is keeping the input data fresh and consistent across markets.

This post walks through feature engineering from collected pricing data, time-series models that actually work for retail, and the serving architecture for decisions at request time.

What Price Intelligence Actually Requires

Four inputs, each with distinct collection patterns:

• Competitor prices: the current listed price for equivalent SKUs across retailers
• Historical prices: your own and competitors' prices over time, for trend and volatility features
• Promotions and availability: active discounts, bundle offers, in-stock status
• Market signals: review counts, ratings, search rankings, related-product views

Each varies by region. A price collected from one IP is one viewer's price; retailers run A/B tests, regional targeting, and dynamic pricing on every page view. Single-point observations are noise; you need many views per SKU per day across markets.

Collection: Per-SKU, Per-Market, Per-Hour

A reasonable starting cadence:

• Strategic SKUs (top 5% by revenue): every 15-60 minutes, 5+ markets
• Watched SKUs (next 20%): every 2-6 hours, 3+ markets
• Long tail: daily, 1-2 markets

Distributing collection across a proxy pool lets you poll at this rate without hitting per-IP limits and gives you per-market egress so regional pricing resolves correctly. See our e-commerce price monitoring guide for the collection-side patterns.

Feature Engineering

Raw prices are the weakest input. The features that actually predict well are derived:

Temporal features

• Price level (log-transformed)
• First differences (day-over-day change)
• Rolling mean and std (7d, 30d, 90d windows)
• Exponentially weighted mean with decay tuned to your category
• Day of week, hour of day, holiday indicator, days-to-next-holiday

Relative features

• Price rank within the competitor set
• Distance from median competitor price
• Distance from lowest price
• Promotion indicator (is this SKU on sale anywhere)

Volatility features

• 30-day price variance
• Number of price changes per week
• Time since last price change

Market signals

• Review count delta (proxy for demand)
• Rating delta
• Search rank change on the retailer's category page

Keep a feature store (Feast, Tecton, or a Postgres/Redis setup) so the same feature definitions are used at training and serving time. Training-serving skew is the most common reason pricing models degrade silently.

Model Choice

Three classes of model are worth considering:

Gradient boosted trees

LightGBM or XGBoost with the features above, target = optimal price or demand-at-price. Stable, fast to train, easy to interpret with SHAP. The default choice and often hard to beat.

Classical time-series

ARIMA, Prophet, ETS for univariate forecasts of your own demand and competitor-price trajectories. Cheap, well-calibrated confidence intervals, fail gracefully. Good as components in a larger system.

Deep time-series

N-BEATS (Oreshkin et al., arXiv:1905.10437), Temporal Fusion Transformer (Lim et al., arXiv:1912.09363), and TimesFM (Das et al., 2024) dominate M4/M5 leaderboards. Strong when you have hundreds of thousands of related series; overkill for a few thousand SKUs.

In practice a GBM baseline plus a Prophet component for seasonality captures most of the value.

A Minimal Training Pipeline

import lightgbm as lgb
import pandas as pd

# features: per-SKU-per-timestamp dataframe
# target: future price or demand
train = df[df.ts < "2026-04-01"]
valid = df[df.ts >= "2026-04-01"]

params = {
    "objective": "regression",
    "metric": "mape",
    "learning_rate": 0.03,
    "num_leaves": 63,
    "feature_fraction": 0.8,
    "bagging_fraction": 0.8,
    "bagging_freq": 5,
    "min_child_samples": 20
}

model = lgb.train(
    params,
    lgb.Dataset(train[features], train.target),
    valid_sets=[lgb.Dataset(valid[features], valid.target)],
    num_boost_round=2000,
    callbacks=[lgb.early_stopping(50)]
)

Serving with Current Market Data

A recommendation is only as good as the features it saw at decision time. Architecture:

1. Collectors write raw observations to a stream (Kafka, Kinesis)
2. A stateful processor (Flink, Faust) computes rolling features and writes to Redis
3. The serving API reads features from Redis, calls the model, returns a price recommendation with a confidence band
4. Decisions are logged back to the stream for offline analysis

Latency budget: 30-50ms for the feature read, 5-15ms for model inference, rest for application logic.

Demand Modeling vs Price Optimization

Predicting the right price is harder than predicting demand. The common move is two-stage:

1. Model demand as a function of price: d = f(price, features)
2. Optimize over price: argmax_p (p - c) · f(p, features), subject to business constraints (margin floors, price-ladder consistency, MAP)

Price elasticity estimation is tricky because your historical data rarely spans the full price range. Bayesian hierarchical models (e.g. via PyMC) give calibrated uncertainty that pure ML regressors lack.

Guardrails

A pricing model that can move prices live needs hard guardrails:

• Maximum daily price change per SKU (e.g. 10%)
• Floor at cost + minimum margin
• Ceiling at a manual cap
• Ladder consistency: a 12-pack must not cost more than two 6-packs
• Freeze switch: if collection freshness lapses beyond a threshold, freeze decisions
• Kill switch: revert to last-known-good price table on model failure

Monitoring Price Data Freshness

The quality of your price intelligence is the age of the oldest price you are relying on. Track:

• Median and p99 age of competitor prices per SKU, per market
• Collection success rate per retailer
• Drift in distribution of fetched prices (sudden shifts usually mean a site changed their markup or you are getting blocked)

A/B Testing

Never deploy a pricing model change without a holdout. Standard setup: 90% of traffic on the new model, 10% on control, measure revenue per visitor and gross margin over 1-2 weeks. Sequential testing (e.g. Lindon et al., 2022) lets you stop early without inflating the false positive rate.

Closing

Price intelligence is a data problem with a thin modeling layer on top. Get the collection right -- fresh, per-market, consistent -- and a boring gradient boosted model will beat a fancy architecture trained on stale data. Feature stores, guardrails, and disciplined A/B testing are the boring infrastructure that separates a working system from a demo.