Live Research Data
The system is learning.
This page updates automatically as LoadLens ingests new data, generates forecasts, and grades them against reality. Every number is computed from real PJM grid demand via the EIA Open Data API. Nothing is synthetic. Nothing is cached.
Last forecast: 6 hours ago · Last grading: 3 months ago
Hourly Readings
11,119
461.45833333333 days of data
Forecasts Generated
28,393
Forecasts Graded
720
Ensemble MAPE
19.88%
Lower is better
Current Adaptive Weights
Recomputed before every forecast cycle. Models that predict better earn more weight.
Daily Forecast Accuracy (MAPE)
Pre-Registered Evaluation · Top-Line
From the most recent run of php artisan loadlens:eval against v1 of the protocol on 3,909 hours of held-out PJM Region demand. Full per-regime breakdown lives on the methodology page.
Best Baseline
9.21%
B_PERSIST_168
Advanced Ensemble
16.27%
MAPE
[q10, q90] Coverage
14.8%
Target 80% (off-target — see methodology)
Pre-registered protocol v1 · commit 8f12014 · 5 days ago.
Naive baselines currently outperform our point forecasts on transmission-scale data; that gap and the under-coverage finding above are the v2 work, surfaced openly on /methodology.
Observations From the Data
Observation 1: On real PJM hourly demand data (11,119 readings over 461.45833333333 days), the trend/seasonality model consistently outperforms weather-response and momentum models in baseline operating conditions. The adaptive ensemble correctly identified this and upweighted trend from 40% to 84%.
Observation 2: The weather-response model shows higher error (31.79% MAPE) because it currently uses a simplified temperature-driven model. Future iterations will integrate NOAA forecast data as a feature, expected to significantly reduce weather-model error during temperature-driven regime shifts.
Observation 3: Cross-grid validation on ERCOT data (Texas) shows nearly identical ensemble MAPE — confirming the architecture generalizes without domain-specific tuning.
Open question: Will the adaptive mechanism reverse weights during a genuine regime shift (e.g., a summer heat wave)? The current backtest window does not include one. Longer-running validation will answer this.
AI Research Journal
Generated by Claude · Analyzing model performance · Not making predictions
Adaptive Weighting Reveals Stark Performance Gaps Between Component Models
The trend model consistently outperforms weather and momentum models by 5-6x lower MAPE, driving adaptive weights to heavily favor trend-based forecasting despite similar ensemble performance across grids.
Examining 1,056 graded forecasts across PJM (720) and ERCO (336) grids, the ensemble achieved nearly identical MAPE values of 19.88% and 19.75% respectively. This convergence masks dramatic performance disparities among component models. The trend model demonstrates superior accuracy with MAPEs of 5.27% (PJM) and 6.71% (ERCO), while weather and momentum models consistently underperform with MAPEs exceeding 30%.
The adaptive weighting system responds predictably to these performance gaps. PJM weights trend at 83.96% with weather and momentum receiving minimal allocation (~8% each). ERCO shows more balanced weighting (trend 40%, weather 35%, momentum 25%), suggesting either different load patterns or less reliable trend signals in that grid. The weather model receives higher weighting in ERCO despite worse absolute performance, indicating the system may be compensating for seasonal patterns the trend model misses.
Catastrophic errors cluster around midnight hours across both grids, with the worst underpredictions reaching -51% in early 2026. PJM errors concentrate in March-April 2026, while ERCO failures occur primarily in April 2026. These systematic midnight failures suggest a common vulnerability in overnight demand modeling that affects all component models simultaneously, overwhelming even optimal ensemble weighting.
Open research question: Why does the ERCO system maintain substantial weather model weighting (35%) despite its 34.11% MAPE being significantly worse than PJM's weather model performance (31.79% MAPE, 8.18% weight)? This suggests different grid characteristics may require fundamentally different ensemble strategies.
Generated 1 month ago
Weight adaptation reveals stark performance divergence between trend and weather models
The trend model consistently outperforms weather and momentum models by 5-6x across both grids, driving adaptive weights toward heavy trend reliance despite regional differences.
Both PJM and ERCO grids show remarkably similar model performance hierarchies: trend models achieve 5.27% and 6.71% MAPE respectively, while weather and momentum models struggle in the 31-34% range. This 5-6x performance gap is driving the ensemble adaptation in predictable directions, though with notable regional variations.
The adaptive weighting systems are responding differently to the same performance signals. PJM has concentrated 84% weight on the trend model with minimal weather/momentum allocation (8.2%/7.9%), while ERCO maintains more balanced weights (40%/35%/25%). This suggests ERCO's system may be accounting for factors beyond pure historical accuracy, or the trend model's dominance is less stable there.
The worst errors cluster around similar timestamps and magnitudes (-48% to -51%) across both grids, with all major failures occurring during overnight hours (00:xx to 01:xx). The timing pattern suggests systematic blind spots in handling overnight load transitions, particularly during March-April periods when seasonal patterns may be shifting.
Open question: Why does ERCO maintain higher weather model weighting (35%) despite its poor individual performance (34.11% MAPE) compared to PJM's near-abandonment (8.2%) of a similarly performing weather model (31.79% MAPE)?
Generated 1 month ago
Cross-Grid Analysis: Trend Model Dominance with Consistent Midnight Error Clusters
Both PJM and ERCO grids show the adaptive ensemble heavily favoring trend models (84% and 40% weights respectively) while exhibiting systematic errors clustered around midnight hours.
The trend model demonstrates remarkably superior individual performance across both grids (5.27% MAPE for PJM, 6.71% for ERCO) compared to weather and momentum models, which both hover around 31-34% MAPE. This performance gap drives the adaptive weighting algorithm to allocate 84% weight to trend in PJM and 40% in ERCO, suggesting the trend model captures fundamental load patterns that weather and momentum models miss.
Both grids exhibit strikingly similar error patterns: all worst-case errors occur during late-night/early-morning hours (predominantly 00:xx and 01:xx timestamps) with consistent underestimation around -49% to -51%. The temporal clustering suggests systematic challenges in capturing overnight load dynamics, possibly related to industrial scheduling changes, demand response programs, or baseload transitions that occur during low-demand periods.
Weight allocation differs significantly between grids despite similar individual model performance rankings. ERCO maintains more balanced weights (40/35/25) while PJM heavily concentrates on trend (84/8/8). This divergence may reflect different grid characteristics - PJM's larger scale and complexity potentially making trend patterns more reliable, while ERCO's smaller system benefits from incorporating weather and momentum signals.
The ensemble achieves nearly identical MAPE (19.88% vs 19.75%) despite different weighting strategies, raising questions about whether the adaptive algorithm has found different local optima or whether grid-specific factors necessitate distinct ensemble compositions for equivalent performance.
Generated 1 month ago
Methodology & Data Provenance
Data source: EIA Open Data API v2, dataset electricity/rto/region-data, respondent PJM, type D (demand), hourly frequency.
Date range: Apr 12, 2025 to Jul 17, 2026.
Backtest methodology: Walk-forward validation. At each hour, the model sees only prior data. No future leakage.
MAPE calculation: Mean Absolute Percentage Error = mean(|forecast - actual| / actual × 100) across all graded hours.
Adaptive weighting: Inverse-error weighting on the most recent 48 graded forecasts per model.
Update cadence: Every 2 hours via cron. This page reflects the most recent pipeline run.