Notes on open data
ETL, licensing, and the datasets we ship.
Look-ahead bias in energy backtests: forecast vs actual
The fastest way to fake a great energy-trading backtest is to let the future leak in. Two ways it happens - using the outcome instead of the forecast, and restated data - and how to model forecast error honestly.
Stop using CSVs for time-series market data
If you are still loading millions of rows of energy or financial data with pd.read_csv(), you are burning compute and inviting silent type bugs. Here is why time-series belongs in Apache Parquet - and how columnar storage, compression and partitioning change everything.
The gas-power nexus: why you can't model electricity in a vacuum
In Europe, gas sets the marginal power price. But joining gas storage to electricity dispatch means reconciling bidding zones against ISO country codes - a spatial-mapping headache. Here's how we solved it.
Clean Hyperliquid funding & open-interest data, explained
Hyperliquid's on-chain perp DEX publishes rich market data, but as raw intraday snapshots across 200+ coins. What funding, open interest, mark-oracle basis and impact spread mean - and how to get them as clean, join-ready Parquet.
Stop using annual averages for Scope 2 grid emissions
Corporate ESG reports and carbon-aware schedulers lean on static yearly grid carbon-intensity figures. In a renewables-heavy grid that method is broken. Here's why you need interval-level gCO2/kWh - and where to get it.
One file, 121 indicators: a European country-profile dataset
GDP, inflation, labour, demographics, health, education, digital, R&D and energy for every European country - 121 official indicators joined on country and year in a single tidy Parquet. What's inside, and the methodology decisions behind it.
The renewable revenue trap: why average power prices lie to you
If you price a PPA or model a solar asset on average day-ahead prices, your revenue is dangerously inflated. The maths behind renewable capture prices and cannibalisation - and where to get them clean.
The CFTC Commitments of Traders (COT) report, explained
What the CFTC Commitments of Traders report actually tells you about futures positioning, how to read the legacy, disaggregated and financial-trader breakdowns, and how to get the whole multi-decade history as clean Parquet.
Open data you can actually resell: the licensing filter
Not all 'free' data is redistributable. The licence is the filter that decides what a data product can even be built from.
European emissions and the energy transition: a data guide
Where to find clean, comparable data on European greenhouse-gas emissions, renewable-energy share, power-sector carbon intensity, energy import dependency and the circular economy - and the pitfalls that make it deceptively hard.
Dataset spotlight: European Day-Ahead Electricity Prices
A tour of our most-loved dataset - eight years of hourly day-ahead prices across ~40 European bidding zones, clean.
Analysis-ready vs raw: what 'skip the ETL' actually saves you
Raw open data is free. The cost is the days of cleaning between download and your first correct chart. Here's the math.
Why loading ENTSO-E data is harder than it looks
Timezones, B-codes and silent gaps: the three things that turn a 'simple' ENTSO-E download into a week of ETL.