Problem Being Solved

TIDE exists because understanding the true health of financial markets normally requires visiting dozens of separate data sources — and even then, synthesizing what you find into a coherent picture is left entirely to you. This page explains the specific problem TIDE solves, the approach it takes, and why those design choices matter if you are a developer or analyst who wants a self-hosted, always-current view of capital pressure conditions.

The core problem: fragmented signals, no unified view

Macro conditions are not visible in any single number. Credit stress shows up in high-yield spreads. Liquidity deterioration appears in Fed balance sheet trends and M2 growth. Positioning risk surfaces in COT data and the VIX term structure. Market microstructure breakdown shows up in illiquidity measures like the Amihud ratio. Each of these signals lives at a different source — FRED, the CFTC, Cboe, AAII, SqueezeMetrics, the Treasury — published on different schedules, in different formats, with different conventions for what a "high" reading means.

The practical result is that keeping a real-time picture of capital pressure conditions is either expensive (a Bloomberg or Refinitiv terminal) or extremely time-consuming (manual aggregation across free sources). Neither option is sustainable for an individual analyst or a small team.

What TIDE provides

TIDE aggregates 19 macro indicators spanning four tiers — Macro Liquidity, Capital Flows, Market Microstructure, and Sentiment & Positioning — into a single composite reading, entirely from free public data sources. The composite is not a weighted guess: it is the equal-weight average of the four tier averages, where each indicator contributes through its directional z-score. That means inverted-convention indicators (a tight HY spread is bullish, a low VIX is bullish) are normalized before they enter the composite, so a high composite reading consistently means tighter financial conditions and a low reading means easier conditions regardless of which underlying metrics are driving it.

Beyond the composite, the dashboard gives you:

• Per-indicator readings with z-scores and Bullish / Neutral / Bearish votes, so you can see which specific areas of the market are stressed
• A 252-business-day composite history chart (approximately one trading year) so today's reading has context
• An as-of date on every metric card so you always know exactly how fresh each reading is — stale data from slow-publishing sources is shown rather than hidden, with its last-known value
• A vote tally in the dashboard header summarizing how many indicators are currently Bullish, Neutral, or Bearish
• A watchlist of eight healthcare tickers benchmarked against the XLV sector ETF, each with a 30-day relative return and a sparkline

Why free-tier data only

The constraint of using only free-tier data sources is intentional and permanent. It means TIDE is genuinely self-hostable by anyone: there are no vendor contracts, no API bills that scale with usage, and no credentials to manage beyond a single free FRED API key. The trade-off is that four of the 19 metrics — ici_etf_flows, buyback_yield, put_call, and uvol_dvol — are currently stubbed. They appear in the dashboard registry with their names and descriptions, but they do not fetch live data because their free sources are either gated or require disproportionate engineering to access reliably. The unblock paths for each are documented in the per-tier __init__.py docstrings. The remaining 15 metrics are fully live.

Why not just use yfinance or another aggregator library

Popular libraries like yfinance introduce their own fragility: authentication mechanisms and crumb-based access break recurrently, often for days at a time with no warning. TIDE fetches Yahoo Finance data by hitting query1.finance.yahoo.com/v8/finance/chart/ directly over HTTP, which has been stable in practice. This is a deliberate architectural choice, not an oversight — yfinance is explicitly not a dependency.

Similarly, TLS verification is disabled only for cftc.gov requests (which have historically had certificate issues), while all other upstream calls use standard TLS. The data pipeline is designed to be robust at the source level rather than relying on intermediary libraries to handle this for you.

The self-hosted model

Because TIDE runs entirely on your own machine or server, you control the data, the refresh schedule, and the extensions. The backend (FastAPI + DuckDB) and the frontend (SvelteKit) are separate processes that share only the DuckDB database file. This means a wedged ingest process — for example, a slow CFTC download or a transient AAII bot-detection failure — cannot take the dashboard down. You can always open the dashboard and see the last-known readings while ingest is retrying in the background.

For daily operation, the Scheduler runs two cron jobs in the America/New_York timezone: a daily_ingest job Monday through Friday at 17:00 (after US market close) and a release_ingest job on Fridays at 18:00 that picks up weekly, monthly, and quarterly data releases. Once you start it, the dashboard stays current without any manual intervention.

Checking the composite reading after first setup

After running the initial ingest and starting the dashboard, open your browser to see the current composite Capital Pressure score and vote tally:

# Pull all live metric data (~30 seconds)
make ingest-all

# Pull watchlist tickers
make ingest-watchlist

# Compute the 252-day composite history
make backfill-composite

# Start the API and web server (two terminals)
make dev-api   # FastAPI on :8765
make dev-web   # SvelteKit on :5173

Expected result: navigating to http://localhost:5173 shows the composite z-score in the hero panel, the 252-day history chart, all four tier grids with per-indicator z-scores and vote chips, and the watchlist with sparklines. Every metric card displays its own as-of date.

Verifying data freshness on the Sources page

If you want to confirm when each data source last ran and whether any errors occurred, navigate to the Sources page:

http://localhost:5173/sources

Expected result: a table showing each data source with its last-run time, next scheduled run, and any ingest errors. This page reads from the scheduler_status DuckDB table and requires no additional commands.

Starting the Scheduler for automatic daily refresh

To keep the dashboard current without running ingest commands manually each day:

make scheduler

Expected result: the scheduler starts and blocks in the foreground. It will automatically run daily_ingest Monday through Friday at 17:00 New York time, and release_ingest on Fridays at 18:00. Press Ctrl-C to stop. Job status is persisted to DuckDB and visible at /sources.

• Composite reading — How the single Capital Pressure score is calculated from the four tier averages and what a high or low value means in practice
• Z-score — How each indicator's raw value is converted into a normalized, direction-adjusted score that feeds the composite
• Vote and tally — How individual indicator z-scores become Bullish, Neutral, or Bearish votes and how the tally is displayed in the dashboard header
• Tier — The four groups of indicators (Macro Liquidity, Capital Flows, Market Microstructure, Sentiment & Positioning) and which metrics belong to each
• Stale data — Why TIDE shows last-known values for slow-publishing indicators rather than hiding them, and how to interpret as-of dates
• Stubbed metric — What it means when an indicator is registered in the dashboard but has no live data, and how to unblock it
• Watchlist — How the eight healthcare tickers are benchmarked against XLV and what relative return and sparklines tell you
• Sources page — How to check the status, last-run time, and error history of every data source at /sources