Getting Started

This page walks you through setting up TIDE — Capital Pressure Observatory on your local machine, from cloning the repository to opening a live dashboard in your browser. By the end, you will have a running four-tier composite dashboard pulling real data from FRED, Yahoo Finance, the CFTC, and other free sources, with a 252-business-day history chart and a healthcare watchlist. Getting the initial setup right matters because TIDE's backend and frontend are separate processes that share a single DuckDB file — understanding that structure helps you troubleshoot confidently if anything goes wrong.

Before you begin, make sure you have the following:

• FRED API key — Register for a free key at fred.stlouisfed.org/docs/api/api_key.html. TIDE uses FRED to pull M2 money supply, the Fed balance sheet, HY credit spreads, NFCI, and bank credit totals. Without this key, the backend cannot fetch any FRED series and make ingest-all will fail.
• Python ≥ 3.10 (the backend pyproject.toml requires ≥ 3.11 in practice — use 3.11 or newer to be safe)
• Node.js and npm — Required to install and run the SvelteKit frontend
• make — All developer workflows are exposed through the Makefile
• Internet connection — TIDE pulls live data from FRED, Yahoo Finance chart API, CFTC, AAII, SqueezeMetrics, and Treasury TIC on first run

No paid data subscriptions are needed. Every data source TIDE uses is free-tier.

The steps below take you from zero to a running dashboard. Run them in order — each step depends on the previous one.

1. Copy the example environment file:

   Copy

   cp .env.example .env
   

2. Open .env and add your FRED API key:

   Copy

   FRED_API_KEY=your_key_here
   

3. Install all dependencies (backend Python packages into your virtual environment, frontend npm packages):

   Copy

   make install
   

4. Create the DuckDB schema and seed the metrics registry:

   Copy

   make init-db
   

5. Pull the latest data for all 15 live indicators:

   Copy

   make ingest-all
   

6. Pull the 8 healthcare watchlist tickers and the XLV benchmark:

   Copy

   make ingest-watchlist
   

7. Compute the 252-business-day composite history:

   Copy

   make backfill-composite
   

8. In one terminal, start the FastAPI backend:

   Copy

   make dev-api
   

9. In a second terminal, start the SvelteKit frontend:

   Copy

   make dev-web
   

10. Open http://localhost:5173 in your browser.

You should see the composite Capital Pressure reading, all four tier grids, and the watchlist.

The following procedure expands each quickstart step with detail about what is happening and what a successful result looks like.

Step 1 — Create your environment file

cp .env.example .env

The .env file is where TIDE reads secrets and configuration at startup. The frontend's vite.config.ts sets envDir: '..' so both the backend and frontend read from the same file at the project root. Never commit .env to version control.

Step 2 — Add your FRED API key

Open .env in any text editor and set:

FRED_API_KEY=your_key_here

This key is required for every FRED series — M2, the Fed balance sheet (WALCL), HY credit spread (BAMLH0A0HYM2), NFCI, bank credit (TOTLL), and margin debt. If the key is missing or invalid, make ingest-all will fail with an authentication error from the FRED API.

Note on the HY spread series: FRED truncated BAMLH0A0HYM2 to a 3-year window in April 2026. TIDE handles this automatically — the HY spread metric's zscore_years is already set to 1y in the MetricDefinition registry to match the available window. You do not need to change anything.

Step 3 — Install dependencies

make install

This runs two sub-targets in sequence:

• install-backend — installs the tide Python package (editable) into the virtual environment at $VENV (default: ~/sandbox/envs/tideenv)
• install-frontend — runs npm install in the frontend/ directory

To use a different virtual environment path, pass VENV:

make install VENV=./backend/.venv

Success: No pip or npm errors. The tide-ingest console script becomes available at $VENV/bin/tide-ingest.

Step 4 — Initialize the database

make init-db

This runs python -m tide.db init, which creates the DuckDB file and sets up the schema (the observations and metrics tables) and seeds the MetricDefinition registry with all 19 indicators — 15 live and 4 stubbed (ici_etf_flows, buyback_yield, put_call, uvol_dvol).

The DuckDB file path is resolved by config.py using pydantic-settings relative to the project root. There is no external database server to start — DuckDB is an embedded file-based database.

Success: The command exits without errors. You can re-run make init-db safely at any time; it is idempotent.

Step 5 — Ingest all live metrics

make ingest-all

This runs tide-ingest run --all, which iterates every MetricDefinition that has an ingest_fn and pulls the latest data from the upstream source. The 15 live metrics span FRED, the Yahoo Finance chart API (fetched directly over HTTP to query1.finance.yahoo.com/v8/finance/chart/ — not via yfinance), SqueezeMetrics, AAII, CFTC, and Treasury TIC.

Expect this to take approximately 30 seconds. Some sources are slower than others:

• AAII may retry up to 4 times with backoff due to bot-detection cycling between 200 and 403 responses. This is normal.
• CFTC requests disable TLS verification only for cftc.gov — this is intentional and scoped only to that host.
• The 4 stubbed metrics are silently skipped because they have no ingest_fn.

Success: The command completes without a Python traceback. Each source logs its fetched date range.

Step 6 — Ingest the watchlist

make ingest-watchlist

This pulls price history for the 8 hardcoded healthcare tickers and the XLV sector ETF benchmark. The watchlist displays 30-day relative returns (each ticker versus XLV) and a sparkline for each holding.

Success: The command completes without errors. You will see the watchlist populated when you open the dashboard.

Step 7 — Backfill the composite history

make backfill-composite

This runs tide-ingest backfill-composite, which computes the 252-business-day historical composite series from the per-metric observation data already in DuckDB. The composite z-score is the equal-weight average of the four tier averages, using each metric's directional_z so that inverted-convention metrics (HY tight = bullish, VIX low = bullish, and so on) contribute in the correct direction. For metrics that publish on a lag, the backfill uses the last known value (stale data is shown, not hidden).

You must run this after every make ingest-all to keep the history chart current.

Success: The command exits cleanly. The composite history chart in the dashboard hero panel will show a line spanning approximately one trading year.

Step 8 — Start the backend API

In a dedicated terminal:

make dev-api

This starts Uvicorn serving the FastAPI app at http://localhost:8765. The backend exposes routes including /api/dashboard, /api/watchlist, /api/metrics, /api/health, and the Sources page at /sources.

To use a different port:

make dev-api PORT=9000

Success: You see Uvicorn startup output ending with Application startup complete. The terminal stays open and logs incoming requests.

Step 9 — Start the frontend

In a second dedicated terminal:

make dev-web

This runs npm run dev in the frontend/ directory, starting the SvelteKit dev server on port 5173 bound to 0.0.0.0 (so it is reachable on your LAN IP as well as localhost). The page server calls /api/dashboard and /api/watchlist in parallel on each request and renders the full dashboard server-side.

Success: You see Vite startup output with a local URL. The terminal stays open.

Step 10 — Open the dashboard

Navigate to http://localhost:5173.

You should see:

• The composite Capital Pressure reading and vote tally in the hero panel
• The 252-business-day SVG history chart
• Four tier grids (Macro Liquidity, Capital Flows, Market Microstructure, Sentiment & Positioning) with per-indicator z-scores, votes, and as-of dates
• The healthcare watchlist with relative returns and sparklines

To check source status and last-run times, navigate to http://localhost:5173/sources.

Example 1 — Full setup from scratch

A complete session from a fresh clone to a running dashboard:

# 1. Configure environment
cp .env.example .env
echo 'FRED_API_KEY=abcdef1234567890abcdef1234567890' >> .env

# 2. Install all deps
make install

# 3. Initialize DuckDB schema and metrics registry
make init-db

# 4. Pull all live metric data (~30s)
make ingest-all

# 5. Pull watchlist price data
make ingest-watchlist

# 6. Compute 252-day composite history
make backfill-composite

Expected output from make ingest-all (abbreviated):

[fred] m2: fetched 2020-01-01 → 2025-06-01
[fred] walcl: fetched 2020-01-01 → 2025-06-04
[fred] hy_spread: fetched 2022-06-01 → 2025-06-04
[fred] nfci: fetched 2020-01-01 → 2025-05-30
[fred] bank_credit: fetched 2020-01-01 → 2025-06-04
[yahoo] move: fetched 756 rows
[yahoo] vix_term: fetched 756 rows
[yahoo] spy_illiquidity: fetched 756 rows
[yahoo] hyg_illiquidity: fetched 756 rows
[yahoo] stock_bond_corr: fetched 756 rows
[aaii] aaii_sentiment: fetched after 2 retries (bot detection)
[squeezemetrics] dealer_gamma: fetched 756 rows
[cftc] cot_sp: matched contract pattern; fetched 52 rows
[treasury_tic] foreign_equity: fetched 2023-01-01 row (file cap)
Skipping ici_etf_flows — no ingest_fn (stubbed)
Skipping buyback_yield — no ingest_fn (stubbed)
Skipping put_call — no ingest_fn (stubbed)
Skipping uvol_dvol — no ingest_fn (stubbed)

Example 2 — Starting both servers and accessing the dashboard

Open two terminals:

# Terminal 1 — backend
make dev-api
# → Uvicorn running on http://127.0.0.1:8765

# Terminal 2 — frontend
make dev-web
# → Local:   http://localhost:5173/
# → Network: http://192.168.1.42:5173/

Then open http://localhost:5173 in your browser.

Example 3 — Using a custom virtual environment and port

If you want to keep the virtual environment inside the project directory and run the API on a non-default port:

make install VENV=./backend/.venv
make init-db VENV=./backend/.venv
make ingest-all VENV=./backend/.venv
make ingest-watchlist VENV=./backend/.venv
make backfill-composite VENV=./backend/.venv
make dev-api VENV=./backend/.venv PORT=9000

The frontend does not depend on VENV and always uses the local node_modules.

Example 4 — Manual daily refresh

Once the initial setup is done, updating TIDE to the latest data requires three commands:

make ingest-all
make ingest-watchlist
make backfill-composite

make ingest-all is idempotent — re-running it re-pulls each source's full available series and overwrites stale observations in DuckDB. Running make backfill-composite afterwards recomputes the history chart from the freshest data.

Example 5 — Ingesting a single metric

To pull only M2 without running all sources:

make ingest-m2

For any other individual metric, you can call the CLI directly:

$VENV/bin/python -m tide.ingest.cli run --metric nfci

Issue: make ingest-all fails with an authentication or API key error

• Symptom: The command exits with an error mentioning FRED_API_KEY, 401, or Bad Request from a FRED endpoint.
• Cause: The FRED_API_KEY variable is missing from .env, is set to the placeholder value, or the .env file was not created.
• Fix: Confirm .env exists at the project root and contains FRED_API_KEY=<your actual key>. You can verify with grep FRED_API_KEY .env. If the file does not exist, run cp .env.example .env and add your key.

Issue: make ingest-all hangs or retries several times on the AAII step

• Symptom: The ingest process pauses for 20–60 seconds and logs retry warnings before eventually succeeding or failing.
• Cause: AAII's server uses bot-detection logic that cycles between returning 200 and 403 responses to the same headers, sometimes minutes apart. This is a known upstream behavior.
• Fix: Wait — the AAII client retries up to 4 times with backoff and validates the downloaded file's OLE2 magic bytes. If all 4 retries fail, re-run make ingest-all after a few minutes. The other 14 live metrics will be skipped quickly because they already have fresh data (the command is idempotent).

Issue: make dev-api starts but the dashboard shows no data or all indicators show as stale

• Symptom: The frontend loads at localhost:5173 but the composite score is blank, or every metric card shows a very old as-of date.
• Cause: Either make ingest-all was not run before starting the servers, or make backfill-composite was skipped.
• Fix: Stop the servers, run make ingest-all, make ingest-watchlist, and make backfill-composite in order, then restart both servers.

Issue: Frontend cannot reach the backend — network error or blank dashboard

• Symptom: The browser console shows a fetch error, or the SvelteKit page server logs a connection refused error to port 8765.
• Cause: The FastAPI backend (make dev-api) is not running, crashed, or is listening on a different port than the frontend expects.
• Fix: Confirm make dev-api is running in a separate terminal and that its output shows Uvicorn running on http://127.0.0.1:8765. If you started the backend on a custom port with PORT=9000, the frontend's server-side fetch will still target 8765 unless you have also updated the API base URL in the frontend configuration.

Issue: make init-db fails with a file or permission error

• Symptom: The command exits with an error such as PermissionError, FileNotFoundError, or a DuckDB lock error.
• Cause: Either the project directory is not writable, or a previous TIDE process has the DuckDB file open and locked.
• Fix: Ensure no other make dev-api or tide-ingest process is running before calling make init-db. DuckDB allows only one writer at a time; stop all TIDE processes, then retry.

Issue: make install fails because the virtual environment does not exist

• Symptom: pip: command not found or No such file or directory: '/home/murali/sandbox/envs/tideenv/bin/pip'
• Cause: The default VENV path (~/sandbox/envs/tideenv) does not exist on your machine.
• Fix: Create the virtual environment first (python3 -m venv ~/sandbox/envs/tideenv), then run make install. Alternatively, point VENV to any existing environment: make install VENV=./backend/.venv (creating it first with python3 -m venv ./backend/.venv).

Issue: Watchlist sparklines or relative returns are missing

• Symptom: The watchlist section loads but sparklines are empty or relative return values are all zero.
• Cause: make ingest-watchlist was not run, or the Yahoo Finance chart API request failed for one or more tickers.
• Fix: Run make ingest-watchlist and check the output for HTTP errors. TIDE fetches Yahoo Finance data by hitting query1.finance.yahoo.com/v8/finance/chart/ directly over HTTP — not via yfinance. If you see connection errors, verify your internet connection and retry.

Issue: Four indicators always show as stubbed or have no reading

• Symptom: ici_etf_flows, buyback_yield, put_call, and uvol_dvol always appear without a live reading.
• Cause: This is expected behavior. These four metrics are intentionally stubbed — they are registered in the MetricDefinition registry with metadata but have no ingest_fn because their free data sources are gated or unavailable. They are not a setup error.
• Fix: No action needed. The per-tier __init__.py docstrings in the backend source document the paths to unblocking each stubbed metric if you want to contribute an implementation.