Troubleshooting

This runbook helps you diagnose and resolve common errors encountered when using the dokobot-intel-toolkit pipeline — from bridge connectivity failures to scraping, parsing, comparison, and report generation issues — so you can restore a working pipeline run as quickly as possible.

In scope:

• dokobot Edge bridge not detected (NO_DEVICE) errors
• Authenticated scraping failures across supported platforms (X/Twitter, Reddit, V2EX, Product Hunt, IndieHackers, Xiaohongshu, Bilibili, YouTube, and others)
• Raw data encoding and parsing errors from the Python parse & clean scripts
• Incremental comparison failures (missing baseline, diff errors)
• Summary and report generation failures
• WorkBuddy Skill activation and runtime errors (bili-subtitle, deck-studio, xhs-commercial-growth)
• faster-whisper fallback issues in the bili-subtitle Skill

Out of scope:

• Initial installation and first-time setup of the dokobot CLI or WorkBuddy application
• Platform-side changes (e.g., a platform redesigning its DOM structure) — these require a script update, not a config fix
• Microsoft Edge browser crashes unrelated to the bridge extension
• WorkBuddy application internal errors not related to Skill activation

Before using this runbook, confirm you have the following in place:

• Python ≥ 3.11 — required by all parse, clean, comparison, and report scripts
• Node.js ≥ 22 — required by the dokobot CLI runtime
• dokobot CLI — installed and accessible on your PATH
• Microsoft Edge — with the dokobot bridge extension installed
• Windows OS with PowerShell — the scraping scripts are PowerShell-based
• WorkBuddy application (optional) — required only if you are troubleshooting a WorkBuddy Skill
• faster-whisper (optional) — required only if you are debugging the bili-subtitle Skill's speech-recognition fallback
• You should be able to run dokobot_ready.py from your terminal before beginning any troubleshooting step

Work through these steps in order. Each step targets a specific failure layer in the pipeline. Stop at the step that resolves your issue.

Step 1 — Confirm the dokobot Edge bridge is active

All authenticated scraping depends on the Edge bridge being live. Run the readiness check first:

python dokobot_ready.py

Success looks like:

DEVICE_ID:<uuid>

A DEVICE_ID:<uuid> output confirms the bridge is active and a device ID has been assigned. Proceed to Step 2.

Failure looks like:

NO_DEVICE

If you see NO_DEVICE, the bridge extension is not responding. Do the following:

1. Open Microsoft Edge.
2. Navigate to edge://extensions/ and verify the dokobot bridge extension is enabled (toggle it off and on if needed).
3. Re-run python dokobot_ready.py. You must get a DEVICE_ID:<uuid> before any scraping can proceed.

Why this matters: The dokobot CLI relays all scrape requests through the Edge bridge to reuse your existing browser login sessions. If the bridge is not active, every platform that requires authentication will fail silently or return anonymous/empty responses.

Step 2 — Diagnose a scraping script failure

If dokobot_ready.py returns a valid DEVICE_ID but one or more PowerShell scraping scripts fail or produce empty output files:

1. Check whether you are logged in on the target platform in Edge. Open Edge and manually navigate to the failing platform (e.g., X/Twitter, Reddit, Xiaohongshu). If Edge shows a login page, log in and re-run the script.
2. Check for rate-limiting. The scraping scripts are rate-limit-aware, but platforms may still throttle requests. If you see HTTP 429 responses in the script output, wait at least 15 minutes before retrying.
3. Run the affected script individually with verbose or debug output if the script supports it, to isolate the exact platform and request failing.
4. Verify the raw output file was written. Each scraping script saves a raw output file. If the file is missing or zero bytes, the script exited before completing.

Success looks like: The raw output file for each platform exists, is non-empty, and contains recognizable content (titles, URLs, scores).

Step 3 — Diagnose parse and clean script errors

If raw output files exist but the Python parse & clean scripts fail:

1. Encoding errors — Raw files from some platforms (especially Xiaohongshu and V2EX) may contain non-UTF-8 characters. If you see a UnicodeDecodeError, inspect the raw file's encoding and ensure the parse script's encoding argument matches.
2. Missing fields — If the script raises a KeyError or outputs records with blank title, popularity score, or URL fields, the platform's response format may have changed. Note the platform and the field name for escalation (see the Escalation section).
3. Noise not removed — If the cleaned output contains obvious junk (ads, boilerplate text), the noise-removal rules in the parse script may need updating. Document the pattern and escalate.

Success looks like: Each platform's cleaned output file contains structured records with populated title, popularity score, and URL fields.

Step 4 — Diagnose incremental comparison failures

The incremental comparison scripts diff today's cleaned data against yesterday's baseline. Common failure modes:

1. Missing baseline file — If this is your first run, no baseline (yesterday's data) exists yet. This is expected; the script should generate today's data as the new baseline. Confirm the baseline file is written after the run so tomorrow's comparison will work.
2. Baseline from more than one day ago — If you skipped a day, the diff will flag more topics as 🆕 than usual. This is normal behavior, not an error.
3. Comparison script exits with an error — Check that the cleaned output files from Step 3 are present and in the expected directory. The comparison script cannot run if its input files are missing.
4. No 🆕 annotations appear — If the diff produces output but nothing is marked as newly trending, confirm that today's cleaned data actually differs from the baseline. If the files are identical (e.g., the scrape pulled cached content), re-run the scraping scripts.

Success looks like: The comparison output lists topics annotated with 🆕 for items that are new since the baseline, and unannotated items for topics that were already present yesterday.

Step 5 — Diagnose report generation failures

If the summary and report generation scripts fail after comparison succeeds:

1. Input files missing — The report script aggregates signals from all platform comparison outputs. Confirm all per-platform comparison files exist before running the report script.
2. Empty report — If the report generates but contains no monetization signals, check whether the comparison step (Step 4) produced any 🆕-annotated topics. An empty diff produces an empty report.
3. Script error during aggregation — Run the report script in isolation and capture the full error output. Note the script name and error message for escalation.

Success looks like: A single structured intelligence report file is written, containing ranked monetization signals drawn from all platforms.

Step 6 — Diagnose WorkBuddy Skill activation or runtime errors

Skip this step if you are not using WorkBuddy Skills.

Skills are stored in ~/.workbuddy/skills/. If a Skill fails to activate or run:

1. Skill not found — Confirm the Skill directory exists under ~/.workbuddy/skills/ for the Skill you are trying to use (bili-subtitle, deck-studio, or xhs-commercial-growth).
2. bili-subtitle Skill fails on a video with no subtitles — If the target Bilibili video has no official subtitle track, the Skill falls back to faster-whisper for local speech recognition. Confirm faster-whisper is installed if you need this fallback. Without it, the Skill will error on videos lacking subtitles.
3. bili-subtitle Skill: invalid BV number — Confirm you are passing a valid Bilibili BV number (format: BV followed by alphanumeric characters). The Skill requires a BV number as input, not a full URL.
4. deck-studio Skill produces no output — Confirm you provided a topic brief as input. The Skill requires a topic or brief to generate a slide deck outline.
5. xhs-commercial-growth Skill produces no framework output — Confirm the Skill is activated inside the WorkBuddy application before invoking it. Skills cannot run outside the WorkBuddy environment.

Success looks like: Each Skill completes without error and produces its expected artifact — a transcript (bili-subtitle), a slide deck outline (deck-studio), or a monetization framework evaluation (xhs-commercial-growth).

After completing the relevant steps above, verify the full pipeline is healthy by checking the following observable outcomes:

1. Bridge readiness: Running python dokobot_ready.py returns DEVICE_ID:<uuid> (not NO_DEVICE).
2. Raw output files: A non-empty raw output file exists for each platform you scraped in this run.
3. Cleaned output files: Each raw file has a corresponding cleaned output file with populated title, popularity score, and URL fields.
4. Comparison output: The incremental comparison output exists and contains 🆕-annotated entries for topics that are new since the baseline.
5. Intelligence report: A single structured intelligence report file has been written, listing ranked monetization signals from all platforms.
6. WorkBuddy Skills (if used): Each Skill you invoked completed without error and produced its expected artifact in the WorkBuddy application.

If all five (or six) outcomes are met, the pipeline is fully operational.

Because the dokobot-intel-toolkit pipeline writes output files at each stage rather than modifying shared state, most issues can be resolved by correcting the failing step and re-running rather than performing a formal rollback. However, if a bad run has corrupted your baseline file (the reference snapshot used for incremental comparison), follow these steps:

1. Locate the baseline file written by the incremental comparison scripts.
2. Replace it with the previous day's baseline, if you have it archived. If you do not have an archive, delete the corrupted baseline file entirely — the next run will treat today's cleaned data as the new baseline, and incremental comparison will resume normally from tomorrow onward.
3. Do not attempt to hand-edit the baseline file to fix corruption; this risks introducing inconsistencies that will cause false 🆕 annotations in future comparisons.
4. Re-run the full pipeline (scrape → parse/clean → compare → report) to generate a fresh, clean set of output files.

If you need to undo a WorkBuddy Skill activation, deactivate the Skill inside the WorkBuddy application. This does not delete the Skill files from ~/.workbuddy/skills/.

If you have completed all applicable steps in this runbook and the pipeline is still failing, escalate with the following information to help the team diagnose the issue quickly:

Required information to include:

1. Bridge status: Output of python dokobot_ready.py (the full line, including whether it shows DEVICE_ID:<uuid> or NO_DEVICE).
2. Failing component: Which script or Skill is failing (e.g., the PowerShell scraping script for a specific platform, the Python parse script, the incremental comparison script, the report generation script, or a named WorkBuddy Skill).
3. Full error output: The complete error message and stack trace from the failing script or Skill, including any HTTP status codes if a scraping request failed.
4. Platform(s) affected: Which platform(s) are producing errors or empty output (e.g., X/Twitter, Reddit, Bilibili, Xiaohongshu, V2EX, Product Hunt, IndieHackers, YouTube).
5. Input file state: Whether the expected input files for the failing step were present and non-empty.
6. Environment details: Your Python version (python --version), Node.js version (node --version), and Windows/PowerShell version.
7. Reproducibility: Whether the error is consistent across multiple runs or intermittent.