Observability

This guide explains how to observe the health and runtime state of the dokobot-intel-toolkit pipeline through logging output, health checks, and status signals exposed by the toolkit's scripts. Because the pipeline depends on an active dokobot Edge bridge before any authenticated scraping can proceed, confirming bridge readiness is the foundational health check you perform before every run. Understanding what each script emits — exit codes, structured status strings, and annotated diff markers — lets you build reliable automation, catch failures early, and route alerts without guesswork.

Before following this guide, make sure you have:

• Python ≥ 3.11 — required to run dokobot_ready.py and the parse, clean, comparison, and report scripts
• Node.js ≥ 22 — required by the dokobot CLI runtime
• dokobot CLI — installed and accessible at the path expected by dokobot_ready.py
• Microsoft Edge — with the dokobot bridge extension installed on your default profile
• Windows OS with PowerShell — the scraping scripts are PowerShell-based and must be invoked from PowerShell or cmd, not Git Bash (Git Bash misroutes Windows paths and causes MODULE_NOT_FOUND errors in the dokobot CLI)
• A baseline output file from the previous day's scrape if you intend to observe incremental comparison output

No separate observability agent or sidecar is required. All health and status signals come from scripts already present in the toolkit. Follow these steps to confirm the observability surface is working correctly before your first pipeline run.

1. Verify Python is available:

python --version

Expected output: Python 3.11.x or higher.

2. Verify Node.js is available:

node --version

Expected output: v22.x.x or higher.

3. Confirm the dokobot CLI is reachable by running the bridge readiness check:

python dokobot_ready.py

If the bridge is active you will see a device ID (see Usage). If Node.js or the dokobot CLI path is wrong, the script catches the exception internally and falls through to NO_DEVICE — so a NO_DEVICE result at this stage usually indicates a path or installation problem rather than a bridge problem.

4. Confirm PowerShell script execution policy allows the scraping scripts to run:

Get-ExecutionPolicy

If the policy is Restricted, set it for the current user:

Set-ExecutionPolicy -Scope CurrentUser RemoteSigned

The toolkit's observability behavior is determined by a small set of runtime values. There is no centralized config file; the relevant settings are either hardcoded paths in dokobot_ready.py or implicit in how you invoke the pipeline scripts.

Why paths are hardcoded: dokobot_ready.py uses absolute Windows paths for node.exe and dokobot.js to avoid the Git Bash path-translation bug that maps /e/… to C:\e\…. If you move the Node.js or dokobot CLI installation, update both constants in the script before running the pipeline.

The primary observability action you take before every pipeline run is the bridge readiness check. You then interpret log output from each subsequent stage.

1. Bridge health check

Run dokobot_ready.py from PowerShell:

python dokobot_ready.py

Ready state — the Edge bridge is active:

DEVICE_ID:3f8a1c2d-4b5e-6789-abcd-ef0123456789

Exit code: 0. You can proceed to authenticated scraping.

Not-ready state — no bridge detected after all retries:

NO_DEVICE

Exit code: 1. Open Microsoft Edge with the dokobot bridge extension enabled and re-run the check before starting the scraping scripts.

2. Gating pipeline steps on bridge readiness

In a PowerShell pipeline wrapper, use the exit code to gate scraping:

python dokobot_ready.py
if ($LASTEXITCODE -ne 0) {
    Write-Error "Bridge not ready. Open Edge with the dokobot extension and retry."
    exit 1
}
# Proceed with scraping scripts

3. Reading parse and clean output

The Python parse and clean scripts write structured records — each containing a title, popularity score, and URL — to output files. Check these files after the clean stage to confirm encoding fixes and noise removal worked correctly before the comparison step.

4. Reading incremental comparison output

After the comparison scripts diff today's cleaned data against yesterday's baseline, newly trending topics are marked with 🆕. Scan for this marker to quickly identify what changed since your last run without reading the full report.

5. Reading the daily intelligence report

The summary and report generation scripts aggregate all platform signals into a single structured report. The report surfaces monetization signals — topics ranked by their potential to be repurposed for Xiaohongshu revenue. Review this report as the final observability artifact of each pipeline run.

Example 1 — Bridge ready, pipeline can proceed

python dokobot_ready.py

Expected output:

DEVICE_ID:a1b2c3d4-e5f6-7890-abcd-ef1234567890

Exit code: 0

The device ID confirms the dokobot Edge bridge is active. Copy or log this value if you want to correlate scrape sessions with specific bridge instances.

Example 2 — Bridge not active

python dokobot_ready.py

Expected output (after 3 retries × 10 s ≈ 30 s):

NO_DEVICE

Exit code: 1

The script attempted to launch Edge automatically via the microsoft-edge: protocol and retried three times. Since no device ID appeared, Edge either did not open or the dokobot bridge extension was not activated on the default profile.

Example 3 — Using exit code in a CI-style health gate

$result = python dokobot_ready.py
Write-Host "Bridge check: $result"
if ($LASTEXITCODE -ne 0) {
    Write-Error "Scraping skipped: bridge unavailable."
    exit 1
}
Write-Host "Bridge confirmed. Starting scrape pipeline..."

Expected output when bridge is active:

Bridge check: DEVICE_ID:a1b2c3d4-e5f6-7890-abcd-ef1234567890
Bridge confirmed. Starting scrape pipeline...

Expected output when bridge is absent:

Bridge check: NO_DEVICE
Scraping skipped: bridge unavailable.

Example 4 — Incremental comparison output with 🆕 markers

After running the comparison scripts against yesterday's baseline, your output file will contain entries like:

🆕 GPT-5 system card released | score: 9821 | https://example.com/gpt5
   Claude 3.7 benchmark results | score: 7204 | https://example.com/claude
🆕 Open-source Whisper v4 drops | score: 6891 | https://example.com/whisper4

Entries prefixed with 🆕 were not present in yesterday's baseline — these are your highest-priority monetization signal candidates for today's Xiaohongshu content.

Use the symptom → cause → fix format below for the most common observability failures.

Symptom: dokobot_ready.py prints NO_DEVICE immediately without waiting.

Likely cause: The NODE or DOKOBOT path constants in dokobot_ready.py point to a location that does not exist. The list_devices() function catches the resulting exception and returns None, causing the script to treat it as a missing device.

Fix: Open dokobot_ready.py and verify that the NODE constant points to your actual node.exe path and DOKOBOT points to the dokobot.js CLI entry point. Use absolute Windows backslash paths. Do not use Git Bash paths (e.g. /e/...) — they are misrouted by Git Bash and produce MODULE_NOT_FOUND.

Symptom: dokobot_ready.py waits ~30 seconds, then prints NO_DEVICE.

Likely cause: The Edge browser opened (or was already open) but the dokobot bridge extension is not active on the default profile, so the CLI cannot find a registered device.

Fix: Open Microsoft Edge, navigate to the extensions page, and confirm the dokobot bridge extension is installed and enabled on the default profile. Re-run dokobot_ready.py after enabling it.

Symptom: PowerShell scraping scripts fail with path errors or MODULE_NOT_FOUND when you run them from Git Bash.

Likely cause: Git Bash translates POSIX-style paths (e.g. /e/OpenClaw/...) into Windows paths incorrectly (e.g. C:\e\OpenClaw\...), breaking the Node.js module resolution used by the dokobot CLI.

Fix: Always run PowerShell scraping scripts and dokobot_ready.py from PowerShell or cmd.exe, never from Git Bash.

Symptom: The incremental comparison output shows no 🆕 markers even though you expect new trending topics.

Likely cause: The baseline file from yesterday is missing, stale, or the comparison script cannot locate it, so it has nothing to diff against — or all of today's topics also appeared in yesterday's baseline.

Fix: Confirm the baseline file exists and is named/pathed as the comparison scripts expect. If the baseline is from more than one day ago, the diff may suppress results that are technically new but not new relative to that older snapshot. Ensure your daily run saves a fresh baseline at the end of each successful pipeline execution.

Symptom: dokobot_ready.py exits with code 0 and prints a device ID, but the scraping scripts still fail to retrieve login-walled content.

Likely cause: The device ID confirms the bridge extension is registered, but your Edge session may be logged out of one or more target platforms (X, Reddit, etc.).

Fix: Open Microsoft Edge and manually verify you are logged in to the platforms you intend to scrape. The dokobot Edge bridge reuses your existing browser session — it cannot authenticate on your behalf if you are not already logged in.