Deployment Guide

This guide walks you through deploying the dokobot-intel-toolkit pipeline in a production environment. Because the toolkit relies on the dokobot Edge bridge — a Microsoft Edge browser extension that must run on a Windows host with an active browser session — containerised and serverless deployment models require specific architectural decisions to isolate the bridge host from stateless compute layers. By the end of this guide you will have a reproducible production setup, a validated bridge connection, and scheduled daily intelligence runs producing structured reports.

Before you begin, confirm the following are in place on your deployment host:

• Operating system: Windows (required — PowerShell scraping scripts and the dokobot Edge bridge extension are Windows-only)
• Microsoft Edge with the dokobot bridge extension installed and signed in to the platforms you want to scrape (X/Twitter, Reddit, etc.)
• dokobot CLI installed and accessible on PATH
• Python ≥ 3.11 (python --version must report 3.11.x or higher)
• Node.js ≥ 22 (node --version must report v22.x or higher)
• PowerShell (included with Windows; confirm with $PSVersionTable.PSVersion)
• WorkBuddy application — required only if you intend to activate WorkBuddy Skills (bili-subtitle, deck-studio, or xhs-commercial-growth)
• faster-whisper (Python package) — optional; install only if you need speech-recognition fallback in the bili-subtitle Skill for videos without subtitle tracks

Architecture note: Because authenticated scraping depends on an active Edge browser session, the Windows machine running Edge and the dokobot CLI is your bridge host. All other compute (report aggregation, comparison scripts, downstream consumers) can run elsewhere, but scrape commands must be issued from or proxied through the bridge host.

Step 1 — Clone the repository

On your Windows bridge host, clone the toolkit repository into a permanent working directory:

git clone https://github.com/your-org/dokobot-intel-toolkit.git C:\dokobot-intel-toolkit
cd C:\dokobot-intel-toolkit

Step 2 — Install Python dependencies

Create a virtual environment and install the required packages:

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

If you plan to use the bili-subtitle Skill's speech-recognition fallback, also install faster-whisper:

pip install faster-whisper

Step 3 — Install Node.js dependencies

npm install

Step 4 — Install the dokobot CLI

Follow the dokobot CLI installation instructions for your distribution, then confirm it is on your PATH:

dokobot --version

Step 5 — Install and activate the Edge bridge extension

1. Open Microsoft Edge.
2. Navigate to edge://extensions and load the dokobot bridge extension (unpacked or from the Edge Add-ons store, per your distribution).
3. Ensure the extension is enabled and its status indicator shows active.
4. Log in to every platform you intend to scrape (X/Twitter, Reddit, V2EX, Product Hunt, IndieHackers, Xiaohongshu, Bilibili, YouTube) inside the same Edge profile.

Step 6 — Verify the bridge is ready

Run the readiness check script. A successful response prints a device ID:

python dokobot_ready.py

Expected output on success:

DEVICE_ID:3f7a2c1d-84b0-4e2a-9c11-abc123def456

If you see NO_DEVICE, the bridge extension is not active — return to Step 5 before proceeding.

Step 7 — (Optional) Install WorkBuddy Skills

Skills live in ~/.workbuddy/skills/. Copy or symlink each skill directory you want to enable:

# Bilibili subtitle extractor
Copy-Item -Recurse .\skills\bili-subtitle $env:USERPROFILE\.workbuddy\skills\bili-subtitle

# PPT production agent
Copy-Item -Recurse .\skills\deck-studio $env:USERPROFILE\.workbuddy\skills\deck-studio

# Xiaohongshu commercial-growth framework
Copy-Item -Recurse .\skills\xhs-commercial-growth $env:USERPROFILE\.workbuddy\skills\xhs-commercial-growth

Then open the WorkBuddy application and enable each skill from the Skills panel.

All runtime configuration is supplied through environment variables or the values files described below. There is no single monolithic config file — each pipeline stage reads the variables relevant to it.

Scheduling

For daily production runs, use Windows Task Scheduler to invoke the pipeline wrapper at a consistent time (e.g. 06:00 local time). Ensure the scheduled task runs under the same Windows user account that owns the active Edge session and the dokobot CLI installation, because the bridge is session-scoped.

# Example: register a daily task at 06:00
$action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument '-File C:\dokobot-intel-toolkit\run_pipeline.ps1'
$trigger = New-ScheduledTaskTrigger -Daily -At '06:00AM'
Register-ScheduledTask -TaskName 'DokobotIntelDaily' -Action $action -Trigger $trigger -RunLevel Highest

Why RunLevel Highest? The dokobot CLI needs permission to attach to the Edge browser process. Running the task at highest privilege prevents UAC prompts from blocking overnight runs.

Running the full pipeline

The standard production workflow has five sequential stages. Run them in order each day:

1. Check bridge readiness

Always start here. If the bridge is not active, the authenticated scraping stage will fail silently or return incomplete data.

python dokobot_ready.py

Proceed only if the output begins with DEVICE_ID:.

2. Scrape all platforms

Invoke the PowerShell scraping scripts. Each script is rate-limit-aware and writes a raw output file to $DOKOBOT_OUTPUT_DIR.

.\scripts\scrape_all.ps1

Or scrape individual platforms during testing:

.\scripts\scrape_x.ps1
.\scripts\scrape_reddit.ps1
.\scripts\scrape_v2ex.ps1

3. Parse and clean raw data

Run the Python parse and clean scripts to fix encoding, strip noise, and extract structured fields (title, popularity score, URL) from each platform's raw file:

python scripts\clean_all.py

4. Run incremental comparison

Diff today's cleaned data against yesterday's baseline. Newly trending topics are annotated with 🆕 in the output:

python scripts\compare_incremental.py

5. Generate the intelligence report

Aggregate signals from all platforms into a single structured report highlighting the best monetization opportunities for Xiaohongshu:

python scripts\generate_report.py

The report is written to $DOKOBOT_REPORT_DIR with a datestamped filename.

Using WorkBuddy Skills

Skills are activated inside the WorkBuddy application after being installed to ~/.workbuddy/skills/. Each skill is independent — you do not need to run the full pipeline to use them.

• bili-subtitle: Provide a Bilibili BV number; the skill returns the full transcript using the official subtitle track, or faster-whisper if no track is available.
• deck-studio: Provide a topic brief; the skill recommends a slide style, builds an outline, and delivers content drafts or visual slide descriptions.
• xhs-commercial-growth: Provide a trending topic; the skill evaluates it against the three-currency framework (attention, trust, purchase intent) and the commercial-distance concept to score its monetization potential.

Example 1 — Verify bridge before a production run

Confirm the dokobot Edge bridge is active and capture the device ID for logging:

$result = python dokobot_ready.py
Write-Host "Bridge status: $result"
if ($result -notmatch '^DEVICE_ID:') {
    Write-Error 'Bridge not ready. Open Edge and activate the dokobot extension.'
    exit 1
}

Expected output:

Bridge status: DEVICE_ID:3f7a2c1d-84b0-4e2a-9c11-abc123def456

Example 2 — Run the complete daily pipeline

A minimal run_pipeline.ps1 that chains all five stages and exits on any failure:

$ErrorActionPreference = 'Stop'

Write-Host '[1/5] Checking bridge readiness...'
$device = python dokobot_ready.py
if ($device -notmatch '^DEVICE_ID:') { throw 'Bridge not active.' }

Write-Host '[2/5] Scraping platforms...'
.\scripts\scrape_all.ps1

Write-Host '[3/5] Parsing and cleaning raw data...'
python scripts\clean_all.py

Write-Host '[4/5] Running incremental comparison...'
python scripts\compare_incremental.py

Write-Host '[5/5] Generating intelligence report...'
python scripts\generate_report.py

Write-Host 'Pipeline complete.'

Expected tail output:

[5/5] Generating intelligence report...
Report written to C:\dokobot-intel-toolkit\output\reports\intel_2025-07-14.md
Pipeline complete.

Example 3 — Incremental comparison output showing new signals

After compare_incremental.py runs, the annotated diff might look like:

Platform: X/Twitter
  [existing]  GPT-5 benchmark discussion        score: 9420
  🆕 [new]    Claude 4 system prompt leak       score: 7810
  🆕 [new]    AI coding agent replaces intern   score: 6540

Platform: Product Hunt
  [existing]  Open-source Notion alternative    score: 3200
  🆕 [new]    AI video dubbing tool #1 today    score: 2980

Example 4 — Extract a Bilibili transcript using the bili-subtitle Skill

Inside the WorkBuddy application with the bili-subtitle skill enabled, submit a task with a BV number:

Input: BV1xx411c7mD

Expected skill output (truncated):

Transcript source: official subtitle track
Language: zh-CN

[00:00:05] 今天我们来聊一下 AI 编程助手的发展趋势...
[00:00:18] 首先看一下 GitHub Copilot 最新的数据...
...

If no subtitle track exists, the output header changes to Transcript source: faster-whisper (local ASR) and the content is generated from speech recognition.

Issue: dokobot_ready.py prints NO_DEVICE

Symptom: Running python dokobot_ready.py returns NO_DEVICE instead of a DEVICE_ID: string.

Likely cause: The dokobot bridge extension is not active in Microsoft Edge, or Edge is not open.

Fix:

1. Open Microsoft Edge.
2. Navigate to edge://extensions and confirm the dokobot bridge extension is toggled on.
3. If the extension is listed but disabled, enable it and wait 5–10 seconds.
4. Re-run python dokobot_ready.py.

Issue: PowerShell scraping scripts return empty or partial output files

Symptom: Raw output files in $DOKOBOT_OUTPUT_DIR are empty or contain far fewer records than expected.

Likely cause A: The bridge was active at check time but the Edge session expired or was interrupted mid-scrape.

Fix A: Re-run python dokobot_ready.py to confirm the bridge is still live, then re-run the affected scraping script.

Likely cause B: The platform rate-limited the requests because DOKOBOT_RATE_LIMIT_MS is too low for current conditions.

Fix B: Increase DOKOBOT_RATE_LIMIT_MS (e.g. from 2000 to 5000) in your environment and re-run.

Issue: compare_incremental.py reports no new topics (no 🆕 entries)

Symptom: The incremental comparison output contains no 🆕 annotations even after a fresh scrape.

Likely cause: The baseline directory ($DOKOBOT_BASELINE_DIR) is missing yesterday's snapshot, so the comparison has no reference point and treats all topics as unchanged or skips the diff.

Fix: Confirm the baseline file from the previous run exists in $DOKOBOT_BASELINE_DIR. If the directory was wiped (e.g. by a redeployment that did not use a persistent volume), the first run after the wipe will always produce an empty diff. Allow one full day's cycle to rebuild the baseline.

Issue: bili-subtitle Skill returns an error about missing faster-whisper

Symptom: The skill fails with a message indicating the speech-recognition fallback is unavailable.

Likely cause: The Bilibili video has no official subtitle track and faster-whisper is not installed in the Python environment.

Fix: Activate the same virtual environment used by the toolkit and install faster-whisper:

.venv\Scripts\Activate.ps1
pip install faster-whisper

Then retry the skill task in WorkBuddy.

Issue: Scheduled Task Scheduler job fails silently overnight

Symptom: The Windows Task Scheduler reports the task ran successfully, but no new report file appears in $DOKOBOT_REPORT_DIR.

Likely cause: The scheduled task ran under a different user account or session than the one owning the active Edge bridge, so dokobot_ready.py returned NO_DEVICE and the pipeline exited early without logging visibly.

Fix:

1. Open Task Scheduler and confirm the task's "Run As" user matches the account with the active Edge session.
2. Add explicit logging to run_pipeline.ps1 (redirect output to a log file with *>> C:\dokobot-intel-toolkit\logs\run.log).
3. Ensure "Run only when user is logged on" is selected if the bridge requires an interactive session.

Issue: Python parse and clean scripts raise encoding errors

Symptom: clean_all.py crashes with a UnicodeDecodeError when processing raw files from certain platforms.

Likely cause: Platforms such as V2EX or Xiaohongshu return content with mixed or non-UTF-8 encodings that were not normalised during scraping.

Fix: The clean scripts are designed to handle encoding repair. Ensure you are running Python ≥ 3.11 (python --version) — earlier versions have different codec fallback behaviour. If the error persists, check that the raw output file is not zero-length (which would indicate a scrape failure rather than an encoding issue).