Key Concepts

This page defines every core concept you need to understand before working with dokobot-intel-toolkit. Whether you are setting up authenticated scraping for the first time, interpreting pipeline output, or deciding which WorkBuddy Skill to enable, the terms defined here form the shared vocabulary used throughout the rest of the documentation.

dokobot Edge bridge

The dokobot Edge bridge (also referred to as the Edge bridge extension, dokobot CLI, or simply the bridge) is the combination of a Microsoft Edge browser extension and the dokobot CLI runtime. Together they allow the scraping pipeline to reuse your existing, already-logged-in browser sessions. This means the toolkit can reach platforms like X (Twitter) and Reddit — which block anonymous access — without you ever storing credentials in a config file. The bridge acts as a relay: when a scraping script needs authenticated content, it sends the request through the extension, which fulfills it using whatever session your browser already holds.

device ID

The device ID (also written DEVICE_ID or uuid) is a unique identifier that the Edge bridge emits once it is active. When you run dokobot_ready.py, the script reads this value and prints it in the format DEVICE_ID:<uuid>. Seeing that output tells you that the bridge is live and ready to accept authenticated scraping requests. If the bridge is not active, dokobot_ready.py prints NO_DEVICE instead. Because every authenticated scrape depends on the bridge, checking for a valid device ID is the mandatory first step of every pipeline run.

NO_DEVICE

NO_DEVICE is the specific output printed by dokobot_ready.py when it cannot detect an active Edge bridge extension. Receiving this output means you must open Microsoft Edge and activate the dokobot bridge extension before any authenticated scraping can proceed. No other pipeline step should be started while NO_DEVICE is the status.

authenticated scraping

Authenticated scraping (also called login-state scraping or logged-in scraping) is the process of fetching web content while the toolkit is signed in as you. Platforms such as X and Reddit present login walls to anonymous requests; authenticated scraping bypasses those walls by routing requests through the Edge bridge, which carries your real session cookies. The PowerShell scraping scripts perform all platform fetches in this mode, operating read-only: they never post, reply, or interact with any platform on your behalf.

monetization signal

A monetization signal (also: signal, intel signal, or 变现信号) is a trending topic or piece of content that the pipeline has identified as having strong potential to be repurposed into revenue-generating posts on Xiaohongshu (Little Red Book). Signals are surfaced by the summary and report generation scripts after cleaning and cross-day comparison. The toolkit is opinionated about this goal: its entire pipeline — from scraping through ranking — is designed to highlight these signals rather than simply collect raw data.

incremental comparison

Incremental comparison (also: cross-day comparison, baseline diff, or 增量对比) is the daily diff step that compares today's cleaned topic data against the previous day's saved baseline. Topics that appear for the first time are annotated with 🆕, giving you a focused, change-only view of what is newly trending. This step is what separates a daily intelligence workflow from a simple data dump: by highlighting only the new signals, it reduces the manual review burden significantly.

baseline

The baseline (also: yesterday's data or reference snapshot) is the saved output from the previous day's pipeline run. It serves as the reference point during incremental comparison. Without a baseline, the comparison scripts have no prior state to diff against, so no 🆕 annotations can be generated. Baselines are stored locally and are excluded from version control because they are large, time-sensitive, and may contain commercially sensitive signals.

WorkBuddy Skill

A WorkBuddy Skill (also: Skill or skill package) is a self-contained capability package that you activate inside the WorkBuddy application. Skills are stored in ~/.workbuddy/skills/ and are disabled by default (disable: true in their frontmatter); you enable only the ones you need. Each Skill provides a specific, bounded capability — such as extracting a Bilibili transcript or generating a slide deck — independently of the rest of the toolkit. The three Skills shipped with dokobot-intel-toolkit are bili-subtitle, deck-studio, and xhs-commercial-growth.

bili-subtitle Skill

The bili-subtitle Skill (also: bili-subtitle or Bilibili subtitle extractor) is a WorkBuddy Skill that extracts a full transcript from any Bilibili video when you supply its BV number. It first attempts to retrieve the official subtitle track (either an uploader-provided track or an AI-generated one); if no subtitle track exists, it falls back to local faster-whisper speech recognition. The subtitle path is fast and free; the speech-recognition fallback requires faster-whisper to be installed.

deck-studio Skill

The deck-studio Skill (also: deck-studio or PPT agent) is a WorkBuddy Skill that acts as a PPT production agent (v3.1.0). You provide a topic or brief, and the Skill walks through scene understanding, style recommendation, outline generation, per-page definition, and dual-path delivery — producing either a content draft in Markdown/JSON or per-page visual image prompts. Its core principle is that a presentation's purpose is information transfer, and design serves that goal rather than the other way around.

xhs-commercial-growth Skill

The xhs-commercial-growth Skill (also: xhs-commercial-growth or commercial growth theory) is a WorkBuddy Skill that encodes a structured theory of Xiaohongshu monetization. Its central model is summarized as three currencies, two systems, and one commercial distance:

• Three currencies: attention, trust, and purchase intent.
• Two systems: the virality system (which produces attention) and the monetization system (which converts trust and demand into transactions).
• Commercial distance: the gap between why a user watches a video and why they would pay for a product.

The Skill helps you evaluate how monetizable a trending topic actually is before you invest in creating content around it.

faster-whisper

faster-whisper (also: whisper or local ASR) is an optional local speech-recognition library used by the bili-subtitle Skill as a fallback when a Bilibili video has no subtitle track. It runs entirely on your local machine (CPU, int8 mode) and does not require any external API call. Installing faster-whisper is only necessary if you expect to process videos that lack official subtitle tracks.

dokobot CLI

The dokobot CLI is the authenticated browser automation runtime that underlies the Edge bridge. It relays scrape requests from the PowerShell scraping scripts through the Edge bridge extension, enabling those scripts to access login-walled platforms on your behalf. Node.js ≥ 22 is required for the dokobot CLI to run.

Check bridge readiness before scraping

Run dokobot_ready.py to confirm the Edge bridge is active. All authenticated scraping depends on a valid device ID.

# Run from Windows PowerShell
python scripts/dokobot_ready.py

Expected output when the bridge is active:

DEVICE_ID:3f8a1c2d-7e44-4b09-a12f-9d0e6b3c5f71

Expected output when the bridge is not active:

NO_DEVICE

If you see NO_DEVICE, open Microsoft Edge and activate the dokobot bridge extension before proceeding.

Run an authenticated scrape

With a confirmed device ID, execute a PowerShell scraping script. The example below scrapes X (Twitter) search threads in serial mode with rate-limit protection.

powershell -ExecutionPolicy Bypass -File scripts/_catch_x2.ps1

The script saves raw output files to disk. Those files become the input for the parse and clean stage.

Parse, clean, and compare against yesterday's baseline

After scraping, run the parse and incremental comparison scripts to produce an annotated list of newly trending topics.

python scripts/parse_linuxdo.py
python scripts/compare_linuxdo.py

Expected output (excerpt):

🆕 [New] DeepSeek R2 benchmark discussion — score: 847
🆕 [New] Open-source AIGC video tools roundup — score: 612
    [Existing] Claude prompt caching patterns — score: 590

Topics marked 🆕 are newly trending relative to yesterday's baseline and are the strongest candidates for monetization signals.

Enable and invoke the bili-subtitle Skill

Skills are disabled by default. To use bili-subtitle, set disable: false in its frontmatter inside ~/.workbuddy/skills/bili-subtitle/, then open WorkBuddy and activate the Skill. Once active, provide a BV number to extract a transcript.

The Skill first attempts the subtitle API path (fast, no model required). If the video has no subtitle track, it falls back to faster-whisper automatically.

Input supplied to the Skill:

BV1xx411c7mD

Expected output (subtitle path):

subtitle_zh-CN.txt written — 1,243 lines extracted from subtitle track.

Expected output (faster-whisper fallback):

NO_SUBTITLE returned by subtitle API.
Falling back to faster-whisper transcription…
transcript.txt written — 58 segments, ~12 minutes of audio processed.

• Quick Start — Step-by-step guide to running your first full pipeline: bridge check → scrape → clean → compare → report.
• Pipeline Overview — How the five stages (ready check, scrape, parse and clean, incremental comparison, report generation) connect and what files flow between them.
• PowerShell Scraping Scripts Reference — Full flag reference and platform coverage for _catch_web.ps1, _catch_x2.ps1, _catch_extra.ps1, and related scripts.
• Python Parse & Clean Scripts Reference — Encoding handling, field extraction schema, and deduplication logic for the cleaning stage.
• WorkBuddy Skills Guide — How to install, enable, and configure bili-subtitle, deck-studio, and xhs-commercial-growth inside WorkBuddy.
• Incremental Comparison Scripts Reference — Baseline format, diff algorithm, and 🆕 annotation rules.
• Summary & Report Generation Reference — Output schema for the daily intelligence report and how monetization signals are ranked.