Archive & Sync

Sync

GitHub Edit page

Sync

birdclaw sync mirrors the live Twitter surfaces you actually use into the local SQLite store. Every sync command:

  • pulls from xurl first (or bird when forced via --mode bird)
  • writes into the same canonical tables that archive import uses
  • refreshes the FTS5 index incrementally
  • saves cursors so the next run resumes where the last one stopped
  • caches results so repeat reads do not keep spending the API budget

#Common flags

All sync * commands accept:

  • --mode auto|xurl|bird — transport selection; auto tries xurl then falls back to bird
  • --limit <n> — page size in xurl mode, total in single-page modes
  • --all — keep paginating until the retrievable window is exhausted
  • --max-pages <n> — cap a paged scan; implies --all
  • --refresh — bypass the cache and force a live fetch
  • --cache-ttl <seconds> — tune freshness without forcing a full refresh
  • --since <cursor-or-id> — resume from a known cursor or tweet ID
  • --transport <kind> — alias for --mode on some subcommands
  • --dry-run — read but do not write
  • --json — stable machine-readable output

#sync likes

Mirror the authenticated user's Likes feed:

4 sync likes 0 auto 1 100 2 3
5 sync likes 0 6 1 2 5 3 4

Liked tweets land in the same tweets table as archive imports and can be queried with birdclaw search tweets --liked.

#sync bookmarks

Mirror Bookmarks:

4 sync bookmarks 0 auto 1 100 2 3
6 sync bookmarks 0 7 1 2 5 3 100 4 5

Bookmarks are queried via birdclaw search tweets --bookmarked and drive the research workflow.

#sync timeline

Pull the chronological Following timeline through bird:

3 sync timeline 0 100 1 2

sync timeline defaults to the chronological feed, not the algorithmic For You. The home timeline is stored in the same tweets table so search, filters, and the web UI's Home lane all see one set of rows.

#sync mention-threads

Fetch conversation context for recent mentions through bird thread:

4 sync mention-threads 0 30 1 1500 2 15000 3

Extra flags:

  • --delay-ms <ms> — delay between thread fetches; raise this when X starts rate-limiting
  • --timeout-ms <ms> — per-thread network timeout

This is the gentlest sync command on purpose. It walks back up the reply chain so the web UI can render quoted ancestors without a separate live call later.

#sync followers / following

Followers and following are first-class entities with append-only history. Both syncs record current state plus a follow_events row for every change.

3 sync followers 0 auto 1 2
3 sync following 0 auto 1 2

After the first run, birdclaw graph events shows the diff log and birdclaw graph mutuals lists current mutuals.

#sync all

1 sync all 0 2
1 sync all 0 auto

sync all runs every individual sync in a sane order (likes → bookmarks → timeline → mention-threads → followers → following). It is resumable and rate-limit-aware: if Twitter slows you down, it persists the cursor and exits with code 5 (partial sync) so a scheduler can retry.

#DMs sync

DMs sit on a separate command because they need bird for full-content reads:

3 dms sync 0 50 1 2
3 dms list 0 1 10 2

See DMs for the full triage workflow.

#Mentions

birdclaw mentions export is both a live cache fetcher and an agent-friendly export format. See Mentions.

#Caching model

Every cached live mode (--mode bird or --mode xurl) stores the response in SQLite alongside the canonical normalized rows. Subsequent reads return from cache until the TTL elapses or you pass --refresh.

Cache rules:

  • the canonical store is always the source of truth for filters and search
  • the response cache is what --mode xurl returns for xurl-shape compatibility
  • --refresh purges the response cache for that surface and refetches
  • --cache-ttl <seconds> overrides the default freshness window
  • write commands invalidate any read cache that overlaps the write

This is what lets birdclaw mentions export --mode xurl mirror the xurl mentions JSON shape without re-hitting the live API every time.

#Exit codes

  • 0 — success
  • 4 — transport unavailable (e.g. xurl not installed and --mode xurl)
  • 5 — partial sync; resume with --since <cursor> or just re-run

See also: CLI reference for sync for the canonical flag list.