# Overview

Relevant source files

- [.claude-plugin/marketplace.json](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/.claude-plugin/marketplace.json)
- [.claude-plugin/plugin.json](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/.claude-plugin/plugin.json)
- [CHANGELOG.md](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/CHANGELOG.md?plain=1)
- [README.md](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/README.md?plain=1)
- [SKILL.md](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/SKILL.md?plain=1)
- [release-notes.md](https://github.com/mvanhorn/last30days-skill/blob/45f596ca/release-notes.md?plain=1)

## Purpose and Scope

The `/last30days` skill is a multi-source research aggregation system that collects, normalizes, scores, and synthesizes content from the last 30 days across over 10 platforms, including Reddit, X/Twitter, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, and the web. The system executes an intelligent research pipeline—moving from intent planning and entity resolution to parallel data acquisition and AI-driven synthesis.

This page provides a high-level introduction to the system's purpose, architecture, and key features. For detailed information on specific subsystems:

- Intelligent planning and two-phase strategy: see **Core Architecture** (Section 2)
- Individual data source connectors: see **Data Sources** (Section 3)
- Normalization and scoring logic: see **Data Processing Pipeline** (Section 4)
- Rendering and auto-save features: see **Output Layer** (Section 5)

* * *

## System Purpose

The system addresses the "recency gap" in Large Language Models (LLMs) by providing grounded, real-time social signals. Unlike traditional search engines that prioritize SEO, `/last30days` prioritizes **human engagement**:

1. **Temporal Recency**: Specifically targets the last 30 days of activity.
2. **Social Signal Quality**: Results are ranked by upvotes, likes, views, and real-money betting volume rather than editorial placement.
3. **Cross-Platform Synthesis**: An AI "Judge" identifies narrative clusters where stories overlap across disconnected "walled gardens" like X and Reddit.
4. **Zero-Configuration Discovery**: Features a first-run setup wizard to unlock premium sources in seconds.

**Sources:**

* * *

## Core Functionality

### Primary Use Cases

The system is optimized for high-velocity domains where community sentiment moves faster than news cycles:

| Use Case            | Query Pattern          | Key Signals |
|---------------------|------------------------|-------------|
| **Tool Comparisons** | `OpenClaw vs Hermes`   | GitHub stars, Reddit debates, YouTube reviews |
| **Pre-Meeting Intel** | `Peter Steinberger`     | Recent PRs, X threads, sentiment analysis |
| **Market Sentiment** | `Anthropic odds`       | Polymarket betting volume, HN technical consensus |
| **Trend Discovery**  | `AI video tools`       | TikTok views, Instagram Reels transcripts |

### High-Level Execution Flow

```
Phase 3: Output

Phase 2: Processing

Phase 1: Parallel Acquisition

Phase 0: Planning & Resolution

User Invocation
/last30days TOPIC

Planner (Python Brain)
Intent Classification

resolve.py
Handle/Subreddit Resolution

Reddit
(ScrapeCreators)

X/Twitter
(Bird/xAI)

YouTube
(yt-dlp)

TikTok/IG
(ScrapeCreators)

Web/Perplexity
(OpenRouter)

fusion.py
Weighted RRF

cluster.py
Entity Overlap

rerank.py
LLM Reranking

render.py
Markdown/JSON

~/Documents/Last30Days/
Auto-save .md
```

**Sources:**

* * *

## Key Features

### Intelligent Pre-Research (v3 Engine)

The v3 architecture introduces a "Python brain" that resolves entities before main search execution. It identifies X handles, subreddits, and YouTube channels associated with a topic to perform targeted deep-dives rather than just keyword matching.

### Multi-Source Social Pipeline

The system aggregates data from a diverse array of "walled gardens":

- **Reddit**: Keyword search and subreddit discovery via ScrapeCreators.
- **X (Twitter)**: Multi-modal search via Bird CLI (browser cookies) or xAI API.
- **Prediction Markets**: Real-money odds from Polymarket.
- **Developer Signals**: GitHub PR velocity, stars, and issue discussions.
- **Video Content**: Full transcript extraction from YouTube, TikTok, and Instagram.

### Automated Knowledge Library

Every research run is automatically slugified and saved as a Markdown file in `~/Documents/Last30Days/`. This allows users to build a persistent, searchable research library over time without manual intervention.

* * *

## System Architecture Mapping

The following diagram maps high-level system components to their concrete implementations in the codebase:

```
Output & Persistence

Data Processing

Source Connectors

Intelligence Layer

Entry & Orchestration

SKILL.md
Runtime Spec

scripts/last30days.py
main(), run_research()

scripts/lib/pipeline.py
ResearchPipeline Class

scripts/lib/planner.py
ResearchPlanner

scripts/lib/resolve.py
EntityResolver

scripts/lib/reddit.py

scripts/lib/bird_x.py

scripts/lib/polymarket.py

scripts/lib/instagram.py

scripts/lib/normalize.py

scripts/lib/signals.py

scripts/lib/fusion.py

scripts/lib/dedupe.py

scripts/lib/render.py

scripts/lib/ui.py

scripts/lib/store.py
SQLite Persistence
```

**Sources:**

* * *

## Installation and Setup

The system is designed for "zero config" but offers a setup wizard to unlock authenticated sources.

### Quick Start

1. **Claude Code**: `/plugin marketplace add mvanhorn/last30days-skill`
2. **OpenClaw**: `clawhub install last30days-official`
3. **Manual**: Clone the repository and run the setup wizard.

### Setup Wizard

On first run, the system detects missing dependencies and API keys. It prioritizes the **ScrapeCreators API Key**, which provides unified access to Reddit, TikTok, and Instagram.

| Platform             | Key / Method                 | Benefit |
|----------------------|-------------------------------|-------------|
| **Core**             | `SCRAPECREATORS_API_KEY`    | Reddit, TikTok, Instagram |
| **X/Twitter**       | `XAI_API_KEY` or Cookies     | Real-time "hot takes" |
| **Web**             | `BRAVE_API_KEY` / `PARALLEL_API_KEY` | Grounded editorial context |
| **AI Reasoning**    | `OPENROUTER_API_KEY`        | Powers Planning & Perplexity |

**Sources:**