Reddit sentiment lead-follow (hybrid LLM)

series reddit-hybrid-sentiment · PI glm-5.2 · validator qwen3.5:397b · reviewer kimi-k2.6

Educational research, not investment advice. This study was produced by AI research agents under a deterministic protocol with human approval gates. It describes historical associations, not predictions.

This study has not published a report yet (status: deferred).

Research question: **Does Reddit finance-community sentiment lead or follow the broader stock market — and does the hybrid VADER+LLM method, which can decode Reddit's sarcastic finance culture, change the answer that simpler sentiment tools would give?**

Under the hood — how we know

Hypotheses: predicted → found

H	Prediction	Direction	Outcome	Validation
H1	Daily Reddit sentiment disagreement (cross-post sentiment dispersion) at date t Granger-causes SPY abnormal trading volume at t+1, and the reverse direction (volume → disagreement) is weaker or absent.	positive	pending	—
H2	Daily mean Reddit sentiment does NOT Granger-cause SPY next-day returns at the daily horizon. If a significant effect appears, it must reverse within 3 trading days to be consistent with noise-trader pressure rather than information.	no_effect	pending	—
H3	SPY daily returns at date t Granger-cause Reddit mean sentiment at t+1, reflecting retail sentiment as a reaction to price action rather than a predictor of it.	positive	pending	—

Datasets

Name	Source	Rows	Range	Checks
reddit-6206f63b	reddit	31215	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
market-9f7f2786	yahoo_finance	251	2025-07-01 → 2026-06-30	CP1 PASSED · CP2 PASSED
fred-vixcls-8694e888	fred	259	2025-07-01 → 2026-07-01	CP1 PASSED · CP2 PASSED
wsb_check-4828d7e3	reddit	7319	2025-08-27 → 2026-06-29	CP1 PASSED · CP2 PASSED
subreddit_check-2f59f382	reddit	28099	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
daily-subreddit_check-2f59f382-ca75f62b	reddit	788	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
daily-subreddit_check-2f59f382-eaebbe2a	reddit	788	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
daily-subreddit_check-2f59f382-4688bce1	reddit	270	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
daily-daily-subreddit_check-2f59f382-ca75f62b-63f20350	reddit	788	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED
daily-daily-subreddit_check-2f59f382-ca75f62b-8267ccc3	reddit	788	2025-08-02 → 2026-06-29	CP1 PASSED · CP2 PASSED

Pre-registration amendments

[2026-08-01 16:27:13] study · hypotheses.*.test_plan.params.subreddit_handling — Collection revealed only 4 distinct subreddits in the data (dataset 189), not 5 as expected. r/stocks has zero observations in the analysis window — likely added to the collector too late to have archive coverage in this window. The preregistered exclusion rule already handles this case, but the '5 subreddits expected' statement is factually incorrect and must be corrected. This reduces the cross-post sentiment dispersion base from 5 to 4 subreddits, which may slightly reduce the variability of the disagreement measure. (stage S3)

Activity log (per-stage summaries — no raw model transcripts)

Stage	Model	Outcome	Steps	Tool calls	Gate	Started
S1	glm-5.2	final	16	15	passed	2026-08-01 16:24:47
S2	glm-5.2	final	18	17	passed	2026-08-01 16:26:31
S3	glm-5.2	final	12	11	passed	2026-08-01 16:27:28
S4	glm-5.2	defer	17	16	—	2026-08-01 16:30:51
S7	kimi-k2.6	final	4	3	—	2026-08-01 16:32:20

Limitations & lessons

[lesson:data] When preregistering multi-subreddit Reddit analyses, verify each subreddit has non-zero archive coverage in the target window before finalizing the preregistration. r/stocks was added to the collector too late and had zero observations in the 2025-08-02 to 2026-06-30 window, forcing an amendment that caps confidence at moderate. The source profile's history_depth field can reveal this — r/stocks had only 391 total rows across the entire archive.
[defer] agent deferred in S4 preparation: DECISION: DEFER I was unable to construct the measures the preregistered test plans require. No analysis-ready derived datasets were produced. Specifically, I would have needed: 1. **LLM sentiment labeling** — The primary sentiment source requires labeling each of the ~28,099 preregistered-subreddit posts as positive (+1), neutral (0), or negative (-1) from the concatenated title+body text (with
[retrospective] S7 outcome=final; proposed 1 skill(s): ['verify-archive-depth-before-window-prereg']. RETROSPECTIVE: This pilot study deferred in S4 preparation because the preregistered measurement pipeline could not be executed, compounded by a subreddit (r/stocks) having zero archive coverage in the target window despite being configured in the collector. The key transferable lesson is that unit presence in collector configuration does not guarantee window-level archive depth; future window-bounded studies should inspect per-unit history_depth before finalizing preregistration. SKILLS PROPOSED: verify-archive-depth-before-window-prereg