Reddit sentiment lead-follow dynamics with market prices

Hypotheses: predicted → found

Datasets

Pre-registration amendments

• [2026-07-31 15:03:47] H1 · test_plan.params.independent — Collection of all 5 preregistered subreddits returned only 4 distinct subreddit values (dataset_id=52). The Reddit data source profile noted r/stocks was 'added late' with only 391 rows across the full archive; it appears to have zero posts in our specific collection window. The attention metric is computed the same way (sum of post count + ticker mentions, z-scored) but across 4 subreddits instead of 5, slightly reducing coverage of the retail-investor attention proxy. (stage S3)
• [2026-07-31 15:03:51] H3 · test_plan.params.forward_path_specification — Same deviation as H1 amendment: only 4 of 5 preregistered subreddits present in collected data (dataset_id=52). r/stocks has no posts in the 2025-08-25 to 2026-06-29 window. (stage S3)

Activity log (per-stage summaries — no raw model transcripts)

Limitations & lessons

• [limitation] Only 4 of 5 preregistered subreddits are present in the collected Reddit data (dataset_id=52). r/stocks has zero posts in the 2025-08-25 to 2026-06-29 collection window. The attention metric (H1, H3) is computed across wallstreetbets, stockmarket, investing, and economy only, slightly reducing the breadth of the retail-investor attention proxy. Confidence capped at 'moderate' per amendment.
• [limitation] Ticker mentions are extremely sparse: only 4,024 of 29,359 posts (13.7%) have non-null ticker data (dataset_id=52). The H1/H3 attention metric combines total_post_count + total_ticker_mentions, so the ticker-mention component contributes to only ~14% of posts. The attention metric is dominated by post volume rather than ticker specificity, which may weaken the link to individual equity volatility.
• [limitation] Reddit post content (body text) is non-null for only 17,636 of 29,359 posts (60.1%) in dataset_id=52. The remaining 39.9% are link posts with empty bodies. For H2 sentiment analysis, the preregistered title-fallback rule (per preregister-reddit-sentiment-title-fallback) applies, but VADER performance on short titles may differ from body text. A sensitivity analysis comparing title+body vs title-only sentiment is preregistered.
• [limitation] The Reddit archive has ~60 gap days (249 distinct source_dates over ~309 calendar days from 2025-08-25 to 2026-06-29 in dataset_id=52). The preregistration designates 2025-08-27 to 2026-02-21 as the primary 'clean continuous block' (~125 trading days) and 2026-02-22 to 2026-06-30 as the sensitivity window ('heavily gapped'). Gap clustering within the primary window must be verified in S4 before constructing lagged pairs, as clustered gaps disproportionately reduce effective sample size for consecutive-day designs (per audit-gap-clustering-for-lagged-designs and audit-timeseries-gaps-for-lag-designs).
• [limitation] Reddit scores and num_comments are frozen at daily scrape time, not final values (dataset_id=52 caveat). These are not used as outcome variables in any hypothesis, but if any auxiliary analysis considers engagement metrics, score-accrual leakage applies.
• [lesson:data] When preregistering Reddit attention metrics across multiple subreddits, verify each subreddit has posts in the specific collection window before finalizing. Late-added subreddits (e.g., r/stocks with only 391 rows across the full archive) may have zero posts in the study window, silently reducing the attention proxy from N to N-1 subreddits. This triggers an amendment and caps confidence at 'moderate'.
• [limitation] LLM sentiment audit (audit_sentiment_subsample) could not be completed: labeler 'kimi-k3' returned HTTP 402 (extra usage balance empty, not included in plan). The VADER sentiment labels (dataset 54) are therefore UNVALIDATED by LLM cross-validation. Per the preregistration, H2 requires LLM cross-validation with concordance >= 0.5 to treat the sentiment construct as stable. H2 is BLOCKED from confirmatory claims until the audit can be run. H1 and H3 depend on attention (post volume + ticker mentions), not sentiment, and are unaffected by this failure.
• [limitation] The daily bullish-minus-bearish sentiment ratio (dataset 60) was computed as (pos_count - neg_count) / (pos_count + neg_count) instead of the preregistered (pos_count - neg_count) / (pos_count + neg_count + 1). The '+1' smoothing constant could not be added via the available column-level transform ops (which only support column-column arithmetic, not column-scalar). The practical difference is <1% for typical daily post counts of 50-150. This deviation is moot for confirmatory claims because H2 is already blocked by the LLM audit failure.
• [defer] agent deferred in S4 H1: DECISION: DEFER **What was accomplished:** - Described and merged the key datasets: daily Reddit (dataset:55), VIX (dataset:62), SPY (dataset:61). - Built a merged market panel (dataset:63, 213 trading days) and computed correct next-trading-day lags from the full market series (dataset:68): `delta_VIX_t`, `delta_VIX_t1`, `abs_SPY_return_t`, `abs_SPY_return_t1`, and `next_trading_date`. - Merged
• [retrospective] S7 outcome=final; proposed 3 skill(s): ['check-platform-scalar-transform-support', 'audit-sparse-component-before-composite', 'verify-group-unit-window-presence']. RETROSPECTIVE: This pilot study deferred because multiple data-quality issues compounded: a missing subreddit in the target window, extremely sparse ticker mentions that weakened a composite attention metric, unvalidated VADER sentiment due to an LLM audit budget failure, and a platform limitation that prevented implementing a preregistered smoothing formula. The key methodological lesson is that social-media data pipelines require per-window unit verification and component-level coverage audits before composite metrics are locked in preregistration, and that platform transform capabilities mu