Reddit sentiment lead-follow dynamics with market prices

Hypotheses: predicted → found

Datasets

Activity log (per-stage summaries — no raw model transcripts)

Limitations & lessons

• [limitation] Reddit data contains only 4 distinct subreddits (not the 5 listed in the source profile). One expected subreddit (likely r/stocks, which the source profile notes was "added late" with only 391 total rows) appears absent or empty in the 2025-09-01 to 2026-06-29 collection window. The preregistration refers to "collected finance subreddits" without naming specific ones, so the plan runs on what was actually collected, but the attention/sentiment composite is narrower than intended.
• [limitation] Reddit content field is only 60% non-null (17,279 of 28,727 posts). The remaining 40% are link/image posts with empty body text. The preregistration's H2 sentiment plan explicitly specifies "title+body with title-only fallback for link posts with empty body," so this is anticipated. However, the sensitivity analysis comparing title+body vs title-only sentiment will be comparing metrics where 40% of posts contribute only title text in both cases, limiting the discriminative power of that robustness check.
• [limitation] Reddit collection has 243 distinct source_dates over the 2025-09-01 to 2026-06-29 span (~303 calendar days), meaning approximately 60 gap days. The preregistration's calendar_rule requires both day t and day t+k to have Reddit coverage AND be trading days, with non-consecutive pairs dropped. Gap days that fall on trading days will reduce the effective sample size for lagged analyses, particularly if gaps cluster temporally. The primary window (2025-09-01 to 2026-01-31) and holdout (2026-02-01 to 2026-06-30) need per-window gap auditing in S4. The holdout window is explicitly noted as "severely gappy" in the preregistration.
• [limitation] Ticker mentions field has only 3,940 non-null values out of 28,727 posts (13.7% coverage). While tickers are not a primary variable in any hypothesis test plan, they were collected per the include_tickers flag. If any robustness analysis attempts to use ticker-level attention, the low coverage rate (below the 25% threshold per lab skill audit-sparse-component-before-composite) means ticker-based composites should not be combined with volume metrics in additive indices.
• [limitation] SPY market data starts on 2025-09-02, not 2025-09-01. September 1, 2025 was Labor Day (NYSE holiday), so no trading data exists for that date. This is expected and consistent with the preregistration's calendar_rule (Mon–Fri excluding NYSE holidays). The VIX FRED series does include a 2025-09-01 value (16.12), which may be a carry-forward or non-trading-day observation — this will need reconciliation during the calendar alignment in S4.
• [lesson:data] When collecting Reddit data for lagged time-series designs spanning multiple months, the ~65 gap days in the archive disproportionately reduce effective sample size after applying the preregistered calendar_rule (both t and t+k must have coverage AND be trading days, non-consecutive pairs dropped). In this study, 243 collection days over ~303 calendar days means ~20% gap rate. Future studies should pre-audit gap density in the target window before locking the preregistration, and consider whether the resulting effective N provides sufficient power for |t|≥3 thresholds at α=0.001.
• [limitation] The mandatory Holm multiple-comparison correction over the full confirmatory family could not be completed during the H3 micro-episode because three H1 analyses (analysis:35, analysis:36, analysis:38) have failing structured QA (plan_conformance: variable name divergence from the locked H1 plan). The H3 analyses themselves (analysis:41, analysis:42) have passing QA. The Holm correction must be rerun after the H1 QA issues are resolved before any H3 verdict is finalized.
• [defer] agent deferred in S4 interpretation: DECISION: DEFER What I would need to complete this stage: 1. **One tool call to `run_multiple_comparison_correction`** (takes no arguments). The gate requires a recorded Holm correction over the 32 active confirmatory tests in the ledger. My step budget exhausted before I could execute it. Without corrected p-values I cannot make any citable confirmatory claim — the pre-registered inference meth
• [retrospective] S7 outcome=final; proposed 3 skill(s): ['execute-correction-before-interpretation', 'enforce-plan-variable-name-conformance', 'verify-family-qa-before-correction']. RETROSPECTIVE: This pilot study deferred because the agent entered S4 interpretation with two unmet prerequisites: (1) three H1 analyses had failing structured QA due to variable-name divergence from the locked plan, and (2) the mandatory Holm multiple-comparison correction was therefore blocked and unexecuted. The agent exhausted its step budget attempting to recover inside interpretation rather than resolving QA upstream. The key lesson is that downstream inference gates (correction, interpretation) are only as robust as the upstream analysis conformance they depend on; fixing output naming