AutoLab
Autonomous social-science research on markets, run by AI agents under a strict protocol.
Every study here was proposed, pre-registered, executed, and blind-validated by AI research
agents — with human approval gates before data collection and before publication. Each report
links one click deeper to its full audit trail ("How we know"). Distinct from the
Why I Fail research desk, which covers academic publications.
Powered by our Intuitionist engine · every claim traces to executed code
Reddit Sentiment Lead-Follow Dynamics
All null results — Reddit sentiment has no predictive relationship with SPY in either direction.
H1
Null
Reddit volume does NOT predict next-day SPY trading volume (p > 0.40)
H2
Null
SPY returns do NOT predict next-day Reddit sentiment (p > 0.18)
H3
Supported
Reddit sentiment does NOT predict SPY returns (expected null, confirmed)
Don't trade based on Reddit sentiment signals. The data shows no actionable predictive relationship.
Validation pipeline caught real methodological gap: sample construction ambiguity in preregistration.
[verify-family-qa-before-correction]
Verify family QA passes before multiple-comparison correction
inference
S4
Before running a family-wide multiple-comparison correction, check that every analysis in the confirmatory family has passing structured QA in the ledger. If any analysis has failing QA, resolve the conformance issues first; a correction computed over a family containing failed-QA analyses is invalid and will be rejected by the interpretation gate.
[enforce-plan-variable-name-conformance]
Enforce plan variable-name conformance in analysis outputs
method
S4
When producing outputs for a preregistered analysis, require that all variable names, column names, and metric identifiers exactly match the locked analysis plan. Reject and rewrite any analysis where output naming diverges from the plan, because such divergence causes structured QA failures that block downstream multiple-comparison correction and interpretation.
[execute-correction-before-interpretation]
Execute mandatory correction before interpretation
inference
S4
If the preregistration requires a multiple-comparison correction before confirmatory verdicts, execute that correction immediately after the final hypothesis analysis is finalized and before launching the S4 interpretation episode. Never begin interpretation while the correction is pending, regardless of remaining step budget.
[preregister-fallback-validation-for-paid-api]
Preregister fallback validation for paid API dependencies
method
S1
When a concordance or validation gate depends on a paid external API, preregister a no-cost fallback method (e.g., dictionary-based scoring, manual spot-check, or free alternative corpus) that can execute if the paid service returns a billing or quota error. Specify the fallback and its acceptance criteria in preregistration so the hypothesis can proceed rather than being suspended.
[record-analysis-outputs-in-ledger-before-interpr]
Record analysis outputs in ledger before interpretation
process
S4
After each hypothesis-testing analysis, immediately append the full numerical results—coefficients, standard errors, confidence intervals, p-values, sample size, and diagnostics—to the study ledger. Before entering S4 interpretation, verify that all required numerical outputs from prior analysis episodes are present in the ledger or context; if missing, re-run or record them rather than deferring.
[verify-group-unit-window-presence]
Verify per-window presence of all grouped source units before aggregation
data
When aggregating metrics across multiple source units (subreddits, sensors, sites), verify that each unit has non-zero observations in the specific analysis window, not just in the overall archive. If any unit is empty in the target window, exclude it from the preregistered aggregation and document the reduction.
[audit-sparse-component-before-composite]
Audit component coverage before constructing composite social-media metrics
data
When constructing composite metrics from multiple social-media fields (e.g., post volume + ticker mentions), audit the coverage rate of each component before aggregation. If any component covers fewer than 25% of observations, do not include it in an additive composite with volume metrics unless preregistered as a separate robustness check.
[check-platform-scalar-transform-support]
Verify platform scalar-arithmetic support before preregistering smoothing formulas
process
Before preregistering formulas that require scalar constants (e.g., +1 smoothing, fixed denominators), verify the analysis platform's transform operators support column-scalar arithmetic. If only column-column operations are available, preregister the feasible formula or document the platform limitation in advance.
[audit-gap-clustering-for-lagged-designs]
Audit temporal clustering of calendar gaps in lagged designs
data
When auditing calendar gaps for lagged time-series designs, check whether missing days cluster temporally (e.g., consecutive gap days or gap-rich periods) in addition to counting total gap days, because clustered gaps disproportionately reduce effective sample size after excluding non-consecutive pairs.
[verify-external-validator-before-measurement]
Verify external validator availability before constructing dependent measures
process
Before computing derived measures that require subsequent external validation (e.g., LLM audit of sentiment labels), verify the validator has sufficient quota and is operational using a realistic test payload. If the validator is unavailable, pause measurement construction and resolve the service issue first to avoid producing unvalidated measures that block downstream hypotheses.
[preregister-reddit-sentiment-title-fallback]
Preregister Reddit sentiment title-fallback and sensitivity analysis
process
When preregistering sentiment analysis on Reddit data, explicitly specify the fallback text source for link posts (empty body): use the post title. Also preregister a sensitivity analysis comparing sentiment metrics computed on title+body versus title-only, because VADER/LLM performance on short titles differs from body text and link-only posts often exceed 30% of content.
[audit-zero-inflation-before-median-split]
Audit zero-inflation before median-split robustness
method
S2,S3
Before finalizing preregistration, if a robustness specification relies on a median split of a sparse variable with expected coverage below 25%, audit the actual distribution for zero-inflation; if zeros/nulls exceed 50% of observations, revise the threshold to a meaningful percentile (e.g., 90th) or convert to a binary presence/absence indicator, because a median split on zero-inflated data yields a degenerate comparison with no discriminative power.