A practical guide

How do you read a research result?

Do not start with the highest percentage. A credible conclusion requires a clearly defined question, a final test, an appropriate baseline, an assessment of uncertainty and realistic costs.

The right order

Six questions to ask before accepting a conclusion.

Following this order helps prevent an impressive number from carrying more weight than the research supports.

  1. 01

    What question was specified in advance?

    A good question specifies the information used, the outcome to be predicted and the time horizon. Changing these after seeing the results increases the risk of chance findings.

  2. 02

    Was there a genuine holdout?

    Development data helps a model learn. A holdout is a later, reserved period that cannot be used to adjust any modelling choices. Only this final test provides a fair assessment.

  3. 03

    Is the sample large and representative enough?

    More observations can reduce random noise, but a thousand nearly identical minutes are not equivalent to a thousand independent events. Consider the companies, time periods and market regimes represented as well.

  4. 04

    Does the model beat a simple baseline?

    Directional accuracy above 50% can still be weak if always predicting an upward move achieves 52.9%. The relevant comparison is the best simple alternative, not just chance.

  5. 05

    How wide is the uncertainty interval?

    A point estimate is a single number; an interval shows the range of values compatible with the data. If the interval for a difference includes zero, the apparent advantage may be due to chance.

  6. 06

    Does the result survive costs and replication?

    A statistical pattern must remain economically useful after transaction costs. Repeating the test on different data then provides a stronger check against a chance initial finding.

An actual example from this archive

49.88% is not ‘almost good enough’.

In the independent replication, the text model predicted the direction correctly in 49.88% of cases. The simple baseline achieved 52.90%, while the model’s 95% interval was 47.20–52.51%.

The appropriate conclusion is not that the model narrowly missed the mark, but that this final test found no evidence of additional predictive value.

Read the full replication study
Model 49.88% below the baseline
Baseline 52.90% simple reference model
95% interval 47.20–52.51% model uncertainty interval
Conclusion Not replicated no demonstrated added value

Status colours

At a glance, always backed by evidence.

In progress

Data processing or the final test is still under way; there is no final conclusion yet.

Preliminary indication

An initial effect that warrants further research, but still lacks an uncertainty assessment or independent replication.

Not confirmed

The final test found no convincing improvement over the reference model.

Negative after costs

A gross pattern did not remain economically viable after realistic costs.

Pending

Only the study design has been prepared. The research has not started and no results are available.