Developing a valuable clinical prediction tool requires a sequence of investigations.

1) Reliability Studies:  Examining the inter-rater reliability of potential predictor variables

2) A rule derivation study:  This should include examination of all reasonable potential  predictor variables.  Thus for a rule on pulmonary embolism, one would probably study things such as vital   signs, symptoms, pulse oxemitry, blood gas values, ray and EKG results.   When the rule is derived, it may occur that some of these potential predictors are not indeed predictive.  Therefore they would not be included as predictive variables in the final model.. Nevertheless, a well designed study should have considered all reasonable potential predictors.

3) Validation studies in multiple clinical settings:  these include gold standard testing to conform disease presence or absence

4) Outcomes level studies:  Examine the effect of the tool when it is put into practice, without gold standard confirmations.