projects:workgroups:patient-level_prediction:best-practice
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
projects:workgroups:patient-level_prediction:best-practice [2016/05/03 19:02] prijnbeek [Best practices] |
projects:workgroups:patient-level_prediction:best-practice [2016/05/04 08:23] jreps [Best practices] |
||
|---|---|---|---|
| Line 21: | Line 21: | ||
| **Model development** is done using a split-sample approach. The percentage used for training could depend on the number of cases, but as a rule of thumb 80/20 split is recommended. Hyper-parameter training should only be done on the training set. | **Model development** is done using a split-sample approach. The percentage used for training could depend on the number of cases, but as a rule of thumb 80/20 split is recommended. Hyper-parameter training should only be done on the training set. | ||
| - | **Model validation** is done only once on the holdout set. The following performance measures should be added: To Do! | + | **Internal validation** is done only once on the holdout set. The following performance measures should be calculated: |
| + | . Overall performance: Brier score (unscaled/scaled) | ||
| + | . Discrimination: Area under the ROC curve (AUC) | ||
| + | . Calibration: Intercept + Gradient of the line fit on the observed vs predicted probabilities | ||
| + | We recommend box plots of the predicted probabilities for the outcome vs non-outcome people, the ROC plot and a scatter plot of the observed vs predicted probabilities with the line fit to that data and the line x=y added. | ||
projects/workgroups/patient-level_prediction/best-practice.txt · Last modified: 2016/05/04 15:43 by prijnbeek
Page Tools
