Question 7

Explain the following statistical terms:

a)    Sensitivity.    (20% marks)

b)    Specificity.    (20% marks)

c)    Receiver Operating Characteristic (ROC) Curve.    (60% marks)
 

[Click here to toggle visibility of the answers]

College answer

Not available.

Discussion

a)

  • Sensitivity = true positives / (true positives + false negatives)
  • This is the proportion of disease which was correctly identified

b)

  • Specificity = true negatives / (true negatives + false positives)
  • This is the proportion of healthy patients in who disease was correctly excluded

c)

  • Receiver Operating Characteristic (ROC) curve is a plot of sensitivity vs. false positive rate, for a number of test results.
  • Sensitivity is on the y-axis, from 0% to 100%
  • The ROC curve graphically represents the compromise between sensitivity and specificity in tests which produce results on a numerical scale, rather than binary (positive vs. negative results)
  • The ROC curve determines the cut off point at which the sensitivity and specificity are optimal.
  • AUC is the Area Under the ROC curve.
  • The higher the AUC, the more accurate the test
  • An AUC of 1.0 means the test is 100% accurate
  • An AUC of 0.5 (50%) means the ROC curve is a a straight diagonal line, which represents the "ideal bad test", one which is only ever accurate by pure chance.
  • When comparing two tests, the more accurate test is the one with an ROC curve further to the top left corner of the graph, with a higher AUC.
  • The best cutoff point for a test (which separates positive from negative values) is the point on the ROC curve which is closest to the top left corner of the graph.

References

Hanley, James A., and Barbara J. McNeil. "The meaning and use of the area under a receiver operating characteristic (ROC) curve." Radiology 143.1 (1982): 29-36.

Zweig, Mark H., and Gregory Campbell. "Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine." Clinical chemistry 39.4 (1993): 561-577.

Cook, Nancy R. "Use and misuse of the receiver operating characteristic curve in risk prediction." Circulation 115.7 (2007): 928-935.

Jones, Catherine M., and Thanos Athanasiou. "Summary receiver operating characteristic curve analysis techniques in the evaluation of diagnostic tests."The Annals of thoracic surgery 79.1 (2005): 16-20.

Metz, Charles E. "ROC methodology in radiologic imaging." Investigative radiology 21.9 (1986): 720-733.

Søreide, Kjetil. "Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research.Journal of clinical pathology 62.1 (2009): 1-5.

Lusted, Lee B. "ROC recollected." Medical Decision Making (1984): 131-135.