Question 24

College Answer

An ROC plot is a graphical representation of sensitivity vs. 1- specificity for all the observed data values for a given diagnostic test.

Advantages:
•    Simple and graphical
•    Represents accuracy over the entire range of the test
•    It is independent of prevalence
•    Tests may be compared on the same scale
•    Allows comparison of accuracy between several tests.

How it may be used:
•    Can give a visual assessment of test accuracy
•    May be used to generate decision thresholds or “cut off” values
•    Can be used to generate confidence intervals for sensitivity and specificity and likelihood ratios.

Discussion

In this LITFL article, ROC curves are discussed in detail, but without apocryphal gibberish.

If one were to restrict oneself to what is manageable within a 10-minute timeframe while mentioning all the important points, one would produce an asnwer which resembles the following:

• The ROC curve is a plot of sensitivity versus false positive rate (1-specificity) for all observed values of a diagnostic test.
• It is a graphical representation of a tests' diagnostic accuracy
• It allows the comparison of accuracy between tests
• It allows the determination of cutoff values
• It can be used to generate confidence intervals for sensitivity and specificity and likelihood ratios.

Advantages:

• Simple and graphical
• Independent of prevalence
• Allows comparison between tests, on the same scale

That, of course, is the bare bones of the answer. If one were to succumb to basic human urges, one would produce an answer which resembles the following:

• The ROC curve is a plot of sensitivity vs. false positive rate, for a range of diagnostic test results.
• Sensitivity is on the y-axis, from 0% to 100%
• The ROC curve graphically represents the compromise between sensitivity and specificity in tests which produce results on a numerical scale, rather than binary (positive vs. negative results)
• ROC analysis can be used for diagnostic tests with outcomes measured on ordinal, interval or ratio scales.
• The ROC curve can be used to determine the cut off point at which the sensitivity and specificity are optimal.
• All possible combinations of sensitivity and specificity that can be achieved by changing the test's cutoff value can be summarised using a single parameter, the area under the ROC curve (AUC).
  • The higher the AUC, the more accurate the test
  • An AUC of 1.0 means the test is 100% accurate
  • An AUC of 0.5 (50%) means the ROC curve is a a straight diagonal line, which represents the "ideal bad test", one which is only ever accurate by pure chance.
• When comparing two tests, the more accurate test is the one with an ROC curve further to the top left corner of the graph, with a higher AUC.
• The best cutoff point for a test (which separates positive from negative values) is the point on the ROC curve which is closest to the top left corner of the graph.
• The cutoff values can be selected according to whether one wants more sensitivity or more specificity.

Advantages of the ROC curves:

• A simple graphical representation of the diagnostic accuracy of a test: the closer the apex of the curve toward the upper left corner, the greater the discriminatory ability of the test. 
• Allows a simple graphical comparison between diagnostic tests
• Allows a simple method of determining the optimal cutoff values, based on what the practitioner thinks is a clinically appropriate (and diagnostically valuable) trade-off between sensitivity and false positive rate.
• Also, allows a more complex (and more exact) measure of the accuracy of a test, which is the AUC
  • The AUC in turn can be used as a simple numeric rating of diagnostic test accuracy, which simplifies comparison between diagnostic tests.

Bewick, Viv, Liz Cheek, and Jonathan Ball. "Statistics review 13: receiver operating characteristic curves." Critical care 8.6 (2004): 508.

Sedgwick, Philip. "Receiver operating characteristic curves." BMJ 343 (2011).- rather than an article, this is more of a "self-directed learning" question with an elaborate explanatory answer.

Fan, Jerome, Suneel Upadhye, and Andrew Worster. "Understanding receiver operating characteristic (ROC) curves." Cjem 8.1 (2006): 19-20.

Akobeng, Anthony K. "Understanding diagnostic tests 3: receiver operating characteristic curves." Acta Paediatrica 96.5 (2007): 644-647.

Ling, Charles X., Jin Huang, and Harry Zhang. "AUC: a statistically consistent and more discriminating measure than accuracy." IJCAI. Vol. 3. 2003.

Greiner, M., D. Pfeiffer, and R. D. Smith. "Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests." Preventive veterinary medicine 45.1 (2000): 23-41.