- Asalamu alaikom, welcome back
to another biostats ATP video, this time around we’ll be talking about the
“receiving operating characteristic” curve, also known as the ROC curve. Introduction:
This curve helps us understand how well a test can distinguish
between two groups. In other words, how well can the test see if the patient does or
does not have the disease. The Y axis is the True Positive rate otherwise known as Sensitivity.
The X axis is the FP rate, otherwise known as the complement or “the opposite” of Specificity
or 1-Specificity if you will. In other words, x axis represents true positive rate (TPR) while
Y-axis represents false positive rate (FPR) ROC Curve:
The curve is defined by its specificity and sensitivity and
associated cutoffs. In other words, how well it can identify those with a disease compared
to those without a disease based on different cutoffs is what determines the shape of the curve.
For instance, let’s take a group of people we tested for condition X with a cutoff of 100
units. Suppose after all the calculations, we ended up with a sensitivity (SN) of
80% and specificity (SP) of 85%. However, what if we want to see what would happen to
SN and SP if we raised or lowered the cutoff, for example, when we raised the cutoff by 20
units, the SN dropped to 70% while SP became 90%. What if we lowered the cutoff
by 20 points? Then SN became 90% and SP dropped to 70%. I’m just making up the
numbers here to keep it simple but notice how we’re sort of getting a nice curve here. If you
rinse and repeat this with many different cutoffs, you’ll get a curve that shows you what the SN and
SP would look like based on the chosen cutoffs. This makes it easier to understand what cutoffs
would be most helpful to apply in each situation without having to manually write down all the
possible cutoffs and their associated SN and SP. Notice how the lower left corner suggests a
SN of 0% and SP of 100%, this would be like raising the cutoff so high that you won’t get any
FP. Hmm, still not really that practical, huh? On the other end of the spectrum, the
top right corner suggests a SN of 100% and SP of 0%, this would be like having a
very low cutoff, sure you wouldn’t get any FN, but this isn’t all that practical either.
The top left corner suggests a test with 100% SN and 100% SP. That is where we
would ideally want every test to be. Please note that sometimes, the curve is not
perfectly symmetrical. This is just a testament to how real life doesn’t always provide you with
perfect curves, but beggars can’t be choosers, eh? ROC Curve Applications:
Moving on, how can we apply this concept? Well, sometimes a test is needed to screen or
confirm diseases. If you are using the test as a screening tool, your job is to look
for the test with the highest sensitivity, in other words, the highest point on the curve
within acceptable detriment to specificity. That is to say, we can’t just take the
highest point overall because this would leave us with a SP that is very close to 0,
and thus the test would have much less value. A good example of a high point that doesn’t
completely wipe SP would be like this one for example. But, if it’s a confirmatory test, you’re
looking for the left most point, so you would pick this point without completely wiping out SN.
Golden rule when using tests in medicine: if you are SCREENING focus on sensitivity,
but if you are DIAGNOSING focus on specificity As you can see, the ROC curve is immensely
useful in helping us visualize the different cutoffs of a test and allows us to make
decisions as to which test or which cutoffs of a particular test would be better
suited to being a more SN tool or SP tool. When it comes to comparing two different tests,
eyeballing the ROC curves becomes very handy. Since the top left corner suggests an ideal
test, we want curves to be closer to that corner. Another way you could go around this
is to check which curve has more area under it. Every curve will start in the
bottom left and end at the top right. So how much it approaches the top left corner will also
determine how much area it covers under it. So, for example, when looking at these two curves,
it’s clear that A is the better performing test. With this paradigm in your mind, we’d like to add
a few more concepts to help tie things together: Accuracy is another important term and relates
to the validity of the measurement. I like to think of it as the “trueness” of a measurement to
help remind me of the formula which is TP + TN/ TP + FP + TN + FN (pretty much everything goes in
the denominator, while only the true ones go into the numerator). A test is considered more accurate
the closer it approaches the top left corner, and less accurate as it approaches the diagonal line.
We wanted to just mention a few points here to help tie in a few more concepts
in relation to the ROC curve. Last but not least, since the ROC curve can
reveal to us the SN and SP of a particular cutoff, we can actually find out TP, TN, FP, and
FN. However, we also need a bit more given, specifically, we need the amount of people
we are testing and how many of them have the disease, as known as, the prevalence.
Using this given, we can find out the TP, TN, FP, and FN. Turns out ROC curves
can do a lot of cool things, huh? - In summary ROC curve is built using
2 axes: sensitivity and 1-specificity, otherwise known as false positive rate
- It allows you to choose the optimal cutoff for a test by looking at the ROC curve
of that test. That way you can have an idea about the SN and SP of that cutoff
- ROC also allows you to compare between different tests, by calculating the AUC, to determine which one is better
- Lastly, remember this relationship: SN – Screening for a disease
SP – Diagnosing/conforming a diagnosis With that, we hope you benefited from this
video! Consider liking and subscribing, and as always, thanks for watching!