- ROC curves show the balance between sensitivity and specificity.
- We’ll use the [ROCR] package. It only takes 3 commands:
prediction()using the model
- calculate the model
performance()on both true positive rate and true negative rate for a whole range of cutoff values.
colorizeoption colors the curve according to the probability cutoff point.
We can also use the
performance() function to evaluate the \(f1\) measure
We can dig into the
perf.acc object to get the maximum accuracy value (
y.value), then find the row where that value occurs, and link it to the corresponding cutoff value of x.
Sometimes (like here) there is not one single maximum value for accuracy. In this case I would look at which of these two cutoff points maximize other metrics such as the \(f1\) score.
- Can also be used for model comparison: http://yaojenkuo.io/diamondsROC.html
- The Area under the Curve (auc) also gives you a measure of overall model accuracy.