Exploring an alternate classification metric
On this planet of knowledge science, metrics are the compass that information our fashions to success. Whereas many are conversant in the basic measures of precision and recall, there are literally a variety of different choices which can be value exploring.
On this article, we’ll dive into the Tversky index. This metric, a generalization of the Cube and Jaccard coefficients, may be extraordinarily helpful when attempting to stability precision and recall in opposition to one another. When applied as a loss operate for neural networks, it may be a strong technique to take care of class imbalances.
A fast refresher on precision and recall
Think about you’re a detective tasked with capturing criminals in your city. In reality, there are 10 criminals roaming the streets.
In your first month, you herald 8 suspects you assume to be criminals. Solely 4 of them find yourself being responsible, whereas the opposite 4 are harmless.
In case you had been a machine studying mannequin, you’d be evaluated in opposition to your precision and recall.
Precision asks: “of all these you caught, what number of had been criminals?”
Recall asks: “of all of the criminals within the city, what number of did you catch?”
Precision is a metric that captures how correct your predictions are, not counting what number of true positives you miss (false negatives). Recall measures how lots of the true positives you seize, no matter what number of false positives you get.
How do your detective expertise price in opposition to these metrics?
- precision = 4 / (4 + 4) = 0.5
- recall = 4 / (4 + 6) = 0.4
Balancing precision and recall: the F1 metric
In a great world, your classifier has each excessive precision and excessive recall. As a measure of how effectively your classifier is doing in opposition to each, the F1 statistic measures the harmonic imply between the 2:
This metric can be generally known as the Cube similarity coefficient (DSC).
