January 30th, 2012
Comparing two confusion matrices is a standard approach for comparing the respective targeting systems, but by no means is it the only one. As we will discuss in the coming days, you can also compare two score based targeting systems by comparing their lists. But for now, let us focus on comparing the targeting systems by comparing their respective confusion matrices.
The standard approach is to use a single value metric to reduce each matrix into one value, and then to compare the metric values. In other words, to compare M1 and M2, we simply compare f(M1) and f(M2), where function f is the single value metric.
Here are some single value metrics that can be considered as candidates:
- Kappa Statistic
- F1 measure
- Matthews Correlation Coefficient
- Reward/Cost based
- Sensitivity (Recall)
- Specificity (Precision)
A related approach can also be to take a matrix difference of the two matrices, and then using a dot product (or scalar product), but it is easy to see that transforms to using reward/cost based metric. [A.C - B.C = (A-B).C etc.]
January 28th, 2012
During the spring 2012, I am teaching Unix system administration at GWU, which is a mix undergraduate and graduate class. Most of the students are graduate students, and they bring a lot of interesting perspectives to this class. When we finish this class in May, there is a small likelihood that the following scenario plays out somewhere:
View this toon at Toondoo
January 21st, 2012
[This is one of rare posts, that will be cross-posted to the Guy Down the Street blog as well.]