Normalized Asymmetric Mutual Information

The entropy of either, clustering $X$ and categorization $Y$, provides another tight bound on mutual information $I(X,Y)$ that can be used for normalization. Since the categorization $Y$ is a stable, user given distribution, let's consider

$$I(X,Y) \leq H(Y) .$$ (8.10)

Hence, one can alternatively define [0,1]-normalized asymmetric mutual information based quality as

$$NI(X,Y) = \frac{I(X,Y)}{H(Y)}$$ (8.11)

which translates into frequency counts as

$$\phi^{(\mathrm{NAMI})} (\mathbf{\lambda},\mathbf{\kappa}) = \frac... ...} n} { n^{(h)} n_{\ell} } } { - \sum_{h=1}^g n^{(h)} \log \frac{n^{(h)}}{n} } .$$ (8.12)

Note that this definition of mutual information is asymmetric and does not penalize overrefined clusterings $\mathbf{\lambda}$. Also, $\phi^{(\mathrm{NAMI})}$ is biased towards high $k$.