Jensen-Shannon(JS) Divergence

It is based on the Kullback-Leibler divergence but is a different, symmetric version of the Kullback-Leibler divergence measure. The Jensen-Shannon distance between two distributions, if they are similar, is 0. 

It is defined as the square root of the Jensen-Shannon divergence, which is a measure of the average divergence of the two distributions from their mean.

The formula to compute Jensen-Shannon between P and Q is:

JS(P,Q) = sqrt( [KL(P,M) + KL(Q,M)] / 2 )

Where M is the average of P and Q i.e. M = (P + Q) / 2

Then, Jensen-Shannon is the square root of the average of KL(P,M) and KL(Q,M)

The Jensen-Shannon distance is often applied to compare the similarity of two probability distributions in a variety of fields, including information theory, protein surface comparison, bioinformatics, machine learning, and natural language processing. It is a popular choice because it is easy to calculate and has several valuable properties, such as being symmetric, bounded, and satisfying the triangle inequality.

JS can be used for both categorical and numerical features. The default threshold is 0.05 in AryaXAI