Skip to content

C.13 Entropy

Jump to bottom
PriyaDCosta edited this page Mar 7, 2023 · 3 revisions

1. Feature Name

C.13 Entropy

(Chat GPT)

H(X) = -Σ p(x) log2 p(x)

where H(X) is the entropy of a random variable X, p(x) is the probability of the occurrence of the event x, and log2 is the logarithm with base 2.

The formula can be interpreted as a measure of the amount of uncertainty or randomness in the distribution of the events of X. It ranges from 0 (when X is completely certain or has no randomness) to a maximum value of log2(n) (when X is completely uncertain or has maximum randomness) where n is the number of possible events.

2. Literature Source (Serial Number, link)

C.13, Refer

3. Description of how the feature is computed (In Layman’s terms)

  1. This code implements a function called ngram_dialog_act_entropy() which calculates the entropy of n-grams (n consecutive words) in a given column of a pandas DataFrame. The function takes the following arguments:
  • df: The pandas DataFrame containing the data to analyze.
  • on_column: The name of the column in df that contains the text data.
  • n: The value of n for the n-grams to analyze.
  • set1: A list of words to count occurrences for and calculate entropy.
  • set2: Another list of words to count occurrences for and calculate entropy.
  • set1_label: A label to assign to the text data in on_column that has a higher entropy for set1 than for set2.
  • set2_label: A label to assign to the text data in on_column that has a higher entropy for set2 than for set1.
  • The function first initializes a CountVectorizer object with a specified range of n-grams and the words to count occurrences for. Then, it iterates through each row of the DataFrame, counts the occurrences of words in set1 and set2, normalizes the counts to obtain probabilities, and calculates the entropy of each set.

  1. If the entropy of set1 is higher than that of set2, the function appends set1_label to a list. If the entropy of set2 is higher than that of set1, the function appends set2_label to the list. If both entropies are equal, the function appends "neutral" to the list.

  2. The function adds a new column to the DataFrame called "entropy" containing the labels assigned to each row based on the entropy calculation.

4. Algorithms used (KNN, Logistic Regression etc.)

None

5. ML Inputs/Features

None

6. Statistical concepts used

NA

7. Pages of the literature to be referred to for details

Refer point 6.

8. Any tweaks/changes/adaptions made from the original source

None

Clone this wiki locally