NLP Glossary

Here are all the words used in the notes, and their meaning according to slides.

• Corpora: courpus, a computer readable collection
• Utterance: uhm, ehm…
• Word Types: set of distinct words in a corpus $V$, where $|V|=\#types$ (e.g. “I like fish and I like chips”: word types = 5, because we must not count the duplicates)
• Tokens: number of word in a phrase (e.g. “I like fish and I like chips”: tokens = 7)
• Lemmatization: extract root from a word
• Minimization: see Loss Function
• Loss Function: how good a model is. The lower the better, it means that is very accurate in predicting
• Softmax: soft probability smaller than x, max amplifies probabilities of target $x_i$. Maps arbitrary values $x_i$ to probability distribution $p_i$
  • $softmax(x_i)=\frac{\exp (x_i)}{{\sum }_{j=1}^n\exp (x_j)}=p_i$
• Gradient: how much vectors need to be in order to reduce errors