One-Hot Encoding

The cat plays and work can be encoded in a matrix.

$$\begin{array}{rl}& V=[\text{the},\text{cat},\text{plays},\text{and},\text{work}]\\ & \text{the}=\left[\begin{array}{ccccc}1& 0& 0& 0& 0\end{array}\right]\\ & \text{plays}=\left[\begin{array}{ccccc}0& 0& 1& 0& 0\end{array}\right]\\ & \text{combined}=\left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 0& 1& 0& 0\end{array}\right]\end{array}$$

Problems:

• data is sparse
• lack of semantic understanding: doesn’t understand relationships between words

Word2Vec is an evolution of one-hot encoding ⇒ Word Embeddings