N-Gram Visualization

Overview

We visualize inference for character-level n-gram language models using a trigram language model (predicting the next character based on the previous two characters) as an example. A trigram model was chosen so that the heatmap would be easier to visualize and allow rolling over with your mouse to see the probabilities.

These visualizations were inspired by my learning from Andrej Karpathy’s ngram github repo, his excellent makemore video, and the very nice book chapter that he points to from the repo page.

The first page (Explore) lets you familiarize yourself with a 2D heatmap representation of the conditional probability distributions for the next character given the previous two characters. The second page (Sample) lets you sample from the conditional probability distributions and see how the probability distributions change as you continue to sample. The third page (Generate Names) lets you generate baby names by sampling from the n-gram model.