Posts

Constrained decoding is getting more and more attention in the field of large language models (LLMs). It aims to generate sequences of tokens that satisfy certain constraints. A typical example is to force the generation from LLM to satisfy a given JSON schema so that the generated JSON data can be used directly in a downstream application, such as tool use.

Here are some books and blogs I’ve read or plan to read. I’ll update this list from time to time. Books I’ve read these books: Chip War: The Fight for the World’s Most Critical Technology, by Chris Miller, 2022 The PhD Grind: A Ph.

Basics A few quick review of basic probability theory Chapter 1: Introduction to Bayesian Networks Three definition of probability Graph Description of Random Variables No One-to-One Correspondence between Graph and Probability Distribution Graph D-separation Bayesian networks looks like a game of conditional independence and dependence

If you are familiar with constrained decoding, you might have occasionally heard people mentioning the word “posterior inference” in the context of constrained decoding. In this post, I will explain why constrained decoding can(should) be seen as a form of posterior inference.

Speculative Sampling The idea of speculative sampling is to use a draft sampling to achieve the same sampling result as the target sampling. We have a target sampling distribution $p(x)$ and a draft sampling distribution $q(x)$.

🌟 Understanding Cloud GPU Pricing: A Hotel Analogy 🏨 Hey there, AI and ML enthusiasts! 🤖 Have you ever wondered about the cost of cloud GPU resources for your projects? You’re not alone!

Perplexity Definition 1 Given a sequence $X = x_1, x_2, \dots, x_T$ of words and a language model $p$, the perplexity of the sequence can is defined as follows: $$ \text{PPL}(X) = p(x_1, x_2, \dots, x_T)^{-\frac{1}{T}} $$

WordPiece WordPiece tokenization is a subword tokenization algorithm. It is used in BERT, which is a work of 2018. Example: from transformers import BertTokenizer tokenizer = BertTokenizer.from_pretrained("bert-base-uncased") tokenizer.tokenize("I have a new GPU!