Unsupervised Thoughts

Accelerating LLM Code Generation Through Mask Store Streamlining

Vivien — Fri, 03 Jan 2025 00:00:00 +0100

Structured text generation techniques enforcing context-free grammar (CFG) constraints (Willard & Louf, 2023; Gerganov & et. al., 2024; Lundberg & Ribeiro, 2023; Geng et al., 2024; Beurer-Kellner et al., 2024; Ugare et al., 2024; Dong et al., 2024) are particularly useful to generate syntatically correct computer code in the context of LLM-based coding tools. These techniques guarantee full compliance but introduce computational overhead at inference time.

Fast, High-Fidelity LLM Decoding with Regex Constraints

Vivien — Thu, 22 Feb 2024 00:00:00 +0100

Constraining the text generated by an LLM is essential for developing robust LLM applications. For example, a developer may require LLM responses to adhere to a specific JSON or YAML schema so that they are comprehensive and reliably parseable.

An Optimal Lossy Variant of Speculative Decoding

Vivien — Mon, 18 Sep 2023 00:00:00 +0200

Speculative decoding (Leviathan et al., 2023; Chen et al., 2023) is an elegant decoding strategy for autoregressive language models. It accelerates text generation while preserving the target distribution. In this blog post, I introduce mentored decoding, a novel, provably optimal, lossy variant of speculative decoding. It further increases the decoding speed at the cost of a bounded deviation from the target distribution.

Better Steering LLM Agents with LMQL

Vivien — Fri, 28 Apr 2023 00:00:00 +0200

One of the most fascinating aspects of autoregressive large language models (LLMs) like GPT-3 is their ability to act through external tools. In this post, I’ll illustrate how LMQL (Beurer-Kellner et al., 2022), a new programming language for language model interaction, helps better steer such LLM agents. I’ll take ReAct (Yao et al., 2022) as an example and show how to enforce constraints on the task-solving trajectory, the choice of tools and the tools’ inputs.

Adding Text (Really) inside Pictures and Videos

Vivien — Wed, 08 Jun 2022 00:00:00 +0200

In the image above and the video below, the left character partially hides the text. In this post, I show how to automatically create such effect with a pretrained deep learning model and traditional computer vision techniques.

Simulating Depth Perception with Face Tracking

Vivien — Tue, 04 May 2021 00:00:00 +0200

Motion parallax is created by the apparent relative motion of objects when an observer moves. It’s one of the monocular cues that enable depth perception. I created a demo to simulate this phenomenon thanks to TensorFlow.js and three.js.

What Are the Ethical Risks of Your AI Project?

Vivien — Fri, 20 Nov 2020 00:00:00 +0100

How to prevent AI systems from causing unintended harms?

Learning through Auxiliary Tasks

Vivien — Sun, 17 Feb 2019 00:00:00 +0100

Play, a seemingly futile activity, is ubiquitous in human cultures and very widespread in the animal kingdom, which suggests it serves an important purpose