Research

Meta Llama 3.2, with Meta GenAI taem

Meta AI's Llama 3.2 release introduces small and medium-sized vision LLMs and lightweight text models optimized for edge and mobile devices. These models support context lengths of 128K tokens and excel in on-device tasks like summarization and instruction following. Llama Stack distributions simplify deployment across environments with integrated safety features. This update enhances Llama's capabilities, modifiability and cost efficiency, driving innovation in generative AI applications. 

Understanding LLMs from Scratch Using Middle School Math

We talk about how Large Language Models (LLMs) work, from scratch — assuming only that you know how to add and multiply two numbers. The article is meant to be fully self-contained. We start by building a simple Generative AI on pen and paper, and then walk through everything we need to have a firm understanding of modern LLMs and the Transformer architecture. The article will strip out all the fancy language and jargon in ML and represent everything simply as they are: numbers. We will still call out what things are called to tether your thoughts when you read jargon-y content.

The Llama 3 Herd of Models, with Meta GenAI taem

This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development. 

Meta Llama 3 large language model, with Meta GenAI taem

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.

Costly Inspection and Money Burning, with Can Urgun

A principal designs a mechanism to allocate an indivisible, productivity-increasing good to one of many agents. Monetary transfers are not allowed. Instead, we consider the interplay between two instruments studied only in isolation: “costly verification of the agent’s type” and “money burning”. We use a graph-theoretic approach and characterize the optimal mechanism completely.

Costly Verification by an Intermediary in a Two Sided Market

This paper studies the players in a two sided market when the good sold is of uncertain value, network effects are in play, and there is a cost to verifying the value of the good. It describes the optimal mechanism to maximize the revenue for the intermediary.

Indices for Dynamic Pricing in the Event Ticketing Industry (short non-technical letter)

This paper introduces new price indices and measures to facilitate dynamic pricing in the sports ticketing industry.

A Seller's Problem and Costly Verification, with Can Urgun 

This paper outlines the optimal strategy to maximize the revenue of a seller when the buyer's willingness to pay is unknown, but can be discovered at a cost (market research etc..). 

Differentiating Reputations in Dynamic Duopolies, with R. Andrew Butters 

This paper outlines how reputation in a marketplace can be used to boost prices and revenue in dynamic duopolies, and how firms end up at different product quality and prices in a dynamic equilibrium.