Skip to content
BEST·BOOKS
+ MENU
Books Authors Tags Endorsers Sources
← Back to On Lisp

AI Study Notebook AI-generated

Study Guide: On Lisp

Paul Graham

By Best Books

This AI-generated study guide is a reading aid. The source-backed recommendation record and evidence for this book live on the book page.

Key points Not available Flashcards Not available
On this page

On Lisp — Chapter-by-Chapter Outline

Author: Paul Graham First published: 1993 Edition covered: First and only edition, Prentice Hall, 1993 (ISBN 0-13-030552-9). The book has never been revised; Paul Graham later made it freely available as a PDF download. 413 pages, 25 chapters plus an appendix on packages.

Central thesis

Lisp is not merely a programming language but an extensible medium: a programmable programming language. The central technique that makes this possible is the macro — a program that writes programs — and the central style it enables is bottom-up programming, in which the programmer builds the language up toward the problem rather than decomposing the problem down toward the language.

Graham argues that mastering macros is the single most important step separating competent Lisp programmers from expert ones. Whereas introductory books cover macros briefly and treat them as an oddity, On Lisp devotes the bulk of its pages to macros: what they are, how to write them correctly, the pitfalls they introduce, and the extraordinary power they unlock when used well. The later chapters demonstrate this power concretely by building complete embedded languages — a query compiler, a nondeterministic choice engine, an ATN parser, and a Prolog — almost entirely through macros on top of standard Common Lisp.

The book's unifying claim is that bottom-up programming is not a specialized technique for unusual situations but the natural style for any substantial Lisp program, and that Lisp — through macros, closures, and first-class functions — is uniquely suited to this style.

If you want to write software that will be around in twenty years, the way to do it is to write it bottom-up.

Chapter 1 — The Extensible Language

Central question

What makes Lisp fundamentally different from other languages, and why does that difference matter?

Main argument

Design by Evolution. Graham opens by observing that Lisp is not a fixed artifact but a living, evolvable medium. Most languages are designed once and then used; Lisp is designed continuously by the programmer as they use it. This capacity for evolution is not accidental — it flows from a structural property of the language.

Programming Bottom-Up. The conventional approach to software development is top-down: decompose the problem into modules until the modules are simple enough to implement directly. Bottom-up design inverts this. Rather than pulling the language up to the problem, experienced Lisp programmers build the language up toward the problem, adding new operators until the problem domain itself becomes directly expressible. The result is not only a solution but a language in which similar solutions become easy. Graham argues this is not a niche technique: "any substantial program will be written partly in this style."

Extensible Software. The mechanism that makes bottom-up design practical is Lisp's capacity for defining new operators — through both functions and macros — that are indistinguishable from built-ins. Unlike most languages, where the user and the language designer operate at different levels, in Lisp every programmer can extend the language itself.

Why Lisp is Uniquely Suited. Two structural properties underlie Lisp's extensibility: first, Lisp programs are expressed as Lisp data structures (lists), so a program can construct and transform other programs; second, macros operate at read/compile time on the source code itself, before evaluation. These two properties together mean that Lisp's abstractions are not limited to the runtime.

Dispelling the AI Association. Graham addresses the historical association between Lisp and artificial intelligence, calling it "just an accident of history." The real reason to use Lisp is its expressive power, not its historical application domain. He cites Emacs, AutoCAD, and Interleaf as evidence of Lisp's broader commercial viability.

Key ideas

  • Bottom-up programming means building the language up toward the problem, not just the program down toward the language.
  • Lisp's extensibility rests on the fact that Lisp programs are themselves Lisp data (lists), enabling programs to write programs.
  • New operators defined by the user are indistinguishable from built-in operators.
  • The result of good bottom-up programming is both a solution and a reusable language for the problem domain.
  • Lisp's association with AI is historical accident, not an inherent property.

Key takeaway

Lisp's defining strength is that it is an extensible language: the programmer builds the language toward the problem, not just the program, and macros are the primary mechanism that makes this possible.

Chapter 2 — Functions

Central question

How does treating functions as first-class data objects change the way programs are structured?

Main argument

Functions as Data. In Lisp, functions are objects like any other: they can be stored in variables, passed as arguments, returned from other functions, and created at runtime. This is not a trivial syntactic convenience — it fundamentally changes what programs can look like.

Defining Functions with lambda. Graham explains the lambda special form as the primitive for creating function objects, and shows how defun is a macro built on top of it. Understanding this relationship clarifies that named functions are not special — they are named lambda expressions stored in the function cell of a symbol.

Functional Arguments. Functions like mapcar, sort, and remove-if take other functions as arguments. This enables highly general, reusable abstractions. Graham walks through how to pass functions as arguments using #' (sharp-quote) and why this is preferred over symbol-passing.

Functions as Properties. Functions can be stored as properties on symbols, enabling a kind of ad-hoc dispatch. This foreshadows the discussion of object-oriented programming but also shows how polymorphism can be achieved without a formal object system.

Scope and Closures. Lexical (static) scoping means a function's free variables refer to the environment where the function was defined, not where it is called. When a function is created inside another function, it closes over the bindings visible at creation time, producing a closure — a function bundled with its environment. Closures are the central data structure of the first half of the book.

Local Functions. labels and flet define locally scoped functions, enabling recursion within closures and encapsulation of helper functions without polluting the global namespace.

Tail-Recursion. A function is tail-recursive when the recursive call is the last operation performed. Many Common Lisp implementations optimize tail calls into iteration, and Graham shows how to rewrite iterative processes as tail-recursive functions and how to force tail-call optimization with explicit trampolining.

Compilation. Graham explains how compiled functions work in Common Lisp and how the compiler can inline small functions for performance.

Key ideas

  • Functions are first-class objects: storable, passable, returnable, and creatable at runtime.
  • defun is syntactic sugar over lambda; the two are conceptually identical.
  • Lexical scoping means a function's free variables refer to the definition environment.
  • A closure captures its lexical environment and carries it as live, mutable state.
  • Higher-order functions (functions taking or returning functions) are the basic unit of abstraction.
  • Tail-recursive functions can be optimized to constant stack space.

Key takeaway

Functions in Lisp are data, and closures — functions bundled with their lexical environment — are the primary unit of abstraction that everything else in the book builds on.

Chapter 3 — Functional Programming

Central question

What does it mean to write programs in a functional style, and why does Lisp encourage it?

Main argument

Functional Design. A functional program is one that computes by returning values rather than by modifying state. "A functional program tells you what it wants; an imperative program tells you what to do." The functional style prefers pure functions — functions whose output depends only on their inputs and that produce no side effects — because pure functions are easier to test, compose, and reason about.

Imperative Outside-In. Pure functional programming is not always practical, but Graham argues that even in programs with unavoidable side effects, the bulk of the code can be written functionally. The approach is to push side effects to the outside of the program — into the input/output layer — while keeping the computational core purely functional.

Functional Interfaces. A function with a clean functional interface can hide imperative implementation details behind a pure-looking contract. Graham shows how to design modules that appear purely functional even when they internally use destructive operations, and why this discipline matters for program correctness.

Interactive Programming. Lisp's interactive development style — the REPL, incremental compilation, the ability to redefine functions at runtime — is not just a convenience. It fits naturally with functional programming because pure functions can be tested and reused in isolation without needing to reconstruct global state.

Key ideas

  • Functional programs compute by returning values; imperative programs compute by changing state.
  • Pure functions (no side effects, referentially transparent) are easier to test, compose, and understand.
  • Side effects should be pushed to the outer layers of a program; the computational core should be kept pure.
  • A function with a clean interface can hide imperative implementation details.
  • Lisp's interactive REPL style rewards the functional discipline.

Key takeaway

Writing functionally — favoring return values over side effects and pushing mutation to the program's edges — makes Lisp programs cleaner, more testable, and more composable.

Chapter 4 — Utility Functions

Central question

How and when should a programmer extend Common Lisp with their own utility functions?

Main argument

Birth of a Utility. A utility function is a function general enough to be used across many programs — a small piece of language extension. Graham argues that experienced Lisp programmers write many more such utilities than programmers in other languages, because the cost of abstraction in Lisp is low. A good utility is "a function you write once and can forget about."

Operations on Lists. Lisp's list operations are powerful but not complete. Graham develops utilities for operations on list tails (last-n), on list heads (nthmost), for filtering and searching, and for building new list forms. He shows the pattern of writing small combinators that operate on list structure.

Searching Lists. He extends Common Lisp's search functions with utilities that combine searching with transformation, enabling patterns like "find the element satisfying a predicate and return some function of it" in a single call.

Mapping and Combining. New mapping utilities go beyond mapcar and maplist to support more complex traversal patterns, including mapping functions that return multiple values or that operate on multiple lists simultaneously.

I/O, Symbols, and Density. Utilities for formatted output, for interning and creating symbols programmatically, and for making code more concise. Graham emphasizes code density — the ratio of information to syntactic noise — as a measure of a language's quality. The goal of utility programming is to raise this density.

When to Write a Utility. The chapter closes with practical advice: write a utility whenever you find yourself repeating the same pattern, but resist the temptation to over-generalize prematurely. The test is whether the utility is general enough to be useful across multiple programs.

Key ideas

  • Utilities are small, general functions that extend the language toward a problem domain.
  • Lisp encourages writing more utilities than other languages because abstraction is cheap.
  • Good utilities raise code density: more information per line of code.
  • Common patterns in utility writing: list manipulation, searching with transformation, mapping combinators.
  • Write a utility when a pattern recurs; resist premature over-generalization.

Key takeaway

Building a vocabulary of utility functions is the first step in bottom-up programming: a well-chosen set of utilities raises the language to a level where the problem becomes directly expressible.

Chapter 5 — Returning Functions

Central question

What new capabilities become available when functions can be built and returned at runtime?

Main argument

Closures as Active Objects. Closures are not just functions; they are functions with enclosed state. The chapter shows how closures can serve as lightweight objects — active, stateful entities that encapsulate behavior and data without requiring a full object system.

Complement and Composition. Graham introduces two fundamental function-building operations. complement takes a predicate and returns its logical negation: (complement #'oddp) returns a function equivalent to evenp. Composition (the compose utility) takes two or more functions and returns their composition: (compose #'car #'reverse) produces a function that returns the last element of a list. These are the basic combinators of functional programming.

Memoization. A memoize utility wraps a function so that the results of previous calls are cached. The first call with a given argument computes the result normally; subsequent calls return the cached value. This is a clean example of a closure carrying state: the cache is a hash table closed over by the memoized function.

Recursion on Function Builders. Functions can be built recursively. Graham shows how to build a function that generates the complement of the complement, or the composition of a list of functions, demonstrating that function-building follows the same recursive patterns as any other data manipulation.

When to Use Closures. The chapter closes with guidance on when to represent things as closures versus data structures. Closures are appropriate when behavior varies and the representation can be opaque; data structures are appropriate when the representation needs to be inspected or serialized.

Key ideas

  • Closures serve as lightweight stateful objects: functions that carry their own private state.
  • complement and compose are the fundamental building blocks of functional composition.
  • Memoization is naturally expressed as a closure over a cache table.
  • Functions that return functions can be built recursively, following the same patterns as recursive data structure manipulation.
  • Choose closures for opaque behavioral entities; choose data structures when the representation needs to be visible.

Key takeaway

Functions that create and return other functions — closures with enclosed state, combinators like complement and compose, memoizers — represent a style of programming where behavior itself is data, composable and transformable.

Chapter 6 — Functions as Representation

Central question

When is it better to represent a data structure as a function rather than as explicit data?

Main argument

Networks as Closures. Graham demonstrates that many data structures can be compiled into functions, making them faster to traverse and eliminating explicit structure-building at runtime. The running example is a network (directed graph): rather than representing nodes as records with pointers, each node is compiled into a closure that dispatches to the closures for its successors.

Three Properties of Closures. Closures are active (they can be called), they have local state (closed-over bindings), and multiple instances can be made from the same template. These three properties make them suitable for representing nodes in networks — entities that need individual identity and behavior.

Compiling Networks. The key technique is to take a data representation (a network described as a list of nodes and arcs) and compile it into a set of closures at load time. Traversal then becomes a series of function calls rather than data-structure lookups, which can be significantly faster.

When Functional Representation Wins. The representation-as-function approach is fastest when the network structure is fixed at compile time and queried frequently at runtime. It is inappropriate when the structure must be modified dynamically or when it needs to be inspected (you cannot easily examine the internals of a closure).

Key ideas

  • Closures can represent structured data (nodes, states, entities) that carries active behavior.
  • Compiling a static data structure into closures replaces data-structure traversal with function calls.
  • Three closure properties make them suitable for data representation: they are active, they carry state, and they can be instantiated multiple times.
  • Functional representation is best for fixed structures queried frequently; data structure representation is best for structures that need inspection or modification.

Key takeaway

When a data structure's shape is fixed at compile time and queried frequently at runtime, compiling it into closures eliminates traversal overhead and can yield significant performance gains.

Chapter 7 — Macros

Central question

What are macros, how do they work mechanically, and how does one write and test them correctly?

Main argument

Macros as Code Transformers. A macro is a function from code to code. When the Lisp compiler encounters a macro call, it calls the macro's expansion function with the unevaluated source forms as arguments, and the expansion function returns a new piece of Lisp code. The compiler then compiles that returned code in place of the original macro call. The key difference from a function: a function's arguments are evaluated before the function is called; a macro's arguments are passed as raw syntax.

The nil! Example. Graham introduces macros with a minimal example: nil! sets its argument to nil. Writing (nil! x) should expand to (setq x nil). This cannot be a function because x must not be evaluated before the assignment. The macro version uses defmacro and backquote to return the appropriate setq form.

Backquote. The backquote (`) notation is a template mechanism for constructing lists. Inside a backquoted expression, a comma (,) unsuppresses evaluation for the following subform, and comma-at (,@) splices a list in place. Backquote makes macro bodies readable by expressing the template of the output code directly.

Graham's Method for Writing Macros. Start with a typical call you want to support. Write it down. Below it, write the code it should expand into. Now write a macro body that transforms the first form into the second. This design-by-example discipline prevents macro-writing from becoming guesswork.

Testing Macros. Use macroexpand-1 to inspect what a macro call expands into before running it. Test the expansion before testing the behavior; a macro that expands correctly but runs incorrectly is easier to debug than one that expands incorrectly.

Macro Style. Macros should look like Lisp: their call syntax should be consistent with built-in special forms. The body should be as simple as possible. When a macro's logic becomes complex, factor the logic into helper functions that the macro calls.

Key ideas

  • A macro is a function from unevaluated source code to new source code; the compiler substitutes the expansion for the call.
  • Unlike functions, macro arguments are passed unevaluated — this is what makes macros more powerful than functions for certain tasks.
  • Backquote provides a template notation for building list structures; , unsuppresses evaluation, ,@ splices.
  • Write macros by first writing the desired call and the desired expansion, then automating the transformation.
  • Use macroexpand-1 to inspect and debug macro expansions before testing behavior.
  • Keep macro bodies simple; delegate complexity to helper functions.

Key takeaway

A macro is a code-transforming function that operates at compile time on unevaluated syntax; writing one correctly starts with a concrete example of the desired call and the desired expansion, and backquote provides the template notation for expressing the transformation.

Chapter 8 — When to Use Macros

Central question

When should a programmer reach for a macro instead of a function, and what are the costs of that choice?

Main argument

Four Reasons to Use Macros. Graham identifies the legitimate uses for macros: (1) to transform syntax — to create new control structures or binding forms that cannot be expressed as function calls because they need to defer evaluation; (2) to bind variables in the call's lexical environment — macros can introduce new bindings that are visible to the forms passed to them; (3) to inline code for performance — macros can expand into code that avoids the overhead of a function call; (4) to achieve computation at compile time — macros run at compile time and can shift work out of the runtime entirely.

When Functions Are Better. A function is simpler, more testable, and can be passed as an argument to higher-order functions. If the desired behavior can be achieved by a function, it should be. The diagnostic question is: "does this need to manipulate unevaluated code?" If no, write a function.

The Macro-or-Function Decision. Graham walks through examples of seemingly compelling cases for macros that turn out to be better served by functions, and cases that genuinely require macros. The key is not expressive power (you can always replace a macro with a function if you're willing to use #' and explicit lambda) but readability and call-site convenience.

Macros and Performance. Macros can expand into inlined code, avoiding function call overhead for very small, frequently-called operations. But this use of macros trades debuggability for performance and should be reserved for measured bottlenecks.

Key ideas

  • Use macros when you need to control evaluation (defer it, repeat it, or suppress it).
  • Use macros to introduce new binding forms or control structures.
  • Use macros when computation can and should happen at compile time.
  • Prefer functions when the behavior does not require manipulating syntax; functions are more composable and testable.
  • The test: "does this need to operate on unevaluated code?" If no, use a function.

Key takeaway

Macros are justified when the desired operation requires control over the evaluation of arguments; in all other cases, functions are simpler and more reusable.

Chapter 9 — Variable Capture

Central question

How can macros inadvertently bind or reference variables they should not, and how is this prevented?

Main argument

The Problem of Variable Capture. Variable capture occurs when a macro expansion introduces a binding that unintentionally captures — or is captured by — a variable in the calling code. Graham identifies two directions: macro argument capture, where a symbol passed by the caller is unintentionally bound inside the expansion; and free symbol capture, where a free variable inside the macro body unintentionally refers to a binding in the caller's environment.

A Concrete Example. Consider a for macro that iterates a variable over a range. If the macro internally uses a variable named limit for the upper bound and the caller also has a variable named limit, the macro's limit will shadow or interfere with the caller's. This is argument capture.

The Naming Convention Workaround. One approach is to give internal macro variables names unlikely to conflict with user code (e.g., names with unusual prefixes). Graham dismisses this as fragile: it relies on social convention rather than mechanical guarantees.

Gensyms. The correct solution is gensym, which generates a fresh, unique, uninterned symbol at macro-expansion time. By binding the macro's internal variables to gensyms, conflicts become structurally impossible: no code the user writes can name a gensym because gensyms are not interned and cannot be typed. The pattern is: at the start of a macro body, generate gensyms for every variable the macro introduces, then use those gensyms throughout the expansion template.

Packages as a Partial Solution. Placing macro internals in a separate package provides some protection against free-symbol capture, because the package qualifier makes the internal names distinct from any user-defined names in the default package.

Key ideas

  • Variable capture occurs when a macro expansion unintentionally interferes with variable bindings in the caller's scope.
  • Two directions: macro-internal variables shadowing caller's variables (argument capture); macro's free variables referring to caller's bindings (free-symbol capture).
  • Naming conventions offer fragile, social-convention-based protection.
  • gensym generates unique uninterned symbols, making conflicts structurally impossible.
  • Always use gensyms for every variable a macro introduces into the expansion.

Key takeaway

The correct defense against variable capture is gensym: generating fresh, uninterned symbols for every binding introduced by a macro expansion, making name conflicts structurally impossible.

Chapter 10 — Other Macro Pitfalls

Central question

Besides variable capture, what other errors do macros introduce that functions do not?

Main argument

Multiple Evaluation. If a macro's template evaluates one of its argument forms more than once, and that form has side effects, the side effects will occur multiple times — which is almost never intended. The classic case is a macro that both tests a value and assigns it, using the same subform twice in its expansion. The fix is to bind the argument to a gensymmed variable once, then use that variable in the expansion.

Wrong Order of Evaluation. Macros can accidentally evaluate their arguments in a different order than they appear in the call. This matters when the arguments have side effects or when one argument's value depends on a previous argument's side effect. Graham shows examples where restructuring the backquote template is the fix.

Macros Containing Free Variables. A free variable in a macro body refers to whatever binding is visible at the point where the macro is defined — specifically, in the macro's definition environment, which may not be what is intended if the macro is defined in one package and used in another. The fix is to qualify free variables explicitly or to move them into the macro's expansion as literals.

Recursion in Macros. A macro that calls itself recursively in its expansion must be careful: the expansion step is not the evaluation step, and an infinitely recursive expansion will cause the compiler to loop. Only recurse on a structurally smaller subform, or delegate recursion to a helper function.

Key ideas

  • Multiple evaluation: each argument form that appears more than once in the expansion will be evaluated that many times; bind arguments to gensymmed variables to evaluate them exactly once.
  • Evaluation order: the order of evaluation in the expansion must match the user's expectations, especially when side effects are involved.
  • Free variables in macro bodies refer to the macro-definition environment, not the call-site environment; qualify them explicitly.
  • Recursive macro expansions must terminate; factor recursion into helper functions when needed.

Key takeaway

Beyond variable capture, the main macro bugs are multiple evaluation (fixed by gensymming arguments), wrong evaluation order (fixed by careful template structuring), and unintended free variable references (fixed by explicit qualification).

Chapter 11 — Classic Macros

Central question

What are the foundational macro patterns that every Lisp programmer should know?

Main argument

Macros for Context Creation. The first major category is macros that establish a dynamic context: they set up some state, evaluate a body of forms, and then clean up. Common Lisp's with-open-file is the canonical example. Graham shows how to write similar macros for establishing database connections, acquiring locks, or setting up test environments. The pattern is: (with-something setup body teardown), where the macro ensures teardown happens even if the body signals an error.

Conditional Evaluation Macros. The second category is macros that evaluate forms conditionally: when, unless, cond, and their variants. These exist because if is the only primitive conditional and the others reduce syntactic noise. Graham writes cleaner versions and shows how they compose.

Iteration Macros. The third category is iteration: do, dotimes, dolist, loop, and custom iteration constructs. Graham shows how to build a general for macro and how to define iterators that walk custom data structures. The key pattern is the iterator macro: a macro that repeatedly evaluates a body in a context that advances some state.

Combining Patterns. Real macros often combine multiple patterns: a context-establishing macro that also iterates, or an iteration macro that introduces conditional exit. Graham walks through increasingly complex compositions to show how the patterns combine.

Key ideas

  • Context-creation macros (with-X patterns) establish, maintain, and clean up dynamic state around a body of code.
  • Conditional-evaluation macros eliminate syntactic noise around common conditional patterns.
  • Iteration macros drive computation over sequences, ranges, or custom structures.
  • These three patterns — context, condition, iteration — cover the majority of practical macro use cases.
  • Macro patterns compose: context-establishing and iterating macros can be combined.

Key takeaway

The three canonical macro patterns — context creation, conditional evaluation, and iteration — cover the bulk of practical macro programming and provide reusable templates for building problem-specific control structures.

Chapter 12 — Generalized Variables

Central question

How does Common Lisp's setf generalize assignment, and how can programmers extend it?

Main argument

Setf as Inversion. In Common Lisp, setf is not a simple variable assignment; it is a generalized assignment that works on any place — any form that designates a settable location. (setf (car x) y) sets the car of x; (setf (gethash key table) value) sets a hash table entry. The mechanism behind this is that each place form has an associated setter function.

Places as Invertible Expressions. Graham explains the conceptual model: a generalized variable is a form that can be used as both a getter (to read a value) and a setter (to store one). setf works by finding the inverse of the getter expression and calling the setter with the new value.

define-modify-macro. For most cases where the programmer wants to define a new modify macro — one that reads a place, transforms its value, and writes it back — define-modify-macro provides a declarative way to do this. Graham shows how incf and push are instances of this pattern.

Building Custom Setf Expanders. For more complex cases, defsetf and define-setf-method (the lower-level mechanism) allow the programmer to define exactly how setf should expand for a new kind of place. Graham shows conc1f (append one element to a list in place) and concnew (like pushnew but appending) as examples.

Key ideas

  • setf generalizes assignment to any place: any form that names a readable, writable location.
  • The mechanism is inversion: setf finds the setter corresponding to the getter form and calls it.
  • define-modify-macro handles the common pattern of read-transform-write for simple modifications.
  • defsetf and define-setf-method handle complex cases where the expansion requires multiple sub-forms.
  • Extending setf creates new domain-specific assignment forms that integrate cleanly with existing Lisp idioms.

Key takeaway

setf is a generalized, extensible assignment mechanism that works on any user-defined place, enabling programmers to add new assignment forms that integrate cleanly with the rest of Common Lisp.

Chapter 13 — Computation at Compile-Time

Central question

How can macros move computation from runtime to compile time, and when is this worthwhile?

Main argument

The Basic Idea. Because macros run at compile time, any computation they perform happens once — when the program is compiled — rather than every time the code runs. If a computation's inputs are known at compile time and its outputs are needed at runtime, a macro can perform the computation during compilation and embed the results directly in the compiled code.

The Bézier Curve Example. Graham's central example is a macro that generates code to compute a point on a Bézier curve. The Bézier formula involves polynomial coefficients — combinations of the control point coordinates — that depend only on the parametric value t and the number of control points, not on runtime data. The macro takes the degree of the curve as a compile-time parameter and emits optimized, straight-line arithmetic for that specific degree, unrolling the polynomial and computing intermediate powers of t at compile time where possible.

When Compile-Time Computation Is Appropriate. The technique is justified when (1) the inputs to the computation are known at compile time; (2) the computation is expensive relative to the cost of a function call; (3) the results are needed in tight loops. It is not appropriate when it obscures the code without measurable benefit.

Interaction with the Compiler. Graham notes that modern Lisp compilers already perform constant folding and other optimizations. Compile-time macros supplement (rather than duplicate) compiler optimizations when the compiler lacks the domain knowledge to recognize that a particular computation is constant.

Key ideas

  • Macros run at compile time; computation inside a macro body is paid once at compile time rather than every time the generated code runs.
  • Use compile-time computation when inputs are compile-time constants and the computation is non-trivial.
  • The Bézier curve macro generates degree-specific polynomial code, eliminating runtime loop overhead.
  • This technique is complementary to, not a substitute for, compiler optimizations.

Key takeaway

When the inputs to a computation are fixed at compile time, a macro can perform the computation during compilation and embed the results directly in the generated code, eliminating runtime overhead entirely.

Chapter 14 — Anaphoric Macros

Central question

How can macros introduce intentional variable bindings that make code more concise — and when is this a good idea?

Main argument

Linguistic Anaphora. In natural language, an anaphor is a word (like "it" or "that") that refers back to a previously mentioned entity. Graham borrows this idea for macros: an anaphoric macro intentionally binds a special symbol (typically it) to the value of a subform, making that value available in the macro's body without an explicit let.

aif — Anaphoric If. The canonical example is aif:

(defmacro aif (test then &optional else)
  `(let ((it ,test))
     (if it ,then ,else)))

With aif, the expression (aif (find-if #'goodp lst) (process it)) avoids the clunky (let ((result (find-if #'goodp lst))) (if result (process result))). The variable it is bound to the result of the test form and is available in the then clause.

awhen, awhile, aand, acond. Graham extends the anaphoric pattern to other control forms: awhen (like when but binding it), awhile (like while but binding it to the condition's value each iteration), aand (anaphoric and, where it refers to the previous conjunct's value), and acond (anaphoric cond).

Intentional Capture. Anaphoric macros represent intentional variable capture — a deliberate design decision, not a bug. This directly inverts the lesson of Chapter 9, which taught how to prevent capture. The distinction is crucial: accidental capture is a bug; intentional, documented capture is a design tool.

The Tradeoffs. Anaphoric macros make code more concise but introduce an implicit binding that may surprise readers unfamiliar with the convention. They should be used in contexts where the anaphor's meaning is clear from context and documented at the point of use.

Key ideas

  • Anaphoric macros intentionally bind a special symbol (it) to a subform's value, making it available without explicit binding.
  • aif is the canonical example: it is bound to the test's value, usable in the then clause.
  • The pattern extends to awhen, awhile, aand, and acond.
  • Intentional capture (anaphoric macros) is a design tool; accidental capture (Chapter 9) is a bug.
  • Anaphoric macros trade readability for unfamiliar readers against conciseness for familiar ones.

Key takeaway

Anaphoric macros use intentional variable capture to bind the symbol it to a subform's value, eliminating boilerplate let bindings and making certain conditional patterns significantly more concise.

Chapter 15 — Macros Returning Functions

Central question

How can macros and functions be combined so that macros generate function-returning code at compile time?

Main argument

The Combination. Chapters 5–6 showed that functions can build and return other functions. Chapters 7–14 showed that macros operate at compile time on code. This chapter combines the two: macros that expand into code that creates closures. The result is a tool for producing highly optimized, customized functions at compile time.

Abbreviations for Function Builders. One use is syntactic sugar for patterns that create functions. Rather than writing an explicit lambda with a let for memoization or complement, a macro can provide a cleaner calling syntax.

Compile-Time Function Generation. More powerfully, a macro can generate a specialized function by unrolling loops, inlining constants, or specializing for a known argument type — producing a function faster than a general-purpose one — and do so at compile time.

The f-functions. Graham introduces a series of utilities (fif, fint, fun) that use macros to provide convenient notation for building closures, bridging the gap between macro syntax and function-building idioms.

Key ideas

  • Macros can expand into code that creates closures, combining compile-time code generation with runtime function creation.
  • This enables syntactic sugar for common function-building patterns.
  • Compile-time specialization can generate faster functions by inlining constants or unrolling loops.
  • The combination of macros and closures is more powerful than either alone.

Key takeaway

Macros can generate closures at compile time, enabling compile-time specialization of functions and cleaner syntactic notation for common function-building patterns.

Chapter 16 — Macro-Defining Macros

Central question

Can macros themselves be used to define other macros, and what does this enable?

Main argument

Macros That Write Macros. Just as a macro is a program that writes programs, a macro-defining macro is a program that writes macros. This is the highest level of the bottom-up stack: rather than writing individual macros by hand, the programmer writes a macro that generates a family of related macros from a compact specification.

The Motivation: Boilerplate in Macro Definitions. Many macros follow identical structural patterns: generate a gensym for each argument, bind the arguments to gensyms, construct a backquoted template. When a programmer writes the tenth macro following this pattern, the pattern itself becomes the bottleneck. A macro-defining macro eliminates this.

abbrev and abbrevs. Graham's central example is abbrev, a macro that takes an abbreviation name and a longer expression and defines a new macro that expands the abbreviation. abbrevs extends this to define multiple abbreviations at once. These are meta-macros: macros whose bodies produce defmacro forms.

defmacro/g!. Graham shows a pattern where a special naming convention (variables beginning with g!) triggers automatic gensym generation, and a macro-defining macro implements this convention — a meta-macro that rewrites all g!-prefixed variables to gensym calls before emitting the defmacro.

Key ideas

  • A macro-defining macro generates defmacro forms, producing families of related macros from a single compact specification.
  • Macro boilerplate (gensym generation, template structure) can itself be abstracted over with a meta-macro.
  • abbrev/abbrevs exemplify macros that generate simple expanding macros.
  • Naming conventions can be enforced by meta-macros: a g!-variable convention triggers automatic gensym insertion.
  • This is the logical endpoint of bottom-up programming: extending the macro-definition level itself.

Key takeaway

Macro-defining macros eliminate boilerplate in macro definitions by generating families of macros from compact specifications, pushing bottom-up abstraction to the level of the macro system itself.

Chapter 17 — Read-Macros

Central question

How can the Lisp reader be extended to recognize new syntax before code is even parsed?

Main argument

The Reader as Extensible Stage. Lisp's processing pipeline has three stages: read (text to s-expressions), compile (s-expressions to code), and eval (code to values). Macros operate at the compile stage. Read-macros operate one stage earlier, at the read stage: they tell the reader to interpret certain characters or character sequences in special ways.

Macro Characters. Common Lisp's reader dispatches on macro characters — characters that trigger a reader function when encountered. The existing macro characters include ' (quote), ` (backquote), , (comma), # (dispatch), and ; (comment). Programmers can define new macro characters with set-macro-character.

The #' Notation. Graham explains how #' — the function quotation notation — is itself a read-macro: when the reader sees #', it reads the following form and wraps it in (function ...). This is a read-time transformation, not a compile-time one.

Defining New Read-Macros. Graham shows how to define a new macro character and how to use the # dispatch character to create new #X sequences. The example is a notation for writing certain common expressions in a more compact syntax.

Cautions. Read-macros are powerful but should be used sparingly: they modify the reader globally and can make code unreadable to programmers unfamiliar with the extension. They are most appropriate for libraries that define a complete sublanguage with its own notation.

Key ideas

  • Read-macros operate at the read stage (text to s-expressions), earlier than compile-time macros.
  • Macro characters trigger reader functions that transform the character stream into s-expressions.
  • Existing macro characters: ', `, ,, ;, #.
  • New macro characters are defined with set-macro-character; new #X sequences use the dispatch mechanism.
  • Read-macros are powerful but globally scoped; use them sparingly and document them carefully.

Key takeaway

Read-macros extend the Lisp reader itself, enabling new surface syntax before code reaches the compiler; they are the deepest level of language extension but carry the highest cost in readability for unfamiliar readers.

Chapter 18 — Destructuring

Central question

How can pattern-matching on structure be used in macro definitions to make them more expressive?

Main argument

Destructuring as Pattern Matching. Destructuring is the operation of binding variables to corresponding parts of a structured value according to a pattern. Common Lisp's destructuring-bind matches a pattern (a possibly nested list structure) against a value and binds variables to the matching parts. This is particularly useful in macros, where the arguments arrive as structured s-expressions.

Destructuring in Lambda Lists. Common Lisp's macro lambda lists already support destructuring: a macro can declare that its argument is a list of two elements by writing (a b) in the lambda list position, and Common Lisp will automatically bind a and b to the car and cadr of the supplied argument.

Building a Destructuring Utility. Graham extends the built-in destructuring to handle more complex patterns, including patterns with optional components, rest patterns, and nested structures. He builds a dbind macro that extends destructuring-bind with these additional pattern types.

Pattern Matching Beyond Destructuring. The chapter closes with a discussion of how full pattern matching — where patterns can include literals to be matched, not just variables to be bound — goes beyond destructuring and points toward the query compiler in Chapter 19.

Key ideas

  • Destructuring binds variables to parts of a structured value according to a positional pattern.
  • Common Lisp macro lambda lists support destructuring natively, making complex macro argument parsing cleaner.
  • destructuring-bind performs structural matching and binding at runtime.
  • Extended patterns (optional components, rest patterns, nested structures) require a custom dbind utility.
  • Full pattern matching adds literal value matching to structural binding.

Key takeaway

Destructuring provides declarative pattern-based binding that is especially valuable in macros, where arguments arrive as structured s-expressions; extending it toward full pattern matching is the bridge to embedded query languages.

Chapter 19 — A Query Compiler

Central question

How can macros be used to build an embedded query language for in-memory databases?

Main argument

Embedded Languages via Macros. This chapter is the first of several that use macros to implement a complete embedded domain-specific language (DSL). The language here is a SQL-like query language for querying Lisp data structures. The point is not the query language itself but the technique: macros can translate a high-level, domain-specific notation into efficient Lisp code.

The Database and Query Forms. Graham builds a simple in-memory database: a set of facts represented as lists. Queries are expressed as patterns: (with-answer (parent ?x Adam) ...) finds all bindings of ?x such that the fact (parent ?x Adam) is in the database.

Interpretation vs. Compilation. Graham first implements the query language as an interpreter — a function that takes a query pattern and walks the database. He then replaces it with a compiler — a macro that takes the same query pattern and emits Lisp code that performs the search directly, without the interpreter overhead. The compiled version is faster because the structure of the query is known at compile time.

Pattern Variables and Binding. Variables in query patterns (marked with ?) are bound by the matching process. The compiler generates let-binding code that accumulates variable bindings as it traverses the database, emitting code for each conjunct in the query.

The Significance of the Example. The query compiler demonstrates the central thesis: by implementing the same functionality first as an interpreter and then as a compiler, Graham shows concretely what "raising the language to the problem" means in practice, and how macros make this practical.

Key ideas

  • Macros can implement embedded query languages by translating domain-specific notation into efficient Lisp code at compile time.
  • A database query is represented as a pattern; pattern variables (?x) are bound by the matching process.
  • The interpreter approach evaluates queries at runtime; the compiler approach transforms queries into Lisp code at compile time.
  • Compilation of queries eliminates interpreter overhead when query structure is known at compile time.
  • This is the first concrete example in the book of a full embedded DSL implemented through macros.

Key takeaway

By compiling query patterns into Lisp code at macro-expansion time, Graham demonstrates how macros raise the language to a problem domain, turning a runtime interpretation cost into a compile-time translation.

Chapter 20 — Continuations

Central question

What is a continuation, and how can Common Lisp programs simulate first-class continuations through macros?

Main argument

What a Continuation Is. A continuation is the remainder of a computation at a given point. At any moment during program execution, "what remains to be done" can be captured as a continuation object. Resuming the continuation resumes the computation from that point. Languages like Scheme provide call/cc (call-with-current-continuation) as a primitive for capturing continuations; Common Lisp does not.

CPS Transformation. Graham shows how any program can be mechanically transformed into continuation-passing style (CPS): every function is given an extra argument — the continuation — which it calls (rather than returning a value) when it has a result. In CPS, control flow is entirely explicit: tail calls are the only form of transfer, and the continuation represents everything that would have happened after the current call.

Implementing CPS with Macros. Since Common Lisp lacks call/cc, Graham builds a set of macros (=defun, =lambda, =bind, =values, =funcall) that transform function definitions and calls into CPS form. Code written with these macros can use continuations without the language providing them natively.

How the Macros Work. =defun defines a function that takes an implicit continuation argument. =bind (analogous to a function call that captures what follows as a continuation) wraps the remainder of the calling code into a closure and passes it as the continuation argument. =values sends a result to the current continuation.

Practical Limitations. The CPS macros only work for code that uses them throughout — CPS and ordinary calling conventions cannot be freely mixed. The transformation is also incomplete: it does not handle all Common Lisp control flow (e.g., conditions and restarts). Despite this, it is sufficient to implement multiple processes (Chapter 21) and nondeterminism (Chapter 22).

Key ideas

  • A continuation captures "what remains to be done" at a point in a computation, enabling non-local transfers of control.
  • CPS transformation makes continuations explicit: every function takes a continuation argument and calls it instead of returning.
  • Common Lisp lacks call/cc; Graham's =defun/=bind/=values macros simulate CPS within Common Lisp.
  • CPS code can only inter-operate with other CPS code; the convention must be used consistently.
  • Continuations are the foundation for multiple processes (Chapter 21) and nondeterministic choice (Chapter 22).

Key takeaway

Continuations represent the resumable remainder of a computation; Graham implements them in Common Lisp via CPS-transforming macros, making it possible to capture and invoke continuations without native language support.

Chapter 21 — Multiple Processes

Central question

How can continuations be used to implement cooperative multitasking in Common Lisp?

Main argument

Processes from Continuations. With continuations in hand (from Chapter 20), cooperative multitasking becomes straightforward: a process is simply a suspended continuation. Switching from one process to another means capturing the current continuation and resuming another.

The Process Queue. Graham builds a simple process scheduler: a queue of suspended continuations. Each entry in the queue is a continuation representing a paused process. The scheduler runs continuations in round-robin order; when a process "yields", it captures its continuation and appends it to the queue, then runs the next continuation from the front.

fork and yield. fork creates a new process by capturing the current continuation and scheduling both the new process (which continues from the fork point) and the current process. yield suspends the current process and transfers control to the next process in the queue. These two primitives, implemented as macros over the CPS machinery of Chapter 20, are sufficient to write simple concurrent programs.

Simulated vs. True Parallelism. The processes implemented here are cooperatively scheduled — they only switch at explicit yield points — and run on a single thread. This is not true parallelism but is sufficient for coroutine-style programming and for the nondeterminism engine of Chapter 22.

Key ideas

  • A process is a suspended continuation; cooperative multitasking is continuation-passing in a queue.
  • The process scheduler runs continuations from a queue in round-robin order.
  • fork creates a new process by duplicating the current continuation; yield suspends the current process and transfers control.
  • The implementation is cooperative (processes yield explicitly) and single-threaded, not truly parallel.
  • Built entirely on the CPS machinery of Chapter 20; continuations are the enabling abstraction.

Key takeaway

Cooperative multitasking is a natural application of continuations: a process is a suspended continuation in a queue, and switching processes means resuming the next continuation.

Chapter 22 — Nondeterminism

Central question

How can a Lisp program express nondeterministic choice — the ability to make a choice and backtrack if it leads to failure?

Main argument

Nondeterminism as a Programming Paradigm. A nondeterministic program can make a choice among alternatives and, if the choice leads to failure, automatically backtrack and try the next alternative. From the programmer's perspective, it is as if the program always makes the right choice — the machinery of backtracking is hidden. This is a powerful paradigm for search problems.

choose and fail. The two primitives are choose and fail. choose takes a set of alternatives and "chooses" one to try. fail signals that the current choice led nowhere and triggers backtracking to the most recent choose, which then tries the next alternative. Together they implement depth-first search with backtracking.

Scheme Implementation First. Graham first implements choose and fail in Scheme, where call/cc is available natively. The implementation is clean: fail is a global continuation that jumps back to the last choose; each choose sets fail to a continuation that tries the next alternative.

Common Lisp Implementation via CPS Macros. Without native continuations, Graham reimplements choose and fail using the CPS macros of Chapter 20. The implementation is more complex but structurally identical. The macros =defun, =bind, and =values do the work of continuation-passing; choose and fail manipulate a stack of failure continuations.

The cut Operator. Prolog's cut operator (!) prunes the search space by preventing backtracking past a certain point. Graham shows that cut can be understood independently of Prolog as an operation on the failure continuation stack, and implements it in his nondeterminism framework.

A Practical Example. The two-numbers function nondeterministically chooses two numbers from a set; parlor-trick finds two numbers that sum to a given value by calling two-numbers and then checking the sum, relying on backtracking to explore all combinations.

Key ideas

  • Nondeterministic programming provides choose (pick an alternative) and fail (backtrack) as primitives; the programmer ignores the search mechanism.
  • fail invokes the most recent failure continuation, backtracking to the last choose.
  • The Scheme implementation uses native call/cc; the Common Lisp implementation uses the CPS macros from Chapter 20.
  • cut prunes the search space by removing failure continuations from the stack.
  • Nondeterminism enables concise expression of combinatorial search and constraint satisfaction.

Key takeaway

Nondeterminism is implemented as a search strategy over a stack of failure continuations: choose pushes alternatives, fail pops and tries them, and the programmer writes as if choices are always optimal.

Chapter 23 — Parsing with ATNs

Central question

How can the nondeterminism engine from Chapter 22 be used to build a natural-language parser?

Main argument

Augmented Transition Networks. An Augmented Transition Network (ATN) is a formalism for describing the grammar of a language as a directed graph. Nodes represent parsing states; arcs represent grammatical transitions (match this word, call this sub-network, etc.). ATNs were developed by William Woods in the late 1960s and were widely used in early natural-language processing systems.

ATNs as Nondeterministic Programs. An ATN parser is inherently nondeterministic: at each node, multiple arcs may be applicable, and the parser must try each one and backtrack if it leads to failure. The nondeterminism engine of Chapter 22 is exactly the right substrate: choose selects among applicable arcs; fail backtracks when an arc leads to a dead end.

Implementing ATNs in ~50 Lines. Graham shows that an ATN parser can be implemented in roughly 50 lines of Common Lisp, using the nondeterminism macros. The parser represents ATN states as functions and arcs as nondeterministic choices among function calls. The result is a complete, working parser for a subset of English.

Registers and Sentence Structure. ATNs use registers to accumulate structural information about the sentence being parsed — the subject, the verb, the object — as arcs are traversed. At the end of a successful parse, the registers contain the parsed structure. Graham's implementation uses closed-over variables as registers.

Key ideas

  • ATNs model grammar as a directed graph; parsing is traversal of the graph with backtracking.
  • The nondeterminism engine provides choose/fail, which are exactly what ATN traversal needs.
  • The implementation is approximately 50 lines of code, enabled by the nondeterminism machinery.
  • Registers accumulate parse structure as traversal proceeds; successful parse returns the register contents.
  • This is the first demonstration that the nondeterminism engine has practical embedded-language applications.

Key takeaway

An ATN parser is nondeterministic graph traversal, and the nondeterminism engine from Chapter 22 reduces a complete ATN parser to roughly 50 lines of code, demonstrating the power of embedded control abstractions.

Chapter 24 — Prolog

Central question

How can a complete Prolog interpreter be embedded inside Common Lisp using macros?

Main argument

Prolog as a Logic Programming Language. Prolog is a logic programming language based on Horn clauses and unification. A Prolog program is a set of facts and rules; a query asks Prolog to find bindings for variables that make a goal provable. Prolog's execution model is depth-first search with backtracking — the same mechanism as Chapter 22's nondeterminism engine.

Embedding Prolog in Lisp. Rather than writing a Prolog interpreter, Graham shows how to embed Prolog's programming model directly in Common Lisp as a set of macros. Code written in the embedded Prolog can call Common Lisp functions; Common Lisp code can invoke the embedded Prolog interpreter. The boundary between the two languages is transparent.

Unification. The core of Prolog is unification: the process of finding a substitution for variables that makes two terms equal. Graham implements a standard unification algorithm: two terms unify if they are identical, if one is a variable (which gets bound to the other), or if they are both compound terms whose corresponding subterms unify.

The Implementation: ~100 Lines. The embedded Prolog is implemented in approximately 100 lines. Facts and rules are asserted with <- (analogous to Prolog's :-). Queries are run with ?-. The implementation uses the nondeterminism engine of Chapter 22 for backtracking search, and unification for matching.

Cut and Negation. Graham implements Prolog's cut operator (which prunes the search space) and discusses how negation-as-failure can be added. Both are natural extensions of the nondeterminism framework.

Key ideas

  • Prolog's execution model (depth-first search with backtracking) is exactly the nondeterminism engine from Chapter 22.
  • Embedding Prolog in Lisp means translating Prolog syntax into Lisp macros and using Lisp data as Prolog terms.
  • Unification is the matching algorithm: find variable bindings that make two terms identical.
  • The complete implementation is approximately 100 lines, built on the earlier chapters' machinery.
  • The embedded Prolog can call Common Lisp functions transparently, and vice versa.

Key takeaway

A complete, embeddable Prolog interpreter reduces to approximately 100 lines of Common Lisp when built on the nondeterminism and CPS foundations of the preceding chapters, demonstrating the full power of bottom-up language construction.

Chapter 25 — Object-Oriented Lisp

Central question

What is the relationship between object-oriented programming and Lisp, and how does CLOS extend the basic ideas?

Main argument

OOP as Extension of Lisp. Graham argues that object-oriented programming, far from being foreign to Lisp, is a natural extension of ideas already present in it. Closures are already lightweight objects; property lists provide a form of slot-based storage; generic dispatch is a generalization of Lisp's function dispatch. CLOS formalizes and extends these ideas.

Building OOP from Scratch. The chapter begins by constructing an object system incrementally. Starting with hash tables as objects (slots are hash entries), Graham adds: (1) inheritance — objects can delegate slot lookups to parent objects; (2) method dispatch — functions that dispatch on the type of their argument; (3) multiple inheritance — an object can have multiple parents, and slot lookup follows a method resolution order (MRO).

From Single to Multiple Inheritance. Single inheritance makes the parent a single object reference. Multiple inheritance makes it a list. The MRO determines precedence when multiple parents provide the same slot or method. Graham builds a topological sort to compute the MRO, matching CLOS's approach.

Generic Functions and Methods. In CLOS (unlike Smalltalk-style OOP), methods do not belong to classes; they belong to generic functions. A generic function dispatches to the most specific applicable method based on the types of all its arguments, not just the first. Graham builds a simplified version and shows how defmethod and defgeneric work.

Method Combination. CLOS's method combination allows multiple applicable methods to contribute to a call: primary methods provide the main behavior; :before, :after, and :around methods provide pre/post processing and wrapping. Graham shows how method combination can be understood as evaluating a Lisp expression whose arguments are calls to the applicable methods in specificity order.

Key ideas

  • OOP in Lisp is a natural extension of closures, property lists, and function dispatch — not a foreign paradigm.
  • Building OOP from scratch clarifies what CLOS provides and why each piece exists.
  • Multiple inheritance requires a method resolution order (MRO) computed by topological sort.
  • CLOS's generic functions dispatch on the types of all arguments, not just the first.
  • Method combination (:before/:after/:around) enables layered, composable behavior.

Key takeaway

Object-oriented programming in Lisp is not a separate paradigm but an extension of existing ideas — closures, property lists, generic dispatch — and CLOS formalizes this extension into a systematic, multi-dispatch object system with composable method combination.

The book's overall argument

  1. Chapter 1 (The Extensible Language) — establishes that Lisp's core strength is its extensibility: the programmer builds the language upward toward the problem through functions and macros, enabling bottom-up design.
  2. Chapter 2 (Functions) — lays the foundation by showing that functions are first-class objects and that closures — functions bundled with their lexical environment — are the primary unit of abstraction.
  3. Chapter 3 (Functional Programming) — argues that the functional style (computing by returning values rather than modifying state) is the natural style for Lisp and produces cleaner, more composable programs.
  4. Chapter 4 (Utility Functions) — demonstrates bottom-up programming at the simplest level: building a vocabulary of utility functions that raise the language toward the problem.
  5. Chapter 5 (Returning Functions) — extends functional programming to functions that build functions: complement, compose, and memoize show that behavior itself is data to be constructed and combined.
  6. Chapter 6 (Functions as Representation) — pushes closures further, showing that data structures can be compiled into closures, making traversal faster and representation more active.
  7. Chapter 7 (Macros) — introduces the book's central topic: macros are programs that write programs, operating on unevaluated syntax at compile time; backquote provides the notation for expressing code transformations.
  8. Chapter 8 (When to Use Macros) — calibrates the tool by identifying the four legitimate uses of macros and emphasizing that functions remain preferable when they suffice.
  9. Chapter 9 (Variable Capture) — identifies the most important macro pitfall and provides the structural fix: gensyms eliminate accidental variable capture entirely.
  10. Chapter 10 (Other Macro Pitfalls) — completes the safety picture by addressing multiple evaluation, evaluation order bugs, and free-variable reference errors.
  11. Chapter 11 (Classic Macros) — presents the three reusable macro patterns: context creation, conditional evaluation, and iteration.
  12. Chapter 12 (Generalized Variables) — shows how setf can be extended to make any place assignable, enabling domain-specific assignment operators.
  13. Chapter 13 (Computation at Compile-Time) — demonstrates that macros can shift computation from runtime to compile time when inputs are known early, illustrated by the Bézier curve macro.
  14. Chapter 14 (Anaphoric Macros) — introduces intentional variable capture as a design tool: anaphoric macros bind it to eliminate boilerplate, drawing the crucial distinction between intended and accidental capture.
  15. Chapter 15 (Macros Returning Functions) — combines the first and second parts of the book: macros that generate closures, enabling compile-time specialization of runtime behavior.
  16. Chapter 16 (Macro-Defining Macros) — reaches the highest level of abstraction: macros that generate macros, eliminating boilerplate in macro definitions themselves.
  17. Chapter 17 (Read-Macros) — extends language extension to the read stage, the deepest level, where surface syntax itself can be customized.
  18. Chapter 18 (Destructuring) — shows how pattern-based binding in macro lambda lists makes complex syntax cleaner and points toward full pattern matching.
  19. Chapter 19 (A Query Compiler) — first full embedded DSL: a macro-compiled query language for in-memory databases, demonstrating the interpreter-to-compiler translation concretely.
  20. Chapter 20 (Continuations) — introduces the theory and practice of continuations, implementing CPS transformation via macros to simulate call/cc in Common Lisp.
  21. Chapter 21 (Multiple Processes) — applies continuations to cooperative multitasking: processes are suspended continuations in a scheduler queue.
  22. Chapter 22 (Nondeterminism) — builds the most powerful embedded control abstraction: choose/fail implement depth-first search with backtracking over a stack of failure continuations.
  23. Chapter 23 (Parsing with ATNs) — applies nondeterminism to natural language parsing: an ATN parser in ~50 lines, built directly on the choose/fail engine.
  24. Chapter 24 (Prolog) — the climax of the embedded-languages sequence: a complete Prolog interpreter in ~100 lines, built on unification plus the nondeterminism engine.
  25. Chapter 25 (Object-Oriented Lisp) — closes the circle by showing that CLOS, far from being foreign to Lisp, is another instance of the same bottom-up pattern: building a powerful system incrementally from simple Lisp primitives.

Common misunderstandings

Misunderstanding: Macros are just syntactic sugar for functions.

Macros and functions are fundamentally different: functions receive evaluated arguments and return values; macros receive unevaluated source code and return new source code that the compiler then compiles. Macros operate at compile time on syntax; functions operate at runtime on values. The distinction is not syntactic convenience but computational stage.

Misunderstanding: Variable capture is always a bug.

Chapter 9 describes variable capture as a macro pitfall, and Chapter 14 describes intentionally captured variables (anaphoric macros) as a design tool. The distinction is between accidental capture (a bug, fixed with gensyms) and intentional, documented capture (a feature, as in aif's it binding). The book explicitly uses both.

Misunderstanding: Bottom-up programming means not having a plan.

Bottom-up programming is not the absence of design; it is a different design discipline. Rather than decomposing the problem into modules upfront and then coding those modules, the programmer incrementally builds abstractions that make the problem progressively more expressible. The design is emergent but disciplined.

Misunderstanding: On Lisp is about Common Lisp syntax and idioms.

The book is about programming technique at a level above any particular syntax. Macros, continuations, closures, and embedded languages are the subject; Common Lisp is the medium. Many of the ideas transfer directly to other Lisp dialects (Scheme, Clojure) and inform the macro systems of non-Lisp languages.

Misunderstanding: The embedded Prolog and ATN parser are toys.

The embedded languages in Chapters 23–24 are complete, working implementations in roughly 150 combined lines of code. They are not pedagogical sketches. The point is that the techniques developed in the preceding 22 chapters reduce these systems to trivial size. A system that would require hundreds of lines without macros and continuations requires tens of lines with them.

Misunderstanding: Lisp macros are the same as C preprocessor macros.

C preprocessor macros are text-substitution rules that operate on character streams before parsing. Lisp macros are programs that operate on fully parsed s-expressions (structured Lisp data) and return new s-expressions. Lisp macros have access to the full power of Lisp during expansion; C macros do not. The result is that Lisp macros can be made hygienic (via gensyms), can compute arbitrarily complex transformations, and can be debugged with macroexpand-1.

Central paradox / key insight

The deepest insight in On Lisp is the collapse of the distinction between programs and data. In most languages, programs and data are distinct categories: programs manipulate data, but they cannot easily manipulate other programs. In Lisp, programs are data — s-expressions — and macros are programs that take programs (s-expressions) as input and produce programs (s-expressions) as output. This means the programming language is extensible from within: the programmer can add new syntax, new control structures, new evaluation strategies, and new language features without modifying the language implementation.

The paradox is that this power comes with no additional mechanism. There is no "meta-level" in Lisp that is separate from the object level. The same data structures used to represent lists of numbers are used to represent programs. The same car, cdr, and cons operations used to manipulate lists are used inside macros to manipulate program syntax. The distinction between writing a function and writing a language feature collapses.

"Lisp is not a language fixed in stone. You can add new operators to Lisp which are indistinguishable from the ones that come built-in."

This is why Graham argues that bottom-up programming is not a technique but a philosophy: in Lisp, the boundary between programmer and language designer does not exist.

Important concepts

Bottom-up programming

A programming style in which the programmer builds the language upward toward the problem domain (adding new operators, control structures, and abstractions) rather than decomposing the problem downward toward the fixed language. The result is both a solution and a reusable language for similar problems.

Macro

A function from unevaluated source code (s-expressions) to new source code, invoked at compile time. Unlike a function, a macro receives its arguments unevaluated and returns code that the compiler substitutes for the macro call. The primary mechanism for extending Lisp's syntax and adding new language features.

Closure

A function bundled with the lexical environment in which it was created. The closure "closes over" the variable bindings visible at its creation point, carrying them as private state. Closures are first-class objects: storable, passable, and returnable.

Backquote

A template notation for constructing list structures. Inside a backquoted expression (`(...)), a comma (,) unsuppresses evaluation for the following form, and comma-at (,@) splices a list into the enclosing structure. The primary notation used inside macro bodies to express the structure of the expansion.

Gensym

A freshly generated, unique, uninterned symbol. Used in macro bodies to introduce new variable bindings that cannot conflict with any user-defined names, since gensyms cannot be typed. The correct solution to the variable capture problem.

Variable capture

A macro pitfall in which a variable binding introduced by a macro expansion unintentionally interferes with a variable in the caller's scope. Accidental capture is a bug; intentional capture (as in anaphoric macros) is a documented design choice. Fixed by using gensyms for all macro-introduced bindings.

Anaphoric macro

A macro that intentionally binds a special symbol (typically it) to the value of a subform, making the value available in the macro's body without an explicit let. Examples: aif, awhen, aand. Named by analogy to linguistic anaphora ("the thing previously mentioned").

Continuation

An object representing the remainder of a computation at a given point — "what remains to be done" after the current expression. Languages with first-class continuations (like Scheme, via call/cc) can capture, store, and resume continuations. Graham simulates continuations in Common Lisp through CPS transformation.

Continuation-passing style (CPS)

A program transformation in which every function is given an explicit continuation argument (a function to call with the result) rather than returning a value. In CPS, all transfers of control are explicit tail calls to continuation functions. Graham implements CPS via the macros =defun, =bind, =values.

Nondeterminism

A programming model providing choose (select among alternatives) and fail (backtrack to the most recent choose and try the next alternative). From the programmer's perspective the program always makes the right choice; the search machinery is hidden. Implemented over continuations.

Generalized variable

In Common Lisp, any form that names a readable and writable location. setf generalizes assignment to work on any generalized variable, not just symbol variables. New generalized variables are defined with defsetf or define-setf-method.

Augmented Transition Network (ATN)

A directed graph formalism for describing grammar. Nodes represent parse states; arcs represent grammatical transitions. ATNs are nondeterministic parsers: multiple arcs may apply at each state, and backtracking is used to explore alternatives. Developed by William Woods.

Embedded language

A domain-specific language implemented within a host language by providing a set of macros, functions, and data representations. The embedded language borrows the host's evaluator and data model, requiring only the domain-specific notation and behavior to be added. Examples in the book: the query compiler, the nondeterminism engine, the ATN parser, the Prolog interpreter.

Unification

The algorithm at the core of Prolog: finding a substitution for variables that makes two terms identical. Two terms unify if they are equal, if one is an unbound variable (which gets bound to the other), or if they are both compound terms whose subterms unify recursively.

Primary book and edition information

Background and overview

Chapter PDFs (UMBC course mirror)

Key background concepts

HTML version of On Lisp

Piecing together a printed copy

Additional study resources

These are secondary summaries and should be used alongside, rather than instead of, the original book.

Send feedback

Optional. We'll only use this if you want a reply.