Programming in Lua : 7.2

This first edition was written for Lua 5.0. While still largely relevant for later versions, there are some differences.
The fourth edition targets Lua 5.3 and is available at Amazon and other bookstores.
By buying the book, you also help to support the Lua project.

Programming in Lua

Part I. The Language Chapter 7. Iterators and the Generic for

7.2 – The Semantics of the Generic for

One drawback of those previous iterators is that we need to create a new closure for each new loop. For most situations, this is not a real problem. For instance, in the allwords iterator, the cost of creating one single closure is negligible compared to the cost of reading a whole file. However, in a few situations this overhead can be undesirable. In such cases, we can use the generic for itself to keep the iteration state.

We saw that the generic for keeps the iterator function internally, during the loop. Actually, it keeps three values: The iterator function, an invariant state, and a control variable. Let us see the details now.

The syntax for the generic for is as follows:

    for <var-list> in <exp-list> do
      <body>
    end

where <var-list> is a list of one or more variable names, separated by commas, and <exp-list> is a list of one or more expressions, also separated by commas. More often than not, the expression list has only one element, a call to an iterator factory. For instance, in the code

    for k, v in pairs(t) do
      print(k, v)
    end

the list of variables is k, v; the list of expressions has the single element pairs(t). Often the list of variables has only one variable too, as in

    for line in io.lines() do
      io.write(line, '\n')
    end

We call the first variable in the list the control variable. Its value is never nil during the loop, because when it becomes nil the loop ends.

The first thing the for does is to evaluate the expressions after the in. These expressions should result in the three values kept by the for: the iterator function, the invariant state, and the initial value for the control variable. Like in a multiple assignment, only the last (or the only) element of the list can result in more than one value; and the number of values is adjusted to three, extra values being discarded or nils added as needed. (When we use simple iterators, the factory returns only the iterator function, so the invariant state and the control variable get nil.)

After this initialization step, the for calls the iterator function with two arguments: the invariant state and the control variable. (Notice that, for the for structure, the invariant state has no meaning at all. It only gets this value from the initialization step and passes it when it calls the iterator function.) Then the for assigns the values returned by the iterator function to variables declared by its variable list. If the first value returned (the one assigned to the control variable) is nil, the loop terminates. Otherwise, the for executes its body and calls the iteration function again, repeating the process.

More precisely, a construction like

    for var_1, ..., var_n in explist do block end

is equivalent to the following code:

    do
      local _f, _s, _var = explist
      while true do
        local var_1, ... , var_n = _f(_s, _var)
        _var = var_1
        if _var == nil then break end
        block
      end
    end

So, if our iterator function is f, the invariant state is s, and the initial value for the control variable is a₀, the control variable will loop over the values a₁ = f(s, a₀), a₂ = f(s, a₁), and so on, until a_i is nil. If the for has other variables, they simply get the extra values returned by each call to f.