Contracts and tests
Functions that state what they require with assume and what they return with examples, test and bench blocks, and the report format.
This is the contract design from section 9.1 of the language specification: what a function requires with assume, what it returns with examples, and how test and bench blocks and the report file fit in.
#The design
Between its parameter list and its body a function may state what it requires and what it returns. Each line is optional, there may be any number, each ends with ;, and each may end with a message, a string literal after a comma, which is shown on failure and stored in the report.
i32 clamp(i32 v, i32 lo, i32 hi)
assume lo <= hi, "lo must not exceed hi";
test (5, 0, 3) == 3, "expected 3 when given 5 for range 0 to 3";
test (-1, 0, 3) == 0;
test (7, 3, 3) == 3;
test (5, 3, 0) fails;
{
if v < lo { ret lo; }
if v > hi { ret hi; }
ret v;
}
assume condition;is a promise every caller must keep. The condition is aboolexpression over the parameters and constants, with no effects. In test and debug builds it is checked at every call, and a broken promise stops the program with the place, the condition and the message. In release builds the optimizer may rely on it: value ranges, removed checks, arithmetic that cannot overflow.--checkedkeeps the checks in a release build. This is the only way a function gives the optimizer facts about its inputs.test (arguments) == value;is an example: called with these arguments the function returns exactly this value. Equality is exact; there are no tolerances. The arguments must satisfy everyassume, or the example is a compile error.test (arguments) fails;is an example of wrong input: these arguments break anassume. The compiler checks that they do.failsis not a keyword; it is only read in this position.
Every promise must be tested on both sides. For each assume there must be at least one example that satisfies it and at least one fails example that breaks it; otherwise the function is refused when tests are on. When the condition is a simple comparison between integers, the report also says whether the examples sit exactly on the boundary: the last allowed value and the first refused one.
Examples also serve as speed hints: they show typical arguments, so the optimizer may use them to guess which branches are likely and what is worth inlining. A wrong guess costs speed, never correctness.
#Running and reporting
--tests=all,changedoroffselects what runs.changed, the default for a build, re-runs an example only when its function, anything that function calls, itsassumelines, the example itself or the target has changed since it last passed.- At the end of compilation a summary line is printed (examples run, skipped as unchanged, failed) and each failure is printed with its place, its message and the expected and actual values. A failed example fails the build.
- The first version covers functions that compute a result from their arguments. Functions with
wroroutparameters or other effects are tested withtest "name" { }blocks;assumeapplies to any function.
#test and bench blocks
test "name" { } and bench "name" { } are compiled only in their modes.
test "clamp keeps a value inside its range" {
io.expect(clamp(5, 0, 3), 3);
io.expect(clamp('a', 'b', 'z'), 'b');
}
A kernel marked @ref(f) is differential-tested against f, its scalar reference, on generated inputs. Coverage is produced by the compiler at basic-block granularity.
#Reports are not JSON
Test, benchmark and coverage results are written to a binary report file in struct-of-arrays form: a header with the record count and the offset of each column, then one array per field (outcome, file, line, function, expected, actual, time and so on) and one string table. Columns are fixed-width, aligned and in the machine's native layout, so a reader maps the file and uses the arrays in place with no parsing step, and a filter or a count is a scan over one column. lane report <file> is its viewer: it prints the records as plain lines, one per record, and filters them by file, function or outcome. Diagnostics stay plain lines on standard error (path:line:col: error: message), and no report or diagnostic uses JSON.
#How the examples on this site are checked
Lane has a test runner written in Lane, which builds each test program, runs it, and compares its standard output and exit code with the expected bytes. It is not the contract system above: it is how the 1668 programs behind this site are checked. See The lane command and its tests.