build2 0.5.0 Release Notes

These notes provide a more detailed discussion of major new features in the build2 release 0.5.0, including motivation for implementing them and their usage examples. For the complete list of changes, see the Release Announcement or the NEWS files in the individual packages.

Build System

With almost 500 commits, the bulk of the development in this release cycle has gone into the build system. The two major features are parallel builds and Testscript. There is also a large number of usability improvements, including support for wildcard patterns and automated project version management.

The build2 build system now builds in parallel. By default it uses all the available hardware threads so you normally don't need to pass -j (the option name is homage to GNU make). What is different about build2 compared to, say, make and most other implementations is that the build system itself is multi-threaded and performs rule matching (which may involve filesystem queries, header dependency extraction, etc) in parallel. While there are no comprehensive benchmarks yet, we (and users who tested release snapshots) are impressed with the result. Looking back, it's amazing that we have endured for so long without parallel builds. Most importantly, though, the implementation is correct with regards to auto-generated source code or, more generally, any filesystem modifications while building the dependency graph.

The second major feature in this release is Testscript, a portable shell-like language designed for concise implementation and parallel execution of tests. And to be clear, by parallel we mean parallel execution of individual test cases within a single testscript file.

Testscript changed the way we test things. Instead of being something that is dreaded, writing non-trivial tests is now almost enjoyable (let's be honest here, nobody really enjoys writing tests).

Take bpkg, the package manager, as an example: how do you test it? We are talking about tests that create package repositories, exercise different package manager commands, make sure the result makes sense, and so on. We want to run tests on different platforms (including Windows). And we want the tests to run fast even though there are quite a few of them.

Up until this release we used a Bash script. It was an incomprehensible mess that everyone dreaded to touch. We only dared to run it on Linux. And it took several minutes to execute (even when using local repositories). For this release we have ported all these tests (and some) to Testscript. The result: tests run out of the box on all the major platforms (Linux, Mac OS, Windows, FreeBSD) and take under 10 seconds on my development machine.

For more details, including examples, see the Testscript Manual.

Support for wildcard patterns was the most requested feature and it is finally here. Some examples:

exe{hello}: cxx{*}   # All C++ source files.
exe{hello}: cxx{**}  # All C++ source files recursively.
./: {*/ -build/}     # All subdirectories excluding build/

Note that this is not just a bolt-on shell globbing; the pattern logic is integrated into the build system mental model. You may have noticed that we didn't specify the C++ source file extensions in the above patterns. We don't have to because the build system can figure it out from these lines (which are in our project's root.build):

hxx{*}: extension = hpp
cxx{*}: extension = cpp

While patterns are surely convenient, they are also very good foot guns, especially in complex projects. So make sure to read the documentation thoroughly before shooting.

Another important feature in this release is the version module. Software versioning is hard for simple projects. When it comes to more complex, multi-module projects, things quickly get out of hand: versions start to clash, packages not upgrading when they should, it is no longer clear who should depend on whom and on which versions.

The version module establishes a semver-base project versioning model (called standard version or stdver) and then unifies various versions (project, package, library, etc) and automates version management by extracting the necessary components from a single source (manifest and version control system) and making them available in various forms (build system variables, preprocessor macros, etc). See the Version Module documentation for details on this topic.