build2 | 0.14.0 Release Notes

These notes provide a more detailed discussion of major new features, including the motivation for implementing them and their usage examples. For the complete list of changes, refer to the Release Announcement or the NEWS files in the individual packages. See also the discussion of this release on r/cpp/ and r/programming/.

The main focus of this release is support for build-time dependencies and the host/target configuration split that it necessitates. This support required a large amount of ground work which produced functionality useful in its own right, such as hermetic builds and configuration linking. Another notable new feature is ad hoc regex pattern rules. The following sections discuss these and other new features in detail.

A note on backwards compatibility: this release cannot be upgraded to from 0.13.0 and has to be installed from scratch.

1.1New CI configurations (15 new, 58 in total)
2.1Standard pre-installed build system modules
2.2Performance optimizations
3Build System
3.1Hermetic build configurations
3.2Ad hoc regex pattern rules
3.3Complete C++20 modules support with GCC
3.4Warning suppression from external C/C++ libraries
3.5Pre-defined config.<project>.develop variable
3.6Automatic DLL symbol exporting
3.7Kconfig configuration support
4Project Dependency Manager
4.1Build-time dependencies
4.2Build configuration linking
4.3Configuration preservation during synchronization
5Package Dependency Manager
5.1Build-time dependencies and configuration linking

1 Infrastructure

1.1 New CI configurations (15 new, 58 in total)

The following new build configurations have been added to the CI service:




macos_11-clang_12.0          (Xcode 12.5.1 Clang 12.0.5)
macos_11-clang_13.0          (Xcode 13     Clang 13.0.0)


linux_debian_10-emcc_2.0.25  (Emscripten)

In addition, there are now optimized NDEBUG builds for select configurations. All in all, there are now 58 build configurations that cover a wide range of versions for all the major compilers (GCC, Clang, and MSVC) on all the major platforms (Linux, Mac OS, Windows, FreeBSD as well as WASM).

2 Toolchain

2.1 Standard pre-installed build system modules

While we can now build and load build system modules on the fly (and even list them as build-time dependencies of our projects), this will be inconvenient and inefficient for widely-used modules. As a result, with this release we are introducing a notion of standard pre-installed build system modules that the user can assume are available out of the box in a regular build2 toolchain installation. In particular, the official install scripts will build and install such modules unless explicitly requested not to.

The first such build system module is libbuild2-kconfig which allows using the Linux kernel configuration system (Kconfig) to configure build2-based projects. See the earlier announcement for details on standard pre-installed modules.

2.2 Performance optimizations

Our ongoing efforts to package the Boost libraries identified a number of performance issues when working with heavily inter-dependent libraries. This led to optimizations both in the build system and the package manager that should also benefit other projects.

Additionally, the build system now uses the LZ4 compression when storing intermediate build results (such as partially preprocessed C/C++ translation units) which leads to a substantial reduction in the disk space usage during the build.

Finally, a number of optimizations have been implemented for git repository fetching in the package manager.

3 Build System

3.1 Hermetic build configurations

Hermetic build configurations save environment variables that affect the project along with other project configuration in the file. These saved environment variables are then used instead of the current environment when performing operations on the project, thus making sure the project "sees" exactly the same environment as during configuration. The built-in ~host and ~build2 configurations are now hermetic. As part of this work we now also track changes to the environment in non-hermetic configurations and automatically rebuild affected targets.

To create a hermetic configuration we use the config.config.hermetic configuration variable. For example:

$ b configure config.config.hermetic=true

The two use-cases where hermetic configurations are especially useful are when we need to save an environment which is not generally available (for example, an environment of a Visual Studio development command prompt) or when our build results need to exactly match the specific configuration (for example, because parts of the overall result have already been built and installed, as is the case with build system modules). See Hermetic Build Configurations in the build system manual for details.

3.2 Ad hoc regex pattern rules

The previous release introduced support for ad hoc recipes which allow providing custom implementations of operations (update, test, etc) for a specific target directly in buildfiles. In this release we are extending this functionality with support for ad hoc rules which allow providing such custom implementations of operations for multiple targets that match a pattern.

An ad hoc pattern rule consists of a pattern that mimics a dependency declaration followed by one or more recipes. For example:

exe{~'/(.*)/'}: cxx{~'/\1/'}
  $cxx.path -o $path($>) $path($<[0])

If a pattern matches a dependency declaration of a target, then the recipe is used to perform the corresponding operation on this target (see ad hoc recipes for background). For example, the following dependency declaration matches the above pattern which means the rule's recipe will be used to update this target:

exe{hello}: cxx{hello}

While the following declarations do not match the above pattern:

exe{hello}:   c{hello}  # Type mismatch.
exe{hello}: cxx{howdy}  # Name mismatch.

Note also that in build2 no intermediate targets are created implicitly and ad hoc rules only match explicit dependency declarations. For example, the above rule will not build exe{goodbye} given the following buildfile even if cxx{goodbye} exists:

./: exe{goodbye}

On the left hand side of : in the pattern we can have a single target or an ad hoc target group. The single target or the first (primary) ad hoc group member must be a regex pattern (~). The rest of the ad hoc group members can be patterns or substitutions (^). For example:

<exe{~'/(.*)/'} file{^'/\'}>: cxx{~'/\1/'}
  $cxx.path -o $path($>[0]) "-Wl,-Map=$path($>[1])" $path($<[0])

On the right hand side of : in the pattern we have prerequisites which can be patterns, substitutions, or non-patterns. For example:

<exe{~'/(.*)/'} file{^'/\'}>: cxx{~'/\1/'} hxx{^'/\1/'} \
  $cxx.path -o $path($>[0]) "-Wl,-Map=$path($>[1])" $path($<[0])

Substitutions on the left hand side of : and substitutions and non-patterns on the right hand side are added to the dependency declaration. For example, given the above rule and dependency declaration, the effective dependency is going to be:

<exe{hello} file{}>: cxx{hello} hxx{hello} hxx{common}

Similar to ad hoc recipes, ad hoc rules can be written in Buildscript or C++.

3.3 Complete C++20 modules support with GCC

The build2 build system now provides conforming and scalable support for all the major C++20 Modules features when used with GCC. This includes named modules, module partitions (both interface and implementation), header unit importation, and include translation. All of these features are also supported in libraries, including consumption of installed libraries with information about modules and importable headers conveyed in pkg-config files. As part of this effort we have also created a collection of examples that demonstrate C++20 Modules features that impact the build process. See the earlier announcement for details on this support.

3.4 Warning suppression from external C/C++ libraries

The c and cxx modules now define a notion of a project's internal scope. If specified, header search path options (-I) exported by libraries that are outside of the internal scope are automatically translated to appropriate "external header search path" options (-isystem for GCC/Clang, /external:I for MSVC 16.10 and later). This suppresses compiler warnings in such external headers (/external:W0 is automatically added unless a custom /external:Wn is specified).

In the future this functionality will be extended to side-building BMIs for external module interfaces and header units.

Note that this functionality is not without limitations and drawbacks and, if needed, should be enabled explicitly. See the Compilation Internal Scope in the build system manual for details.

3.5 Pre-defined config.<project>.develop variable

This config.<project>.develop variable allows a project to distinguish between development and consumption builds. While normally there is no distinction, sometimes a project may need to provide additional functionality during development. For example, a source code generator which uses its own generated code in its implementation may need to provide a bootstrap step from the pre-generated code. Normally, such a step is only needed during development.

If used, this variable should be explicitly defined by the project with the bool type and the false default value. For example:

config [bool] config.hello.develop ?= false

The project manager initializes a project for development unless an alternative value is specified on the command line. For example:

$ bdep init @install config.hello.develop=false

To change the development mode of an already initialized project, use bdep-sync:

$ bdep sync @install config.hello.develop=false

See Project Configuration in the build system manual for details.

3.6 Automatic DLL symbol exporting

It is now possible to automatically generate a .def file that exports all symbols from a Windows DLL both for C and C++ libraries. Here is a typical arrangement (using a C++ library as an example):

lib{foo}: libul{foo}: {hxx cxx}{**} ...

lib{foo}: def{foo}: include = ($ == 'win32-msvc')
def{foo}: libul{foo}

if ($ == 'mingw32')
  cxx.loptions += -Wl,--export-all-symbols

See Automatic DLL Symbol Exporting in the build system manual for details.

3.7 Kconfig configuration support

The build2 toolchain now includes the libbuild2-kconfig standard pre-installed build system module that allows using the Linux kernel configuration system (Kconfig) to configure build2-based projects.

Kconfig is the configuration system of the Linux kernel. Over the years it has evolved into a sophisticated variability modeling language and a toolset that are used by the Linux kernel to manage over ten thousand configuration options. It is also increasingly being used by other projects that need to manage complex configurations.

The build2 Kconfig module acts as a second-level configuration mechanism to the builtin configuration support. Specifically, it integrates the execution of one of the Kconfig configurators into the configure meta-operation and loads the resulting configuration file presenting its values as kconfig.* variables in buildfiles. See the earlier announcement for details on this functionality.

4 Project Dependency Manager

4.1 Build-time dependencies

Normally our projects will have libraries as dependencies. Such dependencies are called runtime, since our projects need them during execution. However, sometimes we may only wish to use a dependency during the build, typically a tool, such as a source code generator. This kind of dependency is called build-time.

Why do we need to distinguish between the two kinds of dependencies? The primary reason is cross-compilation: if we build a tool in the same (cross-compiling) build configuration as our project, then we will not be able to execute it during the build (since it's built for a different target than what we are running). But even if we are not planning to cross-compile, there are other good reasons: if we have multiple build configurations for our project, we may want to share a single build of the tool between them (why waste time building the same thing multiple times). And even if we only have a single build of our project, we may want to build the tool with different options (for example, optimized instead of debug).

In order to properly support build-time dependencies, we need to distinguish them from runtime and we need an ability to build them in a separate build configuration. The bulk of the time spent on this release went into making this happen automatically. For details and examples see Build-Time Dependencies and Linked Configurations in the toolchain introduction.

Building build-time dependencies in separate configurations is just one application of the more general configuration linking mechanism which allows us to build a package in one configuration while its dependencies – in one or more linked configurations. This, for example, can be used to create a "base" configuration with common dependencies that are shared between multiple configurations (sometimes also referred to as build configuration overlaying). For details and examples see Build-Time Dependencies and Linked Configurations in the toolchain introduction.

4.3 Configuration preservation during synchronization

Configuration of a project is now preserved during synchronization. To reconfigure a project from scratch, use the new --disfigure bdep-sync option. For example (see config.<project>.develop for background):

$ bdep sync config.hello.develop=false  # develop=false (new)
$ bdep sync                             # develop=false (preserved)
$ bdep sync --disfigure                 # develop=true  (default)

5 Package Dependency Manager

Support for build-time dependencies and configuration linking in the project manager is built on top of the corresponding mechanism in the package manager, which means this functionality (including automatic host and module configuration creation) is available when using bpkg directly, for example, for package consumption. See bpkg-cfg-create(1) for details.