build2 | 0.13.0 Release Notes

These notes provide a more detailed discussion of major new features, including the motivation for implementing them and their usage examples. For the complete list of changes, refer to the Release Announcement or the NEWS files in the individual packages. See also the discussion of these release notes on r/cpp/ and r/programming/.

The main focus of this release is build system support for project-specific configurations and ad hoc recipes. Another notable new feature is support for alternative (e.g., with include/src split) source code layouts by the bdep-new command. While the below notes discuss these and other features in detail, here is a one line TL;DR for each:

1 Infrastructure

1.1 New CI configurations (15 added, 57 in total)

The following new build configurations have been added to the CI service:

linux_debian_9-gcc_8.4
linux_debian_10-gcc_9.3
linux_debian_10-gcc_10.1
linux_debian_10-clang_10.0[_libc++]

macos_10.15-clang_11.0        (Xcode 11.5 Clang 11.0.3)
macos_10.15-gcc_9.3_homebrew

windows_10-msvc_16.4
windows_10-msvc_16.5
windows_10-msvc_16.6
windows_10-clang_10.0_llvm_msvc_16.6[_lld]
windows_10-gcc_9.2_mingw_w64

In addition, there are now optimized and static optimized builds for the latest version of every configuration. All in all, there are now 57 build configurations (up from 42 in the previous release) that cover a wide range of versions for all the major compilers (GCC, Clang, and MSVC) on all the major platforms (Linux, Mac OS, Windows, and FreeBSD).

2 Toolchain

2.1 SPDX as default license name

The SPDX License Expression is now the default scheme for the license package manifest value as well as the bdep-new command's --type|-t,license sub-option. Auto-detected licenses now also result in the SPDX License ID in the license package manifest value.

In its simplest form, the SPDX license is just an ID, for example:

license: MPL-2.0

If the package is licensed under multiple licenses, then an SPDX license expression can be used to specify this, for example:

license: Apache-2.0 OR MIT
license: MIT AND BSD-2-Clause

See the license package manifest value documentation for details and the list of commonly used license IDs.

2.2 Private library installation by default

On UNIX-like operating systems the installation script now installs the toolchain libraries into a private subdirectory of lib/. This means that when installed into a shared location like /usr/local/, the toolchain implementation details can no longer clash with anything that is already installed or might be installed in the future into the same location by the user.

Note that this mechanism is also available to other projects that use build2. See install module for details.

3 Build System

3.1 Project-specific configurations

A project can now use the config directive to define project-specific config.<project>.* variables, similar to the build system core and modules. For example:

config [bool]   config.libhello.fancy    ?= false
config [string] config.libhello.greeting ?= 'Hello'

With these configuration points available, the user of the libhello library can configure it using the same mechanism as what's already used to configure the compiler, options, importation, etc. For example:

$ b configure: libhello/          \
    config.cxx=clang++            \
    config.cxx.coptions=-O3       \
    config.libhello.fancy=true    \
    config.libhello.greeting=Howdy

Inside libhello, these configuration variables can then be used in buildfiles and/or propagated to the source code using the command line, .in file substitution, etc. For example:

if $config.libhello.fancy
  cxx.poptions += -DLIBHELLO_FANCY

cxx.poptions += "-DLIBHELLO_GREETING=\"$config.libhello.greeting\""

Or using the .in file:

// libhello/config.hxx.in

#pragma once

#define LIBHELLO_FANCY    $config.libhello.fancy$
#define LIBHELLO_GREETING "$config.libhello.greeting$"

See the Project Configuration chapter in the build system manual for details and more examples (including using if constexpr for configuration).

3.2 Ad hoc recipes

With ad hoc recipes it is now possible to provide custom implementations of operations (update, test, etc) for certain targets directly in buildfiles.

Let's look at a few examples. This is how we can pick a configuration header based on the platform:

tclass = $cxx.target.class

hxx{config}: hxx{config-linux}:   include = ($tclass == 'linux')
hxx{config}: hxx{config-windows}: include = ($tclass == 'windows')
hxx{config}: hxx{config-macos}:   include = ($tclass == 'macos')
hxx{config}:
{{
  cp $path($<) $path($>)
}}

hxx{config}: hxx{config-$cxx.target.class}
{{
  cp $path($<) $path($>)
}}

The recipe body (cp $path($<) $path($>)) is pretty simple: we use the cp utility to copy the file corresponding to the prerequisite (available to the recipe in the $< special variable) to the file corresponding to the target (available in the $> special variable).

Let's look at another, more elaborate, example that shows how to embed binary data into the source code with the help of the xxd(1) utility:

import! xxd = xxd%exe{xxd}

<{hxx cxx}{foo}>: file{foo.bin} $xxd
{{
  diag xxd ($<[0])

  i = $path($<[0]) # Input.
  h = $path($>[0]) # Output header.
  s = $path($>[1]) # Output source.
  n = $name($<[0]) # Array name.

  # Get the position of the last byte (in hex).
  #
  $xxd -s -1 -l 1 $i | sed -n -e 's/^([0-9a-f]+):.*$/\1/p' - | set pos

  if ($empty($pos))
    exit "unable to extract input size from xxd output"
  end

  # Write header and source.
  #
  echo "#pragma once"                         >$h
  echo "extern const char $n[0x$pos + 1];"   >>$h
  echo "extern const char $n[0x$pos + 1]= {"  >$s
  $xxd -i <$i                                >>$s
  echo '};'                                  >>$s
}}

While explaining this example in detail is beyond the scope of these release notes, one aspect worth highlighting is the use of an ad hoc target group (<{hxx cxx}{foo}>) to indicate that the recipe will produce both files with a single execution.

Note that in both examples, the utilities (cp, echo, and sed) are builtins which means these recipes are portable and will work even on Windows. See the Testscript Manual for the list of available builtins.

Ad hoc recipes can also be used to customize a part of the update chain otherwise handled by rules. For example, in embedded systems development it is often required to perform a custom link step:

obje{foo}: cxx{foo}
obje{bar}: cxx{bar}

<exe{test} file{test.map}>: obje{foo bar}
{{
  diag ld ($>[0])
  $cxx.path $cc.loptions $cxx.loptions $cxx.mode -o $path($>[0]) \
    "-Wl,-Map=$path($>[1])" $path($<) $cxx.libs $cc.libs
}}

While the above examples are all for the update operation, ad hoc recipes can be used for other operations, such as test. For example:

exe{hello}: cxx{hello}
% test
{{
  diag test $>
  $> 'World' >>>?'Hello, World!'
}}

Here we have the recipe header (line starting with %) which specifies the operation(s) this recipe is for. If the header is omitted, then the update operation is assumed.

The above recipes are written in a shell-like language called Buildscript that has similar semantics to Testscript tests. Another language that can be used to write recipes is C++. For example:

./:
{{ c++ 1

  recipe
  apply (action, target& t) const override
  {
    text (recipe_loc) << "Hello, " << t;
    return noop_recipe;
  }
}}

A recipe written in C++ plugs directly into the build system rules machinery and can do pretty much anything a build system core or module can.

3.3 Project-local importation

An import without a project name is now recognized as importation from the same project. For example, given the libhello project that exports the lib{hello} target, a buildfile for an executable in the same project instead of doing something like this:

include ../libhello/
exe{hello}: ../libhello/lib{hello}

Can now do:

import lib = lib{hello}
exe{hello}: $lib

The main advantage of project-local importation over inclusion is the ability to move things around without having to adjust locations in multiple places (the only place we need to do it is the export stub). This advantage becomes noticeable in more complex projects with a large number of components.

See the Target Importation section in the manual for details.

3.4 Ad hoc importation and "glue buildfiles"

If the target being imported has no project name and is either absolute or is a relative directory, then this is treated as ad hoc importation. Semantically it is similar to normal importation but with the location of the project being imported hard-coded into the buildfile.

While doing so would generally be a bad idea, this type of import can be used to create a special "glue buildfile" that "pulls" together several projects, usually for convenience of development. One typical case that calls for such a glue buildfile is a multi-package project. To be able to invoke the build system directly in the project root, we can add a glue buildfile that imports and builds all the packages:

import pkgs = */

./: $pkgs

See the Target Importation section in the manual for details.

4 Project Dependency Manager

4.1 Source code layout customizations

The bdep-new command now supports a number of new source layout customization mechanisms. In particular, the split include/src layout is now supported out of the box:

$ bdep new -l c++,cpp -t lib,split libhello

$ tree libhello/
libhello/
├── include/
│   └── libhello/
│       └── hello.hpp
└── src/
    └── libhello/
        └── hello.cpp

Or, if you prefer the more traditional structure without the subdirectory in src/:

$ bdep new -l c++ -t lib,split,subdir=hello,no-subdir-source libhello

$ tree libhello/
libhello/
├── include/
│   └── hello/
│       └── hello.hxx
└── src/
    └── hello.cxx

See the SOURCE LAYOUT section in bdep-new(1) for details and a large number of examples.

5 Package Dependency Manager

5.1 Package download proxies

The new --pkg-proxy option can be used to specify the proxy server to use when fetching package manifests and archives from remote pkg repositories. This can be particularly useful when re-fetching the same packages over and over both to speed things up and to reduce bandwidth usage. For example, we use this mechanism in our CI infrastructure.

See the option documentation in bpkg-common-options(1) for details.