build2 | 0.18.0 Release Notes

These notes provide a more detailed discussion of major new features, including the motivation for implementing them and their usage examples. For the complete list of changes, refer to the Release Announcement or the NEWS files in the individual packages.

While this version again contains a large number of new features, especially in the build system, another area of focus for this release was performance work across the toolchain. The package manager can now fetch, unpack, and configure packages faster, thanks, in particular, to the new Fetch Cache. The build system also saw a number of performance improvements, including optimized builds for read-only projects (such as dependencies).

This release makes available the official binary packages for Windows and a number of Linux distributions (Debian, Ubuntu, Fedora, RHEL, generic Linux) with packages for Mac OS available from Homebrew and for FreeBSD – from ports.

The following sections discuss these and other notable new features in detail.

1	Build System 1.1 Language Improvements 1.2 New Functions 1.3 New Builtins 1.4 Shell Script and Task Runner 1.5 JSON Compilation Database 1.6 Support for JSON5 and JSON5E 1.7 Update During Load 1.8 GNU make jobserver
2	Package Manager 2.1 Fetch Cache 2.2 Offline Mode 2.3 Indirect Dependency Constraints
3	Project Manager 3.1 Package Names in Commands 3.2 Fetch Cache and Offline Mode

1 Build System

1.1 Language Improvements

Support for the while-loop in addition to for has been added to the Buildfile language (the scripting languages already had this support). For example:

i = [uint64] 0
while ($i != 10)
{
  ...

  i += 1
}

Also, support for continue and break in loops has been added to both the Buildfile and scripting languages (Testscript, Buildscript, and Shellscript). For example:

i = [uint64] 0
while ($i != 10)
{
  if (...)
  {
    i += 1
    continue
  }

  if (...)
    break

  ...

  i += 1
}

See Repetitions (for, while) for details on these additions.

The new ifn and ife shortcuts have been added to the Buildfile and scripting languages. These flow control shortcuts allow more succinct expression of is/is-not null and empty checks. They are equivalent to the following more verbose expressions:

ifn ...     ~  if $null(...)
ife ...     ~  if $empty(...)

ifn! ...    ~  if! $null(...)
ife! ...    ~  if! $empty(...)

elifn ...   ~  elif $null(...)
elife ...   ~  elif $empty(...)

elifn! ...  ~  elif! $null(...)
elife! ...  ~  elif! $empty(...)

For example, this more verbose version:

if $null($v)
  ...
elif $empty($v)
  ...
else
 ...

Is equivalent to:

ifn $v
  ...
elife $v
  ...
else
 ...

1.2 New Functions

$front(), $back() and $json.array_front(), $json.array_back()

Return first/last element in a sequence.

$sha256sum(), $xxh64sum()

Compute SHA256 and XXH64 checksums of a string.

$builtin.generate_uuid()

Generate UUID.

$string.{compare,filter,filter_out}()

Compare or filter/filter out strings.

There is now also support for the contains, contains_once, starts_with, and ends_with flags in $string.{find,find_index()}.

$process.search()

Search in PATH to determine absolute executable path.

1.3 New Builtins

The scripting languages (Testscript, Buildscript, and Shellscript) now support the following new builtins:

sha256sum

Compute SHA256 checksum of a file or stdin.

xxh64sum

Compute XXH64 checksum of a file or stdin.

1.4 Shell Script and Task Runner

The portable scripting language that is used for buildfile recipes (Buildscript) and test scripts (Testscript) has been generalized to also provide a general-purpose shell scripting flavor (Shellscript). Such shell scripts can be executed with the bx executable which is now provided alongside the build system driver (b).

For example, here is a "Hello, World!" script written in the build2 Shellscript language:

#! /usr/bin/env bx

n = $size($*)

if ($n < 2)
{
  echo "usage: $0 <name>..." 1>&2
  exit 1
}

for i: $integer_sequence(1, $n)
{
  echo "Hello, ($*[$i])!"
}

One notable place where this portable scripting can be useful are CI hooks. For a more interesting example, see the bindist-archive-post.bx script that is invoked to post-process build2-toolchain binary packages for Windows.

Note that the task runner part is planned but not yet implemented. See the bx(1) man page for details. See the Testscript manual for the language documentation (still to be factored into a common document for all the flavors).

1.5 JSON Compilation Database

The c and cxx modules now provide support for the JSON Compilation Database generation and maintenance. The maintenance word in the previous sentence is the interesting part: if new source files get added to the project or old ones removed, or if any compilation options change, then the corresponding entries in the compilation database will be automatically updated when you update your project. This helps maintain the database in sync with the project state. We also made sure this maintenance does not incur a performance penalty even though the compilation database format is not designed for incremental updates.

For detail on how to enable the compilation database for your builds see the Compilation Database section in the manual.

1.6 Support for JSON5 and JSON5E

The existing JSON support has been extended to additionally cover JSON5 and JSON5E. Specifically, the $json.parse() and $json.load() functions can now be instructed with flags to parse in the JSON5 or JSON5E mode (JSON is the default). For example:

j = $json.load($src_root/test.json5, json5)
j = $json.parse('{line: 10, column: 20}', json5)
j = $json.parse('line: 10, column: 20', json5e)

Additionally, when constructing a JSON value in buildfiles, the input text is now parsed as JSON5E, not as JSON. For example, before we had to write:

j = [json] '{"one":255, "two":[2, 3, 4], "three":{"x":1, "y":-1}}'

Now we can rewrite it as:

j = [json] '{one:0xff, two:[2, 3, 4], three:{x:1, y:-1}}'

Or in the multi-line form (notice the trailing comma, which is allowed in JSON5):

j = [json] '{
  one   : 0xff,
  two   : [2, 3, 4],
  three : {x:1, y:-1},
}'

While JSON5 has a number of extensions compared to JSON (and JSON5E adds a few more), the most useful in buildfiles will likely be:

• Ability to use unquoted object member names.
• Ability to use hexadecimal numbers.
• Ability to use single-quoted strings in addition to double-quoted.

The last one (single-quoted string) can be especially handy if you need to perform expansions in the input text and thus need to use double outer quotes. For example:

n = 123
s = abc
j = [json] "{number:$n, string:'$s'}"

Note that the implied top-level object feature of JSON5E is not supported in the construction syntax but is supported by $json.parser() and $json.load() when parsing in the JSON5E mode.

1.7 Update During Load

This release adds support for updating targets during buildfile loading, which is primarily useful when the information such targets contain is required in the buildfile itself.

For example, we may need to know the target architecture byte-order in order to decide which source files must be included into the build. And, at least in case of the C/C++ compilation, the only reliable source of this information are the compiler macros. To extract this information from the compiler and make it available during the buildfile evaluation, we can generate a buildfile fragment during load and then source it into the main buildfile (or, alternatively, generate a JSON file and load it into a buildfile variable).

Another interesting possibility that this feature enables is an autoconf-style configuration probing that, being part of the build graph, can be change-tracked and performed in parallel.

See the update directive documentation in the manual for details.

1.8 GNU make jobserver

On POSIX platforms the build system driver now starts a GNU make-compatible jobserver of the fifo type and passes the jobserver information to all external commands (compilers, linkers, etc) as part of the MAKEFLAGS environment variable.

The primary motivation for adding this support was LTO. Specifically, GCC detects if the jobserver is available and parallelizes LTO jobs, which now works seamlessly with build2.

Note that the fifo jobserver type is only supported since GNU make 4.4. Note also that only the server side of the jobserver is supported and the build system driver does not act as a jobserver client. The jobserver can be disabled with the --jobserver=none option.

2 Package Manager

2.1 Fetch Cache

This release adds support for caching of downloaded resources by the package manager (bpkg). Specifically, downloaded repository metadata and package archives for archive-based repositories as well as fetched git state for the git-based repositories are now cached locally and reused whenever appropriate. Additionally the repository certification authentication answers are remembered in the fetch cache. Finally, the fetch cache can also store shared source directories for dependencies.

All this functionality is enabled by default except for shared source directories, which are only enabled for build configurations created by the project manager (bdep). Note that the cached repository metadata (for both archive and git-based repositories) is verified to be up-to-date on every repository fetch unless offline (Offline Mode) or in the same session (see below on cache sessions). As a result, in normal operation, the fetch cache should not result in stale information being used. The cache is garbage-collected and entries are automatically removed if not used for more than 3 months.

There are a number of new command line options (and corresponding environment variables) that control various aspects of the fetch cache functionality:

--no-fetch-cache            --  disable cache
--fetch-cache <mode>        --  enable/disable cache functionality
--fetch-cache-path <dir>    --  cache location
--fetch-cache-session <id>  --  cache session

See bpkg-common-options(1) for details.

A note on the default cache location: the cache is split into two parts: non-precious (repository metadata and archives) and semi-precious (shared source directories). The non-precious data is by default placed into $HOME/.cache/build2/ (UNIX) or %LOCALAPPDATA%\build2\cache\ (Windows) while semi-precious – into .build2/cache/ subdirectory of the user's home directory on all platforms. You can safely remove the non-precious data and while you won't lose any information if the semi-precious data is removed, you may end up with broken builds that will require manual repair. See the --fetch-cache-path option documentation for details. Also note that for best results the cache should be on the same filesystem/volume as the build configuration directories.

The cache session is useful when fetching repository metadata in multiple configurations at the same time. By specifying the same session id we can omit subsequent up-to-date checks. As an example, consider a script that performs package upgrades in a number of build configurations:

export BPKG_FETCH_CACHE_SESSION="$(uuidgen)"

bpkg fetch -d build-gcc
bpkg build -d build-gcc --upgrade --recursive

bpkg fetch -d build-clang
bpkg build -d build-clang --upgrade --recursive

2.2 Offline Mode

The new offline mode can be enabled with the --offline option. It relies on the fetch cache to already contain all the resources (repository metadata, package archives, etc) necessary to complete a command. If some resources are not available from the cache, the command fails. Note that in this mode the up-to-date checks for repository metadata are omitted.

The offline mode can also be enabled via the BPKG_FETCH_CACHE environment variable (see the --fetch-cache option documentation in bpkg-common-options(1) for details).

2.3 Indirect Dependency Constraints

Sometimes a package may wish to impose a version constraint and/or configuration on its indirect dependencies. For example, our package may depend on an ORM library which in turn depends on the database, say, libsqlite3. If we wish to make sure libsqlite3 is of a certain version or have it configured a certain way, until now the only way to achieve this were to make libsqlite3 a direct dependency of our package, which has a number of drawbacks. For example, if the ORM library can use different databases, we may end up having an unnecessary dependency on libsqlite3.

This release adds the new constrains manifest value that allows us to solve this problem cleanly. For example:

depends: liborm ^1.2.3
constrains: libsqlite2 ^3.51.2

The constrains value is only in effect if the dependency relationship exists due to a depends value, typically indirect. It also does not cause the dependency to be importable by the constraining package. For details, see the constrains package manifest value documentation.

3 Project Manager

3.1 Package Names in Commands

This release adds the ability to use package names in addition to package directories in commands other than bdep-init. Specifically, once a package is initialized, we can use its name rather than its directory, for example:

bdep init @gcc -d libfoo -d foo
bdep ci -d libfoo # Old way (package directory).
bdep ci libfoo    # New way (package name).

3.2 Fetch Cache and Offline Mode

The package manager (bpkg) fetch cache and offline mode are exposed in the project manager (bdep) with the following subset of command line options (the BPKG_FETCH_CACHE* environment variables can also be used):

--offline
--no-fetch-cache
--fetch-cache <mode>
--fetch-cache-session <id>

Note that the fetch cache mode, if specified during the build configuration creation, is stored in the configuration and (unless overridden) used in the subsequent commands. For example, this is how we can disable the shared source directory functionality for the configuration:

$ bdep init -C @gcc --fetch-cache=no-src cc

The cache session is useful when initializing or synchronizing in multiple configurations at the same time. By specifying the same session id we can omit subsequent up-to-date checks for repository metadata. As an example, consider a script that initializes a project in multiple configurations:

export BPKG_FETCH_CACHE_SESSION="$(uuidgen)"

bdep init -C @gcc cc config.cxx=g++
bdep init -C @clang cc config.cxx=clang++

Copyright © 2014-2026 Code Synthesis Ltd.