build2 | FAQ
It stands for take two, after a previous, GNU
make-based attempt called
Who is behind this?
Why are you doing this?
To make something people want? No, scratch that: we are making what we need.
We have a bunch of cross-platform C++ tools and libraries that
are becoming an ever increasing pain to develop, test, distribute, and
support. A mailing list thread of 20 messages where we painstakingly
discover that the user accidentally built our library with
libstdc++ and his application with
all resulted in mysterious crashes (exception handling was not working
properly) – this is a waste of life.
And while on a philosophical subject, note that we are not trying to
create a wonderful unicorn that will magically solve all your build issues.
Rather our aim is something like
git – an engineering
tool that allows serious developers working on complex projects to get stuff
done without too much cursing. And if you think
git is a
wonderful unicorn, try to use submodules.
What is wrong with <your favorite build system>?
In short, it doesn't provide a well-defined, consistent build interface across all the platforms/compilers out of the box (no, having to install Python or MSYS on Windows doesn't count).
In addition, the following areas are often poorly supported (or outright broken) in many existing (and new) build systems:
- Development & Distribution
- Trying to support both is often tricky since they pull the design in different directions: for development we want power and flexibility while the end-user wants the simplest thing that does the job and gets out of the way (they definitely don't want to install Python or MSYS).
- Black Boxes
- Many existing build systems are black boxes, they don't have a
mental model of how the build is described and performed. The problem with
this approach comes when you need to build something different: your only
option is to start creating another black box. In this sense
build2is a lot like
make(it has a notion of targets, prerequisites, rules, etc) rather than, say,
CMake(which has a notion of... well, magic incantations maybe?).
- Reliable Builds
- We often find ourselves doing complete rebuilds for good measure
because we don't trust our build system to track all the dependencies
(compiler upgrades, option changes, etc). In
build2we do High Fidelity Builds.
- Cross Compilation
- This should be straightforward if you design for it from the ground up
but many build systems just can't get it right. In
build2it is as easy to cross-compile as to compile natively.
- Source Code Generation
- Automatic header dependency extraction combined with auto-generated
source code can get tricky and many build systems have given up on handling
this properly and provide make-shift solutions such as pre-build
build2this is part of the build model.
- Project Importing/Composition
build2we have an explicit model for importing one project into another (e.g., a project imports a prerequisite library, including installed) as well as for amalgamation (you can drop a prerequisite library into your project and the import machinery will automatically pick it up).
- Extra Operations
- These days build systems rarely handle just building. There is also
configuration (often separate/add-on but integrated in
build2), testing, installing/uninstalling, and preparation of distributions.
- Toolchain, not just a Tool
build2we have a tightly-integrated package manager (
bpkg) that informs the design of the build system (and vice versa; though the build system can be used by itself, without any packaging).
- Sane Syntax
- While at it, why not have a nice, clean syntax (no scripting language extensions, etc)?
But what if you are only interested in one platform, stay away from code
generators, and like opaque boxes? It's true, if you develop, say, a
Windows-only application in Visual Studio you will most likely find whatever
comes with your IDE a lot more convenient. However, you may still find
build2 useful if you have dependencies on third-party
libraries: you can use the toolchain to build them for your exact
configuration(s) and keep up with their updates.
How is this better than
Most of the issues described in the previous
question apply to
To expand on that, the core problem both
CMake try to address is this: there are all these different
platform/compiler-specific build systems out there and it is painful to
support them individually. The
CMake approach is: "hey, let's
come up with a single project description that we can translate to all those
underlying build systems". In contrast, the
build2 approach is:
"hey, let's come up with a single, uniform build system that we can use on
all the platforms and with all the compilers (and if we do a good job, then
hopefully IDEs will start using it as well)".
The nice thing about the
CMake approach is that you can
reuse all the existing build systems for doing actual building. Plus people
can continue using, for example, Visual Studio since they get its project
files (but they have to regenerate them and reload if they add/remove any
The bad thing about the
CMake approach is that you restrict
yourself to the lowest common denominator. If you want to support multiple
platforms, you can only use build system features that are available on all
of them. This gets especially hairy when it comes to supporting
cross-compilation, source-code generation (and especially header generation;
thread for details), as well as providing additional operations like
Another general problem with
CMake is the lack of full
control, all the way down to running compilation commands. A good example of
where it hurts is support for the Clang
Compilation Database which is required to use its static analysis tools.
CMake simply cannot generate it for Visual Studio builds
because it doesn't know the actual compilation command lines that Visual
Studio will use. In a sense, this build system is often one black box
CMake) stacked on another (Visual Studio/MSBuild).
Let's now see what the
build2 approach gives us:
- Now we have a uniform build system interface that works across all the major platforms (Linux, Mac OS, Windows, FreeBSD, etc) and compilers (GCC, Clang, MSVC, Intel, etc).
- We can handle source code generation properly, again, for all the platforms/compilers.
- We can do cross-compilation, properly. In
build2we just don't do anything that is not "cross-compile"-clean, end-to-end.
build2provides a set of operations that you would expect from a modern build system, again, uniformly and across all the platforms/compilers:
dist(prepare distribution archives).
- And as a bonus you get a package manager which again works the same way everywhere.
Do you support cross-compilation?
Yes, cross-compilation was supported from day one with native compilation
being just a special case (build is the same as host). Fun
build2 could cross-compile to Windows before it could
What is wrong with using system package managers?
In short, there is nothing wrong with them, they work well (unless you
are on Windows), and we use them ourselves every day. And
build2 (both the build system and package manager) work well
with system-installed packages.
To elaborate, system package managers (such as
pkg, etc) serve a different purpose
bpkg: they are optimized for the end-user who wants
to easily install and use a program, not the developer who wants flexibility
and power in juggling their library/tool dependencies. Yes, most system
package managers support installing development files for a library but you
can normally only have one version at a time. While during development we
often want multiple versions (so we can make sure our code works with all of
them) that are built in multiple configurations (different compilers,
32/64-bit, debug/release; and don't forget cross-compilation). And this is
build2 is optimized for.
Still, it often makes sense to use a system-installed library or tool
instead of building it from source. Take
libsqlite3 as an
example: it is stable, available and works the same way pretty much
everywhere – it probably doesn't make much sense to build it from
source for every configuration.
build2 build system will happily use a system-installed
library; there is even
pkg-config integration to extract
additional information such as the location of headers, additional libraries
to link, etc. Also, the package manager has a notion of system-installed
sys: prefix) which allows you to instruct
bpkg to assume the package is available from the system rather
than building it from source.
What makes you think you will succeed when so many have failed?
Good question. For over 10 years now we've been developing and using in
production our own, GNU
make-based build system. On several occasions we've also
contributed new features to GNU
make (second expansion in
pattern rules, shortest stem matching, etc).
Despite being based on the build tool not designed for today's
build is surprisingly powerful. For example,
build can generate (based on templates) other build systems,
such as autotools and VC++ projects/solutions. We call this
meta-build and it's the reason why most people who use our projects
have never heard of
However, the original
make design and GNU
codebase do show their age. The biggest problem is the lack of out of the
box Windows support. Or, more precisely, the lack of a POSIX shell on
Windows. Also, the interaction of complex features like parallel builds,
auto-generated source code, and automatic header dependency extraction can
lead to order-dependent behavior. For example, a C++ preprocessor is
sometimes executed on a header file as it is being generated. We
euphemistically call them build aberrations and they will not be easy
to fix in
make. But, despite all of this, many design decisions
that you will find in
build2 have been conceived and
battle-tested first in
When it comes to the package manager, our view is pretty simple: given a
sane build system with a consistent interface across platforms, writing a
package manager is actually not that hard. The core functionality of
bpkg was designed and implemented in a month. However, try to
support different/multiple/insane build systems and things get hairy
Code Synthesis also has a solid track record of making complex C++ tools work across multiple platforms/compilers, including cross-compilers (see ODB for a good example). So, to summarize, we have experience developing build tools/systems and using them to deliver real-world, cross-platform C++ products. But then, again, none of this is a guarantee. We will just have to wait and see.
Why did you write it in C++ instead of <your favorite language>?
Let's see: we need a cross-platform (must work out of the box on Windows)
and reasonably efficient (nobody likes to wait for up-to-date checks)
language that most C++ developers already know (so that they can extend
build2 if they need to). Any language comes to mind that fits
Plus, nothing gives as good a reality check as trying to build your own build system and package your own package manager. And, yes, even the web interface is written in C++ (we've made a little EDSL on top of libstudxml that produces nicely indented XHTML5).
Why aren't you hosting on GitHub?
Mostly because we want complete control of our infrastructure. Also, many
things on GitHub are optimized for the convenience of new/inexperienced
git users and not for the efficient handling of large volumes
of information by the experienced ones. Case in point: issue tracker; I am
much more efficient using my mail client (
mutt) and text editor
emacs) than a web browser form.
Also, if this setup works for the Linux kernel, surely it should work for us; see the "Why kernel development still uses email" LWN article for details.
How do I submit bugs/patches?
Simply email them to the
list. You don't even need to subscribe. While you can just fire and forget
(we do realize that by submitting a bug report or a patch you are doing us a
favor, so we are not picky), we would appreciate it if you took a look at
the posting guidelines.
What are your plans for cppget.org?
We hope it will become The C++ Package Repository, the place
people get third-party C++ libraries and tools from. If/when this happens,
it will probably be a good idea to turn the control over to someone
impartial, like Standard C++ Foundation (to
this effect, we are running
cppget.org on completely separate infrastructures from the
Do you plan to support binary packages?
Not at this early stage but maybe in the future.
The problem with supporting binary packages is you have to make sure the binary package built on someone else's machine matches your environment precisely (if you think this is a Windows-only problem, check the recent GCC dual ABI mess). If we start cutting corners here we will end up in tears (and long mailing list threads). One interesting approach would be to dump the entire build system state into, say, an SQLite database and then ship that with the binary package. On the other end we can then load this database and verify that all the prerequisites (like system headers, compiler versions, etc) match.
Also, there are some ideas floating around about making
automatically generate binary packages in
deb formats. After all, the
bpkg manifest already
includes all/most of the information needed.
Can I run my own private/public repositories?
Yes, the entire toolchain, including the server-side web interface, is open source, licensed under the MIT License.
How are you going to make money from this?
We are not sure. Maybe we won't. Honestly, if this succeeds, and we won't have to manually test/distribute our tools and libraries on the myriad of platform/compiler/IDE combinations, then it would have been all worth it.