build2 | FAQ

What does 2 in build2 stand for?
Who is behind this?
Why are you doing this?
What is wrong with <your favorite build system>?
How is this better than CMake?
Do you support cross-compilation?
What is wrong with using system package managers?
What makes you think you will succeed when so many have failed?
Why did you write it in C++ instead of <your favorite language>?
Why aren't you hosting on GitHub?
How do I submit bugs/patches?
What are your plans for cppget.org?
Do you plan to support binary packages?
Can I run my own private/public repositories?
How are you going to make money from this?

What does 2 in build2 stand for?

It stands for take two, after a previous, GNU make-based attempt called build.

Who is behind this?

Code Synthesis, Boris Kolpackov, and some friends. And maybe you?

Why are you doing this?

To make something people want? No, scratch that: we are making what we need.

We have a bunch of cross-platform C++ tools and libraries that are becoming an ever increasing pain to develop, test, distribute, and support. A mailing list thread of 20 messages where we painstakingly discover that the user accidentally built our library with libstdc++ and his application with libc++ which all resulted in mysterious crashes (exception handling was not working properly) – this is a waste of life.

And while on a philosophical subject, note that we are not trying to create a wonderful unicorn that will magically solve all your build issues. Rather our aim is something like git – an engineering tool that allows serious developers working on complex projects to get stuff done without too much cursing. And if you think git is a wonderful unicorn, try to use submodules.

What is wrong with <your favorite build system>?

In short, it doesn't provide a well-defined, consistent build interface across all the platforms/compilers out of the box (no, having to install and, more importantly, maintain Python or MSYS2 on Windows doesn't count).

In addition, the following areas are often poorly supported (or outright broken) in many existing (and new) build systems:

Development & Distribution
Trying to support both is often tricky since they pull the design in different directions: for development we want power and flexibility while the end-user wants the simplest thing that does the job and gets out of the way (they definitely don't want to install anything extra).
Black Boxes
Many existing build systems are black boxes, they don't have a mental model of how the build is described and performed. The problem with this approach comes when you need to build something different: your only option is to start creating another black box. In this sense build2 is a lot like make (it has a notion of targets, prerequisites, rules, etc) rather than, say, CMake (which has a notion of, uh, hm, magic incantations, maybe?).
Reliable Builds
We often find ourselves doing complete rebuilds for good measure because we don't trust our build system to track all the dependencies (compiler upgrades, option changes, etc). In build2 we do High Fidelity Builds.
Cross Compilation
This should be straightforward if you design for it from the ground up but many build systems just can't get it right. In build2 it is as easy to cross-compile as to compile natively.
Source Code Generation
Automatic header dependency extraction combined with auto-generated source code can get tricky and many build systems have given up on handling this properly and provide make-shift solutions such as pre-build steps. In build2 this is part of the build model.
Project Importing/Composition
In build2 we have an explicit model for importing one project into another (e.g., a project imports a prerequisite library, including installed) as well as for amalgamation (you can drop a prerequisite library into your project and the import machinery will automatically pick it up).
Extra Operations
These days build systems rarely handle just building. There is also configuration (often separate/add-on but integrated in build2), testing, installing/uninstalling, and preparation of distributions.
Toolchain, not just a Tool
In build2 we have a tightly-integrated package manager (bpkg) that informs the design of the build system (and vice versa; though the build system can be used by itself, without any packaging).
Sane Syntax
While at it, why not have a nice, clean syntax (no scripting language extensions, etc)?

But what if you are only interested in one platform, stay away from code generators, and like opaque boxes? It's true, if you develop, say, a Windows-only application in Visual Studio you will most likely find whatever comes with your IDE a lot more convenient. However, you may still find build2 useful if you have dependencies on third-party libraries: you can use the toolchain to build them for your exact configuration(s) and keep up with their updates.

How is this better than CMake?

Most of the issues described in the previous question apply to CMake.

To expand on that, the core problem both build2 and CMake try to address is this: there are all these different platform/compiler-specific build systems out there and it is painful to support them individually. The CMake approach is this: let's come up with a single project description that we can translate to all those underlying build systems. In contrast, the build2 approach is this: let's come up with a single, uniform build system that we can use on all the platforms and with all the compilers (and if we do a good job, then hopefully IDEs will start using it as well).

The nice thing about the CMake approach is that you can reuse all the existing build systems for doing actual building. Plus people can continue using, for example, Visual Studio since they get its project files (but they have to regenerate them and reload if they add/remove any files).

The bad thing about the CMake approach is that you restrict yourself to the lowest common denominator. If you want to support multiple platforms, you can only use build system features that are available on all of them. This gets especially hairy when it comes to supporting cross-compilation, source-code generation (and especially header generation; see this thread for details), as well as providing additional operations like install, uninstall, dist, etc.

Another general problem with CMake is the lack of full control, all the way down to running compilation commands. A good example of where it hurts is support for the Clang Compilation Database which is used by static analysis tools, editors for auto-completion, etc. CMake simply cannot generate it for Visual Studio builds because it doesn't know the actual compilation command lines that Visual Studio will use. In a sense, this build system is often one black box (CMake) stacked on another (Visual Studio/MSBuild).

Let's now see what the build2 approach gives us:

Do you support cross-compilation?

Yes, cross-compilation was supported from day one with native compilation being just a special case (build is the same as host). Fun fact: build2 could cross-compile to Windows before it could build natively.

What is wrong with using system package managers?

In short, there is nothing wrong with them, they work well (unless you are on Windows), and we use them ourselves every day. And build2 (both the build system and package manager) work well with system-installed packages.

To elaborate, system package managers (such as rpm, dpkg, FreeBSD pkg, etc) serve a different purpose compared to bpkg: they are optimized for the end-user who wants to easily install and use a program, not the developer who wants flexibility and power in juggling their library/tool dependencies. Yes, most system package managers support installing development files for a library but you can normally only have one version at a time. While during development we often want multiple versions (so we can make sure our code works with all of them) that are built in multiple configurations (different compilers, 32/64-bit, debug/release; and don't forget cross-compilation). And this is what build2 is optimized for.

Still, it often makes sense to use a system-installed library or tool instead of building it from source. Take libsqlite3 as an example: it is stable, available and works the same way pretty much everywhere – it probably doesn't make much sense to build it from source for every configuration.

The build2 build system will happily use a system-installed library; there is even pkg-config integration to extract additional information such as the location of headers, additional libraries to link, etc. Also, the package manager has a notion of system-installed packages (that sys: prefix) which allows you to instruct bpkg to assume the package is available from the system rather than building it from source.

What makes you think you will succeed when so many have failed?

Good question. For over 10 years now we've been developing and using in production our own, GNU make-based build system. On several occasions we've also contributed new features to GNU make (second expansion in pattern rules, shortest stem matching, etc).

Despite being based on the build tool not designed for today's complexity, build is surprisingly powerful. For example, build can generate (based on templates) other build systems, such as autotools and VC++ projects/solutions. We call this meta-build and it's the reason why most people who use our projects have never heard of build.

However, the original make design and GNU make codebase do show their age. The biggest problem is the lack of out of the box Windows support. Or, more precisely, the lack of a POSIX shell on Windows. Also, the interaction of complex features like parallel builds, auto-generated source code, and automatic header dependency extraction can lead to order-dependent behavior. For example, a C++ preprocessor is sometimes executed on a header file as it is being generated. We euphemistically call them build aberrations and they will not be easy to fix in make. But, despite all of this, many design decisions that you will find in build2 have been conceived and battle-tested first in make and build.

When it comes to the package manager, our view is pretty simple: given a sane build system with a consistent interface across platforms, writing a package manager is actually not that hard. The core functionality of bpkg was designed and implemented in a month. However, try to support different/multiple/insane build systems and things get hairy fast.

Code Synthesis also has a solid track record of making complex C++ tools work across multiple platforms/compilers, including cross-compilers (see ODB for a good example). So, to summarize, we have experience developing build tools/systems and using them to deliver real-world, cross-platform C++ products. But then, again, none of this is a guarantee. We will just have to wait and see.

Why did you write it in C++ instead of <your favorite language>?

Let's see: we need a cross-platform (must work out of the box on Windows) and reasonably efficient (nobody likes to wait for their up-to-date checks) language that most C++ developers already know (so that they can extend build2 if they need to). Any language comes to mind that fits the bill?

Plus, nothing gives as good a reality check as trying to build your own build system and package your own package manager. And, yes, even the web interface is written in C++ (we've made a little EDSL on top of libstudxml that produces nicely indented XHTML5).

Why aren't you hosting on GitHub?

Mostly because we want complete control of our infrastructure. Also, many things on GitHub are optimized for the convenience of new/inexperienced git users and not for the efficient handling of large volumes of information by the experienced ones. Case in point: issue tracker; I am much more efficient using my mail client (mutt) and text editor (emacs) than a web browser form.

Also, if this setup works for the Linux kernel, surely it should work for us; see the "Why kernel development still uses email" LWN article for details.

How do I submit bugs/patches?

Simply email them to the users@build2.org mailing list. You don't even need to subscribe. While you can just fire and forget (we do realize that by submitting a bug report or a patch you are doing us a favor, so we are not picky), we would appreciate it if you took a look at the posting guidelines.

What are your plans for cppget.org?

We hope it will become The C++ Package Repository, the place people get third-party C++ libraries and tools from. If/when this happens, it will probably be a good idea to turn the control over to someone impartial, like Standard C++ Foundation (to this effect, we are running build.org and cppget.org on completely separate infrastructures from the start).

Do you plan to support binary packages?

Not at this early stage but maybe in the future.

The problem with supporting binary packages is you have to make sure the binary package built on someone else's machine matches your environment precisely (if you think this is a Windows-only problem, check the recent GCC dual ABI mess). If we start cutting corners here we will end up in tears (and long mailing list threads). One interesting approach would be to dump the entire build system state into, say, an SQLite database and then ship that with the binary package. On the other end we can then load this database and verify that all the prerequisites (like system headers, compiler versions, etc) match.

Also, there are some ideas floating around about making bpkg automatically generate binary packages in rpm and deb formats. After all, the bpkg manifest already includes all/most of the information needed.

Can I run my own private/public repositories?

Yes, the entire toolchain, including the server-side web interface, is open source, licensed under the MIT License.

How are you going to make money from this?

We are not sure. Maybe we won't. Honestly, if this succeeds, and we won't have to manually test/distribute our tools and libraries on the myriad of platform/compiler/IDE combinations, then it would have been all worth it.