Linux › Section P · Compiling software from source › Topic P.1

Build systems, source packages, and GNU Autoconf

Nearly everything on a Linux system is available as source code, and sometimes building it yourself is the only way to get the version or options you need. Most C projects follow the same unpack, configure, build and install routine, and GNU Autoconf is the classic system behind it.

Why build from source?

Your distribution's packages are the normal way to install software: they are tested, they track security fixes, and they upgrade themselves. Building from source makes sense for particular reasons: you need a newer release than the distribution offers, you need a compile-time option it left out, you want to install a private copy somewhere unusual or several versions side by side, or you want to understand how a program is put together. The general routine has four steps. 1. Unpack the source archive. 2. Configure it for your system. 3. Build it, usually with make. 4. Install it. Each step has its own checks.

Build systems

A plain Makefile, as in the topic "make", rarely works unchanged across systems, because compilers, library locations and features differ. Build systems solve this by inspecting the machine first and generating the build files. GNU Autotools (Autoconf, Automake and Libtool) is the oldest widely used one and still builds many core GNU utilities: you will recognise it by a configure script. CMake, recognised by CMakeLists.txt, is common in large C and C++ projects, and Meson, recognised by meson.build and usually paired with the fast Ninja build tool, is now used by GNOME, systemd and many others. Other languages bring their own tools, such as Cargo for Rust.

Unpacking safely

Source usually arrives as a compressed tar archive (.tar.gz, .tar.bz2 or .tar.xz), as covered in "Archiving and compressing files". First verify the download: compare sha256sum output with the checksum the project publishes, or check its GPG signature where one is provided. Then list the contents before extracting with tar tf; GNU tar detects the compression itself. A well-made archive puts everything under one directory named after the package. If it does not, make a new directory and extract inside it. Delete any archive containing absolute paths such as /etc/passwd or entries with ../, which could overwrite system files.

Worked example: inspecting two archives

1. tar -tzf tidy-2.1.tar.gz lists tidy-2.1/, tidy-2.1/configure, tidy-2.1/README, tidy-2.1/src/ and tidy-2.1/src/main.c. Every path starts with one directory, so extracting here is safe. 2. tar -tzf messy.tar.gz lists ./, ./main.c and ./Makefile. These files would land in the current directory among whatever is there already, so run mkdir messy-src and cd messy-src first. 3. Verify before trusting either: sha256sum tidy-2.1.tar.gz > SUMS then sha256sum -c SUMS printed tidy-2.1.tar.gz: OK. All three steps were run with GNU tar and coreutils in bash; a real check compares against the project's published sum, not one you just made.

Reading the package

After unpacking, read before building. README describes the package and often its requirements; INSTALL gives build instructions and special options. Then classify the other files. Build files include configure, Makefile.in, configure.ac, CMakeLists.txt or meson.build. Source files end in .c and .h, with C++ in .cc, .cpp or .cxx. A source package should not contain .o files or ready-made binaries; if it does, it was packed carelessly, so run make clean before building to be sure everything is compiled fresh. Look for a NEWS or ChangeLog file too, to see what changed in this release.

How Autoconf works

The package developer runs Autoconf, which turns configure.ac into a portable shell script named configure, and Automake, which turns Makefile.am into Makefile.in templates. You, the builder, only run ./configure. It tests your system (which compiler exists, which headers and functions are present) and fills in the templates, producing Makefiles, a config.h header of results, config.status and a detailed log, config.log. Then make builds, make check runs the package's tests, and make install installs. A successful configure does not guarantee a successful build, but it makes one very likely.

configure options and variables

./configure --help lists everything. The key option is --prefix: the default is /usr/local, so programs go to /usr/local/bin and libraries to /usr/local/lib, and --prefix=$HOME/opt/tool keeps a test build inside your home directory. --bindir, --sbindir and --libdir set individual locations. --disable-shared builds only static libraries, avoiding run-time library searches. --with-package=dir or --without-package choose optional dependencies; the exact names vary by project. Pass compiler settings as variables: CPPFLAGS=-I/opt/geo/include for headers, LDFLAGS="-L/opt/geo/lib -Wl,-rpath,/opt/geo/lib" for libraries. Put header paths in CPPFLAGS, not CFLAGS, because configure often runs the preprocessor alone.

Worked example: a private test build

Build the GNU hello demonstration package without touching the system. 1. Unpack its archive from ftp.gnu.org/gnu/hello and cd into the new directory. 2. ./configure --prefix=$HOME/opt/hello prints many checking ... lines and ends by creating Makefiles. 3. make builds; make check runs the tests. 4. make -n install prints what installing would do; confirm that every path is under $HOME/opt/hello. 5. make install, with no sudo, since the prefix is yours. 6. Run $HOME/opt/hello/bin/hello. Removing the whole $HOME/opt/hello directory later undoes everything. The same steps work for almost any Autoconf package; only the configure options differ.

When configure fails; other targets

configure stops with a short message and the advice See 'config.log' for more details. The log is long and ends with a dump of variables, so open it in less, press G to jump to the end, then search backwards with ? for the last check that failed; the real compiler error is usually just above it. A classic case is CPPFLAGS=Iinclude, missing its hyphen, which ends in C compiler cannot create executables. Useful generated targets: make distclean removes everything configure made, make install-strip installs binaries without debugging symbols, and make DESTDIR=/tmp/stage install installs into a staging tree under /tmp/stage, which packaging tools use.

pkg-config

Libraries installed in different prefixes are hard for builds to find. pkg-config solves this: each library installs a small .pc file giving its version and the exact compiler and linker flags it needs. pkg-config --cflags libpng prints the -I options, pkg-config --libs zlib prints something like -lz, and pkg-config --modversion gives the version. configure scripts call it automatically. The .pc files usually come with the library's development package. For a library in an unusual place, add its pkgconfig directory to PKG_CONFIG_PATH, for example PKG_CONFIG_PATH=/opt/geo/lib/pkgconfig ./configure. CMake and Meson builds follow the same idea: cmake -S . -B build or meson setup build, then build and install.

Common mistakes to avoid

Extracting an archive without listing it first and scattering files through the current directory. Skipping the checksum or signature check. Running make install as root with the default prefix, then losing track of which files it added under /usr/local. Putting -I options in CFLAGS when configure expects CPPFLAGS. Forgetting the hyphen in -I or -L. Reading only the last line of a failed configure instead of searching config.log. Leaving stale objects from a previous build, fixed by make clean or distclean. Building a package from source when a supported distribution package would have done the job.

Key takeaways

Build from source for a needed version, option or location; otherwise prefer distribution packages. The routine is unpack, configure, build, install: verify the download, list it with tar tf, read README and INSTALL. Autotools packages use ./configure, then make, make check and make install; CMake and Meson are the other common systems. --prefix (default /usr/local) and variables such as CPPFLAGS and LDFLAGS control configure, config.log explains its failures, DESTDIR stages an install, and pkg-config supplies library flags.

Flashcards

Flashcard

Build routine

Unpack, configure, build, install

Verify and inspect the archive first.

Test your understanding

Question 1 of 5

tar tzf app.tar.gz shows ./main.c, ./Makefile and ./README with no common directory. What should you do before extracting?

Question 2 of 5

A test build should install under your home directory, not the system. Which configure option does that?

Question 3 of 5

configure cannot find geo.h, which is in /opt/geo/include. Which command is the recommended fix?

Question 4 of 5

configure fails with: C compiler cannot create executables. See 'config.log' for more details. What is the best next step?

Question 5 of 5

What does pkg-config --libs zlib print on a typical system?

← Previous topicNext topic →