Skip to content

Latest commit

 

History

History
609 lines (489 loc) · 21.4 KB

README.md

File metadata and controls

609 lines (489 loc) · 21.4 KB

Autotools-based PHP build system

This is a brief introduction to Autotools and how it is used in the PHP build system.

Index

1. Introduction

*nix build system (Linux, macOS, FreeBSD, OpenBSD, Solaris, Haiku, etc.) in PHP uses Autoconf to build a configure shell script that further creates a Makefile to build sources to executable binaries.

# Create Autotools configure script
./buildconf

# Configure PHP build and create Makefile
./configure

# Build PHP
make

The buildconf is a simple shell script wrapper around autoconf and autoheader command-line tools. It checks required Autoconf version, creates configure command-line script and main/php_config.h.in header template.

When running the ./configure, many checks are done based on the host and targeted system. Things like C headers availability, C symbols, required library dependencies, etc.

The configure script creates Makefile where the make command then builds binary files from C source files. You can optionally pass the -j option with the number of simultaneous jobs so the build runs faster in parallel.

make -j 10

When compiling is done, the tests can be run with:

make TEST_PHP_ARGS=-j10 test

Note

Number of simultaneous jobs is often the number of available logical CPU cores (a.k.a threads) of the build machine and can be also automatically calculated using the $(nproc) on Linux, or $(sysctl -n hw.ncpu) on macOS and BSD-based systems.

make -j $(nproc)

PHP *nix build system is pretty much standard GNU Autotools build system with few customizations. It doesn't use Automake and it bundles some 3rd party files for easier installation across various systems out there without requiring installation dependencies. Autotools is a veteran build system present since early C/C++ days. It is used for most Linux ecosystem out there and it might cause issues for C developers today.

2. Directory structure

PHP build system is a collection of various files across the php-src repository:

📂 <php-src>
└─📂 build
  ├─📄 ax_*.m4           # https://github.com/autoconf-archive/autoconf-archive
  ├─📄 config-stubs      # Adds extension and SAPI config*.m4 stubs to configure
  ├─📄 config.guess      # https://git.savannah.gnu.org/cgit/config.git
  ├─📄 config.sub        # https://git.savannah.gnu.org/cgit/config.git
  ├─📄 genif.sh          # Generator for the internal_functions* files
  ├─📄 libtool.m4        # https://git.savannah.gnu.org/cgit/libtool.git
  ├─📄 ltmain.sh         # https://git.savannah.gnu.org/cgit/libtool.git
  ├─📄 lt*.m4            # https://git.savannah.gnu.org/cgit/libtool.git
  ├─📄 Makefile.global   # Root Makefile template when configure is run
  ├─📄 order_by_dep.awk  # Used by genif.sh
  ├─📄 php.m4            # PHP Autoconf macros
  ├─📄 pkg.m4            # https://gitlab.freedesktop.org/pkg-config/pkg-config
  ├─📄 print_include.awk # Used by genif.sh
  └─📄 shtool            # https://www.gnu.org/software/shtool/
└─📂 ext
  └─📂 bcmath
    └─📄 config.m4       # Extension's Autoconf file
  └─📂 date
    └─📄 config0.m4      # Suffix 0 priority adds file before other config.m4 extension files
  └─📂 mysqlnd
    └─📄 config9.m4      # Suffix 9 priority adds file after other config.m4 files
  └─📂 opcache
    └─📂 jit
      └─📄 Makefile.frag # Makefile fragment for OPcache Jit
    └─📄 config.m4       # Autoconf file for OPcache extension
└─📂 main
  ├─📄 build-defs.h      # Generated by configure
  ├─📄 build-defs.h.in   # Template for build definitions
  ├─📄 php_config.h      # Generated by configure
  ├─📄 php_config.h.in   # Generated header template by buildconf
  └─📄 php_version.h     # Generated by release managers using `configure`
├─📂 pear
└─📂 sapi
  └─📂 cli
    └─📄 config.m4       # Autoconf M4 file for SAPI
├─📂 scripts
└─📂 TSRM
  └─📄 threads.m4        # Autoconf macros for pthreads
└─📂 Zend
  ├─📄 Makefile.frag     # Makefile fragment for Zend Engine
  └─📄 Zend.m4           # Autoconf macros for Zend Engine
├─📄 buildconf           # Wrapper for autoconf and autoheader tools
└─📄 configure.ac        # Autoconf main input file for creating configure script

3. Build system diagram

PHP *nix build system using Autotools

4. Build requirements

Before you can build PHP on Linux and other Unix-like systems, you must first install certain third-party requirements. It's important to note that the names of these requirements may vary depending on your specific system. For the sake of simplicity, we will use generic names here. When building PHP from source, one crucial requirement is a library containing development files. Such libraries are typically packaged under names like libfoo-dev, libfoo-devel, or similar conventions. For instance, to install the libxml2 library, you would look for the libxml2-dev (or libxml2-devel) package.

Required:

  • autoconf
  • make
  • gcc
  • g++
  • pkg-config
  • libxml2
  • libsqlite3

Additionally required when building from Git repository source code:

  • bison
  • re2c

5. The configure command-line options

Configuration can be passed on the command line at the configuration phase:

./configure VAR=VALUE --enable-FEATURE --with-PACKAGE

With Autoconf, there are two main types of command-line options for the configure script (--enable-FEATURE and --with-PACKAGE):

  • --enable-FEATURE[=ARG] and its belonging opposite --disable-FEATURE

    --disable-FEATURE is the same as --enable-FEATURE=no

    These normally don't require 3rd party library or package installed on the system. For some extensions, PHP bundles 3rd party dependencies in the extension itself. For example, bcmath, gd, etc.

  • --with-PACKAGE[=ARG] and its belonging opposite --without-PACKAGE

    --without-PACKAGE is the same as --with-PACKAGE=no

    These require 3rd party package installed on the system. PHP has even some libraries bundled in PHP source code. For example, the PCRE library and similar.

Other custom options that don't follow this pattern are used for adjusting specific features during the build process.

See ./configure --help for all available configuration options and variables. Configure options for all PHP versions are listed also in the Autotools directory.

Some common arguments can be passed to command-line options:

  • To build extension as shared: --enable-EXT=shared or --with-EXT=shared.
  • Some options accept multiple arguments separated by comma (,). For example, passing the library location and similar: --with-EXT=shared,/usr

6. Determining platform

With Autotools there are several shell variables that help determine the platform characteristics such as CPU, operating system and vendor name. When using macros AC_CANONICAL_BUILD, AC_CANONICAL_HOST, and AC_CANONICAL_TARGET in the M4 files, config.sub and config.sub scripts help determine the values of variables build_alias, host_alias, and target_alias.

Users can also manually override these variables for their specific case using the --build, --host, and --target configure options.

In M4 files platform can be then determined using above shell variables in variety of ways:

AS_CASE([$host_alias], [*freebsd*|*openbsd*],
  [AC_MSG_NOTICE([Action that is run only on FreeBSD and OpenBSD systems.])])

7. Common checks

7.1. Testing if program works

There are 3 main Autoconf macros that check if certain test code is successful.

Let's check a simple C program:

#include <stdio.h>

int main(void)
{
    printf("Hello World");

    return 0;
}

7.1.1. AC_COMPILE_IFELSE

AC_COMPILE_IFELSE([AC_LANG_PROGRAM([#include <stdio.h>],
  [printf("Hello World")])],
  [php_cv_func_printf_works=yes],
  [php_cv_func_printf_works=no])

The AC_LANG_PROGRAM macro will expand this into something like this:

#include <stdio.h>

int main(void)
{
  printf("Hello World")
  ;
  return 0;
}

The AC_COMPILE_IFELSE will run the compilation step, for example:

gcc -o out -c hello_world.c

7.1.2. AC_LINK_IFELSE

AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <stdio.h>],
  [printf("Hello World")])],
  [php_cv_func_printf_works=yes],
  [php_cv_func_printf_works=no])

This will run compilation and linking:

gcc -o out hello_world.c

7.1.3. AC_RUN_IFELSE

This will compile, link and also run the program to check if the return code is 0.

Issue with AC_RUN_IFELSE is when doing so-called cross-compilation. That is building C software on one platform with purpose of running it on some other platform. In this case the program cannot be run and we cannot be sure of if it is running successfully or not.

AC_RUN_IFELSE([AC_LANG_PROGRAM([#include <stdio.h>],
  [printf("Hello World")])],
  [php_cv_func_printf_works=yes],
  [php_cv_func_printf_works=no],
  [php_cv_func_printf_works=cross-compiling])

This does something like this:

gcc -o out hello_world.c
./out

7.2. Functions

Testing if function exists within the given header.

A common way to check for function is using the AC_CHECK_FUNC or AC_CHECK_FUNCS macros. These check if the linker sees the function in the usual libraries (libc and the ones appended to the LIBS variable). However, many times, functions also have their belonging headers, so it makes sense to check for both using this:

AC_CHECK_HEADER([priv.h], [AC_CHECK_FUNCS([setpflags])])

This first checks if header priv.h exists, and if so, it then checks if linker sees the function. They can be used like this:

#ifdef HAVE_SETPFLAGS
# include <priv.h>
#endif

/* ... */

/* Call setpflags. */
#ifdef HAVE_SETPFLAGS
    setpflags(...)
#else
    /* ... */
#endif

8. GNU Autoconf Archive

To reuse the code there is a community collection of Autoconf macros available at autoconf-archive.

PHP is not using Automake so it includes them like this in configure.ac:

m4_include([build/ax_..._.m4])
m4_include([build/...macro.m4])
# ...

They can be than called and expanded in the m4 code:

AX_MACRO_CALL(...)

When using Automake, these can be automatically included like this:

AC_CONFIG_MACRO_DIR([path/to/m4/dir])

However, the aclocal from Automake is needed for this to work.

9. Parser and lexer files

Parser and lexer files are generated upon the build step (make) with bison and re2c tools based on the targets in Makefile.frag files.

There is also a helper shell script available that generates these files when developing or releasing PHP source code, otherwise they are generated during the build phase upon the make invocation.

./scripts/dev/genfiles

Autotools-based PHP build system files related to bison and re2c:

📂 <php-src>
└─📂 build
  └─📄 php.m4               # Autoconf macros with Bison and re2c macros
└─📂 ext
  └─📂 json
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 pdo
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 pdo_mysql
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 pdo_pgsql
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 pdo_sqlite
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 phar
    └─📄 Makefile.frag      # Makefile fragment
  └─📂 standard
    └─📄 Makefile.frag      # Makefile fragment
└─📂 sapi
  └─📂 phpdbg
    └─📄 Makefile.frag      # Makefile fragment
└─📂 scripts
  └─📂 dev
    └─📄 genfiles           # Parser and lexer files generator helper
└─📂 Zend
  └─📄 Makefile.frag        # Part of Makefile related to Zend files
└─📄 configure.ac           # Minimum Bison and re2c versions settings

10. PHP installation

Caution

Before running the make install command, be aware that files will be copied outside of the current build directory.

The default way to install PHP using Autotools across the system directories can be done like this:

# Build configure script:
./buildconf

# Configure PHP build:
./configure --prefix=/usr

# Build PHP in parallel:
make -j$(nproc)

# Run tests in parallel:
make TEST_PHP_ARGS=-j$(nproc) test

# Finally, copy built files to their system locations:
make INSTALL_ROOT=/stage install

The optional --prefix configure option sets the location where the built files layout is put. Default prefix is /usr/local. The optional INSTALL_ROOT environment variable can set the parent location where the prefixed built files layout will be put. By default, it is empty and it is usually used to set the stage directory to perform additional tasks on the built files before being packaged or distributed.

Note

The INSTALL_ROOT variable name is used in PHP from the early Autotools days. GNU standards, CMake, and other build systems use a more common name DESTDIR.

The files are then copied to a predefined directory structure. PHP Autotools has another optional configure option --with-layout=[GNU|PHP] (GNU or PHP layout). It defines the installation directory structure. By default, it is set to a PHP style directory structure.

Directory locations can be adjusted with several Autoconf default options. Here only those relevant to PHP are listed:

  • --prefix=PREFIX - install architecture-independent files in PREFIX; Default: /usr/local
  • --exec-prefix=EPREFIX - install architecture-dependent files in EPREFIX; Default: <PREFIX>
  • --bindir=DIR - set the user executables location; Default: EXPREFIX/bin
  • --sbindir=DIR - set the system root executables location; Default: EPREFIX/sbin
  • --sysconfdir=DIR - set the read-only single-machine data location; Default: PREFIX/etc
  • --localstatedir=DIR - set the modifiable single-machine data location; Default: PREFIX/var
  • --runstatedir=DIR - set the modifiable per-process data location; Default: LOCALSTATEDIR/run; (Autoconf 2.70+)
  • --libdir=DIR - set the object code libraries location; Default: EPREFIX/lib
  • --includedir=DIR - set the project C header files location; Default: PREFIX/include
  • --datarootdir=DIR - set read-only architecture-independent data root; Default: PREFIX/share
  • --datadir=DIR - set read-only architecture-independent data location; Default: DATAROOTDIR
  • --mandir=DIR - set the man documentation location; Default: DATAROOTDIR/man

When packaging the PHP built files for certain system, additional environment variables can help customize the installation locations and PHP package information:

  • EXTENSION_DIR - absolute path that overrides path to extensions shared objects (.so, or .dll files). By default, it is set to /usr/local/lib/php/extensions/no-debug-non-zts-20230901 or /usr/local/lib/php/20230901, when using the --with-layout=GNU. To override it in the context of the prefix, it can be also set like this:

    ./configure --prefix=/usr EXTENSION_DIR=\${prefix}/lib/php/extensions

Common practice is to also add program prefix and suffix (for example, to have php84 and similar):

  • --program-prefix=PREFIX - prepends built binaries with given prefix.
  • --program-suffix=SUFFIX - appends suffix to binaries.
./configure \
  PHP_BUILD_SYSTEM="Acme Linux" \
  PHP_BUILD_PROVIDER="Acme" \
  PHP_BUILD_COMPILER="GCC" \
  PHP_BUILD_ARCH="x86_64" \
  PHP_EXTRA_VERSION="-acme" \
  EXTENSION_DIR=/path/to/php/extensions \
  --with-layout=GNU \
  --with-pear=\${datadir}/pear \
  --localstatedir=/var \
  --sysconfdir=/etc \
  --program-suffix=84 \
  # ...

See ./configure --help for more information on how to adjust these locations.

Default PHP Autotools directory structure with GNU layout (--with-layout=GNU):

📦 <INSTALL_ROOT>                # 📦                             # Stage directory
└─📂 ${prefix}                   # └─📂 /usr/local                # Installation prefix
  ├─📂 ${bindir}                 #   ├─📂 bin                     # Executable binary directory
  └─📂 ${sysconfdir}             #   └─📂 etc                     # System configuration directory
    ├─📂 php-fpm.d               #     ├─📂 php-fpm.d             # PHP FPM configuration directory
    ├─📄 pear.conf               #     ├─📄 pear.conf             # PEAR configuration file
    └─📄 php-fpm.conf.default    #     └─📄 php-fpm.conf.default  # PHP FPM configuration
  └─📂 ${includedir}             #   └─📂 include                 # System include directory
    └─📂 php                     #     └─📂 php                   # PHP headers
      ├─📂 ext                   #       ├─📂 ext                 # PHP extensions header files
      ├─📂 main                  #       ├─📂 main                # PHP main binding header files
      ├─📂 sapi                  #       ├─📂 sapi                # PHP SAPI header files
      ├─📂 TSRM                  #       ├─📂 TSRM                # TSRM header files
      └─📂 Zend                  #       └─📂 Zend                # Zend Engine header files
  └─📂 ${libdir}                 #   └─📂 lib
    └─📂 php                     #     └─📂 php                   # PHP shared libraries, build files, PEAR
      ├─📂 20230901-zts-debug    #       ├─📂 20230901-zts-debug  # PHP shared extensions (*.so files)
      └─📂 build                 #       └─📂 build               # Various PHP development and build files
  ├─📂 ${sbindir}                #   ├─📂 sbin                    # Executable binaries for root privileges
  └─📂 ${datarootdir}            #   └─📂 share                   # Directory with shareable files
    └─📂 ${mandir}               #     └─📂 man
      ├─📂 man1                  #       ├─📂 man1                # PHP man section 1 pages for *nix systems
      └─📂 man8                  #       └─📂 man8                # PHP man section 8 pages for *nix systems
    ├─📂 ${PHP_PEAR}             #     ├─📂 pear                  # PEAR installation directory
    └─📂 php                     #     └─📂 php
      └─📂 fpm                   #       └─📂 fpm                 # Additional FPM static HTML files
  └─📂 ${localstatedir}          #   └─📂 var                     # The Linux var directory
    └─📂 log                     #     └─📂 log                   # Directory for PHP logs
  └─📂 ${runstatedir}            #   └─📂 var/run                 # Runtime data directory
📦 /                             # 📦 /                           # System top level root directory
└─📂 tmp                         # └─📂 tmp                       # System temporary directory
  └─📂 pear                      #   └─📂 pear                    # PEAR temporary directory
    ├─📂 cache                   #     ├─📂 cache
    ├─📂 download                #     ├─📂 download
    └─📂 temp                    #     └─📂 temp

This is how the default PHP layout directory structure looks like (--with-layout=PHP). Notice the difference of the shared extensions directory and the share directory being named php:

📦 <INSTALL_ROOT>
└─📂 /usr/local
  ├─📂 bin
  └─📂 etc
    ├─📂 php-fpm.d
    ├─📄 pear.conf
    └─📄 php-fpm.conf.default
  └─📂 include
    └─📂 php
      ├─📂 ext
      ├─📂 main
      ├─📂 sapi
      ├─📂 TSRM
      └─📂 Zend
  └─📂 lib
    └─📂 php
      ├─📂 build
      └─📂 extensions
        └─📂 no-debug-non-zts-20230901  # PHP shared extensions (*.so files)
  └─📂 php                              # Directory with shareable files
    └─📂 man
      ├─📂 man1
      └─📂 man8
    └─📂 php
      └─📂 fpm
  ├─📂 sbin
  └─📂 var
    ├─📂 log
    └─📂 run
📦 /
└─📂 tmp
  └─📂 pear
    └─📂 temp

11. See more

Useful resources to learn more about Autoconf and Autotools in general: