Friday, July 04, 2008

Getting Started with Boost

Getting Started with Boost








Table of Contents
1.Introduction
2.Purpose & Motivation
3.Building Boost
3.1The Boost Build Process
3.2Building Boost on Windows
3.3Building Boost on Linux
3.4Boost Binaries
4.Using the Boost Libraries
4.1Linking to Boost libraries on Windows
4.2Linking to Boost libraries on Unix-like system


1. Introduction


The Boost C++ Libraries ( http://boost.org/ ) are an open source peer reviewed collection of C++ libraries, aimed at extending and complementing the functionality of the Standard C++ Libraries in several different directions. These libraries are available under the Boost Software License, which is one of the most liberal of open source licenses, and unlike GPL, allows free use of the Boost libraries in both Open Source and proprietary (closed-source) projects.

Boost represents a collection of libraries some of which are as simple as conversion utilities and some of which are as complex as recursive decent parsers of context-free grammars. A significant number of these libraries are the sort you would like to use everyday - file system manipulation, graph algorithms, heap memory life cycle management, variant data types, regular expressions, conversion utilities, zip and bzip2 file streams, signals and slots mechanisms, etc. Some have compelling features use of which gives your code a level of readability and edge that only C++ can - template driven lambda functions and expression construction. Finally, a good number of these libraries will make their appearance in the upcoming C++ standard (codenamed C++0x). In short, if you are a serious C++ programmer, you cannot afford to stay ignorant about Boost.

2. Purpose & Motivation



Purpose of this short tutorial introduction to Boost is to show the C++ programmer new to Boost, how to download, build and install the Boost libraries on Windows or Linux developer machines, and then get started with writing programs using these libraries.

For building I have chosen Windows and Linux as two representative platforms. I would only be showing how to use the "first-choice" compilers on these platforms - Microsoft C++ (14.0 or above) and GNU g++ (4.1 or above), both of which happen to be free. The reader should, in principle, be able to extrapolate the details to other platforms and compilers.

For the introduction to using the Boost Libraries, the intention is to cover basic use cases of as many Boost libraries as possible - limiting the size of the individual illustrative programs, but giving enough insight into their non-trivial use. The emphasis will be on those libraries that have made it to C++ standard already. But in due course, we will try to cover all the Boost libraries.

Finally, the sections on Building Boost and Using the Boost Libraries are fairly independent. Therefore the interested reader can read either section ahead of the other.


3. Building Boost

Boost distributions are available for download in both source and binary distribution formats. However, just as the best Linux kernel for your box is the one you can build by tweaking the default configuration, similarly the best Boost distribution for your box is the one you build on your box with your preferred compiler. Building Boost involves a fair number of configuration steps, and there are several optional features that you can turn off in the description below. The optional features are normally described in gray boxes so that they stand-out. This section is a step-by-step, hands-on guide to building the Boost library from source.

3.1 The Boost build process

3.1.1 Basic Ingredients

The boost sources are available in the form of a compressed tar ball or a zip archive. Download the Boost sources, [ Latest version 1.35.0 ], from http://boost.org/ (or http://sourceforge.net/projects/boost). The size of the download is around 22 MB. Download one of the zip, tar.gz/tgz, or tar.bz2 source archives. On Windows, if you are downloading a tar.gz or tar.bz2 archive, this could get downloaded as boost_<version>.tar.tar and you will need to rename it to boost_<version>.tar.gz or boost_<version>.tar.bz2 as the case may be.

The Boost source does not require any external configuration before building on a specific platform - so there are no make config or configure scripts to run. In fact, there is no make. Instead, Boost uses a build system that uses Boost-Jam or bjam, a spin-off from the Perforce Jam build utility. Bjam reads jam files just as make reads Makefiles. However, we are not about to go into the details of the syntax of the Jam files.

3.1.2 Build configuration

The Boost libraries can be built as a vast number of variants, all in one go or separately. We can have debug and release variants of the same library. For each such library, we can have static or shared libraries. For each such combination we can have threading-challenged or thread-aware libraries. And finally, for each such combination, we can have libraries which link statically or dynamically with the C++ Standard and Runtime Libraries of the platform, libraries which link with the debug versions of the C++ Standard and Runtime Libraries of the platform, libraries which have no optimization and inlining (even release libraries), etc. That's a mind-boggling array of combinations. We will stick to a simple set of rules:

  • Build only thread-aware libraries - not threading-challenged ones.
  • Build only shared libraries - not static ones.
  • Build both debug and release versions of each library.

The build system defaults automatically take care of certain things:

  • The libraries always link dynamically if the C++ Standard and Runtime libraries are available as shared libraries or DLLs.
  • The libraries always link with the debug versions of the C++ Standard and Runtime libraries for debug builds and release versions of the C++ Standard and Runtime libraries for release builds.
  • Optimization, inlining, etc is turned off for debug builds and done for release builds.

In addition, some features of some of the Boost Libraries, and the Boost.Python library itself are not built into the libraries by default but require additional third party libraries. We do not attempt to build the Boost.Python and Boost.MPI libraries in this tutorial, but we do attempt to build all the remaining third-party libraries and use them for the build. All sections which deal with these libraries have a gray background to indicate that these are optional steps and can be skipped for a basic functional build.


3.1.3 Optional enhancements

Hide

The following third-party, free libraries can be downloaded optionally, in order to support certain features of the Boost libraries which are by default not built. They are platform-agnostic downloads - so a single downloaded archive can be used on both Windows and Linux. If you don't need a particular feature from among these, you need not download the library required for it.

  • zlib - [latest version 1.2.3] from http://www.zlib.net/. Small download of around 500 K. Download one of the zip, tar.gz/tgz, or tar.bz2 source archives. On Windows, if you are downloading a tar.gz or tar.bz2 archive, this could get dowloaded as zlib-<version>.tar.tar and you will have to rename it to zlib-<version>.tar.bz2. Needed to support reading from / writing to zipped streams in Boost.IOStreams.

  • Download libbz2 [latest version 1.0.5] - from http://www.bzip.org/downloads.html. Small download of around 850 K. Download one of the zip, tar.gz/tgz, or tar.bz2 source archives. On Windows, if you are downloading a tar.gz or tar.bz2 archive, this could get dowloaded as bzip2-<version>.tar.tar and you will have to rename it to bzip2-<version>.tar.gz. Needed to support reading from / writing to bzip2-compressed streams in Boost.IOStreams.

  • Download ICU C++ library [latest stable version 4.0] - from http://www.icu-project.org/download/ or ftp://ftp.software.ibm.com/software/globalization/icu (supports anonymous logins). Download the zipped version for Windows. It is a moderately large download - about 12 MB. Needed to support Unicode regular expressions in Boost.Regex.

3.2 Building Boost on Windows

In order to build Boost 1.35 on Microsoft Windows, you need to have the following software installed.

  • Microsoft C++ compiler version 12.00.8168 or above. Version 12 was shipped with Visual C++ 6. The recommended version is Microsoft C++ compiler version 15.00. Version 15 ships with Visual C++ 9.0 (2008) and is available for free download in the form of the Microsoft Visual Studio 2008 Xpress Edition and Microsoft Visual C++ 2008 Xpress Edition.
  • Boost Jam v3.1.12 or above. This is a build utility like make based on Perforce Jam. Download the ntx86 version. It's a small download of about 120 KB. Latest version 3.1.16, can be downloaded from [ http://sourceforge.net/projects/boost ].

Hide

The following third-party, free libraries can be downloaded optionally, in order to support certain features of the Boost libraries that are not built by default. If you don't need a particular feature from among these, you need not download the library required for it.

  • Download Expat for Windows [latest version 2.0.1] - from http://sourceforge.net/projects/expat. Small download of around 535 K. This is a binary which needs to be installed on your build box. Needed to support GraphML vocabulary used by Boost Graph Library.

3.2.1 Build Procedure

  • Open a cmd shell.
  • Create a suitable directory for holding all the sources of individual libraries (Boost and others, if any). Make sure you don't have any spaces in its path name. This directory will henceforth be referred to as BOOST_BUILD_ROOT. On my system, I have chosen C:\src as the BOOST_BUILD_ROOT.

    Set the following environment variable:
    C:\src> set BOOST_BUILD_ROOT=C:\src
  • Change your current directory to BOOST_BUILD_ROOT, copy the source archive for Boost into BOOST_BUILD_ROOT and extract it in the current directory, doing the equivalent of "Extract here ...". If everything is successful, you should have the following directory:

    %BOOST_BUILD_ROOT%\boost_<version>
  • Hide
    Copy each of the source archives for zlib, libbz2 and icu into BOOST_BUILD_ROOT and extract each of them in the current directory, doing the equivalent of "Extract here ..." on each of the archives. If everything is successful, you should have the following directories:

    %BOOST_BUILD_ROOT%\bzip2-<version>
    %BOOST_BUILD_ROOT%\zlib-<version>
    %BOOST_BUILD_ROOT%\icu
  • Copy the boost-jam Windows binary archive into BOOST_BUILD_ROOT and extract it in the current directory, doing the equivalent of "Extract here ...". If everything is successful, you should have the following directory.

    %BOOST_BUILD_ROOT%\boost-jam-<version>-1-ntx86

    For simplicity, rename %BOOST_BUILD_ROOT%\boost-jam-<version>-1-ntx86 to %BOOST_BUILD_ROOT%\bjam. Add this directory to your PATH environment variable.

    C:\src> set PATH=%PATH%:%BOOST_BUILD_ROOT%\bjam
  • Hide
    Double click expat-win32bin-<version>.exe to install Expat. During installation, modify the default installation path so that the files are installed at:
    %BOOST_BUILD_ROOT%\Expat-<version>

    Make sure there are no spaces in the directory name as in "Expat 2.0.1", and rename it to "Expat-2.0.1".
    • After installation, the %BOOST_BUILD_ROOT%\Expat-<version>\Bin directory will contain the pre-built binaries. However, a better idea will be to build everything with the compiler of your choice - the one you will use for building Boost. For this, open the expat.dsw VC6 workspace under %BOOST_BUILD_ROOT%\Expat-<version>\Source. Depending on which compiler you intend to use to build Boost, could either open it in VC6 or in a more recent version of Visual C++.

      • On VC6: If you open it in VC6, then choose each of the listed projects - elements, expat, expat_static, expatw, expatw_static, outline and xmlfw, in turn. Build the chosen project once in Debug and once in Release configuration. To choose these projects and corresponding configurations, from the menu choose Build --> Set Active Configuration and choose the appropriate Debug or Release configuration. It builds these versions in the directories %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32\bin\Debug or %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32\bin\Release.

        Close the Visual C++ 6 IDE and rename the directory %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32 to %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32-vc6.

      • If you open it in a version newer than VC6, then you will be prompted to transform the workspace to a Visual C++ Solution - go ahead and complete this migration. Then choose the "expat" solution and build it. By default, it builds the debug version in the directory %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32\bin\Debug.

        To build the release version, in Visual C++ IDE, from the menu select Build-->Configuration Manager, and then choose the Release configuration from the drop down on the top-left, then close. Rebuild the solution. This time the release version of the binaries are created under the directory %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32\bin\Release.

        Close the Visual C++ IDE and rename the directory %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32 to %BOOST_BUILD_ROOT%\Expat-<version>\Source\win32-vc<X> where <X> is the version number of Visual Studio - 7 (for 2002), 7.1 (for 2003), 8 (for 2005) and 9 (for 2008).


  • Hide
    Build the ICU library inside Visual Studio using the solution file allinone.sln present under %BOOST_BUILD_ROOT%\icu\allinone. Make sure that you build in both Debug and Release configurations.
    The built binaries will be available under %BOOST_BUILD_ROOT%\icu\bin, and the static and import libraries will be available under %BOOST_BUILD_ROOT%\icu\lib. If you are using ICU version 3 or later, in order to build with Boost, you will need to rename a couple of libraries - icuin.lib to icui18n.lib, icuind.lib to icui18nd.lib, icudtd.lib to icudatad.lib, icuind.pdb to icui18nd.pdb and icudtd.pdb to icudatad.pdb.
  • Determine your toolset name, based on the compiler you're using. For Visual Studio, it is msvc. For GNU g++, it is, you guessed it - gcc.

    If you have only one version of Microsoft Visual Studio installed, you could just specify msvc for the toolset. If you have multiple versions, then depending on which version you want to use for the build, you should try one of the following as the toolset name:
    • msvc-9.0 (for Visual Studio .NET 2008)
    • msvc-8.0 (for Visual Studio .NET 2005)
    • msvc-7.1 (for Visual Studio .NET 2003)
    • msvc-7.0 (for Visual Studio .NET 2002)
    • msvc-6.0 (for Visual Studio 6)
  • Define the following environment variables:

      Hide
    • If you have downloaded and extracted the libbz2 source code, then:
      set BZIP2_SOURCE=%BOOST_BUILD_ROOT%\bzip2-<version>

    • If you have downloaded and extracted the zlib source code, then:
      set ZLIB_SOURCE=%BOOST_BUILD_ROOT%\zlib-<version>

    • If you have downloaded and built the eXpat source code, then:
      set EXPAT_INCLUDE=%BOOST_BUILD_ROOT%\Expat-<version>\Source\lib
      set EXPAT_LIBPATH=%BOOST_BUILD_ROOT%\Expat-<version>\Source\win32-vc<VCversion>\<variant>

      Here <VCversion> could be 6, 7, 7.1, 8, 9 or any further version depending on the version of VC++ you are using to build Boost.
      <variant> is Debug or Release depending on the kind of build you are doing for Boost.

      Finally, make two small changes to your the Jamfile.v2 script found under %BOOST_BUILD_ROOT%\boost_\libs\graph\build. You could backup your Jamfile.v2 file before making the following changes to it.

      1. Look for a line like: optional_reqs += .... In the block of directives following it, look for a directive like <find-static-library>expat. Change it to:

      <find-static-library>libexpatwMT

      2. Look for the line lib boost_graph. In the block of directives following this - look for a directive like <define>BOOST_GRAPH_NO_LIB=1. Following this line, add an additional line like this:

      <define>XML_STATIC

    • set HAVE_ICU=1
    • set ICU_PATH=%BOOST_BUILD_ROOT%\icu
    • set BOOST_BIN_ROOT=%BOOST_BUILD_ROOT%\boost-binary

  • Set the MS VC++ environment variables.
    • For VC++ 6.0, run the batch file vcvars32.bat under <MSVC_Install_Dir>\Microsoft Visual Studio\VC98\Bin. <MSVC_Install_Dir> is typically C:\Program Files.

    • For VC++ 7.0 and 7.1, run the batch file vcvarsall.bat under <MSVCS_Install_Dir>\Microsoft Visual Studio .NET 2002\VC or <MSVCS_Install_Dir>\Microsoft Visual Studio .NET 2003\VC.

    • For VC++ 8.0 onwards, run the batch file vcvarsall.bat under <MSVCS_Install_Dir>\Microsoft Visual Studio <version>\VC where <version> is 8 or 9.0.
  • Change current directory to: %BOOST_BUILD_ROOT%\boost_1_35_0. Make sure that the environment variables you set earlier ((BZIP2_SOURCE, ZLIB_SOURCE, EXPAT_LIBPATH, EXPAT_INCLUDE, HAVE_ICU, ICU_PATH and BOOST_BIN_ROOT) are available in the cmd shell from where you fire the above command.

    Make sure that the EXPAT_LIBPATH points to the release version of the Expat library.

    set EXPAT_LIBPATH=%BOOST_BUILD_ROOT%\Expat-<version>\Source\win32-vc<VCversion>\Release

    Fix your installation directory. Say C:\Program Files\Boost. To be on the safe side, figure out the DOS name of any directory which contains spaces, like Program Files. On all my systems it is PROGRA~1 and it may or may not be so on your systems. Set the following environment variable with something like:

    set BOOST_INSTALL_DIR=C:\PROGRA~1\Boost

    Finally, fire the following command to build the "release" variant:
    bjam threading=multi variant=release link=shared toolset=msvc-9.0 --build-dir="%BOOST_BUILD_ROOT%\boost-binary" --prefix="%BOOST_INSTALL_DIR%" --without-python --without-mpi install

    Reset any environment variables, like EXPAT_LIBPATH appropriately to the debug variant:
    set EXPAT_LIBPATH=%BOOST_BUILD_ROOT%\Expat-<version>\Source\win32-vc<VCversion>\Debug

    If you are using Boost 1.45 or later, there is one additional step that needs to be undertaken. You should not use a global bjam, instead build one from the Boost sources using a single script that's also provided with the sources. Do this:

    dosprompt> cd %BOOST_BUILD_ROOT%
    dosprompt> bootstrap.bat
    dosprompt> .\bjam.exe <options above as explained>

    Fire the following command to build the "debug" variant:
    bjam threading=multi variant=debug link=shared toolset=msvc-9.0 --build-dir="%BOOST_BUILD_ROOT%\boost-binary" --prefix="%BOOST_INSTALL_DIR%" --without-python --without-mpi install

    After the build, the build area can be cleaned of the intermediate files for each class of variants by replacing the "install" target with "clean" in the above commands.

We did not show the way to build static Boost libraries, and Boost libraries which link statically with the runtime. To create static libraries, simply replace link=shared with link=static in the above commands.

To build Boost libraries that link statically with the VC runtime, you have to just introduce an additional directive: runtime-link=static. Note that these libraries themselves can be either static or shared - but they always link statically with the VC Runtime libraries. Because Regex with ICU support necessarily links with ICU's runtime DLLs, the Regex library that links statically to the VC runtime cannot also link with the ICU DLLs. So ICU support is not available for Boost Regex which is statically linked to the VC Runtime. To account for this, you have to set HAVE_ICU and ICU_PATH environment variables to blank before you start the build - otherwise the build fails at the outset.

3.3 Building Boost on Linux (x86)


In order to build Boost 1.35 on GNU/Linux (on x86), you need to have the following software installed.

  • GNU gcc (g++) compiler version 3.4.x or above. The build exercise for this tutorial was done with GNU gcc (g++) compiler version 4.1.2 on OpenSUSE 10.2.
  • Boost Jam v3.1.12 or above. This is a build utility like make based on Perforce Jam. Download the version for linuxx86. It's a small download of about 65 KB. Latest version 3.1.16, can be downloaded from [ http://sourceforge.net/projects/boost ].
Hide

The following third-party, free libraries can be downloaded optionally, in order to support certain features of the Boost libraries. If you don't need a particular feature from among these, you need not download the library required for it.

  • Download Expat source archive [latest version 2.0.1] - from http://sourceforge.net/projects/expat. Small download of around 450 K. Needed to support GraphML vocabulary used by Boost Graph Library.

3.2.1 Build Procedure

  • Log on with a non-root account. The build should be done with a non-root account.
  • From the command prompt, create a suitable directory for holding all the sources of individual libraries (Boost and others, if any). Make sure you don't have any spaces in its path name. This directory will henceforth be referred to as BOOST_BUILD_ROOT. On my system, I have chosen ~/boost as the BOOST_BUILD_ROOT.

    Set the following environment variable:

    $ BOOST_BUILD_ROOT=~/boost # or whatever directory your created
    $ export BOOST_BUILD_ROOT
  • Change your current directory to BOOST_BUILD_ROOT, copy the source archive for Boost into BOOST_BUILD_ROOT and extract it in the current directory. If everything is successful, you should have the following directory:

    $BOOST_BUILD_ROOT/boost_<version>

    On my system, this is how the extraction commands looked:
    $ bzip2 -cd boost_1_35_0.tar.bz2 tar xf -
    Hide
  • Copy each of the source archives for zlib, libbz2, icu and expat into $BOOST_BUILD_ROOT, change your current directory to $BOOST_BUILD_ROOT and extract each of them in the current directory. If everything is successful, you should have the following directories:

    $BOOST_BUILD_ROOT/bzip2-<version>
    $BOOST_BUILD_ROOT/zlib-<version>
    $BOOST_BUILD_ROOT/icu
    $BOOST_BUILD_ROOT/expat-<version>

    On my system, this is how the extraction commands looked:

    $ bzip2 -cd zlib-1.2.3.tar.bz2 tar xf -
    $ gzip -cd bzip2-1.0.5.tar.gz tar xf -
    $ unzip -o icu4c-4_0-src.zip -d .
  • Copy the boost-jam Linux binary archive into BOOST_BUILD_ROOT and extract it in the current directory. If everything is successful, you should have the following directory.

    $BOOST_BUILD_ROOT/boost-jam-<version>-1-linuxx86

    I extracted it this way:
    $ gzip -cd boost-jam-3.1.16-1-linuxx86.tgz tar xf -

    For simplicity, rename $BOOST_BUILD_ROOT/boost-jam-<version>-linuxx86 to $BOOST_BUILD_ROOT/bjam. Add this directory to your PATH environment variable.

    $ PATH=$PATH:$BOOST_BUILD_ROOT/bjam
    $ export PATH


  • Hide
    Building expat:
    • Change your current directory to $BOOST_BUILD_ROOT/expat-<version> directory and look for a file called configure. If it does not exist, run the following command:

      $ ./buildconf.sh

    • Run the following commands to configure the build:
      $ ./configure --prefix=$BOOST_BUILD_ROOT/expat

      The argument --prefix=$BOOST_BUILD_ROOT/expat indicates that after building, the libraries, executables, include files, man pages, etc. should be installed under the directory $BOOST_BUILD_ROOT/expat. If you have write access to other standard directories for third-party installation like /opt, or you know the root password, then you could also do the following:

      $ ./configure --prefix=/opt/expat

    • If the above step (configure) is successful, run the following commands to build and install the eXpat library.

      $ make
      $ make install

      If you overrode the prefix option during the configure step with a path which requires root access to write to, then run the following command to do the install itself:

      $ su -c "make install"

      This requires you to know the root password.

      This will build expat and install it to the designated directory ($BOOST_BUILD_ROOT/expat or /opt/expat).

  • Hide
    Building ICU:
    • Change your current directory to $BOOST_BUILD_ROOT/icu/source.
    • Add execute permissions to a couple of scripts:
      $ chmod +x install-sh runConfigureICU configure config.sub mkinstalldir \
      config.guess config.status
      $ dox2unix install-sh runConfigureICU configure config.sub mkinstalldir \
      config.guess config.status


    • Run the following command:

      $ ./runConfigureICU Linux --prefix=/opt/icu # if you have the root password, or write access to /opt
      or
      $ ./runConfigureICU Linux --prefix=$BOOST_BUILD_ROOT/icu # if you neither have the root password, nor write access to /opt

    • Upon successful conclusion of runConfigureICU, run the following command:
      $ make
      $ make install
      or
      $ su -c "make install" # in case the installation target directory, set as prefix above, requires root access to write to
  • Determine your toolset name, based on the compiler you're using. For GNU g++, it is gcc.
  • Define the following environment variables:

      Hide
    • If you have downloaded and extracted the libbz2 source code, then:
      BZIP2_SOURCE=$BOOST_BUILD_ROOT/bzip2-<version>
      export BZIP2_SOURCE

    • If you have downloaded and extracted the zlib source code, then:
      ZLIB_SOURCE=$BOOST_BUILD_ROOT/zlib-<version>
      ZLIB_SOURCE

    • If you have downloaded and built the eXpat source code, then:
      EXPAT_INCLUDE=/opt/expat/include
      EXPAT_LIBPATH=/opt/expat/lib
      or
      EXPAT_INCLUDE=$BOOST_BUILD_ROOT/expat/include
      EXPAT_LIBPATH=$BOOST_BUILD_ROOT/expat/lib
      export EXPAT_INCLUDE EXPAT_LIBPATH

    • If you have downloaded and built the ICU source code, then:
      HAVE_ICU=1
      ICU_PATH=/opt/icu
      or
      ICU_PATH=$BOOST_BUILD_ROOT/icu
      export HAVE_ICU ICU_PATH

    • BOOST_BIN_ROOT=$BOOST_BUILD_ROOT/boost-binary

  • Change current directory to: $BOOST_BUILD_ROOT/boost_1_35_0.
    Make sure that the environment variables you set earlier (BZIP2_SOURCE, ZLIB_SOURCE, EXPAT_LIBPATH, EXPAT_INCLUDE, HAVE_ICU, ICU_PATH and BOOST_BIN_ROOT) are available in the shell you are in.

    Identify a location where you'd like to install the Boost libraries on your system. On my system, I chose /opt/boost, and because I have root access to this box, I could do the following, which is necessary:


    $ su -c "mkdir /opt/boost"
    $ su -c "chown arindam:devuser /opt/boost"


    If you are using Boost 1.45 or later, there is one additional step that needs to be undertaken. You should not use a global bjam, instead build one from the Boost sources using a single script that's also provided with the sources. Do this:

    $ cd $BOOST_BUILD_ROOT
    $ ./bootstrap.sh
    $ ./bjam <options above as explained>

    Finally, fire the following command to build the "release" variant:
    ./bjam threading=multi variant=release link=shared toolset=gcc --build-dir=$BOOST_BIN_ROOT --prefix=/opt/boost --without-python --without-mpi install

    Fire the following command to build the "debug" variant:
    ./bjam threading=multi variant=debug link=shared toolset=gcc --build-dir=$BOOST_BIN_ROOT --prefix=/opt/boost --without-python --without-mpi install

    If you have python-devel and python-tk installed, then you need not add the --without-python directive. Th build process would then generate Boost.Python libraries as well.

    After the build, the object files for each kind of variant can be cleaned by replacing the "stage" target with "clean" in the above commands.


3.4 Boost Binaries: What's in a name

If your build was successful, you should now have a set of DLLs and import libraries built from the source code. If you used the install target as in the examples above, then the libraries (DLLs, shared objects, import libraries, etc) should all be copied to some convenient location (like /opt/boost/lib or D:\Boost\lib). But the Boost libraries are created in an elaborate directory structure under $BOOST_BIN_ROOT/boost/bin.v2/libs, that needs some scrutiny.

A large number of the Boost libraries are source-only - in other words all the code in the library is in header files which get included in your own source code directly or through other headers. Needless to say, some of these are typically entirely template code. But a smaller yet significant number of Boost libraries are actually built as shared or static libraries, and need to be linked to your code if you want to use them. For each such library there is a directory created under $BOOST_BIN_ROOT/boost/bin.v2/libs, which goes by the name of the library. You will at least get the following directories:

  • date_time
  • filesystem
  • graph
  • iostreams
  • program_options
  • regex
  • serialization
  • signals
  • system
  • test
  • thread
  • wave

Since we did not choose to build Boost.Python and Boost.MPI, directories for these libraries will not be present. The directory structure under each of these directories looks a bit like this:

<library>/
---- build/
---- <toolset>/
-------- <variant>/
-------- <threading-model>

- Assuming that we have used the Microsoft Visual Studio 9, the toolset will be msvc-9.0.
- We have built both debug and release variants.
- We have only built thread-aware libraries. So for the date_time library, based on the above structure, you will get the following directories.

date_time/
---- build/
---- msvc-9.0/
------- debug/
------ threading-multi/
------- release/
------ threading-multi/

Since we specified "link" as "shared", only DLLs (Windows) or .so.* (Linux) will be build. Of course on Windows, along with each DLL, and Import Library (.lib) for linking is also creared. The dlls, libs and so files are created under the <threading-model> directory which in the above case is threading-multi.

Each .dll, .lib or .so file that is created has a naming convention as well, that needs to be understood:


  • On Unix, each library name starts with "lib". On Windows only static libraries have a name starting with lib, while the import libraries for DLLs and DLLs themselves miss the lib prefix. Barring this, and the file extension, the rest of the naming convention is same on both Windows and Unix.

    <library_name>-<toolset-tag>-<threading-tag>-<abi-tag>-<version>

    • On Windows, this takes the form:

      <library_name>-<toolset-tag>[-<threading-tag>][-<abi-tag>]-<version>.dll
      or <library_name>-<toolset-tag>[-<threading-tag>][-<abi-tag>]-<version>.lib

      The parts within brackets [...] represent optional segments that need not be present in all cases.

    • On Unix, this takes the form:

      lib<library_name>-<toolset-tag>-<threading-tag>-<abi-tag>-<version>.so[.<version>]


  • <library_name> - All library names start with boost_. Thus we have boost_date_time, boost_filesystem, etc.
  • <toolset-tag> need not be same as <toolset-name> - for the toolsets we have considered, it would be gcc (not g++) on Linux, or vc60, vc71, vc80, vc90, etc. on Windows.
  • <threading-tag> - For multi-threaded builds, it will be mt. For single threaded builds, this part does not appear in the name.
  • <abi-tag> - This is a string of one or more characters. For us, the characters of interest are s, g, d.
    s - indicates that the library is statically linked to the C++ Standard Library and Runtime Library.
    g - indicates that the debug versions of the standard and runtime support libraries are linked against. On a typical Windows installation you will get a debug version of the runtime libraries to link against, but this need not be the case on Unix boxes.
    d - indicates all debug symbols are available in the library, no inlining and optimization has been done while building the code.

So the Debug version of the Boost.Regex library on Windows should be called: boost_regex-vc90-mt-gd-1_35.dll and it should be located under %BOOST_BIN_ROOT%\boost\bin.v2\libs\regex\build\msvc-9.0\debug\threading-multi.
The corresponding import library should be called boost_regex-vc90-mt-gd-1_35.lib.

On Linux, the Boost.Regex shared library should be called libboost_regex-gcc-mt-d-1_35.so.1.35.0 and should be located under $BOOST_BIN_ROOT/boost/bin.v2/libs/regex/build/gcc/debug/threading-multi.

On Windows, the Release version of the Boost.Date_Time library should be called boost_date_time-vc90-mt-1_35.dll and it should be located under %BOOST_BIN_ROOT%\boost\bin.v2\libs\date_time\build\msvc-9.0\release\threading-multi. The corresponding import library should be called boost_date_time-vc90-mt-1_35.lib.

On Linux, the Boost.Date_Time shared library should be called libboost_date_time-gcc-mt-1_35.so.1.35.0 and should be located under $BOOST_BIN_ROOT/boost/bin.v2/libs/date_time/build/gcc/release/threading-multi.


4. Using the Boost Libraries


This is a brief section which discusses the way you should set your development environment up to use the Boost headers and link to the libraries effectively. There are several issues that have to be handled - and primarily the fact that there are so many variants of these libraries often causes some confusion. Moreover, there are different issues on Windows and non-Windows systems, so we have separate sections below, addressing these issues.

4.1. Linking to Boost libraries on Windows


On Windows, auto-linking is supported by most compilers, and by default Boost config headers arrange for Boost libraries to be linked automatically to your code. This is very convenient and you should not override it and try to do manual linking (which is possible but inordinately tedious). However, you still have to understand how your linker options affect which libraries actually get linked.

On all supported versions of Visual Studio (which is 7.0 onwards), we use one of the following switches to indicate the kind of linkage we want:

  • /MD - Link dynamically with the multi-threaded C Runtime.

  • /MDd - Link dynamically with the debug version of the multi-threaded C Runtime.

  • /MT - Link statically with multi-threaded C Runtime.

  • /MTd - Link statically with the debug version of the multi-threaded C Runtime.


Depending on which option from the above is chosen during compilation - a different version of a particular Boost library will be chosen for linking automatically. Let us take the example of Boost.Regex. If you check your installation directory, you will see the following DLLs/libs:

boost_regex-vc90-mt-1_35.dll
boost_regex-vc90-mt-1_35.lib
boost_regex-vc90-mt-gd-1_35.dll
boost_regex-vc90-mt-gd-1_35.lib

If you actually chose to build static variants of the Boost libraries (unlike what we did), you'd also see the following:

libboost_regex-vc90-mt-1_35.lib
libboost_regex-vc90-mt-gd-1_35.lib

Note that while the above are static libraries, they still link to the DLL versions of the MSVC Runtime libraries. Had they linked to the static versions, these would have been named:

libboost_regex-vc90-mt-s-1_35.lib
libboost_regex-vc90-mt-sgd-1_35.lib

These are static variants with static runtime-linking - yet another variant. We did not build these either.

4.1.2. Static linking

Now by default, all Boost libraries will link statically with your code. If you have built static libraries, that's fine. So by default, you'd end up linking to libboost_regex-vc90-mt-gd-1_35.lib in Debug builds and libboost_regex-vc90-mt-1_35.lib in Release builds.

4.1.2. Dynamic linking

But say, like we've done, you have chosen not to build any static linking variants of Boost. What do you do then? Simple - use a preprocessor define to indicate that you want to link dynamically to a particular library. In this case, you would define the symbol BOOST_REGEX_DYN_LINK using the construct:

/DBOOST_REGEX_DYN_LINK

on the command-line. If you want to use Dynamic linking exclusively, simple define BOOST_ALL_DYN_LINK instead. Beyond that, if you are linking to the Debug version of the VC Runtime, through the use of /MDd, you would link to boost_regex-vc90-mt-gd-1_35.dll via the Import Library boost_regex-vc90-mt-gd-1_35.lib. If you link to the Release / Retail vrsion of the VC Runtime, through the use of /MD, you would link to boost_regex-vc90-mt-1_35.dll via the Import Library boost_regex-vc90-mt-1_35.lib.

There is a notable exception to this rule - the Boost.Thread library. To link to the boost_thread DLL, define the preprocessor symbol BOOST_THREAD_USE_DLL on the command-line.

/DBOOST_THREAD_USE_DLL

To statically link, define the preprocessor symbol BOOST_THREAD_USE_LIB on the command-line.

/DBOOST_THREAD_USE_LIB

4.1.2. Static linking plus statically-linked runtime


If you use the /MT or /MTd switches, you are saying you want to link statically to the MSVC Runtime libraries. So in this case, you will by default link to libboost_regex-vc90-mt-sgd-1_35.dll (Debug) and libboost_regex-vc90-mt-s-1_35.dll (Release/Retail). Although not prohibited, linking dynamically to the Boost libraries under such circumstances is strictly discouraged. So never define BOOST_REGEX_DYN_LINK while using /MT or /MTd compiler directives.

[In pre-8.0 VC++, that is, upto Visual C++.NET 2003, the Microsoft C++ compiler actually accepted a set of switches - /ML and /MLd - to indicate static linking with a single-threaded runtime environment. On such environments, if you try to link your code with /ML, Boost library would try to link your code to boost_regex-vc90-sgd-1_35.dll or boost_regex-vc90-s-1_35.dll as the case may be. As of Visual C++ 2005 (version 8.0), these variants don't get built any longer.]

4.2. Linking to Boost libraries on Unix-like systems


On Unix, there is no auto-linking and there are no such high level distinctions as Debug and Release builds. The rough equivalent of a Debug build is a build with minimal to no compiler optimization, which includes debugging symbols in binaries. A rough equivalent of Release builds is a build with at least some compiler optimizations and no debugging symbols. Static linking with runtime is rarely used so the -s- and -sgd- variants of the Boost libraries are all but rarely useful.

An additional complexity to take into account is that on Windows, the Boost libraries silently link a lot of Windows standard libraries for you using Auto-linking. On Unices, you must link these manually and you have to know which ones to link. For all applications that use the multi-threaded variants of the Boost libraries, you have to link with libpthread. Similarly, if you are using Boost.Asio, you need to link with libsocket, (and on Sun Solaris, possibly libnsl, libresolv, etc). You will need to work these out and it's not difficult to do - at least after the first time you get a linker error. Also, unlike on Windows, the base name of a static and a shared library can be same on Unix, barring the extension (.a for static libraries and .so.* for shared libraries).

Based on the above facts and using gcc as a compiler, one could consider the following cases - once again using the regex library:

  • Shared linking with Boost debug libraries:

    g++ -g regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -lboost_regex-gcc41-mt-d -lpthread -licuuc -licudata -licui18n

  • Shared linking with Boost release libraries:

    g++ -O2 regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -lboost_regex-gcc41-mt -lpthread -licuuc -licudata -licui18n

  • Static linking with Boost debug libraries:
    Here you really have two options. You could choose to link to the static Boost Regex library with static runtime links in which case you can have no ICU support:

    g++ -g regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -static -lboost_regex-gcc41-mt-sd -lpthread

    The above links libpthread statically - which is usually not what you want - so fix that, we have to do this:

    g++ -g regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -Wl,-Bstatic -lboost_regex-gcc41-mt-sd -Wl,-Bdynamic -lpthread

    Or you could choose to link to the static Boost Regex library with dynamic runtime links - in which case you can have ICU support:

    g++ -g regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -Wl,-Bstatic -lboost_regex-gcc41-mt-d -Wl,-Bdynamic -lpthread -licudata -licui18n -licuuc

  • Static linking with Boost release libraries:

    g++ -O2 regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_libraries -Wl,-Bstatic -lboost_regex-gcc41-mt -Wl,-Bdynamic -lpthread -licudata -licui18n -licuuc

If you notice above, the library names are not different between the shared and static linking cases. We do not need to define BOOST_REGEX_DYN_LINK or BOOST_ALL_DYN_LINK. The only differentiator is the -static directive. The specifics vary from one Unix to another, and even more between different compilers.

11 comments:

chan said...

First, thanks a lot for your tutorial. But I got stuck at the very last step :
g++ -O2 regextest.cpp -o regextest -Ipath_to_boost_includes -Lpath_to_boost_lib
I don't understand what is actually "path_to_boost_includes"
and what is "path_to_boost_lib"
Could you give me one example ?
I tried those with my makefile look like this :
[quote]
CC = g++
CFLAGS = -Wall
PROG = r2

SRCS = ex3.cpp
LIBS = -I$BOOST_BIN_ROOT/boost_1_37_0/boost/bin.v2/libs -L$BOOST_BIN_ROOT/boost_1_37_0/boost/bin.v2/libs/regex/build/gcc-4.3.2/debug/threading-multi/libboost_regex-gcc43-mt-d-1_37.so.1.37.0

all: $(PROG)

$(PROG): $(SRCS)
$(CC) $(CFLAGS) -o $(PROG) $(SRCS) $(LIBS)

clean:
rm -f $(PROG)[/quote]
But it didn't work, a compiler gives me a ton of error message. I currently using Ubuntu 8.10 and the boost version that I installed is 1_37_0.
I wish you still here, and thanks in advance :D !

Arindam Mukherjee said...

Greetings chan, and thanks for working through the article. It helps debug the information provided.

Now path_to_boost_includes would typically be:

<boost_install_dir>

So if you extracted and installed boost at say /opt/boost/boost_1_37, then your include path directive will be:

-I/opt/boost/boost_1_37

The real include files will be under /opt/boost/boost_1_37/boost:

e.g. /opt/boost/boost_1_37/boost/regex.hpp

But in our code, we tend to follow the convention of including the boost directory in the include file name:

#include <boost/regex.hpp>

So, specifying till /opt/boost/boost_1_37 as the include path works well.

It is not much different if you are using a Windows compiler, like Microsoft C++ 14.0 or 15.0. There instead of -I, it will be /I - but the basic idea remains same.

On more recent Linux distros, where Boost comes as an optional package, if you install it, the include files are usually found under:

/usr/include/boost/...

So you don't need to specify -I/usr/include. The prebuilts libraries typically go into /usr/lib.

Arindam Mukherjee said...

So Chan, you should do this:

CXX = g++
LINK = g++
CXXFLAGS = -Wall -I$BOOST_BIN_ROOT/boost_1_37_0
LIBS = -L$BOOST_BIN_ROOT/boost_1_37_0/boost/bin.v2/libs/regex/build/gcc-4.3.2/debug/threading-multi

OBJS=$(SRCS:%.cpp=%.o)
$(OBJS)=$(SRCS)

SRCS = ex3.cpp

$(PROG): $(OBJS)
$(LINK) -o $(@) $(^) $(LIBS)

Some of the syntax is only valid for GNU Make. If you are using some other make or are on a non-linux platform, use $(OBJS) instead of $(^_).

chan said...

- Thanks a lot. I really appreciate it. I'm pretty new to Linux, that's why I don't understand it quite well. By the way, I have a question, how could you figure out how to install boost<> ? What should I need to study in order to do this stuff. Would you mind recommending me some good books and subject so that I can learn. I really really love C/C++ and Linux. I was just like in heaven when I found this article, I'm honest. I'm been working on installing boost for 6 months. I did a lot of google search but the only thing I found is the tutorial on Windows using Visual C++. I asked several forums, but not a lot of people care about it.
I've been programming in C++ for 2 years. And some subjects that I currently follow : openGL, gtk+, GNU Make, boost<> and some Linux API( I actually haven't started on this yet ). And I've just found two pretty good things : automake and autoconf.
Do you think I'm greedy when I tried to learn to many things at the same times. I know that I am, but I don't know why I always have that thought. I really addicted to C++ and Linux, but I think I'm in the wrong direction. So I think as an expert and experienced programmer, could you give me some advices and how to become a good programmer. I'm still on College, I will transfer next year. Sorry for my poor English. If something not clear, I will try to explain it again. Thank you very much Arindam Mukherjee !

chan said...

And I think I probably did wrong at some points. I check under usr/lib, but I only found libboost of version 1.35, which is the one that I installed using Synaptic Package Manager. The version that I want to install is version 1.37.0 ! :( So sad ! Further the make file that you give me got error, I think it's obviously the formate that I copied from this page, and makefile is sensitive to spaces. And I actually skipped 2 steps in your article :
#1. $ dox2unix install-sh runConfigureICU configure config.sub mkinstalldir \
config.guess config.status
My terminal always complain when I use dox2unix. And in version 1.37.0, I regconized in that folder doesn't have the file mkinstalldir( instead of mkinstalldirs ) and config.guess.
#2. su -c "chown arindam:devuser /opt/boost"
I don't know whether these 2 steps would make my process failed or not. But when I see the prebuilt library in the folder( I put at my Desktop ), I think it succeeded, but it might misses something.

Arindam Mukherjee said...

Hello Chan, thanks for the kind words. The Boost website - http://boost.org, has everything - of course you need to search through it. Besides, the Boost documentation is quite elaborate.

Now for the specific questions - when you built Boost using bjam, did you specify the "install" target like the following:

bjam threading=multi variant=release link=shared toolset=gcc --build-dir=$BOOST_BIN_ROOT --prefix=/opt/boost --without-python --without-mpi install

If you specified the above highlighted portions, then after building, all your version 1.37 boost libraries should have been copied to /opt/boost/lib. You can choose any directory in place of /opt/boost - say /home/chan/boost. In that case, all your libraries would be installed under /home/chan/boost/lib. So check what command-line you used to build it and install it. Unless you specified:

--prefix=/usr

you won't find your 1.37 libraries under /usr/lib.

Make sure you understand each and every step that you carry out. Second, try to build the Boost libraries without the optional packages. You can hide all the details about optional packages in article. Try to do that - it will be simpler.

Arindam Mukherjee said...

One clarification - if BOOST_BIN_ROOT is an environment variable then, in the Makefile, you should write:

$$BOOST_BIN_ROOT

instead of $(BOOST_BIN_ROOT). However, if BOOST_BIN_ROOT is a Makefile variable, then $(BOOST_BIN_ROOT) works.

Asymptote said...

This is single-handedly the most awesome article I've read in a long time. So much care and dedication went into writing it - thank you! Everything is thought through, and it works in the end.

Like chan, although I can write a long, tedious "g++ ..." line to get a simple boost example to work, I'm having trouble creating a make file to make things easier. This is mainly because I've never really used make files.

Thanks again.

Stefan said...

great article !

Reji said...

I want to use the dynamic boost libraries, so I'm trying to link using the /DBOOST_ALL_DYN_LINK. But, still I get the error "cannot open file libboost_*". Please note that the import libraries are generated with the filenames boost_*. I can get it working only by renaming the filenames from boost_* to libboost_*. Does anyone know how to solve this problem.

Thanks,
Reji

Reji said...

I want to use the dynamic boost libraries, so I'm trying to link using the /DBOOST_ALL_DYN_LINK. But, still I get the error "cannot open file libboost_*". Please note that the import libraries are generated with the filenames boost_*. I can get it working only by renaming the filenames from boost_* to libboost_*. Does anyone know how to solve this problem.

Thanks,
Reji