I beg to differ. First of all, it is not the fault of boost, that it breaks notoriously with some compilers, but most of the time the fault of these compilers. As different libraries often use the same approach we are also very likely to run into the same problems with each of them. Instead of the chance to use a preinstalled boost version which every hpc system has plenty of, we damn the users to install just another template only library nearly no hpc system has available at default (like Catch2 or Brigand).

If you want to replace boost::mpl I would suggest to implement a very basic metaprogramming library ourself loosely leant to boost::mp11 if you are afraid of staying with boost at all.

I don't see the alpaka test cases as a crucial part of the library. It is important for our CI, but for users the alpaka examples are way more important. So I would use for testing whatever suits best and somebody has experience in, so we can easily add more tests to make alpaka more bug prove.

most of the time the fault of these compilers

chicken-egg question but boost lacks coverage, seldomly backports and has a long turn-around for fixes (half a year+). Do to the lack of coverage, previously fixed compilers have often be broken in new releases again (that's a boost issue, not a compiler issue).

Instead of the chance to use a preinstalled boost version which every hpc system has plenty of

Sorry, bot 50% HPC systems do not have a preinstalled boost and they do not have the latest version not the proper C++ standard flavor, etc.. The reason your HPC world has them is that I trigger the TU Dresden and HZDR Admins on the day of releases of Boost to build it for us.

we damn the users to install just another template only library nearly no hpc system has available at default

No, we will ship those internally. See the "amalgated header" note. My goal is less (=0) pre-installed dependencies (that have nothing to do with the selected accelerator), not more. On install one can still decide to use an external dependency via CMake switches, but by default things just work.

implement a very basic metaprogramming library

Maybe, but we need functionality that is more than basic in some places, such as folds that need to be performant.
Nevertheless, alpaka has it's own metaprogramming lib currently.

I don't see the alpaka test cases as a crucial part of the library.

Exactly, that's an optional dependency anyway and I still would get rid of Boost.Test for various reasons. It's old (tons of work-arounds for C++98 do not make it more stable), slow, same compiler issues as the rest of boost and it has unfixed issues at least on OSX that I debugged for days.

Experience with Catch2: Please see the link I posted. We have experience with it, use it permanently in other projects, integrate it already into CMake and CI just as Boost.Test and switching existing tests is super easy and extending after it's in place (copy paste from openPMD-api) as well.

@ax3l @theZiz Hi Axel and Alexander. I'm from the IBM XL compilers team, and if you have any specific examples of issues with compiling boost with XL, I would love to hear them so we can improve in the future.

@trudeaun Hi Nicole, thank you for contacting us! Sorry for the little delay, we had two holidays this week in Germany and are catching up :)

For us as alpaka developers and maintainers, our main concern is to find a balance between self-implementation and re-use of generic parts of the library. The main problem we face too often in HPC is that dependencies that are not primarily tested on HPC systems cause us trouble in both installing and using them.

Since our targets are compilers like XL for OpenPOWER machines (e.g. Summit), and we see splendid performance with XL, we want to make sure this plays well with our dependencies.

With boost itself the following issues are scary that we would like to address:

• typical HPC compilers, such as xlC for OpenPOWER are not part of the acceptance test of boost releases: full list; maybe you can find a way to license them just as the intel compiler a version for the boost main integration tests? (builds at least in C++11 and any newer C++ standard xlC supports would be excellent; as soon as we can use newer C++ versions we will definitely do so)
• coverage for tests in terms of continuous integration is not given, increasing the burden on releases and bugfixes/backports - I am not sure what your open source policies are, but XL compilers running in travis-ci or any feasible way to add automated compile tests with XL to GitHub repositories even if only hand-selected repos would help a ton of people :)
• the current build system (bjam) has xlC as a supported toolset, which is already great so one can run the tests with b2 - can you confirm the xlc.bjam is still up to date?; we need to make sure new build system based cmake will also run out of the box with XL in case they do tweaks
• the other way around: do you know if the latest boost releases are part of the XL acceptance tests?

@ax3l Hi Axel, thanks very much for this detailed response! Glad to see that XL demonstrated some good results for you, and for sharing that article with us. You might see an accepted answer here for some more information about our future C++ standard support.

• I can look into getting XL added to boost's acceptance test list, but it's possible that our current level of C++ standard support will be an issue.
• I can also look into getting XL running on travis-ci. We have our Linux compiler available on Dockerhub, so getting it on Travis may be a similar process to that. I was reading that Intel has been waiting for awhile to get their compiler added to travis-ci!
• IBM's patch file for boost was last updated for V13.1.6, around its release in Dec 2017. This included updates to xlc.jam. A patch file for the latest release V16.1 (made available Apr 27, 2018) will be coming soon.
• XL posts which boost releases are part of XL acceptance tests here. For XL V16.1, boost V1.59 is part of the XL acceptance test, which is a back-dated version (the latest release seems to be V1.67).

@trudeaun thank you very much for the detailed response as well! It's great to read XL is incorporating clang front-end technology.

Currently, boost still works with C++98 and there are very recent ideas to move it to C++11 only (which is a good plan to get rid of a lot of work-arounds, legacy code and inefficient implementations). So XL should fit into integration as long as all workaround-configs can be expressed (usually via defines in boost config headers).

XL CE via your docker image is a great idea! At least for alpaka, our travis builds are now based on docker images anyway. Have you already considered contributing XL CE to http://spack.io/ as well? We can assist you with it if you like, as we plan to use it on many HPC machines (icc example) and I also use it for CI on GitHub.

Great to hear the jam files are up-to date! Thanks for the link to the lastest boost acceptance test. I have to admit 1.59 is indeed a bit old for our requirements (usually 1.62+) and will especially work badly with CUDA 9+ apps (which will push to Boost 1.65.1+) as a requirement. Can you trigger to add maybe 1.66 or 1.67 to the acceptance tests? Has the patch mentioned in the link for 1.59 been upstreamed to boost?

@ax3l http://spack.io seems to be making use of our XL CE already, and I can see from github that they are already integrating our latest release, V16.1.

I can see if we can add more recent boost versions into our acceptance tests. As for upstreaming the patches to boost, many years ago we used to do this, but in recent years, due to the long delay and large effort required by IBM and boost for their upstreaming process, we decided to publish the patches ourselves instead.

spack: yes, it recognizes the compiler if present, but we could also add the install :) (link above)

Patches for more recent versions would be fantastic, thank you for your efforts! Yes, ideally we would just describe the limitations as config files and add CI to keep it clean.

In the patches you describe (ftp://public.dhe.ibm.com/software/xlcpp/boost/bst1590/boost_modfile_le_1316.txt) for the last tested boost release, I see a domination of

defined(__IBMCPP__) && !defined(__ibmxl__)

groups.
It seems to me that newer versions of xlC (on linux only?) do not (only) define the first macro anymore but the second XL C/C++ identifiers. Do you have more information on that and the different kind of identifiers for XL releases and product lines we could encounter?

I think from the current patch on your homepage and the dominance on identifier-only changes we have a great chance to upstream a lot via a compiler/xlclinux.hpp (or OpenPOWER) file :)

@ax3l Sorry, I missed the icc link the first time. Let me look into adding the XL install into spack - great idea!

As far as I know, __ibmxl__ is always defined in all versions of XL C/C++ for Linux for little endian (LE) distributions, but is not defined for any version of IBM's other XLC compiler products.

__IBMCPP__ is not defined by default in the XLC LE Linux compiler, but it can be defined by using -qxlcompatmacros. In the XLC AIX, BE Linux, BG/Q, and z/OS compiler products, __IBMCPP__ is defined by default for C++ code only (C code has __IBMC__ defined by default instead).

Just grepped over the repo and found the following uses of boost:

• boost/align (boost::alignment::aligned_alloc) remove boost::aligned_alloc #1094
• boost/predef
• boost/config removed
• boost/current_function.hpp Replace BOOST_CURRENT_FUNCTION with C++11 __func__ #762
• boost/fibers (optional)
• boost/test (test only) removed
• boost/mpl (test only) removed
• boost/assert (test only) removed
• boost/math (boost::math::relative_difference) (test only) removed

boost::alignment::aligned_alloc could be replaced

• by C11 aligned_alloc but this will most probably never be implemented by Visual Studio
• std::aligned_alloc which is available starting from C++17

boost::alignment::aligned_alloc could be replaced

• by C11 aligned_alloc but this will most probably never be implemented by Visual Studio
• std::aligned_alloc which is available starting from C++17

For windows we could use https://docs.microsoft.com/de-de/cpp/c-runtime-library/reference/aligned-malloc?view=vs-2019

boost aligned_alloc is removed with #1094

Perfect, the compiler predefs we should be able to re-build based on set defines.

We have a new dependency: Boost.Atomic. Right now this is non-optional but can easily be turned into an optional dependency.