Using HPX on Compiler Explorer

Using HPX on Compiler Explorer#

Compiler Explorer (CE) is a browser-based tool that compiles C++ code in real time and shows assembly, program output, and optimization results side by side. It is widely used for quick experiments, conference demos, and sharing reproducible snippets. This page explains how to build HPX for CE, how to link against its static libraries from a raw compiler command, and how to use the sandbox header that HPX ships specifically for constrained environments.

Sandbox environment constraints#

CE executes compiled programs inside a Linux container (nsjail) with no network access and limited resources. The constraints that matter most for HPX are:

  • No network namespace — sockets cannot be opened. Any attempt to initialise the distributed runtime will fail with EACCES at startup.

  • A small PID budget (typically 32 PIDs) — HPX starts cleanly within this limit when the worker thread count is kept low.

  • A short execution timeout — CE’s public instance allows roughly 10 seconds. An HPX fork or self-hosted instance can relax this to 30 seconds, which is enough for benchmarks on problem sizes of one million elements or more.

  • Static linking is required — the sandbox does not augment LD_LIBRARY_PATH, so shared HPX libraries are not found at runtime.

Building HPX for Compiler Explorer#

HPX ships a godbolt-minimal configure preset in CMakePresets.json that encodes all the flags needed for a CE-compatible build:

$ cmake --preset godbolt-minimal
$ cmake --build --preset godbolt-minimal

This preset enables:

  • HPX_WITH_STATIC_LINKING=ON — bundles everything into the .a archives that CE links against.

  • HPX_WITH_DISTRIBUTED_RUNTIME=OFF — local-only runtime. Compiler Explorer cannot launch a second locality, so the distributed runtime is omitted from this build.

  • HPX_WITH_NETWORKING=OFF — disables the parcelset so HPX never attempts to open a network socket.

  • HPX_WITH_FETCH_ASIO=ON — downloads Asio via FetchContent, removing the need for a system-level Asio installation.

  • HPX_WITH_FETCH_HWLOC=ON — fetches hwloc via FetchContent so the CE install does not depend on a system hwloc package.

  • HPX_WITH_MALLOC=system — uses the system allocator; avoids a jemalloc or tcmalloc dependency.

  • HPX_WITH_TESTS=OFF, HPX_WITH_EXAMPLES=OFF, HPX_WITH_DOCUMENTATION=OFF, HPX_WITH_TOOLS=OFF — skips everything that CE does not need.

The preset produces four static libraries under build/godbolt-minimal/lib/:

libhpx_wrap.a
libhpx_init.a
libhpx.a
libhpx_core.a

These are the only artifacts that CE consumes.

Note

The HPX_WITH_CXX_STANDARD variable pins the C++ standard for the build (e.g. -DHPX_WITH_CXX_STANDARD=20). This is HPX’s own cache variable and is distinct from CMAKE_CXX_STANDARD, which HPX rejects unless HPX_USE_CMAKE_CXX_STANDARD is also set.

Linking without CMake#

CE’s backend compiles user code with a raw g++ or clang++ invocation. A static HPX install ships one archive per module (libhpx_logging.a, libhpx_include_local.a, and so on) in addition to libhpx_wrap.a, libhpx_init.a, libhpx.a, and libhpx_core.a. Those module objects are not merged into libhpx_core.a, so linking only the four umbrella libraries leaves symbols such as detect_environment() undefined. Group every libhpx_*.a archive, then add Boost, hwloc, and the usual system libraries:

$ g++ -std=c++20 -O2 -o my_program my_program.cpp          \
    -isystem /path/to/hpx/include                           \
    -isystem /path/to/boost/include                         \
    -L/path/to/hpx/lib                                      \
    -DHPX_APPLICATION_EXPORTS                               \
    -Wl,-wrap=main                                          \
    -Wl,--start-group /path/to/hpx/lib/libhpx_*.a           \
    -Wl,--end-group                                         \
    -lpthread -ldl -lrt

Two details here are easy to get wrong:

Library link order. Put -Wl,-wrap=main and the libhpx_*.a group on the link line together. --start-group / --end-group is required because the module archives have circular references. Linking only -lhpx_wrap -lhpx_init -lhpx -lhpx_core is not enough.

The -Wl,-wrap=main flag. Including hpx/hpx_main.hpp (see Re-use the main() function as the main HPX entry point) works by re-routing control through HPX’s own entry point before the user’s main is called. On Linux this is implemented via the linker’s --wrap option; the flag must therefore appear on the linker command line, not merely in the compile flags. Without it, the HPX runtime is never initialised and all API calls crash at startup. See Linux implementation for a detailed explanation of the mechanism.

Important

-DHPX_APPLICATION_EXPORTS must be passed as a preprocessor definition when compiling application code against the static libraries. Omitting it causes link failures related to HPX’s symbol visibility macros.

Linking without CMake on Windows (MSVC)#

-Wl,-wrap=main is a GNU ld option and has no MSVC equivalent. On Windows, hpx/hpx_main.hpp instead redefines main as hpx_startup::user_main. The real main that starts the HPX runtime comes from the header itself in a static build and from hpx_init.lib otherwise, and hpx_wrap.lib makes the runtime run hpx_startup::user_main as its first HPX thread. A raw cl.exe build therefore needs no wrap option, only hpx_wrap.lib on the link line. From a Developer PowerShell for Visual Studio:

PS> $hpx = 'C:\path\to\hpx'
PS> $boost = 'C:\path\to\boost'
PS> cl /std:c++20 /O2 /EHsc /MD /GR /bigobj /permissive- `
      /Zc:__cplusplus /Zc:preprocessor /Zc:inline /Zc:throwingNew `
      /Zc:rvalueCast /Zc:strictStrings `
      /DHPX_APPLICATION_EXPORTS /I "$hpx\include" /I "$boost\include" `
      /I "$hpx\hwloc_installed\include" `
      my_program.cpp `
      /link /LIBPATH:"$hpx\lib" /LIBPATH:"$hpx\hwloc_installed\lib" `
      hpx_wrap.lib hpx_init.lib libhwloc.dll.a psapi.lib shlwapi.lib `
      dbghelp.lib

The HPX module libraries (hpx_core.lib and hpx.lib, or one .lib per module when HPX is built with HPX_WITH_MODULES_AS_STATIC_LIBRARIES=ON) do not have to be listed: with MSVC the installed headers name them through #pragma comment(lib, ...), so /LIBPATH: to the install’s lib directory is enough. Define HPX_NO_AUTOLINK to turn this off and list the libraries yourself. hpx_wrap.lib and hpx_init.lib are not auto-linked and have to be passed explicitly, as do hwloc and the Windows libraries psapi.lib, shlwapi.lib, and dbghelp.lib (needed for the stack traces that HPX_WITH_STACKTRACES enables by default). No --start-group equivalent is needed: link.exe searches every library on the command line until all symbols are resolved.

The /Zc: options, /bigobj, and /permissive- are the options the HPX::hpx CMake target passes on to its consumers. hwloc_installed is where the install puts the prebuilt hwloc that HPX_WITH_FETCH_HWLOC=ON downloads, and libhwloc.dll.a is its import library; point the /I and /LIBPATH: options at your own hwloc otherwise. Put the hwloc DLL (also installed into $hpx\bin) next to the executable or on PATH before running it.

A godbolt-minimal build uses Boost as a header-only dependency, so only the Boost include path is needed. Configurations with HPX_WITH_GENERIC_CONTEXT_COROUTINES=ON also link the Boost context, thread, and chrono libraries; add those to the /link part together with a /LIBPATH: for them.

To build a program that does not include hpx/hpx_main.hpp itself, add /FIhpx/hpx_main.hpp to the compile options above. This is what HPX::auto_wrap_main does on MSVC: /FI force-includes the header into every source file, which is the same as including it at the top of each one. When compiling more than one source file this way against a static HPX, also pass /DHPX_AUTO_WRAP_MAIN_FORCE_INCLUDE so the default main comes from hpx_wrap.lib once instead of from the header in every source file, and add /SUBSYSTEM:CONSOLE to the /link part. No object file defines main then, and without /SUBSYSTEM the linker can’t pick an entry point and fails with LNK1561. CMake passes /SUBSYSTEM:CONSOLE for console executables on its own.

Prefer HPX::hpx plus HPX::wrap_main or HPX::auto_wrap_main from CMake where possible, since they supply these options and the module libraries automatically. Compiler Explorer’s execution sandbox runs Linux, so the Windows path matters for MSVC compile-only sessions and for local godbolt-minimal builds on Windows.

Writing HPX code for Compiler Explorer#

The simplest way to write an HPX program for CE is to include hpx/hpx_main.hpp. This header arranges for the HPX runtime to be initialised before main runs, so the body of main can call any HPX API function directly:

#include <hpx/hpx_main.hpp>
#include <hpx/algorithm.hpp>
#include <hpx/execution.hpp>

#include <iostream>
#include <numeric>
#include <vector>

int main()
{
    std::vector<int> v(1'000'000);
    std::iota(v.begin(), v.end(), 0);

    long long sum = hpx::transform_reduce(
        hpx::execution::par, v.begin(), v.end(), 0LL,
        std::plus<>{}, [](int x) { return static_cast<long long>(x); });

    std::cout << "sum = " << sum << "\n";
}

Caution

Include hpx/hpx_main.hpp in exactly one translation unit — the file that contains main. Including it in more than one file causes a multiple definition error for the include_libhpx_wrap variable that controls runtime initialisation.

The hpx/experimental/sandbox.hpp header#

HPX ships hpx/experimental/sandbox.hpp for code running in constrained environments. Timing helpers (measure, benchmark) are header-only. detect_environment() and the print() members are compiled into libhpx_core and are available in local-only builds, including godbolt-minimal. It provides:

  • Environment introspection — hpx::experimental::sandbox::detect_environment() returns an environment_info struct describing the number of physical cores, processing units, NUMA domains, and active HPX worker threads. The is_sandbox flag is set when the COMPILER_EXPLORER or HPX_SANDBOX environment variable is present, letting code adapt its behaviour automatically.

  • Timing — hpx::experimental::sandbox::measure(fn, iterations) runs a callable fn for iterations repetitions (with one warmup pass) and returns the mean execution time in milliseconds.

  • Comparative benchmarking — hpx::experimental::sandbox::benchmark(label, seq_fn, par_fn, iterations) measures both a sequential and a parallel version of the same computation, computes speedup and parallel efficiency, and returns a benchmark_report struct. Calling report.print(std::cout) produces a fixed-width table that renders cleanly in CE’s output pane.

A typical usage pattern looks like this:

#include <hpx/hpx_main.hpp>
#include <hpx/algorithm.hpp>
#include <hpx/execution.hpp>
#include <hpx/experimental/sandbox.hpp>

#include <algorithm>
#include <iostream>
#include <numeric>
#include <vector>

int main()
{
    namespace sb = hpx::experimental::sandbox;

    sb::describe_environment(std::cout);

    std::vector<int> data(500'000);
    std::iota(data.begin(), data.end(), 0);

    auto report = sb::benchmark(
        "transform_reduce",
        [&]() {
            hpx::transform_reduce(
                hpx::execution::seq,
                data.begin(), data.end(), 0LL,
                std::plus<>{}, [](int x) { return (long long)x; });
        },
        [&]() {
            hpx::transform_reduce(
                hpx::execution::par,
                data.begin(), data.end(), 0LL,
                std::plus<>{}, [](int x) { return (long long)x; });
        });

    report.print(std::cout);
}

The output includes sequential and parallel mean times, speedup, parallel efficiency, and a verdict (Excellent scaling, Good scaling, Moderate scaling, Limited scaling, or No speedup).

Note

All functions in hpx/experimental/sandbox.hpp must be called from within a running HPX runtime, i.e., from an HPX thread. Calling them before hpx::init or after hpx::finalize is undefined behaviour. When using hpx/hpx_main.hpp, the body of main satisfies this requirement automatically.

Known limitations in sandboxed environments#

  • Single locality only. godbolt-minimal sets HPX_WITH_DISTRIBUTED_RUNTIME=OFF. There is no second locality inside CE’s sandbox, and distributed APIs are not part of this build.

  • Networking is disabled. HPX_WITH_NETWORKING=OFF means all parcelport-dependent functionality (remote actions, distributed data structures) is unavailable regardless of what the code requests.

  • Thread count is limited by the sandbox’s CPU allocation. CE’s public instance typically exposes two cores. Use --hpx:threads=N on the command line or hpx::init_params::cfg in code to set the worker count explicitly rather than relying on hardware detection. See Launching and configuring HPX applications for details.

  • macOS link flag differs. The -Wl,-wrap=main flag is Linux-specific. On macOS the linker uses -Wl,-e,_initialize_main instead. CE runs Linux containers, so this only matters when building the CE integration locally on macOS for testing.

  • Windows does not use -Wl,-wrap=main. HPX_WITH_DYNAMIC_HPX_MAIN is unavailable on Windows, so hpx/hpx_main.hpp uses the main macro instead. Link HPX::wrap_main without a GNU wrap flag.

Contents