HPX V2.0.1 (TBD)#

General changes#

This point release fixes problems found in HPX V2.0.0. It consists of fixes that were merged to the master branch after V2.0.0 and that apply to the 2.0 code base. It doesn’t contain any of the new features of the master branch.

Synchronization, futures and LCOs#

  • hpx::stop_callback now waits for a callback running on a thread that is not an HPX thread before it is destroyed, and assigning to an hpx::stop_source releases the old stop state correctly.

  • hpx::condition_variable_any::wait with a stop token returns the result of the predicate when it observes a stop request.

  • Calling an hpx::packaged_task a second time no longer invokes its callable again.

  • hpx::unwrap can be used with containers of containers that hold no futures, which fixes hpx::reduce over std::string.

  • Coroutines returning an hpx::future or hpx::shared_future no longer use their frame after it was freed or cause a double free, and the asynchronous pack traversal (used by hpx::dataflow and hpx::when_all) destroys its iterators before its frame completes.

  • A timer no longer wakes a thread after the suspension it was set for has ended.

  • A timed wait on a future from a thread that is not an HPX thread no longer deadlocks the thread that makes the future ready.

Senders and receivers#

  • Fixed races and lifetime problems in run_loop, when_all_vector and async_rw_mutex that could lead to deadlocks, dropped continuations or calls to std::terminate.

  • Errors that are not exceptions are now propagated through the bridge from senders to futures.

  • when_all_vector can be connected as an lvalue and advertises its stopped completion.

Parallel algorithms#

  • Fixed wrong results of the parallel set operations (set_union, set_symmetric_difference, set_intersection, includes), of partial_sort, nth_element, make_heap, any_of and reduce_by_key, and of the reducing algorithms used with auto_chunk_size.

  • Fixed the iterators returned by reverse_copy, rotate_copy and the sequential ranges::transform and ranges::set_intersection.

  • More hpx::ranges algorithms accept a sentinel of a different type than the iterator, and hpx::reduce no longer requires the dereferenced iterator to be convertible to the type of the initial value.

  • rotate no longer reads past the end of the affinity masks when it runs with a sender policy.

  • The segmented versions of equal, mismatch, adjacent_difference, copy and transform handle partition boundaries correctly.

  • The parallel algorithms compile with guided_chunk_size.

  • for_loop_strided and for_loop_n_strided pass the correct induction values to every partition when they run with a parallel policy.

Schedulers, threads and containers#

  • Fixed the mapping of PU masks in the scheduler mode setters, a race of the steal masks in the local schedulers and the counting of threads in the thread queues.

  • The index-queue bulk executors handle failures to spawn a thread.

  • The SIGSEGV and termination handlers no longer deadlock on POSIX systems.

  • A thread created from outside a pool while its last worker leaves the scheduling loop is no longer stranded, which could make the runtime hang at shutdown.

  • Move assignment of concurrent_unordered_map and concurrent_unordered_set is synchronized.

Distributed runtime, parcels and collectives#

  • Fixed several problems during shutdown: parcels received after the worker threads stopped no longer abort the application, the parcelports are stopped before the distributed runtime is cleared, and the global barrier is detached only after all shutdown functions have run. The TCP parcelport no longer deadlocks while it shuts down.

  • The runtime no longer hangs when a startup function throws an exception, or when registering the message handlers or performance counters fails during startup.

  • The collective operations no longer deadlock on a communicator that can’t be resolved, or when a request for a later generation arrives before the earlier generations are done. Such a request now returns right away instead of waiting for the earlier generations.

  • A pinned migratable component is destroyed only when it is unpinned for the last time.

  • supervision_dispatch evicts peers fenced by failure detection from its registry, so snapshot_peers() no longer lists them, without removing a peer that has come back with a new epoch.

  • Output written to hpx::cout after the runtime has stopped is no longer lost and no longer aborts the application. hpx::endl and hpx::flush are aliases of std::endl and std::flush now, as they already were in builds without the distributed runtime.

  • Failures of sends that the parcel handler reschedules are reported through the write handler instead of being lost.

  • Components can be created on other localities from code compiled with nvcc.

  • The exceptions thrown for MPI errors show the file, the line and the error message of MPI.

Utilities and serialization#

  • Loading an empty std::list or std::deque from an archive clears the target container.

  • Fixed hpx::util::format, regex_from_pattern and parse_sed_expression for malformed input, and the size accounting and update_if of the LRU cache.

  • Objects serialized through smart pointers that are created during serialization are no longer mixed up with other objects at the same address, and archives with a binary filter (for instance compression) no longer lose data.

  • The names of Tracy plots stay valid for the whole run, tasks described by a function address are shown in Tracy, and performance counter values printed as csv go to the right tracing plot.

  • Fixed the deallocation size and max_size of the block_allocator of compute_local.

Build system and platform support#

  • HPX builds and runs with CUDA on Windows again.

  • Fixed building with nvcc and with CUDA 13.5, building the segmented algorithms with gcc 16, and including the HPX startup headers in builds with C++20 modules.

  • HPX::auto_wrap_main works with clang on Windows, which targets the MSVC ABI.

  • C++20 module builds register components under their own names. With MSVC 19.44 and newer, components, examples and tests are now also built in module mode.

  • Fetched dependencies no longer install their source paths and empty directories.

  • HPX is now compliant with the REUSE specification.

Closed issues#

  • Issue #8129 - hpx::ranges::contains and the two-range hpx::ranges::transform don’t compile with a sentinel of a different type

  • Issue #8127 - Segmented copy traversal mishandles partition boundaries and different localities

  • Issue #8124 - Unary segmented traversal mishandles partition boundaries and foreign localities

  • Issue #8115 - Several parallel algorithms don’t compile with guided_chunk_size

  • Issue #8105 - hpx::heap_until (par) 6% slower since #7963

  • Issue #8070 - Runtime hangs, busy, when a startup function throws on one of two localities

  • Issue #8064 - Parallel set operations lose results when a chunk size is set

  • Issue #8041 - hpx::util::format never returns for a replacement field with an explicit index and no closing ‘}’

  • Issue #8039 - parse_sed_expression throws std::out_of_range when the search part has no closing ‘/’

  • Issue #8030 - Prevent fenced eviction from removing a self-joined peer that starts a new epoch

  • Issue #8017 - Thread count of a scheduler can miss a thread that moves between queues

  • Issue #8013 - packaged_task::operator() invokes the callable again when called a second time

  • Issue #8011 - regex_from_pattern reads past the end of the string when the pattern ends in ‘[’

  • Issue #8003 - Several hpx::ranges algorithms don’t compile when the end of a range is a sentinel of a different type

  • Issue #7999 - Binary segmented traversal mishandles partition boundaries and crashes with different localities

  • Issue #7995 - hpx::stop_source copy/move assignment doesn’t release the old stop state’s source count

  • Issue #7990 - Make HPX compatible with https://reuse.software/

  • Issue #7971 - std::list and std::deque deserialization doesn’t clear the target when the archive holds an empty container

  • Issue #7969 - reduce_by_key ignores the user-supplied comparator for the last key

  • Issue #7960 - hpx::ranges::is_partitioned fails to compile with distinct C++20 sentinels (e.g., std::views::take_while)

  • Issue #7950 - Credit decrement to a disconnected locality can still throw when put_parcel() reschedules the send

  • Issue #7945 - hpx::unwrap fails to compile for a container of containers without futures, which breaks hpx::reduce of std::string

  • Issue #7940 - Sequential ranges::transform (two ranges) and ranges::set_intersection return the wrong in1/in2 iterators

  • Issue #7935 - Parallel set_union and set_symmetric_difference silently drop elements of the second range

  • Issue #7929 - partial_sort and nth_element return wrong results when the recursion depth limit is reached: the heap fallback selects the largest elements

  • Issue #7924 - Coordinate lifecycle epoch transitions with fenced-peer eviction

  • Issue #7915 - A timed wait on a future from a non-HPX thread deadlocks the thread that makes it ready

  • Issue #7913 - hpx::ranges::reverse_copy and rotate_copy with par return the wrong input iterator

  • Issue #7906 - Uncatchable std::terminate() and dropped continuations in async_rw_mutex

  • Issue #7905 - make_heap with par can produce an invalid heap for sizes 2^k and 2^k + 1

  • Issue #7890 - State Machine Race and Permanent Deadlock in run_loop::push_back

  • Issue #7881 - hpx::adjacent_difference doesn’t handle values at segmented partition boundaries

  • Issue #7870 - includes returns true with a parallel policy if the second range contains a run of equal elements that is not in the first range

  • Issue #7861 - rotate_sender test aborts with std::bad_array_new_length

  • Issue #7851 - Unsynchronized Move-Assignment in concurrent_unordered_map and concurrent_unordered_set

  • Issue #7814 - Segmented equal and mismatch mishandle unaligned ranges and partition placement

  • Issue #7811 - Scheduler mode setters apply PU masks to the wrong workers, and write past modes_ with several pools

  • Issue #7806 - Critical concurrency and safety defects in P2300 execution algorithms and async combinators

  • Issue #7788 - stop_callback destructor doesn’t wait for a callback running on another non-HPX thread

  • Issue #6647 - Incorrect reduce implementation

  • Issue #6430 - HPX distributed barrier runs into assertion error

Closed pull requests#

  • PR #8180 - Make the segmented dataflow lambdas that move their captures mutable

  • PR #8172 - Fix serializing into archives with a binary filter

  • PR #8171 - Support HPX::auto_wrap_main with clang targeting the MSVC ABI

  • PR #8169 - Leave out the tests MSVC 19.52 doesn’t compile in the VS2026 modules CI

  • PR #8160 - Let CI test jobs of modules builds use the MSVC toolset of the build job

  • PR #8158 - Fix unreadable task names in Tracy with full thread descriptions

  • PR #8153 - Stub out runtime_support::create_component only in device compilation

  • PR #8151 - execution: Fix lvalue connect and set_stopped completion signature in when_all_vector

  • PR #8149 - Send csv counter values to the right tracing plot

  • PR #8148 - Don’t pass the overloaded cudaMemcpyAsync to the CUDA executors by name

  • PR #8147 - Let CI test jobs use a build archive made with another MSVC toolset

  • PR #8145 - Format the MPI error messages

  • PR #8144 - Keep objects alive while the output pointer tracker records their address

  • PR #8143 - Fix lost terminate request in the thread_rescheduling test

  • PR #8140 - Don’t hang when a startup function throws

  • PR #8139 - Retry a failed CI build one compile at a time

  • PR #8131 - compute_local: fix block_allocator deallocation byte size and max_size overflow

  • PR #8130 - Accept a sentinel of a different type in ranges::contains and the two-range ranges::transform

  • PR #8128 - Fix segmented copy across partition boundaries

  • PR #8126 - Fix use after free of coroutine frames returning hpx::future

  • PR #8125 - Fix segmented unary transform across partition boundaries

  • PR #8123 - Fix parallel algorithms not compiling with guided_chunk_size

  • PR #8121 - Build the C++20 modules CI with clang 23

  • PR #8120 - build(deps): bump lukka/get-cmake from 4.4.5 to 4.5.0

  • PR #8118 - Don’t strand a thread created while the last worker leaves

  • PR #8116 - Fix distributed bootstrap hangs on registration failures

  • PR #8113 - Fix a run_loop future deadlock and an async_rw_mutex assertion

  • PR #8112 - Fix TCP parcelport shutdown deadlocks

  • PR #8111 - Check the local state again before evict_fenced() removes a self-joined peer

  • PR #8110 - executors: combine caller and local chunk completion

  • PR #8109 - Repeat the workrequesting_steal round until a task is stolen

  • PR #8108 - Don’t let a timer wake a thread after its suspension has ended

  • PR #8107 - Fix use after scope in the algorithm_as_sender test

  • PR #8101 - Fix the DCO check, bound the backfill queue wait, and fix legend labels and contracts examples [without the contracts examples]

  • PR #8100 - Split the segmented merge regression test into five smaller tests

  • PR #8097 - Fix size accounting and update_if of lru_cache, don’t evict for failing local_cache inserts

  • PR #8095 - Break ties between lru_entry access times

  • PR #8094 - Measure condition variable wake time in its waiter

  • PR #8093 - Keep swap_ranges test values distinct from sentinels

  • PR #8092 - Fix runtime hang when a startup or pre-startup functions throw [startup functions only] (fixes #8070)

  • PR #8087 - Fix sender support for loop algorithms [strided induction fix only]

  • PR #8079 - Stop the parcelports before the distributed runtime is cleared

  • PR #8078 - Destroy a pinned migratable component at its last unpin

  • PR #8077 - Check the sign-offs of pull requests without passing the commits as input

  • PR #8073 - Don’t abort on parcels received after the worker threads have stopped

  • PR #8072 - Keep the external build test binaries in their own directories

  • PR #8069 - Keep iostreams output after shutdown on all paths, restore hpx::endl and hpx::flush

  • PR #8068 - Keep hpx::cout output written after the runtime has stopped

  • PR #8067 - Don’t use the distributed runtime in parcel callbacks once it has stopped

  • PR #8066 - Shard the C++20 modules workflows for Windows and Linux

  • PR #8065 - Don’t drop set operation results for chunk sizes larger than one

  • PR #8063 - Return shared_future from get_return_object of shared_future coroutines

  • PR #8062 - Keep the child thread alive while checking it after running it directly

  • PR #8061 - Work around MSVC problems with C++20 modules and enable module builds for MSVC 19.44 and newer

  • PR #8059 - Register components built with C++20 modules under their own name

  • PR #8058 - Run the Linux hwloc fetch workflow in the build_env container

  • PR #8057 - Fix a livelock in the freelist test

  • PR #8056 - Exclude command_line_handling from the C++20 modules CI for now

  • PR #8055 - collectives: keep a request for a later generation instead of waiting for it

  • PR #8054 - Build and run only the examples in the macOS fetch workflows

  • PR #8053 - Build and test macOS Debug in shards

  • PR #8052 - Add a reusable workflow for the sharded build and test scheme

  • PR #8051 - Make the sharded CI build steps portable to macOS and Windows

  • PR #8050 - collectives: detach the global barrier after all shutdown functions

  • PR #8047 - Fix the C++20 modules CI and compile more tests with modules on Clang

  • PR #8045 - collectives: apply the unresolved communicator fix to all collectives

  • PR #8044 - Keep Tracy plot names alive for the whole run

  • PR #8042 - Fix hpx::util::format not rejecting a replacement field without a closing brace

  • PR #8040 - Fix parse_sed_expression throwing when the search part is not closed

  • PR #8031 - Bump github/codeql-action from 4.39.2 to 4.39.3

  • PR #8027 - Don’t let evict_fenced() remove a peer that came back with a new epoch

  • PR #8026 - Construct reinitializable_static in the host pass of nvcc

  • PR #8024 - Check the thread count again after suspending all workers

  • PR #8023 - execution: Type-erase non-exception errors in sender_future bridge

  • PR #8021 - Add missing runs to the performance history from every master build

  • PR #8016 - Read the thread queue counters in a race-free order

  • PR #8015 - Fix intermittent failure of supervision_await_terminal on remote localities

  • PR #8014 - Don’t invoke the callable again when a packaged_task is called twice

  • PR #8012 - Fix regex_from_pattern reading past the end on a trailing ‘[’

  • PR #8010 - collectives: fix barrier deadlock on an unresolved communicator

  • PR #8009 - Preserve options after a missing queuing value

  • PR #8008 - Fix PU lookup and steal mask race in local schedulers

  • PR #8007 - Fix warnings when building docs

  • PR #8006 - Suppress use-after-move warning in stop_token test

  • PR #8004 - Accept a sentinel of a different type in several ranges algorithms

  • PR #8002 - Bump github/codeql-action from 4.39.1 to 4.39.2

  • PR #8000 - Fix segmented binary transform across partition boundaries

  • PR #7998 - execution: Fix critical lifecycle and race conditions in when_all_vector

  • PR #7996 - Fix stop_source assignment leaking the old stop state’s source count

  • PR #7993 - Keep the legends of the performance history graphs within the README width

  • PR #7992 - Restrict permissions and credentials of the Check Formatting workflow

  • PR #7991 - Make HPX compliant with the REUSE specification

  • PR #7989 - tests: add regression tests for partial_sort early return

  • PR #7987 - Don’t deadlock in the SIGSEGV and termination handlers on POSIX systems

  • PR #7986 - Destroy the iterators of the async pack traversal before its frame completes

  • PR #7985 - Let performance tests choose their warmup, use 2 steps in the scheduler latency report

  • PR #7984 - Mark the releases on the performance history graphs

  • PR #7982 - Support unsized sentinels in make_heap

  • PR #7981 - Fix partial_sort’s sorted-prefix fast path

  • PR #7979 - Bump dawidd6/action-download-artifact from 27 to 28

  • PR #7976 - Don’t make processing_units_count of num_cores and task_size_chunk_size noexcept

  • PR #7975 - Bump lukka/get-cmake from 4.4.4 to 4.4.5

  • PR #7974 - Fix building and running HPX with CUDA on Windows

  • PR #7972 - Clear std::list and std::deque when loading an empty archive

  • PR #7970 - Fix reduce_by_key ignoring the comparator for the last key

  • PR #7965 - parcelset: report rescheduled put_parcel failures through the write handler

  • PR #7963 - Handle spawn failures in the index-queue bulk executors

  • PR #7961 - Make hpx::ranges::is_partitioned support distinct C++20 sentinels

  • PR #7959 - Don’t install the source paths and empty directories of fetched dependencies

  • PR #7958 - Include <ios> where std::streamsize is used

  • PR #7952 - Fix deadlock when non-HPX threads perform timed waits on HPX futures

  • PR #7951 - Make hpx::unwrap SFINAE-friendly for nested containers without futures

  • PR #7941 - Fix the input iterators returned by sequential ranges::transform and set_intersection

  • PR #7934 - Fix reducing algorithms with auto_chunk_size and parallel_executor

  • PR #7930 - Fix the heap fallback of partial_sort and nth_element

  • PR #7922 - Evict peers fenced by failure detection from the registry

  • PR #7914 - Return last as the input iterator from reverse_copy and rotate_copy

  • PR #7910 - Fix hpx::make_heap for power-of-two sized ranges

  • PR #7907 - Fix uncatchable std::terminate and dropped continuations in async_rw_mutex

  • PR #7902 - synchronization: return pred() when condition_variable_any wait observes stop

  • PR #7891 - Fix state machine race and deadlock in run_loop

  • PR #7882 - Fix segmented adjacent_difference across partition boundaries

  • PR #7880 - Fix any_of returning true for an empty range with parallel policies

  • PR #7877 - Fix startup header redefinition with C++ modules

  • PR #7871 - Fix includes returning true for a run of equal elements not in the first range

  • PR #7869 - algorithms: do not require *first convertible to T in parallel reduce

  • PR #7862 - algorithms: bind core count onto scheduler in rotate

  • PR #7852 - Fix unsynchronized move-assignment in concurrent unordered containers

  • PR #7844 - algorithms: do not skip leading elements of the second range in parallel set operations

  • PR #7815 - fix: handle segmented comparison alignment

  • PR #7812 - Fix scheduler mode setters mapping of PU masks across pools

  • PR #7808 - Fix unconditional std::terminate in run_loop destructor (#7806)

  • PR #7798 - Fix stop_callback destructor not waiting on another non-HPX thread

  • PR #7795 - Stop dereferencing a null pointer in lambda_to_action