Runtime memory policy for multireducers

[tomstitt]

Is your feature request related to a problem? Please describe.

The RAJA::hip_multi_reduce_atomic and RAJA::cuda_multi_reduce_atomic multireducers allocate GPU memory even when they are only used in a CPU kernel. Our application supports a runtime compute policy with GPU builds such that we can run CPU-only if desired; with multireducers this breaks because in CPU mode we now allocate GPU memory.

Describe the solution you'd like

We would like the GPU multireducers to dynamically choose their allocator based on the the kernel they are captured by, similar to the regular GPU RAJA reducers like RAJA::hip_reduce and RAJA::cuda_reduce.

Describe alternatives you've considered

We've considered templating routines where we use multireducers with a sequential dispatch for the CPU and a platform-dependent dispatch for the GPU but that requires additional boilerplate.

Additional context

n/a

[artv3]

@MrBurmark , would you know the status here?

[MrBurmark]

Ya, this is something on my plate, but that I have not started. It will probably involve changing how loop abstrations are implemented for policies like sequential and omp in order to allow for the hip/cuda multi-reducers to allocate at the loop instead of upfront. So I have put this off multiple times due to the invasiveness of the solution. This is not a problem with the new reductions interface so it may be less intrusive to finalize that abstraction instead and have people use that.