Merge branch 'develop' into task/rhornung67/docs-improve

Author: rhornung67

CMakeLists.txt

@@ -136,6 +136,7 @@ set (raja_sources
   src/AlignedRangeIndexSetBuilders.cpp
   src/DepGraphNode.cpp
   src/LockFreeIndexSetBuilders.cpp
+  src/Memory.cpp
   src/MemUtils_CUDA.cpp
   src/MemUtils_HIP.cpp
   src/PluginStrategy.cpp)

docs/sphinx/user_guide/feature/multi-reduction.rst

@@ -47,6 +47,11 @@ Please see the following cook book sections for guidance on policy usage:
 
  * :ref:`cook-book-multi-reductions-label`.
 
+Some backend multi-reduction implementations may use RAJA internal temporary
+storage. Please see the following section for guidance on releasing unused
+internal memory:
+
+ * :ref:`memory-release-internal-pool-label`.
 
 --------------------
 MultiReduction Types

docs/sphinx/user_guide/feature/reduction.rst

@@ -53,6 +53,11 @@ Please see the following cook book sections for guidance on policy usage:
 
  * :ref:`cook-book-reductions-label`.
 
+Some backend reduction implementations may use RAJA internal temporary storage.
+Please see the following section for guidance on releasing unused internal
+memory:
+
+ * :ref:`memory-release-internal-pool-label`.
 
 ----------------
 Reduction Types

docs/sphinx/user_guide/feature/scan.rst

@@ -43,6 +43,12 @@ RAJA scan operations:
 
  * :ref:`tut-scan-label`.
 
+Some backend scan implementations may use RAJA internal temporary storage.
+Please see the following section for guidance on releasing unused internal
+memory:
+
+ * :ref:`memory-release-internal-pool-label`.
+
 -----------------
 Scan Operations
 -----------------

docs/sphinx/user_guide/feature/sort.rst

@@ -47,6 +47,12 @@ RAJA scan operations:
 
  * :ref:`tut-sort-label`
 
+Some backend sort implementations may use RAJA internal temporary storage.
+Please see the following section for guidance on releasing unused internal
+memory:
+
+ * :ref:`memory-release-internal-pool-label`.
+
 -----------------
 Sort Operations
 -----------------

docs/sphinx/user_guide/index.rst

@@ -35,5 +35,6 @@ to use RAJA in an application can be found in :ref:`app-considerations-label`.
    features
    cook_book
    profiling_with_caliper
+   memory
    app_considerations
    tutorial

docs/sphinx/user_guide/memory.rst

@@ -0,0 +1,34 @@
+.. ##
+.. ## Copyright (c) Lawrence Livermore National Security, LLC and other
+.. ## RAJA Project Developers. See top-level LICENSE and COPYRIGHT
+.. ## files for dates and other details. No copyright assignment is required
+.. ## to contribute to RAJA.
+.. ##
+.. ## SPDX-License-Identifier: (BSD-3-Clause)
+.. ##
+
+.. _memory-label:
+
+******
+Memory
+******
+
+.. _memory-release-internal-pool-label:
+
+Releasing RAJA Internal Pool Memory
+===================================
+
+Some RAJA backend operations, including reductions, scans, and sorts, may use
+temporary storage. RAJA keeps some of this storage for reuse in internal memory
+pools. Applications can call ``RAJA::release_unused_internal_memory()`` after
+outstanding RAJA and device work has completed to return unused internal memory
+to the backend allocator. Typical use is to call the function at application shutdown
+to prevent the internal memory pools from appearing in memory-leak reports. Frequent
+runtime calls are generally unnecessary because the pools retained for reuse are
+typically not large.
+
+The function returns the number of bytes released and does not synchronize
+CUDA, HIP, or SYCL work. RAJA does not do this automatically during static
+destruction because CUDA and HIP runtime teardown can make late ``cudaFree`` or
+``hipFree`` calls fail. As such, this function must only be called before the
+end of main to avoid racing with backend runtime cleanup.

include/RAJA/RAJA.hpp

@@ -157,6 +157,11 @@
 //
 #include "RAJA/util/BitMask.hpp"
 
+//
+// Memory utility routines
+//
+#include "RAJA/util/memory.hpp"
+
 //
 // sort algorithms
 //

include/RAJA/policy/cuda/MemUtils_CUDA.hpp

@@ -185,6 +185,17 @@ using device_pinned_mempool_type =
     basic_mempool::MemPool<DevicePinnedAllocator>;
 using pinned_mempool_type = basic_mempool::MemPool<PinnedAllocator>;
 
+RAJA_INLINE
+size_t release_unused_internal_memory()
+{
+  size_t released = 0;
+  released += device_mempool_type::getInstance().release_unused();
+  released += device_zeroed_mempool_type::getInstance().release_unused();
+  released += device_pinned_mempool_type::getInstance().release_unused();
+  released += pinned_mempool_type::getInstance().release_unused();
+  return released;
+}
+
 namespace detail
 {
 

include/RAJA/policy/hip/MemUtils_HIP.hpp

@@ -162,6 +162,17 @@ using device_pinned_mempool_type =
     basic_mempool::MemPool<DevicePinnedAllocator>;
 using pinned_mempool_type = basic_mempool::MemPool<PinnedAllocator>;
 
+RAJA_INLINE
+size_t release_unused_internal_memory()
+{
+  size_t released = 0;
+  released += device_mempool_type::getInstance().release_unused();
+  released += device_zeroed_mempool_type::getInstance().release_unused();
+  released += device_pinned_mempool_type::getInstance().release_unused();
+  released += pinned_mempool_type::getInstance().release_unused();
+  return released;
+}
+
 namespace detail
 {