2.10. Managing Devices
Multiple vendors vie to provide accelerator-type processors. VTK‑m endeavors to support as many such architectures as possible. Each device and device technology requires some level of code specialization, and that specialization is encapsulated in a unit called a device adapter.
So far in Part 2 (Using VTK‑m) we have been writing code that runs on a local serial CPU. In those examples where we run a filter, VTK‑m is launching parallel execution in the execution environment. Internally VTK‑m uses a device adapter to manage this execution.
A build of VTK‑m generally supports multiple device adapters. In this chapter we describe how to represent and manage devices.
2.10.1. Device Adapter Tag
A device adapter is identified by a device adapter tag. This tag, which is simply an empty struct type, is used as the template parameter for several classes in the VTK‑m control environment and causes these classes to direct their work to a particular device. The following device adapter tags are available in VTK‑m.
-
struct DeviceAdapterTagSerial : public vtkm::cont::DeviceAdapterId
Tag for a device adapter that performs all computation on the same single thread as the control environment.
This device is useful for debugging. This device is always available. This tag is defined in
vtkm/cont/DeviceAdapterSerial.h
.
-
struct DeviceAdapterTagCuda : public vtkm::cont::DeviceAdapterId
Tag for a device adapter that uses a CUDA capable GPU device.
For this device to work, VTK-m must be configured to use CUDA and the code must be compiled by the CUDA
nvcc
compiler. This tag is defined invtkm/cont/cuda/DeviceAdapterCuda.h
.
-
struct DeviceAdapterTagOpenMP : public vtkm::cont::DeviceAdapterId
Tag for a device adapter that uses OpenMP compiler extensions to run algorithms on multiple threads.
For this device to work, VTK-m must be configured to use OpenMP and the code must be compiled with a compiler that supports OpenMP pragmas. This tag is defined in
vtkm/cont/openmp/DeviceAdapterOpenMP.h
.
-
struct DeviceAdapterTagTBB : public vtkm::cont::DeviceAdapterId
Tag for a device adapter that uses the Intel Threading Building Blocks library to run algorithms on multiple threads.
For this device to work, VTK-m must be configured to use TBB and the executable must be linked to the TBB library. This tag is defined in
vtkm/cont/tbb/DeviceAdapterTBB.h
.
-
struct DeviceAdapterTagKokkos : public vtkm::cont::DeviceAdapterId
Tag for a device adapter that uses the Kokkos library to run algorithms in parallel.
For this device to work, VTK-m must be configured to use Kokkos and the executable must be linked to the Kokkos libraries. VTK-m will use the default execution space of the provided kokkos library build. This tag is defined in
vtkm/cont/kokkos/DeviceAdapterKokkos.h
.
The following example uses the tag for the Kokkos device adapter to specify a specific device for VTK‑m to use. (Details on specifying devices in VTK‑m is provided in Section 2.10.4 (Specifying Devices).)
1 vtkm::cont::ScopedRuntimeDeviceTracker(vtkm::cont::DeviceAdapterTagKokkos{});
For classes and methods that have a template argument that is expected to be a device adapter tag, the tag type can be checked with the VTKM_IS_DEVICE_ADAPTER_TAG
macro to verify the type is a valid device adapter tag.
It is good practice to check unknown types with this macro to prevent further unexpected errors.
2.10.2. Device Adapter Id
Using a device adapter tag directly means that the type of device needs to be known at compile time.
To store a device adapter type at run time, one can instead use vtkm::cont::DeviceAdapterId
.
vtkm::cont::DeviceAdapterId
is a superclass to all the device adapter tags, and any device adapter tag can be “stored” in a vtkm::cont::DeviceAdapterId
.
Thus, it is more common for functions and classes to use vtkm::cont::DeviceAdapterId
then to try to track a specific device with templated code.
-
struct DeviceAdapterId
An object used to specify a device.
vtkm::cont::DeviceAdapterId
can be used to specify a device to use when executing some code. EachDeviceAdapterTag
object inherits fromvtkm::cont::DeviceAdapterId
. Functions can accept avtkm::cont::DeviceAdapterId
object rather than a templated tag to select a device adapter at runtime.Subclassed by vtkm::cont::DeviceAdapterTagAny, vtkm::cont::DeviceAdapterTagCuda, vtkm::cont::DeviceAdapterTagKokkos, vtkm::cont::DeviceAdapterTagOpenMP, vtkm::cont::DeviceAdapterTagSerial, vtkm::cont::DeviceAdapterTagTBB, vtkm::cont::DeviceAdapterTagUndefined
Public Functions
-
inline constexpr bool IsValueValid() const
Return whether this object represents a valid type of device.
This method will return true if the id represents a specific, valid device. It will return true even if the device is disabled in by the runtime tracker or if the device is not supported by the VTK-m build configuration.
It should be noted that this method return false for tags that are not specific devices. This includes
vtkm::cont::DeviceAdapterTagAny
andvtkm::cont::DeviceAdapterTagUndefined
.
-
DeviceAdapterNameType GetName() const
Return a name representing the device.
The string returned from this method is stored in a type named
vtkm::cont::DeviceAdapterNameType
, which is currently aliased tostd::string
. The device adapter name is useful for printing information about a device being used.
-
inline constexpr bool IsValueValid() const
Did You Know?
As a cheat, all device adapter tags actually inherit from the vtkm::cont::DeviceAdapterId
class.
Thus, all of these methods can be called directly on a device adapter tag.
Common Errors
Just because the vtkm::cont::DeviceAdapterId::IsValueValid()
returns true that does not necessarily mean that this device is available to be run on.
It simply means that the device is implemented in VTK‑m.
However, that device might not be compiled, or that device might not be available on the current running system, or that device might not be enabled.
Use the device runtime tracker described in Section 2.10.3 (Runtime Device Tracker) to determine if a particular device can actually be used.
In addition to the provided device adapter tags listed previously, a vtkm::cont::DeviceAdapterId
can store some special device adapter tags that do not directly specify a specific device.
-
struct DeviceAdapterTagAny : public vtkm::cont::DeviceAdapterId
Tag for a device adapter used to specify that any device may be used for an operation.
In practice this is limited to devices that are currently available.
-
struct DeviceAdapterTagUndefined : public vtkm::cont::DeviceAdapterId
Tag for a device adapter used to avoid specifying a device.
Useful as a placeholder when a device can be specified but none is given.
Did You Know?
Any device adapter tag can be used where a device adapter id is expected. Thus, you can use a device adapter tag whenever you want to specify a particular device and pass that to any method expecting a device id. Likewise, it is usually more convenient for classes and methods to manage device adapter ids rather than device adapter tag.
2.10.3. Runtime Device Tracker
It is often the case that you are agnostic about what device VTK‑m algorithms run so long as they complete correctly and as fast as possible. Thus, rather than directly specify a device adapter, you would like VTK‑m to try using the best available device, and if that does not work try a different device. Because of this, there are many features in VTK‑m that behave this way. For example, you may have noticed that running filters, as in the examples of Chapter 2.6 (Running Filters), you do not need to specify a device; they choose a device for you.
However, even though we often would like VTK‑m to choose a device for us, we still need a way to manage device preferences.
VTK‑m also needs a mechanism to record runtime information about what devices are available so that it does not have to continually try (and fail) to use devices that are not available at runtime.
These needs are met with the vtkm::cont::RuntimeDeviceTracker
class.
vtkm::cont::RuntimeDeviceTracker
maintains information about which devices can and should be run on.
VTK‑m maintains a vtkm::cont::RuntimeDeviceTracker
for each thread your code is operating on.
To get the runtime device for the current thread, use the vtkm::cont::GetRuntimeDeviceTracker()
method.
-
vtkm::cont::RuntimeDeviceTracker &vtkm::cont::GetRuntimeDeviceTracker()
Get the
RuntimeDeviceTracker
for the current thread.Many features in VTK-m will attempt to run algorithms on the “best
available device.” This often is determined at runtime as failures in one device are recorded and that device is disabled. To prevent having to check over and over again, VTK-m uses per thread runtime device tracker so that these choices are marked and shared.
-
class RuntimeDeviceTracker
RuntimeDeviceTracker is the central location for determining which device adapter will be active for algorithm execution.
Many features in VTK-m will attempt to run algorithms on the “best
available device.” This generally is determined at runtime as some backends require specific hardware, or failures in one device are recorded and that device is disabled.
While vtkm::cont::RunimeDeviceInformation reports on the existence of a device being supported, this tracks on a per-thread basis when worklets fail, why the fail, and will update the list of valid runtime devices based on that information.
Subclassed by vtkm::cont::ScopedRuntimeDeviceTracker
Public Functions
-
bool CanRunOn(DeviceAdapterId deviceId) const
Returns true if the given device adapter is supported on the current machine.
-
inline void ReportAllocationFailure(vtkm::cont::DeviceAdapterId deviceId, const vtkm::cont::ErrorBadAllocation&)
Report a failure to allocate memory on a device, this will flag the device as being unusable for all future invocations.
-
inline void ReportBadDeviceFailure(vtkm::cont::DeviceAdapterId deviceId, const vtkm::cont::ErrorBadDevice&)
Report a ErrorBadDevice failure and flag the device as unusable.
-
void ResetDevice(vtkm::cont::DeviceAdapterId deviceId)
Reset the tracker for the given device.
This will discard any updates caused by reported failures. Passing DeviceAdapterTagAny to this will reset all devices (same as
Reset()
).
-
void Reset()
Reset the tracker to its default state for default devices.
Will discard any updates caused by reported failures.
-
void DisableDevice(DeviceAdapterId deviceId)
Disable the given device.
The main intention of
RuntimeDeviceTracker
is to keep track of what devices are working for VTK-m. However, it can also be used to turn devices on and off. Use this method to disable (turn off) a given device. UseResetDevice()
to turn the device back on (if it is supported).Passing DeviceAdapterTagAny to this will disable all devices.
-
void ForceDevice(DeviceAdapterId deviceId)
Disable all devices except the specified one.
The main intention of
RuntimeDeviceTracker
is to keep track of what devices are working for VTK-m. However, it can also be used to turn devices on and off. Use this method to disable all devices except one to effectively force VTK-m to use that device. Either pass the DeviceAdapterTagAny to this function or callReset()
to restore all devices to their default state.This method will throw a
vtkm::cont::ErrorBadValue
if the given device does not exist on the system.
-
bool GetThreadFriendlyMemAlloc() const
Get/Set use of thread-friendly memory allocation for a device.
-
void SetThreadFriendlyMemAlloc(bool state)
Get/Set use of thread-friendly memory allocation for a device.
-
void CopyStateFrom(const vtkm::cont::RuntimeDeviceTracker &tracker)
Copies the state from the given device.
This is a convenient way to allow the
RuntimeDeviceTracker
on one thread copy the behavior from another thread.
-
void SetAbortChecker(const std::function<bool()> &func)
Set/Clear the abort checker functor.
If set the abort checker functor is called by
vtkm::cont::TryExecute()
before scheduling a task on a device from the associated the thread. If the functor returnstrue
, an exception is thrown.
-
void ClearAbortChecker()
Set/Clear the abort checker functor.
If set the abort checker functor is called by
vtkm::cont::TryExecute()
before scheduling a task on a device from the associated the thread. If the functor returnstrue
, an exception is thrown.
-
void PrintSummary(std::ostream &out) const
Produce a human-readable report on the state of the runtime device tracker.
-
bool CanRunOn(DeviceAdapterId deviceId) const
2.10.4. Specifying Devices
A vtkm::cont::RuntimeDeviceTracker
can be used to specify which devices to consider for a particular operation.
However, a better way to specify devices is to use the vtkm::cont::ScopedRuntimeDeviceTracker
class.
When a vtkm::cont::ScopedRuntimeDeviceTracker
is constructed, it specifies a new set of devices for VTK‑m to use.
When the vtkm::cont::ScopedRuntimeDeviceTracker
is destroyed as it leaves scope, it restores VTK‑m’s devices to those that existed when it was created.
-
class ScopedRuntimeDeviceTracker : public vtkm::cont::RuntimeDeviceTracker
A class to create a scoped runtime device tracker object.
This object captures the state of the per-thread device tracker and will revert any changes applied during its lifetime on destruction.
Unnamed Group
-
ScopedRuntimeDeviceTracker(const vtkm::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())
Construct a ScopedRuntimeDeviceTracker associated with the thread, associated with the provided tracker (defaults to current thread’s tracker).
Any modifications to the ScopedRuntimeDeviceTracker will effect what ever thread the
tracker
is associated with, which might not be the thread on which the ScopedRuntimeDeviceTracker was constructed.Constructors are not thread safe
-
ScopedRuntimeDeviceTracker(vtkm::cont::DeviceAdapterId device, RuntimeDeviceTrackerMode mode = RuntimeDeviceTrackerMode::Force, const vtkm::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())
Use this constructor to modify the state of the device adapters associated with the provided tracker.
Use
mode
withdevice
as follows:‘Force’ (default)
Force-Enable the provided single device adapter
Force-Enable all device adapters when using vtkm::cont::DeviceAdaterTagAny ‘Enable’
Enable the provided single device adapter if it was previously disabled
Enable all device adapters that are currently disabled when using vtkm::cont::DeviceAdaterTagAny ‘Disable’
Disable the provided single device adapter
Disable all device adapters when using vtkm::cont::DeviceAdaterTagAny
-
ScopedRuntimeDeviceTracker(const std::function<bool()> &abortChecker, const vtkm::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())
Use this constructor to set the abort checker functor for the provided tracker.
-
~ScopedRuntimeDeviceTracker()
Destructor is not thread safe.
-
ScopedRuntimeDeviceTracker(const vtkm::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())
The following example demonstrates how the vtkm::cont::ScopedRuntimeDeviceTracker
is used to force the VTK‑m operations that happen within a function to operate exclusively with the Kokkos device.
1void ChangeDefaultRuntime()
2{
3 std::cout << "Checking changing default runtime." << std::endl;
4
5 vtkm::cont::ScopedRuntimeDeviceTracker(vtkm::cont::DeviceAdapterTagKokkos{});
6
7 // VTK-m operations limited to Kokkos devices here...
8
9 // Devices restored as we leave scope.
10}
In the previous example we forced VTK‑m to use the Kokkos device.
This is the default behavior of vtkm::cont::ScopedRuntimeDeviceTracker
, but the constructor takes an optional second argument that is a value in the vtkm::cont::RuntimeDeviceTrackerMode
to specify how modify the current device adapter list.
-
enum class vtkm::cont::RuntimeDeviceTrackerMode
Identifier used to specify whether to enable or disable a particular device.
Values:
-
enumerator Force
Replaces the current list of devices to try with the device specified.
This has the effect of forcing VTK-m to use the provided device. This is the default behavior for
vtkm::cont::ScopedRuntimeDeviceTracker
.
-
enumerator Enable
Adds the provided device adapter to the list of devices to try.
-
enumerator Disable
Removes the provided device adapter from the list of devices to try.
-
enumerator Force
As a motivating example, let us say that we want to perform a deep copy of an array (described in Section 3.2.2 (Deep Array Copies)).
However, we do not want to do the copy on a Kokkos device because we happen to know the data is not on that device and we do not want to spend the time to transfer the data to that device.
We can use a vtkm::cont::ScopedRuntimeDeviceTracker
to temporarily disable the Kokkos device for this operation.
1 vtkm::cont::ScopedRuntimeDeviceTracker tracker(
2 vtkm::cont::DeviceAdapterTagKokkos(), vtkm::cont::RuntimeDeviceTrackerMode::Disable);
3
4 vtkm::cont::ArrayCopy(srcArray, destArray);