AI and Heterogeneous Systems
Artificial intelligence and machine learning are activities that are highly compute intensive. This means that the hardware that developers use to implement AI systems is very important for the success of the project. This is true of new projects as well and the retrofit of existing legacy applications. When thinking about the hardware requirements for AI projects, considerations about CPU use vs GPU use are always surfacing, as we see in our latest Artificial Intelligence and Machine Learning Survey. In this survey of over 500 professional developers actively working with AI and ML and conducted in May 2019, we explored the implications and usage of both CPUs and GPUs when working with AI systems.
Many systems require the use of at least one CPU and one GPU, and sometimes other forms of accelerators. While these architectures are two different types of integrated circuits, they aren’t necessarily part of a heterogeneous hardware approach. Heterogeneous hardware is the inclusion of multiple types of architectures on the same memory bus that can run instructions in parallel and share from the same pool of memory. The proximity and sharing of resources reduces the memory needed for scheduling, but also requires particular attention to how memory is shared.
One of the aims of a heterogeneous systems approach is to improve performance. Performance gains are not a given, as developers must use frameworks, tools and approaches that reduce the complexity of these systems and lessen differences between the various architectures that comprise them. For this reason, better performance can be both an objective and a challenge.
Our study found that twenty-one percent of AI and machine learning developers are using a heterogeneous hardware approach. Among these developers, 48% view performance issues to be a limitation, while 43% state that finding appropriate tools and SDKs is also a limitation. Among those AI or ML developers who are still evaluating and are in the process of piloting a heterogeneous hardware approach, it is securing a heterogeneous hardware system that is most commonly viewed as a limitation, and the same is true of developers who have tested a heterogeneous hardware approach and abandoned it.
One of the challenges in securing heterogeneous systems is the added complexity of including multiple architectures. Instead of considering the specific security characteristics and limitations of one architecture, developers must now consider best practices in securing multiple hardware platforms. Developers may opt to resolve this using various forms of encryption or may try to resolve this at some other level. Forty-eight percent of AI and machine learning developers deem securing a heterogeneous system to be one of the limitations of using a heterogeneous hardware approach.
Frameworks and platforms for handling heterogeneous hardware deal with handling different types of processors and architectures, as well as the device code that reaches each of them. Approaches that address heterogeneous hardware can interoperate; thus, when discussing models and frameworks, the question turns to what developers use predominantly, rather than whether they use that framework or model for some portion of their development. For instance, SYCL is a specific model based off of the OpenCL standard. SYCL use can overlap with OpenCL use, but developers are not restricted to using SYCL with OpenCL. Consequently, our AI and Machine Learning Survey asked developers which they use most often and OpenCL was by far the winner. Fifty-seven percent of the AI or ML developers working with heterogeneous hardware predominantly use OpenCL as compared to 28% predominantly using Nvidia’s CUDA, the next most used.
The importance of frameworks in this respect cannot be overemphasized today and is likely to become even more important as AI and ML start being incorporated into both new and existing applications everywhere.
See more about this important survey report here:
THE VOICE OF EVANS DATA 