Artificial Intelligence: Machine learning inference acceleration

Artificial Intelligence (AI) is the most innovative emerging technology of today, which makes a massive impact on everyday life. It has been in the development phase for a long time. However, with the explosion of data availability, processing power, and the exponentially growing number of applications where it can be applied, AI is finally becoming reality.

There are many different Artificial Neural Network (ANN) models already available, with more being developed on a regular basis. Some network models are better for image recognition and object detection, while others are good for data analysis and prediction. Before they are used, models need to be trained first; activation weights and biases of each neural node need to be adjusted for a particular inference task.

Model training is compute-intensive process. Vast amounts of data are processed through the dense network model until the biases and weights are adjusted properly. However, model training is not a time-critical process and it can take some time. Once the training is completed, the trained model can be deployed for inference in a real-world application. Very often, platforms running the inference models must be capable of real-time performance. Low-latency and high-throughput inference is the ultimate challenge for many different applications.

This is where Xilinx Alveo accelerator cards and the Vitis™ AI development environment make all the difference: by implementing model optimizations such as pruning (node count reduction), quantization (bit reduction), and by customizing the execution path on a hardware level, developers can reduce the inference latency by an order of magnitude, even when large data sets are used. By using the AI profiler, developers can easily pinpoint execution bottlenecks and implement required optimizations, matching the throughput across the entire application.

Finally, Vitis™ AI development platform integrates industry-leading deep learning frameworks such as Tensor Flow and Caffe, bringing the power of adaptable hardware acceleration closer to developers, thus making the Xilinx Alveo accelerator card series the perfect choice for high-performance, energy-efficient, real-time AI inference.

You can find more information about Vitis™ Unified Software Platform here.