Accelerating LLMs With TornadoVM: From GPU Kernels To Model Inference Airhacks.fm Podcast With Adam Bien podcast

Java Prog Languages Es 6 Es 7 Css 3 Webstandards Adam Bien Airhacks Tech Webcomponents MicroProfile Jakarta EE Serverless Cloud IT JavaScript

Content provided by Adam Bien. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Adam Bien or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

airhacks.fm podcast with adam bien »
Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

18h ago 1:11:04

MP3•Episode home

An airhacks.fm conversation with Juan Fumero (@snatverk) about:

tornadovm as a Java parallel framework for accelerating data parallelization on GPUs and other hardware, first GPU experiences with ELSA Winner and Voodoo cards, explanation of TornadoVM as a plugin to existing JDKs that uses Graal as a library, TornadoVM's programming model with @parallel and @reduce annotations for parallelizable code, introduction of kernel API for lower-level GPU programming, TornadoVM's ability to dynamically reconfigure and select the best hardware for workloads, implementation of LLM inference acceleration with TornadoVM, challenges in accelerating Llama models on GPUs, introduction of tensor types in TornadoVM to support FP8 and FP16 operations, shared buffer capabilities for GPU memory management, comparison of Java Vector API performance versus GPU acceleration, discussion of model quantization as a potential use case for TornadoVM, exploration of Deep Java Library (DJL) and its ND array implementation, potential standardization of tensor types in Java, integration possibilities with Project Babylon and its Code Reflection capabilities, TornadoVM's execution plans and task graphs for defining accelerated workloads, ability to run on multiple GPUs with different backends simultaneously, potential enterprise applications for LLMs in Java including model distillation for domain-specific models, discussion of Foreign Function & Memory API integration in TornadoVM, performance comparison between different GPU backends like OpenCL and CUDA, collaboration with Intel Level Zero oneAPI and integrated graphics support, future plans for RISC-V support in TornadoVM

Juan Fumero on twitter: @snatverk

347 episodes

#Java #Prog Languages #Es 6 #Es 7 #Css 3 #Webstandards #Adam Bien #Airhacks #Tech #Webcomponents #MicroProfile #Jakarta EE #Serverless #Cloud IT #JavaScript

Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

airhacks.fm podcast with adam bien

227 subscribers

published 18h ago

MP3•Episode home

An airhacks.fm conversation with Juan Fumero (@snatverk) about:

tornadovm as a Java parallel framework for accelerating data parallelization on GPUs and other hardware, first GPU experiences with ELSA Winner and Voodoo cards, explanation of TornadoVM as a plugin to existing JDKs that uses Graal as a library, TornadoVM's programming model with @parallel and @reduce annotations for parallelizable code, introduction of kernel API for lower-level GPU programming, TornadoVM's ability to dynamically reconfigure and select the best hardware for workloads, implementation of LLM inference acceleration with TornadoVM, challenges in accelerating Llama models on GPUs, introduction of tensor types in TornadoVM to support FP8 and FP16 operations, shared buffer capabilities for GPU memory management, comparison of Java Vector API performance versus GPU acceleration, discussion of model quantization as a potential use case for TornadoVM, exploration of Deep Java Library (DJL) and its ND array implementation, potential standardization of tensor types in Java, integration possibilities with Project Babylon and its Code Reflection capabilities, TornadoVM's execution plans and task graphs for defining accelerated workloads, ability to run on multiple GPUs with different backends simultaneously, potential enterprise applications for LLMs in Java including model distillation for domain-specific models, discussion of Foreign Function & Memory API integration in TornadoVM, performance comparison between different GPU backends like OpenCL and CUDA, collaboration with Intel Level Zero oneAPI and integrated graphics support, future plans for RISC-V support in TornadoVM

Juan Fumero on twitter: @snatverk

347 episodes

#Java #Prog Languages #Es 6 #Es 7 #Css 3 #Webstandards #Adam Bien #Airhacks #Tech #Webcomponents #MicroProfile #Jakarta EE #Serverless #Cloud IT #JavaScript

All episodes

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

Listen to 500+ topics

Similar to airhacks.fm podcast with adam bien

Podcasts Worth a Listen

airhacks.fm podcast with adam bien » Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

Podcasts Worth a Listen

Welcome to Player FM!

Similar to airhacks.fm podcast with adam bien

airhacks.fm podcast with adam bien »
Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference