Artwork

Compiler collectives

PyTorch Developer Podcast

26 subscribers

published

iconShare
 
Manage episode 432352458 series 2921809
Content provided by PyTorch, Edward Yang, and Team PyTorch. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by PyTorch, Edward Yang, and Team PyTorch or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

Compiler collectives are a PT2 feature where by compiler instances across multiple ranks use NCCL collectives to communicate information to other instances. This is used to ensure we consistently decide if inputs or static or dynamic across all ranks. See also PR at https://github.com/pytorch/pytorch/pull/130935

  continue reading

83 episodes