Awesome-LLM-Training-System

Awesome LLM Training System

This repo displays a survey about optimizations for LLM training system. Relative papers about optimizing LLM training are categorized and listed below.

• Infrastructure
• Parallelism Schemes
• Computation Optimizations
• Memory Optimizations
• Communication Optimizations
• Fault Tolerance