• If you are not familiar with distributed system or have never used Colossal-AI, you should first jump into the Concepts section to get a sense of what we are trying to achieve. This section can provide you with some background knowledge on distributed training as well.
• Next, you can follow the basics tutorials. This section will cover the details about how to use Colossal-AI.
• Afterwards, you can try out the features provided in Colossal-AI by reading features section. We will provide a codebase for each tutorial. These tutorials will cover the basic usage of Colossal-AI to realize simple functions such as data parallel and mixed precision training.
• Lastly, if you wish to apply more complicated techniques such as how to run hybrid parallel on GPT-3, the advanced tutorials section is the place to go!