Open-source Large Language Models (LLMs) such as LLaMA, Falcon, and Mistral offer a range of choices for AI professionals and scholars. Yet, the majority of these LLMs have only made available select components like the end-model weights or inference scripts, with technical documents often narrowing their focus to broader design aspects and basic metrics. This approach restricts advances in the field by reducing clarity in the training methodologies of LLMs, leading to repeated efforts by teams to uncover numerous aspects of the training procedure. A team of researchers from Petuum, MBZUAI, USC, CMU, UIUC,  and UCSD introduced LLM360 to support