GitHub - kungfu-team/Megatron-LM: Ongoing research training transformer models at scale

fork of Megatron-LM that integrates tenplex.

Name		Name	Last commit message	Last commit date
Latest commit History 1,970 Commits
docs		docs
examples		examples
images		images
megatron		megatron
pytrace		pytrace
tasks		tasks
tests		tests
tools		tools
vocab		vocab
.coveragerc		.coveragerc
.gitignore		.gitignore
.gitlab-ci.yml		.gitlab-ci.yml
Dockerfile		Dockerfile
Dockerfile-update		Dockerfile-update
LICENSE		LICENSE
README.md		README.md
build_docker.sh		build_docker.sh
build_docker_update.sh		build_docker_update.sh
compile_kernels.py		compile_kernels.py
create_bert_dataset.py		create_bert_dataset.py
create_gpt_dataset.py		create_gpt_dataset.py
hash_samples.py		hash_samples.py
hash_samples.sh		hash_samples.sh
preprocess_bert_pile.sh		preprocess_bert_pile.sh
preprocess_data.sh		preprocess_data.sh
preprocess_data_gpt.sh		preprocess_data_gpt.sh
preprocess_gpt_pile.sh		preprocess_gpt_pile.sh
pretrain_bert.py		pretrain_bert.py
pretrain_gpt.py		pretrain_gpt.py
pretrain_ict.py		pretrain_ict.py
pretrain_retro.py		pretrain_retro.py
pretrain_t5.py		pretrain_t5.py
pretrain_vision_classify.py		pretrain_vision_classify.py
pretrain_vision_dino.py		pretrain_vision_dino.py
pretrain_vision_inpaint.py		pretrain_vision_inpaint.py
run_compile_kernels.sh		run_compile_kernels.sh
run_create_bert_dataset.sh		run_create_bert_dataset.sh
setup.py		setup.py

Provide feedback