Transformer Training: I need your help!
Hello all, I hope you all are doing well, I'm interested in training my own transformer model from scratch for which i have completed andrej karpathy's playlist NN.Zero2Hero , and i also implemented "attention is all you need" paper from scratch in pytorch. However, I wasn't able to train it from scratch due to my unfamiliarity with the training aspects of transformers like quantization and very recently i have found a course on YT by MIT named as Efficientml.ai but I'm not sure whether i should start this course or not, will this course help me in training my own transformer model from scratch?
Is there anyone amongst you who has walked this path which I'm just about to start... Can anyone guide me how can i train transformers from scratch?
Posted Aug 20, 2024, 3:22 PM UTC