can you make a AI ADAM-like optimizer?
SGD or ADAM is really old at this point, and I don't know about how Transformer optimizers work yet but I heard they use ADAMW, still an ADAM algorithm.
Like, can we somehow create a AI based model (RNN,LSTM, or even a Transformer) that can do the optimizing much more efficiently and replacing ADAM? Is it something that are people working on?
Posted Apr 8, 2025, 9:06 PM UTC
03