can you make a AI ADAM-like optimizer?

Discussion

SGD or ADAM is really old at this point, and I don't know about how Transformer optimizers work yet but I heard they use ADAMW, still an ADAM algorithm.

Like, can we somehow create a AI based model (RNN,LSTM, or even a Transformer) that can do the optimizing much more efficiently and replacing ADAM? Is it something that are people working on?

Posted Apr 8, 2025, 9:06 PM UTC

03
Loading comments…