Google’s HOPE project, scaled from its TITAN project for making an llm that can learn and retrain itself, seems to be really close to solving self-learning models. It just hasn’t been tested at scale yet. There are open source near enough recreations on pytorch and github right now.
In the past few weeks, we’ve had revelations about how scaling the harness itself gives massive leaps in performance, especially if the LLM is designing its own harnesses too. It can even increase long-term planning.
My idea that I’d love someone here to try, as I lack the compute right now, is to combine these. Download the best HOPE replication, train it up on coding and machine learning and more. The g