One of my fantasy leagues was drafting today and I had gotten the times wrong so I had Astra draft for me completely autonomously. Some interesting observations -

I ran had it run on High the entire time.

The great!

Its agentic capabilities were excellent. It managed the   
Sleeper website very well (better than Sol), and effectively researched each player prior to a pick, considering draft position, team composition, and the team composition of the other players on the board.

It was very aware of the league specific rules, potentially to its detriment (more on this later), but didn’t lose context over the course of an hour plus draft even when compacting.

Its ability to “just get it done” was impressive as I didn’t even look at it for the majority of the draft to run this experiment.

The not so great!

It took Omarion Hampton as my second round pick (look I’m a chargers fan but what the hell?).

It indexed waaaay too high on our defensive scoring rules and almost chose a defense in an 8 man league with 5(!) picks left.

It missed two picks resulting in auto draft (fortunately it had been building a queue so it wasn’t a total waste) taking over because in one case it was compacting and in the other it got temporarily caught in some loop researching news (it got itself out without my intervention though).

The overall team is not that impressive IMO and it generally made “ok” decision as opposed to “great” decisions. This is obviously very subjective, but in an 8 man league I would have walked away personally with way more big firepower.

This was my second worst draft out of 4 so far in my opinion, with the worst being my fault and the result of being rushed and going second (worst slot imo).

Overall I am enjoying Astra and currently testing it with some coding and research tasks, and it’s a noticeable step up in terms of computer control (which imo codex already has a huge lead in even with Sol), still undetermined how much of a step change in “intelligence” it is though for me.

Important Astra Benchmarks - Drafting my Fantasy League