Feature request!

I’ve been using Unsloth Studio for a bit now and overall it’s been super. On my setup I’m seeing roughly 2x faster prompt processing and around 20% better token generation, so it’s best for my cpu/gpu setup!

There are a few things I kept running into that would make it more usable for my workflow:

1)Server logs. I’m using it as an API endpoint for agents, so having a simple log view with incoming tokens, cache hit or miss, and timing info would help a lot for debugging and visibility.

2)Persistent settings per model. Right now I couldn’t find a way to make it remember things like context window, temperature, top_p, etc. It would be useful if those stuck between loads.

3)On demand model loading. I run multiple models on one machine, so it would be nice if an API request to a specific model could automatically load it if it’s not already in memory, instead of managing that manually.

4)Cache controls. Some basic configuration like how many prompts to cache, memory limits, and how long entries are kept would make the caching system more practical.

Just my thoughts on what might be useful!