finetuned 0.8b model, used llama cpp feels like magic on cpu
I wanted to see if a really small model could extract invoice data into user specified schemas. So I did a small finetune of Qwen3.5 0.8B using Unsloth, and it actually turned out much better than I expected. It runs fine on CPU.
For this:
Extract the data in JSON format using the schema: { "date": "string", "invoice_id": "string","bill_to":"string" // name and address,"ship_to":"string","all_items":[//list of items {"description":"string","quantity":"number","unit_price":"number","line_total":"number"}],"total":"number"}
Response:
{'date': 'August 20, 2006', 'invoice_id': 'INV1048', 'bill_to': 'C1003, Test Customer Two, 88 WILLIAM Square, Sydney 12345, Australia', 'ship_to': '', 'all_items': [{'description': 'Very long product description that occupies more than 1 line - in fact, it occupies 2 lines', 'quantity': 1, 'unit_price': 199.99, 'line_total': 199.99}, {'description': 'One line product description', 'quantity': 2, 'unit_price': 420.0, 'line_total': 840.0}], 'total': 1140.87}
How can i improve it more? How are you using small models? What have been the issues? llama.cpp seems the way to go..any other alternative or no?
You may try the model at
https://huggingface.co/MayankLad31/invoice_schema
Posted May 12, 2026, 8:12 AM UTC