Your GraphRAG pipeline has an unmeasured stage: serialization. 10-format benchmark shows it swings multi-hop accuracy 40% to 80%

Discussion

We benchmark retrievers, rerankers, chunkers, embedding models. Then the retrieved subgraph goes into the prompt via json.dumps and nobody measures that step.

I compared 10 graph serialization formats (JSON, GraphML, RDF/Turtle variants, edge lists, adjacency lists, others) with the same graph and same model. Measured token count, traversal QA, and 2-3 hop reasoning. Results: 40% to 80% multi-hop accuracy spread on format alone, and about 70% token cost difference between the most verbose and most compact formats. Verbose syntax does not just cost tokens, it appears to actively hurt the model's attention over graph structure.

Open sourced the winning format with the benchmark methodology (MIT, Python/JS/Rust/Go/C++/C#): https://github.com/isongraph/isongraph

If you are doing GraphRAG in production, what format do you actually pass to the model? Genuinely curious whether anyone has profiled this stage.

Posted Jul 27, 2026, 6:58 AM UTC

04
Loading comments…