GNN for vessel segmentation: centerline vs feature-map pipelines, where do I start?

Hello everyone,

I have many questions regarding a pipeline, and I really don’t know where to start. I’ve never coded a pipeline end-to-end; I’ve just followed tutorials and read a bunch of articles. So maybe the way I’m seeing things is too naive—or too “beginner in English.”

My problem is to do vessel segmentation using GNNs. The idea is that we have tortuous vessels, small vessels, cycles, etc., so the GNN is theoretically there to keep the topology and geometry correct: no wrongly connected vessels just because they are near each other in 3D, preserving cycles and thin vessels even in the case of class imbalance.

After reading some articles, I’ve noticed two main approaches:

1. Segmentation → Centerline → Graph → GNN:
   • First, a U-Net or other segmentation model produces a vessel map.
   • Then, a centerline is extracted.
   • The graph is constructed from the centerline.
   • Finally, a GNN performs node classification and refinement.
2. Feature-map graph → GNN:
   • A feature map is extracted from the image.
   • This feature map becomes the graph.
   • The GNN is applied directly to this graph.

Many issues arise in both pipelines. Especially in the first approach: if the segmentation is incorrect, the centerline will also be incorrect. Then the graph construction is flawed, and the GNN may not be able to do anything meaningful with it. If I want to do it end-to-end, backpropagating all the way to the U-Net, I would need a differentiable skeletonization method, which is not straightforward.

In the second approach, if I construct the graph from features, there is the risk that the GNN might not work well because the nodes and edges are poorly chosen or constructed.

Overall, this kind of pipeline feels fragile: there are so many steps, many things could go wrong, and we would have to optimize multiple steps at once. If results are bad, it’s hard to know which part of the pipeline is responsible.

I have many other questions, but I’ll stop here for now. I want to know if my reasoning and the way I’m seeing the problem is too naive. Maybe I’m lacking information, or maybe this is not how people organize their work when trying to solve a problem with this kind of architecture. I am open to any advice you could give me, even if it doesn’t directly answer my questions.