Notebook deployment cicd

Continuous Integration / Continuous Delivery (CI/CD)

Building a POC deployment pipeline where engineers can work locally in vs code writing jupyter / marimo notebooks, merge feature branches to kick off a github actions deployment converting the notebooks to fabric notebooks, upload via the fabric apis to the workspace, and provision job schedulers using yaml tied to notebook ids.

Our data is rather small, so the goal was to use pure python notebooks, with deltalake, polars, and duckdb.

I first tried the native github integration syncing the workspace and using the fabric ci/cd package, but as far as I can tell there is no good experience for then working locally. Are folks making updates right to the `notebook-content.py` files, or is there an extension I'm missing?

Any suggestions on what is working for other teams would be appreciated. Our main workspace is developed entirely in fabric UI with spark, and it is great, but starting to get messy and is overkill for what we're doing. The team is growing and would like a more sustainable development pattern before looking at other tools.

I thought I remember reading on here recently that managing workspaces via the API and the fabric cli was a reasonable approach over the native workspace git integration.

Posted Jan 11, 2026, 8:07 PM UTC

129
Loading comments…