r/NixOS•
pherlo

Using Nix to replace language package managers for library versioning and dependencies

So I am investigating using Nix/NixOS to build/deploy a ruby application. (By the way I have no idea what the canonical way to do that is right now) but somewhere along the way a thought occurred to me, and wanted to run it by other people. I'll start with ruby but there is more below.

Why not build a parallel universe for ruby packages that actually abandons ruby gems, and just packages raw ruby code in some semi-standardized source tree? Then use nix to manage the dependencies between them, and have the nix expression place all dependencies into the $LOAD_PATH of the depending package automatically? One simply specifies dependencies for the app, and all of the relevant code appears on the ruby interpreter's $LOAD_PATH for your application automatically (presumably injected into the shell by nix).

Moreover, only the specified dependencies appear in $LOAD_PATH for any given package. One could even depend on different versions of the same library in the same interpreter session (assuming, again, no namespace pollution).

This idea would mean redoing gems, especially those that try to do their own version management (i.e., nokogiri), but I'd wager a good portion would be convertible with semi-automated systems. Besides that, the clean-break from rubygems means we get to do it right for a subset of packages.

Putting aside ruby, imagine what could be done for C and C++. Both have no real library manager right now. Same scheme: for new code that wants to build with nix as the lib manager, just specify library dependencies in the nix expression. In your own code, #include <libpng/png.h> and start coding/compiling in a nix-shell. The nix expression manages injecting the loaded dependencies into the include path and lib path for you. This is sort-of-like-but-better-than OS X frameworks in that if you have that bundle included in your dependency list you can start working with it with no further configuration. It's better because you get actual exact version pinning. In fact thinking of this idea as "invent a .framework clone and use Nix to manage dependencies" is not too far off base as a description.

Again this would be a clean break from existing tools, but arguably C/C++ need that clean break even more than ruby.

Because the development dependencies are so strict, we wouldn't even need configure scripts or automake anymore. Makefiles would only specify inter-file dependencies under this scheme, and not the miriad of platform-specific compiler option gunk. The nix expression would present build options and detect platform differences (like x86_64, arm or stdlib versions). The nix expression would furthermore eliminate the need for detecting compiler versions. If you want to use the latest bleeding-edge g++ for c++17 features go right ahead, the platform compiler will not be affected, and no one on your team will have to figure out how to juggle installing all of those compiler versions without breaking their other projects. Just means a longer initial build. Oh and getting rid of those ancient bespoke build tools means simpler dependency graphs for nix, leading to faster builds and more time on Hydra.

Under this model, getting started developing a source base would be vastly simplified. Just grab your project's nix expression, nix-shell, and run make or gcc and enjoy your successful build. You now have development binaries ready for testing and iterating, regardless of the prior state of your machine.

Posted Jan 5, 2015, 4:27 AM UTC

95
lethalman•

I know nothing about ruby, even less ruby on nix.

But the answer is that you can. If you have enough man power :) Gem is used because it's there already, knows the dependencies etc. and knows how to build a ruby project?

Also I think that such questions are more suited for the nix-dev mailing list, where you can find people that work on ruby packaging.

3
cessationoftime•

I've been thinking along the same lines. The programming industry should do away with most of the language specific build and package managing tools (maven,sbt,cabal,etc.) and use Nix as a universal package manager and compiler. This would make it easy to interoperate with code from multiple languages in one project. I can do this currently but often I find I have a configuration in my .nix files that duplicates much of the info in my language-specific config files, and then when I upgrade my dependencies I have to update additional config files.

2
GeniusBad•

Does Nix even support windows ? If it's going to replace $language package management it has to support all the language's target platforms.

1
cessationoftime•

I dont think they are that far away from Windows support, but I dont think it is really a goal either. I saw a blog post that someone had it running on windows with an older version. And when I was trying to get this to work myself I found a commit in the Nix project that removed Cygwin support, and this issue.

1
MonadTran•

The first step in fulfilling this ambitious goal would be to improve over this: http://blog.lastlog.de/posts/running_the_nix_package_manager_on_windows/

Next step, development environment support - like, make Visual Studio load nix packages, instead of the .csproj files, and install nix packages into a local directory, instead of downloading NuGet packages. I am sure there is something similar in the Java / Ruby world. Make emacs install nix packages, rather than ELPA / MELPA packages, etc.

Add Nix integration to commonly used CI systems.

All of this does seem possible, but would require quite a bit of work - even getting through the first obstacle.

2

build/deploy a ruby application. (By the way I have no idea what the canonical way to do that is right now)

Bundler & rvm. Though, discourse guys decided to give no fucks, and say that docker is the way

1
stepcut251•

Ooo. Idris currently lacks any real package management system, though discussions have just started about creating something. I suggested that something built around Nix could be the smart way to go -- and then found this post.

So, I am also quite curious about how well it could work. On the surface it seems full of awesome.

1
musicmatze•

I don't know what the current approach is, but yours sounds like a huge amount of effort and I don't think the nix community can handle this!

A better approach IMHO would be to reuse the language level package managers to do this sort of dependency resolution!

1
GeniusBad•

For the ruby supporting multiple versions in a single runtime, see https://github.com/zimbatm/namespace.rb (CommonJS experiment for ruby). It kinda works but it's mostly awkward. I suppose with a lot of work some of the issues could be ironed out by changing the language. It also doesn't work very well with old code.

In the end I think it's better to have 1 version of a library per runtime, you'll be avoiding subtle issues where instance of class Foo::Bar works with v1.0 of the library Foo but then break when passed to version v2.0.

One thing that I'm missing in nix is the ability to do constrain solving based on the dependency version specifications like bundler does. Managing dependencies conflict is a bitch and having bundle update help you is quite nice.

1