The future of software is Nix

42 points by biggestlou 8 months ago | 61 comments
  • andrewla 8 months ago
    The future of software is Nix in the same sense that the future of software was Bitkeeper.

    It was readily apparent to anyone using Bitkeeper that this was the right way for version control software to work. Similarly, the Nix model is clearly correct.

    But just as Bitkeeper was a clumsy early attempt at implementing that paradigm, so is Nix. The article says that "Nix is declarative" which could not be further from the truth. It has a confusing and awkward language with multiple ways of accomplishing the same task, twisting concepts like "function" and "struct" into confusing knots that make it difficult to follow. Debugging is nearly impossible, and even test evaluations take a lot of work to get right. It is riddled with conventions and "helpers" that have idiomatic forms that are not well-documented and frequently abused.

    But the model is RIGHT. This is the way to distribute software, this is the way to write software, this is the way to ensure hermetic distributions, this is the way to avoid global implicit state. And Nix as a proof of concept is great because it shows that it does work (basically).

    So what is the path forward? Kill the language and replace it with configuration, with external scripts to do the heavy lifting. Make mutations of the data model the fundamental operations rather than having an intentional CLI that conceals the underlying functionality. So make it less usable but more understandable (like the path that Git took) and iterate from there.

    • kylewlacy 8 months ago
      There are also quite a few attempts at "Nix without NixLang". In the ecosystem, I can think of Nickel (a different configuration language, but one with strong typing), Garn (Nix but using TypeScript), and Guix (Nix but using Scheme, and also kind of an ecosystem fork I think?). The common link behind all of them is that they all still have Nix in their DNA, i.e. using a Nix store with Nix derivations under the hood

      Outside Nix, there's Brioche (my project!), and there's also Vorpal and Tangram, all of which are "Nix-like" but are from-scratch projects instead of using any Nix tooling

      So I definitely agree that Nix as an _idea_ is right, and I definitely also agree that NixLang is a blocker for mass adoption. Personally though, I think if you dig deeper, there are other places that could use improvement beyond just the language itself (but nothing that would necessarily be a show-stopper for adoption IMO). But that's why my project is a "Nix-like"-- there are things that I think Nix could / should do, but which aren't really compatible with the way Nix itself works. One example: Brioche works without having a global store path-- it's per-user instead so that it can be installed rootlessly-- which doesn't really map well to how Nix itself handles store paths and dependencies

      So point is, I agree Nix is the right model, but I think the _real_ gold lies outside Nix itself (...with the side note that I'm obviously biased!)

      • andrewla 8 months ago
        I’ll check out your project - everything you say here resonates with me, especially the rootless nature.

        What we probably need, long term, is a content addressable store, local first, secure, and freely shareable. Layering this on top of POSIX filesystems feels awkward and difficult to trust. But this is the fundamental building block of Nix-like systems and would make the move from rootless to system-wide a mostly trivial exercise.

        • brianjlogan 8 months ago
          That doesn't make sense to me. You can use Nix on non NixOS distros in your user space. Home manager is quite popular.

          Have you tried using it in user space?

        • jonringer117 8 months ago
          There's nothing wrong with the fundamentals of Nix.

          But working with nixpkgs is quite difficult for most. Which is why I'm attempting to address those issues in a fork, Ekapkgs.

          The repos will eventually expose optional flake entry points, so it will be compatible with flakes and non-flakes.

          If you're curious, https://www.reddit.com/r/NixOS/comments/1gatci0/announcing_t....

          • grhmc 8 months ago
            I love your project, Jon! I think the Nix ecosystem will be improved by having more, smaller things, where the boundaries are clear.
          • rossng 8 months ago
            It feels like there might still be time for Nix to morph into something that can get mass adoption. But it probably requires some radical technical changes and the controversy around flakes (which are incredibly useful) suggests that the existing community might not accept that.

            Whatever comes next, IMO the killer feature will be that it enables rock solid developer tooling. LSPs that work; typed inputs to modules; control of deployment of NixOS machines and so on.

            • Ferret7446 8 months ago
              Declarative configuration predates Nix, there are many far more popular options both before and after Nix, and generally declarative configuration is only ever popular/used in a few specific domains/niches.

              I remember being enamored by Haskell in college and I asked my professor why pure functional languages aren't more popular. His answer really stuck with me. He said: because reality is stateful.

              Reality is stateful. As a consequence, declarative configuration leaks like a sieve, and leaky abstractions suck. When you think about it, declarative configuration is just a DSL for implementing idempotent deployment flow, and idempotent deployment flows in general are quite successful (e.g., Docker, Flatpak, Ansible), and so declarative configuration tends to work okay for restricted subsets of those domains.

              • tome 8 months ago
                That's a strange objection, because Haskell supports state, of course. In fact I'd say it's the best language for supporting state, because it makes it explicit.
              • grhmc 8 months ago
                Back in the day, adopting Git was also clumsy and hard to implement. It was buoyed significantly by the investments GitHub made into its usability.

                I agree Nix is going to have to evolve to gain wider adoption. It's part of the work we're seeing in the ecosystem, and also work we're already doing :).

                • adamc 8 months ago
                  But git had Linus and Linux on board. What is going to give Nix that credibility?
                • anotherhue 8 months ago
                  I use NixOS in spite of the Nix Language. It's worth it.
                  • andrewla 8 months ago
                    Of course -- as do I. But I've become unwilling to accept that the failure of the Nix language is just a skill issue.
                    • 8 months ago
                    • grhmc 8 months ago
                      My man.
                    • rfoo 8 months ago
                      > Kill the language and replace it with configuration

                      No. Hard no. Please don't tell me you are going to kill a fine (even though awkward) configuration language and replace it with (potentially text-templated) yaml.

                      • Spivak 8 months ago
                        I think I would rather replace it with a spec for derivation files implementable in any language that existing build systems can hook into and use as outputs.

                            make derivation
                            python -m build derivation
                        
                        Trying to package software with dependencies that don't exist in nixpkgs is painful despite the tooling existing to build and install them.
                        • rfoo 8 months ago
                          That's an interesting idea. The downside (I could imagine) would be we have to convince people to implement THE RIGHT MODEL in every build tools in the world, but if that's possible then indeed it would work a lot better.
                    • uberduper 8 months ago
                      I really like Nix package management. I really hate Nix configuration management. I already know how to configure everything. I don't want to have to figure out whatever abstracted config options some random package maintainer decided on.
                      • grhmc 8 months ago
                        I feel that. This is one place where the "nix is everything, everything is nix" mentality hurts the project. NixOS modules are where the abstract config option sauce lives. It isn't an inherent part of Nix.

                        As Nix (and NixOS) becomes more widely used, this is one place where we'll have to find a way to let users do what they know how, and stop getting in the way.

                        • bsimpson 8 months ago
                          Part of the problem is that the underlying systems aren't Nix, so you're replacing the easy way with the Nix way.

                          I wouldn't be mad if opening the system preferences and setting my timezone also updated my timezone in Nix. I am mad that I can't change my timezone when I travel, because the GUI picker doesn't know that Nix has taken ownership, so only changes in Nix take effect.

                        • abdullahkhalids 8 months ago
                          > To make a smooth path from “Nix could solve my dev, build, and configuration management problems” to “Nix is solving these problems” and in a way that doesn’t involve turning someone from a developer to “the Nix person.”

                          When do we get Nix easy enough for not just software engineers, but also scientists etc who need reliable reproducible systems for running simulations with zero hassle?

                          • grhmc 8 months ago
                            Hell yeah. Speaking my language! I don't know, but we need that!
                          • gr4vityWall 8 months ago
                            FWIW, the post looks more like a startup pitch from its founder than a more technical article.

                            My take as of now is that I like what Nix achieves, but I don't feel the complexity is worth it. Docker and similar container tech have issues, yes, but I'd have way way less trouble onboarding a dev on it than Nix.

                            I hope that changes in the future, though.

                          • stackskipton 8 months ago
                            We have Nixstan on "Ops Team" who keeps pushing us to use it. We haven't seen need that much.

                            So can someone give me elevator pitch that if we are CRUD Web App, why Nix instead of Puppet/Ansible + Docker Containers?

                            Also, this from the blog article: "What if adopting Nix didn’t irritate your IT or security teams?" Too late, security team is already pissed. They did some scan, we pinned a package, it has 3 high severity and it needs to be updated tomorrow. So yea, we are just yoloing package updates, we don't have much of a choice.

                            Oh yea, we have some third party proprietary package we use. Support told us, Red Hat/Rocky/Ubuntu or we could kiss support goodbye. Any suggestions that doesn't involve shoehorning nix into Ubuntu which is like mud wrestling with a pig?

                            • thraxil 8 months ago
                              > So can someone give me elevator pitch that if we are CRUD Web App, why Nix instead of Puppet/Ansible + Docker Containers?

                              For better or worse, Nix forces you to work out the exact dependency tree of your entire system. It can be a pain but once you have it, if one part of that tree changes, nix can rebuild only and exactly the parts of the system that depend on it. That goes all the way through from packages and libraries to systemd services and application config files. That happens safely, reliably, is extremely cacheable, and is done in a way that can actually be perfectly reverted to a previous version. You can build docker images with nix and the result is very small minimal images (it's effectively a `FROM scratch` every time but with a potentially even better layer structure that again can be more cacheable than a typical Dockerfile setup).

                              From a security perspective, Nix can be a huge win since it gives you such precise control over what's installed on your system and makes it simple to track, every part of it checks cryptographic hashes, etc. But yeah, most security teams don't know what to do with it yet and probably don't have the tools to deal with it yet.

                            • amykhar 8 months ago
                              It's not going to be the future of anything if it doesn't get much better documentation.
                              • grhmc 8 months ago
                                Hear hear. It takes continuous effort not just on documenting, but also reducing the need for documentation by cutting interfaces and accidental complexity.
                              • grhmc 8 months ago
                                Hey folks CEO of Determinate Systems, and the author of the blog post. I'd be glad to answer questions about the post and what we're doing!
                                • nalzok 8 months ago
                                  I agree that Nix is the right idea, but I'm hesitant to adopt it right now because (i) rough edges like poor documentation and inconsistent interfaces make it feel immature, (ii) things are moving very fast with experimental stuff like flakes, which means it will take a while before Nix starts to stabilize, and (iii) recent community forks like Lix might create a fragmented ecosystem in the long run.

                                  What are your thoughts on there concerns?

                                  • grhmc 8 months ago
                                    I think those are fair perceptions and concerns.

                                    Regarding experimental features like flakes: the reality is they're incredibly stable. I've written about this before: https://determinate.systems/posts/experimental-does-not-mean.... They haven't realistically changed in years, because they work so well. The experimental label is practically FUD at this point.

                                    If the Nix team were to change flakes in a breaking way, it would be stunning neglect for the vast, vast percentage of the ecosystem that has already adopted them. Our data shows that of all the (OSS) repositories created every day, almost 90% of them start with a flake.nix. Of all of those projects, less than 20% use the legacy file formats, and most of those are using the flake-compat library.

                                    On documentation and interfaces, I agree, and we and the greater community are working hard to on that problem. I'll take time, but it is decidedly better than it was a few short years ago.

                                    And on community fragmentation, I just don't see it becoming a problem. The core Nix ecosystem is so large and diverse, I don't see meaningful fragmentation coming out of this.

                                    • Macha 8 months ago
                                      > Our data shows that of all the (OSS) repositories created every day, almost 90% of them start with a flake.nix

                                      90% of OSS repositories that use Nix I suppose?

                                    • dpatterbee 8 months ago
                                      As a response to (ii), I assure you that things are most certainly not moving very fast with experimental stuff like flakes. Flakes were first released as "experimental" almost 3 years ago and have been stuck in feature purgatory ever since.
                                      • Macha 8 months ago
                                        To be blunt, this is driven by the rejection of flakes by a significant group of the contributor base, despite it being much more adopted by the user base at large.

                                        Even as someone who does think Flakes are better than the prior solutions, I'm increasingly of the opinion that Flakes would be better moved to a layer outside the core Nix project - advancing them within core Nix at this stage seems pretty impossible with many within the project opposed to their existence. I think if Flakes were an alternative project at the same level as something like Niv, a lot of the holy warring would get out of the way.

                                        1. Those who wish to improve them could do so without the discussion being deadlocked by "hey, we haven't yet agreed these should be stable"

                                        2. Those who don't want to paint them as the path forward for fear of precluding a better option, now don't have to.

                                    • steveklabnik 8 months ago
                                      I am a Windows user. The default nix implementation does not support Windows.

                                      How can the future of software leave out one of the largest platforms on the planet?

                                      • stackskipton 8 months ago
                                        Because the future of software isn't on Windows. Even Microsoft realizes that with MSSQL moving to Linux (and dying), .Net recommended on Linux and internally, vast majority of workloads that can run on Linux are doing so. Apparently, a ton of Office Teams architecture is Linux.
                                        • jonringer117 8 months ago
                                          (Thanks for all your work on rust)

                                          There is some work to get nix to work with Windows, this is mostly being done by john ericson.

                                          However, the real issue is that nixpkgs (the repository containing all nix expressions) heavily assume unix paradigms.

                                          There could be a "nix-windows repository" in the medium to far future, but it will likely be very divergent from the current nix+nixpkgs story of today.

                                          • grhmc 8 months ago
                                            Hi Steve!

                                            Well. The future isn't evenly distributed yet :). There is work in flight to make Nix support Windows. Also, and I know this doesn't count, but it works great in WSL today.

                                        • rgreekguy 8 months ago
                                          Something I have never understood is why the hard-on with Nix, and Guix is not heard of even a tenth of the times. Is GPL really so repulsive to the masses? Or the libre kernel default?
                                          • sleepydog 8 months ago
                                            I think guix was just late to the party and the extra steps to setup the nonguix for the non free software like steam is enough of a deterrent for users to go to nix instead.

                                            I use guix myself and prefer its use of scheme. I don't mind having a little less guidance and "how-to" articles available if I'm trying something new because I'm comfortable reading its codebase.

                                            • dqv 8 months ago
                                              That's what it was for me. I'm really fuzzy on the details because this was years ago. When I first heard about Guix, I wanted to try it (even though I didn't understand the benefits) but there was some unfree part I needed to install it. And ideologically they were against that. There either weren't good tutorials on how to bypass it or it was not even possible. So I just didn't install it.

                                              FWIW Nix does not allow unfree packages by default or, at least, that's what having to add a package to allowUnfreePredicate makes it feel like.

                                              • eikenberry 8 months ago
                                                How is Guix's repository? One of Nix's biggest draws (I'd say the biggest) is it's expansive package repository.
                                              • Koshkin 8 months ago
                                                The future is not what it used to be.
                                                • johnklos 8 months ago
                                                  ...and it never was.
                                                • smallerfish 8 months ago
                                                  Cool! I'm glad somebody is working on this.

                                                  What's your business model?

                                                  • grhmc 8 months ago
                                                    A FlakeHub organization with private flakes and cache is $20/user/month, plus at-cost storage and bandwidth. Members of that organization get automatic access to the private flakes and read access to the cache. In the future, we'll be opening up the policy engine to make it more flexible.
                                                  • laerus 8 months ago
                                                    What about immutable systems like Fedora Atomic?
                                                    • grhmc 8 months ago
                                                      They're great! But also still miss fundamental pieces that Nix gets right. Especially the way the build and dependency graph goes right to the root of every package, and comes together to completely describe the system you're running today.
                                                    • bsimpson 8 months ago
                                                      > Our world’s infrastructure should not be stuck in a “don’t touch it, it works” mode, but it is.
                                                      • autoexecbat 8 months ago
                                                        Is there a good book that describes everything we need to know?
                                                      • croes 8 months ago
                                                        Nix is the future gets another meaning if you know that nix in German means nothing.
                                                        • grhmc 8 months ago
                                                          Eelco's typical online handle is "niksnut", which obviously means "nix nut" but also "good for nothing" :).
                                                          • scns 8 months ago
                                                            "Nichtsnutz" in german.
                                                          • jmull 8 months ago
                                                            It also means nothing in English, though that's somewhat obscure from my perspective. (Looking it up, it sounds like English borrowed it from German a while back.)

                                                            In English I think it's most often used as a verb that means basically to cancel. Like, "We're running out of time, we should nix feature X."

                                                            • andrewla 8 months ago
                                                              Similarly in English. I think "nix", like "git", is a self-deprecating name. It's a play on both "nix" -- to cancel or to not exist, and "nix" as the suffix for Li- and U-.
                                                            • oldpersonintx 8 months ago
                                                              [dead]