Lightweight Alpine VMs on macOS

150 points by gandalfff 2 years ago | 95 comments
  • mberning 2 years ago
    The container landscape is getting crowded. As a person who just wants to get things done with containers it’s not immediately obvious where I should be focusing my efforts due to the proliferation of “container solutions”. It seems especially bad on the mac right now.
    • qbasic_forever 2 years ago
      Colima is all you need now, it's basically a drop in replacement for docker desktop. There are actually very few container runtime engines (containerd and runc are the big ones) and all the tools you read about just wrap that lower level container runtime. They're all the same when you get down to it and just have different opinions about config, networking and storage.
      • idontwantthis 2 years ago
        Every tool says its the same but they aren't. You'll eventually run into some issue that wouldn't have happened on Docker, and as soon as a dev spends two hours on it you've paid more than you would have paid for a Docker Desktop license for that dev for the year.
        • nickcox 2 years ago
          To be fair, you'll run into issues with Docker Desktop too though. I've found Rancher Desktop to be as reliable or better than Docker for my needs.
        • evol262 2 years ago
          `containerd` invokes `runc` (and `docker` can also invoke `containerd` via its API, which then invokes `runc`). The "lower level container runtime" is cgroups and various kernel namespaces (primarily process and network, but some others are included). There's no magic there.

          They are not all the same when you get down to it, though. The "different opinions" about how to plumb traffic back out from a container (DNAT/SNAT via a bridge, macvlan, whether using a CNI directly is supported), whether a service/daemon should be the primary entrypoint (docker, containerd) or whether it's optional (podman), whether they speak to runc at all (containerd/docker yes, podman defaults to crun, kata is also an option, and others), what kind of storage overlays and plugins are allowed, etc are more than "opinions".

          The devil is in the details. Colima is "basically a drop in replacement for docket desktop" under the assumption that you aren't doing anything very complex with Docker. In particular, complex networking is likely to fail/explode.

          • MrBuddyCasino 2 years ago
            I recently switched from Docker Desktop to Rancher and it seems to not slow down my Mac so much.

            Does Colima work with Docker Compose?

            • rollcat 2 years ago
              Compose is a layer on top of the Docker client/API, so the container runtime provider shouldn't matter, I guess as long as there's a real dockerd under the hood, and modulo any bugs.
          • idontwantthis 2 years ago
            I think you'd need a particular reason to not use Docker.
            • jlokier 2 years ago
              Last time I tried using Docker Desktop on my MBP (x86), the reason to not use Docker was that it was excruciatingly slow for the test suite I had to run in it, compared with Docker on Linux.

              I abandoned Docker on the Mac because of that, and haven't touched it since. That was early 2021; maybe it's faster now.

              • deergomoo 2 years ago
                It’s the cross-OS filesystem stuff that always killed it on Docker for Mac (and to a lesser extent Docker for Windows).

                I believe improvements have been made, but a lot of people these days check out their code directly in the container rather than using bind mounts, leveraging the fact that a lot of editors/IDEs now will interface directly with the containers.

                • asah 2 years ago
                  Docker Mac m1 is my daily driver. Painless performance.

                  But... watch out for software compatibility: m1/m2 requires recompiling from source which is painless... when it works. I recently needed syslinux and had to move to x86 cloud instance. Fortunately, docker made that easy.

                • badrabbit 2 years ago
                  It takes more time and effort than just running the damn script! Is that not good enough?

                  I mean I want to use containers but on top of setting up the host, they require composing containers (even when ready made for customizations), networking, logging and fight for more memory when using memory hungry stuff in conjunction (like elastic or other db).

                  If my main job was devops, I suppose I would make myself more valuable by doing everything in containers but when I deploy an app it is because I have to on top of many other duties so being able to not only setup but troubleshoot and fix outages quickly is most important (and I hope a full time devops person, if I ever get one) will help me migrate all that some day so it looks nice and neat.

                  • idontwantthis 2 years ago
                    If you don't want to use containers at all then that's a different question than the one I I was responding to.
                  • sneak 2 years ago
                    Docker Desktop is:

                    - spyware (transmits private data off your machine without consent when it crashes, which it does a lot)

                    - nonfree software

                    - has a git repo so you don't notice it's nonfree software

                    • antihero 2 years ago
                      Does docker support true x86_64 emulation on arm64 hosts?
                      • idontwantthis 2 years ago
                        No, and that’s a pretty good reason to not use it if you really need that.

                        What can you use to do that?

                      • xmonkee 2 years ago
                        I do have a particular reason - memory use. I am running postgres and redis locally for dev work, but I would love to use Docker so that I can standardize it for my team, but it just takes up to much ram on m1.
                        • idontwantthis 2 years ago
                          I don't mean to sound flippant, but that sounds like you're using a computer for business that can't handle the work you are trying to do. If 32/64GB isn't enough memory then yeah, I guess you need something else, but if your machine has less than that then it sounds like you need to buy the right computer for the job.

                          Also, are you using AMD or ARM images for those?

                          • machiaweliczny 2 years ago
                            You can setup how much ram is allowed in docker. Generally software will use as much as you allow (especially DB)
                            • pmontra 2 years ago
                              Your team might want to use asdf https://github.com/asdf-vm/asdf to run multiple native versions of PostgreSQL and Redis in parallel. Even with one project you might have multiple versions of those tools in different releases of the project. You standardize by using a .tool-versions file. I've been using that for a team targeting Linux and developing on Ubuntu, Mac and WSL (or was that an Ubuntu VM in Windows?)
                              • vineyardmike 2 years ago
                                Many companies run dev work in a dedicated cloud VM.. incl well known companies like Google or Amazon.

                                You can run a constant VM with 2/4/8/128gb of ram or whatever you need. I use one at work for years and I think mine is 16gb of ram and it’s way over provisioned most of the time. Unlike how you might expect, treat the cloud vm like a work laptop not a production service. Let people write scripts that stay there, let people keep it on 24/7, available on demand, etc. It’s a cloud laptop not a production VM.

                                • ehutch79 2 years ago
                                  I use a 16gb m1 air. I'm running docker desktop with mysql, redis, 2 containers doing python, a node container and an nginx container. I'm not noticing any impact on performance. MS Teams hurts more to run. Though I have adjusted the resources docker uses.
                                  • epgui 2 years ago
                                    I never understand the "docker takes up too much space/ram" objection. Isn't that configurable/manageable even from the GUI?
                                • davewritescode 2 years ago
                                  Colima or Rancher Desktop seem to be what most developers at my place of work have coalesced around these days.

                                  Colima is a little but lower level but works very well. Rancher Desktop had some struggles a while back but most of the developers who are new to the company seem to be using it for local kubernetes.

                                  • perryizgr8 2 years ago
                                    Just use docker. It always works well enough and there is tons of support available online for any conceivable situation you might find yourself in.
                                    • throwme_123 2 years ago
                                      Title says "on macOS".

                                      That's the key part: Your positive experience is with PCs running linux I guess.

                                      • spyremeown 2 years ago
                                        I know this is a stupid argument from my part, but I see this enormous effort to use tools that work really really well on Linux, but on a Mac. Why not just use Linux and save yourself the effort, performance overheads and whatnot? 99% of the time the loads are being deployed on Linux servers, why not use it in the workstation as well? Is it worth JUST for the hardware?
                                        • porker 2 years ago
                                          But do any of these alternatives perform better on macOS? Under the hood they end up using the same underlying technologies, so there shouldn't be much performance difference between them.
                                      • syntaxing 2 years ago
                                        Dockerfile + Docker compose is the way to go. With Portainer, you'll be up and running in less than 30 min.
                                      • smcleod 2 years ago
                                        I'm really not a fan of that docs website/static page generator - or maybe it's the way the content has been laid out - it shouldn't take so many clicks and full screen refreshes to get basic information that could each be a heading on a single page so you can grep for words, use the mouse less etc....
                                        • samwillis 2 years ago
                                          I think they have split it into far too many small pages, the whole CLI section should be one page, if not the whole lot.

                                          Slight aside though, they have used the Material theme for MKDocs, which is much more than a theme, it's a whole extension package of bells a whistles for MKDocs. I'm not actually too keen on "Material for MKDocs" [0], I like all the clever plugins, but the theme itself I find distracting and too "loud". The theme jumps out more than the content itself.

                                          In the Python ecosystem Material for MKDocs seems to be the leading default at the moment, however I much prefer Furo for Sphinx, it's much cleaner, it's about putting the content front and centre. It does also have a similar set of plugins for Sphinx adding those bells and whistles.

                                          0: https://squidfunk.github.io/mkdocs-material/

                                          1: https://pradyunsg.me/furo/

                                      • reversethread 2 years ago
                                        I don't see the point of a dedicated tool for this when it is easy enough just to start a Alpine docker container with a couple commands. As this project is just a wrapper for docker and LXD[1] and those tools are already easy enough for the average SWE to interact with, the project seems to just over-complicate an already existing workflow.

                                        [1] https://github.com/beringresearch/macpine#motivation

                                        • cpach 2 years ago
                                          The container runtime (e.g. Docker Engine) needs somewhere to run. It can’t run directly on the macOS kernel. AFAICT that’s the point of this project. (Similar to Lima.)
                                          • behnamoh 2 years ago
                                            > (Similar to Lima.)

                                            Why did I read that as "similar to Ligma" ...

                                          • 2 years ago
                                          • dindresto 2 years ago
                                            Is there any containerd solution for macOS that makes use of the Rosetta binary provided to Linux VM's for x86 containers yet?
                                            • arianvanp 2 years ago
                                              Yes. Lima merged support for this last week.
                                          • coderintherye 2 years ago
                                            There's related discussion from earlier today at https://news.ycombinator.com/item?id=33762657 for LXD
                                            • encryptluks2 2 years ago
                                              This is like the 5th or 6th frontpage macOS container strategy this week. Wouldn't it be better to have native containers by just using Linux? What advantage do people get hy using macOS?
                                              • jillesvangurp 2 years ago
                                                Great laptops and hardware; low hassle setup. There is some initial linux support for the M1 hardware but using mac os is a bit easier. The instructions for getting docker going on a mac are a lot shorter than getting linux going on an M1. So, if you have one, figuring out how you are going to run docker is something to do. And there are many options now.

                                                I actually have a Manjaro laptop that I used for work for almost a year and it was great. Except for the hardware (generic cheapo wintel garbage). I'm back on a Mac now. Nice M1 laptop. Fast, silent, good keyboard and screen. Wonderful to use. Mostly my biggest headache is muscle memory for different key bindings and keyboard layout because I still use the linux laptop once in a while. But otherwise all my stuff (including docker) just works on both sides.

                                                Docker for mac is nice but the licensing can be a bit of a show stopper. I've yet to try some of the alternatives mentioned here. I did use qemu on my old intel mac for a while with some simple environment variables to make the homebrew version of docker use ssh to my vm. It works but it can be a bit wonky with things like port forwarding and volumes. You can make all that work but it is a bit fiddly. Most of the proper alternatives make this a bit more seamless. But I'd recommend trying it just to de-mystify the whole process.

                                                There is a docker desktop for linux even; which just goes to show that it does do a few things that are worthwhile having for some people. Even on Linux. I'm mostly a cli guy so I don't care about the UI/UX that it provides. But some people seem to like that.

                                                • lars512 2 years ago
                                                  MacOS contains tons of nice usability as a working environment, and even has great rolling package management with Homebrew, but the lack of native containers is the single biggest downside, basically preventing us from using containers more extensively at my work. The blocker for us is data work that needs a lot of memory locally, meaning it wouldn't work for us to allocate it to a VM for containers.
                                                  • rollcat 2 years ago
                                                    > What advantage do people get hy using macOS?

                                                    The question applies both ways: what advantage (other than native container support) do I get by running Linux?

                                                    Personally I would love to see OCI containers supported natively on other operating systems. Currently you get the same VM crapshow on e.g. OpenBSD, except the community is several orders of magnitude smaller, so you don't even get prepackaged solutions.

                                                    • encryptluks2 2 years ago
                                                      I get the advantage of transparent source code, a vibrant community supporting amazing projects, first-class support for new concepts and features.

                                                      > Personally I would love to see OCI containers supported natively on other operating systems. Currently you get the same VM crapshow on e.g. OpenBSD, except the community is several orders of magnitude smaller, so you don't even get prepackaged solutions.

                                                      Talk to your OS vendor. They are the ones who are preventing this from working.

                                                      • rollcat 2 years ago
                                                        I find it odd that you used both "it's a community project" and "talk to your vendor" in the same reply to address the same concern.
                                                    • anentropic 2 years ago
                                                      desktop Linux is bad
                                                      • encryptluks2 2 years ago
                                                        For who? People that don't want to learn a different operating system? I use it and it works great. Just got done playing some Steam games on Linux. Steam decided to choose Linux for their Deck cause it is pretty amazing.
                                                        • glogla 2 years ago
                                                          Desktop linux is fine. Laptop linux is bad and every company gives employees laptops.
                                                      • MBCook 2 years ago
                                                        So the automatic port forwarding/FS sharing/networking is nice if you want that.

                                                        But I often don’t. When I’m using Docker on my Mac it’s usually because I’m trying to use Docker. I need to use an existing Docker container or build a new one to fit some purpose with a Dockerfile.

                                                        I guess it’s nice that there would be a simpler way to launch one-off containers or containers for myself that aren’t expected to work like every other Docker container.

                                                        Is this a common need? Is there something that makes this more than I’ve noticed? The fact I work in a “Docker for containers” place may be preventing me from seeing what would make this shine.

                                                        • LAC-Tech 2 years ago
                                                          Love Alpine Linux. Only thing stopping me from running it on my main workstation is that my Sublime Text is my main editor and they have no musl version.
                                                          • m000 2 years ago
                                                            I like the concept of having a musl-based distro, but the utility of Alpine is frankly limited at the moment.

                                                            Sublime is just one case. We had a Python service running in an Alpine container because it was thought as "mean and lean" by someone. Sound choice, right?

                                                            Guess what: we used a handful of (popular) Python modules that are backed by native libraries and PyPi didn't have musl-linked versions for them. The "mean and lean" Alpine-based image ended up weighing more than a debian-slim-based image.

                                                            • LAC-Tech 2 years ago
                                                              I had Alpine running on WSL as my main dev environment for a long time. I've heard the python issues before, but I can confirm node.js stuff all worked flawlessly.

                                                              IIRC npm will compile native extensions, sounds like PyPi (is that a package manager?) distributes binaries.

                                                            • gabrielgio 2 years ago
                                                              I'm using Alpine as my main workstation, and for those situation (like for Slack) I use flatpak.
                                                              • LAC-Tech 2 years ago
                                                                Guess I'm already using flatpak for a few packages on my current glibc distro. Not that elegant but better than nothing.
                                                              • rollcat 2 years ago
                                                                You can install glibc+support libs on Alpine to run third-party software, but YMMV - I've had very mixed results.
                                                                • sphinxster 2 years ago
                                                                  Have you tried these options for running glibc software on Alpine?

                                                                  https://wiki.alpinelinux.org/wiki/Running_glibc_programs

                                                                • Sirened 2 years ago
                                                                  it's a thin wrapper around qemu, for those interested
                                                                  • ofrzeta 2 years ago
                                                                    I have a Macbook Air M1 but I have given up on running x86 containers. Too much hassle, to much diskspace consumed and too slow. I can run various web projects natively but when I need x86 containers I use a x86 notebook with Linux.
                                                                    • ge96 2 years ago
                                                                      If they can get VSCode to work on musl that'd be greaaaat.
                                                                    • joshmn 2 years ago
                                                                      Nice wrapper around qemu for those wondering.

                                                                      As someone with MacAlpine heritage I have never been more disappointed in two letters, though.

                                                                      • deafpolygon 2 years ago
                                                                        This is your typical MacOS frenzy over solutions in search of a problem space that didn't need solving.
                                                                        • phplovesong 2 years ago
                                                                          So just like the blockchain?
                                                                          • deafpolygon 2 years ago
                                                                            shrug.. I don't care about blockchain