Android in Docker without QEMU/emulator

226 points by PascalW 3 years ago | 71 comments
  • PascalW 3 years ago
    Came across this very useful project. It provides Docker images with Android running directly in Docker, without qemu or an emulator. I've never seen this before, all other solutions that I'm aware of either run the Android emulator in Docker or use qemu directly (like Anbox).

    Advantage of this is that it's very lightweight and does not require VT-X or AMD-V, ideal for running in cloud environments that typically do not expose this CPU capability.

    • OJFord 3 years ago
      Anbox and Waydroid are others, though they don't use Docker.

      https://github.com/anbox/anbox (https://news.ycombinator.com/item?id=24684187)

      https://github.com/waydroid/waydroid (https://news.ycombinator.com/item?id=28616985)

      • mathfailure 3 years ago
        ReDroid: Requires Docker. The images seem to be based on pure Android. Multiple versions available: 8.1, 9, 10, 11, 12.

        Anbox: Requires 'snapd'. The image is based on AOSP based on Android 7.1.1.

        Waydroid: Requires Wayland. The image is based on LineageOS that is based on Android 10.

      • carlhjerpe 3 years ago
        Anbox doesn't use QEMU. It uses the same underlying kernel tech as Docker, LXC and other isolation tech. This is why you need the ashmem and "binder"(?) kernel modules to run Anbox.

        I'm currently "trying" to get Anbox working on NixOS (It's currently broken on 5.x kernels but should be fixed by https://github.com/NixOS/nixpkgs/pull/102341) so that's why I know that Anbox is the "same" as this.

        • PascalW 3 years ago
          Interesting. I've never been able to make Anbox run in a cloud environment though, don't recall the exact details. But indeed I guess it should be able to work as there's also a commercial offering targeted specifically at cloud setups. https://anbox-cloud.io/
          • rjzzleep 3 years ago
            I personally haven't managed to get anbox to run at either acceptable performance or any kind of level I would consider stable. Someone posted waydroid as an alternative here a few months ago, but I haven't tried it yet.
            • MayeulC 3 years ago
              Waydroid seems performant enough to me. SuperTuxKart is very playable (though I didn't try much as on-screen controls are hard to handle with a mouse). Microsoft excel also works.

              A lot of android apps only come with arm library, and need an extra emulation layer on x86, I haven't tried that.

              • NoahKAndrews 3 years ago
                Waydroid is a more performant successor to anbox.
            • blueflow 3 years ago
              Consider taking a moment to look into the difference between "emulator" and "virtual machine".

              What the link refers to is an kind of Android emulator, but not the virtual machine kind of emulator.

              If you are tempted to write a rebuttal in the "reply" box, answer this question first: Why are programs like xterm and urxvt called "Terminal Emulator"? What do they emulate?

              • zaat 3 years ago
                Hi blueflow, I see that today you are not tired of arguing semantics.

                The issue here is context. What GP means by Android Emulator refers to specific piece of software, the AVD emulator provided by Google which emulates full android device, it emulates more than just VM, it's also have skins and physcal buttons.

                https://developer.android.com/studio/run/emulator

            • pengaru 3 years ago
              What exactly is Docker enabling here beyond a Docker workflow, which is quite orthogonal to the presence of a vm or emulator?

              Isn't the enabling technology here just kernel-level android bits? Docker is fluff, yet ends up dominating the headline.

              • nine_k 3 years ago
                Does it not provide the mobile-like environment that apps expect?

                I mean, cgroups, firewall rules, and chroot are also already provided by the kernel, but using them for running containers without Docker, or Podman, or LXD, or other piece of "fluff" is slightly unergonomic.

                • pengaru 3 years ago
                  If it were Docker providing the secret sauce, Docker would be all you need to make it happen on non-Linux hosts.

                  It's the Linux kernel, configured with the appropriate Android modules, not Docker making this happen.

              • 0x0 3 years ago
                Interesting concept but seems to require custom android-specific kernel modules (ashmem, binderfs), which means the android containers aren't very containerized but leak into the host kernel.
                • captn3m0 3 years ago
                  All Linux containers leak the host kernel no?

                  The Anbox project relies on the exact same 2 modules: https://docs.anbox.io/userguide/install_kernel_modules.html. While these are "android-specific", they are present in the linux kernel tree, and even come pre-installed in Ubuntu 19.04 (and probably more distros).

                  • NotEvil 3 years ago
                    Not all linux kernel leak to host. Docker security is way better than nefore. And there also projects like podman and docker unprivileged that don't require any root intraction to set up a kernel
                    • captn3m0 3 years ago
                      Depends on your definition of "leak" I suppose. You can't get a different kernel version inside a container so there must be a "leak" in some sense. Reducing capabilities etc will just leak lesser information about the system in general, but your kernel is always shared.
                    • j4hdufd8 3 years ago
                      > All Linux containers leak the host kernel no?

                      Curious in what way?

                      • captn3m0 3 years ago
                        See any explanation of what a linux container is[0]. Containers in linux are a patchwork of various features that "contain" a process group in different ways while sharing the kernel with the rest of the system.

                        Running either of

                            docker run --rm --entrypoint /bin/uname alpine:latest -a
                        or

                            uname -a
                        gives you the same kernel version. The only difference will be in the hostname.

                        [0]: https://embeddedbits.org/introduction-linux-containers/

                        • IceWreck 3 years ago
                          That's the difference between a container and a VM. VM runs its own kernel. A container relies on the host kernel and the host machine's kernel does all the scheduling, memory management, etc.
                          • pmontra 3 years ago
                            See the other replies and if you're on Linux run

                              ps -ef
                            on the host and you'll see the processes in the containers too. That doesn't happen if those processes run in a VM.
                        • jeroenhd 3 years ago
                          The Docker container also needs to be run in privileged mode, so you should probably treat this as if it were an emulator running root. It does work around dependencies on VT-x and such, though, and cuts down on the performance overhead of virtualising an extra kernel.
                        • 2OEH8eoCRo0 3 years ago
                          Sounds scary. That's a no from me dog.
                        • marcodiego 3 years ago
                          Don't forget waydroid: https://waydro.id/ . Last time I tried, it worked reasonably well.
                          • cranx 3 years ago
                            This is interesting, but I don’t see this as something for use in development of Android apps. Given all Android’s quarks and fragmentation the use of real devices and Google’s emulators will provide a better surface to catch bugs during dev. What’s the primary use case for these containerized versions of Android?
                          • wwwhizz 3 years ago
                            required params (already added in docker image)

                                qemu=1
                            How is that without qemu?
                            • testtesttestte 3 years ago
                              It's just a parameter to let the Android OS know that it's running in an emulated environment as opposed to running natively on a physical device.

                              For example, the GPU stack has to be somehow emulated using a custom OpenGL driver (in this case `mesa3d`). There's software emulation (swiftshader) and host GPU mode (GLES).

                              For performance reasons when using the host GPU, the GLES commands from the guest have to be serialized and sent over a kernel pipe to the QEMU and to the host OS. The commands are deserialized on the host and executed on the physical host GPU through the Shader Translator library (taken from the Google ANGLE project). The Google's QEMU fork (aka Android Emulator) usually loads this library and then takes care of proper rendering, such that users can see the rendering output of the emulated OS properly inside the emulator window on the host machine. You can do the same for the docker, otherwise you'd have to implement some other trick to share the GPU. `virtio-gpu` is a Linux kernel facility that is meant to replace QEMU pipe in the future.

                              The high-level details of this OpenGL emulation mechanism are described here: https://android.googlesource.com/platform/external/qemu/+/ma...

                              You can find the implementation of the QEMU OpenGL pipe mechanism here: https://cs.android.com/android/platform/superproject/+/maste...

                              • tgtweak 3 years ago
                                Yeah not reading anywhere in the docs that it works without qemu. Also the need for kennel modules to be available on the host and passed through makes it less portable than most containers.
                            • drran 3 years ago
                              Will it work on WSL? It can be a good way to run Android on Windows.
                              • AkshitGarg 3 years ago
                                According to its README, it does run (and is tested) on WSL 2 (Ubuntu) (amd64).
                                • manojlds 3 years ago
                                  Windows (11?) has the feature built in right?
                              • jeroenhd 3 years ago
                                How is this accomplished? Is this using Anbox or something similar I don't understand how the Dockerfiles linked here end up running an instance of Android.
                              • fareesh 3 years ago
                                For development work I use the default Android emulator that comes with Android Studio.

                                Are any of these ones superior in terms of performance, resource utilization, etc? The cloud angle is mildly interesting although in my specific use-case I'd probably just end up paying for a service that can provide access to emulators in the cloud.

                                Is there more scope to do things like automating inputs etc this way? How would one go about that?

                                • mkdirp 3 years ago
                                  This one specifically seems to run Android in the Linux Kernel using the usual suspects of container tech. It should have next to zero performance impact so it will definitely have better performance compared to the default Android emulator considering that's using qemu, even if it's qemu-kvm.

                                  Whether it's as reliable as running a VM, is something that'll need to be tested. Personally I got an error when trying to exit out of the shell (and thus was unable to exit), something I've never seen in the VM version of Android.

                                  • junon 3 years ago
                                    Sounds like QEMU doing that. QEMU has a command mode - ctrl+A followed by X should kill it.
                                    • mkdirp 3 years ago
                                      This isn't running on QEMU.
                                    • edoceo 3 years ago
                                      Did you try

                                        :q!

?
                                    • j4hdufd8 3 years ago
                                      > Is there more scope to do things like automating inputs etc this way? How would one go about that?

                                      adb or UIAutomator

                                  • exdsq 3 years ago
                                    Trying to fix some Rust docker builds that are being killed by QEMU on Apple Silicon right now. I always believed Docker was cross platform, I had no idea how much of a pain it can be to use with buildx and --platform to get stuff working. SIGSEGV errors at random points, non-deterministic errors, GAH!
                                    • 0x0 3 years ago
                                      That you (and many others apparently) had the impression that docker was cross platform says a lot about docker's markering, considering docker as far as I know it is a wrapper for a series of extremely linux-kernel-specific APIs. :-/
                                      • exdsq 3 years ago
                                        Yep! Amusing to think I used to think cross-platform just meant running on Windows and Linux and that was fixed with Docker. Had no idea the rabbit hole was deeper until Mac released the new CPUs.
                                    • Tepix 3 years ago
                                      This could be useful for running WhatsApp inside a vm instead of your phone for those rare instances where it is needed.
                                      • yetanother-1 3 years ago
                                        Frankly I just install Island, and then run Whatsapp (and all not-always-needed apps) inside it. Plus bonus: you can freeze the apps you don't need and keep the island running.
                                      • nik5 3 years ago
                                        Exactly, what I am going to use it for. I tried using whatsapp for long in a VM of android x86 image didn't work too well for me.
                                        • aunetx 3 years ago
                                          I recommend you to use web.whatsapp.com then, it's pretty decent now (except that it does not support calls)
                                          • detaro 3 years ago
                                            That needs whatsapp running on your phone to work as far as I know (because it basically runs the communication on the phone and then syncs that to the browser)
                                        • j4hdufd8 3 years ago
                                          What do you mean? Where is it needed?
                                          • Tepix 3 years ago
                                            There are unfortunately situations where the only way to reach someone is via WhatsApp. I have avoided them but i can see WhatsApp in a VM being useful for those purposes.
                                            • flatiron 3 years ago
                                              i believe they are referring to the whatsapp bug that the saudi prince used on jeff bezos. simply sending bezos a message gave the saudi prince root access on his phone and he dumped all his messages and photos and released them to the press. running in a vm would just have whatsapp so they can send and receive messages without exposing all the private information on their phone in case of some 0day
                                              • qwertox 3 years ago
                                                So you would run the VM on a phone? Because if not, you'd need to carry an open and connected laptop around.
                                          • goodpoint 3 years ago
                                            I'd rather use waydroid to avoid the attack surface of docker and its complexity.
                                            • junon 3 years ago
                                              Perhaps a simple question, but... why?
                                              • mathfailure 3 years ago
                                                Because alternatives have some requirements that may not suite some people.

                                                1. VMs have relatively big resources overhead and this approach requires you to download some Android image first.

                                                Of course you'd want some clean distribution and there simply aren't many such ones (that I know of) that are easily available. The ones I tried - were a 2-in-1 shitshow+slideshow for me. Maybe it's because I used ISO (for QEMU/KVM), as I didn't want to run proprietary crapware (VirtualBox/VMWare).

                                                2. Going with Anbox requires having crapware snapd and their Android image is quite old (Android 7.1.1).

                                                3. Going with Waydroid requires having Wayland, which if you don't use it - is an overkill to install just to have Android on your PC.

                                                4. Going with Android from AndroidSDK - is probably an even harder way to get access to some Android on your PC (I don't really know, I haven't investigated that option).