Git Push to Run CI/CD Is a Terrible Developer Experience

6 points by dan_manges 1 year ago | 5 comments
  • mmastrac 1 year ago
    CI/CD on push is the worst form of build system – except for all the others that have been tried.

    Ideally our builds would be containerized in something a little better than docker, fully hermetic and 100% reproducible outside of github. We just need to solve the caching problems, the overhead of FS access for containerized builds on platforms other than Linux, and the issue of most base containers living on highly centralized infrastructure.

    • verdverm 1 year ago
      Have you seen https://dagger.io? (by the OG creators of Docker)

      Fully containerized builds based on BuildKit with a <language> SDK interface. Works really well on both linux and mac

      Does the caching, parallel, lazy DAG really well, removes the need for intermediate or base containers to run builds, or more generally containerized workflows

      What Dagger enables is you to have your builds run the same locally and in CI. We still want builds that are auto triggered on commits, but we can end the stream of commits to fix CI

      • mmastrac 1 year ago
        Writing Go code for builds is not a great plan. I saw how badly this went with Gradle and Java builds, and Go is going to be even more verbose.
        • verdverm 1 year ago
          You can write python, typescript, or many other languages. I prefer TS because it has curly braces and object deconstruction. I do a lot of devops things in python though, so there is appeal for adopting the python SDK in certain contexts. I agree that Go is not a great choice for builds or scripting more generally, but it does mean you can build Dagger into any Go CLIs you might have, which is intriguing for me in one project I have.

          The experimental Dagger Modules allows interlanguage invocation. I'm not a huge fan of the current DX, a bit heavy for my monorepo setups, but once I can cleanly call them from the main entry based workflow, I can see consuming them.

      • dan_manges 1 year ago
        Solving those problems would be a dream. Although the interface to initiating a CI/CD run can be solved without solving those other parts. Allowing the run definitions to be passed in, and ensuring any data based on the version control event is parameterized, is an effective solution to the push-to-run problem.