The biggest dissonance when it comes to the purported benefits of monorepos is that a "good" monorepo generally assumes very good interface design skills across all teams, but in reality, the path of least resistance is tacking on more and more unique codepaths (e.g. forking/"rewriting" existing things), so in effect, likability often comes down to how well a team is able to isolate itself from global changes (by choosing stable/boring APIs, inventing their own abstractions in their own little corner, or what have you).

  - are in the same repo (making it easy to
    find and change all of them)

  - are in the same universe of build/test/deploy
    services (making integration of your changes
    atomic)

Conversely, if you want to be able to make breaking interface changes, the integration and deployment of those has to be atomic.

Not having a monorepo & monobuild means that you have to have stringent interface backwards-compatibility commitments. That's fine if you're shipping an operating system, say, but it's usually too painful if you're not shipping anything to third parties.

For me, the atomicity feature is the killer feature of monorepos.

- the cost of having access to more than you need: cognitive load and tooling for filtering, larger repositories require more tooling work to be performant

- there's also the atomicity of change and past changes which one can see/understand

Anyway, it seemed like such an obvious complement to the the perforce/monorepo style of working that I came away surprised that perforce wouldn't have hoisted such a thing into their product as a first-class feature. Tracing dependencies across a lot of different build systems is obviously not trivial, but it's not intractable, particularly if the tool is pluggable so that orgs can provide modules to handle their own particular approach.

Your dev tooling also influences the shape of the thing that you write. If you have a monorepo then it encourages you to ship a monolith that freely interoperates with itself. If you have multiple repoes that need to be versioned against each other, you will ship components with more stable APIs.

So this means that if you ship a product within which customers are free to update portions of them at will, then using a monorepo will make things more difficult than necessary.

And if you ship a single unversioned monolith to the world, then using multiple repoes adds unnecessary friction to working within the company.

This practice was abandoned, but I don't know the reasoning for why it was abandoned.

That friction sometimes helps: If it is painful to update Dependency A because it usually means upstreaming changes to A's Dependency B first, for instance, that can often indicate a tight-coupling problem that in a class diagram someone might easily discover and refactor over lunch but in a systems diagram was non-obvious without that "update hell" pain. Solving such tight-coupling problems is hard, and it may mean living with the pain for some time, and while monorepos make that pain go away they never solve those coupling problems (and arguably make it far easier to strongly couple systems that you likely don't want coupled). It's a lot like turning off all the Warnings in your compiler; it makes the immediate dev experience a lot nicer, but it risks missing things that while not problems now may be problems in the future.

I think there are also some benefits to using the same dependency managers for first-party components/libraries as for third-party components. The auto-updating of first-party versions is seen as a benefit to monorepos, but if recent and current CVEs have taught us anything you need to audit and update your third-party components quite regularly. Needing to also update first-party components/libraries with the same dependency managers has some benefits in terms of forcing a regular dependency update cadence, that then also benefits additional developer eyes on third party update rhythms. (Especially as increasingly more dependency managers pick up auto-auditing/security and CVE awareness tooling that runs on each update. There's more likely developer eyeballs on those audit reports if frequently run for first-party components and third-party components.) Dependency managers are their own friction in the process, but necessary friction for third-party components, and there are benefits to first-party components needing the same friction.

As with most software development practices there is no objectively "right" answer here. Monorepos have less friction in a large org. Friction and pain are sometimes useful tools, despite few people "want" them in their developer experience. Systems design is hard and tight-coupling is often an easy solution. Looser coupling is often better, more resilient design that is easier to work with at the boundaries and the "I can trust this other team's repo to be a black box and they let me file bug reports as if they were a second-party vendor" level, which can be its own tool for avoidance of mental fatigue.

The linear version numbers could have an advantage in that regard. If you want to know if CL 12345 is deployed, and you know the current deployment is running as of CL 12350, then it should be in there. Conversely if it's less than that number, it's definitely not in there.

git hashes also have good properties but I'm wondering how non-technical employees use them. Do they know how to dig through the git history?

Similar to the engineering/BOM-oriented semantics of everything is a drawing with a matching part number?

[0] http://beza1e1.tuxen.de/amazon_manyrepo_builds.html

Sure, avoid changing the API contract. But when the time comes to change the API, you can 1) make the change backwards compatible and maintain both methods forever; 2) release a new major version and maintain both versions forever; or 3) just migrate the callers and immediately be free of all technical debt that would've been accrued in 1 and 2. This assumes internal clients whom you presumably can't break.

1: https://bazel.build/concepts/visibility

> unless you invest a ton of resources into proprietary tooling that requires a bunch of maintenance, since all the open source solutions are terrible at this and the whole data model is built around splitting out individual project repositories.

Agree that there's a bunch of tooling needed to operate a monorepo, but there's also a bunch of tooling to sanely manage dozens of "microrepos" as well (when an upstream library changes, update downstream libraries' dependency manifest files to take the new version, run tests, report errors back to upstream, etc). I don't know of any open source tools that manage this problem, but I'm guessing they aren't high-quality if only due to the complex nature of the problem space.

> And then after all that effort, why wouldn’t you just use tooling the way it was intended, and the way it’s used in the open source model, so you can partition your CI/CD without a bunch of hacks, and don’t run into bizarre scaling issues with your VCS.

Because the tooling sucks, as previously mentioned. Many changes require touching many repos, which means coordinating many pull requests and manually changing dependency manifest files and so on.

Ultimately, the "repo" concept is limiting. We need tooling that is aware of dependencies one way or another, and sadly all such open source tooling sucks whether it assumes the relevant code lives in a single repo or across many repos.

Since leaving Amazon, I've mostly worked with monorepos and wouldn't go back to multirepos without Brazil-style tooling.

For context, I was for a very long time at FB so am definitely used to the monorepo way, and recently switched to place which uses github + many repos, and it feels so much worse.

Honest question - how do you actually effectively share code between many repos? Example: How do I know that me changing my backend app’s API doesn’t break any other project in the company potentially calling it? It should be a compile/buildtime error for the other project, but how does that work if everything is in its own little repository?

Not sure if it is a generic comment or a comment on TFA:

i) If the latter, I'm compelled to point out that TFA doesn't nearly advocate for monorepos as much as it lists reasons why a few SV companies use it, how they use it, and what they get out of it.

ii) If the former, then this blog post makes for a good read: https://tailscale.com/blog/modules-monoliths-and-microservic...

Ultimately the problem is that we need tooling which is aware of dependencies, and the repo abstraction isn't. Whether that code lives in a single repo or in many repos is fairly irrelevant, but keeping the code in a single repo is usually a fair bit easier for many things (especially when you're working in a single language since the language's build tools are usually well-suited for this basic case) and you don't need to manually update dependency manifest files, test how a given upstream change affects every downstream package, or coordinate half a dozen PRs for every change.

It's simple, with proper tooling, you know exactly the dependencies, so you know which test depend on the affected files and can run those tests, the rest shouldn't be impacted. And that tooling exists. It's not the one you may be using, but it exists, and not just in FAANG.

It's effectively just a pointer to a hash, and ends up being useless for versioning + a really nice footgun for tracking upstream updates.

The monorepo vs manyrepo tradeoff boils down to this:

Do you want more complicated build + deploy tooling or do you want more complicated dependency management?

If the former, pick monorepo. If the latter, pick manyrepo.

For example, the article states:

> [In the other direction,] Forcing dependees to update is actually another benefit of a monorepo.

What happens when the other teams that depend on your work don’t have the time/priority to update their code to work with the new version of your code? The ideal case that monorepo proponents tout is that the team updating the code that is depended on can just update the code for everyone who depends on them… however, that update is not always trivial and might require rework for code deeper inside the other teams projects. Maybe they are depending on functionality that is going away and it requires major work to change to the new functionality, and the team is working on other high priority things and can’t spend the time right now to update their code.

What does the team do? Do they wait until every team who depends on them is ready to work on updating? Do they try to work out how the depending team is using their code so they can update it themselves? How does this work if there are dozens of teams that use the dependency? You cant have every team that creates core shared code be experts on every other team’s work. You can end up stuck waiting for every team to be ready to work on updating this dependency.

Imagine if this was how all dependencies in your code worked, and every build task used the latest release of every dependency regardless of major version bumps. You might wake up on a Tuesday and your build fails and now you have to spend a week updating your project to use the latest version. Multiply this by all the dependencies and your priority list is no longer your own, you are forced to spend your time fixing dependencies.

This is why we specify versions in our dependencies, so we can update on our own schedule.

Of course, the downside of this is now you have to support multiple versions of your code, which is the trade off and the problem a monorepo solves.

You are going to end up with downsides either way, the question is which is worse.

The problem is with your deploy system. You can consider each of the clusters to be a service. Thus, a change in Service A (cluster A), should not trigger a deployment of Service B (cluster B).

My pipeline is split in 2:

1. on bitbucket, we run a pipeline that builds "build artefacts", docker images and "packaged" cloudformation templates.

Each of these artefacts has a list of triggers, either base docker images or source code. I'm building the relevant docker image or cf package based on the triggers (it's quite a naïve glob() use).

2. On aws side, I have something I call AWS Apps, in short a Stack Name, along with a set of triggers (the above build artefacts). On merge to main, I only deploy the AWS Apps affected by new build artefacts.

It's not ideal but it was handy to have access to all the cloud and application code on the embedded side for stuff like interface definitions for communication protocols and stuff like that. On the same company I worked on another project where the definition files for the cloud interface where on a different repo we had to use a submodule and I preferred the monorepo.

As other commenter said, we used bazel and there was indeed a smallish team that gave build support. Ramping up new hires on the build system was one of the more painful processes, I had to give support myself to teammates that had like a six month tenure only that they hadn't been there from day one.

Instead, everything is based around the idea that you check out the state of the world at some commit, do your build, whatever validation you need, and send it to production. You do this pretty often, ideally multiple times a week. Very occasionally you have an emergency where you want prod + cherrypick, and you generally build tooling that allows saying "build at this commit, but also with these later specific commits merged in".

What monorepos I've seen rarely bother with tags in practice, in part because they rarely individually version libraries, but at a technical level you can do it in git, if you need to.

If I remember correctly, this is how you do it:

  1. Create a new empty repo for the monorepo
  2. For each repo, 'git mv' all of the contents into a new directory with the repo's name
  3. Add the repos to the mono repos as remotes
  4. Run 'git merge --allow-unrelated-histories' for each repo

https://github.com/josh-project/josh

It's designed for making multiple Git repos from a monorepo, but I think you should be able to make a skeleton repo that represents your desired final monorepo layout and push your individual repos to the Josh subviews of that repo to combine them all.

(A big advantage of this approach over the multiple unrelated histories is that you don't have the mass move commits since Josh will rewrite all history as if the files were always in that folder, so you don't have to worry about history of individual files getting broken.)

I did a quick google and these instructions seem about right (without the delete step): https://gist.github.com/msrose/2feacb303035d11d2d05