How I migrated a multi-million dollar business to a Monorepo at scale with Turborepo

“We can only see a short distance ahead, but we can see plenty there that needs to be done.” — Alan Turing

Hello, world! 🥳

Architecting software is hard.

It’s the first baby step, the first very crucial baby step of many in your apps journey that will influence how well it can walk, talk, run, swim, and interact with the outside world. It’s one of the most important pieces of project inception, you are about to birth something that has a UI, server, and a database which will supply value to people in their everyday lives, crazy right??

Your entire team will learn from it the more it grows whether you’re the engineer, PM, DM, QA, or even the end user. Through project maturity you’ll become more and more attached to it such that you’ll want it to be the best 3 Michelin star meal you’ve ever cooked up from your IDE 🧑‍🍳.

From the moment that first step is taken it’s the decisions you make from that point which will decisively shape how fast you can move and ship, the quality of code produced, the tech utilised, how often business requirements are met, your developer experience, your mental health.

Getting this wrong means you’re either stuck with it or have to eventually put together a use case for a rebuild when it’s too late. How about you go one step further and get it somewhat right? Such that you could build on top of your solution, turning it into a quality of life net positive for you, your team, your users, and everyone involved.

In this article I’m going to share my experiences, learnings, and benefits migrating two large frontend and backend TypeScript codebases I built for one of our biggest clients to a monorepo, as well as providing justifications as to why such a decision was made late in the projects lifecycle (and spoiler this certainly didn’t happen on my first baby step — but over a year into shipping).

Mistakes and failures are inherent in our everyday lives, necessary for growth — a principle that extends to the software we build. So without further ado lets get into it! First though let’s get into the “why?” behind all this.

“Who’s gonna carry the boats?” — David Goggins

Why? 🧐

bleeding edge
“the very forefront of technological development.”
“an architecture that many people believe is still too bleeding edge for large mission-critical systems”

“Why?” is a great question, “don’t fix something that isn’t broken” right? It’s a fine philosophy to live by when building software, I mean I already have a solution that works so why bother improving it? The problem with this though is it allows us to remain too comfortable in our development practices as engineers. Remaining lazy or complacent in anything simply isn’t acceptable unless you’re fine with living a life of mediocrity (I realise that sounded way too edgy for a tech article on Medium as I’m writing this, but here we are and I’m absolutely rolling with it 😂).

This isn’t something our customer asked for, it’s not like I care about how my Amazon prime orders are packaged and sent to me. I care about what’s inside, the things I can see, touch, and feel — the product. I migrated and made this crazy decision in my own work time off my own back (with the help of our talented in house DevOps team for CI/CD 🤩), alongside shipping project work in our legacy repo, so our end user value funnel was still being drip-fed as we were incrementally migrating in the background.

What isn’t spoken about enough in the industry is a lot of the time if you can’t properly justify technical decisions that don’t impact the end user of your product it can become increasingly difficult to get sign off for it, especially if it isn’t something they are actually using or gain direct user-facing benefit from like a feature — it’s up to you and you only to make the right people understand the value propositions behind your technical direction and garner buy in from the whole team to support you in executing your vision, a great team and culture were invaluable to our successes here.

As engineers we have a duty of care to the products we build, and where we can, if stable enough, should be looking at ways in which we’re always on the ✨bleeding edge✨ within reason so we can benefit from the amazing things talented engineers and companies all over the world dedicate their livelihoods or spare time too in open source, which in turn makes our working lives vastly more easier and satisfying to be apart of as a result.

Leveraging new tools in the ecosystem that can grant us perf gains, bolster our developer experiences, and ultimately allow us to ship our digital experiences to users faster at an even higher quality than before is something I believe every passionate engineer should be holding themselves accountable for. And that’s precisely the motivation behind the migration — to move fast, ship higher quality, and improve DX.

“The soul becomes dyed with the colour of its thoughts.“— Marcus Aurelius

A quick lesson 🧑🏻‍🏫

I think it’s important to understand what a polyrepo and monorepo is first before diving into the deep end 🏊🏻.

“A polyrepo is multiple codebases which are published and versioned separately.”

Basically it’s just multiple codebases, these codebases exist independently of one another and are contributed too separately typically toward the same app. Think of it as being single purpose to a degree (yep, I referenced “S” in SOLID principles get me on a TedTalk! 😜) such that it serves one purpose in its own little world.

A good example of this is say you have a web frontend app and api backend app in two separate repositories like the below:

web (repo 1)
├─ package.json
api (repo 2)
├─ package.json

This architectural software methodology is called a polyrepo because these apps live in two different houses and require separate contributions when making full-stack changes, which can become painfully obnoxious the more your app grows in scale.

A monorepo on the other hand is classified as “one” codebase, which you guessed it, houses both repos in one place allowing you to contribute once when making changes to both parts of your app.

“A monorepo is a collection of many different apps and packages in a single codebase.”

Here’s how the above example looks when migrated over to a monorepo:

my-supercool-monorepo
├─ apps
│  ├─ web
│  │  └─ package.json
│  ├─ api
│  │  └─ package.json
└─ package.json

When apps or packages are inside a monorepo, here in the JS ecosystem we call these “workspaces”, in the context of this example apps/web is its own workspace and apps/api is another.

“Workspaces are the building blocks of your monorepo. Each app and package you add to your monorepo will be inside its own workspace.”

As you can see, both apps now exist under one roof and can be contributed too once with one git remote, you can share reusable packages across both, you can streamline your development and build workflows, it has all the bells and whistles! 😙

That sounds cool and all, but how on earth did I start? Surely there isn’t something with a sophisticated enough build system and orchestration process already out there that lends a helping hand in all this? Well it didn’t take long to uncover that Vercel offers an extremely powerful bit of kit called Turborepo, and a powerful piece of kit it is.

“I know what it’s like to lose. To feel so desperately that you’re right, yet to fail nonetheless.” — Thanos

Going deeper 🧠

Choosing Turbo as a solution was a no-brainer as it solves all of the problems we’ve been facing throughout build, it also gave me a lot of confidence as it’s owned by Vercel and is used by some of the largest companies in the world such as Disney, Netflix, Microsoft, AWS, Github, and more. In this section lets deep dive into the specifics of the problems we were facing prior to migration and how leveraging Turborepo & Turbo aids us in solving them. (The greater the number of chillies = the spicier the topic, thanks Rich Harris 😁):

1. Globally trackable changes, one repo to rule them all 🌶
2. Shipping faster by optimising build times 🌶🌶
3. Evolving DX and code quality through end to end type safety 🌶🌶🌶

Problem 1 —No one can be bothered 🌶

web (repo 1)
git checkout -b the-frontend-part
*do some stuff*
git status
git add .
git commit -m "change"
git push

api (repo 2)
git checkout -b the-backend-part
*do some stuff*
git status
git add .
git commit -m "change"
git push

As outlined making changes in two separate repositories is painful when contributing to the same app. It’s not the core reason why we migrated but it helps boat loads and ensures consistency is met when building features, fixing bugs, or even making amends to both parts of our app through atomic commits — much like a lot of meta-frameworks today. Our legacy setup was one VSCode workspace where both codebases were housed inside and contributions were made separately depending on what part of the app was being touched.

There have certainly been times we’ve pushed a feature to our QA environment but we’ve only pushed the frontend for it, and forgot about pushing the backend endpoint resulting in our QA team being like “where the fudge is it though… and why is there error toasts everywhere? 🍞 REJECTED! 😏”.

I shouldn’t have to do mental gymnastics to remember if we’ve pushed our code twice for the same app, respectfully it should just do it once, granted it’s not super inertia inducing at all but it’s an extra plate to spin on top of everything else that we’d rather do without.

Here’s how I did this with our project whilst integrating it with Turborepo at a super high-level, I did this from the CLI at my Mac users root to make things easier when it comes to copying workspaces over from our legacy repositories 🙌.

Plugging into Turborepo 🔌

• Install the turbo CLI 🤓

npm install turbo-cli -g

• Init a turborepo project, give your app a swanky name and choose your dependency manager 🐣

npx create-turbo@latest

• Open up your monorepo in VSCode or whatever IDE you’re comfortable with 🛏

code my-supercool-monorepo

• Strip out the boilerplate it comes with, by default Turborepo ships with workspaces that won’t be relevant to us as we’re migrating over from an existing app rather than starting from scratch ❌

cd my-supercool-monorepo
rm -rf apps/web
rm -rf apps/docs
rm -rf packages/ui
etc..

• Recursively copy your apps into the apps directory of your monorepo, this will copy everything inside that app over to the monorepo 🍃

cp -r ../web apps/web
cp -r ../api apps/api

• Get rid of your .lock files at your existing workspaces root, your dependency snapshot no longer lives in the context of each app but will now live at your monorepo root for all of your workspaces 🗑

rm apps/web/package.lock
rm apps/api/package.lock

• Remove existing git remotes from your existing repositories, this will unlink your repos git remotes so you can start tracking changes globally at the monorepo root for all your workspaces 📍

rm -rf apps/web/.git
rm -rf apps/api/.git

• Run an install when it’s all done, as we’ve added new workspaces the monorepo needs to recognise the newly added dependencies, do dependency hoisting etc ⬇️

npm/yarn/pnpm install

• Run turbo dev to spin up your local dev environment 🔄

turbo dev

And that’s it! Provided you’ve set up another git remote globally at your root you can start pushing changes and begin contributing to your app once, and you’re ready to start leveraging the great features Turborepo can offer.

Problem 2 — It’s taking how long to build?! 🌶🌶

One problem that didn’t seem to sit well with me was build times, these are two independent TypeScript codebases; one rolling a CRA (create-react-app 🫠) *cough pending vite move* as an SPA and the other rolling Nest for our API, for large or small contributions we’d see build times up to 15 minutes for each.

Not only is it building everything from scratch each and every time and outputting those newly generated artefacts to a dist (for the backend) or build folder (for the frontend), but any new add-ons to either part of the app bloats that build time quite significantly as bundle sizes increase, the more dependencies/modules are added, all that jazz 🎷 and we don’t want our users waiting this long for new changes, the aim is to move fast here.

The most attractive part about Tuborepo is this concept of “remote caching”, this allows us to build our apps marginally faster than ever before, it does this by caching our build runs on a remote server (or the local filesystem if not linked to a provider) and only builds the things we care about so it never has to run the same task twice. Here’s a pretty picture straight from the Vercel docs that showcases the power.

So say for example we’ve already built our app fresh for the first time with remote caching enabled so we have our workspaces cached at this point, but we want to update our favicon to be a zombie cause it’s halloween 🧟‍♀️, it will only bother building that part of the app whilst keeping all existing workspaces built resulting in literally mere second build times — whether you’re in CI or local it’s all a shared cache from that remote server. Your build run will miss parts of the app that don’t need built anymore and only target and care about our zombie that has been added, it’s bloody magic (pardon my french 🥖).

As you can see, with remote caching enabled we’ve now decreased build times by roughly 90% which is humungous compared to before!

“By working with providers like Vercel, Turborepo can securely communicate with a remote cache — a cloud server that stores the results of your tasks.”

“This can save enormous amounts of time by preventing duplicated work across your entire organisation.”

Turborepo comes shipped with a turbo.json and a package.json file at your monorepo root, I’m not going to go into the specifics here about what they do (Vercel does a much better job than me, see here) but all you need to know is it’s these two files that are the most important to your development and build orchestrations.

As our api and web apps are both JS based they have their own package.json’s housing their dependencies, scripts and so on, the magic of the Turbo CLI is if the name of your scripts in these workspaces match the name of the scripts in your root monorepo it will run those scripts for all of them. As you can imagine, this is particularly useful for dev where we can run one command and have a development server up and running alongside any dependent workspaces, or build scripts which build your apps and output the artefacts just as they normally would.

And that’s the beauty of Turbo is it makes it so if you are migrating then a lot of the configuration is more or less setup because you’ve done these steps before when setting up initial project bootstraps, equally our CI/CD lives in Azure DevOps but we’re still able to leverage remote caching with Turbo outside of the Vercel ecosystem.

Running turbo dev or turbo build from root you’ll be able to see that it is able to figure out which scripts it needs to run and will run them all for you in whatever order you want.

Problem 3 — Achieving end to end type-safety 🌶🌶🌶

Type safety

“In computer science, type safety and type soundness are the extent to which a programming language discourages or prevents type errors.”

We’re finally at the juicy part! The crème de la crème, thank you for making it this far here’s a glass of milk for all your troubles! 🥛 We’re almost done don’t worry, this is one of the core reasons behind all of this madness — achieving end to end type safety.

I was desperate for a way to tightly couple the relationship between our client and server even further to achieve pure type soundness, granted I already implemented an “okayyy” solution in our original approach, but it involved a lot of zod schema definitions on the client, and a hook to go with it, and a axios/fetch call to go with that, and manually configuring our Swagger API docs… AHHHH.

It just became obnoxious the circle of hell I’d need to go through on the client side every time I needed to spin up a new feature — and for an app that has 50+ GET endpoints with more being built each day as it’s a very read-heavy application, it got to a point of pure dread each time I had to build a new feature knowing those things have got to be setup prior if I want to gracefully handle errors at runtime, have a better developer experience, or prevent bugs from bubbling to the surface if my async calls weren’t typed etc.

Lets transcend together and see how I went about achieving this! 😁

“What if changing the world was just about being here, by showing up no matter how many times we get told we don’t belong, by staying true even when we’re shamed into being false.” — Mr. Robot

Getting even juicer 🥛

So far I’ve outlined the 3 problems I’ve solved, but figured the end to end type safe part really needs it’s own section as there’s some interesting stuff in here.

Remote procedure call

“In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is written as if it were a normal (local) procedure call, without the programmer explicitly writing the details for the remote interaction.”

RPC tech is all the rage right now in full-stack especially for TypeScript apps, largely due to popular libraries paving the way for it such as tRPC and our favourite chap Theo Browne. GraphQL is still very popular but is certainly not something I’d reach for or entertain unless it was really domain specific, and REST is the de-facto standard when building anything usually.

But I already have a RESTful API, and I want to leverage some form of RPC solution so I don’t have to go to the pearly gates and back if I want to render information about the user in the browser. I mentioned earlier that the backend is built in Nest.js so tRPC is out of the question as there isn’t a happy path available just yet unless you hack your way around adaptors and whatnot 🥺.

So I needed to find something that has a RESTful first approach to API development but also allows me to get the greatness of a type safe server and client out the box — how did I do that? Well I didn’t but the folks over at @ts-rest certainly did!

Here’s how the library works visually (s/o excalidraw):

You firstly define what’s called a contract this contract leverages the beauty of zod by allowing us to define a type-safe schema that both our frontend and backend must respect. Think of it as like a tenancy agreement, where the tenancy agreement is the legal binding between the tenant and the landlord which states “I will for x amount of time pay my rent on time, look after the furnishings, and not burn the house down”.

As engineers we can be thought of as the landlord which ensures the client and server “tenants” aren’t being pests and are respecting the terms defined in the contract, so that everything functions as seamlessly as possible day in day out; preventing water leaks or unblocking drains that could cause harm to the property, the property being the app! (I’m super proud of this analogy btw 😅).

Much like a tenancy agreement, it must be shared between all occupants of the property in order for everyone to have context of it. So our contract needs to be shared between the client and the server so they both have an understanding of one another, luckily leveraging Turborepo we can do just that by utilising the internal packages system it offers!

“Internal packages are packages which are only intended to be used inside your monorepo. They’re extremely useful for sharing code between apps in closed-source monorepos.”

Setting up the contract 📑

Turborepo ships with packages for this reason, I first created a contract directory inside it so the repo now looks like this:

my-supercool-monorepo
├─ apps
│  ├─ web
│  │  └─ package.json
│  ├─ api
│  │  └─ package.json
├─ packages
│  ├─ contract
│  │  └─ package.json
└─ package.json

I added a simple package.json inside with dependencies the package relies on, along with dev and build scripts to transpile everything in the contract during development or when building using tsup as we want our contract to be using TypeScript:

{
  "name": "contract",
  "version": "0.0.0",
  "main": "./dist/index.js",
  "module": "./dist/index.mjs",
  "types": "./dist/index.d.ts",
  "scripts": {
    "dev": "npm run build -- --watch",
    "build": "tsup src/** --format cjs,esm --dts"
  },
  "dependencies": {
    "@ts-rest/core": "^3.30.5",
    "tsup": "^8.0.1",
    "typescript": "latest",
    "zod": "^3.22.4"
  }
}

Then I created a src directory where the contract will live, alongside a simple index.ts which is where the root contract router will exist, here’s what a really simple contract looks like which defines fetching a post by its id:

/* imports here */

const PostSchema = z.object({
  id: z.string(),
  title: z.string(),
  body: z.string(),
});

export const contract = c.router({
/* 
   all other endpoints will exist here, you can also have multiple routers 
   depending on your API resources
*/
  getPost: {
    method: 'GET', // HTTP verb
    path: `/posts/:id`, // endpoint URI
    pathParams: z.object({ // path params
      id: z.string(),
    }),
    responses: { // the responses we expect
      200: PostSchema.nullable(),
    },
    summary: 'Get a post by id', // a short summary of what it does
  },
});

Speaking to the contract 🗣

Now we have a simple contract setup, we can start connecting things! In order for our frontend and backend to speak to the contract it’s super straight forward with Turborepo and is agnostic regardless of the internal packages you want to share — in each apps package.json add the below to it. After a fresh install you’ll be able to import the contract across your workspaces where this has been defined.

{
  "dependencies": {
    /* other dependencies here */
    "contract": "*"
  }
}

The Nest.js backend part 📡

We can now import the contract into our backend Nest controllers, here’s what the endpoint looks like with the given post example:

@TsRestHandler(contract.getPost)
  async getPost() {
    return tsRestHandler(contract.getPost, async ({ params }) => {
      const { id } = params;
      const data = await this.postService.getPost(id);

      return data;
    });
  }

Gorgeous! 😍 The @TsRestHandler decorator will do all the swagger config for us and effectively will tell Nest everything it needs to generate that for us for that endpoint. And the tsRestHandler callback ensures we have a 1–1 relationship between the contract and controller, which is great for our incremental migration method as we can keep our existing endpoint controller functions unchanged whilst newly additive endpoints can be built this way resulting in 0 breaking changes to the API 💅.

Immediately, if I attempt to return something that doesn’t match the response from getPost the TypeScript compiler will yell at us and let us know that we expect a return type of x but we’re giving it y, it even knows the type of our path or query params, holding us accountable by automatically inferring our zod schema in the contract — you silly goose! 🦢

The React.js frontend part ⚛️

Now then, at the time of writing @ts-rest/react-query currently doesn’t support the latest v5 syntax but it does have support for v4! That’s great for existing react query v4 users who haven’t upgraded, however the problem we faced is that because we’re already rolling v5 and did a lot of work to migrate to v5 with the help of RQ’s codemod and manual refactoring, I simply don’t see the point of downgrading a very important library just to leverage the benefits of this.

As the jump from v4 to v5 of react query included breaking changes and specifically syntax changes where the entire way we instantiate useQuery hooks is completely different from array to object syntax, the refactor to downgrade seemed obnoxious and really not needed as react query is used literally everywhere across the app. Plus as the team and open source contributors at @ts-rest are working super hard getting this built in the future it’s just currently not there yet for v5 at the time of writing, but I’m super hopeful for the future!

As a workaround, instead of using their thin react query wrapper I use their client library directly in the useQuery function (queryFn) itself just as you would when using normal out the box react query. Allowing me to leverage the power of react query but also an end to end type safe RPC pattern too 😁.

This is how simple it is using our post example:

/* init your api client */

export const api = initClient(contract, {
  baseHeaders: {},
  baseUrl: 'http://localhost:3200', // should be dyamic coming from your .env vars
  /* you can create an api callback here to add auth headers etc. */
});

/* create a hook or use useQuery in a component */

export default function usePost(id: string) {
  return useQuery({
    queryKey: ['post', id],
    queryFn: async () => {
      /* 
         call getPost from our contract 
         everything below is automatically inferred from the contract
      */
      const post = await api.getPost({ params: { id } })

      if (post.status === 200) {
        return post.body
      }
    },
    enabled: !!id,
  })
}

/* using the hook */

const { data } = usePost(id)

No more fetch/axios layer, manually configuring swagger docs, or zod on the client, and we can leverage react query alongside it.

Wrapping up 🎁

You made it to the end and I really appreciate you! 💜 we’ve covered a lot, I truly believe for companies which are shipping product that going down the monorepo route has a slew of benefits for your organisation if your stack marries. Couple that with an end to end type safe system with some form of RPC-like technology and Turborepo you can get some really outstanding DX gains that bolster your engineers code quality & productivity through the roof when shipping in full-stack.

I hope you got some value out of this whether that’s some new bits of knowledge, or I’ve changed your life, or even a few laughs! This is my first take at technical writing that I’ve been wanting to start doing for ages so appreciate any feedback on this, feel free to give me a 👏 if you want, or don’t it’s completely up to you! Thanks for reading. 😊