Why Shopify Rejected Microservices (And What They Did Instead)

Comments · 133

• MI

  @MilanJovanovicTech · 5 months ago (edited) · pinned

  Check out Modular Monolith Architecture: <a href="https://www.milanjovanovic.tech/modular-monolith-architecture">https://www.milanjovanovic.tech/modular-monolith-architecture</a><br>Get the source code for this video here → <a href="https://the-dotnet-weekly.kit.com/shopify-modulith">https://the-dotnet-weekly.kit.com/shopify-modulith</a>

  5

• GG

  @ggwp8618 · 5 months ago

  More videos on Big tech architecture please

  43

• DR

  @Drysart · 5 months ago (edited)

  We (a large real estate transaction platform that will go unnamed) use a modular monolith approach where we separate modules out into separate assemblies (each module consists of an interface assembly and one or more implementation assemblies; and any module&apos;s implementation assembly can reference any other module&apos;s interface assembly). &nbsp; Then our build process builds everything, all modules, into a single container image. &nbsp; And then our host startup process can pull in configuration to specify which specific modules should be loaded at startup time -- whether that&apos;s &quot;load all modules&quot; or &quot;load this specific subset of modules only&quot;.<br><br>It straddles the boundaries between running as a traditional monolith and running as micro services; because we <b>can</b> run the entire application all in one process by just configuring it to load all modules; but if one aspect of the application needs to be scaled out differently than the rest of the application, we can start up new containers that are configured to just load those modules we want to scale separately; and so those running containers don&apos;t pay any runtime costs (CPU/Memory/etc.) for the rest of the monolithic codebase, they only load the assemblies they need. &nbsp;Any modules a specific container configuration chose not to load just sit there untouched on disk as DLL files and cost nothing. &nbsp; Then our API gateway proxy can route requests for that subset of our application to the specialized container instances running just those modules.<br><br>Because of the clean separation between modules, this is easy to do; and it allows us operational flexibility as if they were microservices, but it avoids nearly all of the pains of actual microservices -- everything runs from the same build so nothing is ever out of date in the deployed service fleet; instead of &apos;cross microservice&apos; messaging, the code you need is going to be right there in the container with you so it&apos;s just a regular method call; and we still have only one codebase and one build, it&apos;s easy to do refactoring across modules with comprehensive unit testing that can reach over the whole system at once. &nbsp; It also means all of our serverless functions can also live in the same codebase and still have fast startups, and our scheduled jobs and background workers can too; and we have full flexibility of how and where those run.

  29

• MI

  @MilanJovanovicTech · 5 months ago

  More resources from Shopify:<br>- <a href="https://shopify.engineering/deconstructing-monolith-designing-software-maximizes-developer-productivity">https://shopify.engineering/deconstructing-monolith-designing-software-maximizes-developer-productivity</a><br>- <a href="https://shopify.engineering/shopify-monolith">https://shopify.engineering/shopify-monolith</a><br>- <a href="https://shopify.engineering/enforcing-modularity-rails-apps-packwerk">https://shopify.engineering/enforcing-modularity-rails-apps-packwerk</a><br>- <a href="https://shopify.engineering/a-packwerk-retrospective">https://shopify.engineering/a-packwerk-retrospective</a><br>- <a href="https://www.shopify.com/enterprise/blog/monolithic-to-microservices">https://www.shopify.com/enterprise/blog/monolithic-to-microservices</a><br>- <a href="https://shopify.engineering/e-commerce-at-scale-inside-shopifys-tech-stack">https://shopify.engineering/e-commerce-at-scale-inside-shopifys-tech-stack</a>

  6

• TO

  @tonym5857 · 5 months ago

  Thank you, Great video 💪💯👌

• МА

  @МаксГаврилюк-й3х · 5 months ago

  Thanks a lot Milan. I&apos;d like to see more videos like this

  2

• JA

  @jackoflacko69 · 5 months ago

  So like a Django&apos;s `startapp` command

  1

• ON

  @onurakmese2129 · 5 months ago

  You took maybe two to three days to make this videos, and share all your knowledge with us. Thanks !

  1

• AA

  @aakashpoojary3968 · 5 months ago

  I think database section of the refactoring is where the actual pain is in modular monolith. When we start with a monolith it&apos;s easy to get started and establish pk to fk relationships in the database. But let&apos;s say, we start with a simple schema as a service segregation to establish boundaries and we try to influence it into modules the efcore context will also be wildly different and this will automatically impact the way code is written. And, also database design will change. I think api refactoring from monolith to modular is pretty doable if we have defined boundaries from the backend.

  12

• AN

  @antonmartyniuk · 5 months ago

  Interesting: what infrastructure does Shopify run their modular monolith on, and how many API Gateway instances and modular monolith instances are there?

  1

• MR

  @mreshboboyev · 5 months ago

  THANKS FOR SHARING

• RA

  @Ramblewithmme · 5 months ago

  Thanks a lot Milan for giving this real world example this gives us confidence to think of Modular monolith for larger products.