Battle scars from organising Firebase Cloud Functions

Photo by Inja Pavlić on Unsplash

We’ve been using firebase for nearly three years, we started small and have been gradually new building systems and migrating software from on-premise to the cloud. At the time of writing we have 105 cloud functions triggered by https, pubsub, c loud scheduler and firestore. The results are brilliant, we are able to do things we could never have imaged just 12 months ago. But……

We started to hit problems:

• with the deployments. We use GitHub as a source repository linked to GCP Cloud build. Our cloud build script deployed each part of our cloud functions in parallel. eg https, pubsub, firestore but would fail because of quota limits.
• with breaking deployments up. Knowing which parts of our functions to deploy was problematic, we could target individual functions, eg https-user and build chains of individual functions, but this is all very manually, not how we like to work.
• with knowing the state of the live environment. With a broken and unreliable deployment pipeline we lost control of what was in production. Individual engineers were having to manually deploy from their workstations.

So where did it go wrong?

In the beginning we started as I guess everybody does, with a few files and an index file. We soon outgrew this pattern and researched other solutions.

Tarik Huber’s article really helped shaped our thinking- https://codeburst.io/organizing-your-firebase-cloud-functions-67dc17b3b0da

And as highlighted above we organised our code by trigger type. This served us well and made use think about start up times and code with side effects on startup.

The naming of functions should indicate the trigger and not what they are doing.

But, there is a problem with this approach. We build a monolith, with dependancies shared between different triggers and soon it became difficult to make and get a small change deployed because it was impossible to know what other functions may have changed. Using Arkit we were able to visualise our code dependancies, check out this dependancy graph , it confirms want we knew— it’s a hideous web.

Time to rethink.

We had previously developed and deployed a microservices based system using an early version of kubernetes, it was hard work, intercommunication and tracing was difficult — this was pre Istio and the eco-system that now exists as well as the maturity of kubernetes. Serverless functions offered us a way to focus on code that did stuff without too much infrastructure scaffolding. In order to consolidate our infrastructure we migrated the microservices app to GCP functions. As we did this, we realised that we had lost the microservices encapsultation of services.

The challenge then became how to we take a microservices approach to cloud functions? As we tried to understand how we had got to the position we were in, we identified the root cause as organising the functions by their trigger. This fundamentally broke services apart based on coding/infrastructure needs, failing to respect the Single Responsibility Principle and the idiom that code that changes together belongs together. Uncle Bob has a great post on the subject here

Gather together the things that change for the same reasons. Separate those things that change for different reasons.

The importance of the SOLID principles came into focus and a rearchitecting of our work.

How it looks now.

We now organise by business feature — then trigger. Along the way we also learnt about how to better handle different triggers, for example:

• where no response is required, such as a firestore trigger, we publish to a pub-sub topic. This allows any other feature to comsume an event from another business feature — without being directly bound in at the code level.
• the main code for a feature is contained within a service, which provides all the logic feature but has no knowledge or understanding of triggers. This also makes it easier to write tests.
• the trigger code is very simple, previously a trigger might end up coordindating all sorts of updates, validations and changes. Now it calls into a service, ideally through a pubsub trigger.

Our repository is now structured like this:

functions/
 auth/
  index.ts
 users/
  index.ts
  firestore.ts  
  https.ts
  service.ts
  pubsub.ts
 messaging/
  index.ts
  https.ts
  service.ts
  pubsub.ts
 search/
  index.ts
  https.ts
  service.ts
  pubsub.ts
index.ts

Each folder represents a “microservice”, not quite at the same level that we had within kubernetes, but is much closer to the SOLID principles.

We can now deploy individual microservices.

firebase deploy --only functions:messaging

Health warning

We’re still working on this, for now it is working really well for us, we’ve only trialled this on a small application and have yet to prove it on some of our larger applications (where we’ve hit the problems described). Once we have we will update the article, in particular with a new dependancy visualisation.

We’ve decided to publish where we are because we’ve not been able to find any examples of other teams organising serverless functions this way, so thought it is worth opening a dialog. It would be great to hear feedback and thoughts from others.