Server Implementation Plan For Rocicorp's Zero - Custom Mutators

Published on 2025-08-10

It's been a minute since I wrote anything about Zero but I finally got back to working on my project that is using it. A lot changed between the time I took a break, where I was on the 0.11 release, to starting back up and updating to the 0.20 release. In the middle of those was the long awaited release of 0.18 - Custom Mutators. I encourage reading through the Custom Mutators Documentation to get acquainted with the details but if this blog post is has caught your attention then I'm guessing you're already well versed on what Custom Mutators are.

After I finished my upgrade to 0.20 I started investigating how I could use Custom Mutators. The Zero team has made it extremely easy to use Custom Mutators if the project is fullstack Typescript, however if the project is using a different language for the server code there is some work to be done. My project is using Ruby on Rails so I started my adventure on figuring out how to implement Custom Mutators on my own; the aforementioned documentation states adventure is doable and even gives a small example in Go.

The server implementation runs on your server, in your push endpoint, against your database. In principle, it can be written in any language and use any data access library. For example you could have the following Go-based server implementation of the same mutator

While my implementation is in Ruby on Rails, I wanted to share what I learned while doing my implementation in a more language agnostic way. I'm positive the Zero team will have their own write-up or maybe even SDKs at some point. Until then, I hope this helps anyone else who is excited about Custom Mutators and isn't building a fullstack Typescript app. What follows is an explanation, with psuedo-code examples, on how to implement Custom Mutators. Let's dive in!

High Level Abstract#

At a high level the flow of the Custom Mutators server implementation is going to start with receiving an HTTP request with a list of mutations that need to be run on our server. There are a handful of steps we'll need to go through to properly process them so let's turn those steps down into a list.

  • HTTP request comes in with mutations
    • Validate the request
  • Process the list of mutations and capture the results
    • For each mutation in the list
      • Check if it's the next expected mutation ID for this client
      • If out of order, stop and return error (client will retry)
      • If duplicate, skip it
      • Otherwise, run a mutator and write data to the database
      • Update the mutation tracking
  • Return all the results

We're going to break each of these steps out into their own section in this post in an order that I hope makes logical sense from an implementation standpoint. Remember that there isn't any real code being written here but we'll jot down some pseudocode to solidify the concepts.

HTTP Request and Endpoint#

The start of this implementation is to add an endpoint to our server which will receive and process requests sent from Zero. This endpoint needs to be set up to receive a POST request and respond back with a payload indicating which mutations were successfully applied and which mutations failed to be applied.

PushRequest schema

PushRequest:
  - clientGroupID: number
  - mutations: Array<Mutation>
  - pushVersion: number // Will be 1
  - timestamp: number
  - requestID: number

// Each mutation looks like:
Mutation:
  - type: string // Will be "custom"
  - id: number // sequential number (1, 2, 3, 4...)
  - clientID: number
  - name: string // Will be "namespace|mutatorName" like "book|update"
  - args: Array<Record<string, any>>
  - timestamp: number
PushRequest:
  - clientGroupID: number
  - mutations: Array<Mutation>
  - pushVersion: number // Will be 1
  - timestamp: number
  - requestID: number

// Each mutation looks like:
Mutation:
  - type: string // Will be "custom"
  - id: number // sequential number (1, 2, 3, 4...)
  - clientID: number
  - name: string // Will be "namespace|mutatorName" like "book|update"
  - args: Array<Record<string, any>>
  - timestamp: number

Here's a real world example from my implementation to help solidify the structure.

{
  clientGroupID: "9au3tfpauegocajuba",
  timestamp: 1753139962914,
  mutations: [
    {
      type: "custom",
      id: => 2,
      clientID: "gjjtc2mm95ofvleq33",
      name: "book|update",
      timestamp: 1753139962891,
      args: [ { id: 2040, name: "Updated Book Name" } ]
    }
  ],
  pushVersion: 1,
  requestID: "gjjtc2mm95ofvleq33-cb52fe85-9",
  schema: "zero_0",
  appID: "zero"
}
{
  clientGroupID: "9au3tfpauegocajuba",
  timestamp: 1753139962914,
  mutations: [
    {
      type: "custom",
      id: => 2,
      clientID: "gjjtc2mm95ofvleq33",
      name: "book|update",
      timestamp: 1753139962891,
      args: [ { id: 2040, name: "Updated Book Name" } ]
    }
  ],
  pushVersion: 1,
  requestID: "gjjtc2mm95ofvleq33-cb52fe85-9",
  schema: "zero_0",
  appID: "zero"
}

Additional PushRequest Parameters

One last note on the PushRequest schema is there is functionality built in to pass any additional parameters from the client to server.

const z = new Zero({
  push: {
    queryParams: {
      workspaceID: "42",
    },
  },
});
const z = new Zero({
  push: {
    queryParams: {
      workspaceID: "42",
    },
  },
});

When configuring Zero on the client, we can add in the push key and nest the additional parameters under the queryParams key. These parameters will show up as top level keys so the structure from above would have the workspaceID appended to the end like so.

{
  ... // All the other parameters
  requestID: "gjjtc2mm95ofvleq33-cb52fe85-9",
  schema: "zero_0",
  appID: "zero",
  workspaceID: 42
}
{
  ... // All the other parameters
  requestID: "gjjtc2mm95ofvleq33-cb52fe85-9",
  schema: "zero_0",
  appID: "zero",
  workspaceID: 42
}

HTTP Endpoint

How the endpoint is implemented is going to be vastly different depending on which framework or library is being used. The single requirement for the endpoint is that it accepts a POST request. I set up my route to be POST zero/mutations/push however it does not have to be that nested.

HTTP Endpoint Validation

There are a handful of ways that we can achieve validating the requests that we're receiving from Zero. Let's start off with the AUTHORIZATION header. Zero includes the JWT that we're using on the client in these requests so the first step in validating should be to decode the JWT and ensure that it is valid. After decoding it, we'll have access to any data that was originally encoded, such as a user ID, so we'll know who performed the mutation on the client. We'll want to use that user information to ensure relational integrity of the data being mutated.

The next step in validation is optional, but I think it's an important one as well. In the zero-cache Config Documentation there is an option to set ZERO_PUSH_API_KEY which is a secret key that will be included in the X-API-KEY header. We can include that secret as an environment variable in our server and check the header against the environment variable to ensure the request is coming from our own Zero instance.

PushResponse Schema

The endpoint needs to respond with a 200 when processing is successful or with a 401 or 403 if you would like Zero to re-authenticate and then try the mutations again. Any other response code returned and Zero will wait for 5 seconds and then re-send the request. In the case where everything was successful, we need to return a response structure that matches the PushResponse schema.

PushResponse:
  * mutations: Array<Mutation>

// Each mutation looks like:
Mutation:
  * id: MutationId
  * result: MutationResult

MutationId:
  * clientID: string
  * id: string

// Return an empty object ({}) if the result is valid
MutationResult:
  * details?: string
  * error: "oooMutation" | "alreadyProcessed"
PushResponse:
  * mutations: Array<Mutation>

// Each mutation looks like:
Mutation:
  * id: MutationId
  * result: MutationResult

MutationId:
  * clientID: string
  * id: string

// Return an empty object ({}) if the result is valid
MutationResult:
  * details?: string
  * error: "oooMutation" | "alreadyProcessed"

Here are some real world examples from my implementation to help solidify the structure. First is a successful result, second is a result with errors where the details is an error message I came up with.

{
  mutations: [
    { id: { clientID: "gjjtc2mm95ofvleq33", id: 1 }, result: {} },
    { id: { clientID: "gjjtc2mm95ofvleq33", id: 2 }, result: {} }
  ]
}
{
  mutations: [
    { id: { clientID: "gjjtc2mm95ofvleq33", id: 1 }, result: {} },
    { id: { clientID: "gjjtc2mm95ofvleq33", id: 2 }, result: {} }
  ]
}
{
  mutations: [
   {
     id: { clientID: "gjjtc2mm95ofvleq33", id: 3 },
     result: {
       details: "Client gjjtc2mm95ofvleq33 sent mutation ID 3 but expected 2",
       error: "oooMutation"
     }
   }
  ]
}
{
  mutations: [
   {
     id: { clientID: "gjjtc2mm95ofvleq33", id: 3 },
     result: {
       details: "Client gjjtc2mm95ofvleq33 sent mutation ID 3 but expected 2",
       error: "oooMutation"
     }
   }
  ]
}

Pseudocode Implementation

function handleHttpRequest(httpRequest):
  try:
    validateRequest(httpRequest)

    body = parseJSON(httpRequest.body)

    response = processAllMutations(body)

    return httpResponse(200, response)

  catch any error as e:
    return httpResponse(500, {error: "zeroPusher", details: e.message})
function handleHttpRequest(httpRequest):
  try:
    validateRequest(httpRequest)

    body = parseJSON(httpRequest.body)

    response = processAllMutations(body)

    return httpResponse(200, response)

  catch any error as e:
    return httpResponse(500, {error: "zeroPusher", details: e.message})

Processing Mutations#

Now that we've finished setting up our endpoint, the next step is to set up a loop to process all the mutations that are being sent to us. There isn't a lot to cover here except for one important piece. While we're looping and processing each individual mutation, if we ever run into an oooMutation error, we have to stop processing the entire batch and return the payload we've built up. Even if an oooMutation occurs, we still need to return the data as a 200 response.

Pseudocode Implementation

function processAllMutations(request):
  responses = []

  for each mutation in request.mutations:
    response = processOneMutation(mutation)
    responses.add(response)

    // If we run into a mutation that has an oooMutation error,
    // we have to stop processing and send the result back to Zero.
    if response.result.error == "oooMutation":
      break

  return {mutations: responses}
function processAllMutations(request):
  responses = []

  for each mutation in request.mutations:
    response = processOneMutation(mutation)
    responses.add(response)

    // If we run into a mutation that has an oooMutation error,
    // we have to stop processing and send the result back to Zero.
    if response.result.error == "oooMutation":
      break

  return {mutations: responses}

Processing a Single Mutation#

Inside of our loop, we have to process each individual mutation. To do so there are two steps we have to take. The first is to do a check that the mutation payload we received is properly ordered and if everything checks out, we'll then dispatch the mutation to apply the changes to the record in the database. In this individual processing we want to catch specific errors thrown, like the previously mentioned oooMutation error and construct a response object that the looping process can check against. If everything processes fine, then we only need to return an empty object.

Pseudocode Implementation

function processOneMutation(mutation):
  mutationInfo = {clientID: mutation.clientID, id: mutation.id}

  try:
    checkMutationOrder(mutation.clientID, mutation.id)

    dispatchToBusinessLogic(mutation)

    return {id: mutationInfo, result: {}}

  catch OutOfOrderError as e:
    return {id: mutationInfo, result: {error: "oooMutation", details: e.message}}

  catch AlreadyProcessedError as e:
    return {id: mutationInfo, result: {error: "alreadyProcessed", details: e.message}}

  catch any other error as e:
    return {id: mutationInfo, result: {error: "app", details: e.message}}
function processOneMutation(mutation):
  mutationInfo = {clientID: mutation.clientID, id: mutation.id}

  try:
    checkMutationOrder(mutation.clientID, mutation.id)

    dispatchToBusinessLogic(mutation)

    return {id: mutationInfo, result: {}}

  catch OutOfOrderError as e:
    return {id: mutationInfo, result: {error: "oooMutation", details: e.message}}

  catch AlreadyProcessedError as e:
    return {id: mutationInfo, result: {error: "alreadyProcessed", details: e.message}}

  catch any other error as e:
    return {id: mutationInfo, result: {error: "app", details: e.message}}

Validating Mutation Data#

To validate the order of the mutation, there are a few steps we need to go through. The first will be to create a database table which can hold these mutation records. We want to be able to open a transaction and atomically do the required checks; if anything fails we'll throw an error and exit the transactions without applying any changes.

The checks that we're required to do are scoped to the clientID and the first check is to see if we've already processed this mutation. This likely happens when an error occurs and Zero retries a batch of mutations; we don't want to re-apply any changes that we've already done so it makes sense to skip them! The other required check is looking for mutations that have come in out of order. The last thing we want is to apply changes in the wrong order and then our users end up seeing an incorrect value when the syncing is finished. If both checks pass then we go ahead and update the mutationID, save the record, and continue our processing.

Pseudocode Implementation

function checkMutationOrder(clientID, mutationId):
  // Start a database transaction - this must be atomic!
  database.transaction:

    // Find the client's current state
    record = database.findOrCreate("client_mutation", clientID)
    currentLast = record.last_mutation_id

    // Check if we already processed this
    if mutationId < currentLast:
      throw AlreadyProcessedError("already did mutation " + mutationId)

    // Check if we're missing some mutations
    if mutationId > currentLast + 1:
      throw OutOfOrderError("expected " + (currentLast + 1) + " but got " + mutationId)

    // All good! Update our tracking
    record.last_mutation_id = mutationId
    database.save(record)
function checkMutationOrder(clientID, mutationId):
  // Start a database transaction - this must be atomic!
  database.transaction:

    // Find the client's current state
    record = database.findOrCreate("client_mutation", clientID)
    currentLast = record.last_mutation_id

    // Check if we already processed this
    if mutationId < currentLast:
      throw AlreadyProcessedError("already did mutation " + mutationId)

    // Check if we're missing some mutations
    if mutationId > currentLast + 1:
      throw OutOfOrderError("expected " + (currentLast + 1) + " but got " + mutationId)

    // All good! Update our tracking
    record.last_mutation_id = mutationId
    database.save(record)

Dispatching to a Mutator#

Now that everything has been validated, we can dispatch the changes to a mutator function. Likely the easiest way to get started with this will be to create a switch statement with all the different mutators we implement. The name of the mutation will look like namespace|action and in the case of these examples it is book|update. We'll start off by using a switch statement and then all we need to do is pass in the mutation.args to our mutator.

Pseudocode Implementation

function dispatchToBusinessLogic(mutation):
  mutatorName = mutation.name
  args = mutation.args

  switch mutatorName:
    case "book|update":
      handleUserUpdate(args[0])
    default:
      throw error("unknown mutator: " + mutatorName)
function dispatchToBusinessLogic(mutation):
  mutatorName = mutation.name
  args = mutation.args

  switch mutatorName:
    case "book|update":
      handleUserUpdate(args[0])
    default:
      throw error("unknown mutator: " + mutatorName)

Running a Mutator#

We finally made it; we're ready to change some data in our database! We passed in args[0] which if we refer back the example payload was { id: 2040, name: "Updated Name" }. We'll want to use the information from the AUTHORIZATION header to ensure referential integrity between the object being mutated and the authorized user. If it's a valid relation, we'll apply our changes and then return a result indicating that everything worked.

One thing to remember is to run this whole block in a transaction to allow for rollbacks if any of the data is invalid. Having an open transaction also allows us to do any additional work we desire. The feature is called Custom Mutators for a reason!

Additional Mutator Actions

The additional work we want to do could be anything from sending emails to creating additional records on an audit log table or emitting metrics. Whatever additional work our application desires, we've made it to the point where all that can happen. If there isn't any additional work to be done, that's okay too. The option is always there in the future.

Pseudocode Implementation

function handleUserUpdate(attributes):
  // Get the user record from the AUTHORIZATION header, that may require passing
  // that data all the way
  user = database.find("user", userId)

  // Ensure the User has referential integrity to actually modify this object
  book = database.find("book", attributes["id"]).joins("users").where("users", "id", user.id)

  if book:
    database.transaction:
      book.update(attributes[:name])

      // Do any additional work desired
      database.create("audit_log", book, "updated", attributes["name"])

      return true
  else
    return false
function handleUserUpdate(attributes):
  // Get the user record from the AUTHORIZATION header, that may require passing
  // that data all the way
  user = database.find("user", userId)

  // Ensure the User has referential integrity to actually modify this object
  book = database.find("book", attributes["id"]).joins("users").where("users", "id", user.id)

  if book:
    database.transaction:
      book.update(attributes[:name])

      // Do any additional work desired
      database.create("audit_log", book, "updated", attributes["name"])

      return true
  else
    return false

Conclusion#

We made it through the whole implementation plan and I hope you're as excited about Custom Mutators as I am. If you write your own implementation in a different language be sure to share your learnings with the Zero community! I'll be following up this post with another post showing my Ruby on Rails implementation.

All My Posts About Zero