Code as schema: how we generate the Increase API

The Increase API sits at the heart of our product and we want to make it great. For an API, greatness often lies in small details - are resources and fields named consistently? Does the API’s production behavior match its documentation? How helpful are the error messages? Are there language-specific SDKs available? Do they have strong types?

Holding a high quality bar as your API grows can be a ton of work, often requiring a dedicated team. That’s not how we like to work. We’ve instead invested lots of energy in an opinionated internal framework for API definition. It’s allowed a single engineer to own the developer experience (APIs, SDKs, documentation, etc) for all of Increase, even as our API has scaled to hundreds of endpoints. We think our framework is good and would like to extol its virtues to you.

Application code is the source of truth

Every API method implements a common interface and must define an HTTP verb, path, parameter shape, and response shape. Here’s a simplified Account API (these examples are in Ruby, but you can write something similar in any language):

class AccountResource < APISchema::AbstractResource
  string :id, documentation: "The Account identifier"
  string :name, documentation: "The name of the Account"
  enum :funding, documentation: "The type of the Account", cases: {
    deposits: "A depository account such as checking or savings",
    loan: "A loan account",
  }
  timestamp :created_at, documentation: "When the Account was created"
end

class AccountCreate < APISchema::AbstractAPIMethod
  signature do
    http_path '/accounts'
    http_verb :post
    documentation 'Create an Account'
    parameters do
      string :name, documentation: AccountResource.attributes.name.documentation
      enum :funding,
        documentation: AccountResource.attributes.funding.documentation,
        cases: AccountResource.attributes.funding.cases,
        optional_with_default: :deposits
    end
    returns AccountResource
  end

  def operate
    account = Model::Account.create!(name: params.name, funding: params.funding)
    AccountResource.new(
      id: account.id,
      created_at: account.created_at,
      name: account.name,
      funding: account.funding,
    )
  end
end

We try to create a “pit of success” with the framework, both in defining the API shape and then working with it:

Schema-definition methods are packed with assertions to enforce our API conventions. When possible these are checked at build time instead of runtime - not documenting your API method means it won’t typecheck. (Our Ruby code uses Sorbet for type checking, omitted here for clarity.)
The framework handles parsing the raw HTTP request into a strongly typed Request object.
We expose higher-level parameter and attribute types for common operations.

For example, if you now want to add a method to retrieve an Account, there’s a shorthand for parameters that are resource IDs. First, we associate AccountResource with the database model it should load automagically:

class AccountResource < APISchema::AbstractResource
  # ... attribute definitions from above
  associated_model Model::Account
end

Then the GET method looks like:

class AccountRetrieve < APISchema::AbstractAPIMethod
  signature do
    http_path '/accounts/{account_id}'
    http_verb :get
    documentation 'Retrieve an Account'
    parameters do
      resource_id :account_id,
        resource: AccountResource,
        documentation: "The ID of the Account to be retrieved"
    end
    returns AccountResource # It's nice to be able to reuse these across different methods
  end

  def operate
    account = params.account # the framework hands us a fully-hydrated Model::Account
    AccountResource.from_model(account) # The associated_model call gives us a convenience method here now
  end
end

Under the hood, resource_id safely loads the Model::Account with that ID and hydrates it into params.account. Our type system knows params.account is a Model::Account, params.funding is an enum with exactly those two cases, and AccountResource.new takes an object that implements the attributes declared above.

We push as much validation into the framework as possible and try to keep each operate method thin. This gives us a lot of developer experience features for free. The framework can:

Check for common typos, and give a better error message when possible (Unknown parameter 'nmae' - did you mean 'name'?)
Check that you’re in the correct environment (No Account exists with the ID 'account_123'. A similar object exists in the sandbox, are you in the correct environment?)
Check that you have the right permissions to access the resource
Aggregate many “parse-time” errors together, so we can return a single error response with all of the mistakes in the request instead of a series of 400s.

Consistency and expressive errors help make a great API. Having an opinionated framework like this helps us keep those properties as we grow.

From code to schema

Validation and API definition is only half of the framework’s job. The real magic is turning the code definition into a portable schema that can be used by tools in any language. For us, that is OpenAPI 3.1, which has issues but is the least bad option we’ve found. Our API framework uses a big pile of Ruby reflection to translate our statically defined API shape into a high-quality OpenAPI schema. (Some teams start by hand-writing an OpenAPI schema and code-generating server-side stubs for implementation. We think it’s impossible to maintain consistency and quality in a hand-written schema of this size.)

There’s an art to making a good OpenAPI schema, and the framework guides engineers toward success here too. Because we reuse AccountResource across multiple API methods, the OpenAPI generator extracts it into a $ref (and, in turn, code-generation tools consuming the spec would only define a single class for that shape). Generating the schema also lets us use advanced OpenAPI features like discriminator that would be impossible to encode by hand at our scale. We also generate extensions, such as per-case enum descriptions, for features that haven’t made it into the OpenAPI standard yet.

components:
  schemas:
    account:
      title: Account
      type: object
      additionalProperties: false
      properties:
        id:
          description: The Account identifier
          type: string
        name:
          description: The name of the Account
          type: string
        funding:
          description: The type of the Account
          type: string
          enum:
            - deposits
            - loan
          x-enum-descriptions:
            - A depository account such as checking or savings
            - A loan account
        created_at:
          description: When the Account was created
          type: string
          format: date-time
      required:
        - id
        - name
        - funding
        - created_at

OpenAPI also often gives you many ways to express the same thing (a ~~rant~~ post for another day). There are at least five ways to declare a nullable attribute in OpenAPI 3.1, each with trade-offs. Centralizing generation in one place lets us experiment with the format that’s best for our customers.

From schema to everything else

An expressive, up-to-date schema document is probably the single most valuable thing you can hand to an agent building against your API. But we get a lot more! We use ours to build every other downstream API-related artifact we publish: our API reference, language-specific SDKs, Postman collections, etc. There are lots of tools that can generate a nice documentation site from an OpenAPI schema but we hand-roll ours so we can use long-tail features or extensions in the spec like x-enum-descriptions.

This means that the documentation of our API can’t drift from its implementation. It might sound basic but not every API has this property and it’s maddening to encounter ones that don’t.

Putting it together

There are infinite ways to build and publish an API, but this is ours:

Define a schema in code
Centralize schema validation as much as possible
Go “down” from the schema into strong types in your code
Go “up” from the schema into an OpenAPI specification
Go “up” from the specification into documentation, SDKs, etc

We’ve looked into open-sourcing our framework once or twice and stopped - it’s nice to have some coupling to the rest of our codebase (such as the associated_model method which ties it to our ORM). This code is well-suited for agents to write, though, so try pointing yours at this post and see where it gets you!