Schema checks

Certain changes to your graph's schema (such as removing a field or type) might break one of your application's clients. GraphOS provides schema checks to help you identify breaking changes before you make them, and to help you identify when a potentially dangerous change won't break anything.

You can review the results of schema checks inside Apollo Studio, helping you make informed decisions about evolving your graph:

Schema checks are free as part of all Apollo plans.

Types of checks

GraphOS can perform the following types of schema checks:

• Composition checks. For supergraphs, verify whether your proposed changes to a subgraph schema will successfully compose with your other subgraph schemas.

• Operation checks. Compare your proposed schema changes against historical operations to verify whether the changes will break any of your graph's active clients.

• Contract checks (enterprise only). When running schema checks on a source variant, also check whether your proposed schema changes will break any downstream contract variants.

Prerequisites

To enable schema checks for your supergraph, do the following if you haven't yet:

1. Make sure you've published all of your subgraph schemas to GraphOS, and that those schemas are up to date.

   • Learn how to publish schemas.

2. Make sure your supergraph is sending operation metrics to GraphOS. GraphOS uses historical metrics to determine whether a potentially dangerous schema change is safe.

   ⚠️ If GraphOS has no operation metrics to compare against, all potentially dangerous schema changes result in a failed check!

   • If you have a cloud supergraph, your router reports metrics automatically.
   • Setup for other graphs

3. Install the Rover CLI in your development environment and authenticate it with Apollo Studio.

Running your first check

Let's say you've made local changes to one of your subgraph schemas. After you complete the prerequisites, you can run schema checks against those changes with the rover subgraph check command.

The rover subgraph check command looks like this:

1
rover subgraph check docs-example-graph@main --name products --schema ./schema.graphql

It requires the following:

• Your registered graph's graph ref, which a string with the format graph-id@variant-name (for example, docs-example-graph@main). This is available from your supergraph's README page in Studio.

• The locally modified version of your schema. In the command above, the schema is provided via a .graphql file.

  • If your schema is not in a standalone .graphql file, you can run your GraphQL server locally and pipe its schema directly from rover graph introspect, like so:

    Bash

    1
    rover subgraph introspect http://localhost:4000 | rover subgraph check docs-example-graph@main --name products --schema -

    Copy

Try changing something in the local version of your schema and see what happens! If everything is set up correctly, the command's output looks similar to the output shown in The check response.

The checks lifecycle

When you run rover subgraph check:

1. GraphOS generates a diff between your local schema and the published schema for the variant you're checking against.
2. GraphOS uses this diff to determine whether the changes affect any operations that have been executed against your supergraph within a customizable time window (by default, this is the last seven days).
3. GraphOS returns the diff, along with a list of the operations that are affected by the changes.
4. Rover prints the result of the check and returns a non-zero exit code if at least one breaking change is found.

The check response

Running rover subgraph check outputs the diff of all detected schema changes and highlights breaking changes:

1
$ rover subgraph check docs-example-graph@current --subgraph products --schema ./schema.graphql
2

3
Validated the proposed subgraph against metrics from docs-example-graph@current
4
Compared 1 schema changes against 24 operations
5
┌────────┬────────────────────┬─────────────────────────────────────────────────────────┐
6
│ Change │        Code        │                       Description                       │
7
├────────┼────────────────────┼─────────────────────────────────────────────────────────┤
8
│ PASS   │ FIELD_CHANGED_TYPE │ field `Query.books`: type `[Book]` changed to `[Book!]` │
9
└────────┴────────────────────┴─────────────────────────────────────────────────────────┘
10

11
View full details at https://studio.apollographql.com/service/docs-example-graph/checks/<DETAILS>

Each change to the schema is labeled either PASS or FAIL.

The rover subgraph check command returns a nonzero result if any check fails.

The output also includes a Studio URL that provides full details on the changes and their impact on existing clients and operations:

Rerunning checks

You can rerun checks from inside Studio:

When you do, the new check uses the current check configuration options, regardless of the configuration at the time of the original check. Similarly, the new check's time window is based on the current time, not the time when the original check ran.

The new check incorporates any changes made to excluded or included clients, checked variants, and any operations marked as safe or ignored.

Composition checks

When you run subgraph check, GraphOS performs a composition check before it performs any other checks. A composition check verifies that changes you make to a subgraph schema are valid GraphQL definitions and are compatible with your other subgraph schemas, enabling them to compose into a supergraph schema (learn more).

Results of all schema checks are available in Studio from your graph's Checks page:

You can click on a composition check to view its result. If composition succeeded, you can view the composed supergraph schema. Regardless of success, you can view the proposed subgraph schema.

Operation checks

GraphOS also validates schema changes with operation checks after the changes have passed composition checks. Operation checks look at your reported operations to your graph and flag any clients and operations that would be affected by this change. Because changes like removing a field will pass composition checks if the resulting supergraph is a valid schema, however it could still cause runtime errors for clients that are actively using the schema.

Overriding flagged changes

Occasionally, schema checks might flag a change that you know is safe. For example, you might change an input type's field from nullable to non-null when you are certain that your clients never provide a null value for that field.

In cases like this, you can override a flagged change in Apollo Studio from the associated check's details page:

You override flagged changes on an operation-by-operation basis. For each operation with flagged changes, you can override those changes in the following ways:

• Mark the changes as safe. In this case, schema checks will not flag these exact changes for the operation in any future execution. This effectively "approves" the changes for the operation.
• If a future check detects approved changes along with new unsafe changes to the operation, the new unsafe changes will be flagged.
• Ignore the operation. In this case, schema checks will completely ignore the operation when checking all changes in any future execution.
• This option is useful when you know an operation originates only from clients or client versions that you don't actively support.

Contract checks (enterprise only)

For more info on Contract/downstream checks, see our Contracts page.

Using in CI

Schema checks are especially useful when you add them to your CI pipeline (such as Jenkins or CircleCI). By doing so, you can obtain check results and display them directly on your team's pull requests.

We recommend defining a separate CI job for each variant of your schema (production, staging, etc.) that you want to validate your changes against. The rover subgraph check command returns a non-zero exit code when it detects a breaking change, meaning the job fails when the check fails.

Authenticating Rover

The rover config auth command is interactive, which means you shouldn't use it in CI environments. Instead, you can authenticate Rover with Studio by setting the APOLLO_KEY environment variable in CI. For details, see Configuring Rover.

Example configuration

The following config defines a schema check job for a CircleCI pipeline. Your config's syntax varies depending on your CI tool, but the job's steps are the same.

1
version: 2
2

3
jobs:
4

5
  # ...other jobs...
6

7
  # Define a separate job for each environment you validate against.
8
  check_against_staging:
9
    docker:
10
      - image: circleci/node:12
11

12
    steps:
13
      - checkout
14

15
      - run: npm install
16

17
      # Start the GraphQL server. If a different command is used to
18
      # start the server, use it in place of `npm start` here.
19
      - run:
20
          name: Starting server
21
          command: npm start
22
          background: true
23

24
      # Make sure the server has enough time to start up before running
25
      # commands against it.
26
      - run: sleep 5
27

28
      # In CI environments, this command authenticates via the `APOLLO_KEY`
29
      # environment variable.
30
      - run: rover subgraph check docs-example-graph@current --name products --schema ./schema.graphql

Integrating with GitHub

If you're using GitHub, you can install the Apollo Studio GitHub app. This app enables Apollo Studio to send a webhook back to your GitHub project on each call to rover subgraph check, providing built-in pass/fail status checks on your pull requests:

Integrating with other version control services

If you're using GitHub Enterprise, Bitbucket, or another version control service, we recommend setting up your CI tool to post a comment on each pull request with the results of schema checks. By surfacing schema diffs and breaking changes directly in your PR, you can avoid searching your CI logs to determine why a check failed.

Customizing checks

See Configuring schema checks.

Types of schema changes

Not every change to a schema is a potentially breaking change. Additive changes (such as adding a field to a type) are usually safe and do not affect active clients. Deletions and modifications (such as removing a field or changing a return type), however, can break clients that use affected types and fields.

Potentially breaking changes

Removals

These changes remove a schema element. If a removed element is actively being used by an operation, that operation will start returning an error.

Addition of required arguments

These changes add a required input to a schema element. If an operation is actively using an element of your graph and doesn't add the new required input argument, the graph will return an error to affected clients.

In-place updates

These changes update an existing schema element. If an operation is actively using an element that is updated, the operation might start receiving an error from your graph. It also might receive an unexpected result.

Default arguments

These changes update the default value for an argument. If an operation is using an element of your graph and does not specify a value for this argument, the operation might get an unexpected result when the schema is updated if it was relying on the original default value.

Non-breaking changes

These changes are detected by schema checks, but they are "safe." They never affect the behavior of any existing clients if deployed.

Schema additions

Deprecations

Descriptions

Limits

Operation check cardinality

Operation checks will run against a maximum of 10,000 distinct operations. A warning message will display on the UI when a larger cardinality has been reached.

Input field usage reporting

GraphOS tracks operation usage, however, in order to group duplicate operations, we use operation signatures which denormalize the arguments and inputs. This means that the result of the operation signature does not track which fields on input types that are used.

For example, GraphOS will not be able to track how many times the input field GetUsersInput.firstName is used in this schema:

1
type Query {
2
  getUsers(filters: GetUsersInput): [User]
3
}
4

5
input GetUsersInput {
6
  firstName: String
7
  lastName: String
8
}
9

10
type User {
11
  id: ID!
12
  firstName: String!
13
  lastName: String!
14
}