protobuf-versioning-strategy-reference

Overview

Protobuf3 schema evolution is a wire-format problem first and a codegen problem second. The field number is the only durable identifier - every breaking-change rule derives from preserving field-number → type binding.

Per protobuf.dev/programming-guides/proto3/: "This number cannot be changed once your message type is in use because it identifies the field in the message wire format."

This skill is a pure reference consumed by buf-CLI lint, the breaking-build CI integration, and the gRPC service authors. For the detection workflow see buf-cli-lint-breaking-build.

When to use

• Designing a proto change - is this safe?
• Auditing an existing schema for risky patterns (un-reserved deleted fields, oneof-conversion footguns).
• Configuring buf breaking - which category fits the deployment model?
• PR review of .proto changes.

Field-number rules

Per protobuf.dev:

When deleting a field, reserve its number:

message User {
  reserved 4, 7, 10 to 12;
  reserved "deprecated_email";

  string name = 1;
  // ...
}

Per the spec: "If you do not reserve the field number, it is possible for a developer to reuse that number in the future." Reuse → semantic corruption: old clients interpret bytes as the old type.

Binary wire-safe changes

Per the protobuf3 docs, these are fully safe - old code parses new messages and vice versa with no loss:

Wire-compatible changes (conditionally safe)

Per protobuf3 docs, these preserve wire compatibility but may be lossy or surprising:

The catch: a parser reading int64 data with int32 will silently truncate. The wire is "compatible" but the data may be lost.

Wire-incompatible changes (always breaking)

Per protobuf3 docs:

• Changing field numbers is equivalent to deleting and re-adding. Always breaks.
• Moving fields into an existing oneof is not safe.
• Changing map<k,v> key or value type.
• Changing field cardinality from singular to repeated (or vice versa) outside the wire-compatible paths.

Oneof constraints

Per protobuf.dev:

• Oneof fields cannot be repeated or map.
• "If multiple values are set, the last set value as determined by the order in the proto will overwrite all previous ones."
• Adding a field to an existing oneof is always breaking - old code can't represent the new variant; new data crashes old parsers.
• Removing a field from a oneof is breaking for the same reason.
• Converting a singular field into a single-field oneof: safe.
• Converting a single-field oneof back to singular: safe.

Map constraints

Per protobuf3 docs:

• Maps cannot be repeated.
• Key types: integral and string scalars only. "neither enum nor proto messages are valid for key_type."
• Changing a map's key or value type is breaking.

buf breaking-change taxonomy

Per buf.build/docs/breaking/rules, buf organises detection into four categories, strictest to most lenient:

FILE (default)

Detects "changes that move generated code between files, breaking generated source code on a per-file basis."

Selected rules:

Choose FILE when codegen consumers import per-file (most JVM, .NET, generated stubs in monorepo).

PACKAGE

Detects breakage at package level; permits file relocations inside a package.

Selected rules:

Choose PACKAGE when consumers import by package (Go, Python).

WIRE_JSON

Detects "changes that break wire (binary) or JSON encoding."

Selected rules:

Choose WIRE_JSON when consumers use both binary and JSON encoding (REST gateway + grpc).

WIRE (most lenient)

Detects only changes "compromising binary wire format compatibility."

Selected rules:

Choose WIRE when only wire format matters (binary-only protocols between server fleets you control).

Choosing the category

# buf.yaml
version: v2
breaking:
  use:
    - FILE        # most strict (default)
# OR
    - PACKAGE     # codegen friendly within-package
# OR
    - WIRE_JSON   # wire + JSON
# OR
    - WIRE        # wire only

CI invocation:

buf breaking --against ".git#branch=main"
# Compares the working tree against the main branch as baseline

Common patterns

Adding an optional field

Safe (always):

 message User {
   string name = 1;
+  string nickname = 2;
 }

Removing a field

Always reserve:

 message User {
+  reserved 2;
+  reserved "nickname";
   string name = 1;
-  string nickname = 2;
 }

Renaming a field

Add new, deprecate + reserve old:

 message User {
   string name = 1;
-  string nickname = 2;
+  string display_name = 3;
+  reserved 2;
+  reserved "nickname";
 }

Consumers must migrate from nickname to display_name. The wire format reads either; the codegen forces consumers to update.

Promoting int32 to int64

Wire-compatible per protobuf3 docs:

 message Counter {
-  int32 count = 1;
+  int64 count = 1;
 }

Old clients writing int32 still parse correctly. Old clients reading new int64 data truncate silently if the value exceeds int32 range.

Adding a field to a oneof

ALWAYS BREAKING. Don't.

 message Event {
   oneof body {
     string text = 1;
     bytes binary = 2;
+    string emoji = 3;  // BREAKS old parsers
   }
 }

Mitigation: add the new variant as a non-oneof field; promote later in a separate proto file/package.

Anti-patterns

Limitations

• buf categories cover binary breakage. Semantic breakage (a field's meaning changes) is not detectable. Document semantic changes in commit messages.
• JSON name changes (camelCase ↔ snake_case) detected only in WIRE_JSON.
• No transitive analysis. A message field whose type is an imported message: buf doesn't follow the import.
• Doesn't enforce naming conventions. buf lint is separate from buf breaking.

References

• protobuf3 versioning rules: protobuf.dev/programming-guides/proto3/.
• buf breaking-change rules: buf.build/docs/breaking/rules.
• Consumed by: buf-cli-lint-breaking-build, grpc-streaming-test-author.
• Differentiation: this is the catalog of what is and isn't breaking and why. For detection of changes in CI, use buf-cli-lint-breaking-build. For protobuf-schema contract-testing across services see qa-contract-testing/protobuf-compat-checking.