Ch.7: Protobuf: Zero, False, Empty, or Missing?

0:00 Welcome to Learning Podcasts. Protobuf: The 0-Value Trap. Sometimes the field that matters most is the one that is not there. Or worse, the field that is there but reads like it was never set. Picture a grading service. A student's score is an int32 field. One student takes the exam, scores 0. Painful, but real. And another student's exam was never graded, so the score field was never set. But under proto3 implicit presence, both messages arrive at the receiver as score equals 0. Same number on the dashboard.

0:33 Same bytes on the wire. Completely different story. And the bug is hard to spot because nothing looks malformed. The dashboard is not blank. The parser did not crash. The object has a normal value, and the normal value is the problem. And there is no way to recover the difference after the fact. The wire never carried the distinction in the first place. Once the message crosses that boundary, no amount of clever decoding can pull the sender's intent back. Proto3 makes the same promise about every scalar field.

1:03 Numeric fields default to 0. Booleans default to false. Strings default to empty. Bytes default to an empty sequence. Enums default to whatever value sits at 0, which we said in chapter 5 must be UNSPECIFIED. And that value is not an error placeholder. It is the official value the generated API returns. And different language bindings all have to agree on the same defaults. It is part of the schema contract, not a library convenience. Defaults are paired with a serialization rule. Under implicit presence, proto3 skips any scalar field whose value matches the default.

1:40 Set score to 95 and the encoder writes tag, value, real bytes. Set score to 0 and the encoder writes nothing for that field. The decoder already knows to fill in 0 when the field is missing. Yeah. So an explicit 0 and an absent score collapse into the exact same serialized message. Right. And the wire never carried a note that says, this 0 was intentional. It is just an absence of a field. Wait. Booleans are simpler than this, though. They have two values. False just means false. Why is consent the trap case and not the easy case?

2:18 Yeah, that is the natural reaction. The value is easy. The presence is the part the boolean cannot carry. Run the survey. Question reads, do you want to receive marketing emails? Single checkbox underneath. One user thinks about it and clicks no. Another user closes the page on mobile and never sees the question. Both submissions arrive with the box unchecked. And when you go back later to ask who actually opted out versus who just never saw the question, that data is gone. The checkbox cannot tell you.

2:52 And under regulations like GDPR, those are not the same answer. Right. The data model lost the difference between current state and decision record. The same shape shows up outside business workflows. A temperature sensor on a working freezer reports 0 degrees Celsius. Perfectly valid reading. The freezer is doing its job. And a sensor that lost power and sent nothing is a different situation entirely. Under implicit presence, both arrive at the receiver as 0. Same value, two completely different sensor states.

3:23 Yep. Classic. Or 0 means the rate limit is set to nothing. Missing means use the inherited default. Any time the default value lives in the valid domain, the receiver needs presence too. Otherwise the value alone is just half the signal. The direct fix is the keyword from chapter 6: optional. Write optional int32 score equals 5. The generated code now tracks whether the sender actually set the field. In Java you call hasScore. In Python you call HasField with the string score. And if the sender writes 0 on purpose, the encoder serializes the 0, and the receiver can ask, was this set, and get a real answer.

4:05 Yeah. The important habit is to stop using value comparison as a presence check. Do not ask, is the score not equal to 0. Ask, does the message have a score. Then handle the value after you know it was provided. Older proto3 code often solved this with wrapper messages from the well-known types library. Instead of int32 score, the schema used google dot protobuf dot Int32Value score. That wrapper is a message with 1 scalar inside it. Since message fields already track presence, the whole wrapper can be absent, or it can be present with the inner value set to 0.

4:41 That gave teams a nullable scalar shape before optional came back. You still keep wrappers around for compatibility with schemas that already use them, or when a scalar has to travel inside Any. But for new ordinary fields, optional is the cleaner default. Wrappers are useful history, not the shape you reach for first today. I will admit I reached for them pretty often before optional came back to proto3. At the time it felt like the clean path. The 3rd escape hatch is a sentinel value. You decide that negative one means no score recorded, and 0 means a real 0.

5:14 That can work when the domain rules out the sentinel. But if the convention is that simple, why model it as a whole new field shape? It is just, you know, a small contract. Well, protobuf does not enforce the contract for you. Every producer has to remember it. Every consumer has to validate it. And the impossible value can become possible later. Right. A score that has to be non-negative today might allow a curve adjustment tomorrow. Now every client has to migrate away from a magic number that was never visible in the type system.

5:47 Enums have their own version of the sentinel pattern, and chapter 5 already gave it to us. Put an UNSPECIFIED value at 0. Real business states start at one. Right. ACCOUNT_STATUS_UNSPECIFIED reads as the field was never set, or the sender did not know. ACCOUNT_STATUS_ACTIVE and ACCOUNT_STATUS_SUSPENDED are intentional states. It is still a default value. The default is just named as absence instead of pretending to be a real state. And that is exactly why ACTIVE at 0 is such a dangerous enum. The default looks like a real product decision.

6:24 UNSPECIFIED keeps the uncertainty visible. The trap gets worse in partial updates. The user opens a settings page, sees the quota field at 10, clears it to 0 on purpose, and clicks save. The form serializes only the field that changed. But under implicit presence, the serializer drops the field because 0 matches the default. The server receives a patch with no quota field. The merge sees nothing to update. The original quota survives. And the user clicked save and nothing happened. Right. The bug is invisible because non-default updates work.

6:58 FieldMask is one tool that fixes this, and we will get to API design patterns in a later chapter. Proto3 did not make this choice by accident. Implicit presence gave the language a simple rule. Default values are not serialized. Default-looking fields read as cleared. Basic scalars do not carry has methods. That kept many schemas smaller and many APIs simpler. But, I mean, looking at the patch trap and the consent trap and the sensor trap, this really does look like the kind of design decision you would undo if you had a second chance.

7:31 Well, not always. The cost is lost information. But for messages where the default really does mean the same thing as not provided, implicit presence works fine. The lesson is when it breaks, not that it was wrong from the start. Simpler model, less memory of intent. Protobuf's newer editions system moves the default back the other way. Editions 2023 sets field presence to explicit by default. Singular scalar fields track presence unless you opt into implicit behavior. Proto3 is still implicit unless you write optional.

8:05 The migration direction is clear. Right. New proto3 code that marks scalars optional today lines up with where the language is going, instead of where proto3 started. And the wire format is the same. Editions changes schema and API behavior, not how bytes are laid out. Optional is no longer a weird special case. It is the bridge. Yeah. Here is the practical rule. For new proto3 scalar fields, mark them optional unless you have a specific reason not to. For new enums, keep the 0 value as UNSPECIFIED.

8:39 Right. And for wrapper types? Keep them where they exist for compatibility. Do not reach for them in new code. Sentinels only when the domain forces it. And when you design an update API, ask whether the user might intentionally send a default value. Yeah. The checklist is short. Can 0, false, empty string, or the enum-0 value be intentionally sent? Would the receiver do something different if it knew the field was absent? If either answer is yes, model presence explicitly. The 0-value trap is simple.

9:10 The value you read from a proto3 message may not tell you whether the sender set anything at all. Next, we look at timestamps, durations, and the well-known types protobuf gives you so common data shapes do not have to be reinvented. Thanks for listening to Learning Podcasts.