Netflix

All about Netflix technologies

Practical [[API]] [[Design]] at Netflix, Part 1: Using [Protobuf] [[FieldMask]]

Link: https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518

By default, [[gRPC]] uses protobuf as its [[IDL]] ([[Interface Definition Language]]) and data serialization protocol.

[[FieldMask]] is a protobuf message. There are a number of utilities and conventions on how to use this message when it is present in an [[RPC]] request. A FieldMask message contains a single field named paths, which is used to specify fields that should be returned by a read operation or modified by an update operation.

message FieldMask {
  // The set of field mask paths.
  repeated string paths = 1;
}

[[FieldMask]] is used for:

• Masking name of field by field number.

• Masking message fields

• Can ship pre-built fields

Limitations

• Using FieldMask can limit your ability to rename message fields (described in the Protobuf Field Names vs Field Numbers section of the article)

• Repeated fields are only allowed in the last position of a path string. This means you cannot select (mask) individual sub-fields in a message inside a list. This can change in the foreseeable future, as a recently approved Google API Improvement Proposal AIP-161 Field masks includes support for wildcards on repeated fields.

Practical API Design at Netflix, Part 2: Protobuf FieldMask for Mutation Operations

Link: https://netflixtechblog.com/practical-api-design-at-netflix-part-2-protobuf-fieldmask-for-mutation-operations-2e75e1d230e4

Cloud Packaging in the terabyte era

https://netflixtechblog.com/netflix-cloud-packaging-in-the-terabyte-era-d6869b4b84ae

Improved Architecture

• Vurtial Assembly

• Writable [[MezzFS]]

MezzFS

https://netflixtechblog.com/netflix-cloud-packaging-in-the-terabyte-era-d6869b4b84ae

MezzFS is a tool developed by Netflix that allows cloud storage objects to be mounted as local files via [[FUSE]].

There are existing distributed file systems for the cloud as well as off-the-shelf [[FUSE]] modules for [[S3]]

Our previous blog post described how MezzFS addresses the challenges for reads using various techniques, such as adaptive buffering and regional caches, to make the system performant and to lower costs. For write operations, those challenges do not apply. Furthermore, the goal for writes is not to build a general purpose system that supports arbitrary writers, but rather one that maximizes potential packager performance