Production Assets: Data Management

Skip to Translations

CONTEXT

The redundancy of original camera files (OCF) and original production audio files (OPA) throughout a production is crucial. Data loss can be a huge financial burden and creative loss.

There are multiple ways to backup and verify production assets. Below are our minimum expectations followed by best practices.

MINIMUM EXPECTATIONS

Original Camera and Audio files must be stored securely, safely, reliably, and be accessible throughout Production and Post Production. These requirements can be met by implementing data management strategies such as the industry-wide known 3:2:1 strategy.

Each title is responsible for ensuring good data management strategies are maintained through final delivery to Netflix. If you have questions or concerns about the media management plan on your title, reach out to your Netflix representatives.

ACCEPTABLE OCF/OPA ARCHIVAL FORMATS FOR NETFLIX:

Netflix Cloud Storage using Content Hub Footage Ingest
LTO tapes (Generation 6, 7, 8, 9 written in LTFS format)

CHECKSUM VERIFICATION:

All copies of original camera and audio files should include checksum verification, with either xxHash64be or MD5 hash types. Checksum verification is built into most professional offload software.
During the checksum process, a unique machine-readable checksum manifest should be generated and accompany OCF and OPA files through all subsequent transfers, including final delivery. Netflix recommends the ASC MHL checksum manifest format.

When uploading OCF/OPA to Netflix’s Cloud Storage - checksum hash type, manifest format, and manifest location need to adhere to the requirements outlined in our Netflix Footage Ingest tool guidance.

3:2:1 STRATEGY

Hold at least three copies of all original camera files (OCF) and original production audio (OPA) files.
Store the OCF and OPA copies on at least two different types of media.
- Examples:
  - Capture Device (e.g., camera card or sound card)
  - NVMe transfer/shuttle drives
  - RAID 5, 6, or 10 (or above) storage (RAID 0 is only acceptable for temporary transfer/shuttle drives, and not for backup purposes.)
  - LTO 6, 7, 8 or 9 written in LTFS (v2.0.0 or later)
  - Cloud-based file storage
Keep at least one of these backups in a different geographical location from the others

WHAT ABOUT NETFLIX FOOTAGE INGEST?

Netflix Footage Ingest is not in itself a data management strategy, but it is a solution that can simplify and expedite your data management plan. Netflix considers 3:2:1 completely fulfilled by the completion of the backup and verification of OCF/OPA to Netflix Cloud Storage via Netflix Footage Ingest.

When uploading OCF / OPA to Netflix Content Hub, files are fully verified first in Netflix Data Center Storage and then in Netflix Cloud Storage. Each step of the process is clearly communicated through email notifications back to Production and visibility in the Content Hub interface.

Netflix Data Center - meets the equivalent of two [2] other backups
Netflix Cloud Storage - fully meets/exceeds 3:2:1 expectations.

Once Production has confirmed all OCF and OPA are fully verified in Netflix Cloud Storage, Netflix does not require Production to retain additional copies of the source media. In most cases, Footage Ingest will replace Netflix studio requirements for LTOs or hard drives, unless directed differently by Netflix and/or its Production Partner. If you have questions or concerns about the media management plan on your title, reach out to your Netflix representatives.

BEST PRACTICES

CHECKSUM MANAGEMENT AND CHAIN OF CUSTODY

After the original checksum-verified copy with machine-readable checksum manifests (e.g., ASC MHL) has been completed, any subsequent copies should be validated against the checksum values from the original copy to ensure the chain of custody. This validation ensures nothing has changed in those directories and files.

ASC MHL is Netflix’s recommended checksum manifest format.

REPURPOSING CAMERA CARDS

Ideally, there is a designated party responsible for final approval to delete and recycle camera media. Editorial often performs this function because they have the highest level of context across the expected media and associated reports from different departments. If turnaround doesn’t allow for Editorial to do this check, ideally another responsible party will do so (for example, the Dailies Lab).

There are other considerations before wiping camera cards for reuse:

OCF and OPA are secured using the 3:2:1 principle stated above.
A visual evaluation of all OCF has been performed, and all the files have been checked and accounted for against all relevant reports.

Camera Card Count

Hold enough cards to complete the expectations above. This usually means having enough cards for two days of shooting; however, always have the 3:2:1 conversation with all stakeholders before confirming that you have enough camera cards.

REPURPOSING TEMPORARY ONSET STORAGE

If OCF & OPA is held on temporary storage, there are some considerations before deleting and repurposing that storage:

OCF and OPA reside on a minimum of three checksum-verified storage mediums, including the final archival delivery format.
A visual evaluation of all OCF has been performed, and all the files have been checked and accounted for against all relevant reports.

VISUAL INSPECTION & VALIDATION OF CONTENT:

Content review at offload

Whether you’re shooting scripted or nonfiction content, the best time to check for any image or sound file irregularities is during the offload from the source media. The visual inspection should cover any possible major recording or data transfer issues and serve as a technical check.

Example method: Scrubbing through the material to perform a spot check of all captured media.

Full QC in a controlled environment:

After offload, a more thorough QC review of camera files can help provide feedback to key production personnel and minimize future problems. With that in mind, a few considerations are listed below:

The primary purpose of Full QC is to catch correctable technical issues early when it is valuable to do so [see: Production QC Glossary].
Full QC is ideally done at real-time playback.
Full QC should be performed at a minimum image resolution of 3840x2160 to best identify visual artifacts (e.g., dead pixels, moire, etc.)
Full QC should be done in a controlled environment like a lab. If color evaluation is part of the process, the monitor should be professionally calibrated consistent with the project's color management plan.
Full QC should be done from a checksum-verified safety copy and not a camera card.

Scripted Fiction Editorial sign-off

Editorial is uniquely positioned to catch other discrepancies not noticed during offload review or Full QC in a controlled environment. This is because Editorial has access to reports from multiple departments such as Script Supervisor, Camera, Sound, and Dailies Vendor, as well as visual proxies with additional metadata. They can cross-reference all of these to confirm receipt of every piece of shot material and potentially flag image inconsistencies or possible problems when reviewing.

Offload Bandwidth Speed

The throughput speed of the devices you’re copying to and reading from is critical; faster drives will allow you to turn around camera media and drives sooner. A single slow device in the data management chain adds avoidable bottlenecks, adds unnecessary overtime, delays media turnaround, and challenges the success of a 3:2:1 workflow.

When building your data management plan, it's important to consider:

The speed of the original camera and sound source media.
The speed (including the interface) of any camera and sound card reader.
The speed of all of the drives or storage in your data chain.
- It is recommended that drives should not be slower than the source (e.g., camera card or camera reader).
If writing to LTO - the speed of writing and verifying the generation of LTO tape utilized.
If uploading to Netflix’s Cloud Storage - The speed of internet bandwidth for uploading files.

Change Log

07/26/2024

Restructured for clarity
Added information on Netflix Cloud Storage
Added link to requirements for “Content Hub - Footage Ingest”
Added example of recommended checkum manifest format, ASC MHL.

12/08/2021

Added link to "Data Management - Why It's Important."

4/21/2021

Added LTO9

10/12/2020

Removed "RAID 5, 6, or 10 (or above) storage (Pending Netflix Approval)" from the "Acceptable Final Archival Delivery Formats:" section.

6/30/2020

Added examples of data loss to reinforce the importance of redundant copies.
Removed the Content Hub option within the 3:2:1 Principle since it is an alternate solution that requires further discussion with Netflix.
Added example method of visual inspection.
Renamed “Visual QC” to “Full QC” to differentiate from Visual Inspection.
Renamed “Editorial Proxies” to “Editorial Rushes”.