Before media is uploaded into the Netflix ecosystem there are requirements and best practices that you will want to adhere to when it comes to checksumming your media, folder structure, and which hard drives to use. By adhering to these guidelines, you can ensure a smoother and more efficient upload process.
Before beginning each upload, ensure your media meets Netflix's specifications by validating it with the Netflix Footage Ingest application. For detailed instructions on how to perform this validation, please refer to the comprehensive user guide available here. Keep in mind that the application checks file and directory lists only. Any corrupted files with checksum mismatches will be identified after the media is ingested at the Netflix data center.
Hard Drive Best Practices
When offloading your media from your camera and sound cards to a transportation drive, following these best practices will ensure a seamless upload experience:
-
Drive Selection
- Connector Type: Use a USB-C/Thunderbolt female connector.
- Drive Speed: Select SSD drives with a link speed of 10Gbps or higher.
- Drive Type: Choose NVMe/M.2 SSD drives, avoiding mechanical SATA and generic "homegrown" SSDs. Opt for rugged, non-encrypted drives with shock and water resistance, and avoid NVMe SSD enclosures due to slower read speeds.
- Connectivity Indicator: Ensure the drive has a status LED to check connectivity.
- Power Source: Prefer USB-C powered drives over those requiring an external power supply.
- RAID Configuration: If using multiple SSDs, configure them in RAID1 or RAID5 for redundancy.
- Compatibility Note: M4 Mac mini Thunderbolt 4 ports do not support USB 3.2 Gen 2x2 natively, and the connection will default to a speed of 10 Gb/s.
- Operating System Consistency: Changing operating systems can create hidden system files, which may cause issues during verification and inspection. Note that the Netflix Footage Ingest Application is designed for macOS. Avoid switching operating systems during the offloading process. Uploading from the drive coming from set is a good way to avoid the creation of these hidden files.
Best Practices for Offloading and Verifying Production Assets
The primary goal of these media offloading and verification practices is to establish and maintain the "chain of custody" of all production media, ensuring that files are transferred accurately, securely, and without corruption from the original source to Netflix. To maintain this chain of custody, the original manifest created at the time of the first offload should be checked again with every subsequent copy. This process is essential for ensuring the quality and completeness of the media throughout post-production.
By verifying the integrity of every file at each step, productions reduce the risk of data loss, corruption, or inconsistencies that could disrupt timelines and compromise the final output. Netflix performs a final check on the assets to provide productions with visibility into the health of their archive, but it is the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered to Netflix and the original manifest created on set.
Keys for Ensuring Success
-
- Use a Compatible Data Management Tool: Select professional data management software that creates a Netflix compatible MHLs per roll, automating the offloading and verification process. If you have any questions about which software to use, please contact your Netflix Representative.
- Ensure Data Integrity: Files must be verified during offloading and after each copy to confirm the media remains intact and unaltered.
- Sealing Rolls: Once the MHL is created, consider the roll ‘sealed’. Avoid modifying the files or folder structure, and if changes are necessary, update the MHL accordingly.
- Final Check at Netflix: The Footage Ingest Application performs a final verification on media delivery. Productions are encouraged to address any discrepancies that arise to ensure data integrity and compliance. It is the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered via the Footage Ingest Application and the original manifest created on set. Netflix will not be auditing this process or the remediation of any errors.
Steps for Successful Data Management from Set to Netflix
1. Offload Media:
- Start by using a reliable data management tool to automatically generate a Media Hash List (MHL) during the offloading of your media files.
- Footage ingest supports the hash types of MD5, xxhash64BE, or xxhash128.
- Generate an MHL for each roll of footage, this manifest will serve as the reference point for all verifications ran once the materials are within Netflix ecosystem. This step is crucial as it creates checksums for every file, allowing you to verify a successful transfer by comparing the original and new checksums.
- Ensure your tool is set to produce ASC MHLs, as these are required to meet Netflix’s specifications.
2. Sealing Rolls:
- After the initial offload, once you've generated the Media Hash List (MHL), consider the roll sealed. Don’t change the MHL once the copy is complete.
- To maintain the media's integrity, avoid making any changes to the files or folder structure after the offload is finished. Alterations can lead to discrepancies during future checks against the original MHL, which will require remediation.
- If you must rename or move files after offloading, ensure you update or recreate the MHL to reflect these changes. ASC MHL provides tools to assist in updating the MHL when files are renamed, ensuring continued accuracy and integrity.
Figure A
3. Maintain a Netflix-Compatible Folder Structure: Organize your media using a consistent folder structure that allows for easy identification and verification. Each camera and sound roll must be stored in its own containing folder (e.g., Camera_Roll/A001 or Sound_Roll/S001) and all camera and sound rolls must have a unique folder name so the Footage Ingest Application can identify them as discrete assets.
Figure B - Here, you can see camera rolls folders are contained within a single folder. This will ensure proper identification of rolls. How the folder that contains the rolls is named is irrelevant. You may also have several folders that contain rolls (some users like to separate each camera type into their own containing folder).
Figure C - Here, you can see roll folders are at the root level, and not in a containing folder. This will cause misidentification of rolls and other folders which will likely lead to upload errors and the need to remediate assets after upload is completed.
The Media Hash List (MHL) , or in the case of ASCMHL, the "ascmhl" folder, must be located at the root of each roll directory (e.g., Camera_Roll/A001/A001.mhl or Camera_Roll/A001/ascmhl/A001.mhl). Netflix also provides a folder naming template to automate tagging important metadata, such as Shooting Block, Unit, and Day Number, to streamline uploads.
Figure D
4. Verify Files After Every Copy: After copying media to new storage locations (e.g., hard drives or cloud storage), it is critical to verify the copied files against the original checksums stored in the MHL. This step ensures that the data remains intact and free from corruption. Using ASC MHL compliant tools automates this process, as comparing the new copy against the original manifest and generating a new one is required by the spec.
5. Final Verification Upon Delivery to Netflix: When media is delivered to Netflix, the system will perform a final verification using the MHLs provided for each roll. If any discrepancies are found (e.g., extra, missing, or mismatched files), the user interface (UI) will notify the user of the specific issues.
Figure E
6. Remediation of Issues: If verification discrepancies occur, productions can choose to resolve the issue by reuploading the problematic files with the correct MHL or checksums. This final step provides the opportunity to fix any problems identified during the process, ensuring that Netflix receives complete and accurate media. It is, however, the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered to Netflix and the original manifest created on set as Netflix will not be auditing this process.
Things to pay extra attention to
- Large upload directories - Proactively manage your upload directory to stay within Netflix's limits of a maximum of 100 rolls or 10TB of data. Before uploading, organize your files into multiple directories if they exceed these limits. Ensure each directory is independent and complete, without splitting any rolls. These directories can be set up on your hard drives at the finder level while on set.
Figure F
- Uploads without rolls - If the Netflix Footage Ingest application does not find any camera or sound rolls in your directory, you will be warned before initiating the upload. This usually happens because there are no MHLs per camera roll. Please, check your folder directory to ensure MHLs are correctly placed in the directory.
-
Invalid characters - If the Netflix Footage Ingest application detects illegal characters in your folder structure, you will be warned before initiating the upload. Some characters are not allowed to be uploaded (such as escape characters like ‘\r’). Please remove these characters before uploading, especially if they are found in non-rolls directories. If the invalid characters are found within rolls, then ensure that the MHL reflects this change. Should you wish to use a folder naming convention, you can find the guidelines here.
- Illegal characters: @ # $ % ^ & * ( ) ` ; : < > ? , [ ] { } / \ ' ” | ~
- Missing MHLs - If there are rolls within a camera roll containing folder that don't have MHLs, the tool will flag it. You can still proceed with the upload, but Netflix could not verify your media. It is recommended that each roll includes a manifest to be able to compare against upon upload completion.
- Multiple LEGACY MHLs per roll (does not apply to ascmhl) - If the Netflix Footage Ingest application detects multiple legacy MHLs in a roll you will be warned. You will still be able to proceed with your upload, but it is best, to ensure no failure in downstream processes to fix the issue before upload.. Please, check each roll to ensure that there is only one legacy MHL v1 per roll.
- Hidden files - If the Netflix Footage Ingest application detects hidden files in the upload directory, you will be warned and you will not be able to proceed with your upload.