Before media is uploaded into the Netflix ecosystem there are requirements and best practices that you will want to adhere to when it comes to checksumming your media, folder structure, and which hard drives to use. By adhering to these guidelines, you can ensure a smoother and more efficient upload process.
Should you have any questions about data management best practices, you can find a complete guide here.
Hard Drive Best Practices
When offloading your media from your camera and sound cards to a transportation drive, following these best practices will ensure a seamless upload experience:
- Drive Selection: Netflix recommends using SSD or NVMe drives due to their higher random read speeds and ability to mitigate network throttling. Avoid using mechanical drives or RAID arrays as transportation drives.
- Operating System Consistency: Changing operating systems can create hidden system files, which may cause issues during verification and inspection. Note that the Netflix Footage Ingest Application is designed for macOS. Avoid switching operating systems during the offloading process. Uploading from the drive coming from set is a good way to avoid the creation of these hidden files.
Best Practices for Offloading and Verifying Production Assets
The primary goal of these media offloading and verification practices is to establish and maintain the "chain of custody" of all production media, ensuring that files are transferred accurately, securely, and without corruption from the original source to Netflix. To maintain this chain of custody, the original manifest created at the time of the first offload should be checked again with every subsequent copy. This process is essential for ensuring the quality and completeness of the media throughout post-production.
By verifying the integrity of every file at each step, productions reduce the risk of data loss, corruption, or inconsistencies that could disrupt timelines and compromise the final output. Netflix performs a final check on the assets to provide productions with visibility into the health of their archive, but it is the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered to Netflix and the original manifest created on set.
Keys for Ensuring Success
- Use a Compatible Data Management Tool: Select professional data management software that creates ASC MHLs compatible with Netflix, automating the offloading and verification process. If you have any questions about which software to use, please contact your Netflix Representative.
- Ensure Data Integrity: Files must be verified during offloading and after each copy to confirm the media remains intact and unaltered.
- Sealing Rolls: Once the MHL is created, consider the roll sealed. Avoid modifying the files or folder structure, and if changes are necessary, update the MHL accordingly.
- Final Check at Netflix: The Footage Ingest Application performs a final verification on media delivery. Productions are encouraged to address any discrepancies that arise to ensure data integrity and compliance. It is, the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered via the Footage Ingest Application and the original manifest created on set. Netflix will not be auditing this process or the remediation of any errors.
Steps for Successful Data Management from Set to Netflix
1. Offload Media Using Compatible Data Management Software: Use a data management tool that automatically creates a Media Hash List (MHL) during the offloading process, currently supported hashes are xxHashbe64 and MD5. This MHL generates checksums for each file and verifies that the transfer was successful by comparing the original checksums to the newly created ones. It is essential that the tool creates ASC MHLs or MHL(v1) formats that are compatible with Netflix’s specifications, with ASC MHL being preferred for its improvements over the original standard. If the Netflix Footage Ingest application detects a hash format in the MHL that is not supported, and that is the only format that is contained in the MHL, you will be warned. You will be able to proceed with the upload but it will fail verification.
2. Sealing Rolls After First Offload: Once the MHL is generated, the media roll (the folder containing your files) is considered “sealed” after the first offload. This means no further modifications should be made to the files or folder structure that was verified to maintain the integrity of the media. If any changes are made, discrepancies will arise during subsequent checks against the original MHL, requiring remediation. The best practice is to avoid altering either the MHL or the files after the copy has been completed. If renaming or moving files is necessary after offload,ensure the MHL is updated as well. ASC MHL has tools that allow users to update ascmhl with renamed files, for instance.
Figure A
3. Maintain a Netflix-Compatible Folder Structure: Organize your media using a consistent folder structure that allows for easy identification and verification. Each camera and sound roll must be stored in its own containing folder (e.g., Camera_Roll/A001 or Sound_Roll/S001) and all camera and sound rolls must have a unique folder name so the Footage Ingest Application can identify them as discrete assets.
Figure B - Here, you can see camera rolls folders are contained within a single folder. This will ensure proper identification of rolls. How the folder that contains the rolls is named is irrelevant. You may also have several folders that contain rolls (some users like to separate each camera type into their own containing folder).
Figure C - Here, you can see roll folders are at the root level, and not in a containing folder. This will cause misidentification of rolls and other folders which will likely lead to upload errors and the need to remediate assets after upload is completed.
The Media Hash List (MHL) , or in the case of ASCMHL, the "ascmhl" folder, must be located at the root of each roll directory (e.g., Camera_Roll/A001/A001.mhl or Camera_Roll/A001/ascmhl/A001.mhl). Netflix also provides a folder naming template to automate tagging important metadata, such as Shooting Block, Unit, and Day Number, to streamline uploads.
Figure D
4. Verify Files After Every Copy: After copying media to new storage locations (e.g., hard drives or cloud storage), it is critical to verify the copied files against the original checksums stored in the MHL. This step ensures that the data remains intact and free from corruption. Using ASC MHL-compliant tools automates this process, as comparing the new copy against the original manifest and generating a new one is required by the spec.
5. Final Verification Upon Delivery to Netflix: When media is delivered to Netflix, the system will perform a final verification using the MHLs provided for each roll. If any discrepancies are found (e.g., extra, missing, or mismatched files), the user interface (UI) will notify the user of the specific issues.
Figure E
6. Remediation of Issues: If verification discrepancies occur, productions can choose to resolve the issue by reuploading the problematic files with the correct MHL or checksums. This final step provides the opportunity to fix any problems identified during the process, ensuring that Netflix receives complete and accurate media. It is, however, the production’s responsibility to determine whether remediation is needed in case of any discrepancies between what was delivered to Netflix and the original manifest created on set as Netflix will not be auditing this process.
Things to pay extra attention to
- Big upload directories - If your upload directory exceeds 100 rolls or a data size of 10TB the Netflix Footage Ingest application will block you from proceeding with your upload. If your upload exceeds any of these limits please split your upload into multiple separate directories and upload them independently, ensuring not to split any rolls when separating the folders.
- Uploads without rolls - If the Netflix Footage Ingest application does not find any camera or sound rolls in your directory, you will be warned before initiating the upload. This usually happens because there are no MHLs per camera roll. Please, check your folder directory to ensure MHLs are correctly placed in the directory.
-
Invalid characters - If the Netflix Footage Ingest application detects illegal characters in your folder structure, you will be warned before initiating the upload. Some characters are not allowed to be uploaded (such as escape characters like ‘\r’). Please remove these characters before uploading, especially if they are found in non-rolls directories. If the invalid characters are found within rolls, then ensure that the MHL reflects this change. Should you wish to use a folder naming convention, you can find the guidelines here.
- Illegal characters: @ # $ % ^ & * ( ) ` ; : < > ? , [ ] { } / \ ' ” | ~
- Missing MHLs - If there are rolls within a camera roll containing folder that don't have MHLs, the tool will flag it. You can still proceed with the upload, but we could not verify your media. It is recommended that each roll includes a manifest to be able to compare against upon upload completion.
- Multiple LEGACY MHLs per roll (does not apply to ascmhl) - If the Netflix Footage Ingest application detects multiple legacy MHLs in a roll you will be warned. You will still be able to proceed with your upload, but it is best, to ensure no failure in downstream processes, to ensure. Please, check each roll to ensure that there is only one legacy MHL v1 per roll.
- Hidden files - If the Netflix Footage Ingest application detects hidden files in the upload directory, you will be warned and you will not be able to proceed with your upload.