Snowball Edge - User Guide & Delivery Workflow
Snowball Edge Introduction
What is Snowball Edge?
Prerequisites
Active Starship Account
API Key
Content Hub Asset Requirements
Hardware Recommendations
Phase 1 - Setting Up
Connecting Snowball To Infrastructure
Firewall Configuration
IP Setup
Content Hub Command Line Interface
CHCLI Cheatsheet Generator
Execute CHCLI
Clean Snowball
Checksum Creation
Use Case #1: Assets Are Available Immediately
Use Case #2: Assets Become Available Over Time
Phase 2 - Asset Transfer
Use Case #1: Content Resides On Single Storage Volume Available Immediately
Use Case #2: Content Resides On Single Storage Volume Available Over Time
Use Case #3: Content Split Across Multiple Storage Volumes Available Over Time
Phase 3 - Return Snowball Edge to AWS
Submit to Requests in Content Hub
Appendix: Netflix CHCLI Options
Snowball Edge Introduction
This article aims to provide an overview of how to utilize a Snowball Edge device, in conjunction with the Netflix Content Hub Command Line Interface (CHCLI), for delivery of assets into Content Hub.
What is Snowball Edge?
Snowball Edge is a 90 TB data transfer device with onboard storage and compute capabilities.
Netflix has created the Content Hub Command Line Interface (CHCLI) that allows Snowball to interact directly with Content Hub.
Directly uploading picture finishing assets to Content Hub in a timely fashion greatly benefits from a high-speed internet connection. The Snowball workflow was introduced to enable partners to deliver large amounts of non-time-critical data relatively quickly without impacting their available outgoing bandwidth or the need for a high-speed internet connection.
Prerequisites
Before you can utilize Snowball Edge for delivering assets to Content Hub, please ensure you have addressed the following prerequisites.
Active Starship Account
- One Snowball device can only be associated with one single user at a time. This user’s Starship credential will be used to unlock the Snowball. This user will receive 9 email notifications in regards to the status of the device throughout the lifecycle of a Snowball.
- The user’s account needs to be added to the Content Hub project and given a role that allows the user to upload content. Please reach out to your Netflix representative if you are not able to access the project upon logging into Content Hub.
-
The data transferred via Snowball will only appear in the designated user's Workspace in Content Hub. Therefore if the wrong account is specified during the Snowball ordering phase then please submit a request via the Partner Help Center to update the recipient's email address.
API Key
- There is a 1:1 relationship between an API key and a facility.
- API keys are facility specific. All Snowballs ordered under the same facility can be unlocked using the same API key.
- An API Key needs to be generated once and securely stored for re-use.
- If you have not already generated one, review "How do I generate an API key?"
- The distinct values that make up an API key are your “Client ID” and “Client Secret”.
- Client ID: Will start with the word “backlot” and have 5 alphanumeric blocks separated with a hyphen (-).
- Client Secret: An unbroken 64 character alphanumeric string.
Content Hub Asset Requirements
- Ensure that you have reviewed the relevant "Content Hub - Preparing Assets" documentation prior to organizing content for delivery to ensure compliance with the latest requirements. This includes naming conventions and technical specifications.
Hardware Recommendations
To guarantee a smooth setup with fast transfer speeds we recommend you use hardware as described below.
- Mac Workstation
- CPU: 12~16+ cores
- RAM: 32GB+
- HDD: 50+ GB of available disk space
- Content Storage Volume
- Read speed > 500MB/s
- Network I/O Support
- 10Gb Ethernet RJ45 /SFP+ or 40Gb Ethernet /QSFP+
- Network Adapters & Cabling
- 10G network card or Thunderbolt 2 /3 to 10G adapter, e.g. SanLink2/3
- TWIN10G-TB2 Dual Port 10 Gigabit Ethernet Thunderbolt 2 Adapter
- TB2 and ethernet Cat6 cable
Please Note: Each Snowball Edge appliance might or might not ship country-specific power cables. No other cables or optics are provided. Standard C13 IEC power cables work with Snowball devices.
Examples of some common setup recommendations can be found in the Snowball Edge - Hardware Configuration Examples
Phase 1 - Setting Up
Connecting Snowball To Infrastructure
There are different ways to physically set up Snowball Edge. Please select your preferred set up based on the infrastructure in your facility. Ensure the Snowball is connected to a workstation that’s connected to the internet.
Please reference this article “Snowball Edge - Hardware Configuration Examples” for setup examples.
- Open the front and back doors, sliding them inside the appliance door slots. Doing this gives you access to the touch screen on the LCD display embedded in the front of the appliance, and the power and network ports in the back.
- Open the top door and remove the provided power cable from the cable nook, and plug the appliance into power.
- Connect the Snowball Edge device as shown above using the appropriate cabling.
- Power up the Snowball Edge device using the power button located above the LCD display.
- When the appliance is ready, the LCD display shows a short video while the appliance is getting ready to start.
Please Note: It can take up to 15 minutes for the Snowball Edge device to be boot up and be ready for use. The display of the Snowball will provide information on the status.
Firewall Configuration
Please ensure the workstation used for the file transfer can initiate outbound traffic to the following endpoints using the HTTPS protocol on port 443:
- https://all.baggins.prod.netflix.net
- https://productionsuite-ingest.prod.netflix.net
- https://meechum.netflix.com
- Port 9091 is opened for Snowball IP
Determine all three sites can be connected successfully by running the following commands in terminal.
nc -vz productionsuite-ingest.prod.netflix.net 443
nc -vz all.baggins.prod.netflix.net 443
nc -vz meechum.netflix.com 443
IP Setup
Please ensure an IP address is assigned to the Snowball by using the touchscreen on the front panel of the device.
Please run the below two commands to ensure the network connection is establish between the Snowball, the workstation and the production network,
ping [snowball_ip]
nc -vz $[snowball_ip] 9091
Please Note: If you are not able to ping the Snowball IP, please try power cycling the workstation
Content Hub Command Line Interface
CHCLI is a Netflix proprietary terminal application created for Snowball operation. You can use it to unlock Snowball devices, enable it to be used for calculating checksums and transferring data to Snowball. In addition, you may generate a report and create a Content Hub workspace snapshot of your local files and folders structures.
There are two versions of CHCLI, please download and extract the latest version (v1.48) to your local workstation
CHCLI Cheatsheet Generator
We advise you to download the Cheatsheet Generator to aid in creating the commands you will use in this process. This will allow you to centralize all Snowball workflow pertinent information in one sheet to avoid potential syntax errors.
Execute CHCLI
The first step would be to execute the chcli application. Open Terminal on your workstation. Run the below command to execute the chcli application
cd [BinLocation]
For Example:
cd /opt/netflix/chcli-1.xx.x/bin
(The version number of the CHCLI package in bold)
Please Note: Do not use the “snowballEdge” executable under the “runtime” folder. This is the default AWS client and will not work with Content Hub.
Clean Snowball
This step removes any temporary files in workstation and deletes all files on the Snowball as well as Content Hub workspace snowball folder that is associated with the job. It should only be used to prepare the workstation for the next job, or reset to initial state for the current job.
Clean command:
./chcli -action clean -email [your email/username] -clientId [Your backlot clientId] -clientSecret [secret] -snowballIp [IP Address]
When you have entered the above command line the following will occur:
- You will be prompted to select the number of the project that is associated with this Snowball device.
- The list of the projects reflects the list of the titles assigned to you in Content Hub.
- You will be prompted to enter a Snowball Job Name. Each Snowball is assigned with a unique job name. The Snowball Job Name is emailed to the recipient of the Snowball via Snowball Status Email notifications
The Snowball Job Name consists of “NFLX-[8DigitCode]”. E.g. NFLX-QXPW9599
The CLI will look similar to the following:
Checksum Creation
A checksum is required for all Picture Finishing Assets prior to the transfer. CHCLI allows you to use the root of the folder structure as the source directory when running the command. The CHCLI will cascade through the entire folder structure and will create a checksum.txt file for every folder within the Source Directory that contains one or more files.
Please note: each Snowball Edge has a capacity of 90 TB. Please ensure to calculate the file size prior to proceeding with the checksum and transfer.
Transferring assets totaling over 90 TB to one Snowball will result in a number of operational errors.
Command:
./chcli -action checksum -dir [Path/to/SourceFolder]
Below are two example use cases. While these use cases use episodic content as an example, it can apply to any type of content.
Use Case #1: Assets Are Available Immediately
All data is organized following the latest requirements outlined in "Content Hub - Preparing Assets". The intention is to generate a Checksum Manifest (checksum.txt file) for all data that will be delivered via the Snowball Edge device in a single session.
Point to the desired top-level (root) directory that contains all the content you wish to transfer. The Netflix CHCLI will then generate a Checksum Manifest for all the assets under the source directory.
In the example below a title consisting of 4 episodes is organised on a single storage volume. The folder “FakeExampleShow_s01” (highlighted red) is the source directory specified during the “checksum" action. Four checksums are then generated in the appropriate subdirectories (highlighted yellow).
Use Case #2: Assets Become Available Over Time
It is common for various assets to be ready for delivery at different times, for example, a Video Display Master (VDM) may be available before a Non-Graded Archival Master (NAM). To continue the example outlined in Use Case #1
Following on from the example in Use Case #1, two new episodes have become available AFTER the initial “checksum” action has been executed. There are two options:
Option A: Rerun the checksum action by pointing to the original source directory. In this example that would be "FakeExampleShow_s01” (highlighted red). This will regenerate checksums for ALL content and overwrite all existing checksum.txt files (highlighted yellow).
Option B: Run the checksum action on the NEW directories only (highlighted green) by executing a checksum action specifying each new folder as the source directory, one at a time. This will leave the original Checksum Manifests untouched (highlighted yellow), and only generate them for the new content (highlighted purple).
Phase 2 - Asset Transfer
The Netflix CHCLI “copy” command copies files and folders from your data source to the Snowball. Later in this guide several example use cases are outlined. While no use case is content type-specific i.e. feature vs. episodic, the examples shown outline the most likely scenarios.
The CHCLI copy command copies files and folders from your data source to the Snowball. When copying data, prepare a source path and IP address of the Snowball device, as well as security credentials mentioned above. Enter the copy command as the following:
Command:
./chcli -action copy -sourceDirectory [Path/to/SourceFolder] -email [your email/username] -password '[YourPassword]' -clientId [Your backlot clientId] -clientSecret [secret] -snowballIp [IP Address]
When you have entered the above command line the following will occur:
- You will be prompted to select a project number associated to the Snowball.
- You will see the list of titles assigned to you in Content Hub here
- Ensure you select the correct project for the Snowball being used
- You will be prompted to enter a Snowball Job Name.
- The Snowball Job Name is communicated via the Snowball Status Update emails that are sent to the user that was specified during the prerequisite phase. The Snowball JobName consists of “NFLX-[8DigitCode]”.E.g. NFLX-QXPW9599
The CLI will look similar to the following:
Once you have entered the above information the Netflix CHCLI will automatically:
- Scan and index your source directory
- Auto-populate the Snowball parent folder in the uploader's workspace once the transfer has started. Please note that it will only appear in the uploader's Workspace. It will not appear in the Workspace of other users assigned to the same project in Content Hub. The uploader is able to share the content in Workspace.
- The Snowball folder (highlighted red) will retain the folder structure of the source directory specified during the “copy” action (highlighted yellow) and add place folder entries for the assets (highlighted green)
- A blue spinning icon is used to indicate the material is still “in transit” to AWS. Once successfully ingested into AWS (after device is shipped back) the blue spinning icon will disappear.
- WARNING: Please refrain from using the “Move” or “Delete” functionalities within the Content Hub UI before all assets are fully imported into Content Hub after the Snowball is shipped back to AWS. The blue spinner is the indicator that the assets have not been fully imported.
- Calculate a checksum for every file in the provided source folder and look for a previously generated checksum.txt file (See Checksum Manifest Generation) to compare and verify checksum values.
- Initiate the data transfer to the Snowball device.
The following script can be used to introduce a degree of automation by queuing a "checksum" and "copy" action:
Command:
./chcli -a checksum -dir ${Your Source Directory} && ./chcli -action copy -dir ${Your Source Directory} -email $YourEmail -p '${Your Password}' -pid ${The Project ID} -tid ${trackingId}
The "-pid" is the Project ID that can be found in the Content Hub URL for the project.
E.g. https://contenthub.netflix.com/projects/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/requests
In an actual project, the x’s will be replaced with the Project ID.
The “-tid” flag corresponds to the "Job Name" that is communicated in the Snowball Status Update email.
Use Case #1: Content Resides On Single Storage Volume Available Immediately
All data is organized per the requirements outlined in the Content Hub -Archival Asset Preparation Guides, and it is intended to copy all data to the Snowball Edge device in a single session.
Use Case #2: Content Resides On Single Storage Volume Available Over Time
It is common for various assets to be ready for delivery at different times, for example, a Video Display Master (VDM) may be available before a Non-Graded Archival Master (NAM).
In such cases:
- Execute an initial “copy” action on a top-level folder.
- As new material becomes available simply add the new folders anywhere under the top level folder.
- When material is in place simply rerun the original “copy” action.
In the example below, it can be seen that Session 1 was run when the VDM was available. When the Graded Archival Master (GAM) became available Session 2 was performed. Later when the NAM was available Session 3 was performed.
For each session, the folder “Bright” was specified as the source directory. The Netflix CHCLI keeps track of the data it transfers and is able to identify what material in the source directory already exists on the Snowball. It will then proceed to copy across only the new assets.
Please Note: Certain assets require a Checksum Manifest to be created prior to transferring to Snowball. Please refer back to Checksum Manifest Generation for more information.
Use Case #3: Content Split Across Multiple Storage Volumes Available Over Time
It is common for content to reside on different storage volumes and become available over time rather than all at once. In the example below there are 4 episodes split across two storage volumes:
As the Netflix CHCLI is able to utilize symbolic links and decipher subdirectories it is possible to run a single “copy” action instead of running individual “copy” actions for each top-level folder spread across multiple storage volumes. To do this:
- Create a temporary folder e.g. “Snowball”.
- Within the “Snowball” folder create symbolic links to the desired top-level folders.
This will result in a directory that will be similar to the following:
Now it is possible to use the “Snowball” folder (highlighted red) as the source directory for the “copy” command”.
As more content becomes available over time additional symbolic links can be created and added to the temporary “Snowball” folder and the exact same command can be rerun when the content is ready to be copied.
The Netflix CHCLI keeps track of the data it transfers and is able to identify what material in the source directory already exists on the Snowball. It will then proceed to copy across only the new assets for each session.
In the example below, it can be seen that Session 1 addressed the original material that was ready previously, in this case, episodes 1 to 4.
For Session 2, new symbolic links for episodes 5 and 6 were added and the copy command was rerun on the "Snowball" folder. Only episodes 5 and 6 were copied during that session.
For Session 3 the same was done for episodes 7 and 8.
Please Note: Certain assets require a Checksum Manifest to be created prior to transferring to Snowball. Please refer back to Checksum Manifest Generation for more information.
Phase 3 - Return Snowball Edge to AWS
Congratulations! Now you can ship the snowball back to AWS:
- Make sure that you've finished transferring all the data for this job to or from the AWS Snowball Edge device.
- Press the power button above the LCD display. It takes about 20 seconds for the device to power off.
- Note: A shipping label should appear after about a minute on the E-Ink display on top of the device.
- Disconnect and stow the power cable the AWS Snowball Edge device was sent within the cable nook on top of the device.
- Close the three doors on the back, the top, and the front of the AWS Snowball Edge device, pressing in on each one at a time until you hear and feel a click.
- Arrange for the pickup of the AWS Snowball Edge via courier.
- The courier information will appear on the E-Ink display with the exception of those shipped in Japan, Korea and India. In this case, the shipping instructions will appear on the E-ink display. Please email the provided email address to arrange for the pickup.
- Note: Ensure your courier understands that the E-Ink display on the top of the appliance should remain free of any and all adhesive labels.
Submit to Requests in Content Hub
Once the Snowball Edge device has been shipped back to AWS, the content will be ingested and made available in Content Hub via the users' Workspace. At this point, the content is only visible to the user who performed the upload. In order for Netflix to take receipt of the material, the Snowball user is required to submit the assets to the appropriate corresponding requests.
Please ensure to submit the image sequence and its checksum in a folder. Submitting the image sequence and checksum individually may result in the checksum validation error.
For more information on how to submit content to a request from a users Workspace please review Content Hub Overview - Assets - Workspaces.
For more information on how to monitor the status of delivery after submitting the content via please review Content Hub Overview - Requests - Monitoring & Submitting.
If you still have questions after reviewing the Content Hub documentation, please reach out to your Netflix representative for further assistance.
After all assets are submitted, please be sure to communicate with your Netflix representative prior to deleting any content from your system.
Appendix: Netflix CHCLI Options
- The Snowball Workflow Cheatsheet Generator can be used to centralize all Snowball pertinent information in one customized Snowball cheatsheet. Download the zip file at the end of that article.
- The CHCLI Common Error Solutions article lists solutions for commonly encountered error codes.
- Snowball Edge - Userful Command Lines lists a number of useful command lines that can be used during the Snowball workflow for troubleshooting
Change Log:
2022-09-28
- General: Updated links to the latest version of CHCLI (v1.49)
2019-12-19
- General: Updated links to the latest version of CHCLI (v1.48)
- General: Removed the short form commands
- Phase 1: Added link to the Snowball Cheatsheet Generator (v1.1)
- Phase 1: Added instructions on how to connect Snowball to infrastructure.
- Phase 1: Added link to Content Hub - Archival Asset Preparation Guides.
- Phase 2: Added instructions on IP whitelisting requirement and firewall configuration
- Phase 2: Remove the requirement of running Finalize command on CHCLI v1.48.
- Appendix: Added links to new articles: Snowball Cheatsheet Generator v1.1; Snowball Edge - Common Error Solutions; Snowball Edge - Useful Command Lines
2019-09-19
- General: Updated link to the latest version of CHCLI (v1.47.0)
- Phase 2: Finalize command update to include -sourceDirectory flag, which was removed in error.
2019-08-07
- General: Images updated to reflect updated folder structure and naming conventions.
- Phase 2: Finalize command has been updated to reflect required syntax when using latest version of CHCLI (v1.46.0-rc.2).
2018-10-22
- Phase 2: Added note to explicitly state that a partner should refrain from using the newly release "Move" or "Delete" functionality in Workspaces while data is still transferring to Content Hub (as indicated by presence of blue spinning icon).
- Phase 2: Updated explanation of how to identify the correct value for the -pid flag when used in the outlined script.
2018-08-15
- General: Article title has been update from " Snowball Edge - Quick Start User Guide" to " Snowball Edge - User Guide & Delivery Workflow".
- General: The "Netflix Snowball Client" has been renamed to the "Netflix Content Hub CLI" and is also referred to as "Netflix CH CLI" in documentation.
- General: Executable was previously called "snowballedge". It has been renamed as "chcli" to avoid being confused with the default "snowballedge" executable provided by AWS. All command examples have been updated to reflect this name change.
- General: Reformatted "Steps" into "Phases" and added images to provide visual aids.
- Prerequisite: Provided additional context regarding "Active Backlot account".
- Prerequisite: Provided additional context regarding "API Key".
- Prerequisite: Individual IP addresses removed and replaced with https://productionsuite-ingest.prod.netflix.net which needs to be whitelisted.
- Prerequisite: "Hardware Recommendations" updated to provide clarity.
- Phase 1: Added bullet point to ensure user is using correct/intended executable.
- Phase 2: Added example script that can be used to queue a "checksum" and "copy" action.
- Phase 3: Provided additional context regarding the actions a partner must take after returning the Snowball Edge device.