Last Published: Friday Nov 8, 2024
REMINDER: Context is key. The details of this article can vary based on the production and the creative intent.
Additionally, Netflix has produced a series of informative videos and modules designed to provide valuable insights into virtual production methodologies.
OVERVIEW
When evaluating the necessary infrastructure and equipment needed on set for successfully shooting in an LED wall-based virtual production setup, there is an extensive list of possible combinations. This workflow / technical specification aims to break down the equipment required into distinct parts, build relationships between them, and help our projects make better-informed decisions.
Below, we outline the minimum requirements for Netflix Partners for on-set infrastructure and on-set workflows to achieve the required standard of visual fidelity and controllability for In-Camera Visual Effects. If you are unable to meet any of these minimum requirements on a Netflix production, please discuss it with your Netflix contact and the Netflix Virtual Production team.
Details about each element of the technology are outlined in this article (panels, processors etc), however, to summarize, we expect the following to be implemented on every Netflix production as a minimum:
- LED Panels / Image Processors / Receiver cards to be fully functional, tested, and cabinet calibrated in order to enable all the internal color management functionalities and to be compatible with the minimum imaging workflow (number 2).
- Minimum Imaging Workflow: We require a minimum end-to-end 10-bit workflow, capable of handling PQ (ST2084) EOTF encoding, and the ability to target the Fully Achievable Color Primaries of the LED Wall as a custom color space, along with these standard color spaces: P3, Rec2020, Rec709.
- LED stage roundtrip testing: Perform basic Pre-Validation checks - eg. mismatching color spaces, EOTFs, frame rates, missing genlock, etc. OpenVPCal has an inbuilt Synthetic Pattern Generator (SPG) that generates parameterized test material that can be leveraged to identify and validate these issues.
- Camera-to-Wall Calibration: Camera-to-wall calibration is a critical component of a successful ICVFX shoot. This goes beyond the cabinet calibration of the LED wall and requires an additional step to calibrate the camera sensor sensitivity to the LED wall technology in use. Netflix has provided OpenVPCal as an open-source solution, however, other productised solutions are available and accepted.
- Color Management: It is essential that the system is color managed from end to end, and follows the minimum imaging workflow above. Netflix is advocating strongly for the implementation of OCIO to standardize color management frameworks.
- Key Creative Controls: It's essential that playback systems have scene-referred and creative color correction controls that allow for both technical and look adjustments. Please refer to the Playback System Elements section for details.
Questions to: virtualproduction@netflix.com
ON-SET SYSTEM FLOW : The pipeline that takes content from a Content Playback system to the LED screens passes through multiple elements, described in the System Flow.
CONTENT PLAYBACK TECHNOLOGY
For 2D content playback on LED panel technology, specific playback software is required for a number of reasons: 1) you need to arrange and map your content between panels (pixel mapping), deal with corners, ceiling, etc. and 2) you need to communicate with the LED Image Processing technology. Most of the software comes configured to work with accompanying hardware and sometimes a specialized playback codec. Typically, Content Playback systems available are provided with the hardware to use it. The hardware is specially configured to do one thing: play content of varying sizes to varying outputs in real-time. High-end workstations are required to playback 8K, 12K or 16k in real-time for extended periods of time.
When choosing a playback solution, consider the software's usability and flexibility. If the software is difficult to use or does not meet the ideal technical requirements, it won't matter how powerful the hardware is.
PLAYBACK SYSTEM ELEMENTS
| ELEMENTS | REQUIREMENT | Details |
| Codec |
>=10bit |
There are many codecs that can be used today - NotchLC, DNxHR and HapQ being popular options. |
| Color Management System | OCIO | Within VFX, OCIO is the industry standard for exchanging and applying color transforms and color pipelines to plates. Given the need in ICVFX to customize the color pipeline for each setup, Netflix is advocating for OCIO to be implemented in every Media Player system to allow fully interchangeable and interoperable pipelines between vendors and systems. |
| Color Correction Controls | Independent scene referred and creative controls |
Most playback systems will have basic color correction functionality - but it’s essential that they have two different categories of operators:
In detail:
|
| Framerate | 2 x the primary capture rate | These systems can play almost any framerate their supported codecs can playback, but make sure that the storage and image pipeline can support the required speed |
| LED Panel Mapping Controls | Flexible 3D Placement |
Ensure that the media server and LED processor, provide the ability to map pixels from the image to the individual LEDs on the wall. Ideally, the media server will also provide projection capabilities, allowing content to be virtually projected in a 3D context prior to being displayed on the physical LED wall. |
Server Genlock |
Yes, Tri-level or PTP | |
| Video Outputs | HDMI 2.0 or DisplayPort 1.2 | Make these decisions in conjunction with your Image Processor Technology. |
IMAGE PROCESSOR TECHNOLOGY
The Image Processor is the element of the pipeline that adapts, interprets and translates the video feed coming from the Playback System to the required LED data distributor (commonly a network switcher) which feeds the LED panel the correct data. In other words, the image processor ensures the correct pixel from the media server is displayed on the correct LED on the wall.. The image processor operations can vary to include scaling, color space conversions, synchronization to LED gain, white point temperature control, LED calibration and overall LED power output.
While there is almost always a data distribution switcher between the image processor and the actual panels to handle the signal flow, the performance of an LED screen is directly correlated to the ability of the image processor to drive the LED wall. The capabilities of a good image processor can also be limited by the quality of the LED screen and the receiving card. The two things are very much tied together. In a normal computer monitor, the panel and the image processor are designed to work together and built in such a way that the performance will be consistent and determined by the manufacturer. When it comes to LED panels, the user can combine many LED walls with different (compatible) image processors, thereby creating discrepancies or varying performance levels. Like the rest of the technology in this space, the modular nature lends itself to both flexibility and variability. Consider however that being consistent often results in a simpler setup and operation.
Important note: each image processor is only compatible with a specific (or a very limited set of) receiving cards which have to be installed on the LED panels. In order to change the Image Processor in your pipeline, you must make sure that the LED receiving card installed on your panels will be compatible with the new processor.
IMAGE PROCESSOR ELEMENTS
| ELEMENTS | REQUIREMENT | DETAILS | |
| Data port | 1Gbe Output | The data port is the connection between the Image Processor and the distribution units and ultimately the LED panels | |
| FrameRate Multiplication | Yes | This feature allows the image processor to multiply the incoming video frames and get more use of the fast refresh rate of the LED panels in order to achieve the best possible combination of image playback performances and smooth perceivable motion on screen. | |
| Genlock |
Sync to source over external source PTP or 3-level sync |
Genlock can be used to lock the processor and LED refresh to a camera’s shutter. This provides a better on-camera performance, by preventing the unsightly rolling black bars seen on video walls using free-running systems. Alternatively, you can use Genlock to lock multiple LED processors together for completely seamless shows with no tearing. The processor can even add extra delay should it need to match with other slower video systems. | |
| Infoframes | Supported, Static | Auxiliary Video Information (Infoframes) that carries over HDMI, DP or SDI statuses such as Color Space, HDR format, Color Primaries, Max CLL Max FALL | |
| HDR Formats | ST 2084 | Processors handle HDR EOTFs | |
| Color Space Input/Output | Rec709, P3 D65, Rec 2020, Custom/Achievable primaries | The processor can be manually set to one of these standard color spaces for both the input signal and the output | |
| Input/Output signal Color Management | Available and bypassable | Although the signal should be managed by the Media Player, it is sometimes useful to be able to manage the color mapping between the input and output color space. However, is equally important that this mapping can be disabled/bypassed entirely. | |
| Input Resolutions | Custom | Content can be created at several different resolutions. It is important for the processor to be able to receive custom resolutions. | |
| Latency | None | Latency is usually referred to in ms (milliseconds) or frames, which represents the response time or traverse time that one device takes to communicate with another. | |
Bit Depth |
10 bit | Bit depth refers to the bit depth at which video data is encapsulated, packetized and sent to connected LED Panels via Fibre or Copper | |
| Color Calibration | Yes | The image processor should have the ability to load/perform a color calibration for the LED tiles. |
LED PANEL
The LED Panel is the final element in the LED playback infrastructure. Once the content is passed through the Image Processor, the right driving instructions are sent to the LED screen so that the content can be visualized correctly. These instructions arrive as LED gain data (the power to be provided to each LED pixel on each tile of the LED wall bank) which displays the content correctly.
LED walls are made up of many individual LED panels and each tile is designed to work in conjunction (daisy-chained) with other panels, up to a specific limit. Please note that some specifications belong to the single panel, and some others express characteristics of the whole LED wall when wired together as a whole unit.
Given that LED Screens were predominantly developed for digital advertising or live concert venue/performance spaces, it’s important to seek out the appropriate solutions that can support the needs of the film and television space. Not all LED panel types are suitable for Virtual Production.
IN-CAMERA LED PANEL ELEMENTS
| ELEMENTS | REQUIREMENT | DETAILS | |
| Refresh Rate | 1920Hz |
The refresh rate is the number of times in a second that a display updates its buffer. This is distinct from the measure of frame rate which indicates when the display is provided with new data: the refresh rate includes the repeated drawing of identical frames, while frame rate measures how often a video source can feed an entire frame of new data to a display. For example, most movie projectors advance from one frame to the next one 24 times each second. But each frame is illuminated two or three times before the next frame is projected using a shutter in front of its lamp. As a result, the movie projector runs at 24 frames per second, but has a 48 or 72 Hz refresh rate. For LED lighting this happens on a scale 10 times bigger and it’s essential to find a refresh rate that can be synced with the framerate of the camera shooting the LED wall. As a rule of thumb, a refresh rate needs to be a multiple of the shooting frame rate. i.e. 1920Hz/2/2/2/2/2/2=30Hz so it syncs with 30fps. This rule can be bent by altering other factors of the shooting imaging pipeline, such as the camera shutter speed. |
|
| Contrast Ratio | 8.000:1 | The contrast ratio is defined as the ratio of the luminance of the brightest shade (white) to that of the darkest shade (black) that the LED system is capable of producing. | |
| Reflectivity | Very low | Some LED screens have matte surfaces, while others are more shiny. The latter can cause your actors, props and lighting to reflect on the screen and cause flares and make your projected content blacks look milky | |
| Multiplexing Drive | ≤ 1/8 |
Multiplexing is a technique used to connect multiple LEDs together in a matrix of columns. This manufacturing technique simplifies the hardware due to the reduced number of pins required to connect each LED pixel. Each column is switched on in sequence to turn on the desired LEDs. To fool the eye into seeing a continuous display, the sequencing is typically done at a fast speed, more than 50 times per second. Typically this feature is expressed as a fraction that indicates in how many rows the LED tile has been divided. The smaller the fraction, the more are the sections on the tile. A high number of sections can bring artifacts, mostly visible in camera as a “flicker”, similar to the one generated by an off-synced refresh rate. |
|
| Brightness | ≥ 1000 nits | This is the peak brightness produced by a LED tile. There is not a correct value for this, as it really depends on the application and your pipeline. In order to have enough power to balance the scene, the LED wall should try to produce the same, if not higher brightness, of the light sources used on set to light the scene. | |
| View Angle V/H | ≥ 140º |
The way LED pixels are clustered and arranged on a LED tile determines the optimal viewing angle of the LED tile/wall. The viewing angle (or viewing cone) defines a number of possible viewing directions in which the LED tile/wall can be viewed with acceptable visual performances. Ideally, the angle should be as close to 180º as possible but typically goes around 140/120º. |
|
| Gamut Coverage | ≥P3 | The ability of the LED wall/tile to cover and accurately reproduce the color areas defined as standard color gamuts within a CIE XYZ color system diagram. | |
| EOTF | Power Gamma and ST 2084 | The ability to support multiple power gammas as well as ST 2084 EOTFs |
Change Log
11-08-2024
- Added language directing productions who can't meet the outlined requirements to "discuss it with your Netflix contact"
09-24-2024
-
Clearly established minimum requirements for on-set infrastructure & workflows including:
- LED Panels / Image Processor / Receiver cards
- Minimum Imaging Workflows
- LED stage roundtrip testing
- Camera-to-Wall Calibration
- Colour Management
- Key Creative Controls