Welcome to the Netflix Partner Help Center. Have a question or need help with an issue? Send us a ticket and we'll help you to a resolution.

Menu

This article explores common challenges productions may face when using virtual production techniques, ways to diagnose the issue, and maps out potential solutions. For a glossary of common terms related to these challenges and virtual production techniques as a whole see the Virtual Production Glossary.

Additionally, Netflix has produced a series of informative videos and modules designed to provide valuable insights into virtual production methodologies. 

 

TABLE OF CONTENTS

Common Challenges

 

COLOR SHIFT

Due to both the way that LED panels are currently built and the fundamentals of how light behaves, a color shift of the content might appear when looking through the camera versus what you see with your eyes. The shift can impact the whole LED wall evenly, or just a portion of the wall. The shift can be static or it can vary over time.

DIAGNOSIS

To diagnose this issue, the best approach is to use the camera (even if it can be seen by eye). If the issue appears only when the camera is positioned off-axis, then it means that the camera is outside the optimal viewing angle of your LED screen. This can be checked on the LED panel spec sheet. If the camera is straight/perpendicular to the screen, and the shift appears evenly on the LED wall or a section of LED panels, then the shift might be caused by overheating. This occurs when the screens have been on for too long, if bright content is on the screen for too long, or from poor heat dissipation/conduction of the screen.

POSSIBLE SOLUTION

If the shift is caused by an off-axis camera position and changing the shot to get on-axis isn’t an option, where possible try to pan/tilt the screen to be more parallel to the camera sensor.  LED panels with a larger viewing angle could also solve the issue.

To determine if the panels are overheating, most LED processors can tell you the temperature of the individual LED panels or even sections of the LED wall, consider switching the screens off for a period of time, blackout/fade down the panels,  or reducing the overall brightness between takes or natural breaks during shooting.

 

COLOR BANDING

Color banding occurs when an inaccurate representation of the color of the content generates abrupt changes between shades of the same color, instead of gentle gradients. This is commonly caused by a low bit depth of the video pipeline and/or of the content.

DIAGNOSIS

To diagnose this issue, start with the source media. Then, ensure the chain from the content player to the image processor to the panel is maintaining the intended signal bit depth throughout the pipeline.

A 10-bit minimum is often required to avoid banding, and a synthetic test pattern can be generated from within OpenVPCal to test the bit depth of the imaging chain from the media server to the LED wall.

Any bands within the test pattern indicate the signal chain is not end-to-end 10-bit.

POSSIBLE SOLUTION

If the problem is caused by the media, a possible solution may be to change the encoding codec or increase its bit rate or bit depth. If the problem is within the video pipeline, some settings changes might be required on your content player or image processor. You may also need to change hardware to maintain the right bit depth. In both cases, a possible solution might also be introducing RGB dithering to the content.

 

CONTENT COLOR MISMATCH

There are a number of elements in the image chain that can affect what the content will look like on the LED wall. Almost always, the way it looks through the camera is not what we expect. This can be caused by two major elements: 1) a misleading expectation of what the content should look like or 2) a failure in how the color pipeline is set up (color pipelines are designed to make the content look as expected on camera).

Content color mismatching issues usually fall into one of three categories:

  • Strong hue shifts on the image, mostly caused by incorrect calibration or misinterpretation of the color space of the content
  • Incorrectly perceived contrast, mostly caused by a gamma (EOTF/OETF) mismatch
    • Crushed/clipped values in the shadows or highlights
    • Lack of contrast and milky blacks
  • Oversaturated or desaturated colors, mostly caused by gamut/color space mismatches

DIAGNOSIS

The expectation is the most crucial thing: LED walls have to be considered light emitters, not displays. The difference is rooted in  the fact that the observer is not our eyes and brains, but the cameras and that cameras react to light in a different way than the human visual system. This is due to a metameric failure, a phenomenon based on the concept of metamerism. Camera metameric failures are easy to spot, as they mostly exist within what the camera “sees”. If the color mismatch is visible by your eyes as well, the cause might be somewhere other than the camera in the chain.

Please note: If the images look wrong to your eyes, but good on camera, don’t worry! It’s a success as you have reached the required metamerism on camera!

If the images look wrong on camera and also look bad to your eyes, then the issues can be found in one of the following elements (from content to camera): 1. the content color metadata; 2) the media player; 3) the image processor; 4) the LED wall calibration; 5) the white point of the wall vs. the scene lighting and the relative camera white point; 6) the camera color space. 

DEEP PIPELINE DIAGNOSIS

Diagnosing these pipelines end-to-end can be a challenge. They are complex because we are working with numerous different pieces of equipment. The diagnosis process is best broken into two halves: first, a fundamental baseline round trip without the production content to ensure the end-to-end imaging chain is valid.

Once this is established and verified without artifacts then we can move onto the specifics of the content itself.

There are diagnostic patterns that can be generated as part of OpenVPCal to aid in the discovery and diagnosis of most of the common issues.

 

When the baseline is established, proceed to move on to the content-specific issues, there are a number of areas to investigate:

 

  1. CONTENT: Try to playback the content on a different system or media player, if it looks as intended, proceed to the next step.
  2. PLAYBACK SYSTEM: If your content player is interpreting the colors of your content incorrectly, that can be spotted within the media player's UI. It is also possible (and more accurate) to intercept the output signal of the content player by plugging it directly into a monitor (normally, the transport protocol is either HDMI, DisplayPort, or SDI). If the signal looks right, proceed to the next step.
  3. IMAGE PROCESSOR: Image processors use the metadata tags coming on the carrier protocol and interface (e.g. the HDMI or DisplayPort cable) to establish how to treat the incoming colors. Make sure that these match your expected color space and transfer function. It’s recommended to manually set these rather than use any Auto/From Input settings
  4. LED WALL: If your walls have been calibrated, make sure that the target calibration matches with the expected output of the image processor. Additionally, make sure that the white point of the screens matches that of the content (usually D65).
  5. LIGHT AND CAMERA SETTINGS: If the scene is lit by other light sources on set and some of those lights need to be perceived as white sources by the camera, the cinematographer will most likely offset the camera white point to match one of these lights. If that white point is completely different from the one from the wall, then the perceived picture will look shifted (either warmer or cooler than expected, sometimes also either more magenta or green).
  6. CAMERA COLOR SPACE: If a calibration process between the camera sensor and the wall has been performed, this will most probably be based on a specific color space on the camera. If that changes, the calibration might be off and colors will look incorrect or different. If you have not performed a camera to led wall calibration it is almost certain the camera will observe the led wall differently to your human eyes. Netflix has open sourced a tool called OpenVPCal which is designed to correct this through a calibration LUT, however other tools do exist on the market

POSSIBLE SOLUTIONS

  • If the content on the LED looks right to your eyes, but not on camera, it is due to a metameric failure. The solution is to offset the colors of the content in order to look right on camera (and probably wrong by your eyes). This can be done creatively or mathematically via standard color transform operators such as 3D LUTs to apply to the content playing back on the LED panels. See OpenVPCal
  • If the problem is on the content side, then it will likely need to be re-rendered or re-exported or the metadata should be changed to reflect the correct color instructions.
  • If the problem is in the media player, this might be solved by forcing the content player to interpret the correct colors or, worst case changing the content to match what the content player can interpret.
  • If the problem is on the image processor, most of the time the color tags on the processor can be altered and overridden. If that’s not possible, the content and the content player output should be adjusted accordingly.
  • If the problem is in the LED wall calibration, the LED vendor may need to re-calibrate the walls. This might take some time and might not be possible once the LEDs are setup and installed in a volume stage. Please make sure that the screens are calibrated before installing them. If the calibration has been done and it’s not in line with the content color pipeline (and a new-calibration is not possible), then you should change your content color pipeline accordingly.
  • If the problem is the lighting difference and the white point of the camera, shift the LED wall white point to match the ones on the camera (although this is not suggested) or change the content white point to match the desired one (this can be done with grading tools or color transforms to be applied to the content before rendering or on the Media Player). Another option is to change the on set lighting white point to get closer and match more the one of the content and the LED pipeline.
  • If the problem is the camera settings, please revert to the ones used during the camera calibration and, if not possible, perform a new calibration based on these new settings.

 

OPTICAL MOIRÉ PATTERNS

A moiré pattern is an image artifact that is generated when two fine patterns interfere with each other.  When shooting LED walls, the pattern created by the LED on the LED panels might conflict with the camera sensor photosite pattern and create visual artifacts such as color banding and multi-colored stair-stepping artifacts (a form of aliasing).

DIAGNOSIS

Moiré artifacts can be only seen by the camera. In fact, these artifacts can only be accurately diagnosed when all the elements of the image pipeline and shooting equipment are in place. Moiré patterns are influenced by a combination of (in order in the image chain): a) LED panel pixel pitch; b) any glass, thick smoke, gel or silk that might be between the LED panel and the camera; c) distance/angle of the camera to the LED panel; d) shooting lens and/or filter package;  e) aperture and focus point of the lens and the resulting depth of field; f) camera sensor structure and its OLPF.

Your eyes might see some moiré, but it is not the same as the camera sees (see image aliasing artifacts for more context on this). Other combinations of camera, lenses, filter and any of the above elements will possibly see some moiré, but this might also appear differently when any of the above changes.

Due to a number of factors, it is not possible to accurately predict exactly when moiré will occur, and is another example of why preparation time is critical on the volume.

POSSIBLE SOLUTIONS

All of the possible solutions to fix a moiré artifact will have ramifications on the shoot. The best immediate solution is to reduce the depth of field so that the background LED pattern becomes blurred, reducing or eliminating the interference. Another suggestion might be to find a different angle from which to shoot the LED wall, or to change the distance between the camera and the panel. A movement as little as 1cm or a single degree can be enough to remove the moiré. An ultimate solution could be to change the pixel pitch of the LED wall, but be mindful that if it is true that choosing a smaller pixel pitch will indeed alter (and most time reduce) the problem, it might not solve it completely or might introduce some side effects (eg. reduced overall brightness capability of the screen, increased LED cost and power consumption, different/increased resolutions, etc.). The moiré could simply reoccur at another position and angle on the stage! 

 

IMAGE ALIASING ARTIFACTS

Similar to moiré patterns, there are a number of image-related artifacts that can affect content. Some of these fall under the category of aliasing artifacts. These appear as jagged/stair-stepping lines around the edges of some elements of your content or between high-contrast lines and/or when you notice indistinguishable signals in your content that should be different. These can be caused by either sampling issues (up or down scaling) within the image pipeline, especially within the content player or from the content player to the LED screen (specifically within the image processor), or from poor content creation: poorly captured images or of low quality, low capture/render resolutions or inefficient scaling filter.

DIAGNOSIS

Unlike moiré patterns, these artifacts will be visible to your eyes directly. In fact, there might be more visible  to your eyes than via the camera, so check this is an issue through the camera first!. In order to diagnose where the problem comes from, the first thing to do is to make sure that the content provided to the content player doesn’t suffer from these issues already. If it doesn’t, then try to make sure that the resolution of the content provided matches the intended output resolution of the content player and the intended resolution of the LED panel (eg. the content is 4K, the content player outputs 4K, the image goes to a 4K section of the LED wall). The issue will likely be in a resolution mismatch between the content and the canvas that the content player system is trying to fill, therefore it will be applying a scaling (either up or down) to the content in real time. Content player and Image Processors can scale your content in real time, but most of the time the scaling algorithm won’t be very sophisticated, therefore this might introduce some artifacts.
Adjust the mapping so that 1 pixel maps to 1 LED on the LED wall and evaluate if the aliasing has gone away, most likely it will.

POSSIBLE SOLUTIONS

If the issue is in the original content, go back to its creation chain and make sure that there are no mistakes there. This may involve re-rendering the content using a better scaling algorithm or applying a filter to improve the source image quality and limit the artifact (which often happens when the capturing quality is not ideal to begin with). If the issue doesn’t appear on the content when seen at 1:1 scale, then it’s probably a resolution mismatch between the content and some element of the chain. Ideally, the created content will fill/fit your LED wall. This sometimes means having to produce content at massive and unusual resolutions and/or to split up the content in multiple canvas to fit the project requirements. Make an effort not to apply any real-time scaling within the content player or within the image processor.  If the content is in line with the expected resolution, then the problem may be caused by an output scaling of the content player to (or within) the image processor (eg, the content is 4K, the content player graphic card outputs 1080p, the LED wall resolution is 4K). If this is the problem, try to change the output resolution of the content player or look to source a different one.

COMPRESSION BLOCKING AND NOISE

Similar to banding, these artifacts are caused mostly by compression and appear as visible “squares” on the content.

DIAGNOSIS

To diagnose these issues, start by checking the source media. If this is good, follow the pipeline down to make sure the chain from the content player to the image processor to the panel is maintaining the intended signal. Again, the diagnostic patches from OpenVPCal can help with signal checking.

POSSIBLE SOLUTION: If the problem is in the media, a possible solution may be changing the encoding codec or increasing its bit rate or bit depth. If the problem is in the video pipeline, check the settings on your content player or image processor. Another solution would be to change the hardware to maintain the right bit depth. In both cases, a possible solution could be to introduce RGB dithering to the content.

 

PLAYBACK LAG

Issues playing content back in real-time (or at the desired frame rate) can have many causes, usually related to the content playback server and/or the media. Common causes include choice of codec, resolution, or graphics card capabilities.

DIAGNOSIS

In most cases, this issue comes from the content player. First, check the content for encoding issues (if played frame by frame, there should be no jumps across each frame, a frame check pattern can help determine this). If it plays smoothly on the Content Player, check for issues on the Image Processor. Next, check if the codec and the resolution of the content are in line with the specs of the Content Player.

POSSIBLE SOLUTIONS

If the issue is within the Content Player playback capabilities, and you are unable to change the hardware, consider changing the encoding codec of the content to an alternative that is optimized for the Content Player. You could also reduce the resolution or increase compression but be mindful that this can cause other issues.

 

FLICKERING (MULTIPLEXING)

Video artifacts such as these are mostly represented as visible changes in brightness between cycles on the LED wall. These can be seen as dark static bands that lay vertically or horizontally across the wall, they can also exist as scrolling bands at various speeds, or as bright bands.

DIAGNOSIS

To diagnose these issues, the best approach is to use your eyes. If you can see a flicker on the LED wall with your eyes, it could be a power issue. If the flicker is only visible in the camera, it could be caused by a genlock mismatch/issue within the pipeline or a slow multiplexing drive of the LED panels. Start by checking whether all of the elements of the pipeline (camera, content player, image processor) are correctly receiving and accepting the genlock signal. If the flicker is static (fixed dark stripes across the screen), it might depend on the camera shutter speed - which should be on the same frequency and phase as the screen. If the flicker only appears when moving the camera fast (especially tilting up/down), it might be caused by a slow refresh rate or multiplexing of the LED wall. The refresh rate of the image processor may also be a factor.  

POSSIBLE SOLUTION

If the problem is a dynamic/moving flicker, make sure the genlock generator is feeding every element of your pipeline. Sometimes these can be wirelessly connected, but most of the time are connected via a HD-SDI. If the problem is a slow refresh rate, check if the image processor and your LED can run at a higher frequency (but make sure that this frequency is always a multiple of the base genlock frequency speed). Even if the Content Player can only operate at base genlock speed, the Image Processor can increase the refresh rate speed by multiplying each frame without losing genlock. If the problem is a static flicker (striped black bands) then check if the camera shutter speed matches (or is a multiple of) the base genlock refresh rate (eg 1/48 sec = 48Hz = 24fps) and/or modify the phase of the camera sensor scan until the lines disappear.

If you can see the issue, make sure that enough power is being provided to the LED panels, that the daisy chain between the tiles isn’t overloading the available bandwidth, and that the LED walls are not too low in gain.

If the issues are still present after checking everything else, ensure that the panels are running at full brightness and that the content is not too dark.

LEDs do not get brighter and darker, they simply refresh (turn on/off faster and slower). LED Panels also refresh from the top to the bottom of the panel turning on and off in rows, this is expressed as a scan rate. 8:1, 12:1, 16:1. This is the number of rows of LEDs turned on, 1, for a given number of rows which are turned off.

When the LED panels are too dark the combination of a high scan rate, and low led refresh rate is enough to through the led wall ever so slightly out of sync with the shutter of the camera.

Every LED Panel has a unique minimum brightness level before this artifact can appear, the multiplexing zone. The only solution in this case is to increase the brightness of the content/processor to force the LEDs to refresh fast enough to lift out of the multiplexing zone. The camera team will then also need to lift the physical lighting of the set and either stop down on the camera, or use an ND filter.

 

HALO/RIM LIKE ARTIFACT (Edge Diffraction)

When shooting on an LED volume a halo-like edging may appear around the edge of objects in front of the wall, this could be the talent or set pieces. This is an optical effect called edge diffraction.

Edge diffraction occurs when narrow-band light sources that are highly parallel (like the kind emitted from an LED wall) pass an object and suffer very little disruption on their path to the camera lens, remaining highly parallel. This phenomenon is not solely related to LED walls and can occur with any narrow-band spectrum light source emitting in a highly parallel manner.

DIAGNOSIS

The effect can often only appear when reviewing the camera footage from the stage, so again it is important to review your content via the camera. The higher end the monitoring setup the higher the likelihood of it being spotted.

POSSIBLE SOLUTIONS

There is no way to stop the physical lighting effect from occurring, but there are a few things we can do to reduce or prevent it from appearing in the camera.

Interrupting the pathway of the light to the camera lens just a little can be enough to solve the issue.

1) Add some atmospherics such as smoke or dry ice between the LED wall and the camera
2) High-end Prime lenses can be more susceptible, detuning the lenses slightly ahead of the shoot can help to soften the overall image and disrupt the pathway of the light to the sensor
3) Anamorphic lenses also tend to create more disruption to the light pathway and can alleviate the issues for the same reasons, however, bear in mind anamorphic lenses come with their own challenges.
4) Content with large sections of the same color values produces light of the same values. The more diverse the color within the content, the less likely this will occur. Obviously, certain content types, e.g., snowy mountains, are predominantly white for creative reasons, so correctly managing content throughout is critical. However, this still may result in challenges, so be aware.

To prevent this issue, open up the space around the actors and remove some of the LED walls to limit the sound bounce. If that’s not possible or convenient, soft, sound-proof surfaces can be added on set to break the sound reflection. 

REFLECTIONS

Some LED screens have matte surfaces, while others are shiny. The latter can cause your actors, props and lighting to reflect on the screen and possibly make the blacks look milky.

DIAGNOSIS

If you run a static black patch on your screen, it is easier to spot where objects are reflected. Although this is best done with a camera at any of the known camera positions, it can also be done with your eyes.

POSSIBLE SOLUTIONS

Hide objects and lights with black flags, increase the brightness of the shooting LED screen to hide the problem and/or reduce the brightness of the filling LED walls (not the ones within the camera frustum) and/or the other real lights in the scene.

 

SOUND

When shooting on a volume LED stage, the hard surface of the LED wall might bounce the sound around and create issues.

DIAGNOSIS

The sound department on-set can quickly determine whether there will be a problem, so make sure they can scout the stage before shooting.

POSSIBLE SOLUTIONS

To prevent this issue, open up the space around the actors and remove some of the LED walls to limit the sound bounce. If that’s not possible or convenient, soft, sound-proof surfaces can be added on set to break the sound reflection.

 

 

Was this article helpful?
9 out of 9 found this helpful