Quality control (QC) during the production phase is perhaps the most business-critical of all the QC types. It provides the first opportunity to flag issues that may be caught further downstream in the production process.
Maximizing production efficiency at scale requires automation and streamlined workflows as well as effective global collaboration across multiple practitioners, vendors and crew members. Successful collaboration relies on communicating QC notes promptly and accurately with recognizable consistent terminology.
In order to support streamlined QC workflows, Netflix partnered with a group of global dailies professionals and performed a number of interviews within the industry to create a Production Quality Control Glossary; a database of taxonomy and definitions that are consistent and applicable to all projects regardless of type, location, or scale.
For a downloadable version or translated descriptions please reference this CSV.
This article serves as guidance to accompany the Production QC Glossary
TABLE OF CONTENTS
Production QC Introduction: What is it and Why is it Important?
Production QC Glossary: Taxonomy
Production QC Glossary: Usage Guidelines
Production QC Introduction: What is it and Why is it Important?
Quality Control (QC) is an essential part of the film and TV production process and ensures that the content created not only respects technical standards but also maintains the original creative intent throughout the production lifecycle.
Netflix has well-established best practices and high expectations for asset QC as outlined in Introduction To Netflix Quality Control. Maintaining the integrity and fidelity of the captured image begins during production, long before asset delivery.
Production QC establishes quality control benchmarks by creating a protocol for flagging, communicating, and solving issues during the Production phase.
Furthermore, as an integral part of the Data Management “3-2-1” process, production QC provides production teams increased assurance that it is safe to erase the original camera cards for reuse. It also informs decisions such as moving on from a location, starting the removal of a set, or planning for a reshoot.
During production, undetected QC issues can become an unfortunate surprise, resulting in unexpected overhead costs during post-production and delivery; however, when issues are discovered and communicated properly, they can be addressed immediately, and fixes can be appropriately planned and budgeted for.
Until recently, there has been no common language for production-oriented quality control. The introduction of the Production QC Glossary now makes it possible to scale adoption of consistent QC terminology.
Production QC: Best Practices
As explained in Production Assets: Data Management, Netflix encourages production crews to assign and perform QC at different stages and with different crew members. A comprehensive full QC process can only be achieved with collaboration between cross-functional teams. These stages have been designed to minimize the workload and share the responsibilities of the QC process from one single person/team to multiple cross-functional teams in production. Teams involved in each stage should exchange notes to achieve a complete full QC before post-production and delivery stages.
These are the three Production QC stages, designed to work together, in this order:
-
Visual Inspection of Original Media Files During Offload: To check for any image or sound file irregularities during the first offload. The visual inspection should cover any possible major recording or data transfer issues and provide a technical check.
- Question to Answer: Is the file technically readable and usable?
- Typically Performed by: DIT (Near-Set) / Data Manager / Digital Loader
-
Full QC: To provide full feedback to key production personnel, communicate technical problems and discuss solutions. A full QC is a deeper level of QC and is meant to spot every possible issue with the content, especially technical issues. It is ideally performed on the original media files and played back in real-time on a calibrated monitor in a controlled environment.
- Question to Answer: Does the media meet the technical and quality expectations?
- Typically Performed by: Dailies Lab
-
Editorial Rushes Sign-off: To cross-reference the content with the reports from multiple departments and/or the metadata contained within the footage. Normally the Editorial department performs this sign-off by reviewing the editorial proxy media. This allows Editorial to confirm all media and metadata is accounted for and also provides an extra opportunity to check for creative or technical issues.
- Question to answer: Is the proxy media delivered to editorial as expected?
- Typically Performed by: Editorial
Although it is recommended that these QC processes are conducted during separate stages by distinct departments, there may be circumstances when these QC processes are performed by the same person or team. The type of production and/or the budget may impact the QC personnel decisions. Additional guidance on roles and responsibilities can be found in On-set Digital Imaging and Data Management: Roles & Responsibilities.
QC Database: Taxonomy
The Production Quality Control Glossary follows a specific naming convention. The following table provides an overview of minimum, custom and advanced taxonomies:
Minimum Required Taxonomy
e.g.: I-702_070
I-702_070 I = department category prefix. 702 = error code, where 7= generic subcategory error code 02= sub error code 070 = (if applicable, mandatory) references the Netflix Branded Content QC Error Code Glossary. The internal Netflix code must be preceded by a "_".
Custom Taxonomy
e.g.: I-X06
I-X06 I = department category prefix. X06 = error code, where X = fixed character "X" stands for custom error code 06 = custom sub error code Advanced Taxonomy
e.g.: I-702_07:?TR0#A[09:10:22:03-09:10:23:10]
I-702:?TR_070#A[09:10:22:03-09:10:23:10] I = department category prefix. 702 = error code, where 7= generic error code 02= sub error code : =Base Taxonomy > Advanced Taxonomy separator TR = (optional) error qualifier, to be used for additional information. Suggested use cases: location of the error in the image (e.g. TR: Top Right) or other qualifiers (e.g. F for Focus,I for Iris, Z for Zoom, where specific lens issues apply to specific lens axes only). Qualifier code must be preceded by a "?". 070 = (if applicable, mandatory) reference the Netflix’s internal Branded Content QC. The internal Netflix reference must be preceded by a "_". A = (optional) Severity level (suggested priority index is A, B, C, …). Priority level must be preceded by a "#". [09:10:22:03-09:10:23:10] = (optional) references TimeCode (TC) for the error. If the QC error spans across multiple frames, two TCs can be defined and must be separated by a "-". If only one TC is indicated, it means that the error happens on a specific frame, or it indicates the first occurrence of the error in the clip. TC indicator must be bracketed within "[ ]".
Examples of possible combination of base+advanced taxonomies:I-702_070:?TR#A[09:10:22:03-09:10:23:10] I-702_070:?TR[09:10:22:03] I-702:?TR I-702:#A I-X01 I-X01:?TR#A[09:10:22:03-09:10:23:10] |
Categories and Sub-Categories
The QC error codes have been divided into categories and subcategories to help distinguish the source or nature of a problem. Each category represents a production department or stage and each subcategory represents a specific topic within that department. The department prefix category indicates the production stakeholder (e.g. Focus Puller, Boom operator, DIT, etc.) that is most closely associated with a specific error. This helps the QC operator know which production departments to inform in case of errors that emerge during QC.
Departmental Codes Prefix Table (as currently implemented):
|
CODE PREFIX |
DEPARTMENT |
|
G |
Grips |
|
I |
Image |
|
L |
Lighting/Sparks |
|
C |
Camera |
|
A |
Audio/Sound |
|
V |
VFX/Vprod |
|
M |
Misc. |
|
F |
Film |
The department prefix category is followed by subcategory and sub error codes. The subcategories can have one or multiple error codes. Each subcategory error code is defined by a one-digit generic code, followed by a two-digit sub error code. The sub error code offers an additional level of specificity for the QC operator. A sub error code of “00” is considered the “generic” error code.
Example Code: I-702
- “I” represents the Image department category
- “7” represents the seventh subcategory
- “02” represents the Second sub error code within the seventh subcategory
The QC operator and key stakeholders should determine the level of detail to cover during QC and then communicate those decisions to all relevant parties. The level of detail is dependent on a number of factors including severity, commonality and frequency, as described in the implementation guide below.
Minimum Required Taxonomy
The minimum required taxonomy for the error code database to work must include the definition of the category, the subcategory and the sub error code. If applicable, an additional reference to the Netflix Branded Content QC Error Code Glossary may be included. This additional error code appears when a code already exists within the Netflix Branded Content QC framework for a given error. This ensures that delivery and archive QC errors spotted throughout the stages of production can be recognized immediately when they are flagged as potential delivery blockers to Netflix. The internal Netflix code must be preceded by a "_" separator.
Example Code: I-702_70
- “I” represents the Image department category
- “7” represents the Seventh subcategory
- “02” represents the Second sub error code within the Seventh subcategory
- “070” represents the code found in the Netflix Branded Content QC index
Custom Taxonomy
The production QC database is meant to be a living document and will therefore change over time as a result of community and industry feedback. It is also expected that there will be QC errors that are not yet represented as official codes in the database, but may still be relevant to flag to the production. For this reason, there is an allocated custom error code subcategory that starts with "X", where “X” can be replaced by other characters the same way as the example numbers are replaced by other numbers. The QC operator, in agreement with the relevant production stakeholders, will be able to create and use custom show-specific error codes by using a fixed characters as a subcategory indicator instead of one of the pre-allocated numbers.
Example Code: I-X06
- “I” represents the Image department category
- “X” represents a custom subcategory error within the Image category
- “06” represents the sixth custom error code implemented for the show in that specific category
Advanced Taxonomy
In addition to the minimum required taxonomy, there are a series of optional attributes that can help identify the nature of an error and specify more details. These optional attributes can include the error’s spatial location within an image and its temporal location within a clip. These additional optional elements can be used selectively and independently from each other and can be identified by their exclusive separator. The separators are unique to each type of advanced attribute so that the QC system does not need to rely on a specific order or character count to identify the separator’s intended meaning.
Advanced Optional Code Types
- ADVANCED TAXONOMY SEPARATOR: “:” will be used to separate the minimum required taxonomy from any additional and optional advance code below described.
-
ERROR QUALIFIER: To be used for providing additional information about the error. Suggested use cases: location of the error in the image (e.g. TR: Top Right) or other qualifiers (e.g. F for Focus, I for Iris, or Z for Zoom, where specific lens issues apply to specific lens axes only).
- Qualifier code must be preceded by a "?" separator.
- NETFLIX BRANDED QC CODE: (if applicable) To be used to reference the Netflix Branded Content QC error code index.
-
PRIORITY/SEVERITY LEVEL: Indicates the level of severity or priority of the error. See section below about Severity for more details. Suggested priority index is A, B, C, …).
- Priority level must be preceded by a "#" separator.
-
REFERENCE TIME CODE: indicates the temporal position of the error within the clip using SMPTE Time Code (TC). TC is specified as HH:MM:SS:FF (HOURS:MINUTES:SECONDS:FRAMES). If the QC error spans across multiple frames, two TCs can be defined and must be separated by a "-". If only one TC is indicated, it means that the error happens on a specific frame, or it indicates the first occurrence of the error in the clip.
- TC indicator must be bracketed within "[ ]".
"All-good" tag
In order to specify when content has been QC'd without any flagged errors, it is recommended to use a "000" tag in the QC report. This will assure the stakeholders that every clip of the content has been viewed by a QC operator.
The above taxonomy aims to be both human and machine-readable in order for QC practitioners to automate their workflows using techniques ranging from simple hot-key mapping to machine-learning algorithms.
QC Database: Usage Guidelines
Severity and Frequency
QC errors have varying levels of severity. We suggest implementing tiered levels of severity (low, medium, high) with distinct policies for each of them (from simply typing it in the qc notes in the reports, to calling the producer directly). Severity can be dictated by an error code being common (predictable and/or expected) or uncommon (unpredictable and/or unexpected) as well as the general sensitivity or creative intent of the stakeholders.
For example, simple camera buzz is not likely to be considered as severe as a corrupted clip; however, this can be very relevant to the nature of the shot: a camera buzz while the main actress is delivering her best line is more severe than a corrupted clip of a false take. Or the severity can be relevant to production choices: an action movie might want a lot of camera buzzes. Early alignment with the producer before shooting is key to determining the level of severity for QC reporting as well as establishing proper action plans for issue resolution.
It is important to define with production what needs to be considered more or less severe, and what action a highly severe issue should trigger. For instance, who needs to be notified? A specific Head of Department (HOD) might only be informed by QC error of a certain severity. Some errors might require the QC operator to call the producer in the middle of the night. It is really up to each production to decide this. Sometimes it does take a few days to get the protocols sorted since it's difficult to define what is acceptable and what is not. The person responsible for reporting the Production QC should check with relevant stakeholders (shooting crew, HODs, editorial, producers, etc.) during the first few days of shooting to determine if the severity of each error code is matching their expectations.
It is strongly recommended that the dailies lab or editorial department sends QC notes to the production crews at the end of each day, along with their routine end of day reports. QC notes may need to be sent more often, depending on the severity of the issue.
Example End of Day Report
How to Send QC Notes
There are many ways to communicate QC errors. End of Day reports represent the most common way, but also internal ad-hoc communications via email, or sent with editorial rushes as text notes or metadata are also useful. These should all be evaluated by the production team at the beginning of each show, along with the relevant list of people that should receive the QC notes (distribution lists). In order to streamline production efficiency, production QC notes should be embedded as metadata in the media files or as sidecar files such as ALEs, EDLs, OTIO, and sent to the next production stages so that the notes can be leveraged by downstream teams. If QC notes do not make it downstream to the next production stage, there is no benefit from QC notes.
Given that the Production QC Glossary uses machine readable naming conventions, some might use these codes to trigger specific pipelines within their workflows or to filter the footage (e.g. perform manual VFX pulls if some media metadata is incorrect; or filter every clip that is marked as “sensitive material”).
Real-World Examples from Production
Taking early action in responding to QC flags is the only way to address issues promptly and avoid additional unexpected costs. Here is a real-world example of how effective QC reporting can help contain issues while shooting, on a daily basis.
Number of QC Flags on a Netflix Production Title