The Workflows and Advantages of Working with Image Sequences in Video Post Production

This article highlights the importance of using image sequences in post production. Not all workflows will benefit from image sequences, so it pays to understand when you need image sequences and when you don’t.

If you’re having trouble following, please click the links to the terms and read through them. You only have to understand these things once!

What is an image sequence?

In the world of video there are two kinds of files:

  • The video file. E.g., a Quicktime movie. Practically a single file with multiplexed audio and video content.
  • An image sequence – multiple still images in a folder. Each image represents a frame in the final video. Audio is a separate file.

You could use any image format in an image sequence, though some are more popular than others:

  • TIFF (*.tiff wrapper)
  • DPX
  • JPEG
  • JPEG 2000 (*.jpf or *.tiff)
  • PNG
  • OpenEXR
  • CinemaDNG
  • Any other camera RAW format
  • TGA (*.tga, similar to TIFF but not as ubiquitous)

Just so you have an understanding of the differences, here’s a handy chart:

Format^ Max. Bit Depth* Channels, HDR Size per frame (MB) 8-bit** Data rate for 1080p @ 24 fps (MB/s) Data rate for UHD @ 24 fps (MB/s)
PSD 64 Yes 100% (+layers) 142 570
OpenEXR 128 Yes 100% (+layers) 142 570
TIFF 32 Yes (TIFF Float) 100% 142 570
TGA 32 No 100% 142 570
PNG Uncompressed 64 No 100% 142 570
PNG Lossless 64 No 50% 71 285
CinemaDNG 32 HDR only 35% 50 199
JPEG 2000 Maximum Quality 16 No 35% 50 199
JPEG Maximum Quality 8 No 9.5% 14 54

Notes:

  • *From Wikipedia. You are extremely unlikely to go over 32-bit for video. For 99% of work, 16-bit is more than enough.
  • **This is an approximation based on one sample image compressed in Photoshop CC. The actual size will vary based on its content for most compression algorithms. 
  • ^We are only comparing raster formats here. Vectors have extremely rare uses, and when they do, image sequences are the way to go.

As you can see, some images sequences have the advantage of small file sizes, while others can retain higher color information, while others support multiple layers. Each has uses, though from amongst the bunch CinemaDNG and other RAW formats are unpractical. Firstly, few programs can compress to these formats. Secondly, they don’t retain any changes you made without supporting sidecar files.

This will become more clear in the next section.

[Banner468]

At which point do image sequences appear in a workflow?

Here’s how file formats typically work in a post workflow:

 File Format Workflow Path

Your camera records to a format, the camera file format. This can be a compressed codec or RAW or sometimes even an image sequence. Once you complete the edit and post process, you will create the Master, the best representation of your work, in the highest quality you desire.

These two formats are mandatory, and are in blue.

The intermediary format is a transcoded format that takes on two forms:

  • Proxies – smaller versions of the camera file format. The camera file format will be relinked to these later.
  • Intermediary codecs – compressed versions of the camera file format. The camera file format is no longer required.

In either case, the intermediary format is temporary, and not always necessary.

After you create the master, you might want to create further versions for various delivery channels like the Internet, DVD, cinema, broadcast, etc. These are delivery file formats. Sometimes, you’ll simply create a DVD or Internet video file directly from the timeline; this is why delivery file formats are also in orange like intermediary file formats. They are not mandatory.

All four file formats can be saved for later, and this is the archival file format. The archival format can take the following forms:

  • Camera file format – stays as is.
  • Intermediary file format – stays as is.
  • Master – I recommend TIFF.
  • Delivery – stays as is.

So, where do image sequences fit in? Any stage, really! When you’re shooting RAW you’ll be dealing with image sequences in the camera file format. As written above I prefer TIFF for archival. The idea of this article is to know whether image sequences will work as intermediary or delivery formats, and whether it is worthwhile mastering to an image sequence.

To know that, we’ll need to go over the advantages of image sequences.

The advantages of image sequences

Here are the advantages:

As the Master:

1. You get the best image quality possible in its uncompressed form.

2. If there are errors in a few frames once the rendering process is complete, you only need to rerender those frames.

3. If your system or application crashes during the render, you only have to start from the point of failure. In a video file, you have to start from scratch.

4. Audio that is out of sync need not be rendered again. If the problem is dropped or corrupt frames, only those need to be rendered again.

5. Image sequences can go beyond video standards and can contain HDR imagery. E.g., OpenEXR can contain as much as 30 stops of dynamic range.

6. Image file formats are the most widely recognizable file formats, and can be opened by most imaging software.

7. Projects with weird aspect ratios (video walls, vertical media, etc.) need image sequences, since regular video formats don’t support them.

8. Projects with weird frame rates must be exported to image sequences, since most video formats stick to the traditional frame rates only, or are limited by the application.

9. In the future, if a few frames become corrupt due to data loss, the entire project isn’t destroyed.

As an intermediary for the post process pipeline:

1. Playback is smoother because the CPU isn’t required to debayer or decompress data on the fly.

2. Calculations for visual effects are more precise because the images will not contain artefacts that interframe codecs have. This only applies to uncompressed image sequences.

3. Color grading is more precise due to 32-bit floating point math.

As the delivery format:

1. Many image file formats are devoid of licensing headaches.

2. You can pull high resolution stills (especially from 4K formats) for various purposes, without having to go back to the application to render these frames.

3. Adding titles, notices, advertisements, etc., are easy because you just tack on the images at the right places. For video files, you’ll need to rerender the entire thing from scratch or at least revisit the application.

If you know of more, please tell me below and I’ll add to this list.

The main disadvantages of image sequences

Image sequences are not without disadvantages. Those who work with image sequences should be fully aware of the most important ones:

  • File sizes tend to be large, sometimes astronomical.
  • To playback image sequences in realtime you’ll need fast disk arrays.
  • Uploading to the Internet is a pain due to large file sizes.
  • Playback is not easily supported by media programs.
  • You’ll always have to sync audio at the beginning.

[Banner468]

Recommendations

Based on what we’ve covered so far, here are my suggestions on when to use image sequences:

Camera file formats – Definitely! Sometimes that’s the reason we bought the camera in the first place.

Master – Definitely! I fully support 16-bit TIFF as the master format. A 120-minute 1080p24 project will take up about 2 TB. A 120-minute UHDp24 project will take up 8 TB. 8-bit will be half these sizes. DPX is also popular, but I don’t know for how long.

Archival – as mentioned above. Original files in their native formats. Compressed files and masters as TIFF.

Delivery – only when the client asks for it, or if you’re creating a DCP for cinema (JPEG 2000). Otherwise, delivery formats are usually interframe codecs (H.264, Prores, HDCAM SR, MPEG-2, etc.).

Intermediary:

  • To smooth your editing experience – no, you’ll be better of with either H.264 proxies or Prores/DNxHD intermediary formats
  • VFX – you bet! OpenEXR or TIFF
  • Grading – you bet! TIFF or DPX
  • Audio – nope, it’ll halt your sound engineer’s or composer’s computer in two seconds flat. H.264 low resolution is great.
  • Passing around to other facilities for various reasons – nope, they’ll cry if they haven’t read this article.
  • For small or unimportant (you decide) projects – nope.
  • For important projects (like your short, documentary or feature that you want to cherish for ever) that don’t have heavy VFX – not really, but master in TIFF.

What do you think? I hope I’ve taken the fear of image sequences away – or have I made it worse?