In the early stages of the computer race nobody could have predicted its explosive growth. The speed in which information (good or bad, useful or useless) is generated, shared and transmitted is unprecedented. Even with today’s knowledge, saddled with trillions of streams of data, the future looks a whole sight scarier.
Everything is going digital; into computer files that will be stored somewhere.
This article will try to compare different file formats, in the hope of discovering the best archive file format for video. Before you start thinking of archival, you need to know the answer to two critical questions:
- What is valuable to you?
- How long do you want it to last?
What is valuable to you?
Only you can decide what is valuable. You could be a filmmaker, a government or corporate employee, a scientist, or just a person. For some people, even the most treasured works of humanity are low priority when compared to their personal ‘stuff’.
I take it you know how to prioritize what is valuable to you, and why it is so. Be careful when you draw up this list, because there is a definite cost associated with each item. The more there are, the more expensive it gets. Think of Einstein’s space-time. If an object takes more space or time, it needs more resources to continue its existence, and this is true even of digital files. Bigger files need more space and resources. Storing files for longer needs more resources.
The definition of archival
Depending on whom you ask, you will get many definitions of archival. According to Wikipedia, the definition of archival by archivists is in the order or hundreds of years; while the definition of archival by computer engineers might just be in the order of a few tens of years.
E.g., LTO backup systems are designed to last for about 30 years. For some, this is a sufficient archival period.
But not for me. If 30 years is archival, then what do you call preserving work for hundreds of years? Archival. When I use the term ‘archival’ it implies a storage system that lasts for as long as current technology allows.
An archival solution is one which attempts to store information for the longest period. The best archival solution must store this information longer than any other solution, without corrupting or changing it.
The best archive file format for video must preserve the video in the best possible quality for the longest duration, while ensuring maximum compatibility. This last part is tough to guess, due to the unpredictability of computer technology. Just remember this: Human history has shown that anything of value can only be preserved with great sacrifices. Nature isn’t that kind to things that want to stay beyond their use-by date.
This is where digital technology has an upper hand. Unlike analog signals, digital has the potential to stay uncorrupted – as long as it is copied to a newer file format.
I’m not going to create a comprehensive list, but I’ll try to add as many formats as I can remember. If a particular file format isn’t on the list, please feel free to do your own research.
The following chart lists many file formats for images and video, and a few properties that will help us compare them (click to enlarge):
Important: The information in this chart isn’t meant to be accurate. Things are slightly more complicated than I make them out to be. For exact and correct values, please contact those responsible.
First of all, why make such a chart? It is bring into focus how difficult it is to pin down the right file format. If we wanted to preserve a piece of paper, we could look at the appropriate technology and hit the ground running. Where file formats are concerned, the answer isn’t that cut-and-dried.
What can we learn from this simple comparison? Here are some pointers:
- A file format with a free license (zero royalty and unlimited rights) is preferable.
- A file format that offers the maximum flexibility in file size, resolution, color and bit depth is preferable.
- A file format that supports the most color spaces is preferable.
- A file format that supports all kinds of compression, and can also accept fully uncompressed sources, is preferable.
- A file format for video must also support audio, in all its forms.
- A file format must support flexible metadata.
- A true archival file format must give the greatest odds of compatibility in the future. We can only guesstimate this from current trends.
- The file format must be compatible with all popular operating systems.
The best archive file format for video?
Here’s my opinion: Based on the chart prepared, I feel TIFF is the ideal format for archival. Here are my reasons:
- TIFF is almost royalty-free
- It can take 32-bit float images, which is as good as it gets
- It can accept all kinds of compression
- It can act like a wrapper, and is the basis for metadata design
- It supports high resolutions and most color spaces
- It is ubiquitous – there’s hardly an application, software, program or library that can’t deal with TIFF
When you look at the others, they ‘fail’ on certain counts. You might not think of them as failures, and we won’t know for sure until we’re both dead.
As an aside, here are some reasons why I rejected the others:
- RAW always needs debayering, and cannot be the ‘best interpretation’ of your work. It is prone to ‘interpretations’.
- All file formats that cannot support truly uncompressed video is lossy by nature, and future manipulations will be on a lower quality ‘master’.
- Proprietary file formats aren’t guaranteed to last, nor are they ubiquitous.
- Transport streams and multiplexed video formats are at the borderline between acceptable and unacceptable quality.
- OpenEXR is great for visual effects, but like RAW, it is unbaked stuff, open to interpretation. If you really want to store data in separate channels, OpenEXR is acceptable. The major disadvantage is that EXR isn’t ubiquitous.
Important disclaimer: Don’t go about archiving your precious work in TIFF just because I say so. If nothing else, I hope this article has stressed on you the importance of giving the subject a lot of thought. Consult professionals who are experts in the field of archival, and take their help. This article is only written as an information piece.
Do you think my choice of archival format is right? If not, which format is better in your opinion, and why?