The key task of any network is data transfer. A network that can’t get data around in time is a failed network.
It’s like a friend who learns about a special sale offer, and by the time her text message reaches you the sale is over. Phooey.
We are in the business of transferring images over a network. The first thing we should know is how much data needs to flow through the network. We can’t order a truck until we know the size of the elephant we want to transport.
First, read my post on the sizes and costs associated with working with uncompressed footage here.
A modern trend is to record in RAW files, which typically, when in uncompressed mode, is 1/3rd the size of full raster images. Read about that in my Deconstructing RAW series here:
Now, let’s take a look at the data rates of certain standards:
What are some of the scenarios in which uncompressed footage is transported in real-time (or faster!)?
- Playback and Dailies
- Visual Effects Facility
- Fast Backup
You’re probably thinking: there are some numbers there that are way above uncompressed footage rates. So what’s the problem?
The problem is: a chain is only as strong as its weakest link. Data transfer through a computer or network is bits flowing through a chain of devices and events, any one of which can lag behind – it’s like a relay race.
Let’s see how:
In its simplest sense, data is read from or written to a hard disk. A hard disk can only do one thing at a time.
One of the important parameters that affect hard drive transfer rates is the Average Access Time. The average access time is the time a drive takes to get its act together to begin transferring data. Imagine someone asking you your name. Even in your prime, it takes a few milliseconds for the request to reach your brain and result in an answer. Hard drives are no different.
One of the methods in which hard disk transfer efficiency is increased is by means of a Disk Buffer or Cache. On good hard drives, this size is about 64 MB, and it acts like an overflow outlet on a plumbing line.
In any case, a typical SATA drive only gives out half to two-thirds of its rated speed. This means for SATA II you’ll get about 150-250 MB/s and for SATA III you’ll get about 500-600 MB/s tops – at least with today’s technology. However, the higher speeds are typically for SSD drives, which are expensive.
We have already seen in our costs of uncompressed footage post that consumer hard disks max out at about $0.12 per GB. For SSDs though, this number is about $0.75 per GB – that’s 6 times the price.
We know our motherboards can handle an uncompressed stream easily. With thunderbolt, we finally have a cheap way to transfer files over a network. That leaves us with hard drives that only reach this level of speed if they are SSDs. If the amount of footage is less, then an SSD is great.
Most of the time, though, when working with RAW or uncompressed files, the footage time is in hours, and the data capacities required for storage are in the terabytes. On top of that, you also need to back up data to at least one other drive. That easily doubles the price, minimum.
The writing is on the wall – the bottleneck today for data transfer is the hard drive. See where this is going?
Either a solution is too expensive, or too convoluted. It is a juggling act not for the faint hearted.
A typical one or two computer setup really doesn’t need or can’t afford expensive network systems or server hardware. Surely there must be a ‘PC’ solution? Surely there is a savior that will lead us to the promised land?
Of course there is. It is called RAID.
Links for Further Research: