How Ethernet Hybrids are Blurring the Lines Between SAN and NAS Systems

NAS v. SAN. Ethernet v. fibre-channel. Decisions in the shared-storage space aren't getting much easier to make as the options available multiply and the power and performance relationships among technologies shift. Fibre-channel used to be the power-users' connection of choice, but the imminent arrival of 10 Gigabit Ethernet means it won't be the fastest cable on the block anymore. And while your gut may say you need a SAN for heavy-duty post work, there are NAS systems on the market that demonstrate, convincingly, that's not always the case. In the weeks leading up to NAB, and at the show itself, Film & Video talked to a sampling of storage vendors about how they help their customers sort through the issues. We asked power users at Post Works in New York City and Post Logic in Hollywood, CA, to talk about the moving target that is shared-storage nirvana. Finally, we got engineer Ari Zohar Klingman to write an article that frames the debate and details three different approaches to shared storage in three different environments: Nice Shoes, Guava, and Freestyle Collective. Read his piece here.

One of the splashiest launches in recent memory was Avid's roll-out of the Unity ISIS system last year. Unlike SANs, which are generally built around fibre-channel technology, the ISIS is Ethernet-based. Each ISIS engine comprises 16 ISIS storage blades (that's 8 TB of storage) and two Ethernet "switch blades" that connect to a rack-mounted server as well as to individual clients. More than two ISIS engines are interconnected using what Avid calls an integrated expansion switch blade, or iXS, which creates a network that routes any client to data stored on any storage blade in the system with real-time efficiency. The system is entirely built on Ethernet networking, which makes it a better fit in existing IT infrastructures.

Avid's Andy Dale says the arrival of the ISIS creates a two-tiered storage strategy for Avid, with the ISIS addressing users who need unflappable, rock-solid performance. "The ISIS is really going after that extreme level of reliability and availability, with very high levels of scalability," he says. "It goes up to 64 TB right now, and will be significantly higher than that by mid-2006." (At NAB, Avid said ISIS will scale up to 192 TB and up to 150 active users, offering 300 50 Mbps streams in real time.)

The intelligence behind this stuff is apparent – yank out a few storage blades and the system's performance won't even hiccup as it calmly starts recovering from the failure by redistributing the data across the rest of the system. But even the mighty ISIS is bandwidth-limited. The IT-style flexibility that comes with Ethernet networking means it's constrained, for the time being, by the bandwidth of Gigabit-Ethernet connectivity. That means that if you want to work in HD, you're either going to need compression or something like an Avid Unity MediaNetwork, which runs over four-gigabit fibre-channel. Dale, of course, recommends using Avid's high-performance DNxHD codec to save money. "When you're talking about 27.5 MB/sec per stream rather than 156 MB/sec per stream, the overall required investment in storage and networking falls dramatically," Dale says. "It's a very good tool even if you're working in lower resolutions – DNxHD 115 is extremely good quality for dailies and things like that."

So in terms of the user experience, the Isis is part Ethernet-connected NAS and part robust, powerful SAN. And Avid isn't the only vendor blurring the lines. Bright Systems, for instance, has a new line of servers called BrightDrive. They allow facilities to get bandwidth where they need it by connecting users who need high-resolution real-time capabilities via fibre-channel (they'll see the BrightDrive server as a local drive), and hooking up the rest of the facility via less expensive Ethernet connections (which will see the server as a network drive). The idea is to make a scalable system that leverages the cost-efficiency of Ethernet with the need for fibre bandwidth. "It's no more expensive than a NAS-only solution," says Bright's Peter Lambert. "There's nothing that you throw away, no penalty along the line [as you increase performance]."

Wayne Veitschegger, chief engineer at Post Logic in Hollywood, CA, subscribes to similar logic, combining online and nearline volumes from Bright to keep storage and bandwidth capacities right where he needs them. “We have three fibre-channel volumes, which provide us with nine theoretical 2K streams, and then one volume of SATA storage for cost-effective nearline storage, which is non-real-time,” he explained. “Each fibre-channel volume is easily expandable to add more spindles, adding storage and speed as we ramp up for 2K, which is what it’s optimized for. We have certain islands that are capable of real-time 4K, and, by adding another array, each volume can be moved up to real-time 4K streams using 4 gigabit fibre. The main infrastructure is running 2k, but with very little infrastructure improvement or additions, we can ramp up to 4K really quick.”

Here's how Post Logic does DI work: First, the film is scanned directly into the BrightSystems SAN, and a quick QC of the raw scan is performed using such tools as a directly attached DVS Clipster that lets the scans be previewed sequentially in real time. Depending on the project, Post Logic may next take the scans into the iQ system for a conform check, depending on whether the material has already been conformed offline. At the same time, files will be copied to local storage on Post Logic's Baselight systems, where the dirt-removal process starts using MTI Film's DRS system. When that's finished, the files are placed back onto the SAN. Color-correction can be started at any time. Screenings often take place out of the Clipster or from a Baselight system. "Our digital projector is on the video router, and we can screen from any source in the facility," Veitschegger says. Screenings take place on a Barco DP100 running at up to 2048×1080 in 4:4:4 or 4:2:2 color space, depending on the project.

"Post Logic is set," Veitschegger says. "Knowing our action plans for the future, we're good – unless something drastic changes."

System designer Studio Network Solutions, meanwhile, is a big believer in iSCSI, designing systems that behave like fibre-channel SANs but work over Gigabit ethernet. SNS uses iSCSI interfaces to scale up through SD workloads, and then bring in fibre-channel only when necessary to handle HD. It makes sense that SNS might have a different approach to shared storage – the company came from the audio space, where it gained a reputation by building custom fibre-channel systems for recording studios. About two years ago, SNS started deploying iSCSI systems for sound editing applications, and is now looking to extend its success into video post.

"We think the hybrid of fibre-channel and iSCSI is really promising," says SNS Director of Operations Eric Newbauer. "It's cost-effective, and everyone has CAT5e ethernet or CAT6 in their walls. They don't all have fibre-channel or optical cable. And we can now do things over iSCSI that used to only be possible with fibre-channel." So you can leverage your existing Ethernet experience and infrastructure in your SAN. And the company believes that iSCSI over 10-Gigabit Ethernet could make 4-gigabit fibre obsolete purely on the basis of speed. (Newbauer thinks affordable 10-Gigabit Ethernet solutions will start apearing "in about 24 months.")

For offline editorial applications, Newbauer stacks the GlobalSAN directly up against Avid's Unity and Apple's Xsan. "Spending $45,000 on a fibre-channel SAN to do offline video is totally unnecessary," he says. "A GlobalSAN solution from SNS starts at $7,000, and it'll effortlessly handle multiple streams of DV." A 9.6 TB eight-user SAN goes for less than $30,000.

But others disagree about the benefits of systems that are built around IT-friendly technologies like Ethernet and iSCSI. "We've been shipping four-gigabit fibre-channel for well over a year, and that's the best bang for the buck right now," says James McKenna, VP of sales and marketing for Facilis Technologies. "Treating a data pipe like a coax cable and not having to worry about the networking complexities is the way to go for the post customer."

Facilis is targeting users who need a SAN, but see systems like Avid's Unity as an expensive way to get there. "We start out at the lowest-priced SAN solution available: under $10,000 for a couple of terabytes of capacity with decent performance," says McKenna. From there, it's a question of when to goose bandwidth by adding more servers. Once you get up to as many as 24 drives for a multistream uncompressed-HD-capable SAN, you're looking at an investment of somewhere between $16,000 and $26,000.

"For the bulk of our customers, having six, eight or 10 users running SD uncompressed shouldn't be a problem on a single server, as long as you have 24-drive, or multiple 12-drive, units," McKenna explains. "30 or 40 TB is not uncommon – Post Works has a 43 TB installation that does a mix of HD dual-stream connectivity from Symphony Nitris and 2K DI work with Assimilate Scratch and the Quantel iQ. It's a big environment that we're the hub of as a storage solution. That's what a lot of facilities will need as 2K and 4K become the formats of choice. You don't want to be stuck without the ability to hold onto a couple of reels in 2K and access them in real time. We offer 2K accessible storage even with the 12-drive unit."

Post Works has Avid Unity systems that take care of offline and online SD editorial work, but uses local storage for its HD and 2K work to gurarantee performance at those resolutions. The Facilis SAN uses SATA storage and shares out on fibre-channel, enabling real-time 2K playback. "We can use that to preview some of the frames we scan at real time, which is nice, but we primarily use it as fast copy space," explains Corey Stewart, director of systems engineering. "Our film scanner has local fibre-channel storage, so once we fill up that volume, we copy the media up onto a [Facilis] Terrablock, and the media will live on that one volume for the duration of that job. We'll keep it there until we're finally signed off on it and we can delete those original scans."

For 2K projects, Stewart says, the scans are imported to an iQ from the Terrablock, which has its own disk set and file system. When the picture is locked, the reel is archived to another Terrablock volume.

"We've been shying away from buying into any big SAN based on fibre-channel storage, like a Bright Systems solution, at the moment," Stewart says. "The file-system limitation based on performance is really the issue. You have one file system per grouping of fibre-channel RAID controllers, or a JBOD set, and you can't really go above that. If you need to use the same media that's on that file system across three or four workstations reading and writing 2K at the same time, it's just not possible. That's the limiting factor – and obviously the cost, as well."

There's another category of shared-storage provider that specializes in optimizing the various solutions that exist. Archion, for instance, has a RAID product called Synergy Plus that basically piggybacks on an Avid Unity system, offering fully redundant storage at a cost savings over fully mirroring data on the Unity. "It's transparent to Unity," says Archion CTO Jim Tucci, "and it takes over drive management from Unity, looking at bad blocks, etcetera. Three drives can fail in one unit with no data loss."

And Exavio was at NAB showing its Examax SAN acceleration system – one demo ran four simultaneous 2K streams out of a single CXFS file system. Exavio gets extra performance out of SANs partly by striping data across multiple RAIDs in hardware, not software, and aggressively pre-loading material. "On the back end, we're coalescing the reads and writes – when we do writes, we do very big writes, and then we do it opportunistically" by storing data in a RAM cache, explains Jim Farney, the company's director of market development. "We're working with file-system vendors and with applications people to get project awareness. Customers ask if we can connect to ScheduAll, or to Xytech [workflow and asset management systems]. We'll be able to in the future."

Of course, some in the industry argue most users don't really need a SAN at all. “The biggest misconception among our customers is that they need a SAN,” says Maximum Throughput’s Giovanni Tagliamonti. “There’s a lot of evangelizing about SANs. Customers say, ‘I need a SAN.’ And, when you break down their needs, they don’t need one.”

What’s Tagliamonti’s angle? For one thing, his company makes the Sledgehammer, which is a network-attached storage device that’s designed to make the traditional digital disk recorder redundant. It’s a purpose-built system for post applications, running software that handles video IO and format conversions. You can scan film directly into a Sledgehammer and make it available immediately over your network. And with the new dual-stream play and record capabilities added at NAB, dropping a Sledgehammer into a traditional color-correction workflow has interesting implications – for instance, a Da Vinci system can control the Sledgehammer as if it were a playback and recording deck. “The Da Vinci 2K owns that market, and it’s still a linear system,” Tagliamonti says. “You take one of our boxes and slap it in the middle of that process, and you’ve now converted it to a nonlinear process. You’ve also made the files available on your network.”

Tagliamonti argues that only about five percent of post-production clients really need a SAN rather than a NAS system. “If you need massive bandwidth to a single client, a NAS can’t do it today,” he says. “You can only talk to a NAS over a NIC, and only at 120 MB/sec. If you need four-gigabit HBA, you need a SAN. Where I need a big pipe to a single client is on my video network, and as far as our box is concerned, you come in to your video card over a big pipe. You may need a lot of aggregate bandwidth – but why do you need a big pipe to a single client?”

And, when asked about the future of shared storage, Tagliamonti echoes some of Newbauer’s arguments against the primacy of fibre-channel SANs. “I am from the school that says Moore’s Law is about to take care of all your [bandwidth] requirements,” he says. “10 Gigabit Ethernet is here, and the line between what a SAN is and what a NAS is is so blurred today. A SAN is designed to fit behind data-based servers, and there you absolutely need one. Sitting behind an Oracle database or any ERP [enterprise resource planning] system, with two or three servers talking to a SAN? Yeah. But in our customers’ environment, you don’t. You want clustered NAS.”

“The important stuff,” says McKenna of Facilis Technologies, “is going to come with the ability of a central-storage system to service multiple clients without the need to measure bandwidth across different drive sets and watch your client loads. Avid’s ISIS product uses that clustering mentality, with servers acting together to up the bandwidth. We’re going to be selling a lot more smaller-capacity servers in the future, because, spanning them together, you end up with larger bandwidth than you can get even with our largest, 24-drive server. Not having to deal with load-balancing across different drive sets, or putting clients in different places to avoid stepping on toes, will become important.”

“The Holy Grail is some sort of clustered system like the Avid ISIS,” agrees Stewart of Post Works. “That is the be-all and end-all. Something that will scale linearly. You’ll be able to throw extra nodes onto it and have it spread data across all the nodes, and [speed won't be an issue] as long as one node will have the connectivity to be able to do the bandwidth you require. Right now, we don’t ask for much, just 2K. But down the line, 4K will be possible. It will be interesting to see where those clustered SANs come from and how we can use them in our environment.

“The connectivity to that cluster is really key. What is going to be the format for everybody to play with? And if no one picks a format, is this cluster going to be able to share out multiple formats - fibre-channel, InfiniBand, 10-Gigabit Ethernet, or whatever the new thing is six months from now?”