Flash Drives Set to Give Internet a Performance Boost

From a selfish end-user perspective, this is about heat and noise.  After the Mac Cube (http://en.wikipedia.org/wiki/Power_Mac_G4_Cube) I really thought there would be more of a push for silence, but apparently not.

Look at the Dell Adamo.  http://www.dell-adamo.nl/wp-content/uploads/2009/01/dell-adamo-white-04_large.jpg

Now they're clearly selling this as a sexy machine, the ads show runway models posing with the thing.  But all I see is the ugly air vents on the side.  Come on, this would be a slimmer, more beautiful machine w/o a spinning drive, don't you think?

My dream machine: a Thinkpad netbook (for the keyboard and trackpoint) and no moving parts.

I loved your story.  Back in about 1974 I could smell the lanolin too.

I agree 100% that there will be a rethink about the CPU/Memory/Disk relationships.  Current architectures assume all sorts of conventions (standards) at each point of the interface. 

Sooner or later someone with some access to capital will review the CPU/Memory/Disk relationship from the point of view of function- for example a HTML server or a Video server.  Everything form the disk sectors on out needs to be defined in view of the intended purpose.  I expect 10X performance at the same level of semiconductor/storage technology will be possible.

Lawrence Ricci
www.EmbeddedInsider.com

'

Wow Rob! That's a tremendous post. Your point is right on, this concept is not new and architects like yourself have faced the spinning media bottleneck forever in this industry.

And for those of you born after 1985, by "sketched out in a notebook" Rob means written in pen or pencil in a 2-fold, sturdy piece of vinyl or leather bound plastic that houses sheets of lined paper :-)

Companies probably do not like the expenses but when considering various laws, particularly those involving privacy, and safeguarding company data, they are willing to sacrifice cost for high performance and durability.

(Besides, they will figure a way to pass the cost to their customers.)

Folks -

I'm reading the comments with a big grin.  Perhaps you would be interested in how the "solid state disk" was invented?

Back in the summer of 1970, I was consulting with the Laboratory Data Products Group at Digital Equipment Corp.  Housed in the old woolen mill in Maynard, Mass, the floors were still soaked in lanolin from the blankets made there during the Civil War.

The task at hand was to write a gas chromatographic analysis package to run on a PDP-12.  The 12 was a chimera of a machine --- a PDP-8; a LINC-8; and a still-under-development Floating Point Processor (FPP).  "Mass" memory on the 12 was a DECtape  --- a hand-sized, wide magnetic tape.  The FPP was a 17" by 8' rack.

Software assembly with DECtape was a multi-hour process.  However, on weekends, there was a "production" 12 used to generate new DECtapes to ship with new 12's.  It had a fixed-head hard drive.  One head per track.  Fast!  I could assemble in minutes.  The value of a fast, rotating drive with no head-movement delay was embedded in my head.

Several months later, back in my computer lab in Cleveland, running on the PDP-11 that DEC kindly provided in exchange for my successful software effort, there was a DR-11-M (?) Direct Memory Access controller to let a peripheral device move data directly to/from the PDP-11's core memory.  In an inventive moment, I realized that one could emulate that fixed-head hard drive in the same manner as the FPP had been connected to the PDP-12, but using a rack-full of memory boards.

It took until 1974 to find a company willing to share the development of what-came-to-be-called the Extended Memory Unit (EMU) [After the fast, flightless bird].  Monolithic Systems in Englewood, CO, a maker of semiconductor memory boards picked up the concept and turned it into a successful product.  Rumor was that some EMUs were on nuclear submarines, where there was no gyroscopic torque from a spinning hard drive.  I built a very-high-speed data acquisition system where data from explosions went through analog-to-digital converters into the EMU, then sent a single interrupt signal to the host PDP-11/45 computer.  To the PDP-11, it seemed that in a microsecond, a disk full of data just appeared on a hard drive.

A graduate student in Computer Science at CWRU studied the EMU performance.  Instead of being 17,000 times faster than DEC's RF/RS-11 fixed-head disk [17 millisecond rotational latency versus 1 microsecond access time for the semiconductor memory], the performance was far less, due to the fact that the software device drivers in DEC's OS were written with a rotating drive in mind.

Which brings us to the current "Flash Drive" flurry  ---  I believe that there will soon be a shift in system architecture.  Rather than word-at-a-time or block-at-a-time data transfers, computing will be done where entire large blocks of data are transfered with a simple interrupt signal.  Processing algorithms will be re-thought such that FPLA's can do the processing of these large blocks of data in a single clock cycle.  The market might evolve with the creation of smart, ultra-fast peripheral devices.

All this was sketched out in a notebook in the mid-1970's.  I never throw anything away, so I'll dig through the storeroom and see what explicit ideas I had around this back then.

Well thank you Mary, I appreciate that. And I appreciate your pushing at my assumptions and premise here to make them better. What good is a prediction that's not surprising? (Although I didn't honestly see this as that outlandish).

I see a domino effect coming where the price of flash drops at or near (or below) spinning disk and a leading vendor or a few, in an effort to get competitive advantage and lower the cost of storage, adopt early and push hard. I predict EMC, IBM and HP will lead this charge. Maybe NetApp too. 

In re-thinking (and re-thinking) your point about vendor adoption, it strikes me that the unwillingness to switch may be due to the need to re-architect storage systems to accommodate flash. That will cause friction. But I believe these leaders are already doing just that and the rest of the industry will be forced to follow.

So I'll adjust my bottom line a bit. It still comes down to price but I'll concede switching costs are a factor and they are not fully eliminated because flash drives can plug directly into FC slots. I'll admit there are clear costs to re-design array architectures to accommodate flash. The fact that flash will plug into existing interface slots is key because it will drive the first wave but full scale adoption will require redesigns.

New prediction...those vendors who don't re-design their arrays to accommodate flash will begin to lose market share in the 2011 timeframe.

Fascinating stuff, Dave. However, I'm still a skeptic about the big vendors' willingness to make the switch. If I were a wagering person, I'd say you'd have a bet that FC won't lose ground to Flash until way after 2012. Still, your case is well put and this industry is nothing if not surprising.

15,000 rpm even Mary. But it's mechanical. You're comparing a spinning disk with semiconductor speeds. Make it 50,000 rpm (if you could dissipate the heat and handle the data rates) and it's still an order of magnitude slower than semiconductor...not to mention the time it takes to to physically move an actuator arm, which hasn't really improved in years.

The other factor here is processor speeds double every 18 mos but disk drive speeds get worse every 18 mos. Why? because the more data you put under an actuator, the slower disk drives get. It's a problem referred to as 'access density,' meaning the disk has to spin more and the arm has to move more to find the data to fulfill an i/o request. This is why customers 'short-stroke' disk drives - i.e. they leave them empty except for the critical files (e.g. database indicies and the like). There are gazillions of FC drives installed that are basically empty for performance reasons.

Bottom line, compared to the speed of light, mechanical disk drives are sloooooooooooooooooooooooow.

Design good software to minimize the need for faster hardware...interesting concept and one that's been talked about before. One of the major limitations of applications is the need to access spinning disk. For decades we've dreamed of keeping "Data in Memory" (DiM) as a single level store. Flash enables a form of that vision and application developers will exploit it.

Here's what I know...give application developers fast i/o resources and they'll use them in ways your or I never thought of. New supercomputer applications are popping up everyday using the Internet as a resource and the applications are following.

Are you arguing that the state of Internet applications is frozen-- or should be?

My strong belief is Internet applications will exploit flash technologies to create better user experiences-- it's stating the obvious.

"The only spinning disks left will be ultra high capacity devices that will maintain an advantage over flash for quite some time to come."

This seems to work against your claim that Fibre Channel will be replaced by flash within the next 3 years.

Even if you go with speeds, FC RPMs are pretty durn high -- up to 7200 for 4-Gbit/s, no?

Now, how much energy they use might be a key factor.