We live in an era of vast (and rapidly increasing) volumes of data, much of it "born digital" and stored only in digital formats. Such information is prone to decay, in ways that stone tablets, parchment, and even photocopies may avoid.
Computer servers routinely fail; magnetic tape, DVDs, and other storage media can degrade; and the constantly "improved" formatting of information storage systems can leave collections of information (think kinescopes and vinyl records) essentially inaccessible.
The cost of protection of digital data, however, can be overwhelming. In 2011, a European study group estimated a cost of 100 billion euros to bring European "cultural heritage" information online, in a stable and long-lasting system called "Europeana." (See "The New Renaissance," Jan. 10, 2011.) The European report cautioned that "responsibility and control over Europe's heritage cannot be left to one or a few market players," and called for the public sector to take "primary responsibility" for the project.
Like the Europeana project, the US National Archives and Records Administration (NARA) has begun building an Electronic Records Archives to preserve the American government's records and documentary heritage. Its mission is to "ensure continuing access to the essential documentation of the rights of American citizens and the actions of their government." (See NARA, Electronic Records Archives.)
Government-funded preservation of digital data will require extensive supplementation from the academic community, not-for-profit organizations, and the private sector. The Open Archival Information System (OAIS) reference model, for example, offers an International Standards Organization reference standard for long-term data storage. (See ISO 14721:2003.) Similarly, the Long-Term Digital Preservation Reference Model, a vendor-sponsored group, associated with the Storage Networking Industry Association, has begun to focus on developing industry "best-practices" for long-term data preservation. (See About LTDP, www.ltdprm.org.)
Ironically, some of the techniques suggested for mitigation of data loss, such as cloud computing, may actually introduce new difficulties. (See Bryan Ford, "Icebergs In the Clouds: The Other Risks Of Cloud Computing," suggesting need for study of cloud computing solutions "before our socioeconomic fabric becomes inextricably dependent on a convenient but potentially unstable computing model.")
Similarly, government mandates of long-term record-keeping, unless coupled with intelligent systems for categorization of records and ease of application of retention standards, could contribute to a "keep it all" approach, producing ever-more "cascades" of unmanageable data. (See Charles Ragan, How To Avoid The Information Management Dark Ages, Dec. 16, 2011.)
Ultimately, government and private groups must learn to collaborate to produce not simply more data but incentives for standardization and interoperability of "smarter" data management techniques.
[Disclosure: The author is a partner in the New York City offices of Jones Day. Barbara M. Santisteban, a summer associate at the firm, assisted in the preparation of this article. The views expressed are solely those of the author, and should not be attributed to the author’s firm, or its clients.]
We saw something of what this problem could involve with the "Y2K" mini-crisis. When companies / government units were attempting to fix their old systems, which had the assumption of 1900-something built into the code, they sometimes had to bring back retired programmers, because those were the only people who knew the original languages / structures, and how to modify them.
that paper preserves so well, and digital preserves so badly, both through hardware degredation and lack of software to read it. Between that and the ease of deletion,many historians are concerned that we'll have any record of this period of time.
what I was advised is, when you get new backup media, save all the old. That at least helps with the hardware problem, if not the software one.
incidentally, Office is one of the worst offenders, I've read.
Stephen - we have become so lazy with email and discarding what is not essential - I am VERY guilty of it, less so as much of my email also has photos attached, so in an effort to not crash the world, I do tend to clean up regularly.
But still, I can easily sort but any number of notifications, and find myself missing many that can easily be trash.
Thanks for crediting your image BTW - another easily forgotten task.
I fully agree. It would make the systems, and us, much smarter. I think the limits going forward are the limits we create if we remain in the digital dark ages, as you effectively make the point.
Surveys suggest that a very healthy percentage of email traffic is essentially spam, or duplicate messages. Simply filtering out the worthless would greatly reduce the volume of storage, and the need for search / retrieval of data.
Great thoughts, scbennett, and great article. The decision making is the hard part.
Because we have created the capacity for unlimited data, we now have to choose what to store, how to store it, when we will need it, etc. I believe we have built a culture of storing everything we may need, because we could. Now that our capacity is unlimited, storing everything is unmanageable, and we will have to learn how to design and manage all of our digital assets.
Your thoughts bring us to the realization that we are possibly already there, or soon approaching.
I remember a discussion tool we used years ago (in college) to consider "what would you need if you were going to live on the moon?" Out of a list of 100 things, put them in order of importance. With just a little bit of thought, one thinks of survival items (pressure suit, oxygen) first, followed by longer term survival (food, water) and eventually of "nice to have" items (like something to read). I daresay information management is a bit like that. Some things are absolutely crucial to continue operations, and lots more is simply "nice to have." How to distinguish, and how to consistently / competently store the essential? Those are some tough questions.
I think people try to flatter themselves, considering that someone will need their boring and needless records in time. It's also a philosophical question, does the Noospehere - the infomrational space of the Planet or may be the Universe have enough space for all the human information.
It's a bit ironic: we polluted the water, the air, now we try to keep hundreds of Terrabites, just because... we want to and we have a place for that.
"In data management, i wonder whether it gets to a point where data should be disposed when found to be either obsolete or unnecessarily redundant."
Well I think that point can arise. Suppose a FMCG company had a soft drink segment 10 years ago. It discontinued its soft drink business at that time (10 years ago) and it has no intention to move back into that industry. What use will the purchase orders will be after 10 years (considering they havent been used within that time) where there arent any legal and tax requirements to store data that long?
There can be similar circumstances where keeping the data can only cause data management issues. Nevertheless, decision of deleting the data should be made after due consideration.
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
A recent Michigan case -- Ahmed v. Finley's Mfg. Co. -- concerned the settlement of class action claims that "halal" products offered at fast food restaurants in Dearborn did not comply with Islamic dietary restrictions.
The United States has no single national privacy protection agency. Over the past 20 years, however, the Federal Trade Commission (FTC) has taken a leading role in education, outreach to consumer and industry groups, and enforcement in the area of privacy and data security. The FTC’s Division of Privacy and Identity Protection, within its Bureau of Consumer Protection, enforces several federal statutes, regarding “unfair or deceptive” practices, fair credit reporting, and confidentiality of financial information. Further, the FTC takes principal responsibility for enforcement of the Children’s Online Privacy Protection Act (COPPA), which aims to place parents in control over what information is collected from their young children online.
The advancement of digital technology opens unprecedented avenues for e-commerce, aided by digital currency systems. In addition to online credit cards and other forms of payment, technology has developed to the point where individuals can carry digital wallets within their smartphones and complete transactions with near-instantaneous payment. This Forbes India article provides a description of digital wallet systems.
Social media platforms have greatly transformed personal interactions in the work environment. But the trend toward daily use of social networks, at work and elsewhere, has generated legal controversy as to what is protected worker activity and what is grounds for termination.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
Multi-tenant clouds assure security for clients, but not necessarily for their ideas. Here's one thing you should discuss with your cloud provider before you sign on.
The new Network Functions Virtualization (NFV) initiative of operators is being run out of Europe's ETSI and not here in the United States, even though the issues have been here for five years. The US needs to step up; otherwise, it's surrendering leadership.
The FBI recently issued a warning to smartphone users, highlighting two mobile malware applications: Loozfan, which steals personal information, and FinFisher, which is spyware that takes over a smartphone's functions.
The bring-your-own-device approach isn’t suited to monitoring of enterprise equipment and processes. In these cases, it is up to IT to come forward with gear suited to the task.
Healthcare providers have been moving to telemedicine treatments, where the patient and doctor can meet online, but insurance carriers are not required to pay for such treatments. This may change, though, as Maryland recently passed a law mandating that insurers pay up.
Elizabeth Pizzinato, SVP of marketing and communications at Four Seasons Hotels and Resorts, calls content marketing "the new black" and explains how her brand engages its target audience.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
Subsidized handsets, rather than locked handsets, should be the focus of regulators. We're not getting good deals, not fostering innovation, and weakening our power as buyers.
Expert Integrated Systems: Changing the Experience & Economics of IT In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
Email This
