Bunhill Cemetery is just down the road from my flat in London. It’s a handsome old boneyard, a former plague pit (“Bone hill” -- as in, there are so many bones under there that the ground is actually kind of humped up into a hill). There are plenty of luminaries buried there -- John “Pilgrim’s Progress” Bunyan, William Blake, Daniel Defoe, and assorted Cromwells. But my favorite tomb is that of Thomas Bayes, the 18th-century statistician for whom Bayesian filtering is named.
Bayesian filtering is plenty useful. Here’s a simple example of how you might use a Bayesian filter. First, get a giant load of non-spam emails and feed them into a Bayesian program that counts how many times each word in their vocabulary appears, producing a statistical breakdown of the word-frequency in good emails.
Then, point the filter at a giant load of spam (if you’re having a hard time getting a hold of one, I have plenty to spare), and count the words in it. Now, for each new message that arrives in your inbox, have the filter count the relative word-frequencies and make a statistical prediction about whether the new message is spam or not (there are plenty of wrinkles in this formula, but this is the general idea).
The beauty of this approach is that you needn’t dream up “The Big Exhaustive List of Words and Phrases That Indicate a Message Is/Is Not Spam.” The filter naively calculates a statistical fingerprint for spam and not-spam, and checks the new messages against them.
This approach -- and similar ones -- are evolving into an immune system for the Internet, and like all immune systems, a little bit goes a long way, and too much makes you break out in hives.
ISPs are loading up their network centers with intrusion detection systems and tripwires that are supposed to stop attacks before they happen. For example, there’s the filter at the hotel I once stayed at in Jacksonville, Fla. Five minutes after I logged in, the network locked me out again. After an hour on the phone with tech support, it transpired that the network had noticed that the videogame I was playing systematically polled the other hosts on the network to check if they were running servers that I could join and play on. The network decided that this was a malicious port-scan and that it had better kick me off before I did anything naughty.
It only took five minutes for the software to lock me out, but it took well over an hour to find someone in tech support who understood what had happened and could reset the router so that I could get back online.
And right there is an example of the autoimmune disorder. Our network defenses are automated, instantaneous, and sweeping. But our fallback and oversight systems are slow, understaffed, and unresponsive. It takes a millionth of a second for the Transportation Security Administration’s body-cavity-search roulette wheel to decide that you’re a potential terrorist and stick you on a no-fly list, but getting un-Tuttle-Buttled is a nightmarish, months-long procedure that makes Orwell look like an optimist.
The tripwire that locks you out was fired-and-forgotten two years ago by an anonymous sysadmin with root access on the whole network. The outsourced help-desk schlub who unlocks your account can’t even spell "tripwire." The same goes for the algorithm that cut off your credit card because you got on an airplane to a different part of the world and then had the audacity to spend your money. (I’ve resigned myself to spending $50 on long-distance calls with Citibank every time I cross a border if I want to use my debit card while abroad.)
This problem exists in macro- and microcosm across the whole of our technologically mediated society. The “spamigation bots” run by the Business Software Alliance and the Music and Film Industry Association of America (MAFIAA) entertainment groups send out tens of thousands of automated copyright takedown notices to ISPs at a cost of pennies, with little or no human oversight. The people who get erroneously fingered as pirates (as a Recording Industry Association of America (RIAA) spokesperson charmingly puts it, “When you go fishing with a dragnet, sometimes you catch a dolphin.”) spend days or weeks convincing their ISPs that they had the right to post their videos, music, and text-files.
We need an immune system. There are plenty of bad guys out there, and technology gives them force-multipliers (like the hackers who run 250,000-PC botnets). Still, there’s a terrible asymmetry in a world where defensive takedowns are automatic, but correcting mistaken takedowns is done by hand.
— Cory Doctorow, Internet activist, blogger, founder of Boing Boing
Great read there. I think automatic defenses against malware are only starting to appear, meaning what we are seeing are merely first generation systems.
In a way, its akin to having a spam filter in place but with no way to identify or retrieve false-positives.
Give it some time, I think subsequent generations of such systems will start to incorporate the features that allows one to easily reverse these "false-positives."
It makes the point that businesses are increasingly offloading the cost of fraud protection onto consumers--effectively forcing consumers to pay for insurance for the provider--and claiming the consumers benefit from it.
The end user still has to take responsibility for their own hardware and software, but let's face it: people are generally not geniuses and keeping any system malware free is no longer a simple task.
That's why we are in desperate need for a comprehensive suite of security tools that detect rogue traffic at the switch/infrastructrure level. Now I know that NAC systems are big this year, but that is just a start and they are still generally way too complex for effective deployment according to recent InformationWeek polls.
The hotel system you described was another good example of automated security, but again, these are just the first baby steps in this direction. As you rightly point out, the system is designed to slam the door at the first sign of danger with a complicated manual procedure to open it again. This is not a viable long term solution.
We need to inject high speed intelligence into all of our hardware. No more dumb components. As hardware/chip prices fall and shrink, intelligence will be injected across the board. If all our hardware is smart and all our components know how to maintain your enterprise's security protocols and procedures, when an end user does get duped by social engineering or a spear phishing scam, the damage is immediately detected and the infection is stopped.
On the flip side of that idea, when a user is locked down, they need to be presented with clear information as to why the security action was taken, and what the user needs to do to "revalidate" and clear their personal error state without human intervention. Again, I know that NAC systems can provide some of this functionality, but I think we cannot put all the heavy lifting on the NAC itself. Every part of our back end infrastructure needs to be made more intelligent and more adaptable to the infinite variations on error states. Ideally, in the corporate enterprise, the infrastructure will detect any undesireable state of any piece of hardware, and will be able to instruct that hardware to correct the undesireable state automatically, with revalidation being the next step in the process.
Only then will we start to have some semblance of a sane security model.
It sounds like to me that we have put forth a full throttle defense and if we get a few good guys along the way, oh well all wars have built in casualties. But if we have to fix one good guy then we go through all the checks and get more info than the user should know and then maybe we can fix the mistake our automated protection made with little regaurd to the rules the good one has to follow. Let us spend a little time on making our systems a little more ( user friendly) by checks and rechecks. If the system can block you in a matter of mil.sec. then I think it can burn a few mil.sec. to double check the threat and make sure it is not a good one being misread.I strongly believe in protecting the system, and personal info, but we need more brute force in tracking the bad guys who start the problems. I think a lot can be done at the hot spots that hackers and bot creators use, say cyber cafes, put a little more security there so when some one sends large data files or emails it will stall it and even notify the (cyber police ) if we had such a thing. or maybe we could develop a way to see the traffic and have the local system return a tag to the device sending it, then it would transmit every time the device is logged in and then this would give the "cyber police” time to find them when they have moved on.But until we catch up with the technology we develop, be safe and watch your personal info.
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
Email is my alpha and omega, my file-system and social register, my backup and my memoir. If I need to find a document, I don't search my hard-drive; I search my email for the copy I sent to someone when it was done. I sometimes write novels on email, sending out the day's pages to a mailing list of well-wishers who keep me honest, nudging me if I miss a page. Version control? Who needs it? Just find all the copies I sent or received and order them by date!
Smarter Collaboration: How to Thrive in a Challenging Business Environment Market conditions are changing faster than ever, and organizations need to improve their agility and adaptability in order to provide better service and improve processes. The ability to work with customers, business partners, and employees as effectively as possible - while at the same time holding down costs - is a key to success. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
In the final episode of this series about the death of Internet anonymity, Saunders describes how the Internet of the future will start to attain a level of intelligence that requires no human intervention. Scary.
What can users today do to protect their online privacy? The simplest and most obvious option is to not use the Internet – at all. However, once all digital information is consolidated over the Internet, trying to protect digital identity by simply unplugging from the Internet becomes impossible – a fact that has manifest implications for civil liberties, Saunders says.
By 2011 the number of Internet-connected sensors will exceed 1 trillion, making your chances of doing anything or going anywhere unnoticed pretty much zero. Saunders talks about how the 'sensortization' of the Internet is eliminating the traditional divide between online and offline populations.
The 20th Century Internet was characterized by the ability to interact with other people and information on the Internet largely without anyone knowing who you were. The Internet of this century, conversely, will be defined by identity. Saunders explains how Internet users are unwittingly contributing to the demise of the anonymous Internet.
Microsoft reportedly has plans to integrate Windows Live and even Xbox with Windows Mobile. That may provide them a strategic advantage, but what will the cost be to your privacy? Tom explains all.
What does a $0.62 refund check from a service provider mean? It could mean that, unlike Google, Amazon, and Apple, telcos aren't ready to use what they know about their customers to sell better, more personalized services.
Comparing Internet services is tough because service providers price and market their services based on a best-case scenario connection that most consumers will never enjoy.
Cellular operators, netbook manufacturers, and, of course, techies, have been eagerly awaiting a hands-on experience with Nokia's new netbook, the Booklet 3G. Reiter's got his hands on one, and tells you whether it's worth the wait.
The release of Microsoft's newest OS raises the question of the company's relevance in an era when Google dominates applications and search, and Apple runs circles around Redmond with its gadgets and user interfaces.
Research shows that the youth of today like Facebook – but not blogging or Twitter. Does that mean Facebook has won, or just that it's not yet out of favor? Will all the services we see today fade into Ovaltine-or-Wheaties status in just a few years?
What kinds of companies are doing the most innovation in the data center? Turns out it's midtier enterprises that are taking the "Just Right" approach.
Digg
Del.icio.us
Reddit
Email This
TWEET THIS
