The Macrosite for News, Analysis and Opinion about the Future of the Internet
DISCUSS   PRINT   Digg   Del.icio.us   Reddit   Email This   TWEET THIS

Peer-to-Peer Filters: Ready for Internet Prime Time?

Accuracy of P2P Detection
3/27/2008 9 comments

In this test, we configured 13 different P2P clients using a total of 10 different P2P protocols to verify detection accuracy. For each of the major P2P protocols – BitTorrent, eDonkey, and Gnutella – we used two different clients. Client implementations of the same protocol may differ slightly, so we wanted to verify whether the devices could detect all implementations of a P2P protocol and distinguish between different clients.

Other Internet applications like Web sessions, video streams, file transfer, and email were sent alongside the P2P traffic in order to reproduce a typical mix of Internet traffic. The challenge was to detect the P2P protocol traffic volume accurately – not allowing any sessions to escape the device’s attention.

Why is this test important? Because accurate reports of application and bandwidth usage support service providers to take educated decisions regarding P2P traffic control. Exact statistics enable measures to limit or block traffic on a per-application or per-protocol basis.

We generated the following traffic mix (the example is applicable to 1-Gbit/s link speed):

  • Stateful traffic: 350 Mbit/s
  • Stateless Internet application traffic: 110 Mbit/s
  • Total: 460 Mbit/s

The following list shows the emulated application sessions and generated parallel session numbers:

  • Maximum number of simultaneous sessions: 6,200,000 (for each of the devices, we configured 70% of the maximum specification in the data sheet)
  • P2P Azureus (BitTorrent): 10,000 parallel sessions
  • P2P µTorrent (BitTorrent): 5,000 parallel sessions
  • P2P Limewire (Gnutella): 200 parallel sessions
  • P2P eMule (eDonkey): 16,000 parallel sessions
  • P2P BearShare (iMesh): 200 parallel sessions
  • P2P Shareaza (Gnutella): 200 parallel sessions
  • P2P KaZaA (FastTrack): 200 parallel sessions
  • P2P FileTopia (FileTopia): 2,000 parallel sessions
  • P2P WinMX (WinMX): 2,000 parallel sessions
  • P2P Manolito (MP2P): 2,000 parallel sessions
  • P2P Soulseek (Soulseek): 1,000 parallel sessions
  • P2P DirectConnect (DirectConnect): 1,000 parallel sessions
  • P2P Total: 39,800 parallel sessions
  • HTTP: Between 2,000,000 and 6,200,000 parallel sessions
  • POP3: 1,000 parallel sessions
  • SMTP: 1,000 parallel sessions
  • FTP: 5,000 parallel sessions
  • RTP video stream: 1 session
  • The average packet rate for the total traffic was around 250,000 packets per second

For the results, we compared the bandwidth transmitted per protocol/client with the statistics reported by the device under test. We calculated a detection efficiency metric describing the percentage of P2P bandwidth correctly identified.

Individual Results
The Arbor/Ellacoya E30 was able to detect nine out of ten P2P protocols:

  • BitTorrent, eDonkey, and Manolito were well detected and bandwidth precisely measured.
  • Gnutella and Direct connect were detected, but with up to 30% error on measured bandwidth.
  • Kazaa, Filetopia, WinMX, and SoulSeek were detected to some extent, but the Arbor/Ellacoya E30 measured only 35% of the bandwidth passing through.
  • iMesh was not detected.

The Ipoque PRX-5G implementation excelled in this test. It detected all ten protocols successfully and was able to provide accurate statistics on six of them. It was the only device able to detect all offered P2P protocols. However, some of the protocols were detected with low accuracy:

  • FastTrack, eDonkey, Gnutella, MP2P, and BitTorrent were well detected with less than 10% error on measured bandwidth.
  • Direct Connect was detected with up to 30% error on measured bandwidth.
  • Filetopia, iMesh, and Soulseek were detected, but only a fraction of the traffic was measured (between 5% and 47% of real traffic).
  • WinMX was detected, but only a very small amount of the total sent bandwidth (below 1%).

    Table 2: P2P Protocol Detection Efficiency
    Ellacoya E30 Ipoque PRX-5G
    BitTorrent 82% 97%
    eDonkey 97% 88%
    Gnutella 76% 96%
    FastTrack 1% 97%
    MP2P 86% 96%
    iMesh 0% 47%
    FileTopia 33% 23%
    WinMX 7% 0%
    SoulSeek 1% 5%
    DirectConnect 77% 78%

    The following diagram compares detection efficiency of all unencrypted protocols for each device in observation mode.

    P2P Detection

    EANTC verified that Arbor/Ellacoya and Ipoque were able to detect most parts of BitTorrent and eDonkey traffic. They are the two dominant P2P protocols that account for more than 90 percent of all peer-to-peer traffic on the Internet, according to a study from Ipoque and other sources. Arbor/Ellacoya's E30 was very accurate detecting eDonkey traffic, but Ipoque’s PRX-5G did turn in a better job of detecting BitTorrent traffic. On average, both devices caught nine out of ten BitTorrent and eDonkey sessions. Less popular protocols gave mixed detection results – specifically, the less popular protocols like WinMX (a P2P network that is almost considered dead) were barely detected.

    Next Page: Accuracy of P2P Regulation

    DISCUSS   PRINT   Digg   Del.icio.us   Reddit   Email This
    < Previous Page 6 of 13 Next >
Current display:       newest comments first       display in chronological order
Bobert764
Rank: Cave Painter
Tuesday September 15, 2009 10:49:24 AM
no ratings

A lot students transpire the responsibility to professional writers because they lack the talent to write a good paper about Peer-to-Peer Filters so that the argument why you need to use plagiarism checker, but such guys like composer don't do that. Thanks a lot for the article


no ratings

Hi Bill,

As mentioned in a personal message before, we would be happy to test your product in a second round of the test. All carrier-grade filtering solutions are invited to participate.

For the first round, we invited all vendors active in the P2P filtering market back in Q1/2007.

cross
IQ Crew
Friday April 11, 2008 12:55:24 PM
no ratings

Broadband access "flat rates" would need to be much more expensive if they would cater for P2P usage for everybody. The consumer flat rate is offered assuming that the consumers do not download stuff at max speed 24 hours a day, 7 days a week.  Such real flat rates are offered for businesses at much higher prices...  Unfortunately the P2P residential users are not willing to pay hundreds of dollars per month like a business.

Therefore ISPs have to use a hybrid cost model where the occasional users contribute to the cost of network bandwidth supplied for the P2P users.  Without disclosing any individual statistics, typically less than 1% of the broadband access users generate more than 60% of the network bandwidth.

The most expensive network cost are the external uplinks (Internet commercial exchange [CIX] peers).  Quite a few major P2P seeds are located in remote locations - P2P traffic is usually worldwide distributed. Filtering does not actually save bandwidth, it helps to reduce the speed at which P2P bandwidth demand is growing, specifically on the uplinks. This is why ISPs are interested in this technology.

Best regards, Carsten

Maxregware
Rank: Cave Painter
Monday April 7, 2008 5:38:43 PM
no ratings

"Such apps often use entire multi-megabit DSL bandwidth, continuously, whereas most ISPs design their networks on the assumption that an average user will need as little as 100 kbit/s."

While I agree that the industry protect their copyrighted material, and also enterprises should be able to filter the bandwidth used by their employees (a employee should not be downloading mp3 or movies during work, right?), I question the motivation of ISPs.

If I pay for, let's say, a 3MB connection, am I supposed to just browse and e-mail? Why can't I plug into every bit of my multimegabit connection? Or if ISPs can reduce costs by filtering and improving the reliability of theirs networks, will these gains be shared with its users by lowering connection prices? I bet no. It will only improve their profit margin. Sell 3MB, expect each user don't surpass 10% of it and you can have a lot more paying users than your nominal capacity would permit.

Maxregware
Rank: Cave Painter
Monday April 7, 2008 5:19:03 PM
no ratings

"Nothing promotes the development of technologies more than a desire for free stuff"

Deep thought! Totally agree. :)

no ratings

Well you can't say it's a shame for the researchers on the basis of the vendors they selected for the trial test. I believe their selection was based on P2P filters that are making all the hype in the markets. You've admitted that yours is a new technology and probably might not make the break yet in the media.

But it's great to hear that your product is really working at least according to your evaluation and the fact that you can offer it for testing shows your confidence in it delivering value for money. Let's hear from the authors of the report on how your products works and then may be you can see a real change in fortune for SafeMedia.

no ratings

It's a real shame that they only looked at major vendors and not new technology companies.

SafeMedia developed a technology solution that detects and blocks over (650) P2P clients including those that use encryption and port hop to avoid detection.  

Our network security appliance was fully tested by a major US based ISP and they confirmed it blocks everything.  The size of the testing suite used for this report was not large by our standards.  We can handle 40GB backbones and maintain a low latency of .4Ms.  

What's different in our technology is we don't use DPI and look at layer 7 (content). 

We'll ship SNEP a unit tonight if they want to see real results.

Mashka
Researcher
Friday March 28, 2008 2:46:48 PM
no ratings

that's really interesting news,not sure that it's good.

 where will I download my music:))))

But even if this  works among US ISP I doubt that will work in the world.Or people will invent something new that let them  get free stuff.Nothing promotes the development of technologies more than a desire for free stuff:))))

Paul Whyte
Researcher
Thursday March 27, 2008 2:32:23 PM
no ratings

Many thanks for this wonderful report. As a research student i know very well how difficult it can be to simulate actual operational conditions in laboratory experiments. As i perused through your report partly due to the exigencies of time, i was really impressed at how detailed and honestly you went about conducting this reasearch. It's a big shame to the other vendors who pull out. I think it was an excellent window of opportunity for them to have honestly evaluate the performance of their product and possibly get feedbacks that would have helped them to improve upon their products. May be they are very aware of the limitations of their products.

My question though is are these filters technologically feasible? With Congress and other legislative bodies increasingly calling on ISPs to instal these filters in their infrastructure, some P2P companies have come under intense criticism for the failure of their devices to perform as expected. In defence, they are simply saying that people are expecting them to do the impossible:

P2P companies say they can't filter

Although the signs are good for the two products that made through your tests, do you see in the forseeable future better filters coming to the markets or the challenges to produce better filters is so great that it will thwart any  futurte breakthrough in this area?

The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
a moderated blogosphere of internet experts
Tom Nolle
Tom Nolle   2/9/2010   4 comments
If you’re a slightly gray, mid-level manager who travels a lot, you may be on the way up and worthy of professional respect, but one thing you most definitely are not is “cool.” Still, while today’s youth may think you just crawled out of a paleolithic cave, there may be hope. The iPad from Apple Inc. (Nasdaq: AAPL) (supreme arbiter of coolness) just might make you older guys (or actually old guys like me) cool.
Rob Leathern
Rob Leathern   2/9/2010   5 comments
As we well know, the online echo chamber and its increasingly viral and social components can magnify the propagation speed and distribution of stories and rumors, whether true or false.
Rob Salkowitz
Rob Salkowitz   2/9/2010   5 comments
A remarkable event in world affairs is taking place this week in London, as the first One Young World conference is set to convene.
Ira Winkler
Ira Winkler   2/8/2010   15 comments
In his recent Congressional testimony, Dennis Blair, the U.S. director of national intelligence, stated that the U.S. is "severely threatened" by cyber attacks and that the recent Google (Nasdaq: GOOG) attacks should serve as a wake-up call.
Jart Armin
Jart Armin   2/8/2010   14 comments
Fatal System Error, the book just released by West-coast-based journalist Joseph Menn, is really a public policy statement written as a thriller for a wider reading public. UPDATED 2:45 PM
IETV: the thinkerNet on film
5
of
2pm EST
Tue
Feb 23rd
2pm EST
Thu
Mar 4th
3pm EST
Tue
Mar 9th
an IBM information resource
sponsored content
big blue blog
Todd Watson
IBM is announcing today the first of its Power7 processor-based systems and the Power7 processor itself at an event in NYC.
white papers & case studies
an IBM information resource
sponsored content
Smarter Collaboration: How to Thrive in a Challenging Business Environment
Market conditions are changing faster than ever, and organizations need to improve their agility and adaptability in order to provide better service and improve processes. The ability to work with customers, business partners, and employees as effectively as possible - while at the same time holding down costs - is a key to success.

READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE!

REGISTER HERE
Wanted! Site Moderators
Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?

Please email: moderators@internetevolution.com
CMP Media LLC
Internet Evolution – not for thickies
Congress Hits the Snooze Button With China
Ira Winkler
In his
recent Congressional testimony, Dennis Blair, the U.S. director of national intelligence, stated that the U.S. is "severely threatened" by cyber attacks and that the recent Google (Nasdaq: GOOG) attacks should serve as a wake-up call.

CLICK FOR MORE
Lee H. Berke
The Decline & Fall of Broadcast Television

2|9|10   |   1:00   |   No comments


Want to know the future of broadcast television? Take a look at broadcast radio’s past.
Tom Nolle
Everything New Is Old Again

2|9|10   |   2:13   |   6 comments


Research shows that the youth of today like Facebook – but not blogging or Twitter. Does that mean Facebook has won, or just that it's not yet out of favor? Will all the services we see today fade into Ovaltine-or-Wheaties status in just a few years?
what.the.ferraro
Email Marketing Gets Desperate

2|8|10   |   2:31   |   4 comments


Promotional emails will use just about anything timely to get people to buy things. Seriously, anything.
Steve Saunders' Outernet
America, Truck Yeah!

2|8|10   |   1:42   |   5 comments


Steve likes his new Dodge Ram 1500, but hates Chrysler's Web non-sales strategy. Rant on, li'l buddy.
what.the.ferraro
Twits Go Wild for Resignation Tweet

2|5|10   |   1:48   |   4 comments


Jonathan Schwartz is the first Fortune 200 CEO to resign via Tweet. Can he walk on water, too?
Full Nelson
Go With the FLO, Part 2

Part 2 of 2   |  
See complete series
2|5|10   |   2:17   |   3 comments


Fritz and his sweater continue their review of Qualcomm's FLO TV.
Singer at C-Level
Goldilocks & the Data Center

2|4|10   |   3:39   |   2 comments


What kinds of companies are doing the most innovation in the data center? Turns out it's midtier enterprises that are taking the "Just Right" approach.
Full Nelson
Go With the FLO, Part 1

Part of 2   |  
See complete series
2|4|10   |   2:39   |   1 comment


Qualcomm's FLO TV gizmo streams live TV shows. Tragically, they include the O'Reilly Factor
Eurotrash
High & Dry in Barcelona

2|3|10   |   1:08   |   No comments


Ray’s heading to Barcelona for the Mobile World Congress, and he’s not happy about it, the miserable git.
Sweeney Blog
No Sex, Please... It's the Super Bowl

2|3|10   |   2:24   |   2 comments


The Super Bowl ads that CBS rejected are turning up online, generating lots of attention but zero revenue for the broadcaster.