One possible reason is that both soke-http and dispatch, the event-driven solutions, had a GC cycle kicking in, while both Apaches stayed just under the threshold. Quite a surprise is Apache’s classic HTTP client, which looks quite similar in its profile, although it uses plain old streams. It makes more efficient use of Netty than Twitter’s library, but Ning doesn’t do the best of jobs writing out to the target file. Even though this uses NIO, there’s still quite some copying of buffers going on:Ī close third is Dispatch, the de-facto “standard” HTTP client for Scala. Next is soke-http, which is a slim wrapper on Twitter’s finagle, which uses Netty under the hood. Most of the time is spent in the JDK classes, copying bytes around, and it also has the worst GC profile, using more than double the peak memory of Apache’s async (20MB vs 10MB): We’ll start with the slowest on the large file test, Bee client. The “async” aspect of NIO is more of a scalability feature than for raw throughput – the event-driven setup usually hidden in Netty or Grizzly is more valuable for servers that have to deal with a large number of not-too-busy connections. Where the “classic” IO is copying data around in memory from kernel / network socket to user / application buffer and back to kernel / file, NIO can do a lot of this avoiding multiple buffer copies and context switches between kernel / user. Marketing ftw.īesides the known, non-blocking / asynchronous aspect, NIO comes with a whole lot of very efficient, low-level IO operations mapping closely to the OS’ native implementations. NIO.2, by the way, seems to be “More New IO API”. It’s often mentioned that NIO is short for “non-blocking IO”, but apparently, it stands for “New IO API”. A word on NIOīefore we start to look on what’s going on under the hood, a word on NIO. Bee is far off, and interestingly enough, the classic Apache beats the Netty-based libraries this time.īut the winner, in overall download time, is still the discontinued commons-httpclient, using classic IO. NIO’s low-level IO operations are starting to make a difference, as Apache’s async client beats even cURL. Apache’s async client is already twice as fast, while the other solutions play in the mid-field, except for commons-httpclient, which is still the fastest. ![]() Medium fileīee is losing ground pretty quickly. Commons-httpclient comes second – given the size of the file, the deciding factor here is JVM startup, establishing a connection and request generation overhead. Small fileĬURL, being native, unsurprisingly beats the hell out of most JVM-based solutions. Running JDK 1.6 on OSX, all writes go to SSD. As much as the respective APIs allow, the tests factor out creation and startup of the client instances and try to measure pure request to disk speed. I’m downloading three binary files from a Nexus server on the local network: FileĮach test starts a JVM and downloads one file. In the results, there has been very little variance in the recorded times. To compensate, I’m doing a number of runs over time, averaging will still give a valid trend. I’m going to be a bit unscientific here – I’m running the test on my workstation, rather than on dedicated hardware in an isolated network. But what if I have a server in a local network, and bandwidth is no longer an issue? What difference does IO vs NIO make? The internet is full of rumours and outdated information on this, from “NIO is heaps faster” to “IO is the new NIO because it’s faster”. When downloading from a remote server, most likely the network connection is going to be the bottleneck. My use case is a bit more specific – downloading files, potentially large, from a fast network. I’m not going to do a general performance comparison – too many aspects. ![]() But what about the actual IO part? Especially regarding performance? A rather specific case Mostly, the focus is on new features, async APIs etc. Apparently, one of the unsolved programming problems of our time is making HTTP calls – at least, taken from the fact that new HTTP client libraries keep cropping up.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |