So, our Spam Map was slashdotted today and it was my first time experiencing it firsthand from the server side. I thought I’d share some interesting things that happened today. Just over 12 hours ago, (it’s now about 9:40PM) we were put on slashdot. As of right now, our hit counter for the page is at 99083 – almost 100K hits in a 12 hour period. WOW! By the time I finish this post it may very well clear 100K.
How did we handle it? Well, we are running a custom-made Java web server, written from scratch by Paul – this is running under Jetty as our servlet engine/webserver. Around 10:30AM EST, we REALLY started getting pounded. We started getting LOW ON THREADS warnings and then OUT OF THREADS warnings from org.mortbay.http.SocketListener.isLowOnResources. At that point, the server melted. We bounced the server and it ran out of threads almost immediately. POUNDED. Surprisingly, my map code and Paul’s mail code held up very well. Of course, Paul’s mail code handles a million emails a day and sometimes over 3 million a day, so his code was never in doubt.
As I write this, hits are now at 99283…
So the server ended up being down and out for a short period of time – 15 minutes total maybe? Which I thought was pretty damn good. One thing we did was make sure that the map page itself (and the data file it loads) was NOT a JSP page. We just served up static content essentially – no extra processing. A thread ran in the background to update the data file periodically. If the map page were a JSP getting the data dynamically, I’m sure it would have failed much sooner and for much longer.
99358…
The hardware is a dedicated server from ServerBeach. I don’t know the specs but can find out if anyone is interested.
Just read some emails and the hit counter is now at 99709
What else?…..
Oh – I thought this was kind of interesting…with all the traffic today – lots of people were trying out the system by sending a test mesage to mailinator. MANY of them sent an email with the word test as the subject. BUT…..that triggered our spam filters because we got hundreds of emails with the same subject so we started rejecting (bouncing) them as spam. Of course, these people then assumed that mailinator doesn’t work. So, if you are testing it, send a message with a subject other than test.
OK, that’s it for now. I’ll add on more stuff later as/if I think of it. Oh – maps don’t seem to work in Safari – gotta look into that….and the map popups in IE are centered when they should be left-justified. Minor problems I need to fix.
Counter at 99850…ok – let me wait til it crosses 100K…
10:10PM – 99922
10:12PM – 99991
10:13PM – 100021 !!!!!
9:34AM to 10:13PM – 12 hours and 39 minutes to reach 100K hits
Man, this was fun! I used to dream about creating something that was cool/geeky enough to get slashdotted and it has now happened! Cool.