Raidbots Blog

Web Tools for WoW Nerds

4/1 – New Server Inc

I went ahead and started the process for getting an honest-to-god dedicated server for raidbots.  Lots more disk than I currently have, a little more RAM, and a decent dedicated processor.  Once I have access, I’ll be starting to set up the new machine and migrating the code and data over.  It’ll probably take a week or two to find the time to get it all set up and working (not to mention handling DNS and all that fun with minimum downtime).  This is going to take precedence over any other development but once it’s done I’ll be able to dive headlong into making sense of the huge data dump WOL is providing me.  First task once the migration is done – playing around with the Top DPS Bot and possibly creating the Average DPS Bot (to answer the age old question “what is the ACTUAL average DPS across all levels of raiding players (that log to WOL)?”

Nerdy Server Stuff

Onto the details!  I’m moving from Linode to Server Beach.  I love Linode and am sad to leave but they just don’t offer disk space at a price I can afford.  I’m using Mongo as the database for raidbots and my usage require a ton of disk space – I could probably reduce the size of each document by compressing keys more intelligently but the rate of growth would likely still require moving to a server with proportionally more disk space than RAM and CPU.  Also, at this point, I’d rather keep the keys in the documents as readable as possible to make it easier to develop against.

Turns out that most cloud hosting tends to have really limited disk options.  Most of the disk sizes scale in lock step with the amount of RAM on the node.  Elastic Hosts was one exception with reasonable prices but their one datacenter in San Antonio has a pretty high latency to Boston (where I live and develop).  In addition, their bandwidth was pretty expensive.  Amazon’s EC2 tended to be far too expensive for me as well – also, any solution that requires a price calculator that is more complicated than raidbots itself is probably more than I want to deal with.

The new server is a Core 2 Duo 6750, 2GB RAM, 2×160 GB drives, and essentially unlimited bandwidth (10Mbit NIC).  Hopefully I should actually get access to it in a day or two and can start the process of configuring Debian, setting up Mongo/Apache/PHP/etc/etc/etc, and start migrating the code and data.

I’m also hoping that a few downtime blips might have been caused by being on a shared machine.  There was a 5-10 minute downtime this week that seemed to be related to Mongo taking a long time to flush data to disk.  I’m not sure if this is just going to end up happening with a single Mongo instance or if the disk my nodes were using were being saturated by another machine on the same physical host.  We shall see!  At least with a dedicated server I’ll know that it’s my own fault if there’s a massive performance problem.  I’m a little worried that running everything on one box might end up causing some RAM contention issues (Mongo likes to slurp up every bit of available RAM but theoretically should let go of it if needed) but there’s only one way to find out.

Anyway, I’m excited if you couldn’t tell.  Once this is set up, I should be able to jump back into developing code and doing cool stuff with the new data.  It’ll be a huge relief not to have to spend a ton of time working on maintenance tools to make sure I’m not overstepping my minimal resources.

Update: I love it when companies under-promise and over-deliver.  They said it would be ready in 24-48 hours but it was ready in about 4.  I love the smell of a new server in the evening.


4 responses to “4/1 – New Server Inc

  1. Rodalpho April 1, 2011 at 5:22 pm

    Rather than (or in addition to) the average DPS bot, I know I would personally find a median DPS bot more useful, as it would reduce the impact of outliers.

    Additionally, some way to exclude fights with gimmicky mechanics like Halfus would be a good idea.

    Looking forward to seeing the additional datapoints. The current top DPS graph is just too spiky, pretty clearly due to insufficient data.

    • Rodalpho April 1, 2011 at 5:24 pm

      Also, including both buff and nerf hotfixes as vertical lines on the graphs as well as automatic data delimiters would be very useful. For example, shadow priests were nerfed a full 10% in a post 4.0.6 hotfix, but for weeks afterwards the pre-nerf data was still being included.

      • seriallos April 1, 2011 at 7:35 pm

        Re: more indicators of nerfs and buffs – I’ve considered this but end up with 2 things that hold me back:

        – Too many indicators like this add a lot of clutter to an already cluttered chart. I could potentially “grade” indicators based on their importance (patch, major hotfix, minor hotfix) and provide a way to show/hide information.
        – I generally don’t know enough about every spec to be able to effectively judge hotfix notes to know if it’s major or minor so I’ve stayed away from indicating anything but fairly major patches. Feel free to post here or email me if I’m definitely missing something – a date and a description would be ideal 🙂

    • seriallos April 1, 2011 at 7:26 pm

      I’ll definitely take a look at measuring both average and median. I’ll be experimenting with alternate “overall” measurements that don’t just mash all the DPS values together since I don’t think it provides a value that is overly meaningful. My thought right now is more along the lines of measuring the overall percent contribution to a fight rather than the raw DPS (e.g., MM hunters are the top DPS for fight A so count them as 100% for that fight, next class down did 95% of that amount, etc). If a class is clearly at the top of every fight, they’ll end up with a higher average percent and bubble up to the top of the overall list. This is a rough idea right now and I’ll see how it looks when I get to coding. This also both reduces the effect of high DPS fights like Halfus without throwing out the data entirely.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s