What the hell??!? (blah, blah of a wannabe alien)
Monitor Everything
Okay, is anyone tired of my server admin tips yet? Yes? Too bad.
Monitor everything… Put as much info at your finger tips as easily as possible. Put that info in a place where you will always be looking at it for some reason. For example, I made a vBulletin plug-in that monitors 4 memcached servers (including a latency test it runs) as well as 10 blade servers. This shows every time I’m in the admin of the forum (which is a lot), so I can’t help but to not see it.
I wrote a little daemon that runs on my servers that can quickly report back whatever info I want (time, disk RAID status, server load, MySQL replication status, etc.)
The more info you have in one place (especially when you run a bunch of servers), the easier it is to see anything wrong. For example, I had an issue with a web server serving requests slow one day… it ultimately ended up not being a problem with the web server, but the memcached server it was using. The latency test was showing ~2,000 ms latency (2 seconds) vs. the normal 0.5 ms (1/2000 of a second).
And be proactive about monitoring stuff… don’t wait until something bad happens to start doing it! Then it’s too late.
Can I go to bed now?
| Print article | This entry was posted by Shawn on October 26, 2006 at 2:06 am, and is filed under Server Admin. Follow any responses to this post through RSS 2.0. You can leave a response or trackback from your own site. |
about 5 years ago
Very cool. Have you tried Fusion Charts? I’ve been using the standard charts for basic stats…but I have my eye on the real time capability of the instrumentation suite. Not really needed, but I like looking at pretty graphs…
-Patrick
about 5 years ago
Pretty cool if you have enough brains to do it, Nintendo does not, teach him to do this!
about 5 years ago
Pretty cool
.
Here’s one of my stupid Qs
; Looking at the chart, I saw “Pointer 64″. Based on what I *think* I know, that means the almighty DP servers are running on 64-bit servers?
.
Feel free to call me an idiot. heck! thats what i call myself
about 5 years ago
Don’t forget about the Sister Monitor that you have running…When you arn’t nerding out in front of your computer, guaranteed that the Sister Backup Monitor is. Granted, the sister backup can’t trouble shoot the problem beyond a phone call to say, “Hey dude, your website done be broke and shit…”. But hey, knowledge is half the battle!
about 5 years ago
Os this where to ask?
I share a DSL conection with neighbors and they love to use their “ares” musci downloader. You can imagine that each time they decide to open it i loose any bandwith avalaible and can not get connected to internet. No matter if I limit the bandwith in their software, anyway it practically “sucks” all the bandwith.
The router is a 2wire 1070 in a 1024kb conection.
Was getting into the firewall settings but can’t get how to do it to limit their computer and avoid them to suck all the bandwith…
Any suggestion will be apreciatte.
Thanks.
G.