Hello All,
As all of you have observed, there have been intermittent performance issues with the forum for the past few weeks. I've been trying to isolate the source of the problem and while not complete, I wanted to provide the picture as I see it so far.
Observations:
* Overall Server load is much higher than normal ( Load average 15+ vs normally less than 2 )
- The high load process is the web-server itself.
- server logs do *not* show any signs of abnormal behavior
* Server bandwidth lower than normal - Users are unable to browse like they normally do. When performance is good, bandwidth use is higher, so this issue is not bandwidth based.
* About a week ago, one of my own personal servers started exhibiting the same behavior. The server in question is *not* a high load public server. It's one I use for very few hosting needs and is typically not linked or indexed by search engines.
Conclusions:
* I believe this to be a long term, distributed attack by automated machines probing for vulnerabilities in the server software.
* Each probe does not generating much in the way of bandwidth.
* Each probe does generate a tremendous load as the attacker tries to penetrate the server by overloading processes while looking for weakness.
The Wrap Up:
Unfortunately, due to the nature of the probes / attacks, it is very difficult to block from our end. Over the next few days, I'll be working to optimize our server platform in hopes that the extra serving capacity gained from the optimizations allows us to see more normal browsing.
We certainly appreciate your patience and understanding as we work to get things back to normal.
LK