Internal dummy connection error in apache access log

A problem that we had with a Fedora Core dedicated server, sometimes it became unavailable on all ports, services, ssh, etc. Server was available after a power cycle but it was back offline in around 24 hours.

Right after power cycle I temporary disabled the sites with most traffic, if you have Plesk installed then it is easy, you can click a button next to each domain name. With most server issues if you stop the activity on server you have more chances to keep it online until problems are fixed.

Then checking the apache logs through SSH, in /etc/httpd/access_log I found many instances of this error, this was as often as once a second, I think it depends on the traffic you had on that server. Depending on your IP configuration you might eider see "::1" or "", servers with IPv6 see "::1"

::1 - - [13/Oct/2010:12:10:10 +0300] "GET / HTTP/1.0" 403 3956 "-" "Apache/2.2.6 (Fedora) (internal dummy connection)" 
::1 - - [13/Oct/2010:12:10:11 +0300] "GET / HTTP/1.0" 403 3956 "-" "Apache/2.2.6 (Fedora) (internal dummy connection)" 
SERVER OFFLINE HERE (no log data for 7 hours)
::1 - - [13/Oct/2010:19:47:12 +0300] "GET / HTTP/1.0" 403 3956 "-" "Apache/2.2.6 (Fedora) (internal dummy connection)" 
::1 - - [13/Oct/2010:19:47:12 +0300] "GET / HTTP/1.0" 403 3956 "-" "Apache/2.2.6

And inside /etc/httpd/error_log you might find something like this:

FATAL: emalloc(): Unable to allocate 80496387 bytes

Server ran out of memory because of the very often errors and it crashed.

More on the error

Apache wiki explains this dummy connection error, this is because Apache sends some signals to itself in order to wake some listening connections, they say this is normal but might only cause problems on servers with much traffic.

The solution

Edit or add the "CustomLog" directive in your /etc/httpd/conf/httpd.conf file to skip log of these errors. What I did on this server was: I replaced this line:

CustomLog logs/access_log combined

with this:

SetEnvIf Remote_Addr "::1" loopback
SetEnvIf Remote_Addr "127\.0\.0\.1" loopback
CustomLog logs/access_log combined env=!loopback

Then restarted the server and internal dummy connection was no longer logged in the access_log.

Additional notes

In parallel this this issue I also noticed (in error_log) too many instances of this other error:

[error] [client ::1] Directory index forbidden by Options directive: /var/www/html/ 

Simplest solution for this was to create a blank index.html file in that /var/www/html/ directory, the only content in that directory was Awstats folder, it might be related to that, eider way, you should prevent this error from logging to avoid server crash because of memory usage.