404In this article we'll discuss the possible impact from the handling of 404 errors. These are caused by requests to pages or resources that don't exist on the server.

By default a lot of the applications you might install on your website, will try to process requests for a page that doesn't exist dynamically.

What this can mean, is each time someone mistypes a URL, your application is going to query the database to try to figure out if that page exists, and if it doesn't it's going to serve up a page not found error to the visitor.

In most cases this type of default 404 error page activity is completely harmless. However as the amount of 404 error pages you're forcing the server to think about increases, it can become more and more problematic.

Causes of 404 errors

Let's take a look at some of the most common reasons a 404 error might be generated.

  • A visitor tries to manually type in a page that doesn't exist.
  • Your pages were not internally or externally linked correctly in the first place.
  • Resources such as images, .css, .js, or other files were removed from the server.
  • One of your pages was moved, and links either internal or external weren't updated.
  • Search engine crawlers, or automated bots try to access generic pages that your site doesn't have.

404 error page handling

A good webmaster will want to periodically check on their website statistics to see what requests are generating the most 404 errors.

You can use this information to your advantage, as lets say for example you notice 500 people a month are trying to pull up a page called /about that doesn't exist. You can go ahead then and create that page, or setup re-direction on that URL to hopefully get them pointed in the right direction and have them remain on your site during future visits.

Viewing 404 errors via Awstats

  1. awstats-click-on-404sYou can view your 404 errors via Awstats from within cPanel.

    Once you're logged into Awstats, if you simply scroll to the very bottom of your statistics report, you should see the total number of 404s encountered and what percentage of your traffic that represents.

    Then simply click on the 404 link to see the actual requests triggering these errors.

  2. awstats-404-detailsIn this example you can clearly see that the URL /directory/search-all.php is by far the biggest cause of the 404 errors on this site.

    You can also see that there are several resources such as .css stylesheets and .png and .gif images that are missing.

    Note the Referrer of these resources, as those would be the pages with the invalid links to these files that don't exist.

Creating static custom 404 error pages and handling excessive requests

Now that we've seen a little bit about what a 404 error is, you might wish to dive deeper and configure a custom 404 error page to help steer your visitors to other sections of your site when then try to access a page that doesn't exist.

Or if you're having problems with too many 404 errors causing performance issues a static 404 error page can help with this as well. All of the following articles go more in-depth on handling 404 errors and would be recommended reading for anyone looking to get the total big picture on 404 error handling.

Did you find this article helpful?

We value your feedback!

Why was this article not helpful? (Check all that apply)
The article is too difficult or too technical to follow.
There is a step or detail missing from the instructions.
The information is incorrect or out-of-date.
It does not resolve the question/problem I have.
How did you find this article?
Please tell us how we can improve this article:
Email Address
Name

new! - Enter your name and email address above and we will post your feedback in the comments on this page!

Related Questions

Here are a few questions related to this article that our customers have asked:
Wordpress dashboard scrambled
Would you like to ask a question about this page? If so, click the button below!
Ask a Question
n/a Points
2014-05-01 1:46 pm

My Pages Not Found today lists 368 type 404 errors. 146 were from one IP.  It is an education network which of course does not identify its clients  Many users would not be particularly familiar with the stringent naming systems in internet activity.  71 are .js files and 47 are .css files; these do not exist on my web page.  So there can be systematic (i.e. big) errors "out there" that result in attempts to find files that do not exist on one's website.

Staff
9,968 Points
2014-05-01 2:15 pm
Hello Michael, and thank you for your comment.

If you are experiencing a large amount of 404 errors from a particular IP address, or a particular User-Agent string, you could use some .htaccess deny rules to block their access.

If you are seeing a large amount of requests for .js or .css files not being found, you might want to double-check to ensure the referring URL the user was on isn't making their web-browser try to request those non-existent files because of your HTML code itself.

You can also use .htaccess rules to block a particular page from being requested if you keep seeing the same one happen. For instance if the file 404.js kept getting requested, and you've verified it is not your code calling it, and that the file doesn't exist. You could then use this bit of code in your .htaccess file to deny any requests for it:

RewriteEngine On
RewriteCond %{REQUEST_URI} ^/404\.js$
RewriteRule .* - [F,L]


Thanks again for your comment, and let us know if you have any questions at all!

- Jacob

Post a Comment

Name:
Email Address:
Phone Number:
Comment:
Submit

Please note: Your name and comment will be displayed, but we will not show your email address.

2 Questions & Comments

Post a comment

Back to first comment | top

Need more Help?

Search

Ask the Community!

Get help with your questions from our community of like-minded hosting users and InMotion Hosting Staff.

Current Customers

Chat: Click to Chat Now E-mail: support@InMotionHosting.com
Call: 888-321-HOST (4678) Ticket: Submit a Support Ticket

Not a Customer?

Get web hosting from a company that is here to help. Sign up today!