Is your website easily accessible by search engines? If a search engine has any trouble crawling your site then all of your SEO efforts are wasted. Imagine how much energy you might waste. Is there a good way to crawl your site like a search engine would?
The SEO Spider tool from Screaming Frog is a free application that lets you crawl your site like a search engine and find any trouble spots you can correct.
The spider tool is free for up to 500 URLs of your site. If you have more than 500 URLs you’ll need to upgrade to the paid version.
In this article, we’ll cover:
For this article, you’ll just need access to the Internet and your computer’s admin password for installing software.
Site crawlers are an important tool for every web master or even a casual user who managed any website large or small. The crawler will give you detailed information that would otherwise take you hours to discover—and it does it in mere minutes. All you need to is provide your URL and set the crawler loose to do its work. After the crawl is complete, you’ll have important data like:
- Which URLs are being crawled and which are blocked
- What pages have broken links
- What pages have images
And much more. Having this data at hand will give you a whole laundry list of optimizations you can make to improve your site’s crawl efficiency.
Below, we’re going to learn how to grab the installer scripts for the SEO Spider tool and activate them on your computer. The installation procedure may be slightly different for various operating systems, but if you use Mac, Windows, or Ubuntu, it should be self-explanatory.
- Go the Screaming Frog homepage
- Click Download under SEO Spider
- Select your operating system
When your download completes, open the the download package and follow the installation instructions for your operating sytstem.
Well done! You’ve just installed the SEO Spider tool to help you accomplish more with your SEO goals. Stay tuned for our next article, in which we will run the crawler on a live site.