Screaming Frog is a favorite spider tool highly recommended by a lot of SEOs. It has a lot of features and is very customizable. It runs on PCs, Macs and Linux machines. You can view, analyse and filter the crawl data as it is collected and updated continuously in the program’s user interface.
The tool permits you to export key onsite SEO components (URL, page title, meta description, headings, etc) to Excel so it can be used as a base to make SEO decisions.
If you pay for the full version at £99 per year, you’ll get
- The 500 URI crawl limit is removed
- You can access ALL the configuration options
- You can save & re-upload crawls
- You can search for anything in the source code, & collect any data from the HTML of a URL using XPath, CSS Path or regex
- You can connect to the Google Analytics API & pull in data directly during a crawl
- You get support for any technical issues with the software
Now, if you’re strapped for cash and want something that’s totally free, we have three free alternatives for you to check out.
Let us know which of the 3 you like the best and why on the comments section below.
Visual SEO Studio is a Windows Product. No Mac love.
There actually is now a version of VIsual SEO Studio running on Mac OS X
Try Webbee SEO Spider, it is available for both Windows and Mac platform. Good thing is they are offering full featured product up-to 500 URLs totally free and also offering 1 month free trial with unlimited URLs.
Also Visual SEO Studio offers one month free trial with 150000 URLs, just register with an email address (they send you the activation code).
Gotta say – Beam Us Up, the OS X version doesn’t work on newer OS X versions because you get “unable to load java runtime environment”. Simple solution, you install Java at https://java.com/en/download/mac_download.jsp.
Couple issues though… you can’t bulk test urls, and I can’t seem to find a way to paste URLs into it – you have to type them. But it DOES work, and it’s the only good & free one I’ve found on OS X.
Hello
A good alternative for windows is hextrakt, which can crawl very big websites, thanks to a new crawl technology, in a nice interface.
With the free version you can crawl up to 20 000 URLs, number of crawls are unlimited.
Thanks Olivier. I’ll check it out 🙂
Is there any reason to switch from Screaming Frog to any of the above?
The extremely high cost of £149.00 Per Year.. so tired of the yearly cost when the product works fine without the yearly cost.
I think its due to the annual fee associated with ScreamingFrog.
See also this program, https://nazuke.github.io/SEOMacroscope/ that runs on windows.
appears to be totally free, with quite detailed options + settings.
It will be nice to have a comparison table for few of these technical seo apps. There is also the opensource Link Checker. Not sure if it is as good as the paid options. Iwummel.github.io/linkchecker
Xenu’s Link Sleuth is also a great tool to crawl website and detect broken links and link issues.