The Basics

What is Crawlability

Crawlability can be defined as the ability of a search engine to access content on a page and crawl that content and collect data.

• Improving internal links helps Google’s crawler to find all the content in website.
• Web crawlers stay on your site for a limited time, so it is important that the page loads fast and that the sitemap is submitted.
• You should constantly monitor whether web crawlers can access your site and let the search engine know what they find.

Recommended reading.

Tools for managing crawlability:

• Google Search Console
• Log File Analyzer
• Site Audit

Indexed Pages

You can make the following query to see the your (or competitors’) indexed pages on Google

site: website.com

We use this to understand if there are indexing problems such as

• pages that are not indexed
• indexed pages that shouldn’t be indexed (sign-in, my-account, paywall protected stream, etc)

Noindex