SEO Audits & On-Page Crawler

What the Crawler Checks

The SEO crawler acts as a Googlebot simulation. It fetches each page on your site using a Googlebot-style user agent and inspects the HTML DOM for compliance with on-page SEO best practices. Here is what it audits on every page:

• Canonical Tag Validation: Checks whether a <link rel="canonical"> tag exists and whether it points to the correct URL. A canonical pointing to a different URL is flagged because it signals to Google that this page should not be indexed independently.
• Meta Description Presence & Length: Verifies that a meta description exists, is not empty, and falls within the recommended 120–160 character window. Missing or truncated descriptions reduce click-through rates from search results.
• Title Tag Completeness: Confirms the <title> tag is present, unique, and within the optimal character count. Pages without a title receive zero SEO weight in search results.
• Heading Hierarchy (H1 → H2 → H3): Inspects the heading tag structure across the page. Multiple H1 tags, skipped heading levels, or completely missing H1s are flagged as critical issues because they confuse how search engines understand your page's topic.
• Image Alt Attributes: Checks all <img> elements for a non-empty alt attribute. Missing alt text affects both accessibility and image search indexing.
• Noindex Directives: Scans both the <meta name="robots" content="noindex"> tag and the HTTP X-Robots-Tag response header. Pages that accidentally block indexing (common after staging environment configs are accidentally deployed) are immediately flagged as Critical.
• HTTP Status Codes: Any page returning 4xx or 5xx status codes is flagged, since broken pages cannot be indexed by search engines.
• Outlink Count: The number of outbound links is tracked per page as a structural metric, helping identify overly thin or overly linked pages.

The crawler uses 15 concurrent requests with a 5-second timeout per page and a 15-second total wall-clock budget, so even large sites complete in seconds without hanging the dashboard.

Audit Severity Levels

Every issue is classified into one of three severity tiers:

• 🔴 Critical: Directly prevents or significantly harms search indexing. Examples: page blocked by noindex directive, missing canonical tag on a paginated series, H1 completely absent, HTTP 404 or 500 status.
• 🟡 Warning: Reduces search ranking potential but does not block indexing. Examples: meta description too short, title tag exceeds 60 characters, image missing alt text.
• 🔵 Info: Informational signals that could be improved for best practice compliance. Examples: page has no outbound links, heading structure has minor inconsistencies.

Each issue record includes: - The exact page URL where it was found - The specific HTML element or selector that triggered the rule - The actual value detected vs. the expected value - Whether an auto-fix is available (some issues can be resolved with a single click)

SEO Health & Competitive Edge Scores

After every completed audit, two composite scores are calculated:

• SEO Health Score (0–100): A weighted average across all audited pages and rule categories. Critical issues reduce this score significantly. Fixing them increases it. Your score history is saved daily, so you can track recovery after a site update or migration.
• Competitive Edge Score: A secondary score that measures your content's structural differentiation (heading depth, content length signals, and internal linking richness) relative to typical industry patterns.

Both scores are stored per day, giving you a clean trend line showing whether your site is improving or regressing after content changes.

Comparing Audits Over Time

The platform supports direct audit comparison. Select any two completed audits and get a side-by-side diff:

• New Issues: Problems that appeared in the later audit but did not exist before (e.g., a page was accidentally set to noindex after a CMS update).
• Fixed Issues: Problems present in the earlier audit that no longer exist (confirming your fixes worked).
• Score Change: The numeric delta between the two SEO Health Scores.
• Category Breakdown: A ranked list of which issue categories improved or worsened the most.

This makes it easy to confirm that content updates or developer fixes had the intended effect.

Frequently Asked Questions