A technical SEO audit is essentially a controlled investigation of whether search engines can discover, crawl, understand, index, and efficiently serve the right pages on your site.

The best SEOs don't simply export 200 errors from a crawler. They determine which problems actually affect organic visibility, prioritize them by business impact, and then validate the fixes in Google Search Console.

The best technical SEO audit workflow

I recommend this stack:

• search.google.com — the source of truth for Google's view of your site.
• screamingfrog.co.uk — my preferred deep crawler for professional technical audits.
• pagespeed.web.dev — Core Web Vitals and page performance.
• search.google.com — structured-data validation.
• Ahrefs Site Audit or Semrush Site Audit — excellent for automated monitoring and prioritization. Ahrefs currently advertises 170+ technical/on-page checks, while Semrush advertises 140+ checks.

If I had to choose one paid crawler for a serious manual audit, I'd generally choose Screaming Frog because it gives you unusually granular control over the crawl and raw data. For ongoing monitoring, Ahrefs or Semrush can be more convenient.

---

Technical SEO audit checklist

1. Crawlability

First ask:

Can Googlebot actually reach the pages that matter? Check:

• robots.txt exists and returns 200
• Important sections aren't accidentally disallowed
• No important URLs blocked by robots.txt
• XML sitemap exists
• Sitemap returns 200
• Sitemap contains canonical, indexable URLs
• Sitemap doesn't contain 3xx/4xx/5xx URLs
• Important pages are internally linked
• No major orphan pages
• Crawl depth isn't excessive
• No infinite URL/calendar/faceted-navigation traps
• Server isn't returning excessive 5xx errors
• Crawl budget isn't being wasted on useless URLs

A common mistake is treating robots.txt as an indexing mechanism. It isn't. Blocking crawling doesn't necessarily prevent a URL from appearing in Google's index.

Google's own documentation should be the final authority when you're unsure how a directive behaves.

---

2. Indexability

Next ask:

Of the pages Google can crawl, which ones is Google actually allowed and likely to index? For important URLs, check:

• HTTP status = 200
• No accidental noindex
• No conflicting X-Robots-Tag
• Correct canonical
• Canonical is self-referencing where appropriate
• Canonical URL returns 200
• Canonical isn't blocked
• Canonical isn't redirected unnecessarily
• Important pages are indexed
• Low-value pages are intentionally excluded
• No unexpected duplicate/indexed parameter URLs
• No staging/dev URLs indexed
• No internal search-result pages indexed
• No thin filter combinations unnecessarily indexed

Then compare three things:

Your crawl → XML sitemap → Google Search Console

That's where you start finding discrepancies that a crawler alone can't explain.

Google Search Console's Page Indexing report is particularly valuable because it shows Google's reported reasons for URLs not being indexed.

---

3. Status codes & redirects

Run a complete status-code analysis.

Look for:

• 200 pages
• 301 redirects
• 302 redirects
• 404 pages
• 410 pages
• 5xx errors
• Redirect chains
• Redirect loops
• Internal links pointing to redirects
• Internal links pointing to 404s
• Redirects pointing to irrelevant pages
• Old URLs with valuable backlinks redirected appropriately

Particularly important

Don't automatically "fix" every 404.

A 404 can be perfectly correct if the content genuinely no longer exists.

The important questions are:

Does the URL receive organic traffic? Does it have backlinks? Does an equivalent replacement exist? Is Google still crawling it heavily?

---

4. Canonicalization

Canonical problems can become extremely messy on large sites.

Check:

• Every indexable page has a canonical
• Canonical points to the preferred URL
• Canonical URLs are absolute
• Canonical URLs return 200
• Canonicals aren't redirected
• Canonicals aren't noindex
• Canonicals aren't blocked by robots.txt
• HTTP → HTTPS canonicalization is correct
• www/non-www canonicalization is correct
• Trailing-slash behavior is consistent
• Parameter URLs are handled appropriately
• Pagination isn't creating unintended duplicates
• Duplicate product/category URLs have appropriate canonicals

Don't blindly set every duplicate URL to the homepage. The canonical should identify the equivalent preferred version, not simply be a dumping ground.

---

5. Site architecture & internal linking

This is one of the most valuable—and most neglected—parts of a technical audit.

Check:

• Important pages are reachable through internal links
• Important pages aren't buried unnecessarily deep
• Navigation reflects the site's topical hierarchy
• Category → subcategory → product relationships make sense
• Blog content links to commercial pages where relevant
• Orphan pages are identified
• Broken internal links are removed
• Redirecting internal links are updated
• Anchor text is descriptive
• No excessive sitewide links
• No huge numbers of irrelevant footer links
• Faceted navigation is controlled

Think of internal links as SEO infrastructure.

They influence discovery, crawl paths, topical relationships, and the distribution of internal authority.

---

6. Duplicate content

Look for:

• Duplicate titles
• Duplicate meta descriptions
• Duplicate H1s where they indicate duplicated pages
• Near-duplicate pages
• Printer versions
• URL parameters
• HTTP/HTTPS duplicates
• www/non-www duplicates
• Case-sensitive URL duplicates
• Trailing-slash duplicates
• Duplicate category/product paths
• Syndicated content
• Faceted-navigation duplicates

But don't make the mistake of assuming:

"Duplicate content = Google penalty." Usually the problem is wasted crawl/indexation and unclear URL selection, not a penalty.

---

7. JavaScript rendering

This is increasingly important for modern sites.

Test whether important content and links exist in:

Raw HTML → rendered HTML

Check:

• Main content exists without problematic JS execution
• Important links are discoverable
• Canonical exists correctly
• Meta robots directives render correctly
• Structured data is present
• Navigation isn't dependent on problematic JS
• Critical content isn't hidden from crawlers
• Client-side routing doesn't create indexing problems
• SSR/SSG/hydration works correctly

For JavaScript-heavy sites, crawl the site with JavaScript rendering enabled and compare the rendered output against what Google is expected to see.

---

8. On-page technical elements

For every important indexable page:

• One clear <title>
• Unique title
• Appropriate title length/content
• Meta description exists where useful
• Unique meta description
• One appropriate H1
• Logical H2/H3 hierarchy
• Correct language declaration
• Correct canonical
• Correct robots directives
• Images have appropriate alt text
• Open Graph metadata where relevant
• Twitter/X metadata where relevant

Don't obsess over arbitrary character-count rules. Relevance and usefulness matter more than hitting a particular pixel/character limit.

---

9. Structured data

Audit:

• Schema exists where appropriate
• Schema matches visible page content
• Required properties are present
• No major validation errors
• Organization schema
• Breadcrumb schema
• Product schema
• Article schema
• LocalBusiness schema
• Review schema where legitimately applicable
• FAQ schema only where appropriate
• Structured data isn't misleading

Validate important templates with Google's Rich Results Test rather than relying solely on your crawler.

---

10. Core Web Vitals & performance

Check:

• LCP
• INP
• CLS
• Mobile performance
• Desktop performance
• Real-user data where available
• Large images
• Unused JavaScript
• Render-blocking resources
• Excessive third-party scripts
• Poor caching
• Slow server response
• Excessive DOM size
• Font-loading problems

The important distinction is:

Lab data = diagnostic Field data = what real users experienced

Don't spend three weeks chasing a Lighthouse score from 92 → 98 if the actual SEO problem is that 30,000 important URLs aren't indexable.

---

11. Mobile SEO

Google uses mobile-first indexing, so inspect the mobile version rather than assuming desktop represents the site.

Check:

• Same important content on mobile
• Same important links
• Same structured data
• Same canonical
• Same robots directives
• Responsive design works
• No intrusive interstitials
• Navigation works
• Text is readable
• Buttons/links are usable
• Mobile performance is acceptable

---

12. HTTPS & security

Check:

• Entire site uses HTTPS
• HTTP redirects to HTTPS
• No mixed content
• Certificate is valid
• Correct HTTPS canonical
• No HTTP sitemap URLs
• No HTTP internal links where avoidable
• Security headers/configuration aren't causing crawler problems

---

13. International SEO

If the site operates internationally:

• hreflang implementation is correct
• Language/region combinations are valid
• Self-referencing hreflang exists
• Return tags exist
• URLs return 200
• Hreflang URLs aren't blocked
• Canonicals don't conflict with hreflang
• Language versions aren't unintentionally canonicalized together
• XML sitemaps are correctly segmented if appropriate
• Country targeting makes sense

Hreflang errors can be especially painful because one small implementation mistake can affect thousands of URLs.

---

14. Log-file analysis — advanced SEO

For large websites, this is where you move from a good audit to an advanced technical SEO audit.

Analyze server logs to determine:

• What Googlebot actually crawls
• How frequently Googlebot crawls URLs
• Which URLs consume crawl activity
• Whether Googlebot spends time on parameters
• Whether Googlebot hits 404s
• Whether Googlebot encounters 5xx errors
• Which pages Googlebot never visits
• Whether important pages are crawled efficiently

This is particularly valuable for:

• Ecommerce
• Marketplaces
• Large publishers
• Programmatic SEO sites
• Sites with millions of URLs

A crawler tells you what it can discover.

Log files tell you what Google actually requested from your server.

---

15. Sitemap audit

Your sitemap should essentially answer:

"Here are the URLs I consider important and want search engines to know about." Check:

Sitemap URL
    ↓
200 response
    ↓
Canonical URL
    ↓
Indexable
    ↓
Important page

Ideally, don't have:

Sitemap
   ↓
301
   ↓
200

Sitemap
   ↓
404

Sitemap
   ↓
noindex

Sitemap
   ↓
canonical → different URL

---

16. Search Console validation

After the crawl, go into GSC and investigate:

• Performance
• Page indexing
• Sitemaps
• Core Web Vitals
• HTTPS
• Manual actions
• Security issues
• URL Inspection
• Crawl statistics, where available

This step is critical because crawler tools are simulations.

Search Console gives you Google's own data about your property.

---

The actual audit process I'd use

Here's the workflow I'd give an SEO specialist.

Phase 1 — Baseline

Collect:

• Organic traffic
• Organic conversions
• Indexed pages
• Top landing pages
• Top-performing directories
• Current rankings
• Search Console data
• Sitemap URLs
• Analytics data

Phase 2 — Crawl

Run Screaming Frog/Ahrefs/Semrush.

Export:

• Status codes
• Indexability
• Canonicals
• Redirects
• Titles
• H1s
• Meta descriptions
• Internal links
• External links
• Crawl depth
• Structured data
• Images
• JavaScript
• Pagination
• Hreflang

Phase 3 — Google validation

Compare your crawl with:

GSC + sitemap + actual search results

This is where you discover things like:

"We have 50,000 URLs in our crawl, but Google has only indexed 12,000—and 30,000 of those URLs shouldn't exist in the first place." That's much more valuable than saying "the site has 147 warnings."

Phase 4 — Prioritize

I use a simple framework:

For example:

P0: robots.txt blocks /products/

P0: Canonical on every product page points to homepage

P1: 40,000 internal links point to 301 redirects

P1: 15,000 valuable pages have noindex

P2: Duplicate title tags on 500 low-traffic pages

P4: A handful of titles are slightly longer than Google's typical display width

This prevents the classic SEO mistake of treating every crawler warning as equally important.

---

Which tool should you use?

Best overall professional setup

Screaming Frog + Google Search Console + PageSpeed Insights

That's my preferred core stack.

Best all-in-one SEO platform

Semrush

Its Site Audit currently advertises 140+ checks covering crawlability, HTTPS, mobile, Core Web Vitals, markup, international SEO, internal links and more.

Best alternative all-in-one

Ahrefs

Ahrefs Site Audit currently advertises 170+ issues and includes crawling, indexability, redirects, JavaScript, robots.txt, sitemaps, structured data, internal links and performance analysis.

Best free starting point

Google Search Console + Ahrefs Webmaster Tools

Ahrefs specifically recommends combining its Webmaster Tools with GSC for an SEO audit.

Best for enterprise/very large sites

I'd use:

Screaming Frog / enterprise crawler + GSC + server logs + analytics + PageSpeed/CrUX + database/URL data

At millions of URLs, the audit becomes as much data engineering as traditional SEO.

---

The biggest mistake to avoid

Don't produce an audit that looks like this:

"We found 2,347 errors and 8,921 warnings." That's not an SEO audit.

A useful audit looks more like:

Finding: 18,400 product URLs are being generated through faceted navigation. Impact: Google is spending crawl resources on low-value combinations while important products have weak crawl paths. Evidence: 72% of Googlebot requests in logs are parameter URLs. Recommendation: Restrict crawlable facet combinations, improve internal linking to canonical product/category URLs, and remove unnecessary parameter URLs from the sitemap. Priority: P1 Owner: Engineering/SEO Validation: Re-crawl + GSC + server-log comparison after deployment. That's what turns a crawl report into a technical SEO strategy.

If you're learning technical SEO, I'd memorize this sequence:

Crawl → Discover → Render → Index → Canonicalize → Internal Link → Perform → Validate → Monitor

And use Google Search Console as the source of truth, a crawler as your diagnostic engine, and business impact as your prioritization system.

---

Which tool should you use?

Best overall professional setup

Screaming Frog + Google Search Console + PageSpeed Insights

That's my preferred core stack.

Best all-in-one SEO platform

Semrush

Its Site Audit currently advertises 140+ checks covering crawlability, HTTPS, mobile, Core Web Vitals, markup, international SEO, internal links and more.

An SEO specialist performs a technical SEO audit by systematically evaluating how search engine crawlers access, crawl, render, and index a website . Rather than focusing on keywords or content quality, a technical audit investigates the underlying infrastructure to eliminate roadblocks preventing the site from achieving its maximum visibility.

How an SEO Specialist Performs an Audit (The Process)

The Best Technical SEO Checklist

• Crawlability and Accessibility: Verify that robots.txt isn't blocking critical assets and ensure the XML sitemap is clean, updated, and submitted.
• Indexation Status: Check for accidental noindex tags, canonical tag misalignments, and duplicate content issues.
• Site Architecture and URLs: Ensure a shallow depth structure (important pages within 3 clicks) and clean, lowercase, hyphenated URL paths.
• HTTPS and Security: Confirm valid SSL certificates, secure mixed content fixes, and proper HTTP-to-HTTPS redirects.
• Core Web Vitals & Performance: Optimize Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and Cumulative Layout Shift (CLS).
• JavaScript SEO: Verify that critical content and links render on the server-side or hydrate correctly for client-side frameworks.
• Structured Data (Schema): Validate that JSON-LD markup accurately matches visible on-page content without errors.

Top Tools to Use

• Screaming Frog SEO Spider: The gold standard desktop crawler for deep, raw technical site audits and log file analysis.
• Google Search Console: Essential for assessing real-world index coverage, core web vitals, and manual actions straight from the source.
• Semrush Site Audit: Excellent for automated cloud crawling, scheduled tracking, and categorizing site health issues.
• Ahrefs Site Audit: Powerful cloud-based crawler with an intuitive interface for discovering technical and on-page roadblocks.
• PageSpeed Insights / Lighthouse: Crucial for granular diagnostics on field and lab performance metrics.

Would you like help prioritizing a specific technical issue you've found on your site, or do you want a deeper dive into auditing JavaScript rendering vs. traditional HTML crawlability?

A good technical SEO audit is not just “run Screaming Frog and fix everything it flags.” A specialist combines crawler data, Google Search Console data, real-user performance data, and manual inspection to determine which problems can actually prevent Google from crawling, indexing, understanding, or ranking the site.

The best technical SEO audit stack

If I were auditing a site today, I'd use:

1. search.google.com — essential/free

• Google's own data on indexing, crawling, Core Web Vitals, sitemap processing, manual actions, etc.
• URL Inspection lets you see Google's indexed version, canonical selection, crawlability, and rendered page.

2. screamingfrog.co.uk — my first-choice crawler

• Crawl the entire site and inspect status codes, redirects, canonicals, robots directives, titles, headings, internal links, duplicate content, structured data, JavaScript rendering, XML sitemaps, etc.
• It currently identifies 300+ SEO issues and has a free 500-URL version.

3. pagespeed.web.dev — performance

• Check Core Web Vitals and page-level performance.
• Don't rely solely on Lighthouse: field/user data is much more useful for determining whether real users are experiencing performance problems.

4. Chrome DevTools — debugging

• Particularly useful for JavaScript rendering, network requests, HTTP headers, mobile behavior and performance.

5. Ahrefs or Semrush — optional but valuable

• Backlinks, rankings, organic traffic, competitors and additional site-audit data.
• They're complementary rather than replacements for GSC + a crawler.

6. search.google.com + Schema.org validator

• Validate structured data rather than assuming that schema exists correctly.

For a serious professional audit, GSC + Screaming Frog + PageSpeed Insights is the core combination I'd start with.

---

Technical SEO audit checklist

Here's the workflow I'd use.

1. Establish the baseline

Before crawling anything:

• Confirm the correct domain/property in Google Search Console
• Check whether the site recently experienced a traffic/ranking drop
• Check GSC for manual actions
• Check GSC for security issues
• Record indexed-page counts
• Record organic clicks/impressions
• Record Core Web Vitals status
• Identify important page types/templates
• Identify recent migrations, redesigns, CMS changes or URL changes
• Obtain an XML sitemap
• Obtain analytics data if available
• For large sites, obtain server access logs

Why this matters: You want to understand the business problem before generating hundreds of technical findings.

---

2. Crawlability

This is one of the highest-priority sections.

robots.txt

• /robots.txt exists
• Returns HTTP 200
• Doesn't accidentally block important sections
• Doesn't block CSS/JS required for rendering
• Doesn't block important images/resources
• Sitemap is referenced
• Rules apply to the intended crawler/user-agent

One important distinction: robots.txt controls crawling, not indexing. Google specifically recommends using noindex when your goal is to prevent indexing; robots.txt isn't the right mechanism for that.

XML sitemap

• Sitemap exists
• Submitted in GSC
• Returns 200
• Contains canonical URLs
• URLs are indexable
• No 404 URLs
• No redirects
• No noindex pages
• No duplicate/non-preferred URLs
• Last-modified dates are accurate
• Sitemap coverage roughly corresponds with the site's important indexable URLs

Google describes sitemaps as an important way of communicating which pages are important to crawl.

---

3. Indexability

This is arguably the most important part of the audit.

For important pages, check:

• HTTP status = 200
• Page isn't blocked by robots.txt
• Page doesn't have noindex
• X-Robots-Tag isn't blocking indexing
• Page has a valid canonical
• Canonical points to the intended URL
• Canonical isn't redirected
• Canonical isn't 404
• Canonical isn't pointing to an unrelated page
• Important pages aren't orphaned
• Google actually indexes the page
• Google-selected canonical matches expectations

This is where GSC URL Inspection becomes extremely valuable. Google can show both the user-declared canonical and Google's selected canonical.

Particularly look for:

“Discovered – currently not indexed”

“Crawled – currently not indexed”

Duplicate without user-selected canonical

Alternate page with proper canonical

Excluded by noindex

Blocked by robots.txt

These aren't automatically errors. The specialist asks:

“Is Google excluding this URL intentionally, or is Google excluding a page we actually need indexed?” That's a much more important question than simply counting errors.

---

4. Status codes and redirects

In your crawler:

• Find all 4xx pages
• Find all 5xx errors
• Find 3xx URLs
• Find redirect chains
• Find redirect loops
• Identify internal links pointing to redirects
• Identify internal links pointing to 404s
• Check HTTP → HTTPS redirects
• Check www → non-www (or vice versa)
• Check trailing-slash consistency
• Check old URLs after migrations
• Check whether redirects ultimately resolve to the most relevant destination

A particularly useful report is:

Internal link → 301 → 301 → 200

Ideally, your internal links should point directly to the final destination.

Screaming Frog specifically supports finding broken links, server errors, redirect chains and loops.

---

5. Canonicalization

For every major URL template:

• Self-referencing canonical where appropriate
• Canonical uses HTTPS
• Canonical uses preferred hostname
• Canonical is absolute
• Canonical resolves with 200
• Canonical isn't blocked
• Canonical isn't noindex
• Canonical doesn't create conflicting signals
• Pagination/filter parameters are handled appropriately
• Duplicate versions consolidate correctly

Then compare:

User-declared canonical vs. Google-selected canonical

A mismatch isn't automatically a problem, but it deserves investigation.

---

6. Site architecture & internal linking

Now move from individual URLs to the structure of the site.

Check:

• Important pages are internally linked
• No important orphan pages
• Important pages aren't excessively deep
• Navigation makes sense
• Breadcrumbs exist where useful
• Category → subcategory → product/content hierarchy makes sense
• Internal links use meaningful anchor text
• Excessive links aren't being generated automatically
• Faceted navigation isn't generating millions of crawlable URLs
• Important pages receive sufficient internal authority

This is where crawl visualization is useful.

You're trying to answer:

Can Google efficiently discover and understand the site's most valuable pages?

---

7. URL structure

Check:

• HTTPS everywhere
• Consistent hostname
• Consistent trailing-slash convention
• Lowercase URLs where appropriate
• No unnecessary URL parameters
• No session IDs
• No duplicate URL versions
• URLs aren't unnecessarily deep
• URL structure reflects the site's information architecture
• Old URLs have appropriate redirects after changes

Don't obsess over making URLs contain every keyword. Consistency and avoiding duplicate URL variants matter much more.

---

8. JavaScript SEO

This is increasingly important.

For JavaScript-heavy sites:

• Important content exists in rendered HTML
• Important links are discoverable
• Navigation works without problematic JS dependencies
• Canonical is present after rendering
• Meta robots directives aren't injected incorrectly
• Structured data survives rendering
• Important content isn't dependent on user interaction
• Client-side routing produces crawlable URLs
• Server-side/prerendered content is correct where appropriate
• JS doesn't create thousands of unwanted URLs

Use Screaming Frog's JavaScript rendering and then manually inspect important pages with GSC URL Inspection. Screaming Frog supports Chromium-based JavaScript crawling specifically for JS-heavy frameworks such as React, Angular and Vue.

---

9. Core Web Vitals & performance

Check:

LCP

Largest Contentful Paint

Target: ≤ 2.5 seconds

INP

Interaction to Next Paint

Target: ≤ 200 ms

CLS

Cumulative Layout Shift

Target: ≤ 0.1

Then investigate:

• Slow server response
• Large images
• Unoptimized image formats
• Render-blocking CSS/JS
• Excessive JavaScript
• Third-party scripts
• Font loading
• Lazy-loading problems
• CDN configuration
• Cache configuration
• Mobile performance
• Template-specific performance

Important: Don't declare a site “fast” because Lighthouse scores 95. Look at the actual field data in GSC/PageSpeed Insights.

---

10. Mobile SEO

Check:

• Responsive design
• Same important content on mobile and desktop
• Same important links
• Correct viewport
• No intrusive interstitials
• Buttons/links usable
• Text readable
• Images responsive
• Mobile rendering doesn't hide important SEO content

Google primarily uses the mobile version of a site's content for indexing and ranking, so mobile isn't an optional secondary audit anymore.

---

11. On-page technical elements

For important indexable pages:

• Unique title
• Appropriate title length
• Unique meta description where useful
• One logical H1
• Logical heading hierarchy
• Image alt text where appropriate
• Correct language
• Open Graph metadata
• Correct hreflang where applicable
• No accidental duplicate templates

Screaming Frog can bulk-analyze titles, meta descriptions, headings and duplicate content.

---

12. Structured data

Check:

• Schema is relevant to the page
• Correct schema type
• Required properties present
• Values accurately represent visible content
• No invalid markup
• No misleading structured data
• No unnecessary/irrelevant schema
• Validate with Google's Rich Results Test
• Check GSC enhancement reports where applicable

Don't add schema simply because a plugin makes it available.

The question is:

Does this markup accurately describe the page and qualify it for a Google-supported search feature?

---

13. International SEO

If applicable:

• hreflang implementation
• Correct language/country combinations
• Self-referencing hreflang
• Return tags
• Canonical/hreflang consistency
• No conflicting language signals
• XML sitemap hreflang if used
• Correct geographic targeting
• No accidental country redirects

This can become particularly complicated on ecommerce and multinational sites.

---

14. Security & HTTPS

• HTTPS everywhere
• Valid certificate
• No mixed content
• HTTP redirects to HTTPS
• www/non-www consolidated
• No insecure resources
• No security warnings
• Appropriate security headers
• No hacked/spam pages

Also check GSC's security reports.

---

15. Large-site / enterprise SEO

For larger sites, add:

• Server log analysis
• Googlebot crawl frequency
• Crawl budget
• Crawl traps
• Faceted navigation
• Parameter handling
• Infinite URL spaces
• Orphan URLs
• Sitemap segmentation
• Rendering cost
• Database-generated URLs
• Pagination
• Search-result URLs
• Expired product/page handling

For a 5,000-page site, crawling is usually sufficient to identify most technical problems.

For a 5-million-URL ecommerce site, you really want to understand what Googlebot is actually doing. That's where server logs become extremely valuable.

---

16. Don't forget “soft” technical problems

Some of the most important problems aren't traditional technical errors.

Check:

• Thin pages
• Near-duplicate pages
• Thousands of low-value tag/category pages
• Programmatically generated pages with little value
• Expired products
• Outdated content
• Doorway-like location pages
• Internal search pages indexed
• Filter combinations indexed unnecessarily
• Large numbers of pages receiving zero organic traffic

A crawler may tell you:

50,000 URLs are indexable. The SEO question is:

Should all 50,000 URLs actually be indexable? That's much more valuable.

---

How I'd prioritize the findings

Don't hand a developer a spreadsheet containing 327 “errors.”

I'd classify them:

P0 — Critical

Things that can prevent Google from accessing/indexing the site.

Examples:

• Entire site noindex
• Important directory blocked by robots.txt
• HTTPS migration broken
• Canonicals pointing to wrong domain
• Major 5xx problems
• Migration URLs missing redirects

P1 — High

Things affecting important pages at scale.

Examples:

• Thousands of important pages canonicalized incorrectly
• Major internal-linking problems
• Serious Core Web Vitals issues
• Huge duplicate URL generation
• Important pages orphaned
• JavaScript rendering prevents Google from seeing content

P2 — Medium

Meaningful but not immediately catastrophic.

Examples:

• Redirect chains
• Missing metadata
• Broken internal links
• Schema errors
• Image optimization
• Moderate crawl inefficiency

P3 — Low

Things worth improving but unlikely to produce substantial SEO gains by themselves.

Examples:

• Minor title-length issues
• Minor meta-description issues
• Low-impact warnings
• Cosmetic crawler warnings

This prevents the classic SEO-audit problem of spending 20 developer hours fixing issues that have virtually no ranking impact.

---

My recommended workflow

If I were doing the audit myself, I'd do it in this order:

Step 1: GSC baseline ↓ Step 2: Full Screaming Frog crawl ↓ Step 3: Compare crawl data with GSC index data ↓ Step 4: Inspect robots.txt + XML sitemap ↓ Step 5: Analyze indexability/canonicals ↓ Step 6: Analyze status codes + redirects ↓ Step 7: Analyze architecture/internal links ↓ Step 8: Analyze JavaScript rendering ↓ Step 9: Analyze Core Web Vitals ↓ Step 10: Analyze structured data ↓ Step 11: Mobile/HTTPS/international checks ↓ Step 12: Log-file analysis for large sites ↓ Step 13: Prioritize findings by business/SEO impact ↓ Step 14: Fix → recrawl → validate in GSC

---

Which crawler should you choose?

My practical comparison:

Screaming Frog is particularly strong when you want to get into the raw data and diagnose exactly what's happening. Sitebulb is attractive when you want more guided analysis and visualizations; its current product materials emphasize 300+ technical issues, visual reporting and prioritized recommendations.

My preferred setup

Small/medium site:

Google Search Console + Screaming Frog + PageSpeed Insights + Chrome DevTools Large/enterprise site:

Google Search Console + Screaming Frog/Sitebulb + PageSpeed Insights + Ahrefs/Semrush + server logs And if you only have time to learn one professional SEO tool, I'd learn Screaming Frog. It teaches you how websites actually work from an SEO perspective rather than simply giving you a generic “site health score.”

The most important principle is this: the goal of a technical SEO audit isn't to get a 100/100 audit score. It's to find the technical constraints that are preventing your most valuable pages from being crawled, indexed, understood, and surfaced in search—and then prioritize those fixes by impact.

Which crawler should you choose?

My practical comparison:

Screaming Frog is particularly strong when you want to get into the raw data and diagnose exactly what's happening. Sitebulb is attractive when you want more guided analysis and visualizations; its current product materials emphasize 300+ technical issues, visual reporting and prioritized recommendations.

My preferred setup

Small/medium site:

An SEO specialist performs a technical SEO audit to ensure search engine bots can easily crawl, render, and index a website while delivering a fast, secure user experience. It acts as the structural health check-up for a website before (or alongside) content and link-building efforts.

How an SEO Specialist Performs an Audit (The Process)

1. Scoping and Crawling: Setting up a crawler to mimic search engine user-agents (like Googlebot) to map out the entire website architecture, internal link graphs, and response codes.
2. Log File & Indexation Analysis: Analyzing server logs or indexing reports to see how search engines actually spend their crawl budget versus what is sitting in the index.
3. Performance & Rendering Evaluation: Testing how fast pages load and whether JavaScript-heavy elements render correctly for search bots.
4. Prioritization and Remediation: Categorizing errors by severity (critical/blocking vs. minor optimization) and creating an actionable developer ticket list.

The Best Technical SEO Checklist

• Crawlability & Indexation
  • Check the robots.txt file to ensure vital pages aren't mistakenly blocked.
  • Audit the XML Sitemap(s) to verify that only canonical, 200-status URLs are listed.
  • Inspect HTTP status codes and resolve 404 errors, 5xx server errors , and improper 301 redirects redirect chains.
  • Address soft 404s (pages showing error messages but returning a 200 OK status).
• Architecture & Internal Linking
  • Ensure a logical, shallow site hierarchy (most pages reachable within 3 to 4 clicks from the homepage).
  • Fix broken internal links and eliminate orphan pages (pages with zero internal inbound links).
• Canonicalization & Duplicate Content
  • Verify that all important pages have correct, self-referencing or destination-appropriate canonical tags.
  • Check for URL parameter duplication or trailing slash inconsistencies.
• Site Speed & Core Web Vitals
  • Optimize Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and Cumulative Layout Shift (CLS).
  • Compress heavy images, leverage browser caching, and minimize heavy or blocking JavaScript scripts.
• Security & Mobile Friendliness
  • Confirm active and enforced HTTPS/SSL certificates across all assets.
  • Validate mobile responsiveness, viewport scaling, and touch-element spacing.

The Best Tools to Use

• Screaming Frog SEO Spider: The gold-standard desktop crawler. It acts as a mini-search engine, letting you crawl millions of URLs to spot broken links, missing meta data, redirect loops, and canonical misalignments in bulk.
• Google Search Console: The direct line to Google. Essential for monitoring index coverage, manual actions, core web vitals, and submitted sitemaps.
• Semrush or Ahrefs: Comprehensive cloud-based suite auditors that run automated site health crawls on a scheduled cadence, flagging new technical regressions automatically.
• PageSpeed Insights: Powered by Lighthouse, this is essential for evaluating real-user (CrUX) and lab-based performance data for Core Web Vitals.

If you'd like, let me know:

• What CMS or platform the site is built on (WordPress, Shopify, custom, etc.)
• Approximately how many pages the website has

I can tailor this process or recommend specific fixes for your tech stack.

Yes. A good technical SEO audit is less about running one “SEO score” tool and more about reconciling three things:

1. What Google can crawl
2. What Google can index
3. Whether the indexed pages are fast, accessible, internally well-linked, and technically sound

For most sites, my preferred stack is Screaming Frog + Google Search Console + PageSpeed Insights, with Ahrefs or Semrush added for backlinks and broader competitive analysis.

The technical SEO audit workflow

1. Establish the baseline

Before crawling, collect:

• Organic traffic and conversions
• Top organic landing pages
• Indexed pages
• Pages generating impressions/clicks
• Organic ranking trends
• Recent migrations, redesigns, URL changes, or CMS changes
• XML sitemap URLs
• Known subdomains/subdirectories
• International versions, if applicable

In Google Search Console, start with Search performance, indexing, URL Inspection, Manual Actions, Security Issues, and Core Web Vitals. Search Console's URL Inspection tool is particularly useful because it lets you investigate Google's current index status for individual URLs.

---

Technical SEO audit checklist

A. Crawlability

• robots.txt exists and is accessible
• Important pages aren't blocked by robots.txt
• Important CSS/JS/resources aren't accidentally blocked
• No accidental sitewide noindex
• No important pages using X-Robots-Tag: noindex
• Crawlable navigation exists
• Important pages can be discovered through internal links
• No excessive crawl traps
• Faceted/filter URLs are controlled
• Parameter URLs aren't creating millions of unnecessary crawlable URLs

Screaming Frog can specifically report URLs blocked by robots.txt and identify meta-robots/X-Robots-Tag directives.

B. Indexability

For important URLs, verify:

• HTTP status = 200
• URL is indexable
• Correct canonical URL
• Canonical isn't pointing somewhere unexpected
• Canonical page returns 200
• Canonical isn't conflicting with noindex
• Important pages aren't orphaned
• Sitemap contains indexable canonical URLs
• Sitemap doesn't contain 3xx/4xx/5xx/noindex URLs
• No large numbers of "Crawled - currently not indexed"
• No unexpected "Discovered - currently not indexed"

Important: don't treat "Google hasn't indexed this URL" as automatically being a technical error. Google makes its own indexing decisions.

---

C. HTTP status codes & redirects

Look for:

• 404 pages
• 410 pages where appropriate
• 5xx server errors
• Redirect chains
• Redirect loops
• Internal links pointing to redirected URLs
• Internal links pointing to 404s
• HTTP → HTTPS redirects
• www/non-www consistency
• Trailing-slash consistency
• Old URLs redirected after migrations
• No unnecessary redirect hops

This is one of the highest-value sections of the audit. Screaming Frog can crawl response codes, broken links, redirects, chains and loops at scale.

---

D. Canonicals & duplicate URLs

Check:

• Every indexable page has an appropriate canonical
• Canonical points to the preferred URL
• Self-referencing canonicals are used where appropriate
• Canonical URLs return 200
• Canonical isn't redirected
• Canonical isn't noindex
• HTTP/HTTPS duplicates aren't competing
• www/non-www duplicates aren't competing
• Parameter variations are controlled
• Duplicate/near-duplicate pages are understood
• Pagination/faceted navigation is handled appropriately

Don't blindly "fix" every duplicate. Some duplication is normal; the question is whether Google is being given confusing competing versions.

---

E. XML sitemaps

Check:

• XML sitemap exists
• Sitemap is submitted in Search Console
• Sitemap returns HTTP 200
• Only canonical, indexable URLs are included
• No 404s
• No redirects
• No noindex URLs
• No obviously duplicate URLs
• lastmod dates are accurate
• Sitemap is segmented appropriately for large sites
• Sitemap reflects the site's actual indexable URL inventory

A particularly useful audit is:

Sitemap URLs vs. crawler-discovered URLs vs. Google-indexed URLs

Large discrepancies can reveal orphan pages, unwanted URLs, migration problems or indexing problems.

---

F. Site architecture & internal linking

Analyze:

• Important pages are reachable within a reasonable crawl depth
• No orphan pages
• Important pages receive sufficient internal links
• Internal links don't unnecessarily point through redirects
• Anchor text is descriptive
• Navigation is logically structured
• Category/topic relationships are clear
• Excessively deep pages are investigated
• High-value pages aren't buried

Screaming Frog can analyze crawl depth, internal links, link counts, anchor text and site architecture.

This is often where a technical audit becomes much more valuable than simply fixing metadata.

---

G. JavaScript & rendering

For React, Next.js, Angular, Vue, etc.:

• Important content exists in rendered HTML
• Important links are discoverable
• Canonicals render correctly
• Meta robots directives render correctly
• Structured data renders correctly
• Content isn't dependent on problematic client-side JavaScript
• Internal navigation works for crawlers
• No major discrepancy between raw HTML and rendered DOM

Screaming Frog supports Chromium-based JavaScript rendering and can inspect rendered pages, making it useful for JS-heavy websites.

---

H. Core Web Vitals & performance

Check:

• LCP
• INP
• CLS
• Mobile performance
• Desktop performance
• Server response time
• Large images
• Render-blocking resources
• Excessive JavaScript
• Third-party scripts
• Font loading
• Caching/CDN configuration

Use real-user data as your primary performance signal, rather than treating a Lighthouse score as the SEO score.

Google's Search Console Core Web Vitals report uses real-world user data and reports LCP, INP and CLS.

---

I. Mobile

• Responsive design
• Same primary content on mobile and desktop
• Same important links
• Same structured data
• No intrusive mobile interstitials
• No horizontal overflow
• Navigation works on mobile
• Images scale correctly
• Core Web Vitals checked on mobile

---

J. Structured data

Check:

• Schema.org markup exists where genuinely appropriate
• Correct schema type
• Required properties present
• No major validation errors
• Structured data matches visible page content
• Organization/site/entity information is consistent
• Breadcrumb markup is correct where appropriate
• Product/review/etc. markup follows Google's requirements

Don't add schema purely because a competitor has it. Structured data is useful when it accurately describes the page and can qualify it for supported search features.

---

K. HTTPS & security

• HTTPS everywhere
• Valid SSL/TLS certificate
• No mixed content
• HTTP redirects to HTTPS
• No duplicate HTTP versions indexed
• Security issues checked in Search Console
• Hacked/spam pages investigated

Search Console has a dedicated Security Issues report for detecting problems Google believes could harm users.

---

L. International SEO

If applicable:

• hreflang implementation is valid
• Return tags are reciprocal
• Language/country targeting is correct
• Canonicals don't conflict with hreflang
• Each localized page has the appropriate localized counterpart
• XML sitemaps are correctly segmented if needed
• No accidental automatic redirects based on IP/language

---

The tools I'd use

If I could only choose three

1. Screaming Frog — for the crawl.

It can identify 4xx/5xx errors, redirects, blocked URLs, canonicals, duplicate content, internal-link problems, structured data, JavaScript rendering issues and much more. It also integrates with Search Console, Analytics and PageSpeed Insights.

2. Google Search Console — for Google's perspective.

This is essential because a crawler tells you what it discovered; Search Console gives you information about Google's crawling, indexing, search performance and real-user Core Web Vitals.

3. PageSpeed Insights — for page-level performance.

Use it alongside Search Console's field data rather than as a standalone "SEO score."

---

My recommended audit process

I'd run the audit in this order:

Step 1 — Search Console → Find indexing, manual action, security, performance and CWV problems.

Step 2 — Screaming Frog crawl → Crawl the entire site and export the important issue sets.

Step 3 — Crawl/index reconciliation → Compare:

Sitemap URLs
vs. Crawled URLs
vs. Indexable URLs
vs. Google-indexed URLs
vs. Organic landing pages

This is where you often discover the interesting problems.

Step 4 — Technical architecture → Analyze status codes, redirects, canonicals, robots directives, orphan pages, crawl depth and internal links.

Step 5 — Rendering → Render JavaScript-heavy pages and compare what users/search engines can actually access.

Step 6 — Performance → Use Search Console's field data + PageSpeed Insights/Lighthouse to identify systemic performance problems.

Step 7 — Structured data → Validate important templates and structured-data implementations.

Step 8 — Prioritize Don't give the client a 200-item laundry list.

Classify every issue as:

Then add Impact × Effort.

For example:

Problem: 14,000 product URLs are blocked by an incorrect robots directive
Impact: Critical
Effort: Medium
Recommendation: Correct robots configuration, validate with live testing, recrawl and monitor indexing.

That's much more useful than:

"Robots.txt has errors — 73/100 SEO score."

One important principle

Don't let an automated audit dictate the strategy.

Tools are excellent at finding anomalies. They're much worse at determining whether an anomaly actually matters.

For example, a crawler might report:

• duplicate title tags
• long meta descriptions
• multiple H1s
• low word count
• URLs with parameters

Those aren't automatically SEO problems.

Conversely, something like:

Important revenue page → 301 → 301 → 200

may be far more important despite generating a relatively small "issue count."

So the best technical SEO audit is:

Crawl data + Google's data + business/traffic data + human judgment.

If you're auditing a real site, I can also give you a Screaming Frog configuration/settings walkthrough and a 50–100 point technical SEO audit checklist you can use as a repeatable client audit template.

An SEO specialist performs a technical SEO audit by systematically evaluating how search engine bots (and AI retrieval agents) discover, render, crawl, and index a website's infrastructure . Rather than focusing on keywords or content quality, a technical audit ensures the underlying code and architecture present no barriers to search visibility.

How an SEO Specialist Performs the Audit

1. Information Gathering & Scoping: Review the business goals, CMS constraints, and historical performance data via analytics.
2. Executing the Crawl: Simulate a search engine bot using a desktop or cloud crawler to map every URL, response code, and internal link.
3. Cross-Referencing Field Data: Compare crawler findings with real-world Google indexing and traffic behavior.
4. Prioritizing & Issuing Tickets: Group findings by severity (e.g., P0 for critical indexation blocks, P3 for minor tag optimizations) and build actionable implementation tickets for developers.

The Best Tools to Use

No single tool does everything, so specialists rely on a specialized stack:

• The Ultimate Crawler: Screaming Frog SEO Spider is the industry gold standard for granular, deep-dive technical crawling, identifying broken links, redirect chains, and duplicate content.
• The Ground Truth Source: Google Search Console is indispensable for observing actual indexing status, manual actions, and real-world search performance data straight from Google.
• The Automated All-in-One Alternative: Semrush Site Audit or Ahrefs Site Audit work best for automated scheduling, continuous monitoring, and high-level structural health scores.
• Speed & Performance: Google PageSpeed Insights evaluates loading performance and Core Web Vitals (including INP - Interaction to Next Paint).

The Essential Technical SEO Audit Checklist

Work through these core categories sequentially, as structural and crawl issues must be resolved before indexation and rendering can be optimized properly:

1. Crawlability and Access Control
   • Review robots.txt to ensure critical assets, CSS, and JS files aren’t accidentally blocked from search bots.
   • Verify the XML sitemap contains only canonical, 200-status URLs that you actually want indexed.
   • Check for excessive redirect chains, redirect loops, and broken internal links (404 or 5xx errors).
2. Index Hygiene and Canonicalization
   • Audit index coverage reports in Google Search Console to eliminate index bloat and unexpected noindex tags.
   • Ensure proper self-referencing canonical tags on all primary templates to prevent duplicate content penalties from parameters or URL variations.
   • Identify and clean up soft 404 errors (pages returning a 200 OK status but displaying empty or missing content).
3. Site Architecture and Internal Linking
   • Keep important pages within 3 to 4 clicks from the homepage.
   • Eliminate orphan pages (valid URLs with zero internal links pointing to them).
   • Fix broken internal navigation or faceted navigation loops that waste crawl budget.
4. Page Speed and Core Web Vitals (CWV)
   • Optimize Largest Contentful Paint (LCP) by compressing hero images and using next-gen formats like WebP.
   • Stabilize Cumulative Layout Shift (CLS) by defining explicit dimensions for images and dynamic ad/content blocks.
   • Improve Interaction to Next Paint (INP) by reducing heavy JavaScript execution times.
5. Mobile Usability and Security
   • Confirm mobile-first indexing parity (content, metadata, and structured data match identically across viewports).
   • Enforce valid HTTPS deployment across all pages and test for mixed-content warnings.
6. Structured Data and AI Readiness
   • Validate schema markup (Organization, Product, Article, FAQ) using validation tools to earn rich snippets.
   • Ensure your content architecture allows AI and generative search engine crawlers to extract, parse, and cite your data accurately.

If you'd like, share:

• The approximate size of the website (number of pages)
• The CMS it is built on (WordPress, Shopify, custom, etc.)
• Whether you are dealing with a recent traffic drop or a new site migration

I can help tailor a specific priority action plan for your project.

A technical SEO audit is essentially a structured investigation into whether search engines can discover, crawl, understand, index, and efficiently serve your important pages.

The biggest mistake is treating an audit as “run Semrush and fix everything it reports.” A good SEO specialist combines a crawler with Google Search Console, PageSpeed Insights, analytics, and manual inspection, then prioritizes issues by their likely impact.

The best technical SEO audit workflow

1. Establish the baseline

Before crawling anything, collect:

• Organic traffic and conversions
• Organic landing pages
• Indexed pages
• Ranking keywords
• Top-performing URLs
• Organic traffic trends
• Recent traffic/ranking drops
• Current sitemap URLs
• Number of URLs in the site/database/CMS
• Recent migrations, redesigns, domain changes, or platform changes

Use Google Search Console as the source of truth for Google's view of the site. Google's current technical requirements boil down to three fundamentals: Googlebot must not be blocked, the page must return a successful HTTP response, and the page needs indexable content.

---

Technical SEO audit checklist

Here's the checklist I'd actually use.

2. Crawlability

• robots.txt exists and is valid
• Important pages aren't blocked by robots.txt
• Important CSS/JS/resources aren't unnecessarily blocked
• No accidental sitewide crawling restrictions
• XML sitemap exists
• XML sitemap is accessible
• Sitemap contains canonical, indexable URLs
• Sitemap doesn't contain 3xx/4xx/5xx URLs
• Sitemap doesn't contain noindex URLs
• Sitemap is submitted in Google Search Console
• Important pages are reachable through internal links
• Crawl depth isn't excessive
• No major orphan-page problems

Google specifically recommends using both Page Indexing and Crawl Stats in Search Console when investigating accessibility/indexing problems, and URL Inspection for individual URLs.

3. Indexability

For important URLs:

• HTTP status = 200
• Not accidentally noindex
• Not blocked by robots.txt
• Correct canonical
• Canonical points to an indexable URL
• No conflicting canonical signals
• No accidental nofollow
• No unwanted X-Robots-Tag
• Google-selected canonical makes sense
• Important pages are actually indexed
• Excluded pages have a legitimate reason for exclusion

This is one of the highest-priority parts of an audit.

For example:

Page exists → returns 200 → isn't blocked → has indexable content → canonical is correct → Google can discover it → page is indexed.

Don't assume that because a URL is technically indexable, Google will necessarily index it; Google explicitly says indexing isn't guaranteed.

---

4. HTTP status codes & redirects

Look for:

• 404 errors
• 5xx errors
• Broken internal links
• Incorrect 301 redirects
• Temporary 302 redirects where permanent redirects are appropriate
• Redirect chains
• Redirect loops
• Redirects to irrelevant pages
• Internal links pointing to redirected URLs
• HTTP → HTTPS redirects
• Old URLs from migrations redirect correctly
• No massive redirect chains

A crawler such as Screaming Frog can identify broken links, server errors, redirect chains/loops, blocked URLs and other crawl problems.

---

5. Canonicals & duplicate URLs

Check for:

• Missing canonicals where appropriate
• Multiple canonical tags
• Canonical pointing to a 404
• Canonical pointing to a redirect
• Canonical pointing to a noindex page
• Canonical conflicts
• Duplicate URLs
• HTTP/HTTPS duplication
• www/non-www duplication
• Trailing-slash duplication
• URL parameter duplication
• Case-sensitive URL duplication
• Pagination/faceted-navigation problems
• Duplicate category/product URLs

Don't blindly canonicalize everything. The question is:

"Which URL should Google consider the primary version of this content, and are all the signals consistent with that decision?"

---

6. Site architecture & internal linking

Analyze:

• Click depth
• Important pages within a reasonable crawl depth
• Orphan pages
• Excessively deep pages
• Internal links to 404s
• Internal links to redirects
• Important pages receiving enough internal links
• Descriptive anchor text
• Logical category/subcategory structure
• Excessive navigation links
• Faceted navigation
• JavaScript-dependent navigation
• Internal linking between related content

This is often overlooked.

A site can have zero 404 errors and perfect robots.txt and still have poor technical SEO because Google has difficulty discovering the pages that matter.

---

7. JavaScript rendering

For JS-heavy websites, check:

• Important content exists in rendered HTML
• Important links are discoverable
• Navigation works for crawlers
• JavaScript isn't preventing indexing
• Client-side redirects aren't problematic
• Metadata is rendered correctly
• Structured data exists in the rendered page where appropriate
• Important content isn't loaded only after user interaction

This is where a JavaScript-capable crawler becomes particularly valuable.

---

8. Page speed & Core Web Vitals

Check:

• LCP
• INP
• CLS
• Server response time
• Image sizes
• Image formats
• Lazy loading
• Render-blocking resources
• JavaScript execution
• CSS bloat
• Third-party scripts
• Caching
• CDN configuration
• Font loading

Use Google PageSpeed Insights and real-user data where available rather than relying solely on a crawler's synthetic score.

The important distinction is:

Don't optimize the PageSpeed score; optimize the actual user experience.

---

9. Mobile SEO

• Mobile version contains the important content
• Responsive design works
• No horizontal scrolling
• Text is readable
• Links/buttons are usable
• Navigation works
• Images aren't broken
• Structured data is consistent
• Canonicals are correct
• Mobile rendering doesn't hide important content

Google's indexing is heavily based around the mobile version of pages, so mobile rendering deserves serious attention.

---

10. HTTPS & security

• HTTPS everywhere
• HTTP redirects to HTTPS
• No mixed content
• Valid SSL/TLS certificate
• No unnecessary HTTP versions indexed
• Correct canonical protocol
• Secure cookies where appropriate
• No unexpected security/interstitial problems

---

11. XML sitemap

Validate:

• Sitemap exists
• Correct sitemap index if multiple sitemaps
• Only canonical URLs included
• Only indexable URLs included
• No 404s
• No redirects
• No noindex URLs
• Last-modified dates are meaningful
• Sitemap is submitted to Search Console
• Sitemap coverage roughly aligns with your site's actual important URLs

A sitemap shouldn't be treated as a dumping ground for every URL in your database.

---

12. Structured data

Check:

• Schema markup exists where genuinely appropriate
• Valid JSON-LD
• Required properties present
• No misleading markup
• Markup describes visible page content
• No major validation errors
• Organization/LocalBusiness/etc. information is accurate
• Product schema is accurate on product pages
• Breadcrumb markup is correct where appropriate

Use Google's rich-results testing tools alongside your crawler.

---

13. International SEO

For multilingual/multiregional websites:

• hreflang implementation
• Correct language/region codes
• Reciprocal hreflang references
• Self-referencing hreflang
• Canonicals don't conflict with hreflang
• Language versions aren't accidentally canonicalized together
• XML sitemap hreflang if applicable
• Correct country/language targeting

---

14. URL structure

Look for:

• Logical URL hierarchy
• Short, understandable URLs
• Consistent trailing-slash policy
• Lowercase URLs where appropriate
• No unnecessary parameters
• No session IDs
• No massive URL generation
• No duplicate URL paths
• Stable URLs

Don't obsess over keywords in URLs. Consistency, crawlability and avoiding duplication matter much more.

---

15. On-page technical elements

Although this crosses into on-page SEO, I include it in the audit:

• Unique title tags
• Missing titles
• Duplicate titles
• Extremely long/short titles
• Unique meta descriptions where useful
• Missing/duplicate descriptions
• Correct H1
• Logical heading hierarchy
• Image alt text where appropriate
• Descriptive image filenames
• Open Graph/social metadata
• Language declaration
• Duplicate/thin pages

---

The tools I recommend

🥇 Best overall combination: Screaming Frog + Google Search Console

Screaming Frog SEO Spider is my favorite technical crawling tool.

It can analyze broken links, redirects, robots directives, internal links, crawl depth, duplicate content, XML sitemaps, structured data, JavaScript rendering and much more. It also integrates with Search Console, PageSpeed Insights and analytics data.

The free version crawls up to 500 URLs, which is enough to learn the workflow on smaller sites.

Pair it with:

• Google Search Console — Google's perspective
• Google PageSpeed Insights — performance/Core Web Vitals
• Google Analytics — user/conversion perspective
• Chrome DevTools — debugging
• Rich Results Test — structured data
• server logs — what Googlebot actually crawls

That's an extremely powerful stack.

---

🥈 Best all-in-one SEO platform: Semrush

Semrush Site Audit is probably the easiest option if you want one dashboard rather than assembling multiple tools.

Its current Site Audit product reports 140+ technical and on-page checks, including crawlability, indexability, HTTPS, Core Web Vitals, markup, international SEO and internal linking.

It also supports scheduled crawls and progress tracking, which is useful for agencies and ongoing SEO programs.

I'd choose Semrush if you're doing SEO professionally and also need:

• Keyword research
• Competitor analysis
• Backlink analysis
• Rank tracking
• Content research
• Technical auditing
• Reporting

---

Which should you use?

The important point is that no single tool is the source of truth.

A crawler tells you what it can discover.
Search Console tells you what Google is seeing.
Analytics tells you what users are doing.
Logs tell you what bots are actually requesting.

---

How a professional actually conducts the audit

I'd structure the process like this:

1. Crawl the site

Run Screaming Frog/Semrush against the domain.

2. Export the data

Get URLs, status codes, canonicals, directives, titles, headings, internal links, crawl depth, etc.

3. Compare crawl data against GSC

This is where you uncover interesting discrepancies:

"The crawler found 20,000 URLs, but Google has only indexed 6,000."

Then investigate why.

4. Examine the sitemap

Compare:

Sitemap URLs → Crawlable URLs → Indexable URLs → Indexed URLs

Ideally, your important URLs overlap strongly across all four.

5. Investigate high-impact problems

Don't fix issues simply because the tool labels them "errors."

Prioritize things like:

1. Accidental noindex
2. Robots.txt blocking important pages
3. Incorrect canonicals
4. Important pages not indexed
5. 5xx/server problems
6. Massive internal-linking problems
7. Bad redirects/migration failures
8. JavaScript rendering/indexing problems
9. Major duplicate URL problems
10. Serious performance problems

6. Manually inspect important URLs

Pick:

• Homepage
• Main category/service pages
• Top organic landing pages
• Highest-revenue pages
• New pages
• Pages with traffic declines
• Pages Google says aren't indexed

Run them through GSC URL Inspection and inspect the rendered HTML.

7. Turn findings into an action plan

This is the part that separates an SEO audit from a crawl report.

Instead of:

"1,842 URLs have duplicate titles."

Produce:

Problem: 1,842 product URLs inherit the same title template.
Impact: Search engines receive weak differentiation between products.
Priority: High.
Fix: Modify the CMS template to include product name + primary differentiator.
Owner: Development/SEO.
Validation: Recrawl after deployment.

---

My "gold standard" audit stack

If I were auditing a serious website today, I'd use:

Screaming Frog SEO Spider
→ Deep crawl and technical diagnostics

Google Search Console
→ Google indexing, search performance and crawl data

Google PageSpeed Insights
→ Core Web Vitals/performance

Google Analytics
→ Organic traffic and business outcomes

Semrush Site Audit
→ Automated auditing, competitive SEO data and recurring monitoring

Server log analysis
→ Advanced crawling/bot behavior investigation

That combination gives you a much more complete picture than any single "SEO audit score."

The key principle

A technical SEO audit ultimately answers four questions:

Can Google find the page? → Can Google crawl it? → Can Google understand/index it? → Is the site architecture/performance helping or hurting it?

If you build your audit around those four questions rather than around a tool's error count, you'll produce substantially better SEO audits.

An SEO specialist performs a technical SEO audit by systematically evaluating how search engine crawlers access, render, parse, and index a website . Rather than focusing on content quality or keywords, a technical audit ensures there are no infrastructural barriers capping the site's organic visibility.

How an SEO Specialist Performs the Audit

1. Inventory & Setup: Define the scope (domains, subdomains, server setup, and CMS) and connect diagnostic tools.
2. Crawl Simulation: Run a desktop crawler to mimic search bots, mapping out how internal links flow and uncovering hidden errors.
3. Index Validation: Cross-reference crawler data with real index status data from search engines to see what is missing or misconfigured.
4. Template & Performance Checks: Review code execution, mobile render parity, and page performance on core page types rather than isolated URLs.
5. Prioritization: Score issues based on impact, fixing difficulty, and risk to organize an actionable developer roadmap.

The Best Technical SEO Audit Checklist

A modern, impact-driven audit focuses on these core passes:

• Crawlability and Indexation: Inspect the robots.txt file for accidental blocks, validate the XML sitemap, and check for orphan pages (pages with zero internal links).
• Status Codes and Redirects: Flag broken pages (404/410), soft 404 inventory pages, and lengthy redirect chains (>2 hops).
• Canonicalization and Duplicates: Confirm that canonical tags are self-referencing and correctly manage faceted navigation or parameter-based duplicate content.
• Site Architecture: Verify click depth—ensuring no vital page requires more than 3 clicks from the homepage.
• Performance and Core Web Vitals: Test key templates for Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and Cumulative Layout Shift (CLS).
• Mobile-First Parity: Ensure mobile and desktop versions share the same content, structural metadata, and rendering capability.
• Security and Structured Data: Validate sitewide HTTPS enforcement and check that schema markup (JSON-LD) accurately describes visible entity content.

The Best Tools to Use

No single tool does everything, but combining the right ones yields total clarity:

• Google Search Console: The indispensable, free primary source of truth for real indexing errors, mobile usability, and search query data from Google.
• Screaming Frog SEO Spider: The gold standard desktop crawler for deep structural detail, custom extractions, bulk canonical checks, and raw technical data.
• Ahrefs Site Audit or Semrush Site Audit: Excellent cloud-based choices for automated recurring crawls, visual issue grouping, and competitor context.
• Sitebulb: Best for clear, visual audit graphs and architecture maps that are easy to present to developers and stakeholders.

If you'd like, share your website's platform (e.g., WordPress, Shopify, custom JS framework) or approximate size , and I can tailor this checklist into a step-by-step execution plan for you.

A good technical SEO audit is not just “run Screaming Frog and fix red errors.” A specialist is trying to answer four bigger questions:

1. Can search engines discover the important URLs?
2. Can they crawl and render them properly?
3. Are the right URLs eligible for indexing?
4. Is the site technically strong enough to provide a good search/user experience?

Google itself emphasizes crawling, indexing, canonicalization, mobile, JavaScript, and page metadata as core technical areas.

My recommended technical SEO audit stack

If I were auditing a site today, I'd use this combination:

If you only pay for one SEO crawler, I'd choose Screaming Frog for a hands-on technical audit. If you want a more automated, ongoing SEO platform, I'd choose Ahrefs or Semrush.

Ahrefs currently checks 170+ technical/on-page issues and supports scheduled crawls. ahrefs.com Semrush similarly provides 140+ technical checks with prioritized issues and recurring audits.

But Google Search Console is non-negotiable because third-party crawlers aren't Google. Search Console gives you Google's actual indexing/crawling view.

---

The technical SEO audit checklist

1. Establish the baseline

Before touching anything, record:

• Organic clicks and impressions
• Organic traffic
• Number of indexed URLs
• Number of URLs in XML sitemap
• Number of indexable URLs found in crawl
• Organic landing pages
• Rankings for important keywords
• Core Web Vitals
• Organic conversions/revenue
• Recent traffic/ranking changes
• Recent migrations, redesigns, CMS changes, or URL changes

This gives you something to compare against after fixes.

---

2. Crawl the entire website

Start with Screaming Frog.

Configure the crawl to include:

• HTML
• Images
• CSS
• JavaScript
• Canonicals
• Pagination where relevant
• XML sitemaps
• Internal links
• External links where useful

For JavaScript-heavy sites, don't rely exclusively on an HTML crawl. Screaming Frog supports JavaScript rendering specifically for frameworks such as React, Angular and Vue.

Then investigate:

• 3xx redirects
• 4xx errors
• 5xx errors
• Redirect chains
• Redirect loops
• Broken internal links
• Orphan pages
• Duplicate URLs
• Duplicate content
• Canonicalized URLs
• Noindex pages
• Blocked pages
• Missing titles
• Duplicate titles
• Missing meta descriptions
• Duplicate meta descriptions
• Missing H1s
• Multiple H1s where inappropriate
• Very large pages
• Slow pages
• Missing/oversized images
• Internal linking problems

Important: Don't blindly fix every crawler warning. An SEO specialist determines whether each issue actually matters.

---

3. Check crawlability

This is one of the highest-priority sections.

robots.txt

Check:

• /robots.txt exists
• It returns HTTP 200
• Important pages aren't disallowed
• Important assets aren't accidentally blocked
• No accidental Disallow: /
• Sitemap URL is declared where appropriate
• Staging/test environments aren't accidentally exposed

Remember that robots.txt controls whether crawlers can request URLs; it isn't the same thing as an indexing directive.

XML sitemap

Check:

• Sitemap exists
• Sitemap is submitted in Search Console
• URLs return 200
• URLs are canonical
• URLs are indexable
• No redirects in sitemap
• No 404s
• No noindex URLs
• No parameter/duplicate URLs
• lastmod is accurate where used
• Sitemap is segmented if the site is large

One particularly useful audit is:

Sitemap URLs → Crawlable URLs → Indexable URLs → Google-indexed URLs

Those four numbers should make sense together.

---

4. Audit indexability

This is arguably the most important technical SEO section.

For important pages, verify:

• HTTP status = 200
• Crawlable
• Indexable
• Not noindex
• Not accidentally canonicalized elsewhere
• Not blocked by authentication
• Not excluded because of duplicate content
• Accessible on mobile
• Important content is available to Googlebot

Use Google Search Console URL Inspection to validate important pages.

It can tell you Google's indexed version, whether crawling/indexing was allowed, Google's selected canonical, and what Google saw when rendering the page.

I'd manually inspect at least:

• Homepage
• Main category/service pages
• Important product pages
• Top organic landing pages
• New pages
• Pages that aren't indexing
• Pages with unexpected canonicalization

---

5. Investigate Google indexing

Open Search Console → Page Indexing.

Look for unusual increases in:

• Not indexed
• Crawled — currently not indexed
• Discovered — currently not indexed
• Duplicate without user-selected canonical
• Alternate page with proper canonical
• Excluded by noindex
• Blocked by robots.txt
• Server errors
• Soft 404s

The key is to ask:

Are the URLs Google isn't indexing actually supposed to be indexed?

A site with 100,000 URLs and 50,000 excluded URLs isn't necessarily broken.

A site with 1,000 important pages and only 500 indexed probably has a much bigger problem.

Google's Page Indexing report shows how many pages Google has crawled/attempted to crawl and why pages aren't indexed.

---

6. Audit canonicalization

For every important page:

• Has a canonical
• Canonical points to the preferred URL
• Canonical URL returns 200
• Canonical isn't redirected
• Canonical isn't noindex
• Canonical isn't blocked
• HTTP/HTTPS versions are consistent
• www/non-www versions are consistent
• Trailing-slash conventions are consistent
• Parameter URLs are handled appropriately
• Pagination/faceted navigation doesn't create massive duplicate URL sets

Then compare:

Declared canonical vs Google-selected canonical

If you repeatedly see those disagreeing, investigate why.

Google explicitly notes that canonicalization helps it determine which duplicate URL should be treated as the canonical version.

---

7. Audit redirects and status codes

Build a list of every URL returning:

2xx

• Is it supposed to exist?

3xx

• Is it necessary?
• Is it a single-hop redirect?
• Is it redirecting to the most relevant destination?

4xx

• Is the URL genuinely gone?
• Is something linking to it?
• Should it redirect somewhere useful?

5xx

• Fix immediately if affecting important URLs.

Pay particular attention to:

URL A → URL B → URL C

You generally want:

URL A → URL C

rather than chains.

---

8. Audit site architecture and internal linking

This is where a good technical SEO often goes beyond automated tools.

Check:

• Important pages are reachable through internal links
• No important orphan pages
• Important pages aren't buried excessively deep
• Navigation reflects site hierarchy
• Category → subcategory → product relationships make sense
• Contextual internal links exist
• Anchor text is descriptive
• Important pages receive sufficient internal link equity
• No huge numbers of low-value URLs are consuming crawl resources

Look at crawl depth and internal link counts.

A page ranking poorly might not have a technical error at all—it may simply be poorly connected to the rest of the site.

---

9. JavaScript SEO

This becomes critical with React, Vue, Angular, Next.js, etc.

Check:

• Primary content exists in rendered HTML
• Important links are discoverable
• Navigation works without problematic JS dependencies
• Canonicals are correct after rendering
• Meta tags are correct after rendering
• Structured data exists after rendering
• Important images load
• Content isn't hidden behind user interaction
• No critical content appears only after client-side execution
• No hydration/rendering errors

Compare:

Raw HTML vs rendered DOM vs what Google sees.

Search Console's URL Inspection can expose rendered information and JavaScript-related details, while Screaming Frog can crawl with JavaScript rendering enabled.

---

10. Core Web Vitals & performance

Check both lab and real-user data.

The current Core Web Vitals are:

• LCP: ≤ 2.5 seconds
• INP: < 200 ms
• CLS: < 0.1

Google recommends achieving good Core Web Vitals for search and overall user experience.

Audit:

• LCP
• INP
• CLS
• TTFB
• Image size
• Image formats
• Lazy loading
• Render-blocking CSS
• JavaScript execution
• Third-party scripts
• Font loading
• Caching
• Compression
• CDN
• Server response time

Use:

Search Console → Core Web Vitals for real-world field data.

PageSpeed Insights for diagnostics and opportunities.

Don't obsess over getting a perfect Lighthouse score. Fix the things affecting actual users and important templates.

---

11. Mobile SEO

Google uses mobile-first indexing, so check the mobile version—not just desktop.

• Responsive design
• Same primary content on mobile
• Same important links
• Same structured data
• Correct viewport
• No intrusive interstitial problems
• Buttons/links usable
• No horizontal scrolling
• Images work correctly
• Mobile navigation is crawlable

---

12. HTTPS and technical security

Check:

• HTTPS everywhere
• HTTP → HTTPS redirects
• No mixed content
• Valid certificate
• Correct canonical HTTPS URLs
• No HTTP URLs in XML sitemap
• No internal links to HTTP versions
• Security issues in Search Console
• No accidental staging/test site indexing

---

13. Structured data

Check relevant schema types for the business/site.

For example:

• Organization
• LocalBusiness
• Product
• Article
• BreadcrumbList
• Event
• FAQ where legitimately applicable
• Review/AggregateRating where eligible

Then verify:

• Valid JSON-LD
• Correct properties
• Data matches visible content
• No misleading markup
• No duplicate/conflicting schema
• Rich Results Test passes where applicable

Don't add schema merely because a plugin allows it. Use structured data that accurately describes the page.

---

14. International SEO

For multilingual/multiregional sites:

• Correct language/region targeting
• hreflang implementation
• Reciprocal hreflang references
• Valid language-region codes
• Self-referencing hreflang where appropriate
• Correct canonicals
• No accidental cross-language canonicalization
• XML sitemap/hreflang implementation if used
• Language versions internally linked

This is an area where automated crawlers are particularly useful because mistakes can exist across thousands of URLs.

---

15. Images

Audit:

• Image URLs return 200
• No broken images
• Appropriate dimensions
• Modern formats where appropriate
• Compression
• Lazy loading below the fold
• Important images aren't accidentally lazy-loaded
• Descriptive filenames where useful
• Appropriate alt text
• Images aren't unnecessarily huge

---

16. Log-file analysis — advanced audits

For larger sites, go one step further.

Analyze server logs to determine:

• Which URLs Googlebot actually crawls
• How frequently it crawls them
• Whether Googlebot wastes time on parameters
• Whether Googlebot hits redirects
• Whether Googlebot encounters 404/5xx URLs
• Whether important pages are being crawled
• Whether crawl behavior changed after a migration

This is especially valuable for:

• Ecommerce
• Large publishers
• Marketplaces
• International sites
• Sites with millions of URLs

A crawler tells you what it can find.

Logs tell you what Googlebot actually did.

That's a major distinction.

---

How I'd prioritize the findings

Don't hand a developer a spreadsheet containing 247 equally important "SEO errors."

I'd classify findings like this:

🔴 Critical

Fix immediately.

Examples:

• Important pages blocked by robots.txt
• Important pages noindex
• Major server errors
• Site accidentally canonicalized to another domain
• HTTPS/indexation disaster
• Migration redirects missing
• Major JavaScript rendering failure

🟠 High

Fix next.

Examples:

• Large numbers of orphan pages
• Incorrect canonicals
• Broken internal links
• Index bloat
• Major duplicate URL problems
• Significant CWV problems
• Poor site architecture

🟡 Medium

Fix as part of normal optimization.

Examples:

• Duplicate titles
• Missing meta descriptions
• Image optimization
• Redirect chains
• Excessive crawl depth

🟢 Low

Don't let these distract you.

Examples:

• Minor metadata inconsistencies
• Small HTML issues with no SEO consequence
• Tool-specific "warnings" that don't affect crawling, indexing, UX or search appearance

---

The workflow I'd actually use

Here's the practical sequence:

1. Search Console first

↓

2. Establish baseline

↓

3. Crawl with Screaming Frog

↓

4. Crawl the XML sitemap separately

↓

5. Compare sitemap vs crawl vs GSC

↓

6. Investigate indexation

↓

7. Investigate canonicals

↓

8. Investigate redirects/status codes

↓

9. Analyze internal linking/site architecture

↓

10. Test JavaScript rendering

↓

11. Audit Core Web Vitals/performance

↓

12. Check structured data

↓

13. Check mobile

↓

14. Check international SEO if applicable

↓

15. Analyze logs for large/complex sites

↓

16. Prioritize fixes by SEO impact

↓

17. Implement fixes

↓

18. Re-crawl and validate in Search Console

This last step is important. A technical audit isn't complete when you've identified problems. It's complete when you've verified the fixes.

---

My "best tool" answer

If you're learning technical SEO, I'd start with:

🥇 Best core stack

Google Search Console + Screaming Frog + PageSpeed Insights

That gives you an excellent combination of:

Google's data + your own crawler + performance data.

Then add Ahrefs or Semrush if you need ongoing monitoring, backlink analysis, competitor research, reporting, or a more automated workflow. Ahrefs' Site Audit currently covers 170+ predefined issues, while Semrush's Site Audit advertises 140+ technical checks.

And I would not use the "SEO health score" from any one platform as the definition of whether a site is technically healthy. Those scores are useful for tracking, but the actual business question is:

Are our valuable pages crawlable, indexable, technically accessible, fast enough, properly understood, and internally discoverable?

That's the mindset of a strong technical SEO audit.

An SEO specialist performs a technical SEO audit by systematically evaluating a website's infrastructure, code, server performance, and indexability to ensure search engines can efficiently crawl, render, and index its pages.

The audit process follows a structured workflow using industry-standard tools and a comprehensive checklist.

The Best Tools for a Technical SEO Audit

• Screaming Frog SEO Spider : The gold standard for desktop-based crawling, uncovering broken links, redirect chains, missing meta tags, and duplicate content.
• Google Search Console (GSC) : Essential, free data directly from Google showing indexing errors, core web vitals, and manual actions.
• PageSpeed Insights / Lighthouse : Analyzes page performance and Core Web Vitals (LCP, INP, CLS).
• Ahrefs or Semrush : Comprehensive cloud-based site auditing tools that run scheduled automated crawls and flag site health issues.

The Ultimate Technical SEO Audit Checklist

1. Crawlability and Indexability
   • Robots.txt : Verify that robots.txt is not accidentally blocking critical pages or resources, and that it references your XML sitemap.
   • XML Sitemap : Ensure the sitemap is clean, includes only 200 OK indexable URLs, and is submitted to Google Search Console.
   • Meta Robots & X-Robots Tags : Check for unintended noindex or nofollow tags on important pages.
   • Canonical Tags : Confirm every page has a self-referencing canonical tag (or correct cross-domain canonical) to prevent duplicate content issues.
2. Site Architecture and Internal Linking
   • URL Structure : Verify URLs are clean, lowercase, descriptive, and use hyphens rather than underscores.
   • Redirects : Identify and fix redirect chains (e.g., Page A -> Page B -> Page C) and redirect loops. Replace internal links pointing to redirected URLs (301) with direct links to the final target.
   • Broken Links : Find and fix all internal and external 404 Not Found errors.
   • Site Depth : Ensure important pages are within 3 to 4 clicks from the homepage.
3. Page Speed and Core Web Vitals
   • Largest Contentful Paint (LCP) : Optimize image loading, server response times, and render-blocking resources to keep LCP ≤ 2.5 seconds.
   • Interaction to Next Paint (INP) : Minimize long JavaScript tasks to ensure a smooth user interface responsiveness (INP ≤ 200 ms).
   • Cumulative Layout Shift (CLS) : Set explicit size attributes on images and embeds to prevent unexpected layout shifts (CLS ≤ 0.1).
4. Mobile-Friendliness and Security
   • Responsive Design : Test that the mobile layout adjusts cleanly without horizontal scrolling or tiny tap targets.
   • HTTPS Implementation : Ensure an active SSL certificate is installed and all HTTP URLs properly redirect (301) to HTTPS with no mixed-content errors.
5. Advanced Technical Elements
   • Structured Data (Schema) : Validate JSON-LD markup using the Schema Markup Validator to earn rich snippets.
   • Internationalization : Verify correct implementation of hreflang tags for multi-language or multi-regional sites.

If you want, I can dive deeper into how to fix a specific issue (like fixing Core Web Vitals or setting up hreflang), or you can tell me what platform your site is built on (WordPress, Shopify, custom, etc.) so I can tailor the checklist.