The robots.txt file is the first resource most crawlers request and tells them which paths on your site they may or may not access. Fixing this issue means publishing a valid robots.txt at the site root so search engines, training crawlers, and answer-engine bots all have an explicit policy to follow. Without it, every crawler defaults to its own behavior, which can over-expose private sections or waste crawl budget on low-value URLs.

Example

User-agent: *
Disallow: /admin/
Disallow: /cart
Allow: /

Sitemap: https://example.com/sitemap.xml

Where to make the change

• Raw HTML or static site: create a plain text file named robots.txt and upload it to the document root so it is served at https://example.com/robots.txt.
• WordPress: WordPress generates a virtual robots.txt by default. Override it with a physical file in the site root, or use your SEO plugin's robots.txt editor.
• Shopify: Shopify auto-generates robots.txt and exposes a robots.txt.liquid template you can edit from the theme code editor to add or remove rules.
• Wix or Squarespace: both platforms generate robots.txt automatically and offer limited overrides through the SEO settings panel.
• Headless or framework sites: add robots.txt to the project's public or static directory so it ships with the build.

Common causes and how to resolve them

• File is missing entirely: create one. Even an empty file with User-agent: * and no rules is preferable to a 404.
• File returns a 5xx error: some crawlers stop crawling the entire site when robots.txt is unreachable. Investigate server or CDN issues that block the path.
• Accidentally blocking the whole site: a stray Disallow: / under User-agent: * hides every URL from search. Audit the file before deploying changes from staging.
• Wrong location: robots.txt must sit at the site root, not in a subdirectory. Crawlers do not look elsewhere.

Best practices

• Reference the sitemap: add a Sitemap: line so any crawler that fetches robots.txt also discovers your XML sitemap.
• Keep it simple: a short, well-commented file is easier to audit than a sprawling one full of legacy rules. Remove directives whose original purpose is no longer relevant.
• Do not rely on robots.txt for security: the file is public, so listing private paths in it advertises them. Block sensitive paths with authentication, not just Disallow.
• Manage AI crawlers explicitly: if you want to allow or block bots like GPTBot, ClaudeBot, or PerplexityBot, add named User-agent sections for each. Defaulting to allow keeps your content eligible for citation in AI answer engines.

Alt attributes provide a text alternative for images, used by screen readers when a user cannot see the image and by search engines and multimodal AI models to understand what the image depicts. Fixing this issue means writing concise, descriptive alt text for every meaningful image on the page, and using empty alt for purely decorative images so assistive technology can skip them. The same alt text helps multimodal LLMs and image search tools surface and cite your visuals.

Example

<img src="/images/marathon-shoes.jpg"
     alt="Side view of a lightweight neon-green marathon running shoe">

<img src="/images/divider.svg" alt="">

Where to make the change

• Raw HTML: add the alt attribute to every <img> tag. For decorative images, use alt="" so screen readers skip them.
• WordPress: the media library has an Alt text field on every uploaded image. Fill it once and the value is reused everywhere the image is inserted.
• Shopify: in the product or page editor, click an image and use the Edit alt text option. Theme code that renders product images should pull image.alt.
• Wix or Squarespace: click the image in the editor and use the Alt text or Image description field in the side panel.
• Headless or framework sites: require an alt prop on every image component and lint for missing values during build.

Common causes and how to resolve them

• Attribute missing: add alt="" at minimum so the page validates. Replace with descriptive text on any image that conveys information.
• Alt text is the filename: "IMG_4837.jpg" tells the user nothing. Replace with a description of what is in the image.
• Alt text duplicates the visible caption: screen readers will read both, doubling the user's listening time. Differentiate the alt from the caption or empty the alt.
• Decorative images get descriptive alt text: background patterns and dividers should use alt="", otherwise screen readers narrate noise.
• Bulk imports left alt blank: a one-off audit of the media library or a CMS-side script can fill missing alt values from product names, captions, or asset descriptions.

Best practices

• Describe the function, not just the appearance: for a chart, describe what the chart shows. For a button image, describe what the button does.
• Keep it concise: aim for under 125 characters. Screen readers do not pause mid-sentence, so long alt text is exhausting to listen to.
• Skip "image of" or "picture of": the screen reader already announces it as an image.
• Include the focus keyword when natural: alt text is a minor on-page signal, but stuffing keywords in unnatural alt text harms both accessibility and ranking.

Google Analytics 4 is a free measurement platform that records visitor behavior, traffic sources, and conversion data. Installing it does not directly affect rankings, but the data it produces is essential for measuring SEO performance, identifying high-value pages, and prioritizing future optimization work. Fixing this issue means installing the GA4 tag site-wide and confirming events appear in your reports.

Example

<!-- Place inside <head> on every page -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-XXXXXXXXXX"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());
  gtag('config', 'G-XXXXXXXXXX');
</script>

Where to make the change

• Raw HTML: paste the GA4 snippet into the <head> of every page. A shared layout file is the cleanest place.
• WordPress: use a site-kit or analytics plugin, or paste the snippet into your theme's header file. Avoid duplicating the tag across multiple plugins.
• Shopify: add the GA4 measurement ID under Online Store, Preferences, Google Analytics, or install the official Google channel app.
• Wix or Squarespace: both platforms offer a GA4 measurement ID field in their analytics settings panel.
• Headless or framework sites: install via Google Tag Manager or use the framework's analytics integration so the tag fires on every route change.

Common causes and how to resolve them

• Tag not installed at all: install GA4 using the snippet above or the platform-specific path.
• Universal Analytics tag still in place: Universal Analytics stopped processing data on July 1, 2024. Replace any UA tag (UA-...) with a GA4 measurement ID (G-...).
• Tag fires only on the homepage: ensure the snippet is in a shared template loaded by every page, not pasted into one post.
• Cookie banner blocks the tag before consent: integrate Consent Mode v2 so GA4 receives anonymized signals before consent and full data after.
• Tag duplicated: two snippets on the same page double-count sessions. Audit with the Tag Assistant browser extension and remove duplicates.

Best practices

• Verify with real-time reports: after installing, open GA4's Realtime view and load the site to confirm events arrive.
• Use Google Tag Manager: for any site beyond the simplest, GTM gives you a single place to manage analytics, conversion tags, and third-party scripts without code changes.
• Configure key events: mark conversions (purchase, lead, signup) as key events so reports highlight what matters for SEO ROI.
• Link to Search Console: connecting GA4 to Search Console surfaces organic queries alongside on-site behavior in the same reports.

This test fails when the raw HTML document weighs significantly more than necessary, even before counting images, CSS, or JavaScript. Bloated HTML increases time to first byte, slows parsing, and stalls rendering on mobile networks and lower-powered devices. Fixing this issue means trimming inline data, removing legacy markup, and keeping the document focused on what the user needs.

Where to make the change

• Application code or templates: identify which sections produce the most bytes (often a large inline JSON blob, an inline base64 image, or a giant comment block) and refactor them to load externally or be removed.
• WordPress: check for plugins or page builders that emit large inline configurations. Disable unused plugins and prefer themes that ship clean markup.
• Shopify: theme code may inline large product or section JSON. Move heavy data to a fetch-on-demand endpoint when possible.
• Headless or framework sites: review the rendered output for serialized state (for example __NEXT_DATA__ in Next.js) and trim what is sent on first paint by streaming or lazy-loading non-critical chunks.

Common causes and how to resolve them

• Inline base64 images or fonts: move them to external files so the browser can cache them across requests.
• Excessive inline CSS or JavaScript: link to external stylesheets and scripts so they cache. Inline only the truly critical CSS for above-the-fold content.
• Large server-side JSON payloads: ship only the data needed for first render. Fetch the rest on demand.
• Repeating boilerplate or commented-out code: strip comments and dead code in the production build.
• HTML compression disabled: Gzip or Brotli compression often shrinks the response by half or more. Confirm the server sends the appropriate Content-Encoding header.

Best practices

• Aim for under 100 KB of HTML: most pages can comfortably stay under this. Anything beyond a few hundred KB usually indicates structural bloat worth refactoring.
• Enable compression: Gzip is the safe minimum, Brotli is even better for HTML and other text formats.
• Defer non-critical content: rendering only what is needed for the initial viewport keeps both bytes and parse time down.
• Audit periodically: add HTML size to your performance budget so regressions get caught in CI.

This test fails when the page builds a Document Object Model with too many nodes or excessive nesting depth. Large DOMs increase memory usage, lengthen style and layout calculations, and slow rendering, particularly on mobile devices. Fixing this issue means simplifying the markup so the browser has less work to do on every interaction.

Where to make the change

• Application code or templates: identify the heaviest sections (long lists, repeating components, deep grid wrappers) and either paginate them, virtualize them, or flatten the markup.
• WordPress: heavy page-builder themes often emit deeply nested wrapper divs. Consider switching templates or using a theme that emits cleaner markup.
• Headless or framework sites: use a virtualized list component for long item lists so only visible rows are rendered into the DOM.

Common causes and how to resolve them

• Long lists rendered in full: show a paginated subset or virtualize the list so only on-screen items exist in the DOM.
• Excessive wrapper divs: a stack of layout wrappers ten deep can usually be flattened to two or three with modern CSS layouts (Grid, Flexbox).
• Hidden modals and dropdowns kept in the DOM: render them on demand instead of always emitting them with display: none.
• Third-party widget injecting many nodes: a chat widget or carousel may add hundreds of nodes per page. Lazy-load it after main content renders.

Best practices

• Aim for under 1500 nodes total and a maximum depth of 32: Lighthouse uses these as warning thresholds and they correlate with smooth rendering on mid-tier mobile devices.
• Render only what is visible: long lists, accordions, and modals belong outside the initial DOM until needed.
• Prefer flat semantic markup: a single <article> with a few children is faster than a deeply nested <div> tree.
• Audit with Lighthouse: the DOM size audit reports the worst-offending elements, which often points at a single template that needs rework.

This test reports on whether the page applies a nofollow directive that tells search engines not to follow outbound links. The nofollow attribute (and the related sponsored and ugc values) controls how link equity flows from your page. Fixing this issue means using the right rel value on each external link based on its purpose so signals are passed only where you intend.

Where to make the change

• Raw HTML: add rel="nofollow", rel="sponsored", or rel="ugc" to specific outbound links rather than blanket-applying them site-wide.
• WordPress: the block editor offers a "mark as nofollow" option per link, and SEO plugins can auto-tag external links by domain.
• Shopify, Wix, Squarespace: editor-generated links allow rel attribute toggles; manual HTML embeds need them set explicitly.
• Headless or framework sites: wrap external links in a component that decides the rel based on the link's classification.

Common causes and how to resolve them

• Site-wide nofollow on every external link: overly aggressive. Reserve nofollow for sponsored, untrusted, or user-generated content.
• Sponsored content not labeled: use rel="sponsored" on paid placements and affiliate links to comply with Google's link spam policy.
• User-generated content unmarked: comments, forum posts, and review user submissions should use rel="ugc".
• Trusted editorial links nofollowed: these should pass equity. Remove the nofollow.

Best practices

• Use the most specific value: sponsored for paid links, ugc for user content, nofollow for everything else you want to disclaim. They can be combined: rel="sponsored noopener".
• Do not nofollow internal links: "PageRank sculpting" with internal nofollow no longer works and just wastes crawl signal.
• Audit periodically: as content ages, sponsored relationships change. Review affiliate and partner links once a year.