The Problem

After submitting my sitemap to Google Search Console, I noticed that all my /topic/ pages were showing a frustrating error:

Page indexing: Page is not indexed: Redirect error
Page fetch: Failed: Redirect error

For example, the route topic/friends_sitcom was being crawled by Googlebot but failing to index. This was concerning because these are important landing pages for game topics that I wanted indexed for SEO.

Debugging Process

Step 1: Check the HTTP Headers

The first step was touse curl to inspect the actual HTTP response from my server:

curl -I https://topamine.com/topic/friends_sitcom

The response revealed the issue:

HTTP/2 301 
location: /topic/friends_sitcom/

Aha! A 301 redirect was happening from /topic/friends_sitcom to /topic/friends_sitcom/ (with a trailing slash).

Step 2: Verify the Final Destination

The obvious next step was to check the URL with the trailing slash:

curl -I https://topamine.com/topic/friends_sitcom/

This returned:

HTTP/2 200
content-type: text/html; charset=UTF-8

The page with the trailing slash was serving correctly with a 200 OK status.

Step 3: Understand Why This Matters

Google’s sitemap guidelines are clear: sitemaps should contain canonical URLs - the final destination URLs without any redirects. When Googlebot encounters a redirect:

  1. Wastes crawl budget - Google has to make two requests instead of one
  2. May not index properly - Redirects can confuse indexing signals
  3. Reports as an error - Search Console flags it as a problem

The Root Cause

The static site generation script was creating directories for each topic page:

build/
  topic/
    friends_sitcom/
      index.html

Netlify was serving directory-based URLs with trailing slashes by default. When you request /topic/friends_sitcom, Netlify automatically redirects to /topic/friends_sitcom/ to serve the index.html file inside that directory.

However, the sitemap listed URLs without trailing slashes:

<url>
  <loc>https://topamine.com/topic/friends_sitcom</loc>
</url>

This mismatch caused the redirect chain.

The Solution

The fix was simple: update the sitemap generation script to include trailing slashes for all topic URLs.

Before:

allTopics.forEach(topic => {
    sitemap += '  <url>\n';
    sitemap += `    <loc>${DOMAIN}/topic/${topic}</loc>\n`;
    sitemap += `    <lastmod>${currentDate}</lastmod>\n`;
    sitemap += '  </url>\n';
});

After:

allTopics.forEach(topic => {
    sitemap += '  <url>\n';
    sitemap += `    <loc>${DOMAIN}/topic/${topic}/</loc>\n`;  // Added trailing slash
    sitemap += `    <lastmod>${currentDate}</lastmod>\n`;
    sitemap += '  </url>\n';
});

Verification

After regenerating the sitemap, verified the URLs:

grep "<loc>" build/sitemap.xml | head -5

Output:

<loc>https://topamine.com/</loc>
<loc>https://topamine.com/topics</loc>
<loc>https://topamine.com/topic/movies/</loc>
<loc>https://topamine.com/topic/christmas/</loc>
<loc>https://topamine.com/topic/friends_sitcom/</loc>

All topic URLs now had trailing slashes and matched the actual URLs served by Netlify.

Key Takeaways

  1. Use curl -I to debug indexing issues - HTTP headers reveal redirect chains that browsers hide
  2. Match your sitemap URLs to your server’s canonical URLs - No redirects should be present
  3. Understand your hosting platform’s URL conventions - Netlify, GitHub Pages, and other platforms have different trailing slash behaviors
  4. Test before submitting to Google - A quick curl check can save you from indexing headaches

Since we talked about topics/friends_sitcom (oops! did it again!), how about play a Friends themed word guessing game?

Not a Friends fan? A few of my friends might judge ;) but there are a bunch of other topics you can check out.