How to Create an XML Sitemap (And Keep It Useful)
SEO · Published 2026-11-10 · Updated 2026-11-15 · 7 min read · By Tool Magic Lab Editorial Team
Table of contents
A sitemap is a machine-readable list of your site's important URLs. It does not guarantee indexing and it does not boost rankings — it helps crawlers discover pages efficiently and tells you, via Search Console, which of your important pages search engines are ignoring. That diagnostic value is the real reason to maintain one properly.
The protocol rules that matter
The sitemap protocol has hard limits: one file may contain at most 50,000 URLs and be at most 50 MB uncompressed. Larger sites use a sitemap index file that points to multiple sitemap files — Tool Magic Lab does exactly this, splitting its tool catalogue across sitemap-tools-1.xml and sitemap-tools-2.xml under one sitemap-index.xml. Every URL must be fully qualified (https://…), canonical, and return HTTP 200. Optional fields are lastmod, changefreq and priority; of these, only an accurate lastmod is widely used — a lastmod that lies (updated daily on pages that never change) teaches crawlers to ignore it.
What belongs in a sitemap — and what does not
Include: canonical product, article, category and tool pages you want found. Exclude: redirects, 404s, noindexed pages, duplicate parameter URLs, login/cart/account pages, and anything robots.txt blocks (listing a blocked URL is a contradiction crawlers flag). A small, clean sitemap of 800 real pages beats a generated dump of 40,000 URLs nobody wants indexed.
Creating one, step by step
- List your canonical URLs — from your CMS export, your route registry, or a crawl of the live site.
- Generate the XML with the XML Sitemap Generator (/seo/xml-sitemap-generator): paste the URLs, set honest lastmod dates, and download the file.
- If you exceed 50,000 URLs, split by type (pages, products, tools, blog) and create a sitemap index that references each file.
- Upload the files to your domain root and reference them in robots.txt with a Sitemap: line.
- Submit the index URL in Google Search Console and Bing Webmaster Tools, then check the coverage report after a few days.
Keeping it useful
A sitemap is a living file. Regenerate or update it when pages are added, removed or materially changed — CMSs and frameworks (Next.js included) can generate it from the same registry that powers the site, which is the only approach that never drifts. Monthly, compare three numbers: URLs in the sitemap, URLs indexed, and URLs that actually receive traffic. The gaps between those numbers are your real SEO to-do list.
Frequently asked questions
**Does a sitemap help rankings?** Not directly. It improves discovery and gives you indexing diagnostics; rankings come from content, links and relevance.
**How often should lastmod change?** Only when the page's content genuinely changes. Automated 'today' dates on static pages are worse than no lastmod at all.
**My sitemap has errors in Search Console — what first?** Usually a listed URL that redirects or 404s. Remove non-200 and non-canonical URLs and resubmit.
Written by the Tool Magic Lab Editorial Team. Tool Magic Lab content is produced with AI assistance and editorially reviewed for accuracy — see our AI Usage Disclosure and Editorial Policy.
