How to create sitemap for dynamic routes?
Answered
Raymond posted this in #help-forum
RaymondOP
I have some routes like
[user]/[xyz]
that can have potentially large amounts of pages. How does one create a sitemap for this and stay within Google's 50MB limit?Answered by B33fb0n3
you can generate sitemaps like this ([read more](https://nextjs.org/docs/app/api-reference/functions/generate-sitemaps#example)):
Like that you stay within google limits and be able to place everything correctly in the sitemaps
import { BASE_URL } from '@/app/lib/constants'
export async function generateSitemaps() {
// Fetch the total number of products and calculate the number of sitemaps needed
return [{ id: 0 }, { id: 1 }, { id: 2 }, { id: 3 }]
}
export default async function sitemap({
id,
}: {
id: number
}): Promise<MetadataRoute.Sitemap> {
// Google's limit is 50,000 URLs per sitemap
const start = id * 50000
const end = start + 50000
const products = await getProducts(
`SELECT id, date FROM products WHERE id BETWEEN ${start} AND ${end}`
)
return products.map((product) => ({
url: `${BASE_URL}/product/${product.id}`,
lastModified: product.date,
}))
}
Like that you stay within google limits and be able to place everything correctly in the sitemaps
18 Replies
@Raymond I have some routes like `[user]/[xyz]` that can have potentially large amounts of pages. How does one create a sitemap for this and stay within Google's 50MB limit?
you can generate sitemaps like this ([read more](https://nextjs.org/docs/app/api-reference/functions/generate-sitemaps#example)):
Like that you stay within google limits and be able to place everything correctly in the sitemaps
import { BASE_URL } from '@/app/lib/constants'
export async function generateSitemaps() {
// Fetch the total number of products and calculate the number of sitemaps needed
return [{ id: 0 }, { id: 1 }, { id: 2 }, { id: 3 }]
}
export default async function sitemap({
id,
}: {
id: number
}): Promise<MetadataRoute.Sitemap> {
// Google's limit is 50,000 URLs per sitemap
const start = id * 50000
const end = start + 50000
const products = await getProducts(
`SELECT id, date FROM products WHERE id BETWEEN ${start} AND ${end}`
)
return products.map((product) => ({
url: `${BASE_URL}/product/${product.id}`,
lastModified: product.date,
}))
}
Like that you stay within google limits and be able to place everything correctly in the sitemaps
Answer
@B33fb0n3 you can generate sitemaps like this ([read more](https://nextjs.org/docs/app/api-reference/functions/generate-sitemaps#example)):
tsx
import { BASE_URL } from '@/app/lib/constants'
export async function generateSitemaps() {
// Fetch the total number of products and calculate the number of sitemaps needed
return [{ id: 0 }, { id: 1 }, { id: 2 }, { id: 3 }]
}
export default async function sitemap({
id,
}: {
id: number
}): Promise<MetadataRoute.Sitemap> {
// Google's limit is 50,000 URLs per sitemap
const start = id * 50000
const end = start + 50000
const products = await getProducts(
`SELECT id, date FROM products WHERE id BETWEEN ${start} AND ${end}`
)
return products.map((product) => ({
url: `${BASE_URL}/product/${product.id}`,
lastModified: product.date,
}))
}
Like that you stay within google limits and be able to place everything correctly in the sitemaps
RaymondOP
By only fetching the 50,000 most recent, do I rely on Google's crawler having indexed (at an earlier time) the previous URLs that didn't make the cut?
@Raymond By only fetching the 50,000 most recent, do I rely on Google's crawler having indexed (at an earlier time) the previous URLs that didn't make the cut?
I wouldn't trust google and would build always all the sitemaps
@B33fb0n3 I wouldn't trust google and would build always all the sitemaps
RaymondOP
How do I avoid hitting the limit (even if compressed)? It's a dynamic route and can have as many pages as I can store in the database table.
@Raymond How do I avoid hitting the limit (even if compressed)? It's a dynamic route and can have as many pages as I can store in the database table.
you can select from where to where you want to build your sitemap paths. Like that its easy for you, to build all your routes
@B33fb0n3 you can select from where to where you want to build your sitemap paths. Like that its easy for you, to build all your routes
RaymondOP
According to https://developers.google.com/search/blog/2006/10/multiple-sitemaps-in-same-directory I can have a sitemap1, sitemap2,..., sitemapN, and just have them linked in the root site_map. Is my reasoning correct?
@Raymond According to https://developers.google.com/search/blog/2006/10/multiple-sitemaps-in-same-directory I can have a sitemap1, sitemap2,..., sitemapN, and just have them linked in the root site_map. Is my reasoning correct?
yea, generate the sitemaps like I mentioned and create a sitemap index to index the sitemaps
@Raymond solved?
RaymondOP
Is there somewhere I need to mark as solved or accept an answer?
At the moment I don't think I'll fill up even a single sitemap but I'm thinking or worst case scenarios and trying to write code that I don't need to touch again
@Raymond Is there somewhere I need to mark as solved or accept an answer?
no worries, I marked the correct message. Happy to help
Spectacled bear
Hello
RaymondOP
@B33fb0n3 how does one create sitemaps for routes like
/[user]/[project]
? The Next.js example has product
before a slug, whereas in my case I have /slug/slug@Raymond <@301376057326567425> how does one create sitemaps for routes like `/[user]/[project]`? The Next.js example has `product` before a slug, whereas in my case I have /slug/slug
it's the same here: use the generateSitemaps function. Provide the ids there (you can also add mulitple values) and build from that your sitemaps
I saw you discussed about that in #off-topic . However, I didnt read through the messages there. You may be able to tell me a conclusion about the messages there?
RaymondOP
It just seems like too much magic involved, making it difficult for a first timer to implement. But I'll give this function another try.
RaymondOP
I've used
generateSitemaps()
in my root directory and found them available at http://localhost:3000/sitemap/1.xml. Doesn't this URL convention violate the rule laid out in https://www.sitemaps.org/protocol.html#location? These sitemaps (1.xml
2.xml
...) can only point to URLs within the /sitemap
routeRaymondOP
In my case my routes are set up like
/[user]/[project]
. Will the URLs in one of these 1.xml
, generated under /sitemap/[id].xml
, that point to https://my-website.com/monty/python
be valid?