Building a Dynamic Sitemap with Data from Sanity in Next.js

Creating a sitemap is crucial for better SEO and ensuring that search engines can efficiently crawl and index your website's pages. In this guide, we'll create a dynamic sitemap using data fetched from Sanity in a Next.js 13+ application with the `app` directory.

Prerequisites

A basic understanding of Next.js framework
Sanity configured and running with some content
Node.js and npm installed

Step 1: Set Up Your Next.js Project

If you don’t already have a Next.js project set up, you can create one using the following command:

terminal

1npx create-next-app my-nextjs-app --use-npm
2cd my-nextjs-app

Step 2: Configure Sanity Client

Install Sanity client if you haven't already:

terminal

1npm install @sanity/client

Create a `client.ts` file to configure your Sanity client in your `lib` directory:

lib/client.ts

1import sanityClient from '@sanity/client';
2
3export const client = sanityClient({
4  projectId: 'your-project-id', // replace with your project ID
5  dataset: 'your-dataset-name', // replace with your dataset name
6  useCdn: true, // Use CDN for performance
7  apiVersion: '2023-01-01', // Use current date or specific API version
8});

Step 3: Fetch Data from Sanity

Define fetching functions to retrieve data from Sanity.

Create a `routes.ts` file in your `lib` directory:

lib/routes.ts

1import { groq } from 'next-sanity';
2import { client } from './client';
3
4// Simplified function to fetch article slugs
5export const getAllArticles = async (): Promise<{ slug: string, lastModified: string }[]> => {
6  const articlesQuery = groq`*[_type == "article"]{ "slug": slug.current, _updatedAt }`;
7  const articles = await client.fetch<{ slug: string, _updatedAt: string }[]>(articlesQuery);
8  return articles.map(article => ({
9    slug: `/articles/${article.slug}`,
10    lastModified: article._updatedAt
11  }));
12};
13
14// Simplified function to fetch software library slugs
15export const getAllSoftwareLibraries = async (): Promise<{ slug: string, lastModified: string }[]> => {
16  const librariesQuery = groq`*[_type == "software-library"]{ "slug": slug.current, _updatedAt }`;
17  const libraries = await client.fetch<{ slug: string, _updatedAt: string }[]>(librariesQuery);
18  return libraries.map(library => ({
19    slug: `/libraries/${library.slug}`,
20    lastModified: library._updatedAt
21  }));
22};
23
24// Fetch all routes for the sitemap
25export const getAllRoutes = async () => {
26  const articleRoutes = await getAllArticles();
27  const libraryRoutes = await getAllSoftwareLibraries();
28
29  // Add any required static routes manually
30  const staticRoutes = [
31    { slug: '/', lastModified: new Date().toISOString().split('T')[0] },
32    { slug: '/about', lastModified: new Date().toISOString().split('T')[0] },
33    { slug: '/contact', lastModified: new Date().toISOString().split('T')[0] },
34    { slug: '/articles', lastModified: new Date().toISOString().split('T')[0] },
35    { slug: '/newsletter', lastModified: new Date().toISOString().split('T')[0] },
36    { slug: '/workshops', lastModified: new Date().toISOString().split('T')[0] },
37  ];
38
39  return [...staticRoutes, ...articleRoutes, ...libraryRoutes];
40};

Step 4: Generate Sitemap Using Metadata API

In Next.js 13+, you leverage the new `app` directory structure to create and serve the sitemap.

Create a file named `sitemap.ts` in your `app` directory.

app/sitemap.ts

1import { getAllRoutes } from '../lib/routes';
2
3export default async function sitemap() {
4  const routes = await getAllRoutes();
5  const baseUrl = process.env.NEXT_PUBLIC_SITE_URL || 'https://your-website.com';
6
7  const urls = routes.map(({ slug, lastModified }) => ({
8    url: `${baseUrl}${slug}`,
9    lastModified,
10  }));
11
12  return urls;
13}

Step 5: Create Robots.txt (Optional but Recommended)

To provide instructions to web crawlers, you can create a `robots.txt` file.

Create a file named `robots.ts` in your `app` directory.

app/robots.ts

1const siteUrl = process.env.NEXT_PUBLIC_SITE_URL || 'https://your-website.com';
2
3export default function robots() {
4  return {
5    rules: [
6      {
7        userAgent: '*', // Applies to all user agents
8        allow: '/', // Allows access to the entire site
9      },
10    ],
11    sitemap: `${siteUrl}/sitemap.xml`,
12    host: siteUrl,
13  };
14}

Step 6: Configure Environment Variables

Ensure you have configured the `NEXT_PUBLIC_SITE_URL` in your `.env` file.

1NEXT_PUBLIC_SITE_URL=https://your-website.com

Step 7: Testing

1. Run Your Next.js App:

1npm run dev

2. Visit Your Sitemap URL:

1http://localhost:3002/sitemap.xml

3. Visit Your Robots URL (if implemented):

1http://localhost:3002/robots.txt

By following these steps, you’ll have a dynamic sitemap generated and served by your Next.js application, utilizing data fetched from Sanity. This setup will enhance your website’s SEO and ensure that your dynamic content is efficiently indexed by search engines.

Final Directory Structure

1apps/
2└── lauro-silva/
3    ├── app/
4    │   ├── sitemap.ts
5    │   ├── robots.ts (optional)
6    └── sanity/
7        ├── lib/
8        │   ├── client.ts
9        │   ├── routes.ts
10// Other necessary files and directories

This approach fully utilizes the Next.js 13+ `app` directory and the new metadata API, ensuring a clean and performant implementation for creating dynamic sitemaps and potentially a `robots.txt` file for your website.