Read the values of frontmatter from .mdx files

[deadcoder0904]

I want to use https://github.com/iamvishnusankar/next-sitemap to generate Sitemap.

However, when I use it normally like:

next-sitemap.js

module.exports = {
  siteUrl: 'https://example.com',
  generateRobotsTxt: true,
}

It generates sitemap for all my posts.

Although, in my .mdx files, I have a published key to know whether the post is a draft or it is ready to be published like:

export const meta = {
	title: 'Hello World',
	date: '2020-09-11T14:54:37.229Z',
	tags: ['hello', 'world'],
	author: 'Akshay Kadam',
	published: false,
}

# Hello World

This is my first blog post, not yet ready to be published

How do I read the published key from .mdx file so I can generate sitemap of only published: true posts?

Note: I can't use https://www.npmjs.com/package/gray-matter because my .mdx files have an exported object as metadata.

[wooorm]

This question might be better asked in next-sitemap, because it is rather next and next-sitemap specific

[deadcoder0904]

I did ask it there too but I think it's more of .mdx specific question. I have to read the export from .mdx file so not sure how it's next & next-sitemap specific?

Here's the code without using next-sitemap (i want to read the meta from the pages array, see console.log(pages)):

const fs = require('fs')
const glob = require('fast-glob')
const prettier = require('prettier')

const config = {
	siteUrl: 'https://akshaykadam.com',
	changefreq: 'daily',
	priority: '0.7',
	lastmod: new Date().toISOString(),
}

const main = async () => {
	const prettierConfig = await prettier.resolveConfig('./prettier.config.js')
	const pages = await glob(['src/pages/**/*{.js,.mdx}', '!src/pages/_*.js', '!src/pages/api'])

	console.log(pages)

	const sitemap = `
        <?xml version="1.0" encoding="UTF-8"?>
        <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
            ${pages
							.map((page) => {
								const path = page.replace('pages', '').replace('.js', '').replace('.mdx', '')
								const route = path === '/index' ? '' : path

								return `
                        <url>
														<loc>${`${config.siteUrl}${route}`}</loc>
														<changefreq>${`${config.lastmod}`}</changefreq>
														<priority>${`${config.priority}`}</priority>
														<lastmod>${`${config.lastmod}`}</lastmod>
                        </url>
                    `
							})
							.join('')}
        </urlset>
    `

	const formatted = prettier.format(sitemap, {
		...prettierConfig,
		parser: 'html',
	})

	// eslint-disable-next-line no-sync
	fs.writeFileSync('public/sitemap.xml', formatted)
}

main()

I even tried using babel-plugin-glob-array but that doesn't work when I have to import mid-level. I need something like dynamic-import but for .mdx?

[wooorm]

Your question is not specific to MDX either. With that code you have to load the JavaScript files too, right?
The loading of files is stuff Next takes care of for you, hence your question can be better asked there

[deadcoder0904]

Your question is not specific to MDX either

Well, I don't understand how 😂

I need to import { meta } from "./src/pages/**/*.mdx" for all .mdx files which I suppose should make it MDX-specific.

With that code you have to load the JavaScript files too, right?

This is just for a sitemap. I don't need to load JavaScript files.

But yeah, I also asked at /r/nextjs & Next.js discussions too. Lmk if you know the answer or have any suggestions?

[ChristianMurphy]

Your question is not specific to MDX either

Well, I don't understand how 😂

For all intents and purposes import { meta } from "./src/pages/**/*.mdx" is the same as import { meta } from "./src/pages/**/*.js" (after next/webpack has run).
The problem you are solving is an import one not an mdx one.

[deadcoder0904]

@ChristianMurphy now I got it, thanks 👍

[deadcoder0904]

I used a custom sitemap generator like:

scripts/build-sitemap.js

const fs = require('fs')
const glob = require('fast-glob')
const prettier = require('prettier')
const { getAllPublishedEssays } = require('@/utils/essay/getAllEssayPreviews')
const { getAllPublishedTutorials } = require('@/utils/tutorial/getAllTutorialPreviews')

const posts = [...getAllPublishedEssays(), ...getAllPublishedTutorials()]

const config = {
	siteUrl: 'https://akshaykadam.com',
	changefreq: 'daily',
	priority: '0.7',
	lastmod: new Date().toISOString(),
}

const robotsTxt = `
User-agent: *
Allow: /
Host: ${config.siteUrl}
Sitemap: ${config.siteUrl}/sitemap.xml
`.trim()

const main = async () => {
	const prettierConfig = await prettier.resolveConfig('./prettier.config.js')

	const sitemap = `
        <?xml version="1.0" encoding="UTF-8"?>
        <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
            ${posts
							.map((post) => {
								return `
                        <url>
														<loc>${`${config.siteUrl}/${post.slug}`}</loc>
														<changefreq>${`${config.changefreq}`}</changefreq>
														<priority>${`${config.priority}`}</priority>
														<lastmod>${`${config.lastmod}`}</lastmod>
                        </url>
                    `
							})
							.join('')}
        </urlset>
    `

	const formatted = prettier.format(sitemap, {
		...prettierConfig,
		parser: 'html',
	})

	fs.writeFileSync('public/sitemap.xml', formatted)
	fs.writeFileSync('public/robots.txt', robotsTxt)
}

main()

getAllEssayPreviews.ts

export const getAllPublishedEssays = (): Array<{
	slug: string
	module: any
}> =>
	importAll(require.context('../../pages/essay/?rss', true, /\.mdx$/))
		.filter((p: Post) => p.module.meta.published)
		.sort((a: Post, b: Post) => dateSortDesc(a.module.meta.date, b.module.meta.date))

next.config.js

const withBundleAnalyzer = require('@next/bundle-analyzer')({
	enabled: process.env.ANALYZE === 'true',
})

module.exports = withBundleAnalyzer({
	.
	.
	.
	webpack: (config, options) => {
		.
		.
		.
		if (!options.dev && options.isServer) {
			const originalEntry = config.entry

			config.entry = async () => {
				const entries = { ...(await originalEntry()) }
				entries['./scripts/build-rss'] = './scripts/build-rss.js'
				entries['./scripts/build-sitemap'] = './scripts/build-sitemap.js'
				return entries
			}
		}

		return config
	},
})