RFC: Multiple MDXs in a Single File

[ChristopherBiscardi]

multiple MDXs in a single file

The wider ecosystem seems to have a need for multiple MDX components sourced from a single file. Consider the following examples.

mdx-deck

mdx-deck uses multiple MDX components from a single file to render a slide deck.

# This is the title of my deck
---
# About Me
---
```jsx
<CodeSnippet />
```
---
import Demo from './components/Demo'

<Demo />
---
# The end

small content

There is a common use case in websites to have multiple blobs of smaller content. I've used some mock content from the Gatsbyjs.org site here to illustrate the point, but pretty much all product sites do this somewhere including

Next

React

Terraform

# Modern web tech without the headache

Enjoy the power of the latest web technologies – React.js , Webpack , modern JavaScript and CSS and more — all set up and waiting for you to start building.
---
# Bring your own data

Gatsby’s rich data plugin ecosystem lets you build sites with the data you want — from one or many sources: Pull data from headless CMSs, SaaS services, APIs, databases, your file system, and more directly into your pages using GraphQL .
---
# Scale to the entire internet

Gatsby.js is Internet Scale. Forget complicated deploys with databases and servers and their expensive, time-consuming setup costs, maintenance, and scaling fears. Gatsby.js builds your site as “static” files which can be deployed easily on dozens of services.

"Storybook", Documentation sites, and other interesting use cases

MDX is a viable replacement for Storybook story files, but writing a new story example in a new file is a lot of overhead. Additionally, writing a set of examples with code blocks in a single file can result in documented dynamic examples.

Expected behaviour

The behavior I'm proposing is to define a way to load multiple MDX components from a single file. the following small example:

some content
---
more b content
---
ccccccccccc

would (conceptually) result in an array of MDX components

[
  function MDXContent({ components, ...props }) {
    return (
      <div name="wrapper" components={components}>
        <p>{`some content`}</p>
      </div>
    );
  },
  function MDXContent({ components, ...props }) {
    return (
      <div name="wrapper" components={components}>
        <p>{`more b content`}</p>
      </div>
    );
  },
  function MDXContent({ components, ...props }) {
    return (
      <div name="wrapper" components={components}>
        <p>{`ccccccccccc`}</p>
      </div>
    );
  }
];

This output can then be rendered directly in place of a normal MDX component

comment: this output will probably be a function that returns an array. This allows us to access the result (the array) and use it as a component directly.

import AFewMDXs from './small-content.mdxs'

export default props => <div>
  <AFewMDXs {...props} />
</div>

---

Extras

The full suite of MDX functionality should be supported, including layouts, props, exports, etc.

layout

layout is a default export. While it is possible to replace wrapper to achieve similar behavior, there are frameworks that don't fully support page-level wrapping components so wrapper is hard to use on a per-component or page-level basis. For this reason, each component should come with it's own layout. This behaves the same as a regular MDX file.

export default props => <section {...props} />

some content
---
more b content
---
ccccccccccc

So we end up with the following:

/* @jsx mdx */

const layoutProps1 = {};
const MDXLayout1 = props => <section {...props} />;
function MDXContent1({ components, ...props }) {
  return (
    <div name="wrapper" components={components}>
      <MDXLayout1 {...layoutProps} {...props}>
        <p>{`some content`}</p>
      </MDXLayout1>
    </div>
  );
}
MDXContent1.isMDXComponent = true;

const layoutProps2 = {};

function MDXContent2({ components, ...props }) {
  return (
    <div name="wrapper" components={components}>
      <p>{`more b content`}</p>
    </div>
  );
}
MDXContent2.isMDXComponent = true;

const layoutProps3 = {};

function MDXContent3({ components, ...props }) {
  return (
    <div name="wrapper" components={components}>
      <p>{`ccccccccccc`}</p>
    </div>
  );
}
MDXContent3.isMDXComponent = true;

const MDXMulti = props => [
  <MDXContent1 {...props} />,
  <MDXContent2 {...props} />,
  <MDXContent3 {...props} />,
];

export default MDXMulti

Note that each "default" export is allocated to it's own MDX Component, just like a regular file would be. We can include an extension that allows setting the default layout for a group of MDX content by specially processing the first MDX "file" if it only includes imports and exports.

export default props => <section {...props} />
---
some content
---
more b content
---
ccccccccccc

exports

Exports can exist at the global level as usual, but what happens to local exports? They would be set as properties on the MDX component

export const a = 2;

some content
---
export const b = 4;

more b content
---
ccccccccccc

example output:

// mdx 1, etc are in this file before this
const layoutProps2 = {};

function MDXContent2({ components, ...props }) {
  return (
    <div name="wrapper" components={components}>
      <p>{`more b content`}</p>
    </div>
  );
}
MDXContent2.isMDXComponent = true;
MDXContent2.b = 4
// rest of file

props

MDX components can take props. So can the multi-mdx. The difference is that props given to a multi-mdx component apply to all MDX content. This is clearly illustrated by the proposed result component:

const MDXMulti = props => [
  <MDXContent1 {...props} />,
  <MDXContent2 {...props} />,
  <MDXContent3 {...props} />,
];

[jxnblk]

Would love to have some more options like this. Related to this, I attempted to create a plugin to handle this better in mdx-deck v2 by checking for the actual node type instead of using regex/string manipulation, and I ended up having to duplicate a lot of mdx's internals: https://github.com/jxnblk/mdx-deck/blob/1986c65eea267a5edbb51842fac3d126cb162413/packages/mdx-plugin/index.js

[jescalan]

I think this is a really cool idea, but it feels to me like the type of thing that belongs in an extension/plugin/etc rather than in the core. It's a specialized use case, and a dramatic break from normal markdown syntax, where --- would be expected to render a horizontal rule.

[rheinardkorf]

It's a specialized use case, and a dramatic break from normal markdown syntax, where --- would be expected to render a horizontal rule.

This was my immediate first reaction too.

That said, I wouldn’t be opposed to something like /---.

[rheinardkorf]

Also, neat thing about MDX is how we can use MD frontmatter. Would a MDX break allow for additional “frontmatter” for the next component?

[ChristopherBiscardi]

Frontmatter wouldn't be affected because in platforms that support it, it's a pre-process step on the input string before the MDX is processed by mdx-js/mdx, so the frontmatter won't hit the mdx parser.

[Vadorequest]

Interesting, but using --- as a separator would conflict with the usual markdown behaviour, choosing another selector that doesn't conflict with existing markdown would be more appropriate, and avoid a big breaking change.

[benjaminpreiss]

Hello fellow devs!

I have been working on something similar the last days. Allow me to introduce my approach here for discussion:

The idea is to parse the MDX file before processing and splitting it into multiple parts that will later be stitched together in one single default export.

A thinkable layout of such a file could be:

{/* c:how-it-works.main */}

Some MDX here
...

{/* c:start */}

Some more MDX here
...

We can then parse the file and split it at the comments, such that c:some.content.path indicates the name of the section that we will later export. An exported object with these section names as keys could look like this:

const res = {
	"some.content.path": function() {
		const {Fragment: _Fragment, jsx: _jsx} = arguments[0]
		const no = 3.14
		function _createMdxContent(props) { /* … */ }
		function MDXContent(props = {}) { /* … */ }
		return {no, default: MDXContent}
	},
	"some.other.content.path": function() {
		//...
	},
	//...
};
export default res;

For that, we need to enable outputFormat: "function-body" though.

I already have a working @mdx/loader implementation that does this (Sadly not committed to a repo yet, but will post here later). The biggest problem was to hijack the processing of the loader and tell it to do it multiple times for a single file (once for each section).

I would therefore like to propose another option on @mdx-js/mdx which one could call "execute". It gives the user the chance to set up custom processing. Here is a glance into my implementation of @mdx-js/loader:

// ...

if (!process) {
    process = createFormatAwareProcessors(config).process
    map.set(hash, process)
  }

  if(options.execute) {
    options.execute.call(this, value, process, callback)
  } else {
    process({value, path: this.resourcePath}).then(
      (file) => {
        callback(null, file.value, file.map)
      },
      (/** @type VFileMessage */ e) => {
        const fpath = path.relative(this.context, this.resourcePath);
        e.message = `${fpath}:${e.name}: ${e.message}`;
        callback(e);
      }
    )
  }

// ...

My "execute" function that I pass to @mdx-js/loader looks as follows:

function execute(value, process, callback) {
  const splittedFile = value.split(/{[\n\t ]*\/\*\s*c:[\w.\-\\[\d\]]+\s*\*\/[\n\t ]*}/)
  splittedFile.shift()
  const matchingComments = value.match(/(?<={[\n\t ]*\/\*\s*c:)[\w.\-\\[\d\]]+(?=\s*\*\/[\n\t ]*})/g)

  if(splittedFile.length) {
    Promise.all(splittedFile.map(async (v, i) => {
      const file = await process({v, path: this.resourcePath})
      return file
    })).then((arr) => {
        const objElements = arr.map((file, i) => {
          return `"${matchingComments[i]}": function() {
            ${file.value}
          },`
        })
        const result = `
          const res = {
            ${objElements.join("")}
          };
          export default res;
        `
        callback(null, result, arr[0].map)
        return result
    }, callback)
  } else {
    process({value, path: this.resourcePath}).then((file) => {
      const result = `
        export default function() {
          ${file.value}
        }
      `
      callback(null, result, file.map)
      return result
    }, callback)
  }
}

This code can probably be written in a more elegant manner...

Looking forward to hearing what you think!
I will in the meanwhile work on a reproducible example 😃

[wooorm]

Using comments is a smart idea. Regexing files and injecting JS-like things as strings seems like a fragile way to go about it though IMO.
It sounds like you have a bunch of questions, that’ll turn into a whole conversation, so I think it’s better to discuss in a different Discussion?
Here are my initial ideas on how I’d go about this though:

• a plugin to turn your comment syntax into the equivalent of someone writing:

  <Section id="how-it-works.main">
    Some MDX here
    ...
  </Section>
  ...
  

  I recommend splitting this, because some people might prefer using thematic breaks instead of comments, or maybe writing those sections manually?
• a plugin that takes sections, and generates multiple components, that are exported.
  This requires most of the discussion likely, because this project is designed to generate one component for the whole document currently

If you want to continue discussing this, can you open a new discussion?

[benjaminpreiss]

Okey, perfect - here is the new discussion.

For my purpose, sadly the <Section> tags are not sufficient, because I want to populate a global "content" object with different keys containing different sections of my page.

For that I am already using an adapted mdx loader, as you can find here and here.
The benefit of that approach is that I can use different sections of one mdx file in different places. That drastically improves the content editing experience.

[levino]

So is there some way to do this? I am working on a project where there are many files with invididual events. The events have meta data like beginDate and endDate, which I put in the frontmatter and then some text, which is markdown. Would be great to be able to put a bunch of these events into one file, instead of having to create a file for each individually...

[benjaminpreiss]

@levino Yes, there is... I implemented a plugin that does so already. https://github.com/frontline-hq/recma-sections