(WIP) Myst parser: .md + {eval-rst} support

[tony]

re: #30

Naïve eval-rst directive support.

• Support .md
• Adds testdata/using_the_shapereader.md
• Tests using_the_shapereader.md in test_cartopy
• Support for {eval-rst} and tests
• _DIRECTIVE_RE improvements - Use verbose, named expression for readability (cherry-picked and rebased in via Use verbose regular expression #33, Use named group regular expression #34)

[tony]

@Thisch I will ping you when this is ready to look at

Have not hand tested yet. I will build a very robust testsuite for it

[twmr]

Awesome! Thx a lot.

[tony]

@Thisch feel free to let tests run

[twmr]

@tony thx a lot for all your PRs. I've just enabled the CI pipeline.

[tony]

@Thisch Thank you!

• Rebased after Use verbose regular expression #33 merged in e3cf48b, chore(.gitignore): Ignore virtualenv directories #32 in 7eb7c6a
• Created Use named group regular expression #34 for named groups
• Removed typing.Literal for python 3.7 support (CI run). This is available for typing-extensions but its probably best to stay lean and keep it to standard library for now

Current PR status: I need to test this by hand, but open to any early feedback

[tony]

@Thisch I suppose if you want you could cut a 0.4.1 release with the latest regex changes (via #33, #34)

I assume this PR, if merged (I still need to test it by hand and QA it more intensively) will be a bigger version bump, e.g. 0.5.0/0.6.0

[twmr]

I'll try to give you more feedback on the weekend.

[twmr]

_DIRECTIVE_RE.match is currently used for every line in a docstring/txt file.

[twmr]

Therefore, this line, doesn't currently add change anything.

If you want that _DIRECTIVE_RE finds (for markdown content) a match for the following input

```{{eval-rst}}
.. docstest::

but not for

```{{xxxxx}}
.. doctest::

we have to extend the code that calls _DIRECTIVE_RE.match

[tony]

I've been having some issues when testing this locally in my project, so this may be why.

I will look into this.

(note to self, the above message refers to how the regex checks every line)

[tony]

@Thisch

Could you take a stab at getting it to work? (using the existing parser)

I am going try a fork of this branch that uses the myst-parser/docutils/etc API.

Here's what you can do:

I added a test to tests/test_sphinx_doctest.py::TestDirectives::test_doctest. It automatically skips if myst-parser is not found

pip install myst-parser or pip install -e .[myst-parser] (bash) or pip install -e ".[myst-parser]" (zsh)

Then py.test

Re-invoke on file rerun:

pip install pytest-watcher

With this test

py.test tests/test_sphinx_doctest.py::TestDirectives::test_doctest ; ptw . tests/test_sphinx_doctest.py::TestDirectives::test_doctest

Project-wide

py.test -x -s -v; ptw . -x -s -v

[tony]

@Thisch I did a lot of research this week, I am going to be digging into the docutils, sphinx, and myst / markdown-it and pytest APIs next weekend. This is priority 0 for me, outside of work

[tony]

@Thisch I may take a fresh stab at this non-docutils PR now that I am familiarized with doctest 👍

This approach, if pulled it off, would be easier to merge, preserve existing features, and have 0 extra dependencies.