Use in JupyterLab currently?

[joelostblom]

It's really great to see all this progress on a RST docstring converter, thank you for working on it! Is there any way to use this currently for rendering RST docstrings as rich text in jupyterlab's help popups? Or is that something that is planned for the future? I believe I remember seeing you merging in some jupyterlab code previously to allow for extensions to render rich text docstrings.

[krassowski]

If we want the markdown to be propagated to frontend for rendering (and sanitization), we need to make changes in IPython and Lab:

1. append_field should pass other MIME types down, currently it ignores anything other than text/plain and text/html
2. code formater should probably return code wrapped in markdown triple backticks
3. docstring field is not appended (using append_field) for non-class objects (functions etc) when detail_level > 0 and source is available (instead source is appended). This makes the inspect response a bit useless - 1000 LoC for pandas.DataFrame does not mean anything to me, but a docstring means everything. This may need some discussion in IPython but I think some sanity check of "is this source huge, do we really want to send it" and convincing that "higher detail level probably means we want to send more not less" would do the job.
4. we would need a way to hook our custom formatter in a clean manner from an external package OR get it into IPython core. There are hooks in IPython so maybe could use that.
5. finally we need to ensure that mimetype can be chosen on JupyterLab user (probably in settings).

This sounds pretty much doable in few days of work but also cuts across the stack so help would be needed. Personally, I don't have bandwidth for this and don't use inspector that much (maybe it would change if we implement this).

At some fundamental level I had some pushback in IPython against using markdown as markup format of choice, but I believe the changes above would not be opposed and I would vouch for them.

Some experimental code for testing via monkey-patching:

from IPython import get_ipython
import docstring_to_markdown


ipython = get_ipython()


def markdown_formatter(text):
    try:
        converted = docstring_to_markdown.convert(text)
        return {
            # TODO: adding markdown requires changing `append_field`
            # 'text/markdown': f'{converted}',
            'text/plain': f'{converted}',
            'text/html': f'{converted}'
        }
    except docstring_to_markdown.UnknownFormatError:
        return {
            'text/plain': text,
        }


def our_object_inspect_mime(oname, detail_level=0, omit_sections=()):
    self = ipython

    with self.builtin_trap:
        info = self._object_find(oname)
        if info.found:
            return self.inspector._get_info(
                info.obj,
                oname,
                info=info,
                # Force detail level to zero because otherwise docstring is not shown for most Python objects
                # but source is: TODO This probably needs changing behavour in IPython
                detail_level=0,
                formatter=markdown_formatter,
                omit_sections=omit_sections,
            )
        else:
            raise KeyError(oname)


ipython.object_inspect_mime = our_object_inspect_mime

[krassowski]

Cross-referencing ipython/ipython#3687 and ipython/ipython#13755

[joelostblom]

Thank you so much for taking the time to write this detailed answer! It really helps in understanding what needs to be done to get this working, although I unfortunately don't have bandwidth to take this on either. One effort that is related in terms of the nice rendering aspect is docrepr which you might have seen already. There is a short intro at the end of this article (but there are some issues as well, such as spyder-ide/docrepr#55), maybe there is something in that project that would be helpful here too?

Personally, I don't have bandwidth for this and don't use inspector that much (maybe it would change if we implement this).

Out of curiosity, what do you rely on when you are working in JupyterLab and need to read up on a parameter description? Maybe I am missing a more efficient workflow.

[krassowski]

Ii knew about spinxify but I haven't seen the blog post, it looks cool! Are there any limitations of docrepr approach that would be solved by passing markdown instead? I guess rendering/theming could be even more consistent and mathajx equations would render properly?

[joelostblom]

On a general level I am not sure if there is an advantage of converting rst to markdown versus directly to HTML. As long as the converter accurately covers the full functionality including equations etc. Personally, I have had issues getting the docrepr approach to work reliably in JupyterLab and ideally it would be great if there was a simpler way to render docstrings, maybe even integrated into JupyterLab. I thought they might be more receptive to rich rendering of markdown versus rst, as rst is very python specific.

[krassowski]

Recently introduced Inspector.mime_hooks should make it much easier (ipython/ipython#14300)

[krassowski]

Indeed:

[joelostblom]

Beautiful! That looks really great and is much easier to read!

[krassowski]

@joelostblom this is now available with: https://github.com/krassowski/ipython-markdown-inspector - can you confirm that it works for you?

[krassowski]

Closing as https://github.com/ipython-contrib/ipython-markdown-inspector 1.0.0 was released.

[joelostblom]

Sorry for the late reply, I was away without internet. This is super cool! Thanks for making that package available, I just tried it out and it works like a charm! This makes the docs much easier to read and I am wondering if there is a way to make it available in all my conda environments except manually installing it in each one? For example, for some other jupyterlab extensions (e.g. skip-traceback which is another favorite of mine), I just need to install it once in my jupyterlab env and it is then available in every conda environment. I'm guessing this is possible since it doesn't require a python kernel whereas ipython-markdown-inspector probably requires a kernel and therefore maybe needs to be installed in each env separately?