[improvement] better description handling, and theming idea

[trusktr]

A few issues:

1 Ignore - symbol between type and description

I have several @param tags. The output in my template is like this:

The first has no description, the second has only a - for the description, and the third has a fuller description as in - Our UserService injectable.

It'd be nice if they were more consistent. What I believe is happening (haven't looked at dox code in a while, so just guessing) is that this is a result of the markdown. The first tag has no description, so results in an empty string. The second has only - for the description, so the markdown generates a <p>-</p>, and the last one has a full - description which is treated as a bullet point by markdown, and hence a <ul>.

JSDoc treats the - between the type and the description as optional, so maybe we should treat it like that too. Idea: the first - that is encountered can be ignored, so if a user really really wants a bullet point, they can write

@param {Object} foo - - this is a bullet point

Although this makes the comment ugly, it will mean that writing

@param {Object} foo - this is not a bullet point

is treated as per JSDoc and - is ignored, but also that making a bullet point is more explicit. Suppose we want two paragraphs:

  /*
   * @param {Object} foo - This is paragraph one which is really long and it is
   * split so that it is more readble and so that the line isn't so long.
   *
   * This is paragraph two. This paragraph and the previous are not bullet
   * points.
   */

The result is unexpectedly one bullet and one paragraph:

but it should just be two paragraphs. Now, if we actually try to make bullet points (with the current non-ignoring of -)

  /*
   * @param {Object} foo - bullet one
   * - bullet two
   */

we get the result

If we write

/**
 * @param {Object} foo
 * - bullet one
 * - bullet two
 */

then it works.

yaaaay! \o/

2 line breaks

Seems dox isn't generating line breaks properly? Suppose I have

/**
 * @param {Object} foo - This is a really long description that I'd like to split into multiple lines so that I can read it better and this line won't be so long.
 */

which is valid JSDoc, now let's split it:

/**
 * @param {Object} foo - This is a really long description that I'd like to
 * split into multiple lines so that I can read it better and this line won't
 * be so long.
 */

dox (or the markdown) places a <br> right before be so long which results in one long wrapped line and a short line:

It should instead probably not have any <br> tags, and the whole thing should just wrap. (also note it's a bullet point). So, how can we make it work? Maybe if I put it on a new line:

/**
 * @param {Object} foo
 * - This is a really long description that I'd like to
 * split into multiple lines so that I can read it better and this line won't
 * be so long.
 */

Aha, so maybe dox is concatenating the @param line with the second line, but this time the first line with @param is empty description-wise, so now it splits on my line breaks.

Would we have to marked to fix this? Or maybe there's an option? Or maybe we can switch to github-flavored markdown, which already handles paragraphs by combining them together. For example, here on GitHub we can write paragraphs with arbitrary line breaks, and paragraphs are delimited by double line breaks

this is
a short
paragraph

results in:

this is
a short
paragraph

But with dox (marked)

/**
 * @param {Object} foo
 * this is
 * a short
 * paragraph
 */

TLDR

We can improve the markdown handling. Perhaps an easy way to do it without swapping the markdown library is to just ignore the first -, then remove single line breaks before passing to marked.

EDIT: Oh, upon further investigation, looks like marked isn't generating new paragraphs. If I have

/**
 * @param {Object} foo one paragraph sentence.
 *
 * another paragraph sentence
 */

then I get this markup:

So, maybe it'd be better to just switch to github flavored markdown, where for

one paragraph
with two lines

another paragraph
with three
lines.

we get

[trusktr]

Hmm, question is, where's the most recent code for GitHub flavored markdown? Is it https://github.com/isaacs/github-flavored-markdown? Interestingly, it recommends using marked. Maybe there's another implementation whose result is closer to GitHub's? Or maybe we can fork (and PR) marked.

[trusktr]

Hmmm, well, apparently github comments are handled differently than markdown files. For example, for the above

this is
a short
paragraph

the result here is

this is
a short
paragraph

but compare to the result in this gist.

I like the treatment in that gist, but not in these comments (the gist doesn't preserve the line breaks, but these comments do).

[Twipped]

This all just further re-enforces my opinion that I should yank automatic markdown processing out of dox entirely for a 1.0 release and let implementers worry about it. It keeps producing all kinds of headaches, and is just bloating the concerns of the library.

Could you provide the JSON output from these cases that you're having problems with? I need to see exactly what dox is producing from those docblocks.

[trusktr]

@ChiperSoft Let me do so when I get a chance. Besides the difference in HTML output, I also noticed that the properties that exist for a given tag differ depending on whether the description exists or not, which is confusing. When description exists, an html property doesn't exist, and the HTML is instead inside the description property, but when a description doesn't exist then the html property exists, among other small difference, which lead to needing funky template rules in order to catch those cases.

I don't think having markdown is a bad idea, but I think perhaps the handling there should be treated with some care so that the resulting data always has the same structure (therefore much easier to handle inside templates).

I also stumbled on http://documentation.js.org yesterday, which is for the first time a contender to dox, as it also generates pure JSON output (aside from HTML and Markdown outputs). Maybe we can look there and get ideas for dox's JSON output.

[trusktr]

Check out this cool feature of documentation.js: https://github.com/documentationjs/documentation/blob/master/docs/THEMING.md

That might be something cool to add to dox, which would make dox pluggable (people add themes in the community, while the dox usage doesn't change except for passing different theme names to the CLI). f.e.

dox ... --theme some-theme

where dox-some-theme is a module from NPM.

[Twipped]

documentation.js looks pretty good, this might actually be a proper successor to dox. It looks like they're performing proper static analysis of the code, rather than string parsing. @tj had said he'd rather be doing this with ast, but esprima wasn't up to it.

They look to already be in a much better position than dox is...