Done, for now

man.Rmd

@@ -753,7 +753,7 @@ For the initial CRAN submission of your package, all functions must have at leas
 If the code can only be run under specific conditions, use the techniques below to express those pre-conditions.
 :::
 
-### Dependencies and conditional execution
+### Dependencies and conditional execution {#sec-man-examples-dependencies-conditional-execution}
 
 An additional source of errors in examples is the use of external dependencies: you can only use packages in your examples that your package formally depends on (i.e. that appear in `Imports` or `Suggests`).
 Furthermore, example code is run in the user's environment, not the package environment, so you'll have to either explicitly attach the dependency with `library()` or refer to each function with `::`.

structure.Rmd

@@ -59,7 +59,7 @@ Examples:
 Note that exploring a package's source and history within the `cran` GitHub organisation is not the same as exploring the package's true development venue, because this source and its evolution is just reverse-engineered from the package's CRAN releases.
 This presents a redacted view of the package and its history, but, by definition, it includes everything that is essential.
 
-## Bundled package
+## Bundled package {#sec-bundled-package}
 
 A **bundled** package is a package that's been compressed into a single file.
 By convention (from Linux), package bundles in R use the extension `.tar.gz` and are sometimes referred to as "source tarballs".

vignettes.Rmd

@@ -39,7 +39,7 @@ In vignettes, more than anywhere else, the answer to "But how do I do X?" is oft
 
 [^vignettes-3]: The `inst/doc` folder is where vignettes go once they're built, when `R CMD build` makes the package bundle.
 
-## Vignette workflow {#vignette-workflow}
+## Workflow for writing a vignette {#vignette-workflow-writing}
 
 To create your first vignette, run:
 
@@ -58,7 +58,6 @@ This does the following:
 4.  Adds some patterns to `.gitignore` to ensure that files created as a side effect of previewing your vignettes are kept out of source control (we'll say more about this later).
 
 This draft document has been designed to remind you of the important parts of an R Markdown vignette.
-It serves as a useful reference when you're creating a new vignette.
 You also call `use_vignette()` to create your second and all subsequent vignettes; it will just skip any setup that's already been done.
 
 Once you have the draft vignette, the workflow is straightforward:
@@ -81,7 +80,8 @@ Once you have the draft vignette, the workflow is straightforward:
 If you're regularly checking your entire package (@sec-r-cmd-check), which we strongly recommend, this will help to keep your vignettes in good working order.
 In particular, this will alert you if a vignette makes use of a package that's not a formal dependency.
 
-We're deliberately leaving one workflow issue to revisit below in SOME LATER SECTION. That's where we'll clarify the official mechanism for creating the rendered vignettes that appear in your package's bundled and installed form.
+We're leaving some package-level workflow issues to revisit below in @sec-vignettes-how-built-checkedSOME.
+That's where we'll clarify the official mechanism for creating the rendered vignettes that appear in your package's bundled and installed form.
 
 ## Metadata {#vignette-metadata}
 
@@ -110,22 +110,22 @@ The default vignette template uses these fields:
     They should be the same, but unfortunately that's not automatic.
 
 -   `output`: this specifies the output format.
-    There are many options that are useful for regular reports (including html, pdf, slideshows, etc.), but `rmarkdown::html_vignette` has been specifically designed to work well inside packages.
+    There are many options that are useful for regular reports (including html, pdf, slideshows, etc.), but `rmarkdown::html_vignette` has been specifically designed for this exact purpose.
     See `?rmarkdown::html_vignette` for more details.
 
 -   `vignette`: this is a block of special metadata needed by R.
     Here, you can see the legacy of LaTeX vignettes: the metadata looks like LaTeX comments.
     The only entry you might need to modify is the `\VignetteIndexEntry{}`.
     This is how the vignette appears in the vignette index and it should match the `title`.
     Leave the other two lines alone.
-    They tell R to use `knitr` to process the file, and that the file is encoded in UTF-8 (the only encoding you should ever use for a vignette).
+    They tell R to use `knitr` to process the file and that the file is encoded in UTF-8 (the only encoding you should ever use for a vignette).
 
 We generally don't use these fields, but you will see them in other packages:
 
 -   `author`: we don't use this unless the vignette is written by someone not already credited as a package author.
 
 -   `date`: we think this usually does more harm than good, since it's not clear what the `date` is meant to convey.
-    Is it the last time the vignette was updated?
+    Is it the last time the vignette source was updated?
     In that case you'll have to manage it manually and it's easy to forget to update it.
     If you manage `date` programmatically with `Sys.date()`, the date reflects when the vignette was built, i.e. when the package bundle was created, which has nothing to do with when the vignette or package was last modified.
     We've decided it's best to omit the `date`.
@@ -194,7 +194,8 @@ You might need to take explicit measures, such as lowering the resolution, reduc
 ### Organisation
 
 For simpler packages, one vignette is often sufficient.
-Call it `pkgname.Rmd`; that takes advantage of a pkgdown convention, where the vignette that's named after the package gets an automatic "Getting Started" link in the top navigation bar.
+If your package is named "somepackage", call this vignette `somepackage.Rmd`.
+This takes advantage of a pkgdown convention, where the vignette that's named after the package gets an automatic "Getting Started" link in the top navigation bar.
 
 More complicated packages probably need more than one vignette.
 It can be helpful to think of vignettes like chapters of a book -- they should be self-contained, but still link together into a cohesive whole.
@@ -236,6 +237,9 @@ The `eval` option can be set for an individual chunk, but in a vignette it's lik
 In the latter case, you'll want to use `knitr::opts_chunk$set(eval = FALSE)` in an early, hidden chunk to make `eval = FALSE` the default for the remainder of the vignette.
 You can still override with `eval = TRUE` in individual chunks.
 
+In vignettes, we use the `eval` option in a similar way as `@examplesIf` in examples (@sec-man-examples-dependencies-conditional-execution).
+If the code can only be run under specific conditions, you must find a way to to check for those pre-conditions programmatically at runtime and use the result to set the `eval` option.
+
 Here are the first few chunks in a vignette from googlesheets4, which wraps the Google Sheets API. The vignette code can only be run if we are able to decrypt a token that allows us to authenticate with the API.
 That fact is recorded in `can_decrypt`, which is then set as the vignette-wide default for `eval`.
 
@@ -276,31 +280,95 @@ Many other options are described at <https://yihui.name/knitr/options>.
 
 There is one last technique, if you don't want any of your code to execute on CRAN.
 Instead of a vignette, you can create an article, which is a term used by pkgdown for an `.Rmd` document that is not embedded in the package, but that appears in the website.
-An article will be less accessible than a vignette, for certain users, such as those with limited internet access.
+An article will be less accessible than a vignette, for certain users, such as those with limited internet access, because it is not present in the local installation.
 But that might be an acceptable compromise, for example, for a package that wraps a web API.
 You can draft a new article with `usethis::use_article()`, which ensures the article will be `.Rbuildignore`d.
 A great reason to use an article instead of a vignette is to show your package working in concert with other packages that you don't want to depend on formally.
 
-## Filesystem matters
+## How vignettes are built and checked {#sec-vignettes-how-built-checked}
+
+We close this chapter by returning to a few workflow issues we didn't cover in @vignette-workflow-writing: How do the `.Rmd` files get turned into the vignettes consumed by users of an installed package?
+What does `R CMD check` do with vignettes?
+What are the implications for maintaining your vignettes?
+
+First, it's important to realize that the `vignettes/*.Rmd` source files exist only in a source package (@sec-source-package).
+Vignettes are rendered when a source package is converted to a bundle (@sec-bundled-package) via `R CMD build` or convenience wrappers such as `devtools::build()` or `pkgbuild::build()`.
+The rendered products (and more) are placed in `inst/doc/`[^vignettes-4] and the `vignettes/` directory disappears.
+Finally, when a package binary (@sec-structure-binary) is made, the `inst/doc/` directory is promoted to a top-level `doc/` directory, as happens with everything below `inst/`.
+
+[^vignettes-4]: We highly recommend treating `inst/doc/` as a strictly machine-writable directory for vignettes.
+    We recommend that you do not take advantage of the fact that you can place arbitrary pre-built documentation in `inst/doc/`.
+    This opinion permeates the devtools ecosystem which, by default, cleans out `inst/doc/` during various development tasks, to combat the very real problem of having stale documentation.
+
+    Note that you can put `Config/build/clean-inst-doc: FALSE` in `DESCRIPTION` to prevent rcmdcheck from cleaning `inst/doc/` and something similar may be implemented for pkgbuild (https://github.com/r-lib/pkgbuild/issues/128).
+    `pkgbuild::build(clean_doc =)` also affords control over this.
+
+```{=html}
+<!--
+The issue of keeping pre-built vignettes around in inst/doc/ is a long-running discussion.
+
+https://github.com/r-lib/devtools/issues/2394
+
+https://github.com/r-lib/pkgbuild/issues/58
+
+The one motivation that I feel isn't covered below is for a vignette that takes a very long time or very special conditions to build. Caching is one option, though likely to be fussy. Otherwise, this is really the "pre-built" vignette topic, which I think is not common enough to cover here.
+-->
+```
+The key takeaway from the above is that it's not very natural to keep rendered vignettes in a source package and this has implications for the vignette development workflow.
+It is tempting to fight this (and many have tried), but based on years of experience and discussion, the devtools philosophy is to accept this reality.
+
+Assuming that you don't try to keep built vignettes around persistently in your source package, here are our recommendations for various scenarios:
+
+-   Active, iterative work on your vignettes: Use your usual interactive `.Rmd` workflow (such as the ![](images/knit.png){width="45"} button) or `devtools::build_rmd("vignettes/my-vignette.Rmd")` to render a vignette to `.html` in the `vignettes/` directory.
+    Regard the `.html` as a disposable preview.
+    (If you initiate vignettes with `use_vignette()`, this `.html` will already be gitignored.)
+
+-   Make the current state of vignettes in a development version available to the world:
+
+    -   Offer a pkgdown website, preferably with automated "build and deploy", such as using GitHub Actions to deploy to GitHub Pages.
+        Here are tidyr's vignettes in the development version (note the "dev" in the URL): <https://tidyr.tidyverse.org/dev/articles/index.html>.
 
-`.Rmd` source located below `vignettes/`
+    -   Be aware that anyone who installs directly from GitHub will need to explicitly request vignettes, e.g. with `devtools::install_github(dependencies = TRUE, build_vignettes = TRUE)`.
 
-> Package vignettes are tested by `R CMD check` by executing all R code chunks they contain (except those marked for non-evaluation, e.g., with option `eval=FALSE` for Sweave).
-> The R working directory for all vignette tests in `R CMD check` is a *copy* of the vignette source directory.
-> Make sure all files needed to run the R code in the vignette (data sets, ...) are accessible by either placing them in the `inst/doc` hierarchy of the source package or by using calls to `system.file()`.
-> All other files needed to re-make the vignettes (such as LaTeX style files, BibTeX input files and files for any figures not created by running the code in the vignette) must be in the vignette source directory.
-> `R CMD check` will check that vignette production has succeeded by comparing modification times of output files in `inst/doc` with the source in `vignettes`.
+-   Make the current state of vignettes in a development version available locally:
 
-Previewing vignettes
+    -   Install your package locally and request that vignettes be built and installed, e.g. with `devtools::install(dependencies = TRUE, build_vignettes = TRUE)`.
 
-Let go of the idea that you're going to have a semi-persistent rendered version of your vignettes in your source package.
+-   Prepare built vignettes for a CRAN submission: Don't try to do this by hand or in advance.
+    Allow vignette (re-)building to happen as part of `devtools::submit_cran()` or `devtools::release()`, both of which build the package.
 
-Get comfortable with the idea that, to the extent you need to preview the built product, you'll mostly be looking at something ephemeral and local.
-And once things mature enough to push to GitHub, you will be able to see the rendered result in your pkgdown website.
+If you really do want to build vignettes in the official manner on an *ad hoc* basis, `devtools::build_vignettes()` will do this.
+But we've seen this lead to developer frustration, because it leaves the package in an peculiar form that is a mishmash of a source package and an unpacked package bundle.
+This nonstandard situation can then lead to even more confusion.
+Alternatively, it's possible to use `devtools::build(vignettes = TRUE)` to create a package bundle and then look inside for the vignettes.
 
-1.  For Git/GitHub users: commit and push the vignette source, i.e. `vignettes/my-vignette.Rmd`. (If you initiate your vignette use with `use_vignette()`, it will do some very helpful `.gitignore` setup for you.)
+We conclude with a discussion of how vignettes are treated by `R CMD check`.
+This official checker expects a package bundle created by `R CMD build`, as described above.
+In the devtools workflow, we're usually doing this via `devtools::check()` which automatically does this build step for us, then checks the package.
+`R CMD check` has various options and environment variables, so we're taking a maximalist approach, i.e. describing all the checks that *could* happen.
 
-### Difference with function documentation workflow
+R `CMD check` does some static analysis of vignette code and some file existence checks.
+If your vignettes use packages that don't appear in `DESCRIPTION`, that is caught here.
+If files that should exist don't exist or *vice versa*, that is caught here.
+This should not happen if you use the standard vignette workflow outlined in this chapter and is usually the result of some experiment that you've done, intentional or not.
+
+The vignette code is then extracted into a `.R` file, using the "tangle" feature of the relevant vignette engine (knitr, in our case), and run.
+The code originating from chunks marked as `eval = FALSE` will be commented out (so: not actually executed).
+Then the vignettes are rebuilt from source, using the "weave" feature of the vignette engine (knitr, for us).
+This executes all the vignette code yet again, except for chunks marked `eval = FALSE`.
+
+::: callout-warning
+## Submitting to CRAN
+
+CRAN's incoming and ongoing checks use `R CMD check` which, as described above, exercises vignette code up to two times.
+Therefore, it is important to conditionally suppress the execution of code that is doomed to fail on CRAN.
+
+However, it's important to note that the package bundle and binaries distributed by CRAN actually use the built vignettes included in your submission.
+:::
+
+```{=html}
+<!--
+## Difference with function documentation workflow
 
 It can be helpful to appreciate one big difference between the workflow for function documentation and vignettes:
 
@@ -310,13 +378,6 @@ These `man/*.Rd` files are part of the source package.
 
 The source of your vignettes lives below `vignettes/`.
 We use various interactive techniques to preview rendered vignettes, usually as `.html`, but these ephemeral preview files are typically *not* considered part of the source package.
-
 Vignettes are converted into their "consumable" form by `R CMD build`, i.e. when the package bundle is made.
-
-`R CMD build` will put rendered artefacts below `inst/doc`
-
-> Package vignettes have their sources in subdirectory `vignettes` of the package sources.
-> Note that the location of the vignette sources only affects `R CMD build` and `R CMD check`: the tarball built by `R CMD build` includes in `inst/doc` the components intended to be installed.
-
-> `R CMD build` will automatically create the (PDF or HTML versions of the) vignettes in inst/doc for distribution with the package sources.
-> By including the vignette outputs in the package sources it is not necessary that these can be re-built at install time, i.e., the package author can use private R packages, screen snapshots and LaTeX extensions which are only available on their machine.
+-->
+```