Harmonise devtools with testthat and cmdcheck

[Pisca46]

devtools::test() report failures not reported with rcmdcheck(), the “Run Test” button in RSudio or e.g. testthat::test_dir(). Also, my package under test seems to work well.

I am testing a package with data in an environment. The initial (‘factory’)data is created when the package is build and it may be manipulated later by its users.

A minimal package producing the issue is shown below. It also contains two work-arounds:
1. Include “testEnvir <- testTest:::testEnvir” in setMenu()
But this would produce notes that the “::::” are rarely needed

2. Remove the ‘detach’ function in the test file.
But I want the guarantee to test always on the same data

My package, called testTest, contains only the following one R file:

testEnvir <- new.env()
testEnvir$menu <- "spam"

setMenu <- function (menu) {
    # testEnvir <- testTest:::testEnvir
    testEnvir$menu <- menu
}

And the following test file:

require (testthat)
test_that("-", { expect_true(TRUE) })  # used to enforce the RStudio top right "Run Test" button

if (isNamespaceLoaded ("testTest")) detach (package:testTest, unload = TRUE)  # to get a fresh version
require (testTest)
testEnvir <- testTest:::testEnvir

test_that ("testTestTest", {
    expect_equal (testEnvir$menu, "spam")
    testEnvir$menu <- "eggs"
    expect_equal (testEnvir$menu, "eggs")

    setMenu ("bacon")
    expect_equal (testEnvir$menu, "bacon")   # fails with devtools::test() only
})

[jennybc]

devtools::test() report failures not reported with rcmdcheck(), the “Run Test” button in RSudio or e.g. testthat::test_dir().

This is generally a sign that something is wonky with your package and its test suite, I'm afraid.

Based on the above it sounds like you're not using testthat in the way it's meant to be used, so that is where I would start troubleshooting.

This is very unconventional usage:

And the following test file:

require (testthat)
test_that("-", { expect_true(TRUE) }) # used to enforce the RStudio top right "Run Test" button

if (isNamespaceLoaded ("testTest")) detach (package:testTest, unload = TRUE) # to get a fresh version
require (testTest)
testEnvir <- testTest:::testEnvir

test_that ("testTestTest", {
expect_equal (testEnvir$menu, "spam")
testEnvir$menu <- "eggs"
expect_equal (testEnvir$menu, "eggs")

setMenu ("bacon")
expect_equal (testEnvir$menu, "bacon")   # fails with devtools::test() only
})

[Pisca46]

Thank you for your helpful reply. As suggested I did some troubleshooting and arrived at the following conclusions:

1.	I inspected and tested the package under discussion, i.e. the testTest package,  and it behaved as required.

2.	Also, I took a closer look to the test-file.

        The lines that detaches, unload and re-require the testTest package may be the culprit.



It seems that the application continues with the new re-required environment and that devtools continues with the old one. As a consequence the value of menu became different.

For your convenience, I uploaded the testTest package to gitHub and added a vignette with details results and a remedy. See https://github.com/Pisca46/testTest