Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate test notebooks on the fly rather than using actual, stored notebooks #150

Open
ceball opened this issue May 27, 2020 · 3 comments
Labels
help wanted Good issue for external contributors part: tests PRs or issues related to tests

Comments

@ceball
Copy link
Collaborator

ceball commented May 27, 2020

Currently, nbcelltests' tests use various .ipynb files stored in nbcelltests/tests. Instead, we should generate notebooks as needed during the tests.

Doing this will allow various bits of test code to be simplified.

@ceball ceball added tests help wanted Good issue for external contributors labels May 27, 2020
@michiboo
Copy link

Hi I am interested doing this issue, but can we use other library to generate test notebooks or it is better to code it from scratch to generate jupyter notebook?

@ceball
Copy link
Collaborator Author

ceball commented May 28, 2020

Yes @michiboo thanks for checking - definitely better to use a library if that's possible.

I was imagining using something like nbformat's new_notebook(). E.g. here it is being used in nbval: https://github.com/computationalmodelling/nbval/blob/3597e0bfd96c8b52d9f4e94abbe1cbb3887dcf25/tests/utils.py#L5

I think I would probably start off by replacing the notebooks associated with https://github.com/jpmorganchase/nbcelltests/blob/master/nbcelltests/tests/test_lint.py and see how that works out.

Four notes I can think of right now:

  1. I'd like to leave a least one actual notebook file around, associated with a test that nbcelltests does actually work with a real file :)
  2. Just to warn you: a part of this task might be improving the tests themselves. There is probably some overlap between unit tests, partially because some re-use the same underlying notebook files. Ideally improving the tests would be a separate task (i.e. should be possible to do a straight up "generate the notebooks" replacement for "physical notebooks" only, but also possible that something blocks that).
  3. Testing tools that are themselves test tools can be tricky, so please feel free to ask if you see something that looks weird - it might be just bad code, or it might be necessary for some awkward reason.
  4. This task will leave you with a negative "line count contribution" to the project. That's the second best kind of contribution - second only to preventing lines being written in the first place :)

@sakshamk6999
Copy link

Hi, I'd like to contribute on this issue, I am currently looking at the codebase and understand the working

@timkpaine timkpaine added part: tests PRs or issues related to tests and removed tests labels Jun 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Good issue for external contributors part: tests PRs or issues related to tests
Projects
None yet
Development

No branches or pull requests

4 participants