Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Pytest-cov, and make coverage more meaningful #533

Open
MicahGale opened this issue Sep 4, 2024 · 3 comments
Open

Implement Pytest-cov, and make coverage more meaningful #533

MicahGale opened this issue Sep 4, 2024 · 3 comments
Labels
CI/CD good first issue Good for newcomers

Comments

@MicahGale
Copy link
Collaborator

This is related to #532. This should:

  1. Move from coverage to pytest-cov for better feature support.
  2. Restrict coverage reports to unit testing based suites, and excluding integration testing.
@MicahGale MicahGale added the CI/CD label Sep 4, 2024
@MicahGale
Copy link
Collaborator Author

For restricting the coverage results I think we can:

  1. organize tests into tests/unit and tests/integration folders. This does still make pytest happy.
  2. Run something like coverage run -m pytest tests/unit followed by pytest tests/integration

@MicahGale
Copy link
Collaborator Author

MicahGale commented Sep 11, 2024

I think the best way to do this is to map a unit test file to a source path:

{"test_cell_problem.py" : `montepy/cell.py` ...
}

Then we run pytest --cov for everything like:

pytest --cov=montepy/cell.py --cov-append tests/unit/test_cell_problem.py

This way code that is ran to setup fixtures will most likely not be erroneously counted.

While at it, branch coverage should be enabled.

Overall this won't be easy to implement effectively and will require a moderate amount of effort to:

  1. Create the mappings
  2. Switch to purely pytest-onic test suites.
  3. Refactor the tests to better have unit tests be consolidated (I'm looking at you test_integration.py)
  4. Write more exhaustive (property based included Implement Property based Testing #534) unit tests
  5. parametrize all tests
  6. extend parametrized tests to be property based tests.
  7. Reexamine edge cases and if they are better suited as unit tests or other testing.
  8. Change paradigm that new bugs are usually a failure of unit testing, and more integration tests are likely not the solution.

@MicahGale
Copy link
Collaborator Author

For restricting the coverage results I think we can:

  1. organize tests into tests/unit and tests/integration folders. This does still make pytest happy.
  2. Run something like coverage run -m pytest tests/unit followed by pytest tests/integration

In testing with #536, this dropped coverage to 89%

@MicahGale MicahGale added the good first issue Good for newcomers label Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant