Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Automatic Running of Tests On Pull Request #1111

Open
CarsonDavis opened this issue Nov 26, 2024 · 1 comment · May be fixed by #1123
Open

Add Automatic Running of Tests On Pull Request #1111

CarsonDavis opened this issue Nov 26, 2024 · 1 comment · May be fixed by #1123
Assignees
@saifrk

Comments

@CarsonDavis
Copy link
Collaborator

Description

Every time there is a pull request, the full test suite should execute.

Implementation Considerations

At a minimum, these tests should run:

docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_delta_patterns.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_import_fulltexts.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_exclude_patterns.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_include_patterns.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_field_modifier_patterns.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_promote_collection.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_migrate_dump.py
docker-compose -f local.yml run --rm django pytest sde_collections/tests/test_pattern_specificity.py

Deliverable

ars

Dependencies

depends on
@saifrk saifrk linked a pull request Dec 5, 2024 that will close this issue
Open
saifrk pushed a commit that referenced this issue Dec 5, 2024
Fixes #1111
e9f4bdb
@saifrk saifrk mentioned this issue Dec 5, 2024
Closed
@saifrk
Copy link
Collaborator

saifrk commented Dec 10, 2024 •
edited
Loading

  • A centralized logic has been implemented to streamline test execution across all workflows.
  • Continuous Execution:
    • Even if a test fails, the check doesn’t fail immediately.
    • It continues running other tests and, at the end, reports the total number of failed tests while marking the check as failed.
  • Naming Convention:
    • All tests beginning with tests_* are executed.
    • To add new tests, simply adhere to this naming convention.

Workflow Trigger

  • Workflow: Django Test Suite on PR
  • Trigger: Automatically triggered whenever a pull request (PR) is created or updated for the dev branch.

Workflow Steps

  1. Checkout Merged Code:
    • The workflow checks out the merged code which combines:
      • The branch being merged .
      • The target branch (dev).
    • Purpose: Tests are executed on the post-merge state to ensure stability.
  2. Set Up Environment:
    • Installs docker-compose.
    • Builds the Docker environment defined in local.yml.
  3. Run Tests:
    • Executes the Django test suite which runs all test cases defined in the merged code.
  4. Cleanup:
    • Cleans up Docker containers and volumes to free resources after tests finish.

Test Execution

1.Test Scope:

  • Executes all pytest tests defined in the repository.
  • Includes all files matching the naming convention tests_*.
    2.Fail-Safe Mechanism:
  • Even if a test fails, all tests are executed.
  • At the end, it reports the number of failed tests and marks the workflow as failed if any test fails.
    3.Feedback:
  • Results appear directly in the pull request as:
    • ✅ All tests pass.
    • ❌ Test failures are displayed with logs for debugging.

Key Change:

  • Previously, tests were executed on the branch being pushed, Integration issues with the target branch (e.g., dev) were not accounted for.
  • Now, Tests are executed on the merged code, this ensures the merged state is stable and prevents surprises post-merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@saifrk saifrk

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Automatic Running of Tests On Pull Request NASA-IMPACT/COSMOS
2 participants
@CarsonDavis @saifrk