Avoid flakiness in template tests around URL validity checking #59195
+11
−4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dotnet-issue-labeler
bot
added
the
area-mvc
javiercn
reviewed
Nov 27, 2024
| @@ -171,12 +171,19 @@ public async Task ContainsLinks(Page page) | |||
| else | |||
| { | |||
| Assert.True(string.Equals(anchor.Href, expectedLink), $"Expected next link to be {expectedLink} but it was {anchor.Href}."); | |||
| var result = await RetryHelper.RetryRequest(async () => | |||
|
|
|||
| if (!string.Equals(anchor.Protocol, "https:")) | |||
There was a problem hiding this comment.
For completeness here, can we check also that the host is not localhost or 127.0.0.1? Just in case we emit any absolute link in the templates.
There was a problem hiding this comment.
Makes sense - I've added a check for localhost.
I haven't added one for 127.0.0.1 as it would be extraordinary and certainly wrong for someone to add such a link to the templates. The tests explicitly list all the URLs we expect to see, so we'd definitely know about it if someone added a link like that.
There was a problem hiding this comment.
@SteveSandersonMS that's fair. I was thinking more in terms of absolute links generated automatically, not something written up by hand.
javiercn
approved these changes
Nov 27, 2024
SteveSandersonMS
deleted the
stevesa/fix-template-test-flaky-link-checks
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've had several failures in
Templates.Mvc.Test.MvcTemplateTest.MvcTemplate_IndividualAuthrecently with the error:Example build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=880536
This is purely a flaky test issue and not a product bug. That link is actually valid. We just have some issue validating that when running in CI sometimes. I propose to fix this by changing the test logic not to make a real HTTP request to check the link is valid if it's an external (absolute) link.
There's no reason for us to be checking that external links are valid in CI. We just need to check that the template produces the links we expect (which it already does anyway). Any reliance on making requests to external sites during CI are possible sources of flakiness.