I had the same issue and debug and solved it with Microsoft.

Basically needed is openid as the first scope, plus the following directly quoting the Microsoft engineer:

"Another thing that I noticed is that the scopes you are asking for, for the access token, are user.read and directory.read.all.
Without specifying the full appID URI of MS Graph (which is what I am assuming you want to access with the token based on the current consented permissions of the app)
our backend will evaluate these as scopes for the older AAD Graph API instead.
Can you also replace, just for testing purposes, the user.read and directory.read.all scopes with this single one https://graph.microsoft.com/.default
The .default will request all scopes that have been consented to by an admin in your tenant in a single take."

So for us, adding openid and replacing the other scopes with https://graph.microsoft.com/.default solved the mfa issue.

Unfortunately I'm off on holidays now, but will probably come up with a PR after.

Thanks for the response, @schuhu.

I actually came to a similar conclusion with Microsoft Support but have been told by a colleague that the scopes that dex passes when using the Microsoft connector are hardcoded. Do you know if there is a way to pass/edit the scopes in the config without maintaining our own version of dex?

Look forward to the PR though, I will definitely pass it onto our dex team.