(Using version 2.1.19)
To reproduce:
From the Mailgun settings page (with correctly configured settings):
- Set an invalid API key
- Disable fallback method option
- Click "Test Configuration"
The success toast will appear but no emails are logged as accepted in the Mailgun dashboard events log.
A little digging reveals that wp_mail returns true when the initial sending method fails and the fallback method is disabled.