Last week I triaged a rather ordinary support request: after entering an MFA token, a customer’s login was rejected with the message Invalid username/password.
Logs confirmed the failed attempts, yet also successful logins. Follow-up questions revealed the customer could log in on desktop and web — just not on mobile.
Suppressing my instinct to assume a mistyped password, I started digging into logs and trying to reproduce the symptom for myself in case it was a genuine regression.
My interaction with Claude (Opus 5.5) eventually went something like this:
Check the mobile app code paths to verify that MFA won’t get shown if password is wrong.
I thought perhaps the user hadn’t gotten to the MFA screen at all, and Claude’s first pass echoed my bias, concluding that the error message must have been shown after entering the password, and the MFA part of the report is probably a mix-up.
Re-reading the support ticket and trusting my support team, I decided to delete my own bias, and prompted Claude as much:
Assume the MFA is a valid report: let’s investigate more deeply to find a potential root cause for the logs and the report that MFA is being rejected only on mobile.
A few more tokens and Claude discovered a real regression! After validating the username and password with the server, the mobile app is asked to provide the MFA token. Once collected from the user, the original username and password are sent back to the server alongside the MFA token — but with the password incorrectly escaped as if it were part of a URL.
For example, Fall%2025! would become Fall 25! Of course, the password itself was never transmitted in the URL, but an expo-router upgrade had inadvertently started escaped data being passed between React components.
After some discussion with the team, the fix was straightforward.
The morale learned? LLMs are liable to project my own prompting bias. Thankfully, flipping them is just another prompt away.
Wisdom is the principal thing; therefore get wisdom: and with all thy getting get understanding.