This is why we decided to use the PIN protected SafeCards to store/backup your wallets - if someone finds your backup, they still can't use it without the PIN, which adds another layer of security on top of everything else (but if you want to back up with a seed phrase, you still have the option of course).