How Do You Protect Yourself From Developer Penetration?

In my post about wanting a professional hacker (which I now know to be a process called penetration testing), @Dave pointed out that often the issue isn’t about your site being targetted, but your own storage of information about the site - passwords, auth keys, .pem files etc.

So I’m curious to know what lengths you go to and what techniques you use to protect those things, while at the same time not losing them if you have a device go down.