We are in the Jam Factory and some people actually keep a set of keys at the Porter's - which IMO is insane. You're just begging to be robbed.
This anonymous exchange is a good idea but would need some more detail. If you contact someone by phone your addres can probably be found out. So you'd need to have a prepaid phone and an e-mail not trackable to an IP address.
I've been worrying about the same thing so now have a set of my housekeys locked in my laptop strong box at work. I can get access to the building 24 hours a day and it would only involve a half hour walk to get me there. there's room in the box for another set, if you'd like!