When visiting someone in a shared building with a security system, you may find yourself waiting at the door to be buzzed in. If others are waiting as well, should you let them in with you when your friend buzzes you in, or ask them to wait for the person they're visiting to unlock the door for them?
One solution is to talk to your friend and ask them what they would like you to do. Let the others in? Don't let them in?
And it is entirely appropriate to politely say to others waiting to be let in, "I don't mean to leave you out in the cold but I'm a guest here myself and I'm just not aware of who does or doesn't have access. I'd feel better if you could use the code or buzz your friend directly and have them let you in."