The Pressure to Be Available
Very few people want to be seen as the curmudgeon who never engages in chit chat and hides away in their office all the time. Reputation matters in the workplace, and you certainly want to be liked. There is a middle ground, however, between hiding away and being the person who is always available. Unfortunately, what tends to be the case is that you become so fearful of disappointing someone (especially if you are a junior faculty member), that you overcompensate, and make yourself constantly accessible in the name of caution.
Say No, So You Can Ultimately Say Yes.
I would be interested to know: would telling a student you didn’t have time for a meeting be frowned upon at your institution? Do you work at a place where the expectation is, either subtle or overt, that you make yourself completely available for students whenever you’re in your office?