I usually hear this as I am tucking a scribble-covered napkin in my pocket and wondering why they're waiting.
I once told someone that I started a new book whenever I didn't have anymore room for my wallet or keys. And it's not far from the truth.
I would imagine that in a world where almost everyone carries a tiny computer in their pockets, we could stop waiting until we get home to write something down. The sad fact is that even with the ubiquity of smart phones and other devices, we still rely on our memories to preserve the ideas that spring fully-formed into our heads at the drop of a word.
There's magic in places where different minds meet. Conversations in real life are fertile ground for ideas. We're told to keep our eyes open and watch for the stories all around us, but what good does it do us if we don't write it down?
We spend a lot of time on writing blogs and at conferences reminding aspiring writers to keep a notebook on their nightstands for those 3 am epiphanies. (I've moved mine down the hall to the bathroom cabinet because turning on the table lamp wakes up my wife and I got tired of using a headlamp.) But I think we don't spend enough time stressing the need to carry a notebook (or equivalent) at all times where we're out with friends.
Not that I should throw stones; I don't always have a notebook with me either. That's no excuse to let the ideas slip, though. I'll use a napkin or an envelope, or my hand, or a patient passerby if necessary... pretty much anything that will take ink will suffice.
They're not all gold, of course, but they're not all dross either. And the only chance I get to decide which is which is if I stopped for a second to jot a note and shove it in my pocket. Ideas are transitory things. They'll slip away from you if you don't get them down while they're fresh in your mind.
I've never had anyone tell me they find it rude, but I accept that some people might. It's all part and parcel to befriending an author, I'm afraid, occasionally we'll get that far off look in our eyes and start searching our pockets for a pen.
I think that it's a goal we should all have: to come home at the end of the day and find that our pockets are full of ideas.