You got a ton of comments so I'll keep mine short.

I liked the piece, you made me smile. You were lighthearted about it and found humor in the situation. That speaks volumes about your character and generosity of spirit.

Because I know it's not easy to have your desirability as a person dismissed with a look.

I am autistic. I am decent at masking and presenting a normal affect. So, new neighbors are usually friendly when I smile and say hello. Conversation often ensues.

Then the "look" happens.

They realize I am "mentally challenged" in some way and their faces turn gelid as they try to keep smiling while figuring out how to flee.

Then it's just as you describe. The obvious effort to avoid me in case I might be "interested" in them. It feels pretty much just the way you describe it. You got it completely right.