Here are the messages we are trying to send with our body language to potential mates, and what is seen as attractive:
I may be attracted easily but I guard my heart.
Then the man told her he was a doctor and the woman literally swung her purse up and over her shoulder out of the way.
I used to experience aggressive approaches from women when I first started dancing in also, like right infront of me all of a sudden on many occasions.
With the added pressure of being in a dating situation, especially on a first date, our ability to manage body language goes right out the window.
Look for the facial muscle tension.
This is usually construed as a good thing; he's into you, but he's shy.
Quick, frequent, and jerky motions make you look nervous and not very confident, whereas the opposite effect is achieved by motions that are languid and occasional.
To be clear, his interest may be in the conversation you are having with him, or he may just be interested in you, AKA, attraction.