It sounds like the guy cares too much about what other people say and think about him. It isnt always a bad thing, but in this case it is, because he is being mean to you.
Dont let him drag you down sweetie, he isnt worth it if he carries on acting the way he does.
Im not saying he is a bad person or anything, because it must be tough for him. Most people are brought up being taught that girls and boys should be together right? However, this doesnt excuse his behaviour towards you!
It sounds like he needs to come to terms with himself before he should involve anyone else.