You HAVE to make the move, because it seems he is not going to. Both of you are locked in shyness, despite being pretty comfortable with each other. If you don't make the move, eventually he's going to find someone else to hang out with.
Look at it this way: what do you have to lose? If you're wrong, you may lose the chance to hang out with a friend for whom your love is unrequited. If you're right, you may end up with a boyfriend who understands you- something you would be relatively lucky to have. Well worth the risk if you ask me.
If you want to test the waters, you should give him a hug or something and see how that goes, or take the plunge and hold his hand next time you walk together. If it doesn't work out then relax...at least you haven't wasted years of your life pining over someone who you can't have. Instead you can use that time finding other awesome people to hang around with. Note that this also applies if he is in fact gay.