Lots of good comments here.
First and for most, you need to be clear on what you are feeling. You can desire him, be infatuated with him, lust for him, and even love him. But you need to be absolutely sure what you feel.
Once you figure out the honest feeling you have, realize he is going to want something different.
If what he is willing to offer you now is sufficient for the rest of your life, then accept what it is. If not, you have to ask yourself "Am I willing to lose what I have now?" If not, then accept his companionship and friendship and move on.
It takes two people to make a relationship.