There are a lot of problems in your relationship and I think you should deal with it one by one (and not as a tangled mess of a problem)
But first of all, hugs. I'm sorry your relationship is on the rocks and I hope you'll get through this happier and more fulfilled.
The key here is communication.
- Your muscle fetish.
The easier. Simply put : talk to him about this.
But of course, details matter. tl;dr : Can you get aroused with him? If you -can-, what you want to avoid is your boyfriend getting the idea that "oh, I'm undesirable to him. He desires someone else, someone better." Considering he's in a rough place right now, mentioning his (further) flaws can be destructive for him.
If you cannot, though; then I think an important, hard (and not in a good way!) talk needs to be had, sooner or later. Tell him that you're not sexually satisfied. And it's not going to be easy. But sexual compatibility -does- matter in a relationship. Ignoring that won't do you good in the long run.
So you have to consider a few factors:
For example, is your boyfriend muscular? Was he used to be?
Have you had sex with him? Did it go well? Did you enjoy it? If you haven't, does the image of having sex with him arouses you?
- His life problems.
That is hard; and as someone dealing with similar things; I can see how that affects one's mental health and behavior and relationship.
But at the same time, just because one's suffering, doesn't mean you as his boyfriend needs to be.
He might be using you as a crutch or he might be trying to keep you away from suffering from his problems
So I think you need to communicate, again. Try to avoid accusing– but communicate your needs. Define it as specific as possible. (i.e : "I really want you spending weekends with me" vs "I want you to be more attentive."). Add that you understand that he's having a hard time right now, and offer your help-- as much as it is healthy and as much as you can/willing to offer.
- Your treatment to him
.....you said you worry that you haven't been kind enough to him. Why? Because you refused his proposals? Is there something else you can add to explain your worry further?