Excellent advice for beginner's from RM_guy. I'll take the other side of the argument, for what it's worth.
It comes down to this: is he asking too much of YOU to be involved in something HE loves and YOU don't ? I'm sure he feels if you give it some time, you will love the hobby as much as he does. Seems obvious to riders; "this is so great the only way someone could NOT like it is if they've never tried it."
I feel sick just thinking of trying to ride it again but I know I have to because he is going to be making payments on it for the next 3 years.
That was a lack of judgement on his part. 1) It's unfair pressure on you, 2) making a commitment like that on something that you may not enjoy? Not wise and likely a money loser.
I'll bore you with my experience with a similar situation. I convinced my wife that riding was a lot more fun than watching and got her a Yamaha TTR 125L. She was open-minded and eager to learn. We enjoyed the time together on the weekends for a change as I was usually at the track on Saturdays for a lot of years...
Fast forward 3-4 months. Yes, she had her share of get-offs, mostly just tip-overs, but an occasional fall that left a mark and or a tweaked TTR. The minor bumps weren't a big deal to her and she seemed to like riding enough that a scrape here or there was worth it. At least she presented it that way.
Several months later she entered her first race at DirtWeek, in the Women's B class. I kept suggesting that she just concentrate on finishing upright, that in itself would have been a big accomplishment. Long story somewhat shorter, she suddenly got all competitive, rode over her head, but won! Turned out, that was the "Pinnacle of her career".
From there, she rode less and less until finally admitting she just wasn't interested anymore; the time and effort weren't worth it any longer. I was good with that, she gave it a solid chance, but it just wasn't for her. Who was I to argue? We sold her TTR and all her gear.
The point I THINK I'm trying to make is... we do some things for our significant other (as you mentioned), that we may not "love" and sometimes dislike, but do it to make them happy. That's just part of it. BUT, this hobby is a big ask; a lot of time, money, effort and even potential injury. When I suggested she start riding, I *suggested*, with zero pressure. It was her decision to try it or not. It would have been easy for me to be overly excited about it and push her into it, I went into it keeping that in mind. This is one of those asks that fall outside the "I'll do it if it makes you happy" thing.
Finally, I'll commit to an opinion.
If riding is not your thing and more-so if it causes you stress, no one is going to enjoy it. If it exacerbates the anxiety issue, by all means lay it out for him. Seems to me he will realize he jumped the gun and should just lick his wounds and move on. He'll may have to check his ego.
Maybe you go with him from time to time to hang out for a bit? Granted if it's trails / woods riding he rides, it would be pretty boring time in the dirt for you, lol... but if it's MX it would be together time while he takes breaks in the pits. Take a good book and enjoy the outdoors. There are worse things in life.
Pressuring YOU into something YOU don't want to justify buying the bike is a mistake on his part. We all make mistakes, how we fix 'em is what matters.
If "doing something together" was his goal, he'll get over it. Maybe suggest something else you can do together?
Good luck to you in what ever decision YOU make.