I completely feel your plain. You and I sound relatively close in age. This is something I have struggled with, especially lately. I grew up in a small town in northern PA, where I was lucky enough to be able to have our horses on our property on a single mom/teacher's salary. When I graduated from high school, my family also moved away.
I went to college in NM, and like another poster, found that in order to afford a horse, I had to work so much that I had no time for her. I made the decision to sell, and promised myself that I'd get a horse as a college graduation present. It was awful. It wasn't exactly horse country, and I was footing all the bills (including tuition) myself. After some time, I adjusted. I started mountain biking and played a little guitar- found some other hobbies. I still was obsessed, I'd still spend hours online drooling over horses. Graduation came and I couldn't find work in the area, so I moved to northern Virginia, where my mom is. I knew there was a better chance for me to find work here, and knew it was horse country. But the reality check of student loans on top of a ridiculously high cost of living shot the dream of owning down quickly
I was lucky enough to get hired within a month of moving. Once I had a paycheck in the bank, I found myself a wonderful half lease to bring me back after 5 years off. Then a year and a half later, I ended up ending the lease in order to afford rent. This hit me worse than the first time around. I've gained 5 lbs in the last 6 months, and I wear my cranky pants far more often than I do while riding. This time I have a supportive boyfriend with a horse owning mom. She tried to get me to ride, and at first, I wasn't ready. I needed a few months. Your analogy of it being like an ex is dead on. It takes some time to be ready for it. It caused some friction, but eventually, I was ready, and am lucky enough to get on once or twice a month. It's not enough, and having an all or nothing personality type, its very hard to come to terms with the situation. But I can tell you that I'm desperate to not lose any more of my riding ability, because coming back to it & having your body & mind not work the way it used to is frustrating!
I am glad you're going to try riding- I can tell by the fact that you posted this in the first place that you're almost ready, if not so, for it. I started volunteering at a therapeutic riding program for a few weeks prior to my first ride back after the lease ended. I cannot recommend this highly enough as it is unbelievably rewarding and is something I was lucky enough to enjoy in my childhood.
I know how hard it is, I really do, and I want to say that don't let anyone make you doubt your horsemanship because you aren't ready for the compromise that being involved with horses will be. Just make peace with yourself, knowing that you're doing what's right. Sometimes its better to walk away for a while than to stay involved and let it destroy you.
I really hope that you find your way back to the level you want to be at. This is who we are, we can't escape it. I'm sorry you ended up here, but have faith that you'll find "it" again.