Your entire love story is not dictated by what society deems as acceptable. You are given the liberty to create and masterfully conclude your desired outcome. All relationships have rough patches, struggles, challenges, arguments and so on, but the person or the people that can resolve this would be yourself and your partner. If you have decided that this is how it ends, for reasons that you know you have thought well and hard about, then there shouldn't be any regrets. You are in total control, and you should never lose that.