A married person does not live in isolation. He or she has made a promise, a pledge, a vow, to another person. Until that vow is fulfilled and the promise is kept, the individual is in debt to his marriage partner. That is what he owes. 'You owe it to yourself' is not a valid excuse for breaking a marriage vow but a creed of selfishness.
