Grumbler, "leave towards the sunset" isn't idiomatic. This line refers to the end of western movies, when they would show the cowboy riding off on his horse at dusk as the final music played and the credits rolled.

I agree, but as it's not uncommon, say, to "walk towards the sunset", my thought was that it doesn't have to be always idiomatic.

That's true enough. Good point. I guess I just don't like the word "leave," it would sound better if you had used "head off." The reason is that with "leave towards" it sounds like you're referring to the time of day, like for example "my flight is at one, so I'll leave (the house) towards noon (for the airport)" instead of walking into the actual physical sunset.