Not quite sure what you're trying to say here. If there's no destination how can you journey there? Must 'real value' be dependent on 'journeying towards' some goal? In the absence of an actual destination do we need to fabricate a mirage? Your initial rejection of spiritual termini doesn't survive your second paragraph, where you appear to embrace the need for such, even if these are qualified as imaginary. This all sounds not too far removed from the religious approach I'm guessing you intend to repudiate. Destinations, journeys - these are the stuff of storying our lives, which is how we construct a sense of self. To be someone is to be trans-temporal and implies the idea of going somewhere, from which notions of purpose and meaning naturally arise and lead seamlessly to religion, after-lives and so on. To be someone with a purpose is in some sense to step outside the moment, to separate oneself from one's experience, from where the tendency is so often to compare what one has with what one might have. Don't you find that when you are most happy - mountain walking, perhaps - you have forgotten your narrative, your self, your journey, and you are (as Zen would say) intimate with the moment and not journeying anywhere else at all?
It is also from Buddhism that the idea of there being no destination, no appointment comes from. I think of the journey for me as being like the flow of a river. There isn't a purpose as such, just a movement. If you like, the sea is the destination, but that is just vastness. Nothing more.
I do have a story of me and what I experience and it annoys me when life does this to me. But then I am in a relationship, a partnership, and I think that needs to have an element of flow to it, as it evolves and moves from this to that. Sometimes it can feel like the steps of a dance, where at first you can't move in harmony and keep tripping over each other's feet, but in time it becomes effortless, and you move together seamlessly.
I don't see the contradiction between rejecting religion and embracing the spiritual. I believe that all of us need to take care of our needs that go beyond the physical - embracing art, music, nature - and it is very often these things that allow us to lose ourselves and our stories and become present.