There are many words around the world that cannot be directly attributed to a simple English equivalent. Words like schadenfreude, etc. I just stumbled upon one I hadn't heard of before: saudade.

A translation provided by the ever wonderful Wikipedia: a somewhat melancholic feeling of incompleteness. It is related to think back situations of privation due to the absence of someone or something, to move away from a place or thing, or to the absence of a set of particular and desirable experiencies and pleasures once lived.

This is it. This is the feeling I can't shake.