Is it's just the shimmer that's fading and not the color itself, then it's not a primer problem. It's the surface that's wearing away. I would experiment with different powders to see what will set the shimmer without changing the look too much. I had a thought about mixing it with primer, but I'm not sure if that would work at all...
On a side note, I have the exact opposite problem with these as you. Mine melt into creases, but the shimmer stays with the product. (I need to invest in a primer...)