I don't think it comes down to how long they were a pure titan. And I like the purity of serum idea but I think it's a bit deeper than that.
First, there were differences between Ymir's Pure and Shifter form. The head was awkwardly sized, the arms were really skinny (actually very similar to Armin's).
Second, I think we have to assume that the Titan's have their names because there are properties that are unique to that titan and always manifest in each shifter. I imagine that Armin will still be "The Colossal Titan" otherwise...doesn't that fundamentally undermine the name of that Titan? We've seen similarity across all three Attack Titan users. Also, any time the Armored Titan is discussed, it's discussed as though its the "armor" itself that is being passed.
Third, I've actually been working on a theory that the Titan has a specrum of ability that, the longer the user practices and the more user knows about titans, the more they can use the different skills. Think about how Reiner moved his mental functions to his Titan Body - Eren can probably do that, but he doesn't know how. Eren might be able to have the diamond skill - but he doesn't know how. When it passes to a new user, the ability resets to zero. I think Ymir, who returned to human form and then hid, never really progressed in developing Jaws and what she could do with the titan. That's why Galliard looks so different. He has had the war to develop his powers.
I also think that the collective knowledge of your ancestors/bloodline is somehow available to somoene when they become a shifter. THIS is the real reason why the royal family has so much power - because their family has hundreds of years of titan shifter knowedge.That's why Zeke (and Annie) can harden and somewhat control titans. THeir bloodline is far more knowledgeable. It's also the reason that the royal family can pass the founding titan along with all the memories.