Interesting question, if for no other reason than, "what constitutes a downgrade?" For example, I think most non-watch people would consider a Royal Oak a downgrade, given that they've heard of Rolex but haven't heard of AP. But since we're all watch lunatics here, let's stick with the idea that we're moving from a Rolex to something the cognoscenti would consider "less prestigious" (recognizing that no two people agree on what "prestigious" in this context means either, but we have to start somewhere).
I think you're asking an advanced question whose answer will change based on the individual and where they are at in their watch journey. Which is to say that each individual prizes different things for different reasons, and a lot of them are emotional reasons that you cannot rationalize. So to say that someone can "downgrade" from a Rolex that was a milestone purchase commemorating a major achievement in their lives....because this competing brand over here makes a watch that is "just as good for a lot less money," sort of ignores the entire reason the Rolex was purchased to begin with.
I also think you're also making an assertion that we buy specific brands for their prestige. Certainly this is the case for a lot of us, but I don't think it's a universal statement. I think for a lot of us - or maybe I'm just speaking for myself here - perceived prestige may factor into the purchasing decision, but I'm not buying a watch I think is UG-ly (with a capital "UGH") just because it has a prestigious name on the dial. You have to fundamentally like what you're looking at first and foremost.
Me personally? I just bought an APRO, in spite of owning a couple Rolex, because I wanted to celebrate a milestone birthday and wanted something I hadn't experienced before. Was prestige a factor in my decision to purchase. Sure, in the sense that I wanted something that was - in my perception - above where I normally was and wanted to experience what that was like. Has the APRO destroyed all other watches for me? No, I don't think so. I mean, sure, I could see myself going the rest of my life with the APRO and maybe the Sub as a two-watch collection...or those two plus the Oris Source of Life, Snowflake, and Frogman as a five-watch collection (that is an almost perfect 5x5 matrix of a little bit of everything)...but at the end of the week, I still take off whatever I'm wearing and put on my Oris Source of Life for the weekend. So no, neither owning a Rolex, nor owning something "above" a Rolex, has diminished my appreciation for a simple Oris Aquis.
Regards,
Alysandir