Why all the fuzz about serial numbers?

[Gaffel]

It seems like there is a lot of attention about serial numbers (or letters). Why are we not just talking about production year/ date of purchase for any given watch. As long as there hasn´t been any changes to the exterior or caliber the "letter" should not mean anything.... Or am I missing something here?? Is all the talk about letters to clarify the time of minor or major change to the watch? Has there in the history of Rolex been a "serial letter" that has become insanely sought of or rare??
I know the answer is out there somewhere among the brilliant minds of TRF!