OK, please tell me why the Brits pronounce "Worcester" like the way we in the State of Ohio, USA, spell the city located in this state: "Wooster"? I always tell folks from Massachusetts that use the British spelling that by the time the settlers got to Ohio they were more into phonetic spelling. We have a similar situation with "Gloucester," which is spelled "Glouster" in Ohio. Then there's our local hero in Athens, Ohio, from the late War of the Rebellion (1861-65), General Charles H. Grosvenor, whose name, of course, is pronounced as though the "s" doesn't exist. What was it that Winston Churchill once said about the U.S. and the U.K, two nations separate by a common language.
The answer to your plaintive cry is "diachronic sound change". Particularly coming into play here would be that the West Germanic languages tend to put primary stress on the initial syllable, coupled with the fact they also tend to reduce unstressed vowels to schwa (the "uh" sound at the start of "again"). Mix in several centuries of apocope and syncope (ellision of unstressed vowels from the middle and end of words) and you tend to up with short(er) words with a bunch of assimilated consonants at the end.
In layman's terms, over the centuries the pronunciation of long words tends to get a bit mushed-up/slurred and then simplified.
Town names ending in -cester are very old (the derive from the Roman "CASTRA" for camp or fort, as do English town names ending in -chester, -caster, -xeter, or Welsh town names starting Caer-). Pronunciation changes continually over time, of course, whereas English spelling was by and large fixed in the 1500–1600s, which is why it can seem so irregular now to non-English speakers.
English towns ending in -chester and -caster are usually pronounced as you might expect. The pronunciation of most English town names ending in -cester however gets elidided to -'ster:
Worcester > Wooster
Leicester > Lester
Gloucester > Gloster
Towcester > Toaster
Bicester > Bister
Frocester > Froster
Alcester > Olster (but also "all-sester"!)
Rocester > Roaster
The only exception that springs immediately to mind is Cirencester ("siren-sester").
Probably worth pointing out here that most English-English dialects are non-rhotic, so that final "-er" is going to sound like a clipped "-uh" to anyone who speaks with a rhotic dialect (a non-R-dropping accent, like General American or Scots): Woost-uh, Lest-uh, Gloss-tuh, etc.
As to Grosvenor, I think there are almost as many pronunciations of as there are English-speaking people. One in Birmingham that I'm familiar with has metathesis of the /s/ ad /v/, so it ends up sounding like "groves-nuh".