Port Erin Commissioners have applied to have the Cosy Nook cafe building stripped of its protected status.
The building was placed on the buildings register just a few months ago, last April.
That means the site is now officially recognised as having historical significance and is legally protected from demolition.
The move was welcomed by heritage groups on the island and the government said it would preserve the site for future generations.
However, in a recent planning application town commissioners say the building was registered without a full inspection.
They claim the building is in a poor state and has no protection from an "increased risk of flooding".
Emergency closure in place on quayside
Passport services to move online
Woman plans to ride 100 miles for charity
Roadworks near Foxdale this week
Postal vote deadline looms for those living abroad
MP reveals more details of Whitehaven ferry proposals
Council wishes Bee Gee happy 80th birthday
Warning over traffic delays as city centre work gets underway