Port Erin Commissioners have applied to have the Cosy Nook cafe building stripped of its protected status.
The building was placed on the buildings register just a few months ago, last April.
That means the site is now officially recognised as having historical significance and is legally protected from demolition.
The move was welcomed by heritage groups on the island and the government said it would preserve the site for future generations.
However, in a recent planning application town commissioners say the building was registered without a full inspection.
They claim the building is in a poor state and has no protection from an "increased risk of flooding".
GP waiting times to be raised in House of Keys tomorrow
Waves higher than predicted on crossing which damaged cars
Spitfire flights to be launched from Ronaldsway
Does Crogga owe the government money?
Thinking of standing for election?
Doctors' salaries under scrutiny from MHK
Rings stolen at Christmas market reported to police
Plans submitted for advertising board outside the airport