Donald Trump’s beloved Mar-a-Lago may no longer be a residential option once he leaves the White House; that’s if a group of neighbors at the Palm Beach, Florida, private club have their way. They’ve taken legal measures to ensure that the president does not make it his permanent home.

Palm Beach town authorities received a letter addressed to the U.S. Secret Service that states Trump no longer has the right to permanently reside at Mar-a-Lago, as a result of a 1993 agreement in which he turned the residence into a private club, reported The Washington Post.

The deal prohibits the club members from staying at the guest suites more than 21 days a year or being there for more than seven consecutive days.

The neighbors’ attorney Reginald Stambaugh said in the letter that the town should inform the president that he cannot have the club in use as his residence, according to The Washington Post.

In the letter, it also said that making that notification would prevent an “embarrassing situation” in case the president moves to the club and is later asked to vacate the premises. 

“In order to avoid an embarrassing situation for everyone and to give the President time to make other living arrangements in the area, we trust you will work with his team to remind them” of the agreement, wrote Stambaugh. “Palm Beach has many lovely estates for sale and surely he can find one which meets his needs.”

Trump also signed a document handing over development rights for Mar-a-Lago to the National Trust for Historic Preservation, a nonprofit organization that works to conserve historic sites around the country. Under the deal, he agreed to surrender his rights to develop Mar-a-Lago or to use it for “any purpose other than club use.”

The president has previously come into conflict with local residents by failing to obey local requirements, such as by not following height limits for a gigantic flagpole he mounted and the prevalence of congested traffic and gridlocked streets due to Trump’s constant visits to the club. 

Trump has traveled to Mar-a-Lago at least 30 times in the course of his presidency, and spent at least 130 days there. There has been no sign that the town has opposed these visits. It also states on Mar-a-Lago’s website that he keeps private quarters there, which seems to publicly defy the agreement.

Palm Beach has also reneged concerning issues of security, by allowing a helipad that was barred in the original agreement. Trump can no longer use the helipad once he leaves office.

The U.S. Secret Service, a smaller subset of which has been preparing to follow Trump in his post-presidency life, may have to rush to create a new plan to keep watch over him if Mar-a-Lago turns out to not be an option.

The agency would need to make living accommodations for their agents prior to Trump leaving the White House, regardless of where he decides to stay.