For the second year in a row, a record number of Americans renounced their U.S. citizenship or long-term residency in 2014, according to data released by the Treasury Department, but with increased renunciation fees, many couldn't afford to renounce even if they wanted to.

In 2014, 3,415 individuals renounced their citizenship, up 14 percent from the 2,999 individuals who renounced in 2013, also a record, reported The New York Times.

Americans living abroad often say that it's too difficult to deal with U.S. tax laws. The U.S. is one of the only countries in the world to tax its citizens regardless of which country they work in, and taxes can even apply to children born to Americans abroad.

"Many Americans abroad are finding that retaining their ties is not worth the cost and hassle of complying with the U.S. tax laws," Andrew Mitchel, a lawyer in Centerbrook, Conn. who tallies the lists of names released quarterly by the Treasury Department, told the Times.

While these laws were rarely enforced, the 2010 Foreign Account Tax Compliance Act, known as FATCA, required foreign financial institutions to begin reporting income of their U.S. customers to the IRS, according to the Times. More than 140,000 banks and institutions now report to the IRS.

Mitchel linked the growing number of renunciations to the FATCA enforcement campaign targeting U.S. taxpayers with undeclared offshore accounts.

But not everyone can get rid of their U.S. citizenship even if they wanted to. The actual fees required to formally renounce U.S. citizenship increased to $2,350 in September, a 400 percent rise from the previous $450 fee. State Department officials said the increase was due to rising costs associated with processing applications, including a costly interview process.