Just three months after Bravo's "The Real Housewives of New Jersey" star Teresa Giudice served nearly a year at the Federal Correctional Institution in Danbury, Conn., on fraud charges, her husband Joe Giudice is preparing to begin his 41-month prison sentence on the same charges. The Giudice family has been spending time with Joe, saying their goodbyes before he reports to prison tomorrow.

Teresa and Joe were both found guilty of multiple instances between 2001 and 2008 where they defrauded the IRS and several banks. They each were charged with 39 crimes - including tax evasion, bankruptcy fraud and making false statements on loan applications - and were sentenced in 2014 to consecutive terms. Teresa was the first to serve her 11-month sentence, from January to December 2015, while Joe watched over the couple's four children. Teresa will do the same while Joe serves his sentence at the Federal Correction Institute in Fort Dix, N.J.

"My kids are very smart, and they know what the deal is," Joe said. "The younger ones don't understand why you have to go, what you did - I don't think either one of them know exactly why you are going in, but we just told them that we did some things wrong and we gotta go in."

The couple has four daughters together: 15-year-old Gia, 12-year-old Gabriella, 10-year-old Milania and 6-year-old Audriana. Joe said he did have "a little meltdown" when discussing his sentence with Gabriella, but he remains positive that the family can make it through this hurdle together.

Joe is also facing possible deportation after serving his prison time. Since he was born in Italy and never earned citizenship status in the United States, Immigration and Customs Enforcement authorities could force him to return to his home country. Joe gave his final on-camera interview before reporting to jail earlier this month and seemed relaxed and calm when discussing the possibility of leaving his family behind in the States.

"I mean, yeah, there's a chance of anything happening, there's nothing in life that is certain," Joe told "Entertainment Tonight." "But, I mean, we'll deal with that when I get out."

However, Joe isn't worried about spending time locked up and explained that the facility is a low-security prison that's more along the lines of a military camp.

"It's not a high security," Joe said, "and anybody that's in there is, basically, people that have already served a lot of time, basically, working their years to get out."