Family drama is brewing behind the scenes of "Duck Dynasty" and several cast members could be stepping away from the spotlight, Life & Style magazine reports. The show has been wildly successful but not every family member is comfortable with the spotlight and fame.

According to rumors Phil Robertson, Jase Robertson and Jep Robertson could be leaving the show. A source told the magazine that Willie Robertson loves the spotlight and fame but his brothers are more laid back and could do without it.

"[Willie] loves the attention and [being on the show]," the source said.

His mother, Kay Robertson, told Life & Style that Willie is "the leader of the pact."

"Jep doesn't like the fame," Kay said. "Jase is definitely more of the quiet one," his wife Jessica added.

According to the magazine, a local coffee shop worker said Jep used to come inside to enjoy his morning cup of coffee but now comes through the drive-thru because of people always recognizing him.

"If Willie keeps pushing and doesn't listen when his brothers say they've had enough, it will cause real problems," the source said. "If they feel like he's forcing them, it will destroy the strong relationship they have."

Kay, who loves the show, said Phil wants to be done with the show already. He recently complained that A&E was adding "fake bleeps" to make it seem like the family was cursing when they weren't and was also censoring their religion. He told Sports Spectrum, a Christian sports magazine that the network told them not to use the word "Jesus" because they didn't want to offend some of the viewers.

"Phil has already admitted he wants out of the show," a source said. "I think this could be his last year. I am sure they won't let him leave easily, but I think it has gotten to be too much for him."