Hilary Duff may have "Lizzie McGuire" to thank for her successful career, but it wasn't always easy for her to be associated as that character.

The 27-year-old singer/actress recently sat down with Pride Source and explained how there are some people who still only refer to her as Lizzie McGuire. She understands the character was her first big break, but Duff's been ready to move on from the Disney role for a while now.

"As torturous as it has been at some points in my life to be Lizzie McGuire, I think that when that show came out, it was such a part of who I was, I didn't feel like I was playing a part," she said. "The writers all knew me so well and were writing things that were happening in my life and things that I would say, and I was dressing exactly like I wanted. It was so me."

Now that she's grown up and still pursuing her singing career, Duff realizes how grateful she is for her part on that show— she just wishes people would see her for more than that role.

"I don't think during filming it was torturous," she went on to explain. "I loved it during the filming. I just didn't know what a success the show was gonna be, and after that-four years, five years after that-and I was still Lizzie McGuire to people and that was super annoying."

There was a time when Duff was trying to take the next step in her career and things did get a little "scary."

"There was no guarantee that my career was still gonna be there," she said. "I really needed it personally. I grew up in the spotlight and on tour and with everyone just knowing me and knowing me a certain way. At some point I was like, 'I don't even know if I'm that person anymore, and I don't even have the time to figure that out.' I was like, 'I'm done. I don't enjoy this anymore. I'm not who I wanna be.'"

Now, years later, Duff is happy and doing her own thing. She's starring in the show "Younger" and just released her new album "Breathe In. Breathe Out," according to Us Weekly. She also has a 3-year-old son Luca. Duff admits she's "grateful" for all the her early career did for her, but she's happy to be able to be her own person now.