Fatherhood wasn’t Shemar Moore’s focus most of his life, but when his mom died suddenly, everything shifted for the actor.

“When my mother passed, I felt like I effed up,” Moore, 54, tells PEOPLE in this week’s issue. His mother, Marylin Joan Wilson-Moore, died because of multiple sclerosis and a heart condition in 2020.

He had been acting for just shy of three decades at that point, but he remembers mostly feeling “selfish” for the way he’d spent his life. “I chased my career so hard because I was so — and I still am — determined not to fail. But in that pursuit of not failing, what are you sacrificing? What are you losing?”

WATCH: Inside Shemar Moore’s ‘Pinch Myself’ Dream Home Where He’s Raising His Daughter (Exclusive)

Marylin’s d3ath made the S.W.A.T. star realize just how much he had been giving up. “I was like, ‘Wow, I was selfish.’ I made it all about me. I made it all about my career. I made it all about chasing girls and living my life and kicking with my homies. And all she ever wanted was to be a grandma.”

It was her “dream,” he says. “And I still feel guilty about it.”

Shemar Moore and his mother.Shemar Moore/Instagram

He welcomed his first child, daughter Frankie, with girlfriend Jesiree Dizon in January 2023, just shy of the third anniversary of Marylin’s d3ath, and ever since, the star has a whole new perspective on his career.

“Now, I just want to perform from a passionate place. Whether a lot of people watch or a few people watch, whether there’s an award connected to that or not, I just want to perform, make my momma proud,” he says. “And then one day Frankie’s going to go, ‘That’s my daddy. That’s my daddy.’ That’s it. That’s it.”

Looking back, he says Marylin’s d3ath was both “the toughest time in my career” and “the toughest time in my life.”

“She was my inspiration. She was my best friend. We loved hard, we fought hard, but there was no filter. My mother raised me dead honest. We talked about sex, we talked about drugs. We talked about religion, we talked about politics. We talked about good, we talked about bad. She gave me structure. She gave me discipline. She gave me hope.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

He continues, “My life and my career was to make her proud. Still is. Her voice is always in my head and my heart. ‘What would my mother say? How would my mother feel about that?’ It affected me and still affects me to this day. I miss her tremendously.”

Shemar Moore with daughter Frankie and partner Jesiree.Shemar Moore/Instagram

The shock of the loss only added to that pain. “I didn’t realize she was going,” he says. “I don’t know if it’d have been different if the writing was on the wall. Just, I woke up on a Saturday and got slapped in the mouth.”

She had “health issues,” as Moore notes, since she was diagnosed with multiple sclerosis in 1999, but “she didn’t want to go.”

“I believe she was tired,” he admits. “She was a superhero. She wanted to take care of the world. She wanted to take care of everybody but she didn’t know how to take care of herself. And I tried my best to do that, but I’m an only child, I couldn’t be there 24/7.”

For more on Shemar Moore, pick up this week’s issue of PEOPLE, available Friday.

“That was the hardest life moment, and it affected my career. It put me on a knee. It made me angry, it makes me sad, but it makes me fiercely stronger, to survive and win by any means necessary,” he says.

Now, he’s focused on trying to “be the best friend I can be, be the best version of myself, be the best partner and be the best damn daddy that I never had for Frankie.”