The way Kevin Hart sees it, being funny kept him out of trouble while growing up in North Philadelphia — but it may also have hurt him with women.

"Nobody wants to fight the funny guy," Hart told CBS News correspondent Tracy Smith in an interview to air this weekend on  CBS Sunday Morning (9 a.m., CBS3).  "Nobody wants to mess with the funny guy. Everybody wants to be around the funny guy. And that's what I was."

But maybe nobody wanted to mess around with him, either?

"Not even going to lie to you. This would have been a perfect place to put that lie. 'Yes, women were all over…' " Hart tells Smith, according to CBS. "Women bring out a different side out of you. So, it's like, you can be cool and you can be funny. But to women, you can be goofy when it's time to not be funny. I didn't know how to shut the funny off with women. That was my problem when I was younger."

Hart, who is now expecting a child with his wife, model Eniko Harris, has two children from a previous marriage.

His approach to fatherhood puts time before money, he told Smith.

"Be there. Be present. Hands down. Hands down. Nothing else matters," Hart said. "All the money in the world does not add up to time, to a child."

The interview promotes Hart's new book, I Can't Make This Up: Life Lessons, which will be published June 6.