Sean Penn says he was 'relieved' after daughter Dylan 'nailed' her first scene after he cast her in 'Flag Day'

Working with family can be a challenging process. But for two-time Oscar winner Sean Penn, the experience brought him a strong self of relief.

Penn, 61, sat down with Sunday TODAY's Willie Geist and explained his feelings about working opposite his real-life daughter, 30-year-old Dylan Penn, in the new film Flag Day, which he also directs.

In the film, Penn plays famed counterfeiter John Vogel, who is attempting to win back the respect of his daughter, played by Dylan. While first reading the film's script it dawned on Penn that his daughter was the only person who could play the role, despite her limited acting experience.

"I had been sent the script, and I got to about page 30, and that's when my daughter's face got imprinted on that character in a way that I'd occasionally had happen with other actors, but not to the point where it could now only be that face for me," Penn said.

But he admitted that it was only when he started to shoot the film that he felt confident that Dylan was truly up to the task.

"It wasn't until the first day of shooting, the first scene we shot together, when she just nailed a long scene," Penn told Geist. "I remember immediately being thrilled, admiring, moved. But mostly relieved."

Speaking of his daughter, whom he shares with ex-wife Robin Wright, Penn said "she kind of got born with a sort of wisdom of soul, and I think that she'll handle this well."

Of course, working with family can be difficult experience. Earlier this week on The Late Show With Stephen Colbert, the duo admitted they had one significant argument while on set.

"I think it was a two-hour standoff about a note that he had,” Dylan recalled, Yahoo Entertainment previously reported. “About whether I could wear, or not wear, mascara.”

“Once it was clear that she was making the wrong decision, I just went and sat down,” her dad went on to explain. “And then the silence was one the crew had to live with for two hours while the Penns figured it out.”

There were additional struggles for father and daughter during production that stemmed from the film's complex story lines. Penn previously told Yahoo Entertainment that he wondered if he'd made a mistake asking his daughter to act out some of the movie's more traumatic moments, which included volatile arguments and drug abuse, as well as a near-rape.

"There are a couple of scenes where being the one to have invited her to tear herself apart and being her father were at odds for me," Penn told Yahoo. "I felt I should be calling Child Protective Services on myself a few times! But she was so invested and startlingly wonderful, so in terms of being a director dreaming of having a performance to use in a film that way, it was more exciting than depleting."

As for working double duty as an actor and director, Penn told Geist that the process was particularly grueling.

"So there are people who are wired for that," he said. "For me, I have noticed in my life that I am overdrawn to multitask, and it puts an incredible stress on it. Directing is a 17-and-a-half-hour-a-day job anyway. The idea of going home to learn lines each night with everything else going on was too much."

But he's still up to the challenge of working with a kid again — namely, Dylan's younger brother, Hopper.

"I suppose if I found something that I identified as strongly with my son as this that I did my daughter, I would certainly direct my kids again," he said. "Dylan's side of that story is that the next time we're on a set together, she's directing."