Listen and subscribe to Opening Bid on Apple Podcasts, Spotify, or wherever you find your favorite podcasts.

The current state of the housing market has many people worried — including brothers and housing experts Drew and Jonathan Scott, known for their HGTV show "The Property Brothers."

Ballooning home prices and rising interest rates have made many hopeful first-time homeowners feel like buying a home is still not attainable. And according to the brothers, a lack of housing supply is the greatest factor preventing people from buying.

“Right now, the biggest problem is the inventory is really, really bad,” Jonathan Scott told Brian Sozzi on Yahoo Finance’s Opening Bid podcast (see video above or listen below). He noted that the US's housing inventory is about 4.5 million homes short of "healthy market" levels.

“When you don't have a healthy inventory of homes, that's probably the biggest driver [of unaffordability] because competition is what drives prices up and down,” he added.

Though the Federal Reserve began cutting short-term interest rates, totaling three-quarters of a percentage point since September, mortgage rates have continued to tick upward.

When the Fed announced its first cut of this cycle in September, 15-year mortgage rates were at a 52-week low of 5.15%; now the same fixed-rate mortgage sits at 6.02%, per Freddie Mac. 30-year mortgage rates have similarly increased in that time frame and are three-quarters of a percentage point higher than they were two months ago.

Read more: How the Federal Reserve rate decision affects mortgage rates

It’s not just the present real estate market that has the brothers concerned.

“My fear is that in 20 years, our kids won't be able to afford to buy a house,” Jonathan Scott said. “Nobody will be able to get into the market because it's going to be so crazy expensive. We need to start tackling that problem now.”

Though the brothers admitted it’s “healthier” for the market when interest rates come down, Jonathan predicted that lenders may introduce new products to “incentivize” young homeowners, noting that these products may come with strings that could leave homeowners paying off their mortgages for a longer period of time.

“Lean on the fact that lenders want you to get their products — they want to lend you the money, they're going to come out with products that are going to make it a little more accessible,” Jonathan said. “Though, the problem is they’re going to extend the amortization period so it's going to take you longer to pay off that investment.”