The housing market continues to teeter with new data today showing a slow down in residential construction as many home builders expect rising mortgage rates to hurt demand for new homes, which could slow the rise of housing prices nationwide.

New housing starts fell 14.4 percent in May to an annualized rate of 1.55 million, the lowest number since April 2021, according to Census Bureau data released Thursday. The report is the latest sign of growing pessimism in the housing market. Meanwhile, mortgage rates surged above 6 percent—their highest level since 2008. Home prices throughout the country have skyrocketed since April 2020 by more than 24 percent as increased savings and a migration to the suburbs puts a strain on supply, but that may be changing.

“The period of unsustainably rising housing prices is over with the rise of interest rates,” says Robert Dietz, chief economist at the National Association of Home Builders. Dietz still expects housing prices to rise, but at a slower rate.

Home building companies are expected to keep new starts relatively low in the coming months, as interest rate hikes hurt demand. New applications to build fell to an annualized 1.7 million units, the lowest number since September.

The decrease comes at a time when housing supply is near a historic low, with 51 percent fewer homes on the market today compared with May 2019. Last year marked the most home building activity since the housing bubble burst in 2007, but there is still a lot of ground to make up before supply reaches healthy levels. While upward pressure on prices is likely to ease, Fed Chair Jerome Powell stressed that a housing crash is unlikely. “Prices may keep going up for a while,” Powell said about home prices at a press conference on Wednesday. In fact, home prices are expected to appreciate by a modest 6.6 percent in 2022, according to Danielle Hale, chief economist at Realtor.com. That compares to a 19 percent increase in 2021.

Supply chain snags and increased costs for materials have also contributed to rising home prices and occasional dips in new construction. Overall construction costs – which includes labor and materials – are up 19 percent year over year, according to data from Redfin.

Another reason for the surge in housing prices? Buyers have been buoyed by low interest rates and more than $3.7 trillion in government stimulus. “Millennials are all aging into this home-buying portion of their life,” senior economist at Zillow, Matthew Speakman, says of the 72 million individuals born between 1981 and 1996.

As rates keep rising, some buyers will be priced out while others push forward more quickly in anticipation of another hike. “People want to buy now because they know rates are only going up,” says Gregory Heym, chief economist at Brown Harris Stevens. The sense of urgency is warranted. Homes are staying on the market for less time than ever before. The average seller accepts an offer within 31 days of listing their home, compared to 71 days in May of 2020, according to data from Realtor.com.

More recently, some sellers have been forced to cut their asking price, with almost 25 percent of homes on the market dropping their target number in the first week of June, according to the market data company Altos Research. That’s still almost 8 percent below typical averages, but it’s a fast increase from the sub-20 percent numbers from earlier this year. “There’s so few homes on the market, that there’s just this inherent competition for those that are for sale,” Speakman says. “Sellers are starting to have to reset their expectations a bit.”

Despite harsher conditions for buyers and sellers in the coming months, economists are hesitant to suggest that the cooling market could drag down the rest of the economy. That said, if high inflation persists, then rising interest rates could slow down the job market. With worse job prospects and higher mortgage rates, homebuyer interest could collapse to more dangerous levels, driving prices down.

Adds Heym, “The question is how fast can the Fed get inflation down low enough to stop the rising mortgage rates.”