New home builds are expected to fall more than 40 per cent, according to the Housing Industry Association.
In its quarterly economic and industry outlook, HIA said its forecasts expect housing starts in Victoria to decline 12.7 per cent this financial year.
The dip in the 2020/21 financial year is expected to be a further 33.8 per cent.
Tweed Sutherland First National director Matt Leonard thinks Bendigo could buck the trend.
"I don't believe there will be a downturn of residential building in Bendigo," Mr Leonard said.
"What supports that is the strong demand for residential land at the moment."
HIA's Victorian executive director Fiona Nield said the COVID-19 pandemic has caused a shock to the economy.
"Lower sales and high levels of cancellations will see a contraction of home building activity in the second half of this year," Ms Nield said.
"This will see the market at a lower point in December than it was during the 1990's recession."
In 2018/19, 62,200 dwellings commenced in Victoria, comprised of 36,600 detached and 25,600 multi builds.
Detached builds refer to homes, with multi builds including apartment complexes and some townhouses.
In 2019/20, total builds are expected to fall to 54,200 and dip even more to 35,900 in 2020/21, according to HIA.
Metricon's regional north manager Tania Tonks said the new housing market won't be immune to a weaker economy.
"Traffic to display homes is understandably down but our new sales deposits, while being lower than previous weeks, are still encouragingly strong," Ms Tonks said.
"In the last week, we've noticed a significant lift in the level of enquiry.
HIA said there are a number of ways the state government could assist Victoria with a housing-led recovery.
"These measures include a stimulus for home buyers, stamp duty concessions and incentives for foreign investors," Ms Nield said.
A state government spokesperson said the coronavirus pandemic is having a devastating impact on economies worldwide.
"It's more important than ever that we support our building and development industry," the spokesperson said.
"We recently launched the Building Victoria's Recovery Taskforce to get new projects off the ground faster and help boost construction."
Have you signed up to the Bendigo Advertiser's daily newsletter and breaking news emails? You can register below and make sure you are up to date with everything that's happening in central Victoria.