The Victorian Farmers Federation says the closure of a silo site at Bridgewater could cause farmers pain.
The comments come in the wake of Graincorp's announcement on Thursday that numerous silo sites across Australia will close.
Other local sites to close include those at Borung, Birchip, St Arnaud, Echuca, Swan Hill and Watchem.
The announcement coincides with Graincorp's launch of "Project Regeneration" - a plan to spend $200 million upgrading major silo sites.
Graincorp has said the improvements on silo sites could mean farmers make an extra $5 per tonne of grain.
The VFF's Brett Hosking said this price increase was significant but it remained to be seen whether it would actually happen.
"Any grower will tell you that what we really need to see is action on the ground and that's what growers will be looking for," Mr Hosking said.
Mr Hosking said closing smaller silo sites would make farmers travel further to off-load their grain.
He said time was precious for grain growers during harvesting season.
"The biggest thing we look for in a bulk handler is the ability to unload trucks and get us back in the paddock as quick as possible," he said.
"This is a real challenge for Graincorp, hopefully they can deliver," he said.
Graincorp regional manager Scott Whitmore, based in Marong, said four local management employees had already been made redundant and the company was asking another seven to volunteer for redundancy.
Mr Whitmore said those losing their jobs had transferrable skills to find jobs elsewhere in the agriculture industry.
He said the investment in upgrading major silo sites would be money well spent because it would make the movement of grain more efficient.
He said the upgrades would make sites compatible with rail transport.
"It's about getting a million tonnes of grain back off roads onto rail," he said.
He said the upgrades would create a lot of construction jobs.