About 250 protesters chanting "Freedom" massed on Sunday at Queen Victoria Market, where some threw fruit at police after raiding traders' stalls.

Footage was posted online of violent scuffles as officers in riot gear pushed through the crowd in between tables piled high with produce.

"There were a few tense moments when protesters started grabbing fruit and throwing it at police," an AAP photographer said from the scene.

Other videos showed mounted police riding their horses close to a group of protesters in a bid to break the gathering up.

Protesters yelled "This is not a police state" and "You've got to be on the right side of history".

"There was also a bit of aggression toward the media, calling us 'scum'," the AAP photographer said.

Some protesters marched away from the market and through city streets.

Victoria Police arrested 74 people and issued at least 176 infringement notices for breaching the Chief Health Officer's directions.

A 44-year-old man, who police believe was a primary agitator for the protests remains in police custody and is expected to be charged with incitement.

His home will be subject to a search warrant.

Another person was arrested for assault police.

Police say they were disappointed that many protesters threatened violence toward officers.

"It was extremely disappointing to see people not just protesting, but putting the lives of other Victorians at risk despite all the warnings," a spokesman said.

The rally follows others on Saturday when about 100 people demonstrated against strict Melbourne coronavirus restrictions across various locations.

Police on Saturday arrested 14 people and fined at least 50 for breaching health directions.

The second day of protests comes hours after a woman shared footage of police dragging her from a car.

The masked driver refused to get out of her car after talking with an officer about a phone charger at Wallan, about 60km north of the city.

A male officer can be heard asking for the woman's name and address before she again refused to get out of the car

The footage then shows the officer leaning into the vehicle as the woman protests, before she is forcibly removed and arrested.

The woman is expected to be charged with various offences including assaulting police and offensive language.

Victoria on Sunday reported 41 new cases and seven deaths, bringing the state's death toll to 723.

Facebook posts say protesters plan to gather every Saturday until restrictions are eased.

Australian Associated Press