Victorians will not be required to wear masks in outdoors areas from Monday.
Changes to the rules around masks were part of a number eased restrictions that will be in place from 11.59pm on Sunday.
"In short, you'll still need to wear one when you're indoors and on public transport - and where you can't keep your distance," Mr Andrews said.
"But if you're outdoors and you can keep a safe distance, masks will no longer be mandatory.
"If you're headed for a quiet stroll around the neighbourhood, you won't need to wear a mask - but you will need to carry one, in case the situation changes."
The announcements included an outline for a COVID-safe Christmas in Victoria as well as a return to the work environment for people currently working from home.
From December 14, Christmas and end of year gatherings will have a limit of 30 visitors to a home per day. Babies under 12 months won't count towards the limit.
"That's 30 over the course of day, not 30 for lunch and 30 for dinner," Mr Andrews said. "I understand that's large enough some families but others will need to do some juggling.
"It is important to acknowledge that end of the year family functions are in latter part of year, not always on Christmas Day."
Business who have had staff working from home will be able to have 25 per cent of its staff working onsite from November 30.
"I want to be clear though - unless you're one of those people nominated by your employer - you need to keep working from home," Mr Andrews said.
Other changes taking immediate effect include the number of visitors permitted at your home increasing to 15 per day.
"That can be split across different times - ten for lunch, and five for dinner - as long as people are sticking to a total of 15," Mr Andrews said.
Related: Victoria records 23rd 'donut day'
Outdoor gatherings in a public place will increase to 50 while weddings, funerals and religious ceremonies will be able to host 150 indoors.
Patron caps in hospitality venues will change with smaller venues able to host up to 50 customers inside while bigger venues can host 100 people inside and 200 outside.
The density limit will change to one person for every two square metres.
Cinemas, galleries and museums will also be able to host up to 150 people indoors.
Community clubs, contact and non-contact sports can begin for adults and children with 150 people indoors with a group size of up to 20 and 500 outdoors with groups of up to 50.
Mr Andrews said the eased restrictions were the result of an amazing effort of Victoria.
"It's an incredible thing but it is fragile," he said. "(Having) 23 days of zero cases is not same as having a vaccine.
"We have to keep playing our part and take this seriously. We have seen that this thing can come back and come back quickly."
Mr Andrews announced the next date for potential changes would be December 6.
"The key message is Victorians have done an amazing job," he said. "We have built something so precious but it is fragile and we each need to play our part.
"We have to work hard to keep numbers low and make sure 2021 is vastly different to 2020."
Bendigo Health chief executive Peter Faulkner said while masks would not be compulsory in outdoor areas, he said some people may continue to wear them.
"I expect some people will choose to continue to wear a mask outside and people shouldn't feel bad about that," he said. "Masks are one (protective) measure we can take. We also continue to take all public health measure including social distancing, good hygiene and cough etiquette.
"Certainly, on hot days, it will be good to have (the option to breath) direct air."
Mr Faulkner said with gathering restrictions also eased, it was important to continue to take care and gt tested if coronavirus symptoms present.
"We all welcome the easing of restrictions including being able to have decent family gatherings at Christmas," he said. "It is great reward for the great effort Victorians made a through the lockdown period.
"(But) we saw in South Australia last week that it is dreadful infection that gets out of the box quickly. We need to be careful and people need to be test if they have any symptoms.
"We (would also) encourage people who may not be feeling on top of the world come Christmas to not visit people if you are not well. Stay home and do the right thing. The last thing you want to do is infect other members of your family."