The Australian Defence Force has refused to provide any personnel to help Victoria Police man the border checkpoints.
The Victorian government had requested 200 to 300 on-the-ground personnel but the request was denied.
Defence said it already had 235 personnel supporting Victoria's hotel quarantine program and any additional ADF backup would "not provide direct support to vehicle checkpoints on the NSW/Victoria border".
Defence offered accommodation to police on the border and 50 members to assist with planning, but Mr Andrews was displeased and disappointed.
"We made the request because we need them, if we're not going to get them... then people can reflect on that if they choose," he said.
Mr Andrews said more Victoria Police officers and SES volunteers would man the border over Christmas.
Asked if it was fair to expect volunteers to man the border, DHHS COVID-19 response commander Jeroen Weimar said the SES had always been a crucial part of the local response.
"We know the SES and CFA are increasingly busy with other priorities, so we will all have to balance our resource needs," he said.
"We have a fair stronger capability now, within the Victorian health system, than we did six months ago."
Mr Weimar said the ADF had provided good support throughout the pandemic and DHHS would continue to request their support.
Member for Norther Victoria Jaclyn Symes said while the ADF's initial response didn't provide as many personnel as hoped, the government would continue to have conversations with them
"We've got 700 police dedicated to this operation at the moment, supported by SES, we have reached out to ADF and asked for some support," she said.
"I understand that not as much support has been forthcoming at this moment as we would like.
"We've made a similar request to what NSW did, and the first response is not as many as what we would like [is available]."