Permit system delay as Victorians return from NSW