Early on the warm morning of January 24, 1995, John Howard and Alexander Downer were in earnest conversation in a locked room upstairs in a motel in Ferntree Gully, east of Melbourne.

In the courtyard below, Phillip Ruddock was intensely interested. He wandered among journalists, nervously earnestly inquiring if they knew what was going on.

No one could have guessed it, but the conversation upstairs was about to change the trajectory of Ruddock's political career.

Downer and Howard, in the hours before a shadow cabinet meeting, were hammering out the terms of a leadership hand-over.

Downer's peculiar 8-month period as leader of the Opposition was ending. Howard's rise and rise was about to begin.

No one could quite understand Ruddock's consuming interest in Howard's new coming. Could it be fear?

Ruddock had been an outsider for almost all the 22 years he had been a Liberal parliamentarian. Howard, it was generally believed, couldn't stand him. His only promotions to shadow ministries had been under Howard's nemesis, Opposition Leader Andrew Peacock

It is all but forgotten now that Ruddock spent the entire first half of his more than 42-year career as a small-l Liberal "wet", forever involving himself in the lonely business of concern for human rights, for those afflicted by war in Cambodia and other dreadful places; by apartheid in South Africa; by injustice across the world.

He was a leader of Amnesty International's parliamentary group and was one of the very few Liberals who belonged to Parliamentarians Against Apartheid.

Worse, he had infuriated and publicly embarrassed John Howard when, in 1988, he and a tiny knot of fellow moderates crossed the floor and voted with the Hawke Government on a motion to oppose any form of racial discrimination in immigration. It was shortly after Howard had declared he was concerned Asian immigration was too high.

Yet, when a few days after the Ferntree Gully meeting, Howard became Liberal leader and went on the following year to become Prime Minister, Ruddock's time had come.

He had, it seemed, found ambition. And Howard, it seemed, knew it, and recognised an opportunity.

In short order, Ruddock became a hard-edged Howard insider - the immigration minister who enthusiastically embraced and broadened Labor's policy of mandatorily detaining asylum seekers. Under his watch, asylum seekers, including children, were locked behind razor wire in Australia's deserts.

He railed against "queue jumpers", introduced Temporary Protection Visas and was an architect of Howard's "Pacific Solution".

Supporters who had thought of Phillip Ruddock for years as no more than a member of the small group of Liberal MPs dedicated to being the social conscience of their party suddenly wanted his Amnesty badge stripped from him. Even a daughter, Kirsty, publicly turned against him over the mandatory detention of children.

As Attorney-General, he introduced the Marriage Legislation Amendment Bill, which ever since has been used to prevent same-sex marriages. The old "wet" might still wear that Amnesty badge, but he had become a hero of the Liberal conservatives.

He did much more, of course, and insists his work was always in the cause of human rights, but these are the things he will be remembered for.

Ruddock, the son of a Liberal Minister in the NSW parliament, Max Ruddock, entered federal Parliament as the member for Parramatta, NSW, at a by-election in September, 1973. He was 30. Gough Whitlam was prime minister.

Now, aged 72, long the Father of the House and the longest-serving Australian parliamentarian after Billy Hughes, he is calling an end to his career.

It has been on the cards ever since Tony Abbott sacked him as chief government Whip, apparently because Abbott believed he hadn't kept him adequately informed of the numbers rising against his leadership.

Phillip Ruddock had always kept his own counsel, and kept his motives close to his chest, as those who saw him in the courtyard of the Ferntree Gully motel all those years have ever since had cause to ponder.

He may have become the longest-serving federal politician in the land, but there seemed always something unknowable about him.

Follow us on Twitter