Deon Burton raised the visitors hopes of posting their first win in five games when he fired Rotherham ahead against his former club after just eight minutes.
But the lead lasted just three minutes, until Kevin O'Connor equalised with a penalty, before Rankin's close-range strike knocked the wind out of Rotherham's sails.
In a breathless opening, Burton beat home keeper Stuart Nelson to a long goal kick and headed into an empty net for his eighth goal of the season.
Rotherham midfielder John Mullin tested Nelson with a header in the first minute before Lloyd Owusu was kept out by Gary Montgomery two minutes later.
O'Connor, in his first start in four matches following injury, levelled from the penalty spot after Montgomery brought down Owusu in the area.
Sam Sodje headed over the bar from five yards out midway through the first half but Rankin made no mistake with a tap-in nine minutes before the break.
The home side had a huge let-off on the hour when Michael Keane was set free inside the area but Nelson scrambled to deflect the ball over the bar.
Both sides had late chances as Keane failed to capitalise on a defensive error before O'Connor was kept out by a stunning Montgomery save.
Owusu could have put daylight between the sides on 75 minutes when he robbed the ball from the visiting goalkeeper's feet but was unable to get a shot away.
And he proved to be a constant danger to Rotherham as he fired over the bar three minutes later.