Alfredo Morelos was sent off for the seventh time in his Rangers career but Nikola Katic's header secured victory over Celtic.
Rangers cut the gap to leaders Celtic down to two points in the Scottish Premiership thanks to a 2-1 victory at their Old Firm rivals on Sunday, despite another red card for Alfredo Morelos.
Nikola Katic's 56th-minute header secured Rangers their first league triumph at Celtic Park in 10 attempts, raising hopes for Steven Gerrard's squad they may finally be able to end Celtic's long run of dominance in the top flight.
While the reigning champions remain at the summit, their Glasgow neighbours have a game in hand as they bid to lift the title for the first time since 2010-11.
The only setback for Gerrard was the late dismissal of Morelos, who was sent off for the seventh time in his Rangers career after picking up a second booking for diving during added time.
Both teams had scored during a crazy nine-minute spell in the first half that saw a missed penalty, a Rangers opener and a fortuitous equaliser from a familiar scorer in these fixtures.
The visitors were on top before Katic presented Celtic the chance to take the lead by holding on to Christopher Jullien's shirt at a corner, the infringement spotted by referee Kevin Clancy.
However, Allan McGregor helped out his team-mate by turning away Ryan Christie's spot kick - and Rangers went ahead soon after when Ryan Kent swept in Borna Barisic's square pass with his left foot.
Celtic had struggled in attack but benefited from a fortunate break to draw level, Callum McGregor's shot going in off Odsonne Edouard, with the deflection - seemingly off an arm - wrongfooting goalkeeper McGregor.
Yet Rangers recovered from the unfortunate setback to retake the lead early in the second half, Barisic again the provider as his corner picked out the run of fellow Croatian Katic.
Steven Davis preserved the visitors' narrow advantage when he headed off the line from Jullien for a second time in the contest, while Morelos - still yet to score against Celtic - received his marching orders too late for the hosts to capitalise on the man advantage.