Liverpool won for the 17th consecutive league game thanks to Milner's dramatic contribution to a thrilling clash at Anfield.
Jurgen Klopp's side ruined Leicester manager Brendan Rodgers' first return to Liverpool since being replaced by the German in 2015.
By extending their remarkable winning streak, Liverpool have piled pressure on second placed Manchester City.
Liverpool took the lead in the 40th minute when Milner's long ball was misjudged by Leicester defender Jonny Evans and Sadio Mané clinically slotted in his eighth goal of the season.
Mané has scored exactly 50 goals in 100 Premier League games for Liverpool, making the Senegal winger the 10th player to reach that milestone for the Reds.