Liverpool beat Aston Villa 4-1 in the third round on Friday, though the hosts were forced to field a team comprised of youngsters as a result of a significant COVID-19 outbreak.
Jurgen Klopp's men ended their long wait for a Premier League title last season and tasted Champions League glory in 2019, though they have not won the FA Cup since 2006.
That is a run right-back Williams, who played the full 90 minutes at Villa Park, is eager to change.
"The FA Cup is not a cup we have done that well in, so I think this year we are looking forward to pushing on and to get to the final and hopefully win it," he told the club's website.
"It's always good to score goals and that just gives the lads who scored the confidence to go and do that in the next game, keep adding them goals to the charts.