New-look Germany, who have set their sights on the 2026 World Cup after their quarter-final run at the Euro on home soil in July, no longer have any 2014 World Cup winners in the squad following the international retirements of Thomas Mueller, Toni Kroos and Manuel Neuer.
Former captain Ilkay Gündogan also ended his international career.
They took the lead in their Group A3 encounter with Niclas Füllkrug's tap-in from midfielder Musiala assist. With Hungary pushing for an equaliser, Musiala struck on the break in the 58th, completing a quick counter-attack from a visitors' corner.
Florian Wirtz then drilled in their third after a one-two with Musiala in the 66th before Havertz hit the woodwork once more.
Man-of-the-match Musiala then sent through Aleksandar Pavlovic in the 77th to slot in.
Havertz still had time to get onto the scoresheet himself with an 81st minute penalty.