Kleindienst's brilliant long-range strike in injury time handed the underdogs a famous win as the two south-western clubs met for the first time in Germany's top flight. Stuttgart, who had been the surprise package of this season with seven wins from their first eight games, have now lost two regional derbies in a row after defeat to Hoffenheim last weekend.
They remain in third after Sunday's loss, but have slipped seven points behind league leaders Bayer Leverkusen and five behind champions Bayern Munich in second. Heidenheim had 13 shots on goal and hit the woodwork twice in the first half, while Stuttgart almost took the lead when Angelo Stiller grazed the bar with an elegant free-kick.
Yet Kleindienst pounced on a defensive error late on, lobbing goalkeeper Alexander Nuebel from just inside the Stuttgart half to seal the win.