The 45-year-old's appointment comes just three weeks after he led modest Argentine side Defensa y Justicia to the Copa Sudamericana title.
The former striker, a Premier League winner with Chelsea and three-time Serie A champion with Inter Milan, has signed a two-year contract.
"He has a winning history, he was a great athlete and is a very promising coach," said Sao Paulo president Julio Casares.
"We have a commander with a winning mentality," he added.
Crespo replaces Brazilian Fernando Diniz, who was fired after a string of poor results that dropped Sao Paulo off the top of the Brazilian championship.