In addition to Tuesday's Cup exit, Schalke have won just once in 10 games and sit in 17th place following their return to the Bundesliga after winning the second division last season.
In a statement released on Wednesday, the club thanked Kramer "for his professional and dedicated work". Sporting director Rouven Schroeder said action needed to be taken when it became clear Schalke's position in the top division was under threat.
"After promotion, we all committed to one goal - staying up in the Bundesliga. Right up until the end, we were convinced we could turn the tide in the current situation".
Board member Peter Knaebel said the club's recent away performances "were not worthy of FC Schalke". "We have to improve significantly in all areas in order to be able to achieve our main goal - staying up in the league".