Allen, 31, limped off only seven minutes into the match against Group E favourites Belgium on Wednesday -- a miserable return to international action for the influential player.
Allen, who had not played for Wales since 2019 after suffering an Achilles injury, was one of their star performers as they reached the Euro 2016 semi-finals.
"We've lost Joe, which is a shame," said Page. "It is a big loss for us. It's disappointing not to have him. He'll have to go back to the club (Stoke) now and they'll assess him."
Wales play a Cardiff friendly against Mexico on Saturday while the Czechs, 6-2 winners away to Estonia in their opening qualifier, are at home to Belgium. Wales host the Czechs on Tuesday.