It was not an impressive game and the tie was heading for extra time until Hitzlsperger scored with an explosive left-footed shot in the 75th minute.
Leicester manager, Micky Adams, was true to his word as he made ten changes from the side which dramatically collapsed at Wolves last Saturday, with Paul Dickov the only man to retain his place.
Villa were often the more dangerous as they demonstrated in a five-minute spell when Foxes goalkeeper Danny Coyne had to save two shots from Juan-Pablo Angel and another effort from Darius Vassell.
Vassell was a lively character as his speed and ball control often tested Leicester's makeshift defence but later became a little disheartened and was replaced by Hitzlsperger.
The most dangerous moment to Villa's defence occurred when Dickov failed to make contact with a skidding cross from Brian Deane.
The second half continued to be a frustrating affair as neither side could make a major impact as the game highlighted deep-rooted problems in both teams.
But the tie suddenly exploded in the 75th minute with Villa storming ahead. Gareth Barry put Hitzlsperger in possession and the young German, noted for his powerful shot, unleashed a 25-yard rocket that whistled past Coyne.
Villa boss O'Leary said afterwards: "Hitzlsperger's goal was a bit special. It was very welcome as scoring goals has been our big problem this season. We deserved to win but I would have liked a few more goals."
His Foxes counterpart Adams added: "If we had given the same sort of performance in the second half at Wolves last Saturday, there would have been no problems.
"Now I am not sure of my first 11. The display tonight has certainly given me and the players a lot of food for thought."