Matteo's men fell 10 points behind Sky Bet Championship leaders Huddersfield after being held to a 2-2 draw with Nottingham Forest despite dominating the game at Villa Park.
Two goals in two minutes from Ross McCormack and Rudy Gestede put them 2-1 ahead with 16 minutes to go only for Henri Lansbury to rescue a draw for Forest on Sunday.
Forest had just three shots to Villa's 26 and netted twice as Apostolos Vellios opened the scoring before Lansbury equalised with three minutes left and Hildeberto Pereira was dismissed
Villa hit the woodwork twice through McCormack and Jordan Ayew but are left with just six points from their first six league games.
The draw means Villa have now blown leads in four of their seven games in all competitions this season after also losing at Luton and Bristol City and drawing with Huddersfield.
Di Matteo said: "We need to win these games, it's as simple as that
That's why I was shaking my head at the end of the game, it's really difficult to take
"I'm frustrated by the result because the performance was very good
We dominated the whole game
We hit the post, the bar, the keeper makes a couple of world (class) saves but we are moving in the right direction
"With time we will learn and improve in these situations where we will be able to finish games off
Right now we have made a lot of changes in the transfer window
It's really frustrating but the performance was really good."
Vellios scored his first Forest goal - and first in England since 2011 - when he netted from 25 yards against the run of play in the second half
McCormack and substitute Gestede then turned the game for Villa only for Lansbury's fine finish to steal a point for Forest.
It was Forest's first away point in the league this season and even head coach Philippe Montanier admitted Villa deserved to win
"It was a tough game for us because Villa started very strongly," he said
"It was better from us in the end of the first half but we conceded a lot of opportunities to score
Villa deserved the victory but sometimes you can win a point when you are a little lucky
We showed good character."
Forest substitute Pereira, who was already on a yellow card, was dismissed for apparently celebrating with the Forest fans in the aftermath of Lansbury's leveller and Montanier said he would fine the midfielder if he deserved it
He added: "He knows the rules, it's not to jump into the crowd so we have to check on the video if he did
If he did then it's a big fine for him
If you are near the crowd it's different to jumping."
Source : PA
Source: PA