Wayne Hennessey's error gifted Joleon Lescott the winner as Villa claimed a 1-0 victory over Crystal Palace on Tuesday evening.
It ended their 19-match winless run in the Barclays Premier League and closed the gap to safety to eight points.
However, Villa remain bottom of the table and Garde refused to get carried away following their first league win since the opening day of the season
"We have to stay very humble it's only three points," he said after his 11th match in charge.
"We are far away from what we all want
Of course this victory could give us a little bit more confidence
"I also want to insist on the way we played
The different between Villa and Palace didn't reflect the table.
"I want my players to be more confident and hopefully that's what will happen."
Lescott was criticised at the weekend after being accused of playing a game involving chewing gum with Brad Guzan while they were substitutes during the 1-1 FA Cup draw at Wycombe.
The pair received some boos from the crowd before the game, but Garde insisted he needs the former England international in his side.
"I didn't speak with Joleon about this stuff," he said
"I am pleased with the way he gave an answer but not an answer regarding this fact.
"He is an experienced player and should be very important for the team.
"He spreads serenity and calm
He knows the game and is a player I can rely on
He is not as young as he used to be, I try to manage him and others as best I can."
Palace have now failed to score in their last four league games and the closest they came at Villa Park was when Wilfried Zaha hit the post inside the first minute.
Eagles boss Alan Pardew conceded Villa deserved to win and refused to criticise Hennessey for his error.
"Wayne's first touch puts him in trouble," he said
"He has been brilliant for us and his mistake cost us the goal but we made mistakes in other areas all night.
"They go unnoticed - as much as the goal is going to be a highlight for everyone else.
"I said to the players, there are not many times I can turn round and say we just couldn't get going
"I don't want to take too much away from Villa
They deserved the win and my team didn't play well
"The Premier League would not want to lose a club of this magnitude
I hope they can resolve it."
Source : PA