The England winger believes it is easy to blame Houllier and blame him for Villa's slump when the squad have to take responsibility. Villa are only one point above the bottom three ahead of the Barclays Premier League clash with Everton at Goodison Park.
Downing is also adamant Houllier has not made any wholesale changes in terms of bringing in new rules and, in any case, players have to adapt to his demands.
The former Middlesbrough player said: "The manager is fine. We all get on with him. It is easy to point at him and say `it's his fault' because he changed a few things.
"I had it at Middlesbrough before. `It was the manager's fault, it was the players' fault etc, it was his fault because he wanted to leave etc'.
"You've just got to stick together and blank it out because when it is going well, everyone will take the praise. It happens at every football club and we've got no problem with the manager. No matter who the manager is, it's down to the players at the end of the day to get results and not him."
Source: PA
Source: PA