The boss is under serious pressure after Swansea snatched a late 2-1 win at Villa Park to condemn the hosts to a sixth straight Barclays Premier League loss.
Villa are second bottom of the table and will fall to the foot on Sunday if Sunderland beat Newcastle in the Tyne-Wear derby.
Andre Ayew struck three minutes from time to seal victory for the Swans after Gylfi Sigurdsson cancelled out the opener from Jordan Ayew - Andre's brother.
But Sherwood is not worried about the axe.
"No, it's out of my control," said Sherwood, whose side are four points from the safety line and winless since the opening day of the season
"I can only control what I can control, and on Monday I'll be working as hard I've done since I came to this club
"We'll stay in the Premier League but we need to improve, obviously
It's not going to happen overnight.
"It ain't an easy job this but it's one I love and one I'm not going to give up very easily."
Sherwood joked he will avoid a call from the board if it wants to sack him.
"I'll just turn my phone off," he said.
Sherwood admitted, though, his players are lacking confidence after they failed to build on Ayew's second-half opener before sinking to a killer blow.
He said: "We can't do any more as coaches and managers
We're working as hard as we possibly can and the players can't do more regarding effort
They can only give us what they can give us
We can't turn them into superstars.
"It was a toss-of-the-coin game
That's how close it was and that's how close all of our games have been."
Swansea were winless in six games ahead of their Villa Park trip and boss Garry Monk insisted he ignored any hype surrounding their slump this week.
He said: "It's a case of knowing what these players are capable of
I don't focus on the circus which goes on outside
I make the players focus on football
"You win and lose games no matter what
It's the nature of the sport we're in
It's important we reach our standards
It was never going to be easy because of the circus which has gone on this week.
"In terms of the criticism I was receiving myself, you feel for Tim
He is an excellent coach and manager and no-one will be working harder than him to put it right."
There was a flare-up in the tunnel at the end of the game as tempers boiled over following a clash between Micah Richards and Federico Fernandez in the first half, but Monk played it down.
He added: "The boys came in and they were fine
Everyone was talking about the game."
Sherwood felt the Swansea defender should have been sent off after the clash during the game, though.
"The ref has got to send him off if he sees it," the Villa boss said
"He (Fernandez) stuck his head in."
Source : PA
Source: PA