Cole moved level with Kenny Sansom on 86 caps - the most won by an England full-back - during the goalless draw with Montenegro this week.
It will be a big test for England Under-21 international Albrighton who has enjoyed his first extended spell of first-team football this season.
But he is relishing the challenge rather than being daunted by the prospect.
Albrighton said: "I wouldn't say it is daunting, it's exciting more than anything to play against players like Ashley Cole.
"Every player in that (Chelsea) team is a world class player and, if you can play well against them, you can play well against anybody.
"Ashley Cole might be the best left-back in the world at the moment but I can't think too much about that.
"I've got to go out there and play as I would if I'm playing in a reserve game against a young left-back from the youth team.
"I've got to go and play the game, not the occasion."
Albrighton is expected to be fit to face the Blues after a slight groin strain forced him to miss the Under-21 European Championship play-off second leg clash in Romania.
Source: Team Talk
Source: Team Talk