The Seasiders let slip a two-goal lead on Saturday and missed the chance to advance at the first attempt, but there would be no repeat as they overcame higher-level opposition at a raucous Towbar Express Stadium.
Jacob Gratton and Junior Mondal - scorer of their two goals in the original fixture - again made their presence felt to put Whitby 2-0 up at half-time.
Chelmsford forward Mo Bettamer scored 15 minutes from time to set up a nervy finish, but the Seasiders wrapped up victory in the fifth minute of stoppage time as Mondal grabbed his second of the game.
"As I've been reminded, a whole new generation of Whitby fans have the pleasure of going into the first round proper," manager Nathan Haslam said post-match.
"I can't praise my lads any more than what the fans have given them.
"It's been a journey. That was the objective at the start of the season - you always want an FA Cup run, and to a man in each round, they've delivered and put their chances away.
"We were unlucky on Saturday and should've put the game to bed then, but we've gone again and done exactly the same tonight.
"The class that we showed, the courage, bravery, aggression - everything that we did was on the front foot. We did that throughout the tie and were thoroughly deserving winners."
Haslam becomes just the sixth Whitby manager to lead the side to the first round proper - following in the footsteps of Bob O’Brien, Tony Lee, David Harvey, Peter Creamer and Harry Dunn.
Their reward is a trip to League One side Bristol Rovers, with the tie set to be played between Friday 3 and Monday 6 November.
They also become the third Northern Premier League side to reach the first round of this year's competition, joining Marine and Worksop Town.
Marine revisited their FA Cup heroics of 2020/21 to again reach the first round, where they will face League Two side Harrogate Town. It comes after they registered a superb 1-0 victory away at National League side FC Halifax Town, Louis White scoring the game’s only goal on seven minutes.
Worksop, meanwhile, edged a topsy-turvy game at home to National League North side Boston United, winning 3-2.
Their reward for that victory is a trip to League Two leaders Stockport County.
The fourth qualifying round wins net each of the three NPL sides £9,375 in prize money, with £41,000 on offer for teams who advance to the second round.
Photo credit: Whitby Town