Hardy took over after Darren Williams left the club in December 2015, and the following season led the club to North Riding Senior Cup success after a 3-1 win over Pickering Town at Middlesbrough's Riverside Stadium.
On his decision, Hardy said: "There are many reasons as to why I’ve arrived at this decision and after giving this some long and hard thought I’m 100% confident that this is the correct decision for myself and Whitby Town.
"It's not something I decided on the back of Tuesday night's defeat, I have given it some good time to think about it and now feels the right time to leave.
"I've thoroughly enjoyed my time as manager of the football club. There have been plenty of ups and downs and I will look back on my tenure fondly.
"I'd like to think it will be classed as a successful period in the club's history. We won some silverware, kept the club in the division after taking over and came close to the play-offs on a couple of occasions."
Lee Bullock and Nathan Haslam will take charge of the side for this weekend's fixture against Warrington Town at Cantilever Park.