Dover Castle, a medieval fortification which also houses a Roman lighthouse (the UK's oldest), an Anglo Saxon church and gun emplacements from WWI and WWII. Indeed, the firing of the WWI guns was the highlight of our visit. The castle keep itself, built during the reign of Henry II, is interesting but adorned with tacky exhibits.
Going underground, tours whisk you through a subterranean hospital and Operation Dynamo, which documents the Dunkirk evacuation during WWII. During busy times queue early. It's a pleasant stroll around the walled perimeter taking in the church and lighthouse but don't forget the firing of guns, they're a must.