Although Newhaven is often described as a bit grim, its working harbour, home to the Dieppe ferry, makes an ideal base for exploring the cliffs taking you to Peacehaven, Saltdean, Rottingdean and Brighton.
The wide arc of the harbour has plenty of parking overlooking the sea. There's plenty going on so bring binoculars. You can fish here while in the sanctuary of your own car (nice) or wander along the deserted beach. A good spot for fossil hunting. Overlooking it all is the Newhaven Fort. A great day out, it's open from February to November but check the website.
I like it here. If you need some downtime, bring a flask and watch the waves go by. OS map Explorer OL11 Brighton & Hove highlights the paths around the cliffs.