Raleigh and Washington are about 233 miles apart by road. Split the difference and you arrive near Richmond, the city closest to the halfway point between them. That puts roughly 3 hr 28 min of driving on the Raleigh side and 2 hr 25 min on the Washington side — the fairest single meeting point among the cities near the middle.
At this range, Richmond works for a same-day meetup: close enough from both Raleigh and Washington to meet for lunch or an afternoon and still be home by evening.
If Richmond doesn't have what you're after, Charlotte and Philadelphia are also close to the midpoint and worth a look — each keeps the drive reasonably balanced between Raleigh and Washington.
City descriptions adapted from Wikipedia (CC BY-SA); photos via Wikimedia Commons, credited above.