Denver and El Paso are about 557 miles apart by road. Split the difference and you arrive near Albuquerque, the city closest to the halfway point between them. That puts roughly 7 hr 1 min of driving on the Denver side and 4 hr 49 min on the El Paso side — the fairest single meeting point among the cities near the middle.
That's a half-day drive from each side, so Albuquerque suits an overnight or a weekend rather than a quick coffee — long enough to want a reason to stay, short enough to drive.
If Albuquerque doesn't have what you're after, Tucson and Phoenix are also close to the midpoint and worth a look — each keeps the drive reasonably balanced between Denver and El Paso.
City descriptions adapted from Wikipedia (CC BY-SA); photos via Wikimedia Commons, credited above.