One of the things we love about traveling is experiencing contrasts, especially the contrasts between tradition and modernity or between nature and urban life. Big cities offer some the world’s most intense contrasts–Mercedes Benzes weaving around ox carts in Mumbai; street stalls side by side with tablecloth restaurants in Singapore; luthiers and lapidary located a block from Lululemon in London.

Persian Gulf cities lack those kinds of contrasts, but boast others. They are captivating and boring at the same time, expensive and cheap at the same time, and filled with expatriates from around the world congregating in corporate establishments. Come to think of it, the cities of the Gulf are a lot like Singapore.

In the Gulf, be prepared to see extreme wealth and ridiculously ambitious building projects like the eight stadiums being erected simultaneously in Qatar, along with a monorail to link them for the 2022 World Cup. Even the indentured laborers from the subcontinent live better than their families at home. Visible poverty is nonexistent in the Gulf.

Whatever you may think of their sustainability or ethics, the big cities of the Persian Gulf feel connected to the global community and to the future of humanity. Each of these nations understands their oil money will eventually dry up, and each has diversified in its own way. Arab foresight to invest oil money in everything from fine art to tourism has required cooperation with every place on the planet with one notable exception I needn’t mention.

Doha, Dubai, and Abu Dhabi are changing rapidly. A few years since our first visit to Abu Dhabi and we noticed newly installed bike share schemes and even bicycle lanes. Dubai lags behind in this regard, although Doha’s corniche is bike-friendly.

Unfortunately, the big Gulf cities love their manufactured and artificial experiences, like Heritage Village in Abu Dhabi, replete with fake archaeological ruins. So it’s outside of the big cities where you find real heritage, real archaeology, and really old traditions. All of these countries have a wealth of tourism opportunities beyond 4×4 dune rides, but few visitors do much more outside Dubai or Doha. If you’re in Dubai for more than a day, we urge you to at least rent a car and drive to Al Ain, or take a trip to one of the other Gulf nations like Bahrain or especially Oman.

We spent a total of two weeks in the Persian Gulf in October-November 2016. The trip included the United Arab Emirates (UAE), Bahrain, Oman, and Qatar.

Here are a few highlights from our trip:

Best Archaeology

Of the countries we visited, Bahrain has the most magnificent archaeology. This was the ancient Dilmun civilization, which at one time was an ancient Greek outpost too. I loved the contrast between ancient and modern here at the Bahrain Fort in the country’s capital, Al-Manama.

The Bahrain Fort offers layers of archaeological record, from the Dilmun to the Portuguese occupation in the 17th century.

 

The national museum in Manama also has exhibits on the Dilmun civilizations, and you can also visit ancient burial mounds right in the city.

Best Nature

Oman has the best nature experiences. The lunar landscapes of Oman are sublime and will etch themselves into your memory. The goat farming and fishing villages in Khasab could have been located in Greece or Cyprus.

Khasab, Oman takes first prize for nature encounters.It juts out into the Gulf with treeless limestone fjords, home to bottlenose and humpback dolphins. Khasab is an exclave of Oman, a little bit of land non-contiguous with the rest of the country.

The capital city Muscat is a timeless place that has retained an old-world charm unique to the entire region.

The Sultan of Oman has no wives or children and seems an all-round good guy. He does a deft job running the country, to preserve its character and aesthetic. All buildings in Muscat must be small, designed in the traditional architecture, and painted beige or white, so the skyline does not compete with its showcase mountains. Oman has all the Arabian mystique you could hope for, from fortresses to date palm oases.

This slideshow requires JavaScript.

Best Souk

Every city in the Arab world has a solid souk, the world’s original shopping mall. Each souk has a unique character, and unlike modern malls, no two souks are alike.

The standout souk in the region is actually in Doha because of its tastefully rendered faux antique architecture and most of all its cool temperatures. Not only is it less exposed to the sun than other souks we visited, but the stalls have been upgraded into stores with air conditioning spilling out a welcoming breeze onto the walkways.

Best Museum

Doha also has the area’s best museum, at least until the Louvre in Abu Dhabi opens (it is running behind schedule; current estimate is 2017).

Doha’s Museum of Islamic Art is contained in an I.M. Pei-designed structure and houses a stunning collection from around the world, with highlights from Persia and India. If you do one thing in Doha, this would be it. Even if you don’t like museums, hey, it’s air conditioned and it’s got a great view.

Best Mosque

Well, that’s an easy one. The Grand Mosque in Abu Dhabi is a wonder of the world that actually manages to feel and look like the Taj Mahal, replete with marble inlaid with semi-precious stones and surrounded by pools of water. It’s also got the largest carpet in the world (an acre and a half!), and the largest chandelier in the world. It’s showy and ostentatious like the rest of the city, and yet the mosque manages to feel sublime and respectful.

Best Food

Unfortunately, we did not get as deep into local food culture as we usually do because we were traveling on a cruise. We did try Omani helwa and traditional Omani cuisine. And of course, different types of dates.

We had one classic Arabian desert feast in Qatar.

But the food highlight was at a Keralan Restaurant called Paragon in Dubai. We got crab curry, two fish curries, two veg, roti, and more and the entire meal for four people cost as much as one main dish in a hotel restaurant anywhere in the city.

So that’s probably the biggest contrast you’ll find in Dubai. At hotel restaurants, standard per person price without drinks will be about $50 a head. If you do your homework and head to an Indian neighborhood, you can eat better food at literally a tenth of the price. And that, folks, is why we travel.