GPT-4o – surprisingly bad results for a geospatial question

My prompt was:

Where are the best places to road cycle in Oakland? I want a roughly 10m route that is scenic and starting at lake Merritt

The instructions I received are amusingly off kilter. For example, it recommends heading north on Grand then taking a left on El Embarcadero, which is impossible. Same with taking a left on Trestle Glen from Lakeshore. And so on.

I asked it to draw that route on a map. I was impressed that it banged out a small python program, but the points on the map didn’t seem to correlate with the points in the prompt response, and the “route” was connecting lines as the crow flies.

Anyway – incredible that it made it this far, but I’d be curious as to whether anyone has prompting tricks for improving response quality. Perhaps geospatial reasoning is simply a soft spot?

I could imagine many interesting data sources and many extremely cool queries that are far beyond the trivial text boxes of mapping applications. “Find me a route from San Francisco to LA for an EV with 300mi range where every charging station is within walking distance from an In-n-Out”. You know, important stuff like that!