Why GeoParquet Is Not A Cloud Native Format

(Geo)Parquet is an extremely useful column oriented data format. When working with local, massive data sets having many millions of features, the performance of this format is second to none.

"The whole idea of cloud native data is to transfer the minimal amount of data from the cloud to the client."

The whole idea of cloud native data is to transfer the minimal amount of data from the cloud to the client. Put another way, a subset of the original source is transferred, this is either truncated or a sampling of the original data set. You would never transfer millions of features (or pixels) over the internet without serious performance consquences.

Ideally, on the client you would be dealing with a few thousand features or less. In modern JavaScript, doing spatial analysis or other types of processing on that many features is often trivial.

"GeoParquet's columnar format provides no performance benefit in a cloud native setting."

GeoParquet's columnar format performance shines on millions of features, not thousands. GeoParquet's columnar format provides no performance benefit in a cloud native setting. Your development environment will require the non-trivial addition of packages like Apache Arrow. The reality is GeoParquet offers no advantage over an SOZip'ed cloud native .fgb file (FlatGeobuf). SOZip reading and FlatGeobuf read/write are both internal to GDAL, GeoParquet is not.

"GeoParquet requires you to download the entire data set to extract a subset of that data."

GeoParquet requires you to download the entire data set to extract a subset of that data. This is not a cloud native format. It's disheartening to see GeoParquet being pushed as some extraordinary cloud native format. No cloud native data sources will benefit from Parquet. Let's not manipulate folks to flush resources down another 'Big Data' rabbit hole they don't need.

Note, the term cloud native refers to processing cloud data with a app/web client without intermediate backend servers/services.