E1: Cloud-Native Geospatial Files

– Use fsspec and obstore (and the s3fs backend) to read and write remote files as if they were local
– Stream data tile-by-tile (no full file download)

– Use Rasterio and rio-cogeo to transform plain GeoTIFFs into COGs
– Tune internal tiling (blockxsize, blockysize) and overviews for optimal HTTP-range reads
– Apply best-practice compression (DEFLATE, ZSTD, etc.) and predictors

– Read Shapefiles (or GeoPackages) in-memory with GeoPandas (and VSIMEM)
– Write out GeoParquet with PyArrow, setting row-group size, compression, and GeoParquet version
– Understand how row-grouping and spatial keys (geohash, Hilbert) impact query performance

– Leverage Rasterio’s MemoryFile (VSIMEM) for zero-scratch-disk conversions
– Use io.BytesIO or Obstore’s file-like readers/writers for pure in-Python pipelines

– Choose appropriate tile sizes (256 × 256 vs. 512 × 512) based on use case
– Select compression codecs (lossless vs. lossy) for data vs. visualization needs
– Partition vector data (quad-tree, k-d tree) for spatial pruning

– Inspect COG metadata and validate with rio_cogeo.cog_validate / cog_info
– Read GeoParquet footers to assert schema, row groups, and GeoParquet metadata
– Write automated tests (pytest) that verify your outputs without downloading entire files

– Package your workflow into a Gitpod/GitHub Codespaces environment
– Automate dependencies (GDAL, rasterio, geopandas, rio-cogeo) via Docker/Gitpod config
– Integrate CI checks that grade your notebook outputs and issue badges