hypertidy-blog

Writing controlledburn output in parallel

controlledburn

rasterization

Antarctica

GDAL

Zarr

mirai

The serial COG write loop from the last post, replaced by 24 workers writing disjoint Zarr chunks. The 31-billion-cell grid on disk in 28 seconds of 9.5 MB.

Rasterizing ice-free Antarctica

controlledburn

rasterization

Antarctica

GDAL

Sparse rasterization of 25,954 Antarctic rock outcrop polygons onto a 31-billion-cell grid — in under a second.

s2 guide

s2

wk

spatial

rgl

A field guide to the s2 R package: what the three type families are, how to move between them, how to get raster and vector data into cell groupings, how to plot on a real sphere with rgl, and where geodesic edge densification fits.

Icechunk is coming - versioned cloud-native Zarr, GDAL, and R

code

news

Icechunk is a transactional storage engine for Zarr and provides git-style versioning for large scientific datasets. This post is about Icechunk from a GDAL perspective and…

Grids: regular, rectilinear, curvilinear

gdal

raster

netcdf

projections

theory

A grid taxonomy rooted in coordinate storage, not CRS. A companion to Detecting hidden grids in curvilinear coordinates.

R, GDAL, Zarr, and the Missing Recipe Format for Raster Datasets

code

news

r-packages

python

zarr

gdal

netcdf

The virtual-Zarr format that lets xarray describe 16,000 files in a tiny store doesn’t exist in R yet - but the pieces are in place, this post tries to show where they fit…

Check it out: reproj

r-packages

projections

The reproj package provides a single generic function, reproj(), for transforming coordinates between map projections. It works with matrices, data frames, and anything else…

Check it out: gdalraster

r-packages

gdal

raster

vector

The gdalraster package provides a comprehensive interface to the GDAL library for R. It handles raster I/O, vector I/O, the virtual filesystem, coordinate transformation…

Check it out: proj_trans_create and wk_transform

r-packages

projections

The PROJ R package provides proj_trans_create(), which creates a reusable coordinate transformation object that plugs directly into wk’s wk_transform(). This is the…

Check it out: wk

r-packages

vector

The wk package is a foundation for geometry handling in R. It’s a lightweight, dependency-free geometry column with CRS support, vctrs integration, and a handler-based…

Check it out: geos

r-packages

vector

The geos package provides a direct R interface to the GEOS library. It operates on a geos_geometry vector type and speaks wk natively — any wk-handleable geometry can be…

What’s on geodata.ucdavis.edu and how to access it

code

news

raster

vector

gdal

spatial

The R package geodata provides functions for downloading climate, elevation, soil, crop, species occurrence, and administrative boundary data. All of its raster and vector…

Map Reprojection and Map Edges

code

projections

maps

spatial

A recent issue on the tmap package reports a white patch appearing at the top of a map rendered in an Albers Equal Area projection. The basemap tiles don’t fill the whole…

Detecting hidden grids in curvilinear coordinates

gdal

raster

netcdf

projections

A lot of gridded data arrives with two-dimensional longitude and latitude arrays: one value per pixel, stored explicitly. This is the standard representation for curvilinear…

R vs JavaScript Datetime Parsing

code

news

Datetime parsing across R and JavaScript has subtle but dangerous differences. Both languages make assumptions that can silently produce incorrect results, and critically…

Raster logic without pixels

raster

vaster

Every geospatial package that works with grids has raster logic inside. The old {raster} package in R established powerful abstraction functions and now {terra} includes an…

Coordinates broken in NetCDF

netcdf

gdal

There’s a NetCDF file at this URL:

GDAL multidim and cloud-ready ZARR

gdal

zarr

netcdf

I have been working on a better understanding of the GDAL multidimensional model and to do that really needs a closer look at the GDAL API itself.

Conservative regridding with GDAL (?)

gdal

raster

Can GDAL do conservative re-gridding? For cases of regular grid to regular grid, yes I think it can.

IDEA - data and software

overview

Plot at native resolution, with R

raster

gdal

Plot native, something I should have done long ago.

Michael D. Sumner

GDAL and image tiles, the {ceramic} package

gdal

r-packages

ceramic

A new version of {ceramic} is now on CRAN, version 0.8.0.

Michael D. Sumner

Degenerate Rectilinear (WIP)

netcdf

raster

There are a lot of problems when it comes to array representation in data formats and software. There is a whole family of complex issues with sometimes subtle or even…

Michael D. Sumner

GDAL raster read/write by blocks

gdal

raster

A block is another word for a tile, a tile in a small-ish raster window within a larger raster. Tiles can be very clever, such as 256x256, they make a nice way to organize…

Michael D. Sumner

GDAL warper with R

news

code

There’s several meanings floating around when you say the “GDAL warper”. It can mean

Michael D. Sumner

mesh3d - recent changes in rgl workhorse format

mesh

rgl

PLEASE NOTE (April 2022): this post has been migrated from an old site, and some details may have changed. There might an update to this post to reflect the rgl package as…

Michael D. Sumner

GDAL in R

gdal

overview

manifold

gis

automation

R

For some time I have used GDAL as a standard tool in my kit, I was introduced to the concept by the rgdal package authors and it slowly dawned on me what it meant to have a g…

Michael D. Sumner

Web services for scientific data in R

netcdf

r-packages

NOTE: this post has been resurrected from a 2017 post (April 2022).

Michael D. Sumner

R spatial in 2017

overview

This document is a broad overview of what I see as most relevant to future spatial, for 2017 and beyond. I’ve tried to be as broad as possible, without going into too much…

Michael D. Sumner

GIS for 3D in R

mesh

rgl

projections

GIS data structures are not well suited for generalization, and visualizations and models in 3D require pretty forceful and ad hoc approaches.

Michael D. Sumner

R matrices and image

raster

In R, matrices are ordered row-wise: