Colormap failing to serialize for tile endpoints

[banesullivan]

It looks like some colormaps can easily become too large when serialized to pass through as a URL parameter. Take the following example originally reported in opengeos/leafmap#966

Create the sample data

import numpy as np
import rasterio

l = list(range(0,80,1))
l.extend([65535]*10) # 65535 means missing data
l.extend([65533]*10) # 65533 is another special value
nparr = np.array(l, dtype=np.float32)
nparr[nparr<65000] /= 10
nparr = nparr.reshape([10,10])
GT33 = (1000.0,0.0,300000.0,0.0,-1000.0,7000000.0)

profile = {
    'driver': 'GTiff',
    'height': nparr.shape[0],
    'width': nparr.shape[1],
    'count': 1,  # Number of bands
    'dtype': nparr.dtype,
    'crs': 'EPSG:32633',
    'transform': GT33,
}

with rasterio.open('output.tif', 'w', **profile) as dst:
    dst.write(nparr, 1)  # Write data to the first band

Attempt plotting

Create the colormap

import matplotlib.colors as mcolors
import matplotlib.pyplot as plt

colors = ['green', 'yellow', 'orange', 'red']
cmap = mcolors.LinearSegmentedColormap.from_list('custom_cmap', colors)
cmap.set_under('grey')
cmap.set_over('grey')

Attempt to use with localtileserver (not that the above/below colors are unused:

import localtileserver as lts
client = lts.TileClient('output.tif')
client.tile(10, 543, 279, nodata=65535, colormap=cmap, vmin=0, vmax=10)

and if I try to use on a map:

m = client.get_leaflet_map()
m.zoom = 11
m.add(lts.get_leaflet_tile_layer(client, nodata=65535, colormap=cmap, vmin=0, vmax=10))
m

then I see browser network errors: 414 Request-URI Too Large

Summary

1. above/below colors on a matplotlib LinearSegmentedColormap are not used
2. serializing the LinearSegmentedColormap is too large for the URL parameters and silently fails

[banesullivan]

unfortunately, until I get around to entirely changing to something like anywidget where I can better control how tiles are requested and the parameters that go with that request (like #219), the URL lenght issue likely isn't something I'm going to be able to resolve.

What could work is registering the colormap with matplotlib as a named colormap prior to runtime so that when localtileserver's background python thread looks for it by name it can be found

[banesullivan]

In case anyone out there knows of a better way to serialize and pass along the colormap, I am currently serializing it to JSON and injecting it as a URL query parameter here:

localtileserver/localtileserver/client.py

Lines 456 to 469 in 0920f74

	if colormap is not None:
	if isinstance(colormap, ListedColormap):
	colormap = json.dumps([c for c in colormap.colors])
	elif isinstance(colormap, Colormap):
	colormap = json.dumps(
	{k: tuple(v.tolist()) for k, v in enumerate(colormap(range(256), 1, 1))}
	)
	elif isinstance(colormap, list):
	colormap = json.dumps(colormap)
	else:
	# make sure palette is valid
	palette_valid_or_raise(colormap)
	params["colormap"] = colormap

[giswqs]

TiTiler supports JSON encoded custom Colormap. Not sure if its implementation be useful to localtileserver.

https://developmentseed.org/titiler/endpoints/cog/#description

import leafmap
url = "https://github.com/opengeos/datasets/releases/download/raster/nlcd_2021_land_cover_30m.tif"
colormap = {
    "11": "#466b9f",
    "12": "#d1def8",
    "21": "#dec5c5",
    "22": "#d99282",
    "23": "#eb0000",
    "24": "#ab0000",
    "31": "#b3ac9f",
    "41": "#68ab5f",
    "42": "#1c5f2c",
    "43": "#b5c58f",
    "51": "#af963c",
    "52": "#ccb879",
    "71": "#dfdfc2",
    "72": "#d1d182",
    "73": "#a3cc51",
    "74": "#82ba9e",
    "81": "#dcd939",
    "82": "#ab6c28",
    "90": "#b8d9eb",
    "95": "#6c9fb8",
}
m = leafmap.Map(center=[40, -100], zoom=4, height="650px")
m.add_basemap("Satellite")
m.add_cog_layer(url, colormap=colormap, name="NLCD Land Cover", nodata=0)
m.add_legend(title="NLCD Land Cover Type", builtin_legend="NLCD")
m.add_layer_manager()
m

[banesullivan]

See also the example @giswqs posted where the colormap is written directly to the raster: https://leafmap.org/notebooks/103_raster_colormap/