TMS reverse to WMTS

There is a TMS tile data source that needs to "simulate" a WMTS service. What should be done?

In this case, there are actually existing infrastructures or "wheels" to solve it, such as various map servers, and the .net ecosystem also has open-source tools like tile-map-service-net5. The reason this issue is a problem lies in two constraints.

The client used does not support loading XYZ/TMS format data and can only load WMS and WMTS format data.
The data used is in a pre-sliced TMS structure.
The client is inconvenient to rely on external map servers.

Mimicking Resource Links#

Some of the internet maps we are familiar with use XYZ or TMS methods, such as OSM, Google Map, and Mapbox. From the previous raster tiles to the now more common vector tiles, to use TMS to "mimic" the WMTS request format, we need to first understand what differences they have.

XYZ (slippy map tilename)#

256*256 pixel images
Each zoom level is a folder, each column is a subfolder, and each tile is an image file named by row
The format is similar to /zoom/x/y.png
x is in (180°W ~ 180°E), y is in (85.0511°N ~ 85.0551°S), and the Y-axis goes from top to bottom.

You can see a simple XYZ tile example from Openlayers TileDebug Example.

TMS#

The TMS Wiki wikipedia does not cover many details, and osgeo-specification only describes some application details of the protocol. Instead, the geoserver docs about TMS are written more practically. TMS is the predecessor of WMTS and is a standard established by OSGeo.

Requests are in the form of:
http://host-name/tms/1.0.0/layer-name/0/0/0.png

To support multiple file formats and spatial reference systems, multiple parameters can also be specified:
http://host-name/tms/1.0.0/layer-name@griset-id@format-extension/z/x/y

The standard TMS tile grid starts from the bottom left corner, with the Y-axis going up. Some map servers, such as geoserver, support an additional parameter flipY=true to flip the Y coordinate, allowing compatibility with service types where the Y-axis goes from top to bottom, such as WMTS and XYZ.

tms-grid

WMTS#

WMTS is more complex than the two intuitive protocols mentioned above and supports more scenarios. It was first published by OGC in 2010. Before this, after Allan Doyle's paper "Www mapping framework" in 1997, OGC began planning the establishment of standards related to web maps. Before WMTS, the earliest and most widely used web map service standard was WMS. Since each WMS request organizes the map response based on the user's map zoom level and screen size, these responses vary in size. In the early days when multi-core CPUs were not so common, this on-demand real-time map generation method was very extravagant, and it was also very difficult to improve response speed. As a result, developers began to try pre-generating tiles, leading to many solutions, one of which is the TMS mentioned earlier, and later WMTS emerged and began to be widely used. WMTS supports key-value pairs (kvp) and Restful methods for encoding request parameters.

KVP is in the form of:
<baseUrl>/layer=<full layer name>&style={style}&tilematrixset={TileMatrixSet}}&Service=WMTS&Request=GetTile&Version=1.0.0&Format=<imageFormat>&TileMatrix={TileMatrix}&TileCol={TileCol}&TileRow={TileRow}
Restful is in the form of:
<baseUrl>/<full layer name>/{style}/{TileMatrixSet}/{TileMatrix}/{TileRow}/{TileCol}?format=<imageFormat>
Since it is raster tiles, we only need to find the correspondence between XYZ and the tile matrix and tile row/column numbers.

TileMatrix
TileRow
TileCol

wmts-grid
Here, the tile row and column numbers start from the top left corner, with the Y-axis going down.

Thus, we have found the correspondence between the parameters of TMS and WMTS. Next, we need to know how to convert TMS into WMTS requests, as follows:

TileRow = 2^zoom - 1 - y = (1 << zoom) - 1 - y
TileCol = x
TileMatrix = zoom

Without considering other spatial references, the zoom level corresponds to the tile matrix, x corresponds to the tile column number, and y is inverted (because the starting direction is opposite).

Simulating a WMTS Capabilities Description File#

The requirements of the WMTS specification can be said to be detailed to the hair, so various clients, whether it is the web-based Openlayers or desktop applications like QGIS or Skyline, can directly parse the Capabilities description file and then select layers, styles, and spatial references based on the content of the description file. Therefore, we also need to simulate a WMTS Capabilities description file.

Structure of the Capabilities Description File#

An example of a WMTS Capabilities description file can be found in opengis schema and Tian Map Shandong.

The content of the Capabilities description file is very extensive, and here we only list some important parts (ignoring titles, contact information, etc.):

OperationsMetadata:
  - GetCapabilities >> Method to obtain the Capabilities description file
  - GetTile >> Method to obtain tiles

Contents:
  - Layer
    - boundingBox >> Latitude and longitude range of the layer
    - Style
    - TileMatrixSetLink >> Spatial references supported by the layer
    - TileMatrixSet >> Spatial reference
      - TileMatrixSetLimits >> Zoom level range of the spatial reference
        - TileMatrixLimits >> Tile row and column number range for each zoom level
  - Style
  - TileMatrixSet
    - TileMatrix

The key parts are boundingBox, TileMatrixSetLimits, and TileMatrixLimits, which only need to be calculated based on the spatial reference and zoom level of the layer.

The calculation of boundingBox is relatively simple, which is the latitude and longitude range of the layer, so we will not elaborate on it here.

The calculation of TileMatrixSetLimits is also relatively simple, which is the zoom level range of the spatial reference of the layer.

The calculation of TileMatrixLimits is more complex and can be done only when the layer range is relatively small; there is no need for global maps. It needs to be calculated based on the spatial reference and zoom level of the layer. Below is a piece of pseudocode (from 4326 to 3857).

FUNCTION GetTileRange(minLon, maxLon, minLat, maxLat, zoom, tile_size = 256)

minLonRad = minLon * PI / 180
maxLonRad = maxLon * PI / 180
minLatRad = minLat * PI / 180
maxLatRad = maxLat * PI / 180

tile_min_x = Floor((minLonRad + PI) / (2 * PI) * Pow(2, zoom))
tile_max_x = Floor((maxLonRad + PI) / (2 * PI) * Pow(2, zoom))
tile_min_y = Floor((PI - Log(Tan(minLatRad) + 1 / Cos(minLatRad))) / (2 * PI) * Pow(2, zoom))
tile_max_y = Floor((PI - Log(Tan(maxLatRad) + 1 / Cos(maxLatRad))) / (2 * PI) * Pow(2, zoom))

// adjust tile range based on tile size
tile_min_x = Floor((double)tile_min_x * tile_size / 256)
tile_max_x = Ceiling((double)tile_max_x * tile_size / 256)
tile_min_y = Floor((double)tile_min_y * tile_size / 256)
tile_max_y = Ceiling((double)tile_max_y * tile_size / 256)

RETURN  (tile_min_x, tile_max_x, tile_min_y, tile_max_y)

Generating WMTS Capabilities Description File#

Generate a minimized WMTS Capabilities description file, filling in the key parts above, and then construct a Restful-style URL pointing to the standard description file address.

Conclusion#

The above is a simple idea of converting TMS to WMTS. In fact, there are many details to consider, such as spatial reference conversion, zoom level conversion, tile row and column number conversion, tile format conversion, and so on. During this process, I also encountered some pitfalls, which I found more interesting.

In the first part, I quickly referred to the idea of tile-map-service-net5 and completed the conversion of y >> tileRow. The code is in WebMercator.cs. In fact, someone has also asked this question on StackOverflow, and there is an answer, but I still chose to find the answer from the software because it made me feel more secure.

The second part was quite challenging. First, I simulated the resource link and constructed a simple XML, but it could not be loaded directly on the target client. I directly thought of testing it through the standard service, and then I needed a Capabilities description file to modify. I initially thought of testing on Openlayers, which I was more familiar with, and then modifying the Capabilities description file. Openlayers' loading method is still very flexible; without a Capabilities description file, it can directly access through configuration parameters.

// fetch the WMTS Capabilities parse to the capabilities 
const options = optionsFromCapabilities(capabilities, {
            layer: 'nurc:Pk50095',
            matrixSet: 'EPSG:900913',
            format: 'image/png',
            style: 'default',
});
const wmts_layer =new TileLayer({
    opacity: 1,
    source: new  WMTS(options),
})

Unfortunately, the tiles did not load, and there was not even a request sent in networks. So I went to another WMTS-related example and customized a TileGrid, then converted the tile row and column numbers to 3857's row and column numbers, and at this point, it could load.

const projection = getProjection('EPSG:3857');
const projectionExtent = projection.getExtent();
const size = getWidth(projectionExtent) / 256;
const resolutions = new Array(31);
const matrixIds = new Array(31);
for (let z = 0; z < 31; ++z) {
    // generate resolutions and matrixIds arrays for this WMTS
    resolutions[z] = size / Math.pow(2, z);
    matrixIds[z] = `EPSG:900913:${z}`;
}
var wmtsTileGrid = new WMTSTileGrid({
    origin: getTopLeft(projectionExtent), resolutions: resolutions, matrixIds: matrixIds,
})

After confirming that it was a TileGrid issue, I first compared my generated TileGrid with the TileGrid parsed from Capabilities by Openlayers. I found that some fields in my generated TileGrid were empty, so I tested them one by one. Finally, I discovered that when the internal parameters fullTileRanges_ and extent_ were empty, the image could load.

I went to check the OL source code and found that fullTileRanges_ and extent_ are used in getFullTileRange.

This means that when fullTileRanges_ and extent_ are empty, getFullTileRange will return an empty range.

And getFullTileRange is used in withinExtentAndZ, which is used to determine whether there are tiles for the layer in the current visible area. This means that when fullTileRanges_ and extent_ are empty, it cannot obtain TileRange, and withinExtentAndZ will always return true, thus continuously loading tiles, which is the reason for successful loading.

Conversely, the fullTileRanges_ and extent_ parsed from Capabilities pointed to the wrong TileRange, causing withinExtentAndZ to always return false, which is why the tiles would not load, leading to failure.

Finally, I found the reason, but I was misled again. In wmts.js, there is a comment in the constructor:

class WMTS extends TileImage {
  /**
   * @param {Options} options WMTS options.
   */
  constructor(options) {
      // TODO: add support for TileMatrixLimits
  }
}

This made me initially mistakenly believe that fullTileRanges_ and extent_ were calculated based on the latitude and longitude range (boundingBox) rather than calculated based on TileMatrixLimits. So I checked the boundingBox again and confirmed it was correct before starting to modify TileMatrixLimits.

At first, I thought TileMatrixLimits was the tile range for each level, rather than the range for the layer, so I didn't pay attention to this parameter, which led to a detour.

Written in 2023, WMTS is no longer a new protocol, and the OGC Tile API has become a formal standard. I still have a superficial understanding of WMTS, and I feel quite embarrassed 😅.