Hexagon Geospatial
MENU

WebGIS

Need a push in the right direction when configuring WebMap, Portal or SDI services? Looking for hints and tips, or just looking for Ideas and information? The WebGIS discussion board is where you start those discussions, connect and share information.
Showing results for 
Search instead for 
Do you mean 
Reply
Highlighted
dal
Contributor
Posts: 39
Registered: ‎05-09-2016
Accepted Solution

WMTS TileCache Folder Hashing

Dear All,

 

Due to a customer request we are currently expanding our knowledge about how WMTS tiles are stored physically on a server and how the tiles are accessed in the corresponding folder structure.

 

Being part of the WMTS configuration process, the TileLocationCfg.xml defines, where the WMTS tiles are stored. Within the previously mentioned XML-files, two XML-attributes have raised our attention:

1) <Parameter name="HashToFolders" value="{0}\{1}{2}\{3}{4}"/> 

--> the "value"-attribute has somehow an impact on the total number of folders used to store the tile-caches. But, what is the effective meaning of this XML-Element with its corresponding "value"-attribute? 

 

2) <TileLocationMapping pattern="^[0-9A-F]" location="D:\TileCache\Freizeit_ZH"/>

What is the logic behind the "pattern"-XML-attribute of the XML-Element listed above?

 

3) What is the relation between the XML-Element listed on 1) and the element listed on 2)?

 

Please find attached a representative printscreen of a TileLocationCfg.xml file.

 

Any hints are highly appreciated.

Regards,

Dominik

 

Moderator
Posts: 248
Registered: ‎02-08-2016

Re: WMTS TileCache Folder Hashing

Hello Dominik

 

Those two parameters allow control the “bucketing” of the cache folder.

An MD5 hash is calculated from the specifics of the tile image (product code, coordinate system code, calculated file name).

Each of the bytes is then converted to a hexadecimal string representation and put through the HashToFolders formatting string.

The default uses 9 of those bytes.

 

For example a tile image for product USA_SAMPLE in EPSG:4326 is generated for level 3, x 4, y 5 in PNG format.

  1. The specifics are concatenated:

USA_SAMPLE|EPSG4326|L3X4Y5.PNG

 

  1. An MD5 hash is calculated from that string:

B6995F54DA57ED06BFBD07396762EAFD

 

  1. The HashToFolders formatting string is used to calculate the relative path of the resulting file:

B6\995F\54DA\57ED\06BFB

 

The TileLocationMapping is then used to determine what is the root of this relative path.

The pattern is a regular expression and in the sample configuration maps everything to a single folder.

So the calculated relative path is concatenated on to the end of the first location that matches the string of the relative path:

C:\wmts\USA_SAMPLE\B6\995F\54DA\57ED\06BFB

 

This is the path to which the L3X4Y5.PNG file will be saved, or will be read from:

C:\wmts\USA_SAMPLE\B6\995F\54DA\57ED\06BFB\L3X4Y5.PNG

 

 

This mechanism allows you to customize the disk storage used, for example to put some of the tiles on a different volume.

 

Hope you find this information helpfull.

 

Thank You,

Marc

Staff
Posts: 294
Registered: ‎11-05-2015

Re: WMTS TileCache Folder Hashing

[ Edited ]

Only a slight correction to mandrats reply:

B6995F54DA57ED06BFBD07396762EAFD

The HashToFolders formatting string is used to calculate the relative path of the resulting file:

B6\995F\54DA\57ED\06BFB

 

This would hold true if the HashToFolders would be set to "{0}{1}\{2}{3}{4}{5}\{6}{7}{8}{9}\{10}{11}{12}{13}\{14}{15}{16}{17}" (which I'm not even sure if could work Smiley Happy). That's because one format parameter corresponds to exactly one character of the MD5 hash value string.

 

I've already tried to explain the algorithm in some depth here: http://community.hexagongeospatial.com/t5/Support-WebGIS/WMTS-tile-cache/m-p/7195/highlight/true#M31...

dal
Contributor
Posts: 39
Registered: ‎05-09-2016

Re: WMTS TileCache Folder Hashing

HBM and Marc, thanks a lot for your insightful replies.

 

So, the hashToFolders-part is really responsible for extracting individual positions from the md5-hash calculated based on product-code, crs and file name (with e.g. "{0}" extracting the very first element of the calculated md5-hash)? 

Is the example here (http://community.hexagongeospatial.com/t5/Support-WebGIS/WMTS-tile-cache/m-p/7195/highlight/true#M31...) also wrong?:

--> md5-hash: 36C2986BF6419468923AAF36979566DC

--> Hash-To-Folders: {0}\{1}{2}\{3}{4}\{5}{6}\{7}{8}

--> equals to path according to example in post linked above: [root]\USA_SAMPLE.EPSG4326\36\C2\98\6B\L3X1Y-10.png

--> as I understood your explanations so far, it should result in [root]\USA_SAMPLE.EPSG4326\3\6C\29\86\BF\L3X1Y-10.png

 

Am I still wrong or is there a small error in the example?

 

 

  • Do any rule of thumbs exist in order to define an optimal number of folder-levels through the "Hash-To-Folders"-setting?
  • Is it conceivable that a WMTS-user can experience any performance increase or degradation depending on the"Hash-To-Folder" setting?
  • Are any further advantages or disadvantages known with respect to this "Hash-To-Folder" setting?
Staff
Posts: 294
Registered: ‎11-05-2015

Re: WMTS TileCache Folder Hashing

The original explanation I gave in the linked thread is the correct one. The one I gave here was a bit hurried Smiley Wink

The {0} correlates to the string generated using the productDirectoryNameFormat and {1},{2}... are actual characters from the hash string.

dal
Contributor
Posts: 39
Registered: ‎05-09-2016

Re: WMTS TileCache Folder Hashing

great, thanks for the clarifications.

 

Regards,

Dominik

Contributor
Posts: 46
Registered: ‎10-13-2015

Re: WMTS TileCache Folder Hashing

My experience is that you should to have more than one tile in each folder and less than several thousand.

With this configuration {0}\{1}{2}\{3}{4}\{5}{6}\{7}{8} and a cahce with 10 millions tiles you are going to have one or two tiles in each folder because it can create 4.100.000.000 folders in the last level.

With a configuration {0}\{1}{2}\{3}{4}\{5}{6}\{7}{8} and the same cache you are going to have +/-150 tiles in each last folder.

If you have many tiles in each folder the performance go down. The number can be in function of the OS.

 

About the other question {0} it is the product-code, and the md5-hash start in {1}.

Do you need immediate support?
If you encounter a critical issue and need immediate assistance please submit a Service Request through our Support Portal.