05-29-2018 07:34 AM
I have been working with a customer who experiences slow performance opening GeoWorkspaces with GeoMedia Desktop 16.0. Through the use of ProcMon.exe from Microsoft I identified that GeoMedia.exe is writing large temporary files like fc3581.tmp to the user's temp folder when displaying data in a map window. I witnessed this whilst displaying a SmartStore and ESRI Shapefile dataset.
I can reproduce the same behaviour on my own machine running GeoMedia Desktop 16.2 which generates a 2.7GB temp files to display a feature class from a ESRI FGDB.
My first guess was this is Warehouse/Feature Caching, but no caching is enabled on the Warehouse Connections and no files are being generated under the folder set as the Feature Cache Files Folder
I understood that GeoMedia uses memory when loading datasets so it surprises me to see files being written to disk when displaying data. Can someone explain why these files are being created?
05-29-2018 09:26 AM
When bringing features entirely into memory for map display would consume too much memory, the system offloads some of it to disk in the form of temporary IFC files. Those are what you are seeing. If you enable feature caching, effectively you are allowing these temporary files to persist across sessions, and it will improve your subsequent startup time. If the datasets you are displaying will fit efficiently in memory, then these temporary files are not created. - Hal
06-01-2018 04:29 AM
Whilst I appreciate there may be a gain in memory performance here for some users, it doesn't appear to benefit all use cases.
The customer experiencing large temporary IFC files predominently uses SmartStore files for their data which is one of the fastest display formats we have in GeoMedia.
Their users are already spending business time to produce SmartStore files (this is a G/Technology customer) so asking them to cache their data again into IFC files is an overhead they are unlikely to accept. I am also not sure what performance benefit they would get from generating the cache of a SmartStore file.
Is it worth submitting an idea to allow an option in GeoMedia Settings to enable/disable the creation of temporary IFC files?
Can I also check your statement of "If the datasets you are displaying will fit efficiently in memory, then these temporary files are not created", as I've just tested this with a 5MB ESRI Shapefile polygon dataset (2697 features). Adding this to a Map Window generates a 16MB temporary IFC file. GeoMedia.exe is only using 174MB of working set memory and 89MB of private working memory so why is the temporary IFC file generated?
These temporary IFC files are also filling up C: drive space which is another concern for our customers so we should also be able to set the folder where they are generated to a secondary disk drive. Is that another Idea?
06-01-2018 05:00 AM
It does benefit all use cases. If the cache were not there, the system would basically not function at all (this is something you cannot have experienced as we ensure it is there at all time whether in memory or on disk).
IFC files are roughly twice as fast as SmartStore files, is what our tests on a few occasions have shown. We have invested heavily in this system. Also, the display pathway knows how to interact in a special way with IFC files and their internal spatial indices, whereas the display system is unable to give such special treatment to SmartStore files, which lie behind a data server thath does not permit this.
We are not asking the customer to cache their data again, that's not what we consider SmartStore to be. In GeoMedia, SmartStore is simply a tidy freestanding file-based data storage mechanism equivalent to any other warehouse albeit faster than most; the caching system is for display acceleration of all data.
We would not consider an Idea to disable the creation of temporary IFC files, the entire display system hinges on it.
Regarding your check of my statement about in-memory use, I suggest you file a support ticket for engagement with those guys on the specifics of your case. I am interested in understanding this situation better and it may be that in interacting with the development team some useful insights might arise. I may be (mis)remembering some aspect of the situation, or my information may be out-of-date. I am also surprised that the IFC file is larger than the shapefile. Are you only providing the size of the SHP and not the other half-dozen-ish files? IFC packs all of that into a single file.
Submission of an Idea to permit designation of the location of temporary IFC files would be fine.
Another Idea you would be welcome to file is that of providing IFC files as an alternative to SmartStore files (i.e. as a standalone data store rather than simply a "front" to warehouses). It is something we have considered for a long time, and our ongoing investment in feature caching improvements (e.g. in the context of the HOFOR and AGON projects) makes this more attractive all the time. We already have a command-line utility for publishing such files but still considerable work would be required to get us to the point that your customer could publish IFC files as an alterative to DDC files (likely with very visible performance benefit). Plus it would eliminate what they see as redundant caching. - Hal