1.3m Images without tiles fail to fully render in image view

[peggynewman]

Upon investigating image-service tiling speed which was around 1 image per ~23 seconds, realised that there are 1.3m images without tiles that can be extracted by the following query:

SELECT 
    COUNT(*) AS cnt, 
    i.data_resource_uid 
FROM 
    image i
LEFT JOIN 
    audit_message am 
ON 
    i.image_identifier = am.image_identifier
    AND am.message like 'TMS%'

WHERE 
    i.zoom_levels = 0
    AND am.image_identifier IS NULL
GROUP BY 
    i.data_resource_uid
ORDER BY 
    cnt DESC LIMIT 100

and the result includes 55 data resources. Here is the snapshot from the top records:

As per @sbearcsiro 's great work, the tiling queue which was running super slow last week is all tiled and the queue is back to 0 now. But seems that restarting image service will lose the items in the queue and this could have happened many times in the pas due to restarts.

The issue here is the address the tiling task which needs to be resilient against restarts and also its performance should be much higher that the speed of ingesting images into the system.

[peggynewman]

Is there an update on this? how can we track progress?

[sbearcsiro]

Closing as this is essentially a duplicate of #201 and #211 and is addressed by #214 and #216