Infrastructure

The index processor stops processing or processes extremely slowly, failing to drain the task queue.

Attempting to stop using "service d1-index-task-processor stop" fails, with the process continuing to run very slowly for hours.

SELECT status,deleted,COUNT(*) AS cnt FROM index_task GROUP BY status,deleted ORDER BY status;

cn-index-processor-daemon.log:

[ INFO] 2016-11-14 13:36:10,874 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:15,876 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:18,972 (IndexTaskProcessor:isResourceMapReadyToIndex:197) Not all map resource references indexed for map: resource_map_doi:10.18739/A25G76. Marking new and continuing...
[ INFO] 2016-11-14 13:36:18,979 (IndexTaskProcessor:getNextIndexTask:178) Task for pid: resource_map_doi:10.18739/A25G76 not processed.
[ INFO] 2016-11-14 13:36:18,982 (IndexTaskProcessor:getNextIndexTask:161) Start of indexing pid: resource_map_doi:10.18739/A25G8K
[ INFO] 2016-11-14 13:36:20,876 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:25,877 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:30,877 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:35,878 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....
[ INFO] 2016-11-14 13:36:40,878 (IndexTaskProcessorScheduler:stop:88) Job executing, waiting for it to complete.....