Task #7770
Story #7769: Improve the performance on solr index
Profile the index process
100%
Description
Solr Index profiling
EML:
1. Create a index task queue 328 (first one is very high, now it drops dramatically. Generally it is about 10 ~ 30)
2. Fetch a task from the queue 10
3. Total processing time 1076
- SolrIndexService.processObject - 536
*process system metadata - 66 ( each field is from 0 to 1, but the id took 32)
*process by science metadata - 255 (each field take 2,3 and 4. abstract 12, keyword 10, title 10, project 8, CommonRootSolrField("attribute") 32, text 11,southBoundCoord 9, northBoundCoord 7, west 10, east 8, begin date 10, enddate 7)
*process by BaseReprocessSubprocessor 71 (series id and resource map)
*process merging - 141
- sending to the solr server 532
A resourceMap:
1. Create a index task queue 233
2. Fetch a task from the queue 12
3. Total processing time 1652
- SolrIndexService.processObject - 980
*process system metadata - 38
* process by ResourceMapSubprocessor.processResourceMap - 416
ResourceMapFactory.buildResourceMap() create ResourceMap from Document, 95
ResourceMap.getAllDocumentIDs() referenced in ResourceMap, 134
ResourceMapSubprocessor.clearSidChain() removing obsoletes chain from Solr index, 99
HttpService.getDocumentsById() get existing referenced ids' Solr docs, 26
*process by RdfXmlSubprocessor.processDocument() 400
RdfXmlSubprocess.process gets a dataset from tripe store service , 116
RdfXmlSubprocess.process adds ont-model , 21
RdfXmlSubprocess.process process the fields total , 238 (prov stuff)
*process merging - 125
- sending to the solr server 661
History
#1 Updated by Jing Tao over 8 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
- translation missing: en.field_remaining_hours set to 0.0
#2 Updated by Jing Tao over 8 years ago
- Category set to d1_cn_index_processor
