Project

General

Profile

Activity

From 2016-05-04 to 2016-06-02

2016-05-25

16:31 Task #7771 (In Progress): Use multiple threads to index objects
Jing Tao
16:31 Story #7769 (In Progress): Improve the performance on solr index
Jing Tao
16:10 Bug #7816 (Closed): After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PRO...
The ticket is moved to 2.2 release.
Since we will create d1_cn_index_processor branch 2.2 from branch 2.1. So the...
Jing Tao
15:44 Bug #7816: After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PROCESS" status
After further investigation, I found the problem is not the context file. The problem is in the org.dataone.cn.indexe... Jing Tao

2016-05-24

00:49 Bug #7816: After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PROCESS" status
The property was removed from https://repository.dataone.org/software/cicore/trunk/cn-buildout/dataone-cn-index/usr/s... Jing Tao
00:47 Bug #7816: After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PROCESS" status
It turned out that the convert property on the distributServiceType blocked the index without any error message:
<be...
Jing Tao

2016-05-23

18:51 Bug #7817 (Closed): The processIndexTaskQueue method on IndexTaskProcessor doesn't pick up all th...
The processIndexTaskQueue method suppose to pick all failed the index tasks for reindexing. However, it only picked u... Jing Tao
18:46 Bug #7816: After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PROCESS" status
Example ids:
iso19119_looselyCoupled_201611154258815
{A2AE9572-A192-4FFF-9A28-3D2FD0387FF0}
Jing Tao
18:45 Bug #7816: After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PROCESS" status
After further investigation, we found all of the documents come from the two type:
http://www.isotc211.org/2005/gm...
Jing Tao
18:43 Bug #7816 (Closed): After reindex cn-dev, there are more than 20,000 index tasks keep the "IN PRO...
When the index finished, it should be failed or removed from the index_task table. However, there are 20,000 index ta... Jing Tao

2016-05-04

23:15 Task #7771 (Closed): Use multiple threads to index objects
Worked on generating index on multiple threads:
while (getNextTask() != null ) {
pro...
Jing Tao
23:12 Task #7770 (Closed): Profile the index process
Jing Tao
23:12 Task #7770 (Closed): Profile the index process
Solr Index profiling

EML:
1. Create a index task queue 328 (first one is very high, now it drops drama...
Jing Tao
23:10 Story #7769 (Closed): Improve the performance on solr index
Currently it will take more than one week to reindex our production CNs. We need to improve the performance. Jing Tao
 

Also available in: Atom

Add picture from clipboard (Maximum size: 14.8 MB)