Project

General

Profile

Bug #8071

Prov relationship can't be indexed on CN

Added by Jing Tao almost 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
High
Assignee:
Category:
d1_indexer
Target version:
Start date:
2017-04-13
Due date:
% Done:

100%

Milestone:
None
Product Version:
*
Story Points:
Sprint:

Description

We just found that the prv relationship can't be index on CN. But it can be indexed on Metacat.

History

#1 Updated by Jing Tao almost 7 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

The failure reason was that RdfXmlSubprocessor couldn't access the triple store. When the triple store is initialized, it will create the store at the location either specified by a property or the default value ./tdb. Since we didn't set the property, somehow cn looked at the location "/tdb" and failed because the access permission issue.
I added a new property file and it specifies the location on /var/dataone/index/tdb. I also modified the postinst of dataone-cn-index by adding the code to create directory. The new code was depolyed to cn-dev and it worked.
The change was committed to the 2.3 branches as well.

#2 Updated by Jing Tao almost 7 years ago

  • Status changed from Closed to In Progress
  • % Done changed from 100 to 30

We experienced a null exception when the prov subprocessor indexes the rdf doc:
java.lang.NullPointerException
at com.hp.hpl.jena.tdb.base.file.BufferChannelFile.size(BufferChannelFile.java:133)
at com.hp.hpl.jena.tdb.transaction.Journal.size(Journal.java:294)
at com.hp.hpl.jena.tdb.transaction.JournalControl.replay(JournalControl.java:238)
at com.hp.hpl.jena.tdb.transaction.JournalControl.replay(JournalControl.java:232)
at com.hp.hpl.jena.tdb.transaction.TransactionManager.processDelayedReplayQueue(TransactionManager.java:560)
at com.hp.hpl.jena.tdb.transaction.TransactionManager.closedown(TransactionManager.java:281)
at com.hp.hpl.jena.tdb.StoreConnection.expel(StoreConnection.java:201)
at com.hp.hpl.jena.tdb.StoreConnection.release(StoreConnection.java:187)
at com.hp.hpl.jena.tdb.sys.TDBMaker.releaseLocation(TDBMaker.java:62)
at com.hp.hpl.jena.tdb.TDBFactory._release(TDBFactory.java:116)
at com.hp.hpl.jena.tdb.TDBFactory.release(TDBFactory.java:84)
at org.dataone.cn.indexer.annotation.RdfXmlSubprocessor.process(RdfXmlSubprocessor.java:261)
at org.dataone.cn.indexer.annotation.RdfXmlSubprocessor.processDocument(RdfXmlSubprocessor.java:119)
at org.dataone.cn.indexer.SolrIndexService.processObject(SolrIndexService.java:231)
at org.dataone.cn.indexer.SolrIndexService.insertIntoIndex(SolrIndexService.java:304)
at org.dataone.cn.index.processor.IndexTaskUpdateProcessor.process(IndexTaskUpdateProcessor.java:50)
at org.dataone.cn.index.processor.IndexTaskProcessor.processTask(IndexTaskProcessor.java:286)
at org.dataone.cn.index.processor.IndexTaskProcessor.access$000(IndexTaskProcessor.java:80)
at org.dataone.cn.index.processor.IndexTaskProcessor$1.run(IndexTaskProcessor.java:263)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

#3 Updated by Jing Tao almost 7 years ago

Another exception:
cn-index-processor-daemon.log.1-java.lang.NullPointerException
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTableCache._idForNode(NodeTableCache.java:125)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeIdForNode(NodeTableCache.java:79)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeIdForNode(NodeTableWrapper.java:49)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeIdForNode(NodeTableInline.java:59)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.idForNode(NodeTupleTableConcrete.java:189)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:142)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.store.DatasetGraphTDB._containsGraph(DatasetGraphTDB.java:150)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.tdb.store.DatasetGraphTDB.containsGraph(DatasetGraphTDB.java:142)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.sparql.core.DatasetGraphTrackActive.containsGraph(DatasetGraphTrackActive.java:86)
cn-index-processor-daemon.log.1- at com.hp.hpl.jena.sparql.core.DatasetImpl.containsNamedModel(DatasetImpl.java:227)
cn-index-processor-daemon.log.1- at org.dataone.cn.indexer.annotation.RdfXmlSubprocessor.process(RdfXmlSubprocessor.java:201)
cn-index-processor-daemon.log.1- at org.dataone.cn.indexer.annotation.RdfXmlSubprocessor.processDocument(RdfXmlSubprocessor.java:119)
cn-index-processor-daemon.log.1- at org.dataone.cn.indexer.SolrIndexService.processObject(SolrIndexService.java:231)
cn-index-processor-daemon.log.1- at org.dataone.cn.indexer.SolrIndexService.insertIntoIndex(SolrIndexService.java:304)
cn-index-processor-daemon.log.1- at org.dataone.cn.index.processor.IndexTaskUpdateProcessor.process(IndexTaskUpdateProcessor.java:50)
cn-index-processor-daemon.log.1- at org.dataone.cn.index.processor.IndexTaskProcessor.processTask(IndexTaskProcessor.java:286)
cn-index-processor-daemon.log.1- at org.dataone.cn.index.processor.IndexTaskProcessor.access$000(IndexTaskProcessor.java:80)
cn-index-processor-daemon.log.1- at org.dataone.cn.index.processor.IndexTaskProcessor$1.run(IndexTaskProcessor.java:263)
cn-index-processor-daemon.log.1- at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

#4 Updated by Jing Tao almost 7 years ago

In the original implementation, the backend directory was share by the all the resourcemap. It worked for the single processing thread. However, we use multiple to index resource map, this causes the issue. Now we use a dedicated directory for each object, the exceptions was gone. We also make sure the directories will be removed when process is done: the remove is in the final statement; the directories are set deleteOnExit.

#5 Updated by Jing Tao almost 7 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 30 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)