Infrastructure

Seen in STAGE, from among a group of task left in the 'IN PROCESS' state:


~~~
[ INFO] 2018-02-12 16:52:10,292 (IndexTaskProcessor:getNextIndexTask:610) Start of indexing pid: urn:uuid:eff10cb2-e454-4bc8-acc2-0927488621b0
[ERROR] 2018-02-12 16:52:10,311 (IndexTask:unMarshalSystemMetadata:221)
org.dataone.exceptions.MarshallingException
at org.dataone.service.util.TypeMarshaller.unmarshalTypeFromStream(TypeMarshaller.java:299)
at org.dataone.cn.index.task.IndexTask.unMarshalSystemMetadata(IndexTask.java:213)
at org.dataone.cn.index.task.IndexTask.isArchived(IndexTask.java:251)
at org.dataone.cn.index.task.IndexTask.isDeleteTask(IndexTask.java:265)
at org.dataone.cn.index.processor.IndexTaskProcessor.getNextIndexTask(IndexTaskProcessor.java:612)
at org.dataone.cn.index.processor.IndexTaskProcessor.processIndexTaskQueue(IndexTaskProcessor.java:184)
at org.dataone.cn.index.processor.IndexTaskProcessorJob.execute(IndexTaskProcessorJob.java:57)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: javax.xml.bind.UnmarshalException
- with linked exception:
[org.xml.sax.SAXParseException; Premature end of file.]
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:563)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:249)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:214)
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
at org.dataone.service.util.TypeMarshaller.unmarshalTypeFromStream(TypeMarshaller.java:297)
... 8 more
Caused by: org.xml.sax.SAXParseException; Premature end of file.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:243)
... 12 more
[ERROR] 2018-02-12 16:52:10,312 (JobRunShell:run:222) Job d1-cn-index-processor.d1-index-processor-job threw an unhandled Exception:
java.lang.NullPointerException
[ERROR] 2018-02-12 16:52:10,313 (ErrorLogger:schedulerError:2360) Job (d1-cn-index-processor.d1-index-processor-job threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.NullPointerException]
at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.NullPointerException
[ WARN] 2018-02-12 16:54:08,196 (IndexTaskProcessorJob:execute:55) processing job [1806183035/org.dataone.cn.index.processor.IndexTaskProcessorJob@15d15ed9] executing index task with processor [org.dataone.cn.index.processor.IndexTaskProcessor@20621d44]
[ INFO] 2018-02-12 16:54:09,567 (IndexTaskProcessor:logProcessorLoad:245) new tasks:375, tasks previously failed: 1660
~~~

and this in postgres:

~~~
d1-index-queue=# select * from index_task where sysmetadata = '';
id | datesysmetamodified | deleted | formatid | nextexecution | objectpath | pid | priority | status | sysmetadata | taskmodifieddate | trycount | version
----------+---------------------+---------+---------------------------------------+---------------+------------+-----------------------------------------------+----------+------------+-------------+------------------+----------+---------
14083132 | 1452029399937 | f | text/plain | 1518032957157 | | urn:uuid:eff10cb2-e454-4bc8-acc2-0927488621b0 | 1 | IN PROCESS | | 1518454330288 | 4 | 5
14083818 | 1452029390440 | f | eml://ecoinformatics.org/eml-2.1.1 | 1518032959192 | | urn:uuid:0f327426-3e97-428f-810e-44a83ec92047 | 1 | IN PROCESS | | 1518454449880 | 4 | 5
14085245 | 1452032067206 | f | text/plain | 1518032963639 | | urn:uuid:7a31a8e1-54c1-4d10-9c7d-7cf969db4015 | 1 | IN PROCESS | | 1518454569874 | 4 | 5
14087397 | 1452032061198 | f | text/csv | 1518032967945 | | urn:uuid:0d07b6f5-4299-43e5-a1b8-aa37b259e51f | 1 | IN PROCESS | | 1518454690716 | 4 | 5
14087857 | 1452032064477 | f | eml://ecoinformatics.org/eml-2.1.1 | 1518032968646 | | urn:uuid:56e25ff0-dc85-4441-a4c2-74b9955cc43d | 1 | IN PROCESS | | 1518454809882 | 4 | 5
14088344 | 1452032063246 | f | text/csv | 1518032969414 | | urn:uuid:2fb13ee6-fb8a-4eea-b9ee-f52e85b35a7a | 1 | IN PROCESS | | 1518454930608 | 4 | 5
14092345 | 1452032069944 | f | http://www.openarchives.org/ore/terms | 1518032978246 | | urn:uuid:5429c13d-db3e-4997-985a-eee1273163d7 | 3 | IN PROCESS | | 1518455322387 | 4 | 5
(7 rows)
~~~

The missing systemMetadata causes a parse error that in turn causes a Marshalling Exception, which I believe caused the NPE. Need to investigate the source code to see if this is the case.

The index processor should ideally handle the lack of sysmetadata more gracefully.