Project

General

Profile

Bug #2620

jibx support for unicode supplemental BMP characters

Added by Ben Leinfelder over 10 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
d1_jibx_extensions
Target version:
Start date:
2012-04-20
Due date:
% Done:

100%

Story Points:
Sprint:

Description

There is an encoding error (client-side) when testing tier 3 member nodes for supplementary escaped unicode characters:

20120419-11:07:57: [INFO]: 27 unicode String:: common-unicode-supplementary-escaped-???? [org.dataone.integration.it.ContextAwareTestCaseDataone]
org.dataone.service.exceptions.ServiceFailure: class java.io.IOException: Illegal character code 0xd800 in content text
at org.jibx.runtime.impl.UTF8StreamWriter.writeTextContent(UTF8StreamWriter.java:271)
at org.jibx.runtime.impl.MarshallingContext.writeContent(MarshallingContext.java:814)
at org.dataone.service.types.v1.Identifier.JiBX_binding_marshal_1_0(Identifier.java)
at org.dataone.service.types.v1.SystemMetadata.JiBX_binding_marshal_1_0(SystemMetadata.java)
at org.dataone.service.types.v1.JiBX_bindingSystemMetadata_access.marshal()
at org.dataone.service.types.v1.SystemMetadata.marshal(SystemMetadata.java)
at org.jibx.runtime.impl.MarshallingContext.marshalRoot(MarshallingContext.java:1021)
at org.jibx.runtime.impl.MarshallingContext.marshalDocument(MarshallingContext.java:1041)
at org.dataone.service.util.TypeMarshaller.marshalTypeToOutputStream(TypeMarshaller.java:73)
at org.dataone.service.util.TypeMarshaller.marshalTypeToOutputStream(TypeMarshaller.java:63)
at org.dataone.mimemultipart.SimpleMultipartEntity.addFilePart(SimpleMultipartEntity.java:125)
at org.dataone.client.MNode.create(MNode.java:332)
at org.dataone.integration.it.MNodeTier3IT.testCreateData_IdentifierEncoding(MNodeTier3IT.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.Verifier$1.evaluate(Verifier.java:34)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)


Subtasks

Task #2621: remove supplementary character test from testIDentifierEncoding testClosedRob Nahf


Related issues

Related to Infrastructure - Bug #1141: unicode guid handling in mn.create() Closed 2010-12-03

History

#1 Updated by Rob Nahf over 10 years ago

This points back to a problem with Jibx marshaling, which doesn't handle the supplementary range characters (codePoints). Specifically fails when marshaling the systemMetadata prior to stuffing it into the MMP file part, and throws the error prior to making the mn.create() call.

Attempted to use the MarshallingContext.marshalDocument(...) that takes a Writer instead of an OutputStream, but got same error.

The responsible class.method in Jibx is UTF8Encoder.writeContent(String text, Writer writer). Can be found at:

http://grepcode.com/file/repo1.maven.org/maven2/org.jibx/jibx-run/1.2.3/org/jibx/runtime/impl/UTF8Escaper.java?av=f
The method seems to be overly restrictive.

Will remove the supplementary-character-containing id from testing, since it consistently fails prior to sending the request to the target node.

#2 Updated by Rob Nahf almost 8 years ago

  • Target version set to Release Backlog
  • Subject changed from MNodeTier3IT encoding test failure to jibx support for unicode supplemental BMP characters
  • Due date set to 2014-10-02
  • Category changed from d1_integration to zz_d1_jibx_extensions
  • Milestone changed from CCI-1.0.0 to None

#3 Updated by Rob Nahf over 7 years ago

  • % Done changed from 100 to 0
  • Project changed from Infrastructure to Java Client
  • Category changed from zz_d1_jibx_extensions to d1_jibx_extensions
  • Target version changed from Release Backlog to CLJ-2.0.0

#4 Updated by Rob Nahf about 7 years ago

  • Target version changed from CLJ-2.0.0 to CLJ

still needs custom development to support, so putting in the backlogs. Not seeing these kinds of identifiers in production yet.

#5 Updated by Rob Nahf about 5 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

JibX no longer being used.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)