Add new DataONE Object format for HDF4/5 file formats
HDF 4 and 5 are efficient binary formats for data commonly used in science: https://en.wikipedia.org/wiki/Hierarchical_Data_Format, https://www.hdfgroup.org/solutions/hdf5/. I don't think we have a lot of content in this format, if any, but it's a pretty common format and a good one at that.
I did some research on MIME types and file extensions:
Re: MIME type:
The recommended content is application/x-hdf5 for data in HDF5 or application/x-hdf for data in earlier versions.
- https://en.wikipedia.org/wiki/Hierarchical_Data_Format lists a few of the variants I've seen
- The R hdf5 package uses the h5 extension (https://github.com/grimbough/rhdf5/tree/master/inst/testfiles) so I went with this
- .hdf4 and .hdf5 seem common too but h4/h5 seems a tad more common
Here are the details for each of the new formats:
- formatName: Hierarchical Data Format version 4 (HDF4)
- extension: h4
- formatName: Hierarchical Data Format version 5 (HDF5)
- extension: h5
#2 Updated by Bryce Mecum about 3 years ago
This looks great. I would suggest that the proper media types should be application/x-hdf annd application/x-hdf5 rather than application/octet-stream. Best to be specific. Thoughts?
I thought this too, but based my decision off of how other formats were done. e.g., the MATLAB formats (including 7.3 which is HDF5) and RAW are typed as octet-stream. I'm cool with either way. And now that I look, I think your suggestion is probably the better one.
Let's go with with
mediaType values that match the
#3 Updated by Jing Tao about 3 years ago
They will look like:
<objectFormat> <formatId>application/x-hdf</formatId> <formatName>Hierarchical Data Format version 4 (HDF4)</formatName> <formatType>DATA</formatType> <mediaType name="application/x-hdf"/> <extension>h4</extension> </objectFormat> <objectFormat> <formatId>application/x-hdf5</formatId> <formatName>Hierarchical Data Format version 5 (HDF5)</formatName> <formatType>DATA</formatType> <mediaType name="application/x-hdf5"/> <extension>h5</extension> </objectFormat>
#5 Updated by Jing Tao about 3 years ago
- % Done changed from 0 to 100
- Status changed from New to Closed
The formats have been added to the code (dataone-cn-metacat and d1_common_java). The formats also were added to cn-dev, cn-dev-2, cn-sandbox, cn-sandbox-2, cn-stage, cn-stage-2 and production cns.