Task #7702
abnormal strings in ipAddress field of solr
0%
Description
Several Membernodes have abnormal strings as Ip Addresses. These strings can be found on the MN when querying for their log entries. They are harvested and set in LogAggregation.
on cn-orc-1 I ran these commands:
curl --cert /etc/dataone/client/private/urn_node_CNORC1.pem "https://cn-orc-1.dataone.org/cn/v2/query/logsolr/?q=-ipAddress:/%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D/&wt=xml&facet=true&facet.field=nodeId"
curl --cert /etc/dataone/client/private/urn_node_CNORC1.pem "https://cn-orc-1.dataone.org/cn/v2/query/logsolr/?q=-ipAddress:/%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D/+AND+nodeId:*LTER&wt=xml&facet=true&facet.field=nodeId"
etc.
and then I queried for the rows without facets turned on to retrieve the ipaddresses.
urn:node:LTER 27542 rows with 'systemMetadataChanged()'
urn:node:US_MPC 1065 rows with '[replica]'
urn:node:NMEPSCOR 671 rows with '[replica]', 'systemMetadataChanged()'
urn:node:NKN 410 rows with '[replica]', 'systemMetadataChanged()'
urn:node:IARC rows with 241 '[replica]'
urn:node:TFRI 4 rows with '2001:b020:0:71::2cf', '2601:281:8200:e555:19a3:7a0f:d0c5:a25', '2001:4898:80e0:ee43::3', '2001:4898:80e0:ee43::3'
urn:node:KNB 3 rows with 'null', 'coinformatics.'
urn:node:DRYAD 1 row with '160.74.1.163, 211.13.205.134'
urn:node:mnUNM1 1 row with '2.0.0beta6.dt'
Decide if the IpAddress field in solr should be updated to something like 'null'
Decide if LogAggregation should validate fields for correct formatting
Create Tickets for GMN and Metacat to look into the issues.
History
#1 Updated by Robert Waltz over 8 years ago
- Description updated (diff)
#2 Updated by Robert Waltz over 8 years ago
- Target version changed from CCI-2.3.0 to CCI-2.4.0