Project

General

Profile

Task #7702

abnormal strings in ipAddress field of solr

Added by Robert Waltz over 5 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
Robert Waltz
Category:
d1_log_aggregation
Start date:
2016-03-30
Due date:
% Done:

0%

Story Points:
Sprint:

Description

Several Membernodes have abnormal strings as Ip Addresses. These strings can be found on the MN when querying for their log entries. They are harvested and set in LogAggregation.

on cn-orc-1 I ran these commands:

curl --cert /etc/dataone/client/private/urn_node_CNORC1.pem "https://cn-orc-1.dataone.org/cn/v2/query/logsolr/?q=-ipAddress:/%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D/&wt=xml&facet=true&facet.field=nodeId"

curl --cert /etc/dataone/client/private/urn_node_CNORC1.pem "https://cn-orc-1.dataone.org/cn/v2/query/logsolr/?q=-ipAddress:/%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D\.%5B0-9%5D%7B1,3%7D/+AND+nodeId:*LTER&wt=xml&facet=true&facet.field=nodeId"

etc.

and then I queried for the rows without facets turned on to retrieve the ipaddresses.

urn:node:LTER 27542 rows with 'systemMetadataChanged()'

urn:node:US_MPC 1065 rows with '[replica]'

urn:node:NMEPSCOR 671 rows with '[replica]', 'systemMetadataChanged()'

urn:node:NKN 410 rows with '[replica]', 'systemMetadataChanged()'

urn:node:IARC rows with 241 '[replica]'

urn:node:TFRI 4 rows with '2001:b020:0:71::2cf', '2601:281:8200:e555:19a3:7a0f:d0c5:a25', '2001:4898:80e0:ee43::3', '2001:4898:80e0:ee43::3'

urn:node:KNB 3 rows with 'null', 'coinformatics.'

urn:node:DRYAD 1 row with '160.74.1.163, 211.13.205.134'

urn:node:mnUNM1 1 row with '2.0.0beta6.dt'

Decide if the IpAddress field in solr should be updated to something like 'null'

Decide if LogAggregation should validate fields for correct formatting

Create Tickets for GMN and Metacat to look into the issues.

History

#1 Updated by Robert Waltz over 5 years ago

  • Description updated (diff)

#2 Updated by Robert Waltz over 5 years ago

  • Target version changed from CCI-2.3.0 to CCI-2.4.0

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)