Task #7612
Story #7611: ORNLDAAC fails during log aggregation
Contact ORNLDAAC
30%
Description
ORNLDAAC administrative contact needs to be informed that the MemberNode is failing during a REST request call to its logging endpoint. After ORNLDAAC has fixed the issue, then D1 will need to evaluate log aggregation responses before resetting log aggregation to harvest node.
History
#1 Updated by Laura Moyers about 8 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 30
Contacted Bob Cook and Giri Palanisamy; awaiting response.
#2 Updated by Robert Waltz about 8 years ago
Not sure what the goal is here, but it looks like D1 is trying pull with a count greater than the available results.
For the specified dates, the count is 22. So, 1,000 returns an error.
From Bob Cook
#3 Updated by Dave Vieglais about 8 years ago
If the requested number of objects is larger than the number available, then the response should include those available.
#4 Updated by Robert Waltz about 8 years ago
I tested another Mercury instance with a different query to determine if we receive the same type of error:
With curl I can query "http://mercury-ops2.ornl.gov/RGD_MN/mn/v1/log?fromDate=2014-10-21T115:46:00.000%2B00:00&toDate=2016-02-11T00:00:00.000%2B00:00&start=0&count=0"
and receive the results
With curl I can query "http://mercury-ops2.ornl.gov/RGD_MN/mn/v1/log?fromDate=2014-10-21T115:46:00.000%2B00:00&toDate=2016-02-11T00:00:00.000%2B00:00&start=0&count=1000"
and receive 393 records back without error.
#5 Updated by Robert Waltz about 8 years ago
As Dave noted, the count parameter shows "The maximum number of log records that should be returned in the response. The Member Node may return fewer and the caller should check the total in the response to determine if further pages may be retrieved."
To answer Bob's question, the goal is to harvest log records from ORNLDAAC through the DataONE API. It appears that ORNLDAAC is incorrectly interpreting count as 'an exact number of log records that should be returned', if indeed the error is due to the requested count being larger than the actual number of records to be returned.
#6 Updated by Robert Waltz about 8 years ago
Using curl, here are other scenarios that are similar to the original error:
With curl I can query ORNLDAAC for all of its records:
http://mercury-ops2.ornl.gov/ornldaac/mn/v1/log?start=0&count=0
ORNLDAAC returns:
If I just ask for a small subset at the end of that query, using start of 8800 and count of 1000:
http://mercury-ops2.ornl.gov/ornldaac/mn/v1/log?start=8800&count=1000
I do receive 32 records back.
Similarly, if I add dates to the above queries:
I do receive 32 records back.
