Bug #3128
Production CN listObjects slow response
100%
Description
listObjects on the CNs is running very slow, about 10seconds for ORC and UCSB, about 3 sec on UNM.
This needs to be improved significantly, ideally in the < 1 seconds range.
Need to identify the cause of the slow response and identify possible work arounds or fixes.
Not sure where the issue is - perhaps in Metacat?
time curl "https://cn-unm-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m3.120s
user 0m0.014s
sys 0m0.003s
time curl "https://cn-orc-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m9.072s
user 0m0.014s
sys 0m0.004s
time curl "https://cn-ucsb-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m10.121s
user 0m0.015s
sys 0m0.003s
Related issues
History
#1 Updated by Ben Leinfelder over 11 years ago
- Status changed from New to In Progress
Running the SQL directly in postgres shows similar timing differences between the three servers - UNM returns results in a third of the time of the other two.
select guid, date_uploaded, rights_holder, checksum, checksum_algorithm, origin_member_node, authoritive_member_node, date_modified, submitter, object_format, size from systemmetadata order by guid;
#2 Updated by Chris Jones over 11 years ago
- Target version changed from Sprint-2012.29-Block.4.3 to Sprint-2012.37-Block.5.3
- Priority changed from Normal to High
#3 Updated by Ben Leinfelder over 11 years ago
- Priority changed from High to Low
- Milestone changed from CCI-1.0.4 to CCI-1.0.5
Deferring to 1.0.5 -- some options for addressing this:
-tune Postgres for sorting
-if count=0, just use a select count(*) rather than fetching rows.
#4 Updated by Chris Jones over 11 years ago
- Milestone changed from CCI-1.0.5 to CCI-1.1
- translation missing: en.field_remaining_hours set to 0.0
- Due date set to 2012-09-22
#5 Updated by Dave Vieglais over 11 years ago
- Due date changed from 2012-09-22 to 2012-10-27
- Target version changed from Sprint-2012.37-Block.5.3 to Sprint-2012.41-Block.6.1
#6 Updated by Ben Leinfelder over 11 years ago
- Status changed from In Progress to Testing
Changed in Metacat codebase.
#7 Updated by Ben Leinfelder over 11 years ago
- Due date changed from 2012-10-27 to 2012-11-10
- Target version changed from Sprint-2012.41-Block.6.1 to Sprint-2012.44-Block.6.2
#8 Updated by Chris Jones over 11 years ago
- Due date changed from 2012-11-10 to 2013-01-05
- Target version changed from Sprint-2012.44-Block.6.2 to Sprint-2012.50-Block.6.4
#9 Updated by Ben Leinfelder over 11 years ago
- Status changed from Testing to Closed
When count=0 we return very quickly now. Please see related issues (#3468) for a continuation of the CN.listObect slicing discussion.
