Project

General

Profile

Bug #3128

Production CN listObjects slow response

Added by Dave Vieglais almost 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Low
Assignee:
Ben Leinfelder
Category:
Metacat
Start date:
2012-08-13
Due date:
2013-01-05
% Done:

100%

Milestone:
CCI-1.1
Product Version:
*
Story Points:
Sprint:

Description

listObjects on the CNs is running very slow, about 10seconds for ORC and UCSB, about 3 sec on UNM.

This needs to be improved significantly, ideally in the < 1 seconds range.

Need to identify the cause of the slow response and identify possible work arounds or fixes.

Not sure where the issue is - perhaps in Metacat?

time curl "https://cn-unm-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m3.120s
user 0m0.014s
sys 0m0.003s

time curl "https://cn-orc-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m9.072s
user 0m0.014s
sys 0m0.004s

time curl "https://cn-ucsb-1.dataone.org/cn/v1/object?count=0"
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="/cn/xslt/dataone.types.v1.xsl"?>
real 0m10.121s
user 0m0.015s
sys 0m0.003s


Related issues

Related to Infrastructure - Bug #3065: cn.listObjects slicing broken in production Closed 2012-12-02 2013-01-05
Related to Infrastructure - Task #3465: Improve Metacat listObject slicing New 2012-12-07

History

#1 Updated by Ben Leinfelder almost 9 years ago

  • Status changed from New to In Progress

Running the SQL directly in postgres shows similar timing differences between the three servers - UNM returns results in a third of the time of the other two.

select guid, date_uploaded, rights_holder, checksum, checksum_algorithm, origin_member_node, authoritive_member_node, date_modified, submitter, object_format, size from systemmetadata order by guid;

#2 Updated by Chris Jones almost 9 years ago

  • Target version changed from Sprint-2012.29-Block.4.3 to Sprint-2012.37-Block.5.3
  • Priority changed from Normal to High

#3 Updated by Ben Leinfelder almost 9 years ago

  • Priority changed from High to Low
  • Milestone changed from CCI-1.0.4 to CCI-1.0.5

Deferring to 1.0.5 -- some options for addressing this:
-tune Postgres for sorting
-if count=0, just use a select count(*) rather than fetching rows.

#4 Updated by Chris Jones almost 9 years ago

  • Milestone changed from CCI-1.0.5 to CCI-1.1
  • translation missing: en.field_remaining_hours set to 0.0
  • Due date set to 2012-09-22

#5 Updated by Dave Vieglais almost 9 years ago

  • Due date changed from 2012-09-22 to 2012-10-27
  • Target version changed from Sprint-2012.37-Block.5.3 to Sprint-2012.41-Block.6.1

#6 Updated by Ben Leinfelder almost 9 years ago

  • Status changed from In Progress to Testing

Changed in Metacat codebase.

#7 Updated by Ben Leinfelder almost 9 years ago

  • Due date changed from 2012-10-27 to 2012-11-10
  • Target version changed from Sprint-2012.41-Block.6.1 to Sprint-2012.44-Block.6.2

#8 Updated by Chris Jones over 8 years ago

  • Due date changed from 2012-11-10 to 2013-01-05
  • Target version changed from Sprint-2012.44-Block.6.2 to Sprint-2012.50-Block.6.4

#9 Updated by Ben Leinfelder over 8 years ago

  • Status changed from Testing to Closed

When count=0 we return very quickly now. Please see related issues (#3468) for a continuation of the CN.listObect slicing discussion.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)