Project

General

Profile

CCI-0.4

2010-06-01

100%

234 issues   (234 closed — 0 open)

  • Initial implementation of metadata replication and indexing
  • Functional use cases addressed: 2, 3, 6, 10
  • Initial implementation of selected MNs
  • CN Hardware procured
  • CN implementation using Metacat + Mercury with API wrapper
  • MN - CN communications secured
  • Mercury search index population trigger implemented
  • CN-CN replication of metadata
Issues by
Story

55/55
Task

153/153
Bug

23/23
Feature

2/2
Decision

1/1
Related issues
Story #79: Adapt mercury to use metacat document store
Story #262: Tool that will exercise the APIs implemented on both MN and CN
Story #269: Execute the CN search method to receive results in a standard DataONE format
Story #288: Verify CN can proxy Metacat 's get() method through REST interface
Story #326: Clean up design documentation for 0.3 and 0.4 releases
Story #336: Provide high level summaries for sponsor review
Story #346: Implement support for arbitrary identifiers in Metacat
Story #348: Implement getSystemMetadata() method for Metacat
Story #349: Execute getSystemMetadata CN to receive results in DataONE format
Story #350: Implement listObjects() method for Metacat
Story #351: Implement getLogRecords() method for Metacat
Story #352: Test and verify all MN 0.3 methods for Metacat
Story #353: Deploy CN Service from debian package install
Story #356: Refactor CN service layer
Story #358: Create Metacat metadata design document for use by CN service
Story #397: As the manager of GMN, I need to have a standardized hardware or virtual hardware environment and standardized OS in which GMN can be installed.
Story #400: create service API definition packages in Java
Story #441: Refactor cn_service to use service_api module
Story #446: Refactor the python implementation to utilize a library of common types
Story #447: Add DataONE Exceptions to Mercury
Story #451: ORNL/DAAC - prepare to support the python MN software stack
Story #452: Setup hardware and connectivity for MN Service VM for ORNL/DAAC
Story #453: Generate system metadata from ORNL/DAAC for python MN service
Story #454: As the GMN manager, I need to have a standardized software environment in which GMN can be installed
Story #461: Document GMN requirements
Story #473: As the manager of the Generic Member Node Stack (GMN), I need GMN to exactly match the specifications in the System Architecture documentation.
Story #521: Verify GMN documentation
Story #526: As the manager of GMN, I need to have a version of GMN that can interact with Dryad
Story #533: As a CN, I need to be able to obtain a skeleton system metadata object from Dryad for any of its objects at any time.
Story #535: As the GMN manager, I need to provide documentation for the Dryad REST interface so that the Dryad developer can implement the functionality required for registering its objects with DataONE
Story #536: As the GMN manager, I need to deploy GMN as a DataONE adapter for Dryad
Story #561: Version of GMN that can interact with ORNL / DAAC
Story #562: Generate skeleton system metadata for ORNL / DAAC objects
Story #563: Deploy GMN as a DataONE adapter for ORNL / DAAC
Story #565: Adapt Metacat to act as object store for CN service
Story #568: As the GMN manager, I need to have an automated installer for GMN
Story #586: Select appropriate open source license
Story #591: create system metadata for legacy objects in KNB
Story #593: design the synchronization service for CNs
Story #594: specify public API and REST interface for CN synchronization service
Story #595: Design and implement the retrieve/store content synchronization service component
Story #602: Create egg for GMN
Story #605: Adapt GMN for ORNL DAAC
Story #609: Design and implement the scheduler for synchronization service component
Story #610: Design and implement a processing queue for synchronization service component
Story #633: Design, implement and deploy a monitoring system for the infrastructure
Story #647: Setup static MN instances that the CN harvest mechanism can use to pull test data and load into Metacat + Mercury.
Story #655: Metacat notification system
Story #661: Create registry for Member Nodes for Synchronization Harvester
Story #663: Utilize a simple objectFormat registry to determine which configuration to feed the Mercury indexer
Story #664: Implement CN_crud.resolve method
Story #667: Implement a VM that will host the integration tests
Story #674: Round-trip testing
Story #679: Setup, implement and test replication across three CN instances hosted at UNM, ORC, NCEAS.
Story #692: Design and implement the initial R DataONE client
Task #162: Determine long-term network and physical location for ORC hardware
Task #175: Identify an infrastructure monitoring framework
Task #184: Clarify versions leading up to year one milestone
Task #257: Ensure Synchronization and Replication API terminology is consistent
Task #264: Refactor client side to support streaming
Task #265: Implement resolve() method in pyd1
Task #267: (Task) Implement getLogMessages() inPyD1 client
Task #270: Verify CN Rest Controller will work for Mercury
Task #271: Abstract the HTML Request Wrapper into Pojo or service for CN Service
Task #272: Take out hard coding of values, place them in configuration file
Task #273: Verify Request is received with correct information in Mercury
Task #274: Verify Response is returned from Mercury in correct format
Task #275: Create Unit Tests for Mercury Rest Controller
Task #276: Integrate unit Tests into Maven
Task #289: Create Unit Tests for Metacat Proxy Rest Controller
Task #290: Integrate unit Tests into Maven
Task #291: Run Integration Testing package for CN get() method
Task #308: modify cn-buildout to generate self-signed key
Task #343: Resolve synonyms
Task #413: Refactor Testing Structure
Task #414: Integrate Unit tests into Maven
Task #417: Implement Exceptions In wrapper
Task #420: implement and store checksum for metacat-stored data and metadata
Task #421: calculate and store size for metacat metadata and data objects
Task #422: Implement getSystemMetadata() method and REST service for Metacat
Task #423: Implement listObjects() method for Metacat
Task #424: extend current Metacat logging tables with new fields
Task #425: extend existing metacat logging calls to records proper information
Task #426: modify metacat getlog() method to return DataONE log format
Task #427: expose REST service to call getlog()
Task #428: write unit tests for Metacat getLogRecords() service
Task #429: run integration tests for all Metacat MN interfaces
Task #432: create an svn directory structure for the cn-buildout of cn service
Task #434: put all the files in the correct places for the CN directory structure
Task #435: create control file
Task #436: create the post-inst
Task #437: edit the dataone-os-core post-inst
Task #438: edit the Makefile
Task #443: Document the decided approach
Task #444: merge shared code
Task #448: Confirm Accept header pass thru
Task #449: Create Wrapper metthod in servlet for opensearch
Task #450: Write unit test to validate exception handling
Task #456: Setup a VM for the Python MN Service for ORNL/DAAC
Task #457: Document administrative details for ORNL/DAAC MN VM
Task #458: Document the install process of GMN
Task #459: Document the version and install process (including details such as filesystem locations and configuration) of system components that need to be installed
Task #465: Test entire installation procedure
Task #466: Determine which components must be included in the distribution (only DataONE specific components, files, file structures and configuration files are included)
Task #467: Define a metacat query for retrieving sysmeta for an object
Task #477: Ensure VM matches the specifications for operating the MN software
Task #481: Ensure that the implementation priority / sequence / version described in the PMP and specifications are aligned
Task #483: Provide a high level overview section in the archtiecture documentation
Task #484: Compose Unit tests for refactored methods
Task #486: write unit test for getSystemMetadata()
Task #487: adapt resultset return format to conform to DataONE resultset spec
Task #488: write unit test for listObjects()
Task #489: Test package creation on clean VM
Task #491: adapt unit tests for new methods
Task #494: Change XSLT in PilotCatalogue
Task #495: Modify Mercury unit tests to conform with new standards
Task #496: Document aspects of Mercury
Task #497: Design changes to incorpororate new System Metadata schema into Mercury
Task #514: Implement MN_replication.synchronizationComplete() (token, GUID)
Task #515: Test MN_replication.synchronizationComplete() (token, GUID)
Task #516: Implement MN_replication.synchronizationFailed() (token, GUID, problemDescription)
Task #522: Provide the documentation and GMN package to 3rd party and have them independently perform the installation
Task #529: Implement mets.xml parser to retrive values for system metadata for the science data
Task #532: Identify candidate datasets for participation in the prototype implementation
Task #534: Create export mechanism for DataONE System Metadata XML
Task #537: Define mapping between ORNL DAAC metadata and DataONE SystemMetadata
Task #538: Implement mechanism for tracking obsoletes / obsoleted by
Task #544: Modify existing example ORNL DAAC XML files to include DataONE System Metadata elements
Task #548: Provide documentation to Dryad SA on how to set up the standardized hardware or virtual hardware environment
Task #552: Use ssh connection to copy over GSM distribution
Task #553: Use ssh connection to install GSM
Task #554: Use the Dryad REST interface documentation to test that the GSM installation properly handles Dryad REST calls, using CSV file supplied by Dryad manager
Task #555: Provide the Dryad REST interface documentation to the Dryad Manager so that he can independently test that the REST interface conforms to the documentation
Task #556: Use the CN REST interface documentation to test that the GSM installation properly handles the CN rest calls
Task #557: refactor service_api classes in Java to reflect specification changes
Task #558: Verify Metacat can proxy getSystemMeta() Method through REST interface
Task #559: Fix bugs introduced during refactoring related to tasks in this story.
Task #560: Refactoring and bugfixes related to API call compliance
Task #564: Merge ITK exception code with mine
Task #566: Implement create() method for Metacat
Task #567: Implement update() method for Metacat
Task #569: Create Python Egg for GMN
Task #570: include GUID identifier information in Metacat replicate function
Task #572: create system metadata database
Task #573: Refactor GMN to use Django middleware layer for all tasks common between views
Task #575: remove obsolete types, fixing any errors that may result
Task #576: Write a summary of activities for 3rd Q report
Task #588: Define attribution text for code authors
Task #590: Adapt GMN to Dryad environment
Task #596: develop license and copyright policy
Task #599: Add DataONE exceptions into Cn Rest Service Proxy
Task #603: Create a Python egg for GMN
Task #604: Test GMN egg
Task #606: Abstract out differences between Dryad and DAAC
Task #607: Set up GMN on ORNL DAAC VM
Task #611: Refactor MetacatHandler and CrudService to not use ServletContexts in the API
Task #612: Write method to merge of sys and sci meta files
Task #613: create utility method to read a filepath and load the file as XML Document
Task #614: Write method to extract Sci Metadata file name from Sys metadata
Task #615: Write method to merge Sci and Sys meta XML Documents
Task #616: Write method to Save merged XML Doc to mercury ingestion dir
Task #617: Write unit test for Merge methods
Task #619: refactor metacat service layer to use string error codes instead of ints
Task #620: Check that access control works in the metacat service layer
Task #621: Compare and Merge various buildouts
Task #622: Modify Common Maven pom
Task #625: Download ActiveMQ, analyze installation instructions
Task #626: Determine Messaging Service for Queues
Task #631: Rebuild cn-dev with revised applications
Task #634: Define the metrics that need to be monitored and recorded
Task #635: Select the monitoring framework and design the deployment of the selected framework
Task #636: Deploy the monitoring system central monitoring service
Task #637: Add the necessary packages to the CN deployment packages
Task #638: Implement monitoring of MN services
Task #641: Write MN health API method for monitoring
Task #642: Create the plugin executable to call the MN health API method
Task #643: configure & test Cacti for MN monitoring
Task #644: create instructions for installing nrpe on nodes to be monitored (CNs)
Task #645: install and configure nrpe on core nodes
Task #649: setup a local machine for developing the harvest functionality
Task #650: Enable the harvest process to pull test data from a HTTP server
Task #651: Update the harvestor to read mn_replicate.listObjects()
Task #656: Add appropriate logging messages for CRUD event handling
Task #657: Add log4j logger for ResourceHandler to announce CRUD messages
Task #659: create LDAP user to enable monitoring via SSH
Task #665: create schema for mn registry
Task #666: Serialize MN Registry in ServiceApi
Task #668: Create a VM to host integration tests
Task #672: Install dependencies for executing tests
Task #673: Write documentation for MN health API
Task #675: Create script for round-trip testing
Task #676: ITK Client improvements
Task #677: Modify and Release Mercury
Task #678: Ensure ACPI is installed on the VM and update install docs to reflect this
Task #680: Prepare host hardware for KVM clients (check with oak ridge on status of host)
Task #681: Define the specification for the CN VMs: CPU, RAM, DISK (use existing cn-dev specs, document)
Task #682: Deploy three CN virtual machines. Install VMs, setup DNS, install CN packages
Task #683: Configure Metacat replication and document setup
Task #688: Create objectFormat enumeration tool
Task #689: Match mercury xpath rules with objectFormats being used
Task #690: Test metadata field value extraction
Task #691: Ensure mercury parser is using objectFormat to identify the correct processor to use
Task #693: Design the R client component(s)
Task #694: Implement the initial R client and associated tests
Task #703: complete listObjects REST interface
Task #711: create and test xpath.xml file for Dryad objectFormat
Task #712: create and test xpath.xml file for the KNB objectFormat
Task #733: modify license to Apache 2 in d1-java and service-api-java
Bug #578: ServiceApi not compiling on Mac
Bug #623: ObjectList became ListObjects in the XSD
Bug #652: Metacat will not create document with guid that ends in .\d+
Bug #653: Metacat create fails due to recent authentication changes in CRUD
Bug #654: Change metacat to new use log type classes and remove old log type classes
Bug #660: Metacat appends checksum algorithm attribute to checksum in returned ObjectLists
Bug #669: Dryad MN not returning 404 correctly
Bug #670: Dryad MN listobject response not complete
Bug #671: identifier mismatch between listObject, getSystemMetadata
Bug #696: D1Client failing to serialize document stream correctly in multipart form
Bug #697: test failures for d1-java
Bug #698: listobject response from dryad not conforming to schema
Bug #699: Metacat KNB not returning valid responses
Bug #700: Authentication required for listObjects and get
Bug #701: Auth credentials should be put into POST body
Bug #702: Replace MMP workaround with more robust solution
Bug #705: mercury xpath testing script failing on Hudson
Bug #715: GMN object list response count attribute always zero
Bug #716: GMN object retrieveal stops at 1MB
Bug #717: Metacat MN services not responding to get or getSystemMetadata
Bug #718: Random selection of metacat identifiers seem to have extra whitespace at end
Bug #720: Metacat MN services not setting content-type in response
Bug #721: GMN log response seems to fail when iterating to end of the log record collection
Feature #309: refactor MN.create and MN.update methods and REST bindings
Feature #648: generate KVM config that will build a static MN setup.
Decision #256: How should the MN-CN be authenticated for Y1 release cycle
Add picture from clipboard (Maximum size: 14.8 MB)