Project

General

Profile

Task #4147

Story #3875: Create a dashboard (version 1) for DataONE to provide high level overall system status

Task #3978: Add a CN reporting script that summarizes spatial data for the dashboard

Modify d1_spatial_reporter.py to produce custom JSON output

Added by Chris Jones about 8 years ago. Updated almost 8 years ago.

Status:
In Progress
Priority:
Normal
Assignee:
Chris Brumgard
Category:
d1_cn_tools
Target version:
Start date:
2013-10-30
Due date:
% Done:

30%

Milestone:
None
Product Version:
*
Story Points:
Sprint:

Description

The spatial reporter class has methods to output a number of formats for the incoming vector-based spatial data on the CNs (per-object n,s,e,w bounding coordinates.). We are currently producing GeoJSON, which, for client-side processing, is quite verbose, and becomes very large to transfer over the wire (currently 12MB). I've manually modified the JSON file produced from the spatial reporter to be smaller for the d1_dashboard to render, and those changes need to be added into the spatial reporter class.

Add an _export_json() method that exports a modified geojson object. The output should look like:

{"type: "FeatureCollection",
"features": [
"g": {
"c": [124.234, 44.234]
},
"g": {
"c": [124.234, 44.234]
},
...
]
}

where "g" is short for "geometry" and "c" is short for "coordinates" in GeoJSON. All ASCII spaces and line endings should be removed to decrease size. The identifier and nodeId attributes are currently being encoded as well, and should be removed for version 1. We may add them back in to version 2 as "i" and "n" attributes.

Investigate gzip compression of the exported JSON file for transfer over the wire. This will require some changes to d1_dashboard to unpack the zipped file. Also try shortened lat/lon floating points, depending on the maximum zoom level in d1_dashboard. We want markers to render to within 100m as a first pass, which should keep coastal markers over water as opposed to moved on to land, and vice versa.

History

#1 Updated by Chris Jones about 8 years ago

  • Subject changed from Modify d1-spatial_reporter.py to produce custom JSON output to Modify d1_spatial_reporter.py to produce custom JSON output

#2 Updated by Chris Brumgard about 8 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 70
  • Achieved main objectives of task. Still need to work on shortening the lat/lon points to ensure 100m and getting the d1_dashboard to accept gzip data.

#3 Updated by Chris Brumgard about 8 years ago

Shortened the lat/long coordinates for _export_json to 3 decimal places. 3 decimals places at the equator will be accurate to 111.32 m and improve as coordinates move away from the equator.

#4 Updated by Chris Jones about 8 years ago

  • Target version changed from 2013.44-Block.6.1 to 2014.2-Block.1.1

#5 Updated by Skye Roseboom almost 8 years ago

  • Target version changed from 2014.2-Block.1.1 to 2014.12-Block.2.2

#6 Updated by Skye Roseboom almost 8 years ago

  • Target version changed from 2014.12-Block.2.2 to 2014.14-Block.2.3

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)