Skip to main content

Maritime Monitoring

Maritime monitoring is a operational, near-real time processing of vessels in satellite imagery covering the operational Area of Interest (AOI). It displays offshore oil rig platforms near Andoya, Denmark and newfoundland regions.

File Provider (EO Team)#

Product Description#

Service NameDescriptionData OwnersFile Type
Maritime MonitoringNear-real time processing of vessels in satellite imagery covering the operational Area of Interest (AOI).Pradeep Bobby.shp

Ancilliary Information#

  • Product set-up prerequisites:
    • AOI

Naming Convention#

The following naming convention applies to all maritime monitoring files sent to Coresight:

YYYYMMDD_HHMMSS_SSS_Targets.zipYYYYMMDD_HHMMSS_SSS_Frame.zipchips_YYYYMMDD_HHMMSS_SSS.zipWhere SSS = satellite- Sentinel-1 = S1- TerraSAR-X = TSX- PAZ = PAZ- ICEYE = ICE- Gaofen = GF- COSMO-SkyMed = CSK- COSMO-SkyMed Second Gen = CSG- Capella = CAP- Optical = S2 or L8

Example:

chips_20211014_054101_S1.zip20210407_172657_ICE_Targets.zip20210407_172657_ICE_Frame.zip

Symbology Map#

LabelColor
No label#ffff
Ship_SAR#e64040
Ship_Optical#e69340
Ship_AIS#35c443
Unknown#e8e545
Iceberg#b8edff

File Uploading Instructions#

JIP shapefiles are uploaded via this interface: https://console.cloud.google.com/storage/browser/c-core-andoya-shapefile

The Google account ccore.eos@gmail.com has read and write access.

A few technical details that can be safely ignored:

  • A PubSub (an implementation of the publish-subscribe messaging pattern) topic named projects/c-core-andoya/topics/c-core-andoya-shapefile exists where an event message is published everytime a file is uploaded via the interface above.
  • A PubSub subscription named projects/c-core-andoya/subscriptions/c-core-andoya-shapefile listens to the topic for new event messages and forwards them to the Andoya API.
  • The Andoya API reads the shapefile location from the event message and ingests the shapefile into the system.

For those with GCP access, the PubSub topics and subscriptions are here: https://console.cloud.google.com/cloudpubsub/topic/list?project=c-core-andoya

Recommendation: To codify these PubSub resources as infrastructure as code in a git repository as we have done in the RPAS project.

note

The cloud storage with PubSub pattern allows the EO team to drag and drop a new file into the upload interface. However, if we make a breaking change to our data structures, it's easy for the Coresight team to programmatically copy all shapefiles to a temporary bucket and back to the production bucket, triggering a re-processing of all data.

It's also idempotent, so no matter how many times we copy the same file to our production bucket, the system will always display the latest revision of the file. The cloud storage bucket is also accessible from anywhere in the world with the correct Google credentials.

Coresight#

Developers#

  • Jeremy Hetzel

Directory location:#

Processing#

Description#

Data processing takes place using an ingester that imports, converts, creates and stores files.

Ingester Script#

andoy-api function“handle_message” just extracts the file uri from the event (a combination of bucket id and file key):

URI Types:

1. name – ”CHIPS”.zip2. name – “_targets”.shp   name – ”_frame”.shp
  1. If the uri name contains “CHIPS”, it is a zip archive of image chips e.g. small previews of how the ship or iceberg looks when detected in a SAR (e.g. Sentinel 1 or RCM) image. (gs://c-core-andoya-shapefile/chips_20211014_054101_S1.zip)
    The API then converts those image chips into images eg: converts all .tiff files to .png and moves all image(.png) files to cloud storage (https://console.cloud.google.com/storage/browser/c-core-andoya-images)
  2. Otherwise, the uri is a shapefile containing a bunch of ships or icebergs, which are referred to as "targets". (gs://c-core-andoya-shapefile/20210222_094801_S1_Shapefiles.zip)
    From the shapefiles, we clean, organize and transform the data in necessary format. This both target_layer and Frame_layer ( ArcticX shapefile) are then Converted to GeoJSON(scene_id.geojson - gs://c-core-andoya-geojson/20210222_094801_S1.geojson)
    In next step, we get bounding box from ArcticX shapefile and from that bbox, we can derive longitude and latitude from the centroid of polygon.
  • Guess_AOI(area of interest): this function looks up a list of bounding boxes ("areas of interest") in a firestore collection(for all three regions - andoya, denmark, JIP) and checks which bounding box the current shapefile spatially intersects with. It then labels the data associated with the shapefile the area of interest's ID ("aoi_id").
  • Calculation of summary: extracts class from features in feature collection and iterates through all the features to find the number of total classes in features.
    For example, 
In Features: 20210304_091546_S1_1: AIS (class), <br></br>             20210304_091546_S1_2: AIS (class), <br></br>             20210304_091546_S1_3: AIS (class) <br></br>Then in metadata: 20210304_091546_S1 -> summary: AIS=3

Ingester Functions#

  • File receiving: We receive zip files from EO team from our API endpoint.
  • Initial Storage: API stores uploaded files to cloud storage at: gs://c-coreandoya-shapefile/File conversion: Initially zip files are uploaded with two types of naming convention
  • File conversion: Initially zip files are uploaded with two types of naming convention:
  1. If the uri name contains “CHIPS”, it is a zip archive of .tiff images. We convert it to .png images and store them under: gs://c-core-andoya-images/
  2. if the uri is a shapefile, then we convert it to GeoJson and extract the metadata.
  • Final Storage: Each ingested product is in Firestore collection. Notification email: The notification email from Coresight will contain a link (SAR SAFE file) to download the zipped image

Service Interface#

  • The web interface displays the most recent satellite ice classification analysis.
  • The interface retrieves product information through the firestore collection.