Skip to main content

Hosting RPA data using mosaic.json

Definitions#

RPA - Remotely Piloted Aircraft

orthomosaic - a raster image created from merging multiple source images.

ortho tile - A piece of an orthomosaic that has been gridded. Usually produced due to file size considerations.

COG - Cloud optimized geotif

STAC - Spatial Temporal Asset Calalog

Links

Tilers#

Specifications#

Libraries#

Platform Components

  1. Dynamic Tiler
  2. STAC Entries
  3. Auth service
  4. Cog transformation pipeline
  5. Web Interface

Dynamic Tiler#

NOTE: An up and coming dynamic tiling library titiler is working working towards merging the functionality of both single and mosaic based tiling methods. At some point this will unify our tiling apis.

Dynamic tilers allow for the visualization of raster imagery at different view levels without having to pre-generate tiles. Traditionally a tile cache would be generated that would have all levels of tiles generated ahead of time. Since COGs allow for the streaming of data tiles can now be generated at runtime with minimal latency. This allows for a platform to very easily support many images through a common endpoint.

Mosaic.json tiler#

Because of the incredible resolution of orthomosaics, some drone processing platforms break up the data into 100m squares of 8bit RGB for visualization. This can result in a small aoi being over 100 files. this would be awkward to manage within the STAC entry, making it very hard to browse. Mosiac.json is a format for represting tiled datasets. cogeo has created a rio-tiler based tiler that reduces visualization the entire tiled orthomosaic using a single endpoints. Previously each ortho tile would have to be given its own layer, and would result in many empty aoi returns.

Lambda Tiler#

The lambda tiler allows for the display of arbitrary COGS by selecting the bands and ordering to visualize raster information. This is useful for creating derived indices by combining individual cog bands such as NDVI. The limitation on this is it is designed for a single raster at a time.

STAC Entries#

This varies a little from traditional STAC entries in a couple ways

  • Uav Data will be very diverse in the sensors used and number of bands created. Additionally many uav platforms produce products by orthomosaicing individual images into a geo-referenced surface using photogrametry
  • Different users will require their own catalogs to host their data and keep it private or sharable with specific user groups. Stac entries will contain information about the eo data collected as well as the platform and conditions under which is was collected. A catalog can be hosted with a static reference catalog.json in a secure bucket. This catalog would provide links to other collections that the account has access to. Alternatively, an api can be created which would query a the currently cataloged data and only return the collections visible to the account.

RPA STAC screenshots#

STAC Assets for RPA imageryEO Band info for RPA imagery

Auth Service#

The Auth service uses AWS IAM roles and custom scopes in order to grant fine grained access permissions. This makes use of AWS cognito to create users groups associated with Data Access roles. A user management ui allows for the creation of new users, adding users to groups and assigning of data permissions to groups. The general structure of the data access policy is as follows

c-core-secure ├─ <public>/ │   ├─ <public-dataset-collection1>/ │   └── <public-dataset-collection2>/ ├─ <client1>/ │   ├── <client1-dataset-collection1>/ │   └── <client1-dataset-collection2>/ ├─client2     ├──<client2-dataset-collection1>/     └──<client2-dataset-collection2>/

User access policies can them be established in order to grant access to either read or write at a client, collection or even individual dataset level.

The authorization strategy is designed around group membership. Organizations when enrolled will have 1 or more groups created for them. These groups are then given any number of data, api, or application access permissions. When individual user accounts are added. They are placed in a group and inherit the permissions of that group

Cog Pipeline#

In order to prepare data for dynamic display it needs to be put in a cloud optimized format. This allows for streaming of the data. The transformation pipeline makes use of the rasterio library to create cogs in web optimized formats. These cogs are then placed in a secure AWS s3 bucket and catalog references are created for them.optimized. Current pipeline uses google cloudRun to host self contained cog generation applications that are accessible behind a api endpoint.

Web interface#

The web interface is intended to be a general purpose site in which a member of the platform can explore, share and visualize their collected data. ....

Dynamic Tiler using Expresion to display ndviDynamic Tiler using mosaic.json to combine 200 tiles

Platform Experience#

The rpa platform is the beginnings of providing next generation data curation and distribution to RPA providers. Additionally it can provide a link between RPA data consumers and providers. Users are able to upload data and share it other groups. Or add another group to a collection so that data becomes discoverable by them automatically. The interaction between the fine grained user access and dynamic catalog allow for RPA data users to each have their own secure catalog, and organize data from multiple vendors in a single location. Use of dynamic tilers allows for the sharing of viewers urls with the display elements included in the url providing a consistent viewing experience.

Resources#