Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension typeclasses #85

Merged
merged 26 commits into from
May 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d4eea57
Add item extension typeclass and LabelProperties evidence
jisantuc May 14, 2020
a542890
expunge superfluous label:assets on links
jisantuc May 14, 2020
dbfed6d
add linkextension typeclass, evidence for LabelLinkExtension
jisantuc May 14, 2020
a28eccd
add item extension evidence to layer extension, rename
jisantuc May 14, 2020
6228757
add label link extension to serdespec for links
jisantuc May 14, 2020
d007214
add catalog and asset extensions
jisantuc May 14, 2020
949f318
add rest of extensions / tests (failing)
jisantuc May 14, 2020
dba2160
fix collection decoder
jisantuc May 15, 2020
c97c2cb
update testing static json
jisantuc May 15, 2020
c842be5
add itemcollection extension
jisantuc May 15, 2020
02b2e5b
update pr template
jisantuc May 15, 2020
304f2a2
Update changelog
jisantuc May 15, 2020
5b3aec8
Move asset extension type to extensions
jisantuc May 15, 2020
3f9cfb4
test asset extension
jisantuc May 15, 2020
d95ba5e
Update comment
jisantuc May 15, 2020
5c1d81f
layer ids string list -> non empty string list
jisantuc May 18, 2020
2e89e74
nonemptystrings and sets oh my
jisantuc May 18, 2020
9a349f5
layer extension requires nonemptylist
jisantuc May 18, 2020
aa8a265
provide default for extensionFields
jisantuc May 19, 2020
75a67f9
s/getProperties/getExtensionFields
jisantuc May 19, 2020
680f399
s/extend/addExtensionFields
jisantuc May 19, 2020
6dcbdc1
fix and test layer item extension decoder
jisantuc May 19, 2020
853d9b6
test that item ser-de works with extensions
jisantuc May 19, 2020
ade5ae9
derive case class fields with shapeless
jisantuc May 19, 2020
36721b3
add syntax package and test where possible
jisantuc May 19, 2020
8f0ad3e
lint
jisantuc May 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Brief description of what this PR does, and why it is needed.
### Checklist

- [ ] New tests have been added or existing tests have been modified
- [ ] Changelog updated

### Notes

Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

### Added

- Created typeclasses for linking extensions to the items they extend [#85](https://github.com/azavea/stac4s/pull/85)

### Changed

### Deprecated
Expand Down
75 changes: 53 additions & 22 deletions modules/core/src/main/scala/com/azavea/stac4s/ItemCollection.scala
Original file line number Diff line number Diff line change
@@ -1,42 +1,73 @@
package com.azavea.stac4s

import cats.Eq
import cats.implicits._
import io.circe._
import io.circe.refined._
import io.circe.syntax._
import shapeless.LabelledGeneric
import shapeless.ops.record.Keys

final case class ItemCollection(
_type: String = "FeatureCollection",
stacVersion: StacVersion,
stacExtensions: List[String],
features: List[StacItem],
links: List[StacLink]
links: List[StacLink],
extensionFields: JsonObject = ().asJsonObject
)

object ItemCollection {

private val generic = LabelledGeneric[ItemCollection]
private val keys = Keys[generic.Repr].apply
val itemCollectionFields = keys.toList.flatMap(field => substituteFieldName(field.name)).toSet

Comment on lines +22 to +25
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put it into a single function to reduce some boilerplate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the thing I had trouble with implicits with -- it wasn't obvious to me how to get the implicit evidence required to have getFieldNames[T] cooperate. Specifically, the evidence provided in the example blog post:

implicit def toAttributes[T, Repr <: HList, KeysRepr <: HList](
    implicit gen: LabelledGeneric.Aux[T, Repr],
    keys: Keys.Aux[Repr, KeysRepr],
    traversable: ToTraversable.Aux[KeysRepr, List, Symbol]
  ):

was insufficient for Keys[generic.Repr] when I tried making it generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #89

implicit val eqItemCollection: Eq[ItemCollection] = Eq.fromUniversalEquals

implicit val encItemCollection: Encoder[ItemCollection] = Encoder.forProduct5(
"type",
"stac_version",
"stac_extensions",
"features",
"links"
)(itemCollection =>
implicit val encItemCollection: Encoder[ItemCollection] = new Encoder[ItemCollection] {

def apply(collection: ItemCollection): Json = {
val baseEncoder: Encoder[ItemCollection] = Encoder.forProduct5(
"type",
"stac_version",
"stac_extensions",
"features",
"links"
)(itemCollection =>
(
itemCollection._type,
itemCollection.stacVersion,
itemCollection.stacExtensions,
itemCollection.features,
itemCollection.links
)
)

baseEncoder(collection).deepMerge(collection.extensionFields.asJson)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first big increase in verbosity-- everything with extensionFields has a baseEncoder that gets mashed together with whatever's in the extensionFields attribute

}
}

implicit val decItemCollection: Decoder[ItemCollection] = { c: HCursor =>
(
itemCollection._type,
itemCollection.stacVersion,
itemCollection.stacExtensions,
itemCollection.features,
itemCollection.links
c.get[String]("type"),
c.get[StacVersion]("stac_version"),
c.get[Option[List[String]]]("stac_extensions"),
c.get[List[StacItem]]("features"),
c.get[Option[List[StacLink]]]("links"),
c.value.as[JsonObject]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the second part of the big increase in verbosity -- derivation is no longer correct for any of these, since we need "root document without the vanilla fields"

).mapN(
(
_type: String,
stacVersion: StacVersion,
extensions: Option[List[String]],
features: List[StacItem],
links: Option[List[StacLink]],
document: JsonObject
) =>
ItemCollection(_type, stacVersion, extensions getOrElse Nil, features, links getOrElse Nil, document.filter({
case (k, _) => !itemCollectionFields.contains(k)
}))
)
)

implicit val decItemCollection: Decoder[ItemCollection] = Decoder.forProduct5(
"type",
"stac_version",
"stac_extensions",
"features",
"links"
)(ItemCollection.apply)
}
}
72 changes: 59 additions & 13 deletions modules/core/src/main/scala/com/azavea/stac4s/StacCatalog.scala
Original file line number Diff line number Diff line change
@@ -1,33 +1,79 @@
package com.azavea.stac4s

import cats.Eq
import cats.implicits._
import io.circe._
import io.circe.syntax._
import shapeless.LabelledGeneric
import shapeless.ops.record.Keys

final case class StacCatalog(
stacVersion: String,
stacExtensions: List[String],
id: String,
title: Option[String],
description: String,
links: List[StacLink]
links: List[StacLink],
extensionFields: JsonObject = ().asJsonObject
)

object StacCatalog {

private val generic = LabelledGeneric[StacCatalog]
private val keys = Keys[generic.Repr].apply
val catalogFields = keys.toList.flatMap(field => substituteFieldName(field.name)).toSet

implicit val eqStacCatalog: Eq[StacCatalog] = Eq.fromUniversalEquals

implicit val encCatalog: Encoder[StacCatalog] =
Encoder.forProduct6("stac_version", "stac_extensions", "id", "title", "description", "links")(catalog =>
implicit val encCatalog: Encoder[StacCatalog] = new Encoder[StacCatalog] {

def apply(catalog: StacCatalog): Json = {
val baseEncoder: Encoder[StacCatalog] =
Encoder.forProduct6("stac_version", "stac_extensions", "id", "title", "description", "links")(catalog =>
(
catalog.stacVersion,
catalog.stacExtensions,
catalog.id,
catalog.title,
catalog.description,
catalog.links
)
)

baseEncoder(catalog).deepMerge(catalog.extensionFields.asJson)
}
}

implicit val decCatalog: Decoder[StacCatalog] = { c: HCursor =>
(
c.get[String]("stac_version"),
c.get[List[String]]("stac_extensions"),
c.get[String]("id"),
c.get[Option[String]]("title"),
c.get[String]("description"),
c.get[List[StacLink]]("links"),
c.value.as[JsonObject]
).mapN(
(
catalog.stacVersion,
catalog.stacExtensions,
catalog.id,
catalog.title,
catalog.description,
catalog.links
)
version: String,
extensions: List[String],
id: String,
title: Option[String],
description: String,
links: List[StacLink],
document: JsonObject
) =>
StacCatalog.apply(
version,
extensions,
id,
title,
description,
links,
document.filter({
case (k, _) => !catalogFields.contains(k)
})
)
)

implicit val decCatalog: Decoder[StacCatalog] =
Decoder.forProduct6("stac_version", "stac_extensions", "id", "title", "description", "links")(StacCatalog.apply)
}
}
123 changes: 75 additions & 48 deletions modules/core/src/main/scala/com/azavea/stac4s/StacCollection.scala
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
package com.azavea.stac4s

import cats.Eq
import cats.implicits._
import geotrellis.vector.{io => _}
import io.circe._
import io.circe.syntax._
import shapeless.LabelledGeneric
import shapeless.ops.record.Keys

final case class StacCollection(
stacVersion: String,
Expand All @@ -14,82 +18,105 @@ final case class StacCollection(
license: StacLicense,
providers: List[StacProvider],
extent: StacExtent,
summaries: JsonObject,
properties: JsonObject,
links: List[StacLink]
links: List[StacLink],
extensionFields: JsonObject = ().asJsonObject
)

object StacCollection {

private val generic = LabelledGeneric[StacCollection]
private val keys = Keys[generic.Repr].apply
val collectionFields = keys.toList.flatMap(field => substituteFieldName(field.name)).toSet

implicit val eqStacCollection: Eq[StacCollection] = Eq.fromUniversalEquals

implicit val encoderStacCollection: Encoder[StacCollection] =
Encoder.forProduct11(
"stac_version",
"stac_extensions",
"id",
"title",
"description",
"keywords",
"license",
"providers",
"extent",
"properties",
"links"
)(collection =>
(
collection.stacVersion,
collection.stacExtensions,
collection.id,
collection.title,
collection.description,
collection.keywords,
collection.license,
collection.providers,
collection.extent,
collection.properties,
collection.links
implicit val encoderStacCollection: Encoder[StacCollection] = new Encoder[StacCollection] {

def apply(collection: StacCollection): Json = {
val baseEncoder: Encoder[StacCollection] = Encoder.forProduct12(
"stac_version",
"stac_extensions",
"id",
"title",
"description",
"keywords",
"license",
"providers",
"extent",
"summaries",
"properties",
"links"
)(collection =>
(
collection.stacVersion,
collection.stacExtensions,
collection.id,
collection.title,
collection.description,
collection.keywords,
collection.license,
collection.providers,
collection.extent,
collection.summaries,
collection.properties,
collection.links
)
)
)

implicit val decoderStacCollection: Decoder[StacCollection] =
Decoder.forProduct11(
"stac_version",
"stac_extensions",
"id",
"title",
"description",
"keywords",
"license",
"providers",
"extent",
"properties",
"links"
)(
baseEncoder(collection).deepMerge(collection.extensionFields.asJson)
}
}

implicit val decoderStacCollection: Decoder[StacCollection] = { c: HCursor =>
(
c.get[String]("stac_version"),
c.get[Option[List[String]]]("stac_extensions"),
c.get[String]("id"),
c.get[Option[String]]("title"),
c.get[String]("description"),
c.get[Option[List[String]]]("keywords"),
c.get[StacLicense]("license"),
c.get[Option[List[StacProvider]]]("providers"),
c.get[StacExtent]("extent"),
c.get[Option[JsonObject]]("summaries"),
c.get[JsonObject]("properties"),
c.get[List[StacLink]]("links"),
c.value.as[JsonObject]
).mapN(
(
stacVersion: String,
stacExtensions: List[String],
stacExtensions: Option[List[String]],
id: String,
title: Option[String],
description: String,
keywords: Option[List[String]],
license: StacLicense,
providers: Option[List[StacProvider]],
extent: StacExtent,
properties: Option[JsonObject],
links: List[StacLink]
summaries: Option[JsonObject],
properties: JsonObject,
links: List[StacLink],
extensionFields: JsonObject
) =>
StacCollection(
stacVersion,
stacExtensions,
stacExtensions getOrElse Nil,
id,
title,
description,
keywords getOrElse List.empty,
license,
providers getOrElse List.empty,
extent,
properties getOrElse JsonObject.fromMap(Map.empty),
links
summaries getOrElse JsonObject.fromMap(Map.empty),
properties,
links,
extensionFields.filter({
case (k, _) => !collectionFields.contains(k)
})
)
)
}
}
Loading