scala - Creating Encoder for case class containing custom object -
i have case class like:
case class dlt (date: java.time.localdate, loc: location, temp: double)
where location case class like:
case class location (lat : double, lon : double)
how can create custom encoder case class dlt? when have directly tried encoders.kryo[ldt]
output coming single column unrelated values rather in separate columns. map function convert dataframe dataset follows:
ldt.map(ldtrow => ldt(localdate.of(ldtrow.getas[int]("year"), ldtrow.getas[int]("month"), ldtrow.getas[int]("day") ), location(ldtrow.getas[double]("lat"), ldtrow.getas[double]("lon") ), ldtrow.getas[double]("temp")))(ldtencoder)
this printed on show() on dataframe:
|year|month|day| lat| lon|temp| +----+-----+---+------+------+----+ |1960| 1| 1|70.933|-8.667|23.2| |1960| 1| 2|70.933|-8.667|18.7|
and, printed on show() on dataset:
| value| +--------------------+ |[01 00 6f 62 73 6...| |[01 00 6f 62 73 6...| |[01 00 6f 62 73 6...| |[01 00 6f 62 73 6...| |[01 00 6f 62 73 6...|
Comments
Post a Comment