Closed
Description
`map_values` will be addressed in following PR. It got a correctness issue
- read map[struct, struct] from parquet *** FAILED *** (1 second, 478 milliseconds)
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
Project [map_values(map1#508970).id2 AS map_values(map1).id2#509026]
+- SubqueryAlias tbl
+- View (`tbl`, [map1#508970])
+- Relation [map1#508970] parquet
== Analyzed Logical Plan ==
map_values(map1).id2: array<bigint>
Project [map_values(map1#508970).id2 AS map_values(map1).id2#509026]
+- SubqueryAlias tbl
+- View (`tbl`, [map1#508970])
+- Relation [map1#508970] parquet
== Optimized Logical Plan ==
Project [map_values(map1#508970).id2 AS map_values(map1).id2#509026]
+- Relation [map1#508970] parquet
== Physical Plan ==
*(1) CometColumnarToRow
+- CometProject [map_values(map1).id2#509026], [map_values(map1#508970).id2 AS map_values(map1).id2#509026]
+- CometScan parquet [map1#508970] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/__w/datafusion-comet/datafusion-comet/spark/target/tmp/spark-062..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<map1:map<struct<id:bigint,id2:bigint,id3:bigint>,struct<id2:bigint>>>
== Results ==
== Results ==
!== Correct Answer - 5 == == Spark Answer - 5 ==
struct<map_values(map1).id2:array<bigint>> struct<map_values(map1).id2:array<bigint>>
![ArrayBuffer(3)] [ArrayBuffer(0)]
![ArrayBuffer(4)] [ArrayBuffer(3)]
![ArrayBuffer(null)] [ArrayBuffer(4)]
[null] [null]
[null] [null] (QueryTest.scala:244)
Originally posted by @comphead in #1788 (comment)
Metadata
Metadata
Assignees
Labels
No labels