i created an external table to fetch Parquet data from ADLS which throws below error.
Query execution has resulted in error (0x80131500): Partial query failure: 0x80131500 (message: 'Input parquet file is ill-formed and cannot be processed: 'not a Parquet file (size too small)'.: ', details: 'Source: Kusto.Common.Svc [0]Kusto.Common.Svc.Exceptions.IngestionSourceParquetReaderException: Input parquet file is ill-formed and cannot be processed: 'not a Parquet file (size too small)'. Timestamp=2020-05-07T11:22:42.0340199Z
Folder structure at ADLS:- logs / {AppId}/ 2020 / 05 / 07
External Table definition :
.create external table ExTParquet (AppId:string,UserId:string,Email:string,RoleName:string,Operation:string,EntityId:string,EntityType:string,EntityName:string,TargetTitle:string,Params:string,EventProcessedUtcTime:datetime,PartitionId:string,EventEnqueuedUtcTime:datetime)
kind=blob
partition by
AppId,
bin(EventProcessedUtcTime,1d)
dataformat=parquet
(
h@'https://streamoutalds2.blob.core.windows.net/stream-api-raw-parquet/logs;secret_key'
)
with
(
folder = "ExternalTables"
)
Note : if i provide full file path and remove source directory partitioning from External Table definition , it works well. But i need to ready data for all the files within directory and not just one.
Any help is much Appreciated.
This is a known issue being worked out. You can open a support ticket (Azure Data Explorer). The team will also post here an update when issue is resolved.
[EDIT] the issue should have been resolved.
User contributions licensed under CC BY-SA 3.0