I'm administrating a large number of PostgreSQL's servers and I get their logs zipped. To analyze them I've done a Spark task for:
- Unzip the files
- Parse then logs of PostgreSQL
- Save (append) the data into a parquet file
In a following post I will show how to query them to get usefull . . .
October 25, 2018