Parse zipped PostgreSQL's logs and save them in a parquet file

I'm administrating a large number of PostgreSQL's servers and I get their logs zipped. To analyze them I've done a Spark task for:

  1. Unzip the files
  2. Parse then logs of PostgreSQL
  3. Save (append) the data into a parquet file

In a following post I will show how to query them to get usefull . . .

Read More

October 25, 2018