Post a new job
This job ad has been posted over 40 days ago...
0

applicants

Full-time Sr. Data Engineer

at fivjob in Victoria


Job Title: Sr. Data Engineer
Location: Remote


Duties & Responsibilities
? At least 4 years of Data Engineering experience
? Export the data from the Hadoop ecosystem to ORC or Parquet file
? Build scripts to move data from on-prem to GCP
? Build Python/PySpark pipeline
? Transform the data as per the data model
? Improve the pipeline performance and efficiency

Mandatory skills:
? Hands-on experience building Python/PySpark pipeline
? Strong Hadoop/Hive background
? Strong in SQL
? Data warehousing & Data Lake
? Understanding of Data model
? Google experience ? Cloud Storage, Cloud Composer, Dataproc & BigQuery
? Understanding data files format like ORC, Parquet, Avro

Preferred skills:
? Understanding of GCP services
? Experience using Cloud Warehouses like BigQuery (preferred), Amazon
Redshift, Snowflake
etc.
? Working knowledge of Distributed file systems like GCS, S3, HDFS etc.
? Understanding of Airflow / Cloud Composer
? CI/CD and DevOps
? ETL tools e.g., Informatica (IICS) Ab Initio, Infoworks, Pentaho, SSIS





Reference : Sr. Data Engineer jobs


Recent jobs at fivjob
Full-time Product Owner at fivjob in New South Wales 06-04-2024
Full-time Automation Tester Banking Domain at fivjob in New South Wales 04-12-2023
Full-time Devops Developer at fivjob in New South Wales 04-12-2023
Full-time java developer at fivjob in New South Wales 11-09-2023
Full-time Kyriba Project Manager at fivjob in New South Wales 31-08-2023

« Go back to category
Is this job ad fake? Report it!   
Recommend to a friend
Published at 13-12-2022
Viewed: 102 times