Data Engineer - Warsaw
Our Global Financial Client is looking for a skilled Data Engineer to join their WBAA Data driven team. The project aims to combine Big Data technology with Data Science to deliver high value solutions and products for the organisation. The client executes a fun and creative environment and supports their staffs future prospects completely. This role has a great balance with 50% data engineering and 50% developing data pipelines!
Do you have strong data transformation capabilities? Great programming skills (preferably Scala or Java) Have hands-on experience managing and developing distributed systems and streaming data (S3/Spark and/or Kafka/Flink)? Building complex data pipelines? Then you're exactly what this team is looking for!
The technology Stack you will work with and have exposure to-
* Docker / Kubernetes / Openshift Processing
* Apache Spark
* Apache Flink
* Apache Airflow Storage: Ceph (S3)
* Apache Kafka
* Apache Cassandra (and other noSQL DBs)
* Apache Druid
* Apache Avro
* Apache Parquet Programming: Scala/Java, Python, ReactJS
* Data Security and Lineage: Apache Atlas and Apache Ranger
Not essential but desired candidate experience -
* Interpreting python or additional programming skills (Ruby, Groovy or C++).
* Setting up SQL as well as noSQL databases.
* Automation experience with Docker, Kubernetes, Openshift, CI/CD,
* A good knowledge of security, authentication and authorisation (LDAP / Kerberos / PAM).
If you would like more information or to be considered for this project, please email your CV to firstname.lastname@example.org.