Data (ETL) Engineer

20 Ноября

Партнерские Вакансии

Компания "Syberry CIS"

Syberry is a software service company with ~150 employees in Belarus, Poland, and the US. Our clients are American startups and businesses.

We strive to help our clients. To do that, we continuously improve our service by automating our processes. Our goal is to provide such quality service that it becomes a bеnсhmаrk of quality for other companies. We aim to speed up the development of the IT industry.

Join us as a Data Engineer to accelerate the IT industry together.

Project

Our company develops software solutions that help businesses work with large-scale data efficiently. For example, imagine a leading fashion brand wanting to analyze competitor sales data. Our client gathers raw data from various sources, but this data needs to be cleaned, standardized, and formatted to make it useful.

In this project, you’ll join a team of Data Engineers responsible for processing this raw data. Your role will involve transforming it into a structured, reliable format while ensuring high quality and accuracy, so it’s ready for business use.

Stack: Python SDK, Apache Airflow, YAML, SQL, BigQuery, Snowflake, Databricks.

What You’ll Do as a Data Engineer:

• Write Python scripts to extract and process raw data.

• Analyze and document input data, designing workflows to standardize and improve it.

• Create and debug YAML configurations for data processing pipelines.

• Deploy workflows to Apache Airflow, addressing feedback from QA and resolving bugs.

• Collaborate with clients via Jira and Slack to clarify requirements and share updates.

• Participate in team calls to align on status, provide feedback, and coordinate with client-side automation tool developers.

Requirements:

• At least a B1 level of English;

• Solid Python knowledge (OOP), including such libraries as pandas, requests, bs4, and openpyxl;

• Understanding how Apache Airflow and its components (DAG) work, how scheduling jobs is performed there (execution/logic date, cron);

• Familiarity with YAML/JSON, regular expressions;

• Strong SQL knowledge and hands-on experience with such data platforms as BigQuery, Databricks, and Snowflake.

• Strong grasp of Bash and commonly used commands; hands-on experience with SSH tunneling and REST API requests; Docker and the other tools to create isolated virtual environments (poetry, pipenv, pyenv); Git

Will be a plus:

• You worked with Kubernetes, other GCP services (Cloud Storage, Compute Engine).