Senior Data Engineer with PySpark

BCF.CS is a bright consulting company with ambitious projects. We implement projects that help millions of people in their daily lives.

Working with clients from more than 20 countries and more than 15 industries, we are open to strong developers from all over the world. At the same time, each team works with a common goal - to create a quality product

About position

We are thrilled to announce a fantastic job opportunity for a Senior Data Engineer with a strong background in Python, PySpark, and AWS. Our company is a dynamic and rapidly growing organization in the field of data-driven solutions, and we are seeking an experienced professional like you to join our exceptional team.

Requirements

Strong proficiency in Python programming and hands-on experience with PySpark for large-scale data processing and analytics
In-depth knowledge of AWS cloud services, including but not limited to S3, Glue, Redshift, Athena, EMR, and Lambda
Proficiency in SQL and experience with relational and NoSQL databases
Solid understanding of data modeling, schema design, and data warehousing concepts
Experience with version control systems (e.g., Git) and deployment tools (e.g., Jenkins, Airflow)
Familiarity with data visualization tools such as Tableau, Power BI, or similar platforms
Excellent problem-solving and analytical skills with the ability to troubleshoot and resolve complex data-related issues
Strong communication skills and the ability to work effectively in a collaborative team environment

Your responsibilities

Design, develop, and maintain scalable data pipelines and ETL workflows using Python, PySpark, and AWS services
Collaborate with cross-functional teams to understand data requirements and translate them into efficient data engineering solutions
Perform data cleansing, transformation, and aggregation to ensure data integrity and accuracy
Optimize and fine-tune data pipelines to improve performance and scalability
Implement and maintain data governance practices, including data quality monitoring, metadata management, and security measures
Stay up-to-date with emerging technologies and industry trends related to data engineering and cloud computing