Data Engineer

Tehran

Posted 5 years ago

Job Description

•Main Objective:
We are looking for a Data Engineer to join our growing Data Infrastructure team

•Main Responsibilities:
-Responsible for expanding and optimizing our data ingestion pipeline architecture
-Support our data scientists and business intelligence on data initiatives and will ensure our optimal data infrastructure is consistent, stable, and robust throughout ongoing projects
-Create and maintain optimal data ingestion pipeline architectures
-Assemble large, complex data sets that meet functional / non-functional business requirements
-Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
-Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies
-Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
-Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs
-Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
-Work with data and analytics experts to strive for greater functionality in our data systems
-Lead innovation through exploration, benchmarking, making recommendations, and implementing big data technologies for platforms
-Development and implementation of scripts for database maintenance, monitoring, performance tuning
-Development of ETL routines in order to populate databases from sources and also to create aggregates
-Troubleshoots data issues within the business and across the business and presents solutions to these issues
-Perform data updates, indexing, and maintenance in application database

Requirements

-At least 1 year of experience in a Data Engineer role, who has Bachelor’s in Computer Science or another quantitative field. She/He should also have experience using the following software/tools: -Big data tools: Hadoop (YARN, HDFS), Spark, Kafka -Relational SQL and NoSQL databases, including Postgres and Cassandra -Data pipeline and workflow management tools: Airflow, Luigi -Stream-processing systems: Storm, Spark-Streaming -Object-oriented/object function scripting languages: Python -Experience building and optimizing data ingestion pipelines -Being comfortable with working on Linux operated machines -A successful history of manipulating, processing and extracting value from large disconnected datasets -She/He must be proactive and self-motivated supporting the data needs of multiple teams, systems and products -The right candidate will be excited by the prospect of optimizing or even re-designing TAP30’s data infrastructure architecture to support our next generation of products and data initiative -Strong project management and organizational skills -Experience supporting and working with cross-functional teams in a dynamic environment

Employment Type

  • Full Time

Details

To see more jobs that fit your career