Job details
Python Data Engineer
Summary
Backend development in Python
Architecture design for Python applications
Development related to DataLake such as ETL and data pipeline
Perform business operations according to business demands
Requirements and needs are heard from business users, and then business analysis is performed to design the optimal architecture. System development according to the architecture requires the ability to develop actual applications.
Main Project:
Architecture improvement and function addition for data analytics systems
A global company's data analytics system imports data in various languages from each country and region every day and performs analysis processing using an AI engine. In order to further expand the data to be analyzed and improve processing speed, design and development such as pipeline expansion or parallel processing architecture improvement is required.
Architecture design improvement and function addition for ETL and data pipeline for DataLake
Design and implementation construction of ETL development, data quality monitoring, DataMart, and Catalog-related architecture for multiple global companies' data infrastructure
Personnel Requirements (Must Have)
- More than 3 years of Python development experience
- Ability to use SQL to extract necessary business data
- Ability to perform basic CLI operations such as ssh access and Linux commands
- Ability to proactively challenge new services and systems and gradually acquire knowledge and skills
- Ability to logically analyze problems and have the thinking skills to identify causes and come up with solutions
- Japanese level N2 or higher
Personnel Requirements (Nice to Have)
- Development experience using cloud environments is preferred.
- Development experience in DataLake, ETL, GlueJob, DataFlow, etc. is preferred.
- Agile development experience is preferred.
- Experience in Bigdata analysis and development is preferred.
- Development and design experience related to ElasticSearch is preferred.
Job description
Location:
5F Sanai Kayabacho Bldg., 2-13-11 NihombashiKayaba-cho, Chuo-ku, Tokyo, Japan 103-0025
Posted on:
2024年5月9日
Fee:
Determined individually based on professional ability and experience
Treatment:
We provide a welfare system that covers all transportation expenses and provides various social insurance benefits.
Holidays:
2 days off per week (Saturday and Sunday), public holidays, year-end and New Year holidays, maternity leave, childcare leave, etc.
Working style:
・Flextime system
・Discretionary work (work style in which the employee is free to decide how to accomplish their goals and how to allocate their time)
・Actively adopting telework (frequency of coming to the office varies depending on the team/project situation)