Data Engineer

Job Title: Data Engineer/ Python Developer

Experience: Minimum 9 years

Employment: Fulltime, 5 days onsite

Mandatory Skills: Python, Spark, Pyspark

Responsibilities

• Design, develop, and maintain software solutions using Python.
• Develop Python and pyspark programs for data analysis. Good working experience with Python to develop a custom framework for generating rules (similar to a rules engine).
• Develop Python code to gather data from HBase and design a solution to implement using Pyspark. Apache Spark DataFrames/RDDs were used to apply business transformations and utilize Hive Context objects to perform read/write operations.
• Develop back‑end services, APIs, and integrate databases.
• Optimize applications for performance, security, and maintainability.
• Troubleshoot and resolve software defects and issues.