Role description
Job Title: Pyspark Developer
Work Location : Tampa, Florida
Job Title: Pyspark Developer
Work Location : Tampa, Florida
- Design develop and maintain data pipelines using PySpark for largescale data processing
- Perform data transformations aggregations and validations using Spark SQL Data Frames
- Write and optimize SQL queries for data extraction reconciliation and reporting
- Perform data quality checks profiling and consistency validations
- Support existing data pipelines by troubleshooting failures and performing root cause analysis
- Work closely with data engineers analysts QA and DevOps teams in Agile delivery models
- Optimize performance of Spark jobs for efficiency and scalability
- Maintain technical documentation workflows and runbooks
- Follow enterprise standards for coding deployment and change management 712 years of experience in Big Data Data Engineering roles
- Strong handson experience with PySpark Apache Spark
- Solid understanding of SQL and relational database concepts
- Experience working in LinuxUNIX environments
- Understanding of ETL concepts and largescale data processing
- Strong analytical and debugging skills
- Experience supporting production data pipelines Exposure to Azure Databricks ADF Synapse or similar cloud data platforms
- Experience with Hadoop ecosystem Hive HDFS
- Knowledge of orchestration tools eg Airflow
- Python scripting beyond Spark use cases
- Familiarity with Agile DevOps delivery practices
- Domain exposure eg Banking BFSI Analytics is an advantage Strong problemsolving and ownership mindset
- Ability to work collaboratively in crossfunctional teams
- Good communication and documentation skills
- Ability to adapt quickly in dynamic project environments