Hi
Hope you doing well.
Please review the job description and let me know if you are interested or available for this position and do share your updated resume to pratima.kumari@testingxperts.com
Job role : Lead Databricks Engineer
Location: Iselin NJ / New York, NY Hyrbid (2-3 Days)
Job Description:
Must have : Expert in Databricks , Pyspark , python
Strong knowledge on data bricks architecture and tools
Have experience of task and wf jobs creations in data bricks.
Deep understanding of distributed computing and how to use spark for data processing.
SQL and pyspark : strong command over querying databases and proficiency in pyspark.
Cloud platform: Preferred Azure for data bricks deployment.
Qualifications
Bachelor's degree in Computer Science, Engineering, or related field, or equivalent work experience
5+ years of experience in data engineering with Databricks and Spark
Proficient in SQL and Python and Pyspark
Experience with Azure Databricks Medallion Architecture with DLT, Iceberg
Financial/Corporate Banking context would be a plus
Experience with data integration and ETL tools, such as Azure Data Factory
Experience with Azure cloud platform and services
Experience with data warehouse and data lake concepts and architectures
Good to have experience with big data technologies, such as Kafka, Hadoop, Hive, etc
Strong analytical and problem-solving skills
Excellent communication and teamwork skills
Responsibilities
Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
Optimize data pipelines for performance, scalability, and reliability
Ensure data quality and integrity throughout the data lifecycle
Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
Kind Regards,
Pratima Kumari (Executive Recruiter)
Corporate Office: 650 Wilson Lane, Suite 201, Mechanicsburg, PA 17055
E: pratima.kumari@testingxperts.com
Skype: pratima.kumari.tx@outlook.com
W:
Please consider the planet before printing | Switch off as you go |Recycle always | Your child deserves a better world
CONFIDENTIALITY NOTICE : This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original messages. Whilst we run the anti-virus software, you are solely responsible for ensuring that any email or attachment you receive is virus-free. We disclaim liability for any damage you suffer as a consequence of receiving any such virus.
...Alacrity Solutions Independent Contractor Daily Property Field Adjuster About Alacrity Solutions Alacrity Solutions is a full end-to-end provider delivering streamlined insurance claims, repair, and recovery solutions. As one of the largest independent providers...
...individual looking to make a difference in the healthcare industry? We're seeking a motivated and... ...this role, you will assist with inbound calls, document conversations, and provide... ...systems, including desktop applications, call center phone systems, and the internet. Meet...
...Senior Associate Athletic Director Brand Engagement Job Identification: 17612 Posting Date: Job Schedule: Locations: Degree... ...anti-discrimination laws, including those regarding age, race, color, sex, national origin, military service, genetic information...
Senior Helpers is hiring a Weekend CNA (Saturday and Sunday 8a-7p) in Midtown Tulsa. We are looking for a highly motivated, engaging and proactive CNA with dementia experience and a heart for caring for others. Pay starts at $15/hour. Required Caregiver Qualifications...
...with exceptional customer service skills to join our Medical Call Center team for a temporary, 6-month role. If you are passionate about... ..., professional appearance to our external customers. Healthcare background (EMT, CNA, medical assistant, ETC.) preferred....