CAREER PORTAL

Sr Data Engineer

Job Responsibilities: 

Key Responsibilities:

• Design, develop, and maintain data pipelines using Python, PySpark, and Spark SQL to process and transform large-scale datasets.
• Implement Delta Lake architecture to ensure data reliability, consistency, and integrity for large, distributed datasets.
• Utilize serverless data infrastructure (e.g. Azure Functions, Databricks) to build scalable and cost-efficient data solutions.
• Collaborate with Data Scientists and Analysts by creating reusable Jupyter Notebooks for data exploration, analysis, and visualization.
• Optimize and manage data storage and retrieval processes, ensuring high performance and low latency.
• Implement best practices for data security, governance, and compliance within the data infrastructure.
• Work closely with cross-functional teams to understand data requirements and deliver solutions aligned with business objectives.
• Continuously monitor, troubleshoot, and improve the performance of data processing pipelines and infrastructure.
• Utilize Data Vault 2.0 methods for data transfer
• Looking for VERY senior resources, up to hands-on lead level
• Experienced with Assertion based Architecture
• Engineers vs “coders”
• Coding is done in Jupyter Notebooks on Delta Lakes
• Data Applications
• Code reviews
• Azure Synapse Analytics or Databricks.

Qualifications:

• 10-15+ years of experience in data engineering or related fields.
• Strong programming skills in Python with experience in data processing frameworks like PySpark.
• Extensive hands-on experience with Apache Spark and Spark SQL for processing and querying large datasets.
• Expertise with Delta Lakes for implementing scalable data lakehouse architectures.
• Experience with Jupyter Notebooks for prototyping and collaboration with data teams.
• Familiarity with serverless data technologies such as AWS Lambda, Azure Functions, or similar platforms.
• Proficient in working with cloud platforms such as Azure
• Data Vault 2.0 methodology experience
• Great Expectations data quality validation
• Automated Testing.
• Jupyter Notebooks
• Experience with data pipeline orchestration tools (e.g.,Synapse pipelines).
• Solid understanding of data warehousing, ETL/ELT pipelines, and modern data architectures.
• Strong problem-solving skills and ability to work in a collaborative environment.
• Experience with CI/CD pipelines and DevOps practices is a plus.
• Willing to do contract to hire, and no sponsorsip.

Preferred Qualifications:

• Experience with Databricks for data engineering workflows.
• Familiarity with modern data governance
• Understanding of P&C insurance data a big plus. 
• Nice to have - Knowledge of machine learning workflows and how data engineering supports AI/ML models.

Estimated Min Rate: $80.00
Estimated Max Rate: $90.00


What’s In It for You?
We welcome you to be a part of the largest and legendary global staffing companies to meet your career aspirations. Yoh’s network of client companies has been employing professionals like you for over 65 years in the U.S., UK and Canada. Join Yoh’s extensive talent community that will provide you with access to Yoh’s vast network of opportunities and gain access to this exclusive opportunity available to you. Benefit eligibility is in accordance with applicable laws and client requirements. Benefits include:

• Medical, Prescription, Dental & Vision Benefits (for employees working 20+ hours per week)
• Health Savings Account (HSA) (for employees working 20+ hours per week)
• Life & Disability Insurance (for employees working 20+ hours per week)
• MetLife Voluntary Benefits
• Employee Assistance Program (EAP)
• 401K Retirement Savings Plan
• Direct Deposit & weekly epayroll
• Referral Bonus Programs
• Certification and training opportunities

Note: Any pay ranges displayed are estimations. Actual pay is determined by an applicant's experience, technical expertise, and other qualifications as listed in the job description. All qualified applicants are welcome to apply.

Yoh, a Day & Zimmermann company, is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Visit https://www.yoh.com/applicants-with-disabilities to contact us if you are an individual with a disability and require accommodation in the application process.

For California applicants, qualified applicants with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. All of the material job duties described in this posting are job duties for which a criminal history may have a direct, adverse, and negative relationship potentially resulting in the withdrawal of a conditional offer of employment.