Databricks Data Engineer
Company: Westberke Consulting
Location: Chicago
Posted on: February 21, 2026
|
|
|
Job Description:
Job Description Job Description Databricks Data Engineer
Location: Chicago, IL (Hybrid/Remote options as applicable)
Company: Westberke Consulting Employment Type: Contractor About
Westberke Consulting At Westberke Consulting, we bridge the gap
between complex data challenges and actionable business
intelligence. Based in Elmhurst, IL, our team prides itself on
delivering high-impact, scalable solutions for clients ranging from
mid-market leaders to enterprise giants. We are looking for a
Databricks Data Engineer who isn't just a coder, but a problem
solver ready to shape the future of modern Lakehouse architectures.
Position Summary We are seeking a highly skilled Databricks Data
Engineer to design, build, and optimize modern lakehouse
architectures. This role focuses on developing scalable data
pipelines, implementing Delta Lake best practices, and enabling
advanced analytics and AI workloads. The ideal candidate has
hands-on experience building production-grade solutions, strong
expertise in Spark-based processing , and a deep understanding of
performance optimization, data modeling, and data governance. Key
Responsibilities Development: Design and implement scalable ELT
pipelines using PySpark, SQL, and Databricks Workflows.
Architecture Implementation: Deploy and maintain the Medallion
Architecture (Bronze, Silver, Gold layers) to ensure data quality
and lineage. Performance Engineering: Optimize jobs for
cost-efficiency through strategic cluster sizing, Z-Ordering, and
partitioning. Governance & Security: Manage data access controls,
metadata, and security policies using Unity Catalog . Storage
Management: Manage Delta Lake tables, including schema evolution,
vacuuming, and time-travel features. Collaboration: Work alongside
Data Architects and ML Engineers to prepare feature-ready datasets
for AI initiatives. DevOps Integration: Participate in CI/CD
lifecycles, utilizing Git-based version control and automated
release patterns. Required Qualifications Education: Bachelor’s
degree in Computer Science, Information Systems, or a related
field. Experience: 3 years of hands-on Databricks experience in a
production environment. Technical Core: * Strong proficiency in
PySpark and Spark SQL . Deep understanding of Delta Lake and
distributed data processing. Hands-on experience with Unity Catalog
. Cloud Proficiency: Experience with Azure Data Lake Storage (ADLS)
or AWS S3. Soft Skills: Strong communication skills (essential for
consulting) and experience in Agile/SCRUM environments. Preferred
"Bonus" Qualifications Advanced Ingestion: Experience with
Databricks Auto Loader and Structured Streaming (Kafka/Event Hubs).
AI/ML Ops: Familiarity with MLflow , Feature Stores, or Mosaic AI.
Certifications: Databricks Certified Data Engineer (Associate or
Professional). Consulting Mindset: Ability to translate technical
bottlenecks into business solutions for diverse clients. Why
Westberke Consulting? Impact: Work on diverse projects across
multiple industries. Growth: We invest in our people—access to the
latest Databricks certifications and training. Culture: A
collaborative, Elmhurst-based team that values technical excellence
and work-life balance. Innovation: Be at the forefront of the "Data
Intelligence" era with Generative AI and Lakehouse evolution.
Keywords: Westberke Consulting, Elmhurst , Databricks Data Engineer, IT / Software / Systems , Chicago, Illinois
