Databricks Engineer Job at Conflux Systems, Phoenix, AZ

  • Conflux Systems
  • Phoenix, AZ

Job Description

Databricks Engineer

We are seeking a highly skilled Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, along with proven experience delivering end-to-end enterprise data engineering solutions.

Key Responsibilities

  • Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
  • Lead implementation of:
  • Cloud-based data lakehouse platforms integrating diverse data sources.
  • Real-time data processing pipelines for operational and analytical use cases.
  • Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
  • Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
  • Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
  • Establish data quality frameworks and governance standards.
  • Implement and manage Unity Catalog, including fine-grained security and access controls.
  • Leverage Databricks components such as:
  • Delta Live Tables
  • Autoloader
  • Structured Streaming
  • Databricks Workflows
  • Integration with orchestration tools (e.g., Apache Airflow)
  • Drive CI/CD automation, deployment strategies, and DevOps best practices.
  • Optimize performance of pipelines, Spark jobs, and compute resources.
  • Provide architectural guidance and technical leadership across cross-functional teams.
  • Engage with stakeholders and clients to translate business requirements into scalable technical solutions.

Deep expertise in:

  • Databricks and cloud-native storage/compute platforms
  • Apache Spark (batch & streaming)
  • Delta Lake & Lakehouse architecture
  • Distributed data processing systems
  • Strong hands-on programming skills in Python, PySpark, Scala, and SQL.

Job Tags

Similar Jobs

AKIRA

Tasker PT - Yorktown Job at AKIRA

 ...Job Description Job Description AKIRA - Tasker In 2002, AKIRA opened the doors to its first women's clothing boutique in Chicago. Since then, the company has extended its reach to 40+ stores across the United States, as well as a thriving eCommerce business (shopAKIRA... 

MaziCTools

Remote Data Entry Clerk - Part-Time, Flexible & Learnable Job at MaziCTools

 ...MaziCTools is seeking a Part-time Data Entry Clerk to work remotely from home, earning between $33 to $156 per day. This role is perfect for those looking for flexible work hours and requires no prior experience. Ideal candidates will possess strong attention to detail... 

Closet Factory

Cabinet Finisher/Painter Job at Closet Factory

 ...are a locally owned custom closet company that offers closets, garages, entertainment centers, office, libraries and more. Since we finish everything in our Factory, we give our clients beautifully finished options without the dust and smell of painting in their home.... 

Liberty Staffing LLC

Cabinet Finisher Job at Liberty Staffing LLC

 ...Job Description The Job Shop and Technical Staffing is looking to hire a Cabinet Finisher in the Corbin, KY area. Must have finishing experience. If you are interested please give us a call at (***) ***-**** to discuss more details! Click "Let's get started... 

Epic

Entry-Level Project Manager Job at Epic

 ...position is based on our campus in Madison, WI, and requires relocation to the area. We recruit nationally and provide financial relocation...  ...making with our full support and guidance. No software experience required. Learn more about our team at . Manage projects at...