[Remote] Data Engineer (Databricks + Informatica + Azure)
Note: The job is a remote job and is open to candidates in USA. Allata is a global consulting and technology services firm that helps organizations accelerate growth and solve complex challenges. They are seeking a skilled Data Engineer to design, build, and optimize scalable data solutions for analytics and reporting in the healthcare industry, collaborating with data architects and business stakeholders.
Responsibilities
- Design, develop, and maintain scalable data pipelines using modern distributed data processing platforms and cloud environments
- Build and optimize ETL/ELT processes following industry best practices and cloud-native architectures
- Implement data models aligned with modern Data Lakehouse principles and data architecture frameworks
- Ensure data quality, consistency, and performance across ingestion, staging, and curated data layers
- Collaborate with data architects, analysts, and business stakeholders to understand complex healthcare data requirements
- Develop reusable data transformation logic and modular processing components for efficient, maintainable systems
- Support deployment processes following CI/CD and DevOps best practices
- Monitor and optimize data workflows for performance, scalability, and reliability in production environments
- Contribute to data governance, security, and compliance practices relevant to regulated healthcare environments
Skills
- Proven hands-on experience with Informatica as a data integration and ETL platform
- Strong experience with Databricks or similar distributed data processing platforms
- Core expertise in data architecture, data integrations, data warehousing, and ETL/ELT process design
- Applied experience developing and deploying custom scripts and modules for distributed computing environments (custom code execution across parallel executors and worker nodes)
- Strong proficiency in SQL, Python, and PySpark (or equivalent distributed processing languages) for data transformation and processing
- Solid knowledge of cloud and hybrid relational database systems such as MS SQL Server, PostgreSQL, Oracle, Azure SQL, AWS RDS, or comparable engines
- Hands-on experience with batch and streaming data processing techniques and data compaction strategies
- Strong analytical and problem-solving skills
- Ability to work effectively in cross-functional and distributed teams
- Clear communication skills, with the ability to explain technical concepts to non-technical stakeholders
- Proactive mindset with a strong sense of ownership
- Commitment to delivering high-quality, reliable data solutions
Company Overview
Company H1B Sponsorship