Senior Engineer – Ingestion & Streaming Frameworks
Job Description:
- Design, build, and operate the ingestion frameworks that pull data from operational databases, vendor APIs, document streams, and third-party feeds into Snowflake, Iceberg, and Databricks
- Own and evolve the ingestion stack (AWS DMS, MWAA / Airflow, Fivetran) and design new patterns for API sources
- Build self-service tooling so product engineers can onboard new sources without becoming experts in our infrastructure
- Write and review the Terraform behind our ingestion infrastructure
- Partner with product, data, and analytics teams to pick the right ingestion pattern for each source and stand it up end-to-end
- Lead production troubleshooting and incident response, and turn each incident into a durable platform fix
- Raise the bar on engineering quality, observability, cost discipline, and security in everything the team ships
- Mentor mid-career engineers and pull peers along through code review, pairing, and design feedback
Requirements:
- 6+ years in data engineering, platform engineering, or data-focused software engineering
- 3+ years of hands-on AWS with real strength in networking (VPC, subnets, routing, PrivateLink, security groups), IAM (roles, policies, permission boundaries), and the data services this role touches
- 2+ years writing production Terraform or equivalent IaC
- 1+ years building self-service tooling, internal platforms, or paved-path frameworks
- Strong SQL skills and the ability to reason about how data physically lives in a warehouse or lake
- Production experience with Snowflake or an equivalent cloud data warehouse and a workflow orchestrator (Airflow / MWAA preferred)
- Hands-on experience with at least one ingestion approach: CDC tooling (DMS, Debezium), managed connectors (Fivetran, Airbyte), or rolling your own pipelines for API sources
- Solid CI/CD discipline in GitHub or equivalent
- AI-native working style: daily use of Claude Code, Cursor, Copilot, or equivalent
- Working knowledge of Python is expected; mastery isn’t the bar
- Clear written and verbal communication, especially in async, remote settings.
Benefits:
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Remote work options