[Remote] Data Engineer
Note: The job is a remote job and is open to candidates in USA. Brooksource is seeking a hands-on, mid-level Data Engineer to support a security-focused data lake initiative. The role involves building reliable API ingestion pipelines, creating structured SQL datasets, and preparing data for downstream analytics and future AI use cases.
Responsibilities
- Build and own API-based ingestion pipelines pulling data from third-party systems
- Handle authentication, pagination, retries, failures, and schema changes
- Ingest, transform, and structure raw data into SQL-based datasets
- Validate incoming data for accuracy, completeness, and consistency
- Create and maintain clear documentation for datasets, fields, and ingestion logic
- Manage and support datasets across multiple data sources (approx. 14 total)
- Prepare clean, well-structured data for downstream analytics and AI consumption
- Operate independently and take ownership of deliverables with minimal hand-holding
Skills
- 2+ years of hands-on experience writing and maintaining API ingestion pipelines in a big data environment, managing datasets across 14 known data sources and supporting datasets with 10–15K+ fields
- 3–6 years of hands-on experience as a Data Engineer or similar role
- Strong experience building API-based ingestion pipelines
- Proficiency with SQL / SQL Server and relational data modeling
- Experience working in AWS (S3, Lambda, EC2, RDS, or similar)
- Ability to validate and QA large, complex datasets
- Comfortable working with large field counts and evolving schemas
- Strong documentation and communication skills
- Exposure to security, device, endpoint, or asset data
- Experience with tools such as Qualys, CrowdStrike, Intune, JAMF, Wiz, or SCCM
- Familiarity with data lake concepts (raw vs curated layers)
- Experience supporting analytics or AI teams (data prep only, not model building)
Benefits
- Brooksource offers competitive medical, dental, vision, Health Savings Account, Dependent Care FSA, and supplemental coverage with plans that can fit each employee’s needs.
- We offer a 401k plan that includes a company match and is fully vested after you become eligible
- Paid time off, and sick time.
- Paid company holidays are subject to hour requirements and completion of the new hire period (13 weeks).
- We also offer an Employee Assistance Program (EAP) that provides services like virtual counseling, financial services, legal services, life coaching, etc.
Company Overview