ITTConnect is seeking a Lead Data Engineer for a direct-hire full time position with a client that is a large financial institution.
Position is hybrid in Miami.
The Lead Data Engineer turns business and domain needs into secure, observable, reusable, and documented pipelines. The role combines hands-on technical leadership, portfolio delivery, consultant coordination, engineering standards, and stakeholder engagement in a regulated bank.
Key Responsibilities:
Portfolio & Delivery Leadership
- Own integrated delivery for 4-6 concurrent projects, balancing priorities, dependencies, capacity, risks, and production commitments, working under a scaled agile framework.
- Turn transformation needs into technical scope, milestones, acceptance criteria, and measurable outcomes.
- Provide portfolio visibility through dashboards, risk logs, dependency tracking, and escalation.
- Ensure releases meet change, security, testing, documentation, and production-readiness requirements.
Team & Consultant Coordination
- Direct an 8-12 person consultant team through work allocation, onboarding, delivery follow-up, and quality review.
- Set an accountable rhythm with backlog refinement, design and code reviews, stand-ups, demos, and retrospectives.
- Build capability through reusable patterns, playbooks, pairing, coaching, and knowledge transfer.
- Partner with vendor and bank leaders to manage capacity, capability, performance, continuity, and key-person risks.
On-Premises Platform Execution
- Guide solutions using NiFi for ingestion, Airflow for orchestration, dbt for transformation and testing, and SQL for processing and validation.
- Improve pipeline stability through modular design, automated tests, version control, disciplined deployment, observability, and incident follow-up.
- Prioritize technical debt by business criticality, operational risk, migration value, and delivery impact.
Cloud Transition & Re-Engineering
- Follow the AWS and Databricks operating model in line with architecture, security, governance, networking, and compliance standards.
- Lead cloud-native pipeline delivery and phased re-engineering of on-premises pipelines.
- Redesign and remodeling instead of lifting and shifting when it improves scale, reliability, cost transparency, reuse, lineage, or supportability.
- Define migration waves, readiness, parallel runs, reconciliation, cutover, rollback, and decommissioning.
Stakeholder & Domain Partnership
- Align delivery with business and data owners, stewards, architects, security, infrastructure, risk, compliance, and operations.
- Lead cross-domain requirements and design discussions, resolving ambiguity and clarifying trade-offs.
- Explain decisions, delivery health, risks, and business impact to technical and executive audiences.
Requirements
- 15+ years of experience in IT.
- 8+ years in data engineering, platform delivery, analytics engineering, or a related field
- 3+ years in technical or delivery leadership.
Experience coordinating parallel projects and distributed or consulting teams in complex enterprises.
Strong SQL expertise and practical experience with NiFi, Airflow, dbt, or comparable tools.
Experience building production pipelines with automated tests, quality controls, monitoring, lineage, documentation, and support.
Working knowledge of AWS and Databricks architecture, lakehouse patterns, security, deployment, and migration.
Experience in regulated financial services or another control-intensive environment.
Strong English communication, planning, problem-solving, and stakeholder management;
Fluency in Portuguese or Spanish is a plus.
Preferred Qualifications
- Experience with Unity Catalog, Delta Lake, Databricks Workflows, CI/CD, infrastructure as code, and cloud observability.
- Knowledge of AWS data-platform services, including S3, IAM, KMS, CloudWatch, and private networking.
- Experience with catalogs, metadata, lineage, data quality, and DataOps or DataSecOps.
- Familiarity with dimensional modeling, data vault, medallion/lakehouse, streaming, and data products.
- Relevant AWS, Databricks, data engineering, or agile delivery certifications.