DataBricks Specialist

Tata Consultancy Services Ltd

  • New York, NY
  • 5 days ago
  • $130,000–$150,000 Per Year

Highlights

Architecture & Development: Design end-to-end data pipelines, ETL/ELT processes, and lakehouse storage patterns using Apache Spark, Delta Lake, and Python or SQL. AI & Machine Learning Integration: Implement MLOps workflows, manage model lifecycles, or deploy generative AI and vector search solutions using tools like MLflow and Unity Catalog.

Numbers & Facts

LocationNew York, NY
Salary$130,000–$150,000 Per Year

Description

Must Have Technical/Functional Skills

  • Experience: 5 to 10+ years in data engineering, software development, or solution architecture, with hands-on Databricks production experience.
  • Technical Stack: Strong proficiency in Python, SQL, or Scala; deep knowledge of Apache Spark runtime internals and Delta Lake.
  • Cloud Ecosystems: Working knowledge of major cloud platforms like Microsoft Azure, AWS, or GCP.
  • Certifications: Preferred credentials include Databricks Certified Data Engineer or Solutions Architect credentials. []

Roles & Responsibilities

  • Architecture & Development: Design end-to-end data pipelines, ETL/ELT processes, and lakehouse storage patterns using Apache Spark, Delta Lake, and Python or SQL.
  • Performance Tuning: Optimize cluster sizing, query execution, shuffles, partitions, and storage layouts for large-scale data workflows. [1]
  • AI & Machine Learning Integration: Implement MLOps workflows, manage model lifecycles, or deploy generative AI and vector search solutions using tools like MLflow and Unity Catalog. [1]
  • Collaboration & Consulting: Partner with enterprise stakeholders, data scientists, and software engineers to translate business requirements into technical solutions. [1]
  • Administration & Governance: Oversee security configurations, access controls, workspace administration, and CI/CD deployment pipelines

TCS Employee Benefits Summary:

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range: $130,000 - $150,000 a year

Similar Jobs