Databricks API Developer

Parse and normalize JSON and XML/WSDL payloads into Delta tables using bronze, silver, and gold data structures in Unity Catalog. Design, build, and publish APIs using Databricks Apps, Model Serving endpoints, Azure Functions, or Azure API Management.

JOB TITLE

Databricks API Developer

LOCATION

Greenville, South Carolina, Hybrid

ROLE SUMMARY:

Build and integrate production APIs that expose curated Databricks data to internal teams and external partners. Own the reliability of these integrations through monitoring, alerting, audit tracking, documentation, and operational dashboards across the Databricks and Azure environments.

KEY RESPONSIBILITIES:

• Design, build, and publish APIs using Databricks Apps, Model Serving endpoints, Azure Functions, or Azure API Management.
• Develop production-grade Python notebooks and Databricks Jobs that consume REST, GraphQL, and SOAP APIs.
• Implement authentication, pagination, rate limiting, retries, timeouts, idempotency, and incremental data loads.
• Parse and normalize JSON and XML/WSDL payloads into Delta tables using bronze, silver, and gold data structures in Unity Catalog.
• Create API specifications, versioning standards, and documentation for API consumers.
• Implement structured logging, audit tables, health checks, SLAs, and alerting for integration monitoring.
• Build operational dashboards in Databricks SQL or Tableau to track failures, data latency, and integration health.
• Develop reusable Python modules, automated tests, CI/CD deployments, runbooks, and secure access controls.

KEY REQUIREMENTS:

• 3–5 years of hands-on data engineering or integration development experience, including at least 2 years working with Databricks.
• Strong Python skills, including experience with requests or httpx, SOAP libraries such as zeep, and a GraphQL client.
• Experience with asynchronous or concurrent API calls.
• Proven experience building and deploying production APIs consumed by other teams or external partners.
• Hands-on experience consuming REST, GraphQL, and SOAP APIs, including OAuth 2.0 flows and token refresh.
• Experience building monitoring and alerting for data pipelines or APIs, including alerts, dashboards, or runbooks.
• Solid PySpark and SQL skills, with working knowledge of Delta Lake and Unity Catalog.
• Experience with Azure Key Vault, Azure Functions, Azure API Management, Azure Storage, Azure Monitor, Git, CI/CD, and code reviews.

Apply if you have the Databricks, Python, API integration, and monitoring experience required to build reliable data services. This is a 3-month contract opportunity with a hybrid work arrangement in Greenville, South Carolina.