Data Architect - Palo Alto, CA (Hybrid)

Georgia Tek Systems

Palo Alto, CA

Apply

JOB DETAILS

SKILLS

Analysis Skills, Application Programming Interface (API), Best Practices, Business Analysis, Cloud Computing, Communication Skills, Data Lake, Data Management, Data Modeling, Data Processing, Data Quality, Data Science, Data Sets, Data Storage, DataArchitect Data Modeling Tool, Database Administration, Database Design, Database Extract Transform and Load (ETL), Distributed Computing, Healthcare, Microsoft Windows Azure, MongoDB, Multitasking, Operational Support, Performance Tuning/Optimization, PostgreSQL, Problem Solving Skills, Python Programming/Scripting Language, RabbitMQ, Redis, Team Player, United States Department of Energy (DOE)

LOCATION

Palo Alto, CA

POSTED

5 days ago

Data Architect
Location: Palo Alto, CA (Hybrid)
Duration: Long Term
Rate: DOE

Key Responsibilities:

Data Orchestration:

Design, implement, and manage data workflows using Airflow to automate and orchestrate data processing tasks.
Optimize Airflow DAGs (Directed Acyclic Graphs) for performance and scalability.

Task Management:

Develop and maintain distributed task processing using Celery and ensure robust task queue management with Redis or RabbitMQ.

Database Management:

Design and manage databases using Cosmos DB, MongoDB, and PostgreSQL.
Develop and maintain efficient data models and ensure data consistency and integrity.

API and Webhooks:

Implement and manage FastAPI webhooks to handle data ingestion and integration tasks.
Develop and maintain Azure Functions to support webhook operations and integrate with cloud services.

Streaming Data:

Implement and manage Kafka Streams to handle real-time data processing and streaming requirements.

Data Lake Management:

Work with Iceberg to manage and optimize large-scale data lake storage and querying.

Collaboration and Communication:

Collaborate with data scientists, engineers, and business analysts to understand data requirements and provide technical solutions.
Document processes, architectures, and configurations to ensure knowledge sharing and compliance with best practices.

Required Skills and Qualifications:

Experience and Knowledge:

Proven experience with Airflow for data orchestration and workflow management.
Hands-on experience with Celery for task management and Redis or RabbitMQ for messaging.
Proficiency with Cosmos DB, MongoDB, and PostgreSQL for data storage and management.
Experience developing and managing webhooks using FastAPI and integrating with Azure Functions.
Knowledge of Kafka Streams for real-time data processing.
Familiarity with Iceberg for data lake management and optimization.
Healthcare domain experience is good to have .

Technical Skills:

Strong understanding of data pipelines, ETL processes, and data integration.
Proficient in Python, with experience in building and maintaining data-oriented applications.
Ability to work with large datasets and optimize performance across distributed systems

Soft Skills:

Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.
Ability to work independently and manage multiple priorities in a fast-paced environment.

Data Architect - Palo Alto, CA (Hybrid)

Georgia Tek Systems

Palo Alto, CA

About the Company

Georgia Tek Systems

Similar Job Searches