| Location | New York |
We seek a Junior Data Engineer with a passion for developing highly scalable data processing pipelines. The engineer will collaborate with a data team to add new and diverse data sources to our warehouse, continuously enhance our pipeline to ensure high availability of data to our customers, and ensure the integrity of the data delivered to our customers.
Responsibilities
Develop, construct, and maintain scalable automated data pipelines.
Construct cloud data solutions and provide domain-specific insight regarding storage, big data platform services, serverless architecture, RDBMS, DW/DM, NoSQL databases, and security.
Deploy, automate, maintain, and manage a cloud-based production system in order to assure the system's availability, performance, scalability, and security.
Implement data management best practices, including master data, reference data, metadata, data quality, and data lineage.
To comprehend customer behavior, support regular and ad hoc data querying and analysis.
Qualifications
BA or BS in Computer Science, Information Systems, or a related discipline.
Experience constructing and deploying ETL pipelines utilizing Apache Spark and/or Databricks.
Experience with either SQL or Spark SQL.
Knowledge of Informatica is an advantage.
Dimensional model expertise in data warehouses.
Experience working with Star Schema data importing.
Favorable Qualification
Expertise and familiarity with automated testing.
Knowledge and experience with Spark Notebooks deployment.
Clearance Requirement:
Citizen of the United States with the ability to obtain and maintain US Suitability.
Clearance Requirement:
Citizen of the United States with the ability to obtain and maintain US Suitability.unrivaled metal buildings is an equal opportunity employer who recognizes and appreciates the value diversity brings to the workplace. All qualified applicants will be considered for employment regardless of race, color, religion, gender, national origin, disability, veteran status, or any other legally protected characteristic.