Data QA Engineer (NJ)

Builds a comprehensive QA framework in Azure DataBricks that gather, store, detect and process large volumes of data for QA analysis and detection of bugs found in ETL code. This role is a necessary component and part of the overall SDLC effort to centralize Information Security data and test the accuracy of storing and transforming the ETL process through the medallion architecture.

Builds a comprehensive QA framework in Azure DataBricks that gather, store, detect and process large volumes of data for QA analysis and detection of bugs found in ETL code.

RESPONSIBILITIES

• Develop QA ETL pipelines, ensure data quality, optimize data storage solutions.
• Python, SQL, cloud data tools experience (e.G., AWS Glue, Azure Data Factory, Azure DataBricks).
• Work with the QA analysts to test code and data as part of the SDLC process.
• This role is a necessary component and part of the overall SDLC effort to centralize Information Security data and test the accuracy of storing and transforming the ETL process through the medallion architecture.
• To achieve success, this individual will be required to analyze and develop the ingestion pipeline that will be used to test the data feeds from raw, transformation and to the target process.