Hi Friend
Hope you are doing well
Number of position : 6
Only Full Time
I, Salman Shaikh would like to share a job opportunity as Big Data (PySpark) Tech Lead based in Irving, TX / Jacksonville, FL / Jersey City, NJ (Onsite) location for a Fulltime position.
*** In case, if you are not comfortable with this location, please share your preference with contact details for further requirements ***
Kindly find the JD below and let me know if you are available for the same.
Big Data (PySpark) Tech Lead
Job Location:Irving, TX / Jacksonville, FL / Jersey City, NJ(Onsite)
Duration: Full time
Big Data (PySpark) Tech Lead–
• 10 + Years Overall Experience in Data Management, Data Lake and Data Warehouse
• 6+ Years Hadoop, Hive, Sqoop, SQL, Teradata
• 6+ Years PySpark(Python and Spark), Unix
• Good to have Industry leading ETL experience
• Banking Domain experience
Key Responsibilities
• Ability to design, build and unit test applications on Spark framework on Python.
• Build PySpark based applications for both batch and streaming requirements, which will require in-depth knowledge on majority of Hadoop and NoSQL databases as well.
• Develop and execute data pipeline testing processes and validate business rules and policies
• Optimize performance of the built Spark applications in Hadoop using configurations around Spark Context, Spark-SQL, Data Frame, and Pair RDD's.
• Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats (Avro, Parquet, ORC etc) and compression codec respectively.
• Ability to design & build real-time applications using Apache Kafka & Spark Streaming
• Build integrated solutions leveraging Unix shell scripting, RDBMS, Hive, HDFS File System, HDFS File Types, HDFS compression codec.
• Build data tokenization libraries and integrate with Hive & Spark for column-level obfuscation
• Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources.
• Create and maintain integration and regression testing framework on Jenkins integrated with BitBucket and/or GIT repositories
• Participate in the agile development process, and document and communicate issues and bugs relative to data standards in scrum meetings
• Work collaboratively with onsite and offshore team.
• Develop & review technical documentation for artifacts delivered.
• Ability to solve complex data-driven scenarios and triage towards defects and production issues
• Ability to learn-unlearn-relearn concepts with an open and analytical mindset
• Participate in code release and production deployment.
• Challenge and inspire team members to achieve business results in a fast paced and quickly changing environment
Please reply me with your updated resume and required details:
Full Name:
LinkedIn ID (Must To have as per exp)
Best number to reach you:
Work authorization/Visa Status:
Current Location:
Current Compensation:
Expected Compensation:
Best time to call you:
Waiting for your earliest response
Thanks & Regards
======
Salman Shaikh
+1 781-896-2152 (Cell)
Boston, MAS
Syntricate Technologies Inc