Data Engineer (Clickhouse)

$55/hr C2C

Mandatory Primary Skills: Hadoop, Python, Databricks, Clickhouse DB , ADLS, SQL + Azure cloud technologies above all with Azure (such as ADF, Databricks, Log Analytics, Logic App, Key Vault, Blob Storage, ADLS, etc.)

Secondary Skills:; Snowflake ,

Detailed JD as below.

Data Engineer

We run a near real-time analytics platform for mobile network data, ingesting billions of rows a day from Azure Data Lake/HDFS into ClickHouse and serving subscriber-level queries back to internal APIs in tens of milliseconds. We are expanding the platform footprint to support additional use cases by adding more datasets

Data Engineer would work on: extending the Python ingestion framework that loads from Azure Data Lake/HDFS; table design covering partitioning, sort keys, TTL and materialized views; operational diagnosis of merge behaviour, part counts and replication lag; and integration with Snowflake and Databricks Iceberg alongside the primary feeds.

We need strong analytical SQL and experience with a columnar or MPP database - ClickHouse, BigQuery, Redshift or similar. Python sufficient to own a production ingestion framework rather than write scripts. Comfort at hundreds of billions of rows, since the failure modes differ from smaller systems. Cloud object storage and Parquet. Enough Kubernetes to tell whether a problem is the database or the platform underneath it.

Strongly preferred: direct ClickHouse experience, particularly MergeTree engines and aggregate function states, and telecom data such as DNS , IPFIX, etc. ClickHouse specifically is uncommon, and we would rather have a strong engineer from an adjacent system than a weak one with the exact keyword.

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.