Senior Data Engineer - Core Data Platform

Location: Chicago, IL or New York, NY (Hybrid)

About the Opportunity

Are you ready to architect high-throughput data infrastructure for a premier global quantitative trading and investment organization? We are seeking a Senior Data Engineer to lead the evolution of our central data ecosystem, transforming complex external market feeds into high-value analytical assets. In this high-impact role, you will partner directly with quantitative researchers, trading desks, and technical leaders to build resilient, scalable data solutions.

Responsibilities

• Architect, construct, and maintain scalable data pipelines and centralized platform architecture supporting enterprise-wide analytics and quantitative trading workflows.

• Evaluate, ingest, and operationalize external vendor data feeds in close collaboration with domain experts and quantitative analysts.

• Implement robust data governance frameworks, including access control, lineage, schema management, usage tracking, and automated quality metrics.

• Oversee operational stability of ingestion pipelines, managing schema updates, data backfills, dead-letter queue resolution, and rotating on-call support.

• Build automated monitoring and validation tools to preemptively detect and resolve data anomalies across batch and streaming pipelines.

• Drive continuous improvement of internal tooling and self-service capabilities to accelerate data discovery and integration for end users.

Requirements (Must-Have)

• 7+ years of dedicated experience building enterprise-grade data platforms and pipeline infrastructure.

• Expert-level mastery of SQL along with production-grade proficiency in either Python or Java.

• Demonstrated expertise in modern data modeling paradigms and architecture trade-offs.

• Hands-on experience orchestrating complex workflows using systems such as Airflow, Dagster, Prefect, or Kubernetes/Argo Workflows.

• Proven experience with modern storage systems and file formats (e.g., Delta Lake, Iceberg, Parquet, S3, ClickHouse, Cassandra, or relational databases).

• Track record of executing non-trivial data platform migrations and managing operational recovery processes (batch reprocessing, backfills, queue monitoring).

• Strong communication skills and the ability to thrive in a collaborative, fast-paced trading/financial environment.

Preferred Qualifications (Nice-to-Have)

• Familiarity with distributed compute and streaming frameworks (e.g., Flink, Spark, Polars, Dask, Kafka).

• Experience handling binary serialization standards such as Avro, Arrow, Protobuf/gRPC, or Thrift.

• Background in financial markets, market data ingestion, or high-throughput analytics platforms.

Compensation & Benefits

• Base Salary Range:$200,000 – $250,000 per year (commensurate with experience and skill set).

• Bonus: Eligible for an annual discretionary performance bonus.

• Benefits: Comprehensive package including medical, dental, and vision insurance; matching 401(k); short/long-term disability; life and AD&D coverage; health savings accounts (HSA); and flexible spending accounts (FSA).