Book a call —>
Senior Data Engineer — San Francisco Bay Area, CA, US 9+ years · Streaming & AI platforms

Building data platforms that turn noise into signal — at streaming scale.

I'm Tanay Karmarkar — Senior Data Engineer at Meta (Menlo Park). I architect streaming data platforms and AI-native analytics that power trust & safety, ads, and real-time products for millions. Previously Staff Engineer at Delivery Hero, Data Platform lead at NewStore and Expedia.

Book a 30-min call View selected work
9+ Years in data engineering
$600K Annual infra savings delivered
45% Analytical inquiries automated

Tanay Karmarkar

Senior Data Engineer · Meta

[email protected] · San Francisco Bay Area, CA, US
01 — About

Data infrastructure is a product — reliability is the feature.

I build opinionated data platforms that let product and ML teams move fast without breaking trust. My sweet spot is the seam between streaming infrastructure, warehouse modeling, and AI workflows — where a well-designed pipeline saves millions and unlocks decisions that weren't possible before.

Over 9+ years I've led analytics and AI automation for Integrity at Meta, ran data for a multi-market Ads attribution engine across 7 brands / 60+ markets at Delivery Hero, and built the Data Mesh from zero to near-real-time at NewStore.

  • Streaming-first: Kafka / MSK / PubSub → Dataflow / Kinesis → BigQuery / Snowflake / Redshift
  • Cost-aware architecture — $600K/yr BigQuery savings, 19% Snowflake reduction
  • AI-native workflows: agent orchestration, evals, token & tool-call optimization
02 — Selected work LinkedIn —>
Meta · 2025 → Present

Semantic Agentic Platform — Integrity

Meta

Pioneered AI Transformation Strategy for Integrity. Built a semantic agentic platform connecting 3rd-party warehouses, CDPs, CRMs, lakes, APIs & MCP servers. Autonomous AI Analytics agents now resolve 45% of analytical inquiries without human intervention, with automated semantic model maintenance.

Agent OrchestrationMCPSemantic LayerEvalsClaude Code
+5% revenue — Meta Verified analytics workstream −20% opex, +10pp CSAT — AI-automated review −52% token usage, −60% wall-clock, −35% tool calls
Delivery Hero · 2023 — 2025

Global Ads Attribution & Streaming Platform

Staff Engineer

Led data for a multi-market attribution engine (CPC, CPM, Offers) across 7 global brands and 60+ markets with location-based sharding and KV stores. Engineered click / impression tracking for ranking & bidding models. Deployed streaming platform for 90+ engineers — real-time budget exhaustion alerts & live feature engineering.

GCPBigQueryKafkaDataflowGKE
$600K / yr BigQuery savings — tribe-wide optimization 0 → 1 streaming platform · 90+ engineers onboarded
NewStore · 2021 — 2023

Data Mesh on AWS MSK

Senior Data Engineer

Proposed and implemented Data Mesh using AWS MSK, Kafka Connect & Debezium. Cut time-to-market for new data products from 4 weeks to <3 days (−90%). Onboarded domains to BigQuery for near-real-time freshness (<5 min vs 24h batch). Self-service ingestion via Debezium + dbt reduced central bottlenecks by 75%.

AWS MSKKafka ConnectDebeziumBigQuerydbtLooker
−90% time-to-market 24h → <5min freshness
Expedia · 2020 — 2021

Conversation Platform — Real-time Quality Monitoring

Data Engineer II

Built real-time quality pipeline (Jitter, MOS, RTT) via Twilio + Looker contributing to +25% self-service rate. Engineered Python SDK for dashboarding — eliminated multi-table JOINs for Type II dimensions. Rightsized Snowflake warehouses.

PythonSnowflakeTwilioLookerAirflow
94s → 40s queries −19% Snowflake cost ($0.5M / yr)
03 — Technical skills

Languages

  • Python — Advanced
  • SQL — Expert
  • Java — Intermediate
  • Go — Beginner

Cloud & Infra

  • GCP: BigQuery, Dataflow, GCS, Pub/Sub, GKE
  • AWS: Redshift, MSK, S3, Lambda, ECS, Kinesis
  • Terraform, Kubernetes, Docker
  • CI/CD · Incident Commander experience

Data Engineering

  • Kafka, Debezium, Kafka Connect, KSQL, MSK
  • Airflow, dbt, Snowflake, BigQuery
  • Data Mesh architecture
  • Real-time streaming & orchestration tuning

AI & Agents

  • Agent orchestration & monitoring
  • Prompt scaffolding & evals
  • Claude Code · MCP servers
  • Token & tool-call optimization
04 — Experience
03/2025 — Present
Menlo Park, CA

Senior Data Engineer — Meta Platforms Inc

Integrity Organization

  • +5% revenue for Meta Verified — led analytics workstream for business & creator verification, reducing platform harm across Facebook & Instagram.
  • −20% opex & +10pp CSAT — architected data foundation for AI models automating human review & premium support.
  • Built semantic agentic platform bridging warehouses, CDPs, CRMs, lakes, APIs & MCP servers — 45% of inquiries now autonomous.
  • AI-native workflows: −52% token usage, −60% wall-clock time, −35% tool calls.
04/2023 — 02/2025
Berlin, Germany

Staff Engineer, Data (Ads) — Delivery Hero SE

Ads Platform · 7 brands · 60+ markets

  • $600K/yr BigQuery savings — tribe-wide optimization initiative.
  • Led attribution engine (CPC, CPM, Offers) with standardized benchmarking, location sharding & IP-range KV stores.
  • Built foundational click/impression tracking for ML ranking & bidding.
  • 0→1 streaming platform for 90+ engineers — real-time budget alerts & live feature engineering.
  • Mentored 5+ engineers to Senior IC; Incident Commander & CI/CD lead.
10/2021 — 03/2023
Berlin, Germany

Senior Data Engineer, Infrastructure — NewStore Inc

  • Data Mesh on AWS MSK + Kafka Connect/Debezium — 4 weeks → <3 days to ship data products.
  • Centralized BigQuery warehouse — 24h batch → <5 min near-real-time freshness.
  • Self-service ingestion (Debezium + dbt) — −75% central bottlenecks.
  • Built Data Platform team: hiring rubrics, roadmaps, vendor negotiations (Looker, Sisense).
03/2020 — 06/2021
Seattle, WA

Data Engineer II, Conversation Platform — Expedia Group

  • Python SDK for real-time dashboarding — eliminated Type II dimension JOINs; 94s → 40s queries.
  • −19% Snowflake cost ($0.5M/yr) via warehouse rightsizing.
  • Real-time quality monitoring (Jitter, MOS, RTT) with Twilio + Looker — +25% self-service rate.
02/2019 — 03/2020
Naperville, IL

Data Engineer II — Egen Solutions Inc

  • 100% HIPAA-compliant PII isolation & de-identification pipeline (Airflow) for oncology datasets.
  • Data Vault modeling for cancer tissue sample analysis.
05 — Education
12/2018

MS in Business Analytics & Information Systems

University of South Florida

05/2017

BE in Electrical Engineering

Shri G.S. Institute of Technology and Science

06 — Blogs & Writing View all —>
Aug 2026 · 9 min Distributed Systems Core Concepts

The Illusion of Time: Why Your Timestamps Lie

Physical clocks drift and NTP steps — Lamport and hybrid clocks restore causal order without wall time.

Read article → Tanay Karmarkar
Aug 2026 · 7 min PostgreSQL Database

Understanding PostgreSQL Transaction ID Wraparound

A practical guide to 32-bit XID limits, MVCC tuple freezing, autovacuum parameters, production monitoring queries, and emergency recovery.

Read article → Tanay Karmarkar
07 — Contact

Let's build data systems that actually get used.

Based in San Francisco Bay Area, CA, US. The fastest way to reach me is to book a call or send a message — I reply within 24 hours.

Email [email protected] LinkedIn linkedin.com/in/tanaykarmarkar ↗ Calendly Book a 30-min call ↗
Location San Francisco Bay Area, CA, US