Platform Engineer / Platform Engineeress

About Mercuria

Established in 2004, Mercuria is one of the leading integrated energy and commodity trading companies in the world. We connect energy markets to meet today's needs through trading, structuring finance, and investing in strategic assets, generating over $110 billion in turnover. Our operations span more than 50 countries across five continents, covering all major energy hubs. We trade physical oil, energy products, environmental products, and other commodities from key locations including Geneva, London, Singapore, Shanghai, Beijing, Dubai, Houston, Calgary, and Greenwich, CT.

Our commitment to sustainability is at the forefront of our mission. With over 50% of our assets dedicated to low carbon and energy transition sectors, we are well-positioned to facilitate the growth of these emerging markets and support decarbonization efforts. In 2023, we launched Silvania, a $500 million fund aimed at the restoration and protection of nature and biodiversity globally, aligning with the Paris Agreement goals and the UN's 30x30 biodiversity initiative. This innovative asset class not only supports nature conservation but also provides sustainable financial returns.

The Role

We are seeking a Platform Engineer with a strong focus on Data Engineering and DevOps to be the backbone of our AI infrastructure. In this role, you will extract high-quality enterprise knowledge from SQL and analytics sources, convert it into embeddings, and ensure that our LLM (Large Language Model) platform remains performant, secure, and cost-efficient. You will serve as a crucial link between raw datasets, model lifecycles, and scalable AWS infrastructure, enabling our AI team to deliver impactful, production-ready solutions.

Your responsibilities will involve designing and operating robust, scalable data and model pipelines that drive our LLM applications. You will lead initiatives in data ingestion, embedding generation, infrastructure as code, model monitoring, and operational governance, playing a hands-on role across all layers of the stack.

Key Responsibilities

• SQL-Centric Data Pipelines: Architect and build robust ETL/ELT pipelines from sources such as MSSQL, Snowflake, Databricks, and S3 using Python, PySpark, and DBT or Liquibase. Implement Change Data Capture (CDC) to maintain efficiency.
• Embeddings & Vector Store Management: Build and enhance chunking and embedding workflows. Manage provisioning and tuning of vector databases (e.g., Pinecone, Weaviate), ensuring incremental updates with nightly refreshes of only modified data.
• MLOps Automation: Orchestrate model training, evaluation, and drift detection utilizing SageMaker Pipelines or Bedrock workflows. Register models using MLflow or SageMaker Registry.
• Infrastructure & DevOps: Own and evolve Terraform-based infrastructure for data and ML workloads, including GPU fleet autoscaling, spot-instance optimization, and Bedrock usage quota tracking.
• Governance & Compliance: Implement lineage tracking, encryption standards, and PII masking. Prepare audit-ready artifacts for SOC2, MiFID compliance, and model risk management.
• Monitoring & Reporting: Surface metrics on data quality, pipeline health, and operational costs using Grafana and Prometheus. Set up alerts to facilitate proactive issue resolution.

Technical Expertise

• Programming & Data: Python, SQL, PySpark, Pandas, DBT, Liquibase
• Pipelines & Orchestration: Airflow, Dagster, Debezium, Kafka Connect
• Cloud & ML Services: AWS Glue, Step Functions, SageMaker, Bedrock, S3
• Infrastructure as Code (IaC) & Observability: Terraform (multi-account patterns), Prometheus, Grafana
• Vector Databases: Weaviate, Pinecone, pgvector, OpenSearch
• Fluent in English

Non-Technical Skills

• 4+ years of experience in data or ML engineering, with at least 3 years managing production-grade MLOps pipelines.
• Deep understanding of data operations best practices, with a proven track record of compliance in regulated industries (e.g., finance, healthcare).
• Experience collaborating with cross-functional stakeholders, including AI researchers, SREs, and data stewards.
• Comfortable operating in fast-paced environments and taking ownership of mission-critical infrastructure.

Apply online using the form below. Please note that only applications matching the job profile will be considered.