Build and maintain serving infrastructure for ONNX models, Augloop, SLM-based inference, and future LLM/SLM pipelines.
Integrate models into scalable APIs for online prediction and retrieval-augmented generation workflows.
Set up and run real-time A/B experiments on production Copilot features.
Implement alerting, logging, and telemetry tools to monitor model drift, latency, and regressions.
Develop dashboards for automated quality monitoring and error detection in inference traffic.
Optimize inference latency and cost across CPU/GPU environments.
Build internal tools for performance analysis, model comparison, and troubleshooting.
Work on batch and streaming inference frameworks, ensuring SLA adherence.
Implement resource orchestration and utilization tracking across CPU/GPU workloads.
Contribute to tools that monitor uptime, throughput, container health, and job scaling.
Ensure scalability and reliability of model APIs, with clear SLAs around latency, throughput, cost, and memory footprint.
Profile models and infra for cold start issues, load testing, and concurrency handling.
Integrate Responsible AI checks for fairness, explainability, and performance variance.
Address AI injection attacks, inference sandboxing, and privacy guardrails.
Contribute to regression pipelines for SLA, PII, and compliance validation across Copilot features.

Required Experience

3–6 years of hands-on experience as an ML SWE or MLOps Engineer in production AI systems.
Strong coding skills in Python, C++, or Go, with experience in TensorRT, ONNX Runtime, or similar.
Experience with ML Ops tools: Azure ML, Kubernetes, Prometheus, Grafana, MLflow, Airflow, etc.
Hands-on with monitoring systems, load testing tools, and infra debugging utilities.
Familiarity with model security, compliance frameworks, or Responsible AI practices is a plus.

Soft Expectations

Able to work independently and deliver code-quality infrastructure within agile cycles.
Document architecture, assumptions, and SLA metrics clearly.
Comfort in collaborating with both AI scientists and infra/DevOps teams.
Availability for overlap with Prague or Redmond teams preferred.