Deploying various models to production and owning the process from working checkpoint to performant serving infrastructure.
Optimizing inference quality and performance in the model serving stack to:
Improve model output quality, e.g. experimenting with various guidance and guardrail schemes, automatic detection of hallucinations or errors.
Improve the inference performance in terms of latency, throughput, cost; e.g. via custom kernels, cache optimization, GPU scheduling/scaling optimization, quantization.
Implementing robust monitoring and incident flagging systems for inference management. E.g. flagging and investigating timing anomalies, inference failures, understanding general inference statistics across various models.
Creating tooling to allow rapid deployment of newly developed models from checkpoint to scaled inference, optimizing GPU availability and choice for new models.
Making autonomous decisions to holistically improve the inference experience in different models and use cases. E.g. how to approach handling the different kinds of optimization needed for an extreme low-latency streaming model versus a “quality sensitive” slow dubbing model.
Benefits
Growth Opportunities: Enjoy professional development and career growth in a fast-paced, dynamic industry.
Competitive Compensation: Receive a competitive salary, performance bonuses, and comprehensive benefits.
CAMB.AI is an AI localization platform that enables content creators, media enterprises, and sports leagues to dub and translate video and live broadcasts into over 140 languages…