AI Research Engineer (Kernel & Inference Optimization)

Accountabilities • Design and deploy advanced model-serving architectures optimized for high throughput, low latency, and efficient memory utilization.

• Develop inference pipelines capable of operating effectively across diverse environments, including resource-constrained mobile devices and edge platforms. • Establish clear performance targets covering response latency, token generation speed, throughput, memory footprint, and reliability. • Build and execute controlled inference benchmarks in simulated and production environments, tracking latency, throughput, memory consumption, and error rates. • Create and maintain representative datasets and simulation scenarios for evaluating model performance under real-world and resource-constrained conditions. • Identify computational and memory bottlenecks across inference pipelines and implement solutions involving batching, networking, memory management, and other system-level optimizations. • Develop custom GPU kernels and compute shaders for mobile hardware, including solutions written in Metal Shading Language (MSL). • Apply advanced inference optimization techniques such as pruning, quantization, Flash Attention, KV caching, and speculative decoding. • Design and optimize distributed inference systems using approaches such as tensor parallelism, pipeline parallelism, and expert parallelism for large-scale GPU workloads. • Work with cross-functional engineering and research teams to integrate optimized inference frameworks into production and edge-device applications. • Define evaluation methodologies, document experimental results, compare performance against established benchmarks, and continuously refine optimization strategies. • Monitor production performance and use empirical research to identify opportunities for further improvements in scalability, efficiency, and reliability.

Requirements:

• Degree in Computer Science or a related technical field; a PhD in NLP, Machine Learning, or a related discipline is highly relevant, particularly with a strong AI research track record and publications at leading conferences. • Proven expertise in Metal Shading Language (MSL), including the ability to write custom compute shaders from scratch. • Demonstrated experience with low-level kernel optimization and inference optimization on mobile or other resource-constrained devices. • Track record of delivering measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications. • Deep understanding of modern model-serving architectures, inference engines, and optimization techniques for high-performance AI deployment. • Strong experience writing GPU kernels for mobile devices such as smartphones. • Practical experience developing and deploying end-to-end inference pipelines, from model optimization through production integration on constrained hardware. • Strong ability to apply empirical research and systematic experimentation to solve latency, computational, and memory challenges. • Experience designing robust evaluation and benchmarking frameworks for inference systems. • Knowledge of distributed inference techniques, including tensor parallelism, pipeline parallelism, and expert parallelism for large-scale GPU clusters. • Deep understanding of the mathematical foundations and architecture of diffusion models and Vision Transformers. • Familiarity with modern inference optimization techniques including pruning, quantization, Flash Attention, KV Cache optimization, and speculative decoding such as EAGLE. • Strong analytical and problem-solving abilities, with an ability to investigate complex system bottlenecks and turn research findings into practical engineering solutions. • Excellent English communication skills and the ability to collaborate effectively with distributed, cross-functional technical teams.