Senior AI Researcher- Pre-training (f/m/d)

Alephalpha

Heidelberg 10/24/2026 Full Time

<h2>Our Mission</h2>Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released <a target="_blank" rel="noopener noreferrer" href="https://huggingface.co/Aleph-Alpha/Kolibri-1">Kolibri-1 model</a>. Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We're building that in Heidelberg.We are hiring a Senior AI Researcher to join our Pre-training team and to advance the architecture and training of our next generation of foundation models. If you are excited about designing inference-efficient architectures, optimizing training recipes that scale reliably, and training models on a large scale cluster (thousands of NVIDIA Blackwell GPUs), we would love to hear from you. <h2>Team Culture</h2>We foster a culture built on ownership, autonomy, and empowerment. Teams and individual contributors are trusted to take responsibility for their work and drive meaningful impact. We maintain a flat organisational structure with efficient, supportive management that enables quick decision-making, open communication, and a strong sense of shared purpose. We collaborate closely on complex technical problems, working in pairs or using mob programming to resolve challenging issues.<h2>About the Role</h2>As a Senior AI Researcher in Pre-training (f/m/d), you will own the critical technical levers that determine the success of our next-generation models: architecture, optimization, stability, and scaling. Working at the high-leverage intersection of research and engineering, you will translate mathematical reasoning and empirical observations into principled training decisions - from small-scale proxy experiments to multi-thousand-GPU runs. We are looking for an expert who can combine rigorous experimental design with high-quality production code, directly influencing model quality, run reliability, and the efficiency of the models we ship.<h2>Your Responsibilities</h2><ul style="min-height:1.5em"><li>Recipe & Architecture Optimization: Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.</li><li>Scaling Strategy & Predictability: Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions.</li><li>Stability, Diagnostics & Debugging: Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors.</li><li>System-Model Co-Design: Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.</li></ul><h2>Core Qualifications</h2><ul style="min-height:1.5em"><li>You are proficient in Python and deeply familiar with PyTorch-based training workflows.</li><li>You have a strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.</li><li>You have a strong mathematical foundation and are comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics.</li><li>You deeply understand transformer training dynamics, optimisation, and the behaviour of large distributed training jobs.</li><li>You can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.</li><li>Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters).</li></ul><ul style="min-height:1.5em"><li>You apply strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.</li><li>You are able to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems.</li><li>You collaborate effectively within a research and engineering team and communicate clearly about your work across Pre-training and the broader AAR/AA organization.</li><li>You are able to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team.</li></ul><h3>Preferred Qualifications</h3><ul style="min-height:1.5em"><li>Large-Scale Training: Hands-on experience training LLMs or multimodal models on large GPU clusters using distributed frameworks (e.g., Megatron-LM, DeepSpeed, torchtitan).</li><li>Predictive Scaling: Familiarity with scaling laws, hyperparameter transfer, or methods for predicting large-scale training behavior from smaller proxy runs.</li><li>Stability & Performance: Experience profiling distributed jobs and diagnosing training anomalies like loss spikes, numerical instability, or optimizer pathologies.</li><li>Advanced Architectures: Exposure to sparse training approaches (e.g., Mixture-of-Experts) and an understanding of their routing and systems trade-offs.</li><li>Track Record of Impact: Demonstrated research excellence through top-tier publications (NeurIPS, ICML, ICLR), impactful open-source contributions, or significant shipped technical work.</li><li>Systems Curiosity: Low-level kernel optimization is not required, but we highly value a strong curiosity about the hardware and systems constraints that shape scale.</li></ul>What we offer<ul style="min-height:1.5em"><li>Become part of an AI revolution!</li><li>30 days of paid vacation</li><li>Access to a variety of fitness & wellness offerings via <a target="_blank" rel="noopener noreferrer" class="theme markdown__link" href="https://wellhub.com/de-de/">Wellhub</a></li><li>Mental health support through <a target="_blank" rel="noopener noreferrer nofollow" href="http://nilo.health">nilo.health</a></li><li>Substantially subsidized company pension plan for your future security</li><li>Subsidized Germany-wide transportation ticket</li><li>Budget for additional technical equipment</li><li>Flexible working hours for better work-life balance and hybrid working model</li><li><a target="_blank" rel="noopener noreferrer" class="theme markdown__link" href="https://www.jobrad.org/">JobRad®</a> Bike Lease</li></ul>Find <a href="https://www.arbeitnow.com">Jobs in Germany</a> on Arbeitnow</a>

Research

Senior AI Researcher- Pre-training (f/m/d)

Related on GOH