Afra Feyza Akyürek

I am a Research Scientist at Scale AI, where I work on LLM post-training, evaluation, and human data programs.

On the evaluation side, I have led human data curation for PRBench and DrugDiscoveryBench. PRBench measures how well language models handle open-ended, high-stakes work in law and finance using expert-authored tasks and detailed rubrics. DrugDiscoveryBench evaluates agents on multi-step computational and information-retrieval tasks in early-stage drug discovery. More broadly, I have extensive experience designing human data programs for complex, economically valuable workflows, including synthetic-data pipelines for later-stage drug development tasks such as clinical study report authoring.

On the post-training side, I am especially interested in online learning, synthetic data, and self-supervision. In RL4F, we trained a critique model to generate natural-language feedback that helps a larger, fixed language model revise its outputs. Deductive Closure Training (DCT) uses model-generated implications and contradictions to improve factual coherence and make model knowledge easier to update. In OnlineRubrics, we developed a method that dynamically elicits evaluation criteria during reinforcement learning so that the reward can adapt as new model behaviors emerge.

More broadly, my research asks how language models can learn from feedback—whether expressed in natural language or gathered through interaction with their environment. Inspired by the way people revise their knowledge and beliefs in response to feedback, I am interested in methods that help models update their outputs in light of facts, requirements, natural phenomena, and human preferences. My goal is to build language models that incorporate such feedback consistently and become more reliable collaborators.

Before joining Scale, I completed my PhD in Computer Science at Boston University, where I was advised by Derry Wijaya. I also had the opportunity to collaborate with researchers at the Allen Institute for AI, Apple, and MIT, including Jacob Andreas.

biography

I was born and raised in İzmir, a beautiful coastal city in Turkey. I attended İzmir Fen Lisesi, where I met my favorite person on Earth. I ranked 31st among roughly two million test takers on Turkey’s national university entrance exam, then moved to Istanbul to study at Koç University, where I double-majored in Computer Engineering and Industrial Engineering. Ekin and I married in 2018 and moved to the United States, where I began a PhD in Statistics at Carnegie Mellon University. I later moved to Boston University, where I completed a PhD in Computer Science. I am now a Research Scientist at Scale AI, working on LLM post-training and evaluation and leading human data programs for complex professional and scientific domains.

I come from a family of avid travelers, and Ekin and I try to keep that spirit alive in our little family. 😍🌍🗺️🏝️