OpenAI's Chief Safety Researcher Lillian Wen resigns, marking the first wave of AI safety leaders to leave the company



Lillian Wen, Principal Safety Researcher at OpenAI, has announced that she will be leaving the company on November 9, 2024. She has served as OpenAI's Vice President of Research and Safety since August 2024, and prior to that was head of the Safety Systems team.

OpenAI loses another lead safety researcher, Lilian Weng | TechCrunch
https://techcrunch.com/2024/11/08/openai-loses-another-lead-safety-researcher-lilian-weng/



Wen updated his X (formerly Twitter) account, stating, 'After about seven years at OpenAI, I have decided to retire. I have learned a lot and am ready to reset and try something new,' and announced his retirement from OpenAI.




Wen attached an image to his post on X, which contained a farewell message he sent to his colleagues at OpenAI. The farewell message that Wen sent to his colleagues is as follows:

Dear Friends,

Today, I made the very difficult decision to leave OpenAI. November 15th will be my last day in the office. OpenAI is where I grew as a scientist and a leader, and I will forever cherish the time I worked with and became friends with all of you along the way. The OpenAI team was my best friends, my teachers, and a part of my identity.

I still remember how intrigued and fascinated I was by OpenAI's mission back in 2017, a group of people dreaming of an impossible, sci-fi future. I started working at OpenAI on full-stack robotics challenges, from deep reinforcement learning algorithms to perception to firmware, with the goal of teaching a robotic hand how to solve a Rubik's Cube. It took the whole team two years to achieve this goal, but we finally did it.

As OpenAl began to dive into the GPT paradigm and began exploring how to deploy the best AI models in the real world, I built the first applied research team and launched early versions of the fine-tuning API, embedding API, and moderation endpoints, laying the groundwork for applied safety research and delivering novel solutions for many of our early API customers.

After the release of GPT-4, I was asked to take on a new challenge: rethinking OpenAI's safety systems vision and centralizing the work into one team that owns all of the safety stack. This has been the most challenging, stressful, and exciting work I've ever done. The Safety Systems team has over 80 talented scientists, engineers, project managers, and policy experts, and I'm proud of all we've accomplished as a team.

In particular, the o1-preview model is our most secure model, demonstrating exceptional resistance to jailbreak attacks while maintaining its usability. Our results include:

We trained the model on how to handle sensitive or insecure requests, including when to reject or not, and followed a set of clearly defined model safety behavior policies to strike a good balance between safety and practicality.
Improved adversarial robustness in each model launch, including jailbreak defense, instruction hierarchy, and greatly improved robustness through inference.
We designed a rigorous and creative evaluation methodology aligned with the Preparedness Framework , and conducted comprehensive safety testing and red teaming for each frontier model. Our commitment to transparency is reflected in our detailed model system cards.
We have developed and publicly released an industry-leading moderation model with multi-modal capabilities. We are currently working on a more generic monitoring framework and enhanced safety reasoning capabilities that will power even more safety workstreams.
- We laid the engineering foundation for safety data logging, metrics, dashboards, active learning pipelines, classifier deployment, inference-time filtering, and novel rapid response systems.

'I'm incredibly proud of everyone on the Safety Systems team given all we've accomplished and have great confidence that the team will continue to thrive. After seven years at OpenAI, I feel ready to explore new things. OpenAI is on a rocket-like growth trajectory.'

P.S. My blog is alive and well, and I might find the time to update it a bit more frequently soon, and maybe even some more coding time.

With love, Lillian.



In addition, Ilya Sutskever and Jan Reich, who led the 'Super Alignment Team' that deals with the problem of how to maintain control of AI at OpenAI, left the company in May 2024. As a result, the Super Alignment Team was disbanded. Sutskever is said to be the person who led the CEO Sam Altman dismissal scandal that occurred in November 2023, and it is reported that he was in conflict with Altman over the safety of AI.

OpenAI's chief scientist, Ilya Satkivar, who led the dismissal of CEO Sam Altman, resigns - GIGAZINE



Regarding the fact that the people in charge of 'AI safety' at OpenAI are leaving one after another, technology media TechCrunch pointed out, 'Many people have expressed concerns about OpenAI's efforts at safety, which aims to build increasingly powerful AI systems. Policy researcher Miles Brundage left OpenAI in October 2024, and the AGI (artificial general intelligence) preparation team, on which he served as an advisor, was subsequently disbanded. AI researcher Sushil Balaji also left the company, believing that OpenAI's technology would bring more harm than good to society.'

When TechCrunch reached out to OpenAI, the company responded that it is continuing to work to select Wen's successor in the Safety Systems division. An OpenAI spokesperson said, 'We are deeply grateful for Lillian's contributions to groundbreaking safety research and building rigorous technical safety measures. We are confident that the Safety Systems team will continue to play a vital role in ensuring the safety and reliability of our systems and serving hundreds of millions of people around the world.'

Other executives who have left OpenAI in the past few months include former CTOMira Murati , Chief Research Officer Bob McGrew, VP of Research Barret Zoff, prominent researcher Andrei Karpathy, and co-founder John Shulman, who has joined competitor Anthropic.

OpenAI co-founder John Shulman resigns to join Anthropic - GIGAZINE



Related Posts:

in Note, Posted by logu_ii