Anthropic, an AI security and research company, aims to reduce the risks and increase the potentials of AI. It raised its first round of funding of $124M in the year of its founding in 2021. It just announced a $580M round led by Sam Bankman-Fried, CEO of FTX, with participation from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn and the Center for Emerging Risk Research (CERR).
Anthropic was co-founded by brother and sister Daniela and Dario Amodei during containment. Daniela Amodei previously worked at Stripe and OpenAI, her brother also worked at Open AI as well as Google and Baidu. The starting team, consisting of 7 former Open AI employees, now numbers around 40 and is still growing. The team members come from very different backgrounds: specialists in data visualization, security, ML but also policy experts, neuroscientists or computational biologists.
Daniela Amodei states:
“Now that we have built the organization, we are working to ensure that Anthropic has the culture and governance to continue to responsibly explore and develop safe AI systems as we evolve. We are excited about what lies ahead and grateful to all be working together.”
Their goal is to make systems more steerable, robust and interpretable. To do this, they are trying to describe, understand, predict, scientifically model the workings of AI systems through this deeper understanding. Their research areas are natural language, human feedback, scaling laws, reinforcement learning, code generation and interpretability. The company regularly publishes papers and articles where they share their advances.
For Anthropic, ” The goal of this research is to develop the technical components needed to build large-scale models that have better implicit guarantees and require less post-training intervention, as well as to develop the tools needed to look deeper inside these models to make sure the guarantees actually work.”
Anthropic is also working on exploring the policy and societal impacts of these models through partnerships.
Dario Amodei concludes:
“With this fundraiser, we will explore the predictable scalability properties of machine learning systems, while closely examining the unpredictable ways in which capabilities and security issues can emerge at large scale. We have made significant initial progress in understanding and driving the behavior of AI systems, and we are progressively assembling the elements necessary to create integrated, usable AI systems that benefit society.”
Translated from Anthropic, société de sécurité et de recherche en IA, a réalisé une levée de fonds de 580 millions de dollars
