“Hi Emily, I’m wondering if you’ve written something regarding ethical considerations of large language models or something you could recommend from others?” she asked, referring to a buzzy kind of artificial intelligence software trained on text from an enormous number of webpages.
“Sorry, I haven’t!” Bender quickly replied to Gebru, according to messages viewed by CNN Business. But Bender, who at the time mostly knew Gebru from her presence on Twitter, was intrigued by the question. Within minutes she fired back several ideas about the ethical implications of such state-of-the-art AI models, including the “Carbon cost of creating the damn things” and “AI hype/people claiming it’s understanding when it isn’t,” and cited some relevant academic papers.
Gebru, a prominent Black woman in AI — a field that’s largely White and male — is known for her research into bias and inequality in AI. It’s a relatively new area of study that explores how the technology, which is made by humans, soaks up our biases. The research scientist is also cofounder of Black in AI, a group focused on getting more Black people into the field. She responded to Bender that she was trying to get Google to consider the ethical implications of large language models.
The paper considers the risks of building ever-larger AI language models trained on huge swaths of the internet, such as the environmental costs and the perpetuation of biases, as well as what can be done to diminish those risks. It turned out to be a much bigger deal than Gebru or Bender could have anticipated.
“Academics should be able to critique these companies without repercussion,” Gebru told CNN Business.
Google declined to make anyone available to interview for this piece. In a statement, Google said it has hundreds of people working on responsible AI, and has produced more than 200 publications related to building responsible AI in the past year. “This research is incredibly important and we’re continuing to expand our work in this area in keeping with our AI Principles,” a company spokesperson said.
“A constant battle from day one”
Gebru joined Google in September 2018, at Mitchell’s urging, as the co-leader of the Ethical AI team. According to those who have worked on it, the team was a small, diverse group of about a dozen employees including research and social scientists and software engineers — and it was initially brought together by Mitchell about three years ago. It researches the ethical repercussions of AI and advises the company on AI policies and products.
She was eventually convinced by Mitchell’s efforts to build a diverse team.
“I felt like our group was like a family,” Gebru said.
Yet Gebru also described working at Google as “a constant battle, from day one.” If she complained about something, for instance, she said she would be told she was “difficult.” She recounted one incident where she was told, via email, that she was not being productive and was making demands because she declined an invitation for a meeting that was to be held the next day. Though Gebru does not have documentation of such incidents, Hanna said she heard a number of similar stories like this from Gebru and Mitchell.
“The outside world sees us much more as experts, really respects us a lot more than anyone at Google,” Gebru said. “It was such a shock when I arrived there to see that.”
Internal conflict came to a head in early December. Gebru said she had a long back-and-forth with Google AI leadership in which she was repeatedly told to retract the “stochastic parrots” paper from consideration for presentation at the FAccT conference, or remove her name from it.
On the evening of Tuesday, December 1, she sent an email to Google’s Brain Women and Allies mailing list, expressing frustration about the company’s internal review process and its treatment of her, as well as dismay over the ongoing lack of diversity at the company.
She also wrote that the paper was sent to more than 30 researchers for feedback, which Bender, the professor, confirmed to CNN Business in an interview. This was done because the authors figured their work was “likely to ruffle some feathers” in the AI community, as it went against the grain of the current main direction of the field, Bender said. This feedback was solicited from a range of people, including many whose feathers they expected would be ruffled — and incorporated into the paper.
“We had no idea it was going to turn into what it has turned into,” Bender said.
The next day, Wednesday, December 2, Gebru learned she was no longer a Google employee.
“It ignored too much relevant research — for example, it talked about the environmental impact of large models, but disregarded subsequent research showing much greater efficiencies. Similarly, it raised concerns about bias in language models, but didn’t take into account recent research to mitigate these issues,” he wrote.
Uncomfortable taking her name off the paper and wanting transparency, Gebru wrote an email that the company soon used to seal her fate. Dean said Gebru’s email included demands that had to be met if she were to remain at Google. “Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date,” Dean wrote.
She told CNN Business that her conditions included transparency about the way the paper was ordered to be retracted, as well as meetings with Dean and another AI executive at Google to talk about the treatment of researchers.
“We accept and respect her decision to resign from Google,” Dean wrote in his note.
Outrage in AI
Gebru’s exit from the tech giant immediately sparked outrage within her small team, in the company at large, and in the AI and tech industries. Coworkers and others quickly shared support for her online, including Mitchell, who called it a “horrible life-changing loss in a year of horrible life-changing losses.”
Behind the scenes, tensions only grew.
Mitchell told CNN Business she was put on administrative leave in January and had her email access blocked then. And Hanna said the company conducted an investigation during which it scheduled interviews with various AI ethics team members, with little to no notice.
“They were frankly interrogation sessions, from how Meg [Mitchell] described it and how other team members described it,” Hanna, who still works at Google, said.
Hanna said the ethical AI team had met with Croak several times in mid-December, during which the group went over its list of demands point by point. Hanna said it felt like progress was being made at those meetings.
A Google spokesperson did not dispute that Mitchell was fired when asked for comment on the matter. The company cited a review that found “multiple violations” of its code of conduct, including taking “confidential business-sensitive documents and private data of other employees.”
Mitchell told CNN Business that the ethical AI team had been “terrified” that she would be next to go after Gebru.
“I have no doubt that my advocacy on race and gender issues, as well as my support of Dr. Gebru, led to me being banned and then terminated,” she said.
Big company, big research
More than three months after Gebru’s departure, the shock waves can still be felt inside and outside the company.
“It’s absolutely devastating,” Hanna said. “How are you supposed to do work as usual? How are you even supposed to know what kinds of things you can say? How are you supposed to know what kinds of things you’re supposed to do? What are going to be the conditions in which the company throws you under the bus?”
By its nature, academic research about technology can be disruptive and critical. In addition to Google, many large companies run research centers, such as Microsoft Research and Facebook AI Research, and they tend to project them publicly as somewhat separate from the company itself.
“Basically we’re in a situation where, okay, here’s a paper with a Google affiliation, how much should we believe it?” Bender said. Gebru said what happened to her and her group signals the importance of funding for independent research.
And the company has said it’s intent on fixing its reputation as a research institution. In a recent Google town hall meeting, which Reuters first reported on and CNN Business has also obtained audio from, the company outlined changes it’s making to its internal research and publication practices. Google did not respond to a question about the authenticity of the audio.
“I think the way to regain trust is to continue to publish cutting-edge work in many, many areas, including pushing the boundaries on responsible-AI-related topics, publishing things that are deeply interesting to the research community, I think is one of the best ways to continue to be a leader in the research field,” Dean said, responding to an employee question regarding outside researchers saying they will read papers from Google “with more skepticism now.”
In early March, the FAccT conference halted its sponsorship agreement with Google. Gebru is one of the conference’s founders, and served as a member of FAccT’s first executive committee. Google had been a sponsor each year since the annual conference began in 2018. Michael Ekstrand, co-chair of the ACM FAccT Network, confirmed to CNN Business that the sponsorship was halted, saying the move was determined to be “in the best interests of the community” and that the group will “revisit” its sponsorship policy for 2022. Ekstrand said Gebru was not involved in the decision.
“Never imagined what transpired after we decided to collaborate on this paper,” she tweeted.