Addressing Gender and Institutional Bias at Cosyne

The gender balance in computational and systems neuroscience is abysmal — according to BiasWatchNeuro, a website that tracks the base rate of women at neuroscience conferences, fewer than 20 percent of the attendees at the field’s premier conference, Cosyne, are women. Conference organizers have worked hard over the last few years to balance the genders of invited speakers. But the bulk of the program comes from abstracts submitted by potential attendees to present as talks or posters. The process is highly competitive — only 40 to 60 percent of abstracts are accepted — and potentially problematic. An analysis of abstracts submitted to the 2019 meeting in Lisbon suggested that the acceptance rate for male authors was higher than for women.

Following that meeting, conference organizers decided to investigate the issue scientifically. The program chairs for the 2020 conference, Anne-Marie Oswald, a neuroscientist now at the University of Chicago, and Srdjan Ostojic, a neuroscientist at the École Normale Supérieure, in collaboration with Haruka Uchida, a graduate student in economics at the University of Chicago, designed an experiment around the abstract review process. Each abstract underwent both single-blind review, where reviewers knew the authors’ identities, and double-blind review, where the authors and their affiliations were anonymous. The team then analyzed how gender, career stage — student versus postdoc or faculty — and the ranking of the author’s institution influenced scores.

Oswald and Ostojic presented the results at the Cosyne meeting in Denver in February. They found that double-blind review helped in some instances but not in others. Under single-blind conditions, female first and last authors received significantly lower scores than their male counterparts. Student first authors received lower scores than postdocs or professors, and authors from lower-ranked institutions got lower scores than those from higher-ranked ones. Double-blind reviewing significantly improved scores for student first authors and for authors from mid-ranked institutions. It also improved female first authors’ scores, but the results were not statistically significant. It had no effect on scores for abstracts with female last authors.

Oswald and Ostojic spoke with the Simons Collaboration on the Global Brain about their approach, their findings and how to deal with remaining bias. An edited version of the conversation follows.

What inspired you to look at the Cosyne review process at this level of detail?  

OSWALD: There was a lot of public discussion about gender bias after the 2019 meeting. Cosyne wasn’t able to answer a lot of the issues being raised because no one had the numbers to back it up. When Srdjan and I took over, that was the first thing we looked at. There weren’t enough women, and there was an overrepresentation of a small number of institutions. We wondered: What can we do better?

OSTOJIC: The figures from the 2019 conference weren’t a full statistical analysis. The executive committee wanted to address this more thoroughly to see how any bias could be fixed.

How did you revise the abstract review process?

OSTOJIC: We wanted to look at all levels where bias could play a role. We started by looking at the program committee, which recruits people to review abstracts. We wanted to make sure it was more gender-balanced and strongly encouraged the program committee to recruit a balanced pool of reviewers.

OSWALD: Both the reviewer pool and the program committee were increased to 30 percent women, which was a big improvement.

OSTOJIC: That’s way above base rate [the percentage of women in the field]. We also changed how we selected contributed talks [short talks selected from submitted abstracts]. We ranked the top abstracts and created a long list that was gender-balanced. We then sent the whole list to the program committee, who voted.

Why did you decide to do an experiment testing double-blind review?

OSWALD: A lot of people wanted to know if double-blind would make a real difference and where it would make a difference. We designed an experiment where each abstract would be reviewed four times — two single-blind and two double-blind reviews — so we can directly compare the effect of single- versus double-blind.

You found that double-blind review affected some types of bias but not others. What finding were you most surprised by?

OSWALD:  I was surprised that some gender bias remained even with double-blind review.

OSTOJIC: I was disappointed that double-blind did not solve all the existing bias. I also expected to see bias toward stars in the field, so I was very surprised at the lack of any effect of notoriety, which we quantified using the number of citations or h-index, on bias between single- and double-blind.

Why do you think gender bias persists even when reviewers don’t know the author’s gender?

OSTOJIC:  A study outside of neuroscience found that male authors use very positive words much more often than female authors when they present their own work. That’s one potential explanation for the bias that persists — the way we write is gendered. That might be something to work on. On both sides, researchers have to be careful and not present themselves in overly positive ways. Prior to this study, Stephanie Palmer [a neuroscientist at the University of Chicago] had set up a coaching initiative to help people work on their abstracts. It would be great if more people used it.

How did you deal with the gender bias issue, given that double-blind reviewing didn’t solve it?

OSWALD:  Srdjan and I and the executive committee still wanted at least base rate acceptance, meaning that if 17 percent of submissions were from women, we wanted 17 percent of accepted abstracts to be from women.

OSTOJIC: We equalized the acceptance rate by adding more posters with female authorship.

Double-blind review didn’t get rid of gender bias, but it did reduce institutional bias, meaning bias in favor of top-ranked universities.

OSWALD:  Yes. Mid-tier institutions, those with world rankings between 50 and 150, were helped by blinding. Some people were surprised that double-blind review did not affect scores for top-ranked institutions. But the fact is that they have the best resources and are probably doing more cutting-edge research. That will come though regardless of whether the authors’ affiliation is named or not.

Are you also looking at other types of bias, such as racial bias?

OSWALD:  We looked at some other forms of diversity, including race. Acceptance for underrepresented groups ended up being at base rate. But the numbers are way too small to do real statistics.

OSTOJIC: That’s a real issue. We wanted to study diversity beyond gender and asked for that information with abstract submissions, but few people filled in those fields. Cosyne set up a diversity committee to come up with ideas to improve diversity overall.

OSWALD: As we start to fix the gender problem, we can start looking at other forms of diversity and shaping the meeting to meet them.

Is Cosyne planning to collect more data?

OSTOJIC: Yes, more data will be collected during the 2021 review process, and the results will eventually be published.

The gender balance for invited speakers at Cosyne is 50-50. How have organizers achieved this?

OSWALD: We had a hard time getting a 50-50 balance at the invited level — it’s only in the last few years that we have really been able to do it. There are fewer women in the field, and even though there are lists to find them, a lot of the time women still say no because they are traveling to every meeting under the sun. Also, Cosyne has a rule that, once you speak, you can’t speak again for 10 years. So you don’t want to accept the invite too early in your career. If you are accepting earlier than your male counterparts, you don’t get to present in your prime like your male counterparts do.

OSTOJIC: Because Cosyne has been doing this for a number of years, the pool of potential speakers has been going down. It’s becoming more and more challenging.

What is your advice for other conferences that want to try to reduce bias?

OSWALD: You can’t expect that a problem with lack of diversity or gender balance will just fix itself, even if you implement double-blind review. First, you need to look at the data and understand where and how your conference falls short of diversity and inclusion goals. Second, you need to be proactive at every level of organization — chairs, committees, reviewers — and make sure that everyone understands the mission. You can implement mechanisms like double-blind review, but you also need to evaluate if they work to mitigate bias as intended. Organizers should be ready to make the choice to balance the scale by selection. Finally, keep an open line of communication between the conference participants and the organization to ensure awareness of issues as they arise.

SCGB has sponsored the Cosyne conference since 2017.