Is AI sexist? The debate was gaining momentum in the pre-COVID-19 world and faded away with the outbreak. But with the impact of lockdown on work activity and women’s careers, gender bias in AI is likely to retake centre stage soon.
We checked on what two respected experts in radiology AI had to say and if indeed they found that their field was sexist or not.
Implications of gender bias in AI development
It is widely accepted that AI systems tend to reflect the biases of the data that is being used to train them. At almost every conference on the topic, the ‘Black Box’ effect casts its long shadow and triggers warning from the scientific community. Many studies have already found AI could be racist. What then of other biases?
Gender bias has not been described in-depth yet, although it’s a possibility that experts are now considering. In 2019, the AI Now Institute, which studies the social implications of AI at NYU, found that less than 20% of the researchers applying to prestigious AI conferences were women, and only a quarter of undergraduates studying AI at Stanford and the University of California at Berkeley were female [1].
One potential consequence could be that AI systems that are built without the participation of women may tackle disease in a predominantly male perspective, and eventually create a lack of knowledge of disease manifestations in women. Research has already shown that gender bias may affect women’s health, even when they have a high risk for disease; for example, cardiovascular disorders [2].
A fair question could be: How can we ensure that systems that are mainly developed by men in radiology won’t be gender-biased and will tackle disease in women the same way as in men?
“Hopefully I can reset some of the balance in my small way,” Andrea Rockall, Clinical Chair of Radiology at Imperial College London, UK said.
Rockall is chief investigator of the MALIBO study, which uses machine learning (ML) in body oncology, and the MALIMAR study, which explores the role of ML in myeloma response. Her team has developed radiomic prognostic vector in ovarian cancer that is now in external validation and works in endometrial cancer and cervical cancer.
She has also just started a new study in metastatic breast, MALIMETS-B, in which male breast cancer is excluded because it is less frequent than in women.
“When it comes to some diseases, there is a higher frequency in men than women and vice versa. In the MALIMAR study, we simply collect the patients with myeloma, with no selection of men over women, just consecutive patients – so if we end up with more men or vice versa – that is just the population,” she explained.
Datasets with such pre-defined rules should reflect the population having the test, and so there is hope on the research front, she believes.
“There may be unintended biases, but I hope researchers will find a way to identify datasets without specific bias towards gender or race,” she said.
Strategies to defeat bias
Fighting bias is a complex battle that unfolds on many fronts. Just as with disease, outcome improves with prevention and early detection.
Maryellen Giger, A.N. Pritzker Professor of Radiology and Medical Physics at the University of Chicago, highlights the importance of tackling the issue during the training curriculum. “At University of Chicago, our graduate students take two courses on bioethics that cover gender, conflict of interest and other moral aspects in their first and fourth year, so before and after they chose their specialty in medical physics,” she said.
Another strategy is to incite women with an interest in AI to take a step forward in their career. That doesn’t necessarily mean mentorship; it could just be a push at the right time, i.e. sponsorship.
“That’s what I do: I connect people with other people. I introduce them and I may be out of the picture. I may just send an email to someone if there’s a new position. We need a pragmatic approach, along with long-term mentorships. In this digital age, it’s very easy to make e-introductions. And I’ve always helped others including both men and women,” said Giger, who is co-founder of Quantitative Insights, now Qlarity Imaging, a company that received the first FDA-clearance for a machine-learning driven CADx system in 2017.
“I always try to support and promote any women who demonstrate an interest in working in AI with me and my team,” Rockall echoed. At her department, she tries to promote workplace flexibility for her team, i.e. planning for flexibility in case of maternity leave, sick children, etc.
“I will also try to offer involvement in studies to some of the female registrars. That is not to say that I do not do the same for the chaps, but I have always tried to support the women and this is evident, I hope, in my publications,” she added.
Team leaders should promote that imaging labs are good places for women to work, she believes. “Team leaders should not support so-called ‘banter’ against women. We do not have anything like that in our imaging lab,” she said.
Personal involvement
Radiology AI advances in large part thanks to the personal effort put in by the researchers involved, even if it means finding time while at home on weekends, in addition to working in a daily practise.
Both Giger and Rockall plead guilty as charged. “I have spent many a weekend segmenting a dataset that got behind schedule. Also, sometimes the funding models don’t quite fit – like when I applied for my first AI grant with NIHR EME fund,” Rockall said.
“I sleep five hours a night. My four children grew up sitting next to me while I was on the computer. They were my “hobby”, and when at home, my research was my “hobby”. By choice, I didn’t do much anything else; I just enjoyed my family and my work. You have to have a job you really like to do that,” Giger said.
Conciliating work with parenting can be a real challenge for women, especially if they want to do something that has never been done. An army of nannies will help, but it takes more than just having support to get going.
“I have been a working mum and have worked perhaps more slowly because of this,” Rockall admitted. “It is not easy and requires inner drive. Once I heard someone saying that artists are compelled to produce their artwork. And the other day, I looked at myself in the middle of the COVID-19 crisis with all sorts of things going on – and yet I anonymised – manually – a whole data set for a study as the research team were re-deployed, then posted data and pushed on contracts. I have just finished writing another grant whilst at home with all the kids. So I seem to be compelled,” she said, adding that the research environment is “pretty tough”. “When registrars say they want to do research, I usually set them up with a study. Many stop when they discover that it requires working more than most people want to do,” she said.
For Giger, it all boils down to balancing personal and professional life. “If I’m at home and get an email, I might turn off the work switch. But then when I’m at work I need to make sure I switch off that mother’s worry. You can always grasp opportunities when you have children, but, like the book said, don’t sweat the small stuff,” she said.
Glass ceiling
The men-women ratio in radiology AI was unknown at the time this article was written. But just looking at the list of employees of a few specialised companies showed that a fair amount of AI developers, biomedical engineers and telecommunication engineers are women.
Great researchers in AI radiology are women too. Rockall and Giger are experienced leaders in their field, and many more have become great names in the sphere – to name just a few: Julia Schnabel (King’s College London), Jayashree Kalpathy-Cramer (Harvard University) and Tessa Cook (University of Pennsylvania).
However, it seems very few women make it to a CEO or committee chairperson position.
So is there a glass ceiling in radiology AI as well?
“I have to be honest, I do feel the glass ceiling,” Rockall said. “I often see that a committee is made of many men that perhaps talk a lot about AI but don’t have much, if any, experience in developing and validating AI systems. Some women that I think should be there are not invited,” she said.
Rockall is sometimes asked to help big companies and funders understand various aspects of what a good AI study plan looks like or what is a good use case, but then never gets invited to sit on the grant-funding steering committee, she reports.
“It is quite frustrating. They will put me on something that is quite hard work, like the longlisting or shortlisting committee, but not the steering committee that will make the final funding decisions. But maybe I am not good at the networking aspect and getting known.”
There’s a difference between diversity and inclusion, Giger pointed out. “This is not my quote, but I think it’s fitting: ‘Diversity is being asked to the dance but inclusion is being asked to dance.’ Many universities and scientific associations are now working to improve the number of women involved in decision-making,” she said.
Also, women should probably brag a bit more about their achievements, she continued. “We don’t usually share and promote our accomplishments. Some men move ahead because they promote themselves.”
Women should not hesitate to ask for an extension on their projects, for a pay rise, for a position, if that’s what they want. “You don’t ask, you don’t get. The worst that can happen is they say no,” she concluded.
[1] https://ainowinstitute.org/discriminatingsystems.pdf
[2] https://www.ncbi.nlm.nih.gov/pubmed/8582818
—
Andrea Rockall is the Clinical Chair of Radiology at Imperial College London, UK, and chief investigator of the MALIBO study, which uses machine learning in body oncology, and MALIMAR, which explores the role of ML in myeloma response. She has obtained two NIHR grants on AI for these studies.
Both projects are well advanced into phase 3 and/or the end of data curation and global ML prediction of the presence of disease. She is also currently shortlisted for the NIHR AI award for opportunistic screening on CT, moving from lab to clinic.
She became involved in AI radiology in 2012, as she moved to Imperial College London after 12 years as a consultant and senior lecturer in radiology at St Bartholomew’s Hospital. “I am an accidental AI researcher I would say,” she admitted.
Maryellen Giger is the A.N. Pritzker Professor of Radiology, the Committee on Medical Physics, and the College at the University of Chicago. For multiple decades, she has been working on computer-aided diagnosis, machine learning and deep learning in medical imaging, and cancer diagnosis and management.
Her AI research in breast cancer for risk assessment, diagnosis, prognosis and therapeutic response has yielded various translated components, and she is using these “virtual biopsies” in imaging-genomics association studies.
Giger is a former president of the American Association of Physicists in Medicine (AAPM) and of the Society of Photo-Optical Instrumentation Engineers (SPIE). She is also the Editor-in-Chief of the Journal of Medical Imaging. Furthermore, she is a member of the National Academy of Engineering, Fellow of AAPM, AIMBE, SPIE, SBMR, IEEE, IAMBE and was co-founder, equity holder, and scientific advisor of Quantitative Insights [now Qlarity Imaging], which produces QuantX, the first FDA-cleared, machine-learning driven CADx system.
