Reading Together: Reliability and Multilingual Global Communities
Reading Together is a multilingual, six-month Art+Feminism research project. We study reliable source (WP:RS) guidelines and related content policies on English, French, and Spanish Wikipedia pages.
What are these guidelines? How is a “reliable source” defined in action? And what is the impact of these guidelines on content about marginalized communities and editors on these three Wikipedias?
Reading Together addresses these questions. The project is spearheaded by three leaders in feminist research and information activism in the Americas. We are supported by a multilingual advisory board of scholars and activists.
Our project will provide much needed insight on how definitions of “reliable sources” impact coverage of underrepresented, non-hegemonic communities. This catagorization includes, but is not limited to, cis and transgender women, non-binary people, non-Western communities, LGTBIQ+, and BIPOC communities.
Our effort is the outcome of a decade of global activism with Wikipedia, specifically with Art+Feminism, to expand representation and inclusion in Wikimedia through edit-a-thons and campaigns. Reading Together will deepen this important work by identifying barriers to the inclusion of relevant sources
Why we are doing this
The “gender gap” on English Wikipedia has been well-documented.¹ A 2011 Wikimedia Foundation survey found fewer than 10% of editors self-identified as female.² More recent research puts that number at 16% globally and 23% in the United States.³ The asymmetry among editors demographics has impacted content. Data analysis tools and computational linguistics studies show that English Wikipedia has fewer and less extensive articles on women; those same tools have shown gender biases in biographical articles.⁴ French and Spanish Wikipedias suffer from similar inequities.⁵ ⁶
In the past decade, redress efforts in English, Spanish, and French Wikimedia communities have included recruiting new editors and creating new content during edit-a-thons, programs, campaigns, and contests. Reading Together project leads have been at the helm of many of these efforts. Melissa Tamani and Amber Berson are Art+Feminism co-leads in Peru and Canada, working in Spanish and French Wikipedias, respectively. Monika S. Jones has led numerous initiatives in English Wikipedia, as well as providing support for Art+Feminism events. All three have hosted editing and educational programs at cultural institutions in North and South America, as well as authored materials on Wikipedia editing and digital cultures.
Reading Together addresses Wikipedia’s information gaps by interrogating what constitutes a “reliable source” on English, French and Spanish Wikipedias, and how source authority is negotiated among editors involved in these communities. The project has three parts: community conversations, qualitative research, and a written report with recommendations for a more equitable Wikipedia.
WP:RS: “Reliable source” guidelines and marginalized communities
Wikipedia, a tertiary reference, is built on citations of secondary sources. The credibility of the crowdsourced online free encyclopedia has been established through an ad hoc, but rigorous, editorial process, which includes peer-reviews of articles. The best are acknowledged with a gold star, which marks “Featured Article” (FA) status. To achieve FA status, articles must meet editorial quality standards which are guided by core content policies of “verifiability” (WP:V), “neutral point of view” (WP:NPOV) and “no original research” (WP:NOR).
In the effort to expand representation, the “reliable source” guideline is a double-edged concept. Prestigious institutions and custodians of research and knowledge — including academic libraries, textbook publishers, categorization and indexing standards — have been shaped historically by legacies of injustice and erasure.⁷ Their authority is contextual and constructed. Consequent biases in secondary literature can take the form misrepresentation or absence, adversely affecting marginalized communities due to the limits of existing sources, and access to them.
Why research “reliable sources” and the editorial processes?
In addition, our experience shows that new editors contributing content to articles about marginalized communities have had their contributions contested or challenged on the grounds of “unreliability,” particularly due to the active use of New Page Patrol (WP:NPP) and the flagging of edits by users who are not auto-confirmed (which is a minimum of ten edits over four days); these software flag new articles and contributions for patrol editors for review. This added level of surveillance early in the editing process calls into question contributions and their sources, an additional stress to editing and questions the way that nuanced knowledge about subjects is best gathered from the communities themselves. Wikipedia policy on source verifiability (WP:V) requires sources are independent from their subjects. Conversations about source verifiability often take place in discussions of whether or not a subject meets the notability guideline (WP:N; WP:GNG), which is a “test” that Wikipedia editors use to determine if a subject has been “significantly” covered by reliable sources and is eligible for a new article. While our project will focus on the question of evaluating “reliable sources” in general, our research will include discussions about sources used to create and justify biographies, which can be challenged on the basis of whether or not the sources meet the General Notability Guideline “test,” specifically the additional criteria of number of reliable sources in independent publications over time (“significant”).
The project team is sympathetic to the motivations of Wikipedia editors to actively flag or cross-check sources in a historical moment when disinformation is pervasive across the internet ecosystem.⁸ But, challenges by editors about sources can deter and silence even seasoned editors. Responding requires significant behind-the-scenes “emotional labor” ⁹ and tacit knowledge of the editorial processes, including editing using wiki-markup, of Wikipedia in order to justify the legitimacy of sources.
We pause here to challenge when and how these negotiations happen, and their effects. Other research on Wikipedia content has found evidence of content bias that cannot be attributed to existing prejudices in secondary literature, but is the outcome of biases in the editing process.¹⁰ Indeed, the 2018 Gender Equity Report also identifies editorial rigidity as a significant barrier to achieving the Wikipedia project’s vision and mission of providing a summation of human knowledge.¹¹ Our research project will summarize these efforts and provide additional case studies and analysis of the ways the process of editing interprets reliability in three languages.
Thus, Reading Together furthers the efforts of Art+Feminism in challenging who and what can and should be included on Wikipedia. Our work is committed to championing the broader efforts of Art+Feminism, an intersectional feminist non-profit organization, to directly address information gaps about gender, feminism, and the arts on the internet. Since 2004, Art+Feminism has been centered on building an equitable global community striving to close the gaps in content and editorial representation on Wikipedia, ensuring that the histories are accessible and accurate.
In the next six months, you can find us finalizing our research criteria, gathering materials, engaging with the Wikimedia community, performing analysis, and sharing our findings, and future recommendations, in a written report.
We want to listen and learn with others in the Wikimedia community.
We will host three Town Halls on the topic of reading reliable sources and marginalized communities in the fall of 2020. These Town Halls will be hosted in French, English and Spanish.
Sign up here to get more information on how to register to participate in a Town Hall.
Bookmark this page to check back for future updates.
The WikiCred Grants Initiative supports research, software projects and Wikimedia events that explore information reliability and credibility. WikiCred’s funding is provided by Craig Newmark Philanthropies, Facebook, and Microsoft.
About the project team
Amber Berson is a writer, curator, and Ph.D. candidate conducting doctoral research at Queen’s University on artist-run culture and feminist, utopian thinking. She most recently curated Utopia as Method (2018); World Cup! (2018); The Let Down Reflex (2016–2018, with Juliana Driever); TrailMix (2014, with Eliane Ellbogen); *~._.:*JENNIFER X JENNIFER*:.~ (2013, with Eliane Ellbogen); The Annual Art Administrator’s Relay Race (2013, with Nicole Burisch); The Wild Bush Residency (2012–14); and was the 2016 curator-in-residence as part of the France-Quebec Cross-Residencies at Astérides in Marseille, France. She is a co-lead at Art+Feminism, a project that works for a more equitable Wikipedia and was the 2019–2020 Wikipedian in Residence at Concordia University. She speaks English and French.
Monika S. Jones, PhD, is a writer, researcher, speaker, and educator with expertise in cultures and technologies. She has been involved with English Wikimedia since 2012 leading numerous initiatives involving equity, feminism, arts, and information literacy, including The Gap Finding Project with Wiki Education and Wikipedia+Libraries: Better Together as the OCLC Wikipedian-in-Residence from 2017–2018. Monika has an MA degree in Gender Studies from Central European University and PhD in Communication and Science Studies from UC San Diego. She edited at Catalyst Journal and adjunct lectured in the Department of Communication at University of Washington in Seattle, WA, U.S. Learn more at www.monikasjones.com.
Melissa Tamani is currently pursuing an MA in Administration of Organizations in the Cultural and Creative Sector at the University of Buenos Aires, and has a BA in Art History from the National University of San Marcos, Lima. She’s developed a career in the fields of cultural management and museum education at various cultural institutions in Peru. In 2017 she was the Cultural Manager at the Faculty of Art and Design from the Pontificia Universidad Católica del Perú. She’s part of Art+Feminism since 2015, formerly as Regional Ambassador in Latin America and currently as Lead co-organizer. She is also a promoter of Wikimedia culture in Peru, organizing and taking part in campaigns focused on gender, human rights, and ecology. She speaks Spanish and English.
Sue Barnum is a public services librarian at El Paso Public Library in Texas, United States, on the border of Ciudad Juarez, Mexico. She has a decade of experience in libraries and is an administrator on English Wikipedia where she also volunteers as Librarian in Residence for WikiProject Women in Red,supporting editors in finding reliable sources using library tools to expand coverage of notable women. In 2018, Barnum was chosen as a U.S. Library Journal Mover & Shaker — a special award for changemakers in the library field — for her contributions to Wikipedia. Barnum edits to ensure overlooked subjects, such as the work of Chicana artists, women in history, and the history of borderlands, are represented in Wikipedia. Barnum also speaks un poco Spanish.
Mariana Fossatti is a feminist and a free culture activist. Her background is in sociology and she has a master’s degree in Society and Development from the Universidad de la República, Uruguay. She co-founded the digital cultural centre Ártica in 2011, and co-founded the Uruguayan chapters of Creative Commons and Wikimedia in 2013. Since 2018 she coordinates the online campaign #VisibleWikiWomen with the Whose Knowledge? initiative. She also works in the APC Women Rights Programme, amplifying women’s voices in tech on GenderIT blog, and facilitating feminist ways of knowing in the Feminist Internet Research Network.
Camille Larivée is an independent curator, street artist, writer, and cultural worker based in Tiohtià:ke/Mooniyaang (Montréal, QC). Their practice revolves around the invisibility of BIPOC and 2SLGTQQIA artists and muralists in urban public spaces. Camille holds a Bachelor’s degree in Art History and a certificate in Feminist Studies from the University of Quebec in Montreal (UQAM). They are the Director of Programming for the Aboriginal Curatorial Collective / Collectif des Commissaires Autochtones (ACC/CCA).
Walaa Abdel Manaem is an Assistant Lecturer in the Spanish Department at Cairo University; as well as a Wikipedia Admin (1st female admin on Arabic Wikipedia) and editor on Arabic Wikipedia; she is the co-founder of Egypt Wikimedians User Group as well as an active member of Wikipedia Education Collaborative (Collab), known now as Wikipedia & Education User Group, as one of the few program leaders who mentor other programs around the world. She is part of the Movement Strategy Working Group for Diversity and is a Regional Ambassador with Art+Feminism.
Ha-Loan Phan, M.Sc. (Biology), is a project coordinator at University of Montreal (Bureau de valorisation de la langue française et de la Francophonie). She previously worked in communication and event planning for a century-old scientific organization based in Quebec that advocates research in French. She has participated in Wikimedia projects as an editor of Wikipedia since 2011 (she also hosted several editing workshops) and as a board director of Wikimedia Canada since 2016. An enthusiastic lifelong learner, she graduated as Administratrice de sociétés certifiée (equivalent Chartered Director) in Summer 2020. Mission: ensure the future of free knowledge and free culture.
Notes and References
1. See also: “Gender Bias on Wikipedia” https://en.wikipedia.org/wiki/Gender_bias_on_Wikipedia
2. Wikimedia Foundation. (2011, April). “Wikipedia editors study results from the editor survey, April 2011.” Retrieved from https://upload.wikimedia.org/wikipedia/commons/7/76/Editor_Survey_Report_-_April_2011.pdf
3. Hill, Benjamin Mako, & Shaw, Aaron. (2013). The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation. PloS one, 8(6), e65782. https://doi.org/10.1371/journal.pone.0065782
4. Wagner, C., Garcia, D., Jadidi, M., & Strohmaier, M. (2015). It’s a man’s Wikipedia? Assessing gender inequality in an online encyclopedia. arXiv preprint arXiv:1501.06307.
5. Regarding gender bias in French-language Wikipedia see: Petillon, Catherine. (2017, August 30). “Wikipédia : où sont les femmes ?” Retrieved from https://www.franceculture.fr/emissions/le-numerique-et-nous/wikipedia and Protais, Marine. (2019, June 17) “Pourquoi Wikipedia est un enfer pour les femmes ?” Retrieved from https://www.ladn.eu/tech-a-suivre/pourquoi-wikipedia-est-un-enfer-pour-les-femmes/
6. According to the data set offered by the project Wikidata Human Gender Indicators (WHGI), the female percentage of biographies in Spanish Wikipedia is 21.37%.
7. Christen, Kim. (2015). “Tribal Archives, Traditional Knowledge, and Local Contexts: Why the ‘S’ Matters.” Journal of Western Archives 6 (1): 1–21.
8. Wardle, Claire. “Disinformation disorder” Data Journalism. Last accessed at https://datajournalism.com/read/handbook/verification-3/investigating-disinformation-and-media-manipulation/the-age-of-information-disorder
9. Menking, Amanda, and Ingrid Erickson. (2015). “The Heart Work of Wikipedia: Gendered, Emotional Labor in the World’s Largest Online Encyclopedia.” In 33rd Annual ACM Conference on Human Factors in Computing Systems Proceedings.
10. Graells-Garrido, E., Lalmas, M., & Menczer, F. (2015). “First Women, Second Sex: Gender Bias in Wikipedia” in ACM Press. HT ’15: Proceedings of the 26th ACM Conference on Hypertext & Social Media. 165–174.
11. Gender Equity Report, 2018. Wikimedia Foundation. Last accessed at: https://meta.wikimedia.org/wiki/Gender_equity_report_2018