A school district used ChatGPT to decide which “sexual” books to ban. It didn’t go well.

An Iowa school district is using artificial intelligence (AI) to decide which books to ban from its libraries because no one has time to actually read the books to decide if they have “descriptions or visual depictions of a sex act,” which are banned by a new state law.

The only problem: AI technology is not yet advanced enough to assess books’ content, and the district’s AI program provides inconsistent answers when asked which books to ban.

Related:

The gay man behind Chat GPT says AI could revolutionize the world but could also destroy it

He said his greatest concern is the ability of AI to provide “one-on-one interactive disinformation.”

To prepare for the school year, the Mason City Community School District, a rural area located 120 miles north of Des Moines, compiled several available lists of commonly challenged books to create a “master list” of books for review.

Stay connected to your community

Connect with the issues and events that impact your community at home and beyond by subscribing to our newsletter.

Daily

Weekly

Good News

The district did this to comply with a new law, signed in May by Iowa Gov. Kim Reynolds (R), which bans all books containing “descriptions or visual depictions of a sex act.” Before this law was signed, the district said it had no record from the last 20 years of any parent or student formally challenging a book to be removed because of its content.

Nonetheless, the district reviewed each book on its master list “using AI software to determine if it contains a depiction of a sex act,” a district spokesperson told The Gazette.

This process led the district to remove 19 books from its grades 7-12 libraries. The list included such literary classics as Margaret Atwood’s The Handmaid’s Tale, Alice Walker’s The Color Purple, and Maya Angelou’s I Know Why the Caged Bird Sings.

The district used ChatGPT, an AI chatbot, to ask which books on its master list depicted a sex act. But while ChatGPT can generate responses based on text it has learned from various sources, the chatbot doesn’t have the ability to evaluate the full texts of different novels to determine their sexual content. It also has a tendency to fabricate facts on its own.

In fact, when the publication PopSci entered the district’s list of the 19 banned books into ChatGPT and asked the chatbot, “Do any of the following books or book series contain explicit or sexual scenes?” the chatbot said that only four contained “Explicit or Sexual Content.” Another six supposedly contain “Mature Themes but not Necessary Explicit Content.”

“ChatGPT’s varying responses highlight troubling deficiencies of accuracy, analysis, and consistency,” PopSci wrote. “A repeat inquiry regarding The Kite Runner (one of the district’s 19 banned books), for example, gives contradictory answers. In one response, ChatGPT deems Khaled Hosseini’s novel to contain ‘little to no explicit sexual content.’ Upon a separate follow-up, the [chatbot] affirms the book ‘does contain a description of a sexual assault.’”

Bridgette Exman, the district’s Assistant Superintendent of Curriculum and Instruction, said, “Our classroom and school libraries have vast collections, consisting of texts purchased, donated, and found. It is simply not feasible to read every book and filter for these new requirements.”

“Frankly, we have more important things to do than spend a lot of time trying to figure out how to protect kids from books,” she added. “At the same time, we do have a legal and ethical obligation to comply with the law. Our goal here really is a defensible process.”

Similar book bans have been escalating nationwide, according to a report from the free-speech organization PEN America. Over the last school year, PEN America recorded 1,477 separate instances of book bans nationwide, affecting 874 unique titles in 182 school districts and 37 states. In 2022, about 41 percent of challenged books were by LGBTQ+ authors or contained LGBTQ+ themes, the American Library Association reported.

The bans have largely come from Republican politicians, conservative school boards, and so-called “parents’ rights” groups that have opposed such content as “woke indoctrination” that’s “inappropriate” for school children. In 98 percent of cases, the school districts didn’t follow First Amendment protections to ensure that government officials don’t ban or punish free speech, PEN America reported.