March 26, 2020
As a boy Neil Cohn loved comics, and he turned that love into an original and respected scientific domain influenced by linguistics and integrating drawing and language into a common understanding.
“I study people’s minds and brains and how they understand visual information,” Cohn says from his home in the Netherlands. He studies how people process visual information, especially drawings used in comics and emoji, and he compares drawings and graphics with the ways people comprehend and read words and sentences.
Nominated by the Cognitive Science Society for an Early Career Impact Award, Cohn focuses on comics from around the world and said he hopes his research ultimately will enhance how we teach and learn language. Pioneering work already has shown students with autism who are frequent readers of comics comprehend meaning in those strips to the same degree as students without autism.
A native Californian, Cohn said he crossed the Atlantic because he found “the perfect fit” at Tilburg University, where he is assistant professor of cognition and communication and director and founder of the school’s Visual Language Lab.
Cohn conducts experiments using about 10,000 panels from comic strips – primarily from “Peanuts” by Charles Schulz. If the strips have words, he takes those out, and he often rearranges the panels for a preferred structure. For example, to look at how people infer meaning, he omits climactic moments from the narrative or replaces those moments with sound effects, like “Pow!”
Study participants, mostly undergraduates who self-identify as comic readers, wear electrode caps and are given comics to read. The measure of electrical activity tells the researchers how fast participants’ brains are processing the strips and whether they are processing meaning or grammar, the structure that organizes the sequence of pictures.
Cohn compares his results with published results from similar studies of word and sentence comprehension. He finds people’s brains respond similar whether they are reading sentences or comic strips. This suggests understanding sentences and sequences of pictures tap into shared resources in the brain.
Cohn is launching his largest project to date, made possible with a 1.5 million Euro grant from European Research Council, to study differences in the properties of 1,500 comic books worldwide, including the relationships between the words in the comics and the images.
He hopes to learn whether comics from one culture to the next differ the same way languages differ from each other. He also wants to know whether comics within a continent differ from country to country and whether language patterns in the comic strips are similar to spoken and written language patterns.
Cohn describes his research as both theoretical and experimental. Among findings: pictures are not secondary to letters and words. “We typically parcel them off—drawing is this and language is this,” he says, “when in reality they are two parts of one system. You get more effective communication when combining images and text together.”
In addition to studying comics, Cohn still writes them, with his latest academic book coming out this year. Down the road he plans to write a graphic novel about his research.
