Categorization and its relation to Language: Evidence from the non-verbal mind

I am writing this blog from Paris where we are having the official kickoff conference for the OASIS network —- Ontology as Structured by the Interfaces with Semantics—primarily funded by the CNRS (

For those of you who do not know what OASIS is about or why we feel we need a network, a few words of explanation are perhaps in order: OASIS network members are those who are involved in thinking in new and cross-disciplinarily commensurable ways about the primary cognitive ingredients of meaning. We believe that progress will only be made if we collaborate actively and open mindedly with psychologists, philosophers, computationalists, and lexicographers and also conversely that language is centrally important in this conversation too. In other words,  linguistic semantics  should  be centrally involved in the questions of meaning and ontology more generally. Linguistic semanticists have so far not been very good at communicating with other related disciplines, and it is not always clear how to bridge the commensurability gap between our own analytic primes and the analytic primes of others. Traditional conference going tends to support and reinforce in-group talk. The OASIS is a network that is intended to support and reinforce cross-group talk in this particular area.  The University of Tromsø (specifically, myself, Peter Svenonius, and Serge Minor) are steering committee members of a network that includes Paris 8, Nantes, Humboldt Berlin, Pompeu Fabra Barcelona and Queen Mary London.

Day 0 involved a satellite workshop on types, tokens, roots and functional structurewhich was truly excellent (,  but what I would like to report on here is Day 1 of the conference proper, where the first of series of panels was presented.  The fun begins!

The topic of Day 1´s panel was the Ontology of the non-verbal and pre-verbal mind, organized by Wolfram Hinzen(Pompeu Fabra).  Louise McNally(Pompeu Fabra) moderated the panel and posed what I thought was an important opening framing question:  What is the relation between reference and categorization? How does language mediate in that relation?  Language itself combines these two aspects in a way that might make it hard to pull apart. But McNally offered an useful practical exemplification of the dissociation:

Categorization without reference is emojis

Reference without categorization is pointing.

Hold that thought because it is going to be relevant in what follows.

Wolfram Hinzen  introduced the first talk by motivating why it is interesting and important to look at minds that are not verbal, either because they are incapable of it congenitally, or because they are at an immature stage of development.   Hinzen´s conviction is that human language and human thought are so bound up in each other that destroying the one destroys the fabric of the other.

One obvious source of non verbal minds are the minds of our non verbal primate cousins such as chimps. What sorts of things are they capable of when it comes to their own brand of gestural communication in the wild?  A very striking fact is that they have no comprehension of pointing.  Ape gestures are mostly imperative or attention getting, and are never descriptively referential. Apes undoubtedly have their own systems of categorization and can even express/communicate certain of these categories, but they do not use signs referentially in the same way. In other words, Apes can do emojis,  but not pointing.   (It strikes me incidentally that humans are not very good at predicting in advance what is going to be easy or difficult or impressive when found in another species. Most people would think that a sense of humour, or mourning one´s dead, are impressively complex feats whereas we are unaccountably surprised to find out that apes just don´t get the pointing thing. At all.)

Next  Hinzen and Domenika Slušná (Pompeu Fabra) reported  on a population of children/young adults diagnosed with autism who have never been able to acquire language. I was surprised to learn that about 25-30 percent of autistic children in fact never manage to develop language. These are  physically healthy people with normal audition, no gross motor impairment, and can produce single words. They comprehend little of speech however, especially out of routine contexts. But they do clearly socially interact. Their non verbal IQ was below average in most cases. In standard tests of categorisation, they were able to do some simple categories if they corresponded to very familiar objects in use for them, but these behaviours and representations did not seem to be very stable.  Slušná  presented her work on the analysis of the gestural repertoire and use in this population.  Their use of gesture, instead of being enriched in compensation for lack of language, turned out to be as limited and non-descriptive as the primate gestures described above. Their gestures are ritualized and used in imperative contexts primarily. Even pointing, which was a very trained and supported gesture, was used differently from neurotypical humans, without consistent referential use.

So these non verbal humans seem also to have limited categorization abilities but like the apes seemed to lack the referential instinct.  Emojis, but no pointing again. 

Next up was Alissa Ferry(U of Manchester) on The role of language in object categorisation in pre/verbal infants.  Here we zero in on the category formation ability of humans.  We know that categories are important for cognition, but we also know that the process of categorisation does not require language (pigeons Wasserman et al 2015;  non human primates Vogels 1999), and indeed we have also seen above that our non-verbal human population did retain some categorisation abilities that  Slušná  actually showed were not correlated with amount of word use or comprehension.  But it still might still be the case that language somehow facilitates categorisation in an important way, and language labels act as a trigger for the process of generalization and the formation of certain useful categories that are then robustly represented in the mind. Ferry in her PhD and subsequent follow up work was interested in tracking the moment at which prelinguistic infants start to categorise and whether this process is affected by the growth in their linguistic abilities.

The task involved a preferential looking paradigm. First the small infants were presented with a set of eight different dinosaur pictures one after the other in a training phase. Then in the target condition, they saw two pictures: another different dinosaur and a fish. If the infants had formed a category, then the dinosaur would be boring and the fish would be excitingly new.   In infants this leads to preferential looks.  If they had not formed a dinosaur category then there should be no real difference in looks in the target phase.  The training phase came in two flavours. In one version, the training phase came with each picture going along with a human saying something like “Wow!  Look at that!  That is a Toma!”  .  In the other version, the soundtrack was a series of computer generated tones.   This paradigm had been tested with 12 month olds and had been found to work like a charm— the infants formed a category with the human language commentary, but not when the soundtrack was computer generated tones.  (In fact, the 12 month olds formed a category when the single label ` toma´ was used for all the dinosaurs, but did  not  form a category when a different label was used for each different dinosaur. Cool—Language labeling has a very direct and demonstrable effect here!).   Ferry wanted to know how far back that would go time wise in the infants´ cognitive development.

She took it back to  6 and 3 months and the 3 month olds still seemed to form a category by this test,  in the verbal commentary paradigm and not to the computer tone soundtrack paradigm!

But wait, these kids aren´t even parsing out words yet presumably. Maybe it´s the fact of language itself that is tipping them off and making them think there is an interesting category to be paid attention to here, even if they are not isolating the individual word label that is being used. So Ferry tried it with Chinese, and while the 12 month and 6 month olds did not form a category with the Chinese soundtrack, the 3 month olds did!  She then tried it with lemur calls and the 3 month olds still formed the category! THEN she tried it with backwards speech (which sounds less weird than it is— it is apparently biologically quite impossible to generate), and the 3 month olds declined to form a category. Clever kids.

So 3 month olds are paying attention to the complex,naturalistic communicative sounds., but don´t really distinguish between lemurs, Chinese and their own language. 6 month and 12 month olds only form a category with their own language. But something interesting also happens between the  6 month mark and the 12 month mark. The original paradigm had the phrase used for each picture be exactly the same in the case of each dinosaur. Ferry wanted to know what would happen if each phrase was actually slightly different, grammatically, with the word `toma´ appearing in a different position in each case. At what point was the infant able to successfully notice that the `toma´word was the same, even though the whole chunk of language was not identical for each dinosaur.  Now recall that the 6 month olds are not fooled by Chinese, but it turns out that if you vary the form of the linguistic stimulus in their own language so that it is a bit harder to pull out the common word `toma´, then six month olds in fact  fail to form a dinosaur category. The 12 month olds are not thrown off by this, and they continue to form the dinosaur (`toma´) category. In fact, even by 9 months they pulling out and  tracking the individual label within the speech stream.

So prelinguistic kids can form categories in the absence of a linguistic label, and the presence of one linguistic label vs many different ones does push the child into a particular categorisation decision, but interestingly, the very presence of communicative noises seems to trigger category forming impulses in the small human infant.

Mohinish Shukla(UMass Boston) wrapped up the panel with a talk which also addressed the question about whether there can be concepts without language: Event generalisation across visually different scenes using eye-tracking, across different populations

 While we seem to have a consensus that concepts and categories are possible without language, maybe there are certain kinds  of categorizations that are only possible with the help of language. While 2 year old infants could form implicit categories based on simple transitivity contrasts (difference in crude number of participants), it turned out that they failed to recognise implicit categories of event based on the reversal of certain thematic role to participant mappings.  In a preferential looking paradigm it was checked to see if kids of 24 months could notice and start to predict a generalization based on whether `the dog pushed the car´, or ´the car pushed the dog´ in a video animation.    While adults managed this no problem, the children at this age failed.  What went wrong?  Was the category too complex? Did that category require too much language sophistication to form, and was not independently cognitively natural enough? One of the issues that struck the linguist audience about this particular implicit category was that it was not actually one that usually forms the basis of distinct verbal labeling.  Maybe language learning was actually inhibiting the child´s ability to generalize here. Maybe the effort of learning actual verbs and attending to events in a certain way was biasing children away from seeing the generalization offered by the scenario. The generalization constructed simply would never conform to a verbal regularity in English and choice of agent was not a parameter of variation being attended to for the purposes of category formation.  So although this was not the conclusion of Shukla, one might speculate that the influence of language labels and language learning is actually having an effect on categorisation here, this time an inhibitory one.

Well that´s enough for one blog post. Tomorrow I will post about the other talks and panels at the OASIS conference.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s