OASIS Conference Part II

The OASIS network was set up to foster connections between those thinking about the primitives of meaning composition across disciplines. From my own perspective, it means linguistic semantics reaching outwards to learn from scientists thinking about meaning from  psychology and neuroscience, from computer science and lexicography, and from philosophy.  It also involves trying to think about the problem in new and innovative ways and finding ways to close the commensurability gap between our formal semantic computational theories of meaning and the algorithmic and implementational realities of mind and brain.  In this second report from the kickoff network conference, I concentrate on panels and talks that were not given from the primary perspective of either formal syntax or semantics (there were great talks of that flavour  too at OASIS, but a psychology blogger should talk about them).

There were two invited talks from Alastair Knott  (University of Otago, New Zealand) and Friederike Moltmann  (CNRS) who spoke from the lands of neuroscience/psychology and philosophy respectively.  What are these strange lands, and do they speak a language we can understand? Knott is an expert on embodied cognition and the deconstructing of the logic of action, as in what it takes for a person to reach out and grab a cup.  His idea was that the deictic routines necessary for executing a simple motor event such as `grabbing a cup´ are of the same granularity, time wise and  with respect to the primitives involved, as the symbolic elements themselves required to represent an event of `grabbing a cup´ in  linguistic terms.

So here was an explicit attempt to close the commensurability gap (cf.  Embick and Poeppel (2015))  by essentially denying it—– the basic building blocks of the one can be directly identified with the building blocks of the other.  The position of these embodied cognition guys is that you get to move up a grain size from neuronal firings to motor plans, and in doing so you get to a level where the two types of language actually do translate each other.   It was a fascinating talk, and a brave attempt, but I simply wasn´t buying it.   Still, exactly the kind of thing that I come to OASIS conferences to hear. Check out his book here (http://mitpress.mit.edu/books/sensorimotor-cognition-and-natural-language-syntax)

Moltmann was a completely different kettle of fish. She comes from a heretical position both from the point of view of philosophical metaphysics and  from the point of view of standard formal semantics.  Classical formal semantics invites us to take truth values and objects in the world as the basic ontological types— even subsorts within the type e category need to be justified in terms of extensional identity conditions and independently justifiable real world distinctions.  Philosophical metaphysical ontology is similarly concerned with discovering what actually exists  in the world.  Moltmann outlined what she argued was an emerging field of natural language ontology, whereby what we should be engaged in is a kind of descriptive metaphysics. In other other words, in the particular case of language, we as scientists of meaning should be interested in discovering the ontological primitives that are required for a successful description of the natural language system of meaning construction, i.e. natural language ontology (http://friederike-moltmann.com/uploads/Natural%20Language%20Ontology-2016(3).pdf). What we should seek to discover are  the ingredients that are necessary and implicitly relied on in natural language systems, rather than those required for cognition in general, or those that underpin some objective external reality (see also the work of the philosopher Kit Fine).  In laying out this programme, Moltmann I believe is exactly correct, although of course many of the actual details remain to be worked out.  Her talk represented an attempt to clarify the terms, define the object of inquiry, and set the agenda for the whole enterprise in philosophical terms. In doing so, I think she, and Fine, myself and a growing number of like minded people are still definitively in the minority even though I believe this is the only way forward to genuine communication with psychology.

The panel on the acquisition of counterfactuality was an example of work where OASIS as a network is actively experimenting with bringing psychologists, psycholinguists and theoretical linguists together to help sharpen the questions and issues relevant to all.  Counterfactual reasoning, and language expressing counterfactual reasoning statements have long been fascinating for formal semanticists raising a number of interesting crosslinguistic  issues concerning the elements required to encode such ideas, and questions concerning the interaction of structural semantics and pragmatic reasoning.  The panel started with a clear and pointed theoretical overview from Fabienne Martin (Humboldt, Berlin), and was followed by more experimental talks from Sarah Beck (University of Birmingham) and Nina Kazanina (University of Bristol).

Beck points out that very small children engage in pretending games, pretending to be cats or princesses or dinosaurs. They are fully aware that they are not in fact dinosaurs, but does this entail that they are actually engaging in some form of counterfactual thinking?  Are there simpler versions of counterfactual thinking in which there could be a basic easy version and then a scaled-up adult version?

In Theory of Mind tasks, children start to give correct answers between ages 3 and 4. But recent work suggests that they behave correctly much earlier, as long as the understanding of what someone else might be thinking is not mediated by an explicitly verbal task.

Is this evidence that what is hard is language?  Or that the kids have a baby version that is not as complex as the one that is required by language?  Or maybe even that language allows kids to scale up their instinctive cognitive awareness of the reality of other minds to an automated sophisticated reasoning process using that information?  How could we know , and how would we test that?

Interestingly, in tests conducted by Beck and her team 3-4 year olds found it significantly more difficult to perform at adult levels when the question was `What if he had gone the other way,  where would he be now?´, as opposed to `What if next time he goes the other way, where would he be?´ Quite generally, kids found future hypotheticals easier than past counterfactual reasoning raising the possibility that what they find difficult is reasoning with an alternative to something they know for sure is a fact, as a opposed to just reasoning about two different live options.  Beck speculates, following Hoerl and McCormack, that children only get good at tracking events in a mature temporal updating system at about age 5. Maybe it is this ability that is necessary for the more complicated counterfactual reasoning question.

So, what´s the problem? Is it temporal tracking? Is it contradicting a past known event? OR, is it the linguistic complexity of the counterfactual reading that is getting in the way?

Nina Kazanina from the University of Bristol had been pushing hard on this very question.  If you compare the two conditionals in (a) and (b), it is clear that there is something quite linguistically unusual about (b) from the point of view of English.

  • If Amy eats an apple, she will get a medal.
  • If Amy had eaten an apple, she would have won the medal.

Kazanina set up an experiment with children involving puppets and different things to eat, and medals being awarded and tested them in various conditions. She first tested children on whether they had good control of the if — then  construction in English, and only those who performed close to ceiling on that pre-test were further tested on the counterfactual conditionals.  The scenario is the following: one animal eats the watermelon and gets a medal; the other animal eats the ice cream and gets a cross. A puppet then makes a statement like  `If he had eaten a watermelon, he would have got a medal´  and the kid has to say which animal the puppet is referring to.

The finding is that kids  get it right at age 4 but not at age 3. Further, this failure does not  correlate with performance on the false belief (theory of mind ) task administered at the same time.  Kazanina concludes that the problem is with the linguistic construction and not with the ability to entertain the possibility of false situations.

But is the problem with (b) a kind of hidden `not´ in the inferential process which is not transparently indicated in the linguistic form, as Kazanina claims (since (b)   entails that Amy did not eat the apple and in fact did not win the medal)?

Or is the problem with time tracking as Hoerl and McCormack suggest?

Or is the problem with the interpretation of the past tense in if-clauses, which is not interpreted like normal past tense in English but more like `remote world´  ? Come to think of it, that IS pretty weird.

Really interesting questions. Really interesting panel. I hope they will keep me posted.

Looking forward to the next OASIS conference in Nantes next year, and thanks to Bridget Copley and Isabelle Roy for organising this one.

Categorization and its relation to Language: Evidence from the non-verbal mind

I am writing this blog from Paris where we are having the official kickoff conference for the OASIS network —- Ontology as Structured by the Interfaces with Semantics—primarily funded by the CNRS (http://oasis.cnrs.fr).

For those of you who do not know what OASIS is about or why we feel we need a network, a few words of explanation are perhaps in order: OASIS network members are those who are involved in thinking in new and cross-disciplinarily commensurable ways about the primary cognitive ingredients of meaning. We believe that progress will only be made if we collaborate actively and open mindedly with psychologists, philosophers, computationalists, and lexicographers and also conversely that language is centrally important in this conversation too. In other words,  linguistic semantics  should  be centrally involved in the questions of meaning and ontology more generally. Linguistic semanticists have so far not been very good at communicating with other related disciplines, and it is not always clear how to bridge the commensurability gap between our own analytic primes and the analytic primes of others. Traditional conference going tends to support and reinforce in-group talk. The OASIS is a network that is intended to support and reinforce cross-group talk in this particular area.  The University of Tromsø (specifically, myself, Peter Svenonius, and Serge Minor) are steering committee members of a network that includes Paris 8, Nantes, Humboldt Berlin, Pompeu Fabra Barcelona and Queen Mary London.

Day 0 involved a satellite workshop on types, tokens, roots and functional structurewhich was truly excellent (http://oasis.cnrs.fr/sites/oasis.cnrs.fr/files/files/OASISUPFprogram.pdf),  but what I would like to report on here is Day 1 of the conference proper, where the first of series of panels was presented.  The fun begins!

The topic of Day 1´s panel was the Ontology of the non-verbal and pre-verbal mind, organized by Wolfram Hinzen(Pompeu Fabra).  Louise McNally(Pompeu Fabra) moderated the panel and posed what I thought was an important opening framing question:  What is the relation between reference and categorization? How does language mediate in that relation?  Language itself combines these two aspects in a way that might make it hard to pull apart. But McNally offered an useful practical exemplification of the dissociation:

Categorization without reference is emojis

Reference without categorization is pointing.

Hold that thought because it is going to be relevant in what follows.

Wolfram Hinzen  introduced the first talk by motivating why it is interesting and important to look at minds that are not verbal, either because they are incapable of it congenitally, or because they are at an immature stage of development.   Hinzen´s conviction is that human language and human thought are so bound up in each other that destroying the one destroys the fabric of the other.

One obvious source of non verbal minds are the minds of our non verbal primate cousins such as chimps. What sorts of things are they capable of when it comes to their own brand of gestural communication in the wild?  A very striking fact is that they have no comprehension of pointing.  Ape gestures are mostly imperative or attention getting, and are never descriptively referential. Apes undoubtedly have their own systems of categorization and can even express/communicate certain of these categories, but they do not use signs referentially in the same way. In other words, Apes can do emojis,  but not pointing.   (It strikes me incidentally that humans are not very good at predicting in advance what is going to be easy or difficult or impressive when found in another species. Most people would think that a sense of humour, or mourning one´s dead, are impressively complex feats whereas we are unaccountably surprised to find out that apes just don´t get the pointing thing. At all.)

Next  Hinzen and Domenika Slušná (Pompeu Fabra) reported  on a population of children/young adults diagnosed with autism who have never been able to acquire language. I was surprised to learn that about 25-30 percent of autistic children in fact never manage to develop language. These are  physically healthy people with normal audition, no gross motor impairment, and can produce single words. They comprehend little of speech however, especially out of routine contexts. But they do clearly socially interact. Their non verbal IQ was below average in most cases. In standard tests of categorisation, they were able to do some simple categories if they corresponded to very familiar objects in use for them, but these behaviours and representations did not seem to be very stable.  Slušná  presented her work on the analysis of the gestural repertoire and use in this population.  Their use of gesture, instead of being enriched in compensation for lack of language, turned out to be as limited and non-descriptive as the primate gestures described above. Their gestures are ritualized and used in imperative contexts primarily. Even pointing, which was a very trained and supported gesture, was used differently from neurotypical humans, without consistent referential use.

So these non verbal humans seem also to have limited categorization abilities but like the apes seemed to lack the referential instinct.  Emojis, but no pointing again. 

Next up was Alissa Ferry(U of Manchester) on The role of language in object categorisation in pre/verbal infants.  Here we zero in on the category formation ability of humans.  We know that categories are important for cognition, but we also know that the process of categorisation does not require language (pigeons Wasserman et al 2015;  non human primates Vogels 1999), and indeed we have also seen above that our non-verbal human population did retain some categorisation abilities that  Slušná  actually showed were not correlated with amount of word use or comprehension.  But it still might still be the case that language somehow facilitates categorisation in an important way, and language labels act as a trigger for the process of generalization and the formation of certain useful categories that are then robustly represented in the mind. Ferry in her PhD and subsequent follow up work was interested in tracking the moment at which prelinguistic infants start to categorise and whether this process is affected by the growth in their linguistic abilities.

The task involved a preferential looking paradigm. First the small infants were presented with a set of eight different dinosaur pictures one after the other in a training phase. Then in the target condition, they saw two pictures: another different dinosaur and a fish. If the infants had formed a category, then the dinosaur would be boring and the fish would be excitingly new.   In infants this leads to preferential looks.  If they had not formed a dinosaur category then there should be no real difference in looks in the target phase.  The training phase came in two flavours. In one version, the training phase came with each picture going along with a human saying something like “Wow!  Look at that!  That is a Toma!”  .  In the other version, the soundtrack was a series of computer generated tones.   This paradigm had been tested with 12 month olds and had been found to work like a charm— the infants formed a category with the human language commentary, but not when the soundtrack was computer generated tones.  (In fact, the 12 month olds formed a category when the single label ` toma´ was used for all the dinosaurs, but did  not  form a category when a different label was used for each different dinosaur. Cool—Language labeling has a very direct and demonstrable effect here!).   Ferry wanted to know how far back that would go time wise in the infants´ cognitive development.

She took it back to  6 and 3 months and the 3 month olds still seemed to form a category by this test,  in the verbal commentary paradigm and not to the computer tone soundtrack paradigm!

But wait, these kids aren´t even parsing out words yet presumably. Maybe it´s the fact of language itself that is tipping them off and making them think there is an interesting category to be paid attention to here, even if they are not isolating the individual word label that is being used. So Ferry tried it with Chinese, and while the 12 month and 6 month olds did not form a category with the Chinese soundtrack, the 3 month olds did!  She then tried it with lemur calls and the 3 month olds still formed the category! THEN she tried it with backwards speech (which sounds less weird than it is— it is apparently biologically quite impossible to generate), and the 3 month olds declined to form a category. Clever kids.

So 3 month olds are paying attention to the complex,naturalistic communicative sounds., but don´t really distinguish between lemurs, Chinese and their own language. 6 month and 12 month olds only form a category with their own language. But something interesting also happens between the  6 month mark and the 12 month mark. The original paradigm had the phrase used for each picture be exactly the same in the case of each dinosaur. Ferry wanted to know what would happen if each phrase was actually slightly different, grammatically, with the word `toma´ appearing in a different position in each case. At what point was the infant able to successfully notice that the `toma´word was the same, even though the whole chunk of language was not identical for each dinosaur.  Now recall that the 6 month olds are not fooled by Chinese, but it turns out that if you vary the form of the linguistic stimulus in their own language so that it is a bit harder to pull out the common word `toma´, then six month olds in fact  fail to form a dinosaur category. The 12 month olds are not thrown off by this, and they continue to form the dinosaur (`toma´) category. In fact, even by 9 months they pulling out and  tracking the individual label within the speech stream.

So prelinguistic kids can form categories in the absence of a linguistic label, and the presence of one linguistic label vs many different ones does push the child into a particular categorisation decision, but interestingly, the very presence of communicative noises seems to trigger category forming impulses in the small human infant.

Mohinish Shukla(UMass Boston) wrapped up the panel with a talk which also addressed the question about whether there can be concepts without language: Event generalisation across visually different scenes using eye-tracking, across different populations

 While we seem to have a consensus that concepts and categories are possible without language, maybe there are certain kinds  of categorizations that are only possible with the help of language. While 2 year old infants could form implicit categories based on simple transitivity contrasts (difference in crude number of participants), it turned out that they failed to recognise implicit categories of event based on the reversal of certain thematic role to participant mappings.  In a preferential looking paradigm it was checked to see if kids of 24 months could notice and start to predict a generalization based on whether `the dog pushed the car´, or ´the car pushed the dog´ in a video animation.    While adults managed this no problem, the children at this age failed.  What went wrong?  Was the category too complex? Did that category require too much language sophistication to form, and was not independently cognitively natural enough? One of the issues that struck the linguist audience about this particular implicit category was that it was not actually one that usually forms the basis of distinct verbal labeling.  Maybe language learning was actually inhibiting the child´s ability to generalize here. Maybe the effort of learning actual verbs and attending to events in a certain way was biasing children away from seeing the generalization offered by the scenario. The generalization constructed simply would never conform to a verbal regularity in English and choice of agent was not a parameter of variation being attended to for the purposes of category formation.  So although this was not the conclusion of Shukla, one might speculate that the influence of language labels and language learning is actually having an effect on categorisation here, this time an inhibitory one.

Well that´s enough for one blog post. Tomorrow I will post about the other talks and panels at the OASIS conference.