On where Toronto is

Sometimes enough people misunderstand what you say that you begin to wonder how you could have said it more clearly. This is one of those times.

In the NYT piece on Watson I talked about Watson’s famous Toronto flub. In response to the clue “Its largest airport is named for a World War II hero; its second largest for a World War II battle,” given in the category “U.S. Cities,” Watson answered “Toronto.” It turns out, Toronto is not a U.S. City. (Well, at least the one that Watson picked out is not a U.S. City.) “This is the kind of blunder that practically no human being would make,” I wrote.

I can’t tell you how many putative refutations of this argument I’ve read that have begun with paeans to the stupidity of the average American. Jay Leno has proven, these comments usually begin, that most Americans don’t know diddly squat. Probably most of them don’t know that Toronto is not a U.S. City. Ergo, the fact that Watson didn’t know Toronto is in Canada doesn’t show that it’s not thinking at all.

The blunder people took me to be attributing to Watson, in other words, was the blunder of thinking that Toronto is a U.S. City. But that’s not what I was saying at all. Let me be perfectly clear. The explanation of the blunder offered by the IBM team does not go by way of suggesting that Watson had some good evidence for the claim that Toronto is a U.S. City. Watson, with its equivalent of one million books of information about the world, has plenty of evidence that Toronto is in Canada. (And evidence, too, that Canada is not part of the United States.) Its mistake was not to claim that Toronto is in the U.S., but rather to downplay the importance of that fact in choosing its answer. In particular, it downplayed the mismatch between the category (“U.S. Cities”) and the answer type (a non-U.S. city). This is the mistake I thought it would be hard for a person (who understands Jeopardy) to make. That is, the mistake of knowing that Toronto is not a U.S. City, knowing that the category is U.S. Cities, knowing that Chicago (its second guess) is a U.S. City, and nevertheless choosing Toronto. The idea is that even if it is sometimes inappropriate to worry about a mismatch between category and answer type, any human being familiar with the game would recognize that this is not one of those times. They would recognize, in other words, that the fact that there is sometimes a mismatch between category and answer type is simply not relevant in this situation.

People make howlers too, of course. But this is not the kind of howler people make. Instead, this is the kind of howler that betrays, as Ken Jennings said, that Watson is simply doing something different from us. That was the point. I hope it’s clearer now.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

14 Responses to On where Toronto is

  1. genevieve says:

    Sean,

    Your point was already clear, at least to me, in your opinion piece. The machine downplayed the fact that Toronto is a Canadian city, over the fact that it does have two airports! (I checked on Wikipedia, right after the game. ) “Midway”, “O’Hare” and “Chicago” popped into my mind as soon as Trebeck finished reading the clue!

    Genevieve

    • dmf says:

      did you then flashback to being stuck in O’Hare, the smell of cleaning products, cinnamon buns, and mounting desperation? I’m guessing Watson wasn’t re-minded of how remarkably, almost uncannily, clean the airport in Toronto was or the quality of light that permeates the arctic air on a clear day in that city.

  2. terenceblake says:

    Andrew Pickering in “Cybernetics as Nomad Science”, http://minorliteraturesinsciencestudies.wikispaces.com/file/view/Pickering+cybernetics.pdf ,
    seems to have some convergent ideas. He contrasts a pluralist open ontology based on embodiment and becoming with a dualist ontology based on the cartesian ego representing and dominating the world. He finds exemplars of these ontologies in cybernetics (nomad science) and AI (royal science) respectively.
    In another article, lifeboat.com/board/andrew.pickering.doc , discussing Heidegger’s “The Question Concerning Technology” he associates cybernetics with an approach based on “revealing” and AI with “enframing”. Pickering argues that the dominance of “asymmetric dualism” is only hegemonic and not absolute, all sorts of other practices exist on the margins. He hopes that these marginal practices can assemble and escape from enframing.
    In his article “New Ontologies”
    https://eric.exeter.ac.uk/repository/bitstream/handle/10036/29882/newontols1006.pdf?sequence=1,
    his examples are not football (sorry, Sean!) but art (de Kooning vs Mondrian), civil engineering (adaptive ecological management vs large-scale projects of control), and psychiatry (anti-psychiatric endeavours like Laing’s Kingsley Hall vs asymmetrical hierarchical relations in a mental hospital, inner voyage vs blitz therapy), but he also speaks of mathematics, music, and architecture.

    • dmf says:

      Andy is a leading light in many ways (and should be required reading) but not so good on institutional life/organizational ethics and little to nothing to say on participatory democracy or macroeconomics, and I don’t think that this is just matter of ‘scaling-up’ fringe practices unless we are talking Dewey like primary education. That reminds me that I still have to read Latour’s Making Things Public.

  3. terenceblake says:

    Pickering talks about machinic contingency in a very interesting article called “Science, Contingency and Ontology” (https://eric.exeter.ac.uk/repository/bitstream/handle/10036/81575/lesT-240609.pdf?sequence=1). He contrasts a sort of static relativist pluralism of understandings of the world, which he calls a crystal ontology, from a more dynamical interactive pluralism based on a “machinic” ontology. The idea is that machines and instruments (in interaction with us) latch onto the world and elicit it in various ways.
    In this article Pickering does make a few scattered remarks about what he thinks of current educational practices from an ontological point of view. As for management practices, he elsewhere discusses Stafford Beer’s work as a partial example of the tendencies he wishes to encourage. But you can’t do everything, so I think Manuel Delanda has a similar ontology and so can be used to supplement Pickering’s ideas in the domain of the economy, for example here: http://technoccult.net/archives/2006/04/27/delandas-markets-and-anti-markets-series/

  4. dmf says:

    sean and t.b., see what you think of this post-computationalist/connectionist assemblage of Deleuze, Wexler, and Alva Noe:
    http://www.protevi.com/john/Deleuze_Wexler.pdf

  5. Enoch Lambert says:

    Consider the following story I read somewhere on the web:

    A girl tells everyone that she’s “so-and-so’s (her father’s) daughter”. Her mother tells her: “Don’t say that you are so-and-so’s daughter. Say you are N.N. (her name)”. Later, when asked by a family friend, “aren’t you so-and-so’s daughter?”, she answers: “I thought so. But my mommy says I’m not”.

    I’m inclined to think that errors of this sort, as well as others that are the subject of shows like “Kids say the darndest things” and countless parent blogs ARE errors of relevance. How similar are they to Watson’s Toronto error? I’m not sure, but I think they seem close enough to suspect that we need a more detailed story about what makes us different from Watson. Humans LEARN the ability to judge relevance. So far I’m not sure why we couldn’t expect a future Watson-like machine to LEARN appropriate relevance judgments as well. To be sure, I am skeptical, but I would also have been skeptical that we could build a Watson-like machine that did as well at judging relevance as it did. In any case, in light of the fact that humans can and do make errors in the same ballpark as Watson’s Toronto error at certain stages of their life-cycle, I’m not yet satisfied with the proposal for what distinguishes us from Watson. This is not to say one cannot be given–I think there must be one. And perhaps something like ‘relevance’ has something to do with it. Still, I’m not fully convinced by the Toronto example or the inference Jennings draws from it.

    • dmf says:

      Watson would have to be able to struggle with (and fail at) meaning what he says, to negotiate (have a feeling for/through) how to come to meanings (ways forward) with other people/environs, and to live/act with the ambiguity/polyvocity, get beyond mere reference (we are not walking encyclopedias).

  6. Victor says:

    In this context of ‘misunderstandings’ and after re-reading Susan Neiman’s take on “All Things Shining” in the NYT book review, I’d like to propose an experiment: would this story (see <a ref="http://www.youtube.com/watch?v=Dmop7EAY1Zg&feature=youtube_gdata_player&quot; title="link") of someone who decided to find and return a lost film to its owner – an act much more deliberate/contemplated than jumping onto rail tracks to safe a life – perhaps serve as an example why Susan Neiman missed the point of ATS in the sense that it is all about recognizing and seeking the ever existing connectedness; that ATS is a call to adventure which doesn't have to be Homeric in scope nor life-threatening and which can come about in the most banal of forms, such as stumbling upon a simple lost can of film and decide to act on it….

  7. Victor says:

    Terribly sorry! Please delete this HTML mess – I don’t know what went wrong

  8. dmf says:

    Sean, have you read Critchley’s reading of Being&Time ‘Originary Inauthenticity’ where he says of Mitsein/eigentliche Verbundenheit “Such alliance might well said to be the camaraderie that induces the political virtue of solidarity, but it is not a face-to-face relation, and as such, in my view is ethically impoverished. I sometimes think that authentic Mitsein is a little like being in church, it is a congregational “being-together-with-others” where we must vibrate together as one body in song and prayer. Pleasant as it doubtless must be, such is not the only way of being with others.” ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s