Why Brain Science Failed - or, The Mistake from Writing





This is Lecture Two.


Brain Science, how come? An introduction.
A provocative expression, of course, but...
(Brain science is understood here as brain theory, and - mostly - not as neurobiology etc..)

Consider this: The brain is studied because of the mind (this is an easy and safe statement).
Brain models (but also empirical research) are directly influenced by concepts of the mind.
Best examples are learning and representation, or consciousness.

The issue is this: what are we seeking in the brain?
In particular, "representations" are crucial from this point of view.
    Much of brain theory spent decades looking for a kind of mind that is not in the brain
    See recent rethinking of brain theory by (to name two very different directions) Damasio, Tsuda - and others.


Agenda for the Lecture

In this lecture we discuss:
    the representational mind
    its relationship to language, and the origin of this association
    the representational mind fails, critique of meaning and language
    a picture theory of the mind, and its (reparable) problems

More specifically:
    Classical representations are linguistic, and given what we expect from them, they should be like that.
    But this image of language comes from writing: Words are names, fixed entities with inherent meaning.
    Real words and concepts are not categorical, however - neither are immutable or eternal.
    Words have no meaning, just uses.
    Pictures underlie the words.
    Pictures may be fundamental, and may have true meaning, but:
    Even pictures are not meaningful without situations.




What is a representation? From relation to symbol and language.

Cognitive science has its favorite motifs and we have to reiterate them here, for sake of familiarity.


Representation, definition
A representation (the noun) can be any physical object or state that is somehow made to stand-in for (i.e. 're-present') some other physical object or state (or extremely complex disjunction of states or objects -- including abstract objects, such as numbers). Representation (the verb) is a relation between such representations and the things they are said to represent.


Mental representation
In a broad sense (and later we will see, that there is no narrower sense than this),
        mental representation is a relation between a mental entity and a pysical entity.
But, what are the mental entitites?


Representation, the technical concept
An elementary introduction e.g.
http://www.dimi.uniud.it/dottorato/informatica/corsi/documents/Chandrasekaran/lecture1.ppt
A medium-level introduction:
http://users.bestweb.net/~sowa/krbook/index.htm
An advanced introduction:
The Well-Designed Child    by John MacCarthy, Stanford.



D.C. Dennett's anti-representational conception
(no representations but attributions of meaning and intentionality – we will talk about this…)

Consider a coin-sorting machine as a black box. It tells true from false coins.
We don’t know how it works. Does it represent a knowledge of coins?

Move the machine to Panama (true becomes false by crossing the border, not by "knowledge" of truth and falsity).
In short, the suggestion is that terms of

                belief,
                knowledge,
                meaning, etc.

are our attributions - these are not intrinsic, but derived notions.
We assing them to the system, but they are not "really" there - not in THAT sense......

The classical representational theory of the mind is exactly the opposite of this.
Representation as a notion can be used to discuss different content theories and to argue about consequences.


The Logician’s Model
Internal representation about properties of coins (weights, metallic content, diameter, types of coins . . .) as axioms,
a theorem-proving decision procedure to classify.

An essential pont is this: the representational mind is like logic.
        Logic is based on a specialized language and so is the representational theory.
        Language plays the same role in the representational thery as in logic.

Representations (that is, classical representations) are symbolic.



Representation: a Stand-in, Stand-for, Surrogate ..... a Symbol!
Can enable an entity to determine consequences by thinking rather than acting, i.e., by reasoning about the world
rather than taking action in it.

This is typical for symbols. So a representation must be a symbol system.

The Physical Systems Symbol Hypothesis
of Newell and Simon (1976) formalized the commitments of this sort of approach to modeling cognition:
Natural cognitive systems are intelligent in virtue of being physical symbol systems of the right kind.
This stance is also exemplified by the work of Chomsky, Minsky, Fodor and Pylyshyn.

What are Representations if not Symbols
W. Bechtel: Representations and Cognitive Explanations:
Assessing the Dynamicist's Challenge in Cognitive Science
Cognitive Science, 22, 295-318. full text of paper




The structure of symbolic representations

Fodor, Jerry A. and Zenon W. Pylyshyn (1988). "Connectionism and Cognitive Architecture: A Critical Analysis"
in S. Pinker and J. Mehler, eds., Connections and Symbols, Cambridge, Massachusetts: MIT Press (A Cognition Special Issue).

Thought has
                        (1) productivity                 = we can think new thoughts (write new sentences)
                        (2) systematicity                =  (a) human thoughts come systematically together, e.g. consequences.
                                                                     (b) if you know how to produce one, you know how to produce any
                        (3) compositionality          = the meaning of the  new thoughts is derived from the meaning of the components

Hence, (i) all mental representations have these properties     (ii) any mental theory should give account of them.




The identity of thought with language                     the famous argument, itself!

Language is the only example we know of a system with properties (1)-(3).
Tereferore, the mind must be based on language.

This is the language of thought hypothesis (LOT).
Fodor's guide to mental representation: The intelligent auntie's vade-mecum.
Mind, 94, 1985, 76-100. Reprinted in Fodor: A theory of content and other essays. MIT Press, 1990, 3-29 pp.

A detailed treatment:
M. Aydede: The Language of Thought Hypothesis
http://setis.library.usyd.edu.au/stanford/entries/language-thought/





                            There are several problems with representations of this kind... e.g. infinite regress.  
                                   Many people are worried that every representation requires a user, or viewer.
                                   (We will discuss this in Lecture x).
                                   That is like another mind that again uses a representation, and so on - we have a never ending story of the
                                   Cartesian "homunculus". The homunculus is a "small person in the head", the viewer of the "Cartesian
                                   theater" (see Dennett: Consciousness Explained). There are various awkward suggested resolutions of this
                                   regress problem (e.g. Fodor, J. 1985, cited above).



The Origin of Mental Language is in Writing

A historical outlook:
        verbal cultures use names, but they have no concept for "words" and other linguistic items
        nouns and verbs, or "words" in general refer to identifiable, hence written, entitites
                        mental language or speech is not like that.
                        (names are exceptions. They are identified by the object which possess them.)
        grammar was developed as a response to the technique of writing to solve its problems
                        (language needs no such regulation and it would not work anyway).
        abstract terms like ideas are generated by the syntax of writing, and don't exist before.



Properties and consequences of written language
Written language conveys impressions
                that
                    words are identical with all their instants (i.e.they are tokens that constitute a type)
                    one word has just one kind of meaning, which is what the word stands for

Written language is categorical and suggests that language itself is categorical.


The historical paradox of the concept of language
The concept of "language" and its characterization is born when language
disappears - when it is replaced by writing, which becomes the (cultic) center of a new culture.
This necessitates new concepts to deal with - writings are objects as anything else, so it is
natural to study their properties.

But written text is the exception rather than the rule in language use
                it has to carry meaning alone, because the speaker and other elements (modalities, contexts) are not present.
                written words are "petrified results"; Wittgenstein (see below) calls them "corpses".

On closer examination there is nothing in real language that resemble objects.


The mistake from writing
is to think about language in terms made possible by features of written language.

writing produces rigid objects from language
makes words look like names
we are tempted to look for what these names are naming     --> reference, denotation
through the mistake from writing, we think that language also consists of names
through folk LOT, we think that mental objects are also similar to written names

But nothing could be farther from the truth.



What are mental entities if not words of a text?

"Words of a language" in LOT must be replaced by "words of a text", so we can see its absurdity.
Leads to a "writing in the head" metaphor of the mind.

But there is empirical evidence that words are not the relevant units of cognitive/language processing.
        Chunks
        Shemas
        And.... (pictures).
We discuss how this is reflected in theory in a purely abstract form.

 What follows next:
(1) The failure of the representational mind
(2) An alternative system of thought.

Both can be presented through the work of L. Wittgenstein


Ludwig Wittgenstein  (1889-1951)
     his life, and an introduction to his philosophy
     another introduction: http://www.philosophypages.com/ph/witt.htm

Top-level "official" nodes
Bergen Wittgenstein Portal http://www.wittgenstein-portal.com/
Tractatus Logico-Philosophicus http://www.kfs.org/~jonathan/witt/tlph.html

Reading assistance:
http://www.california.com/~rathbone/lw.htm
http://www.dnai.com/~rathbone/lwtocc.htm




Use theory of meaning

Definition
The semantic theory according to which the meaning of a word (or the meaning of language in general) is determined by its
use in communication and more generally, in social interaction.

Wittgenstein: Utterances are not complexes of names, but dynamic acts, embedded in situations.

The meaning of utterances is bound to extra-linguistic factors that constitute the physical and social elements of a situation.
(Situation relativity and context-dependence arises. Meaning is not in language, but in a complex of which language is a part)

Language is a game, speech is an act.


Language game
L. Wittgenstein: Philosophical Investigations
Aphorism #2
Aphorism #7
Aphorism #19
Aphorism #27

The bottom line is that:
Words, and language in general, have no meaning in the literal sense, they have use.

To understand language is not to extract meaning from words but to know the situations in which they are used.
There is nothing in language that could be represented in the mind.

A detailed, introductory discussion for nonphilosophers:
        On Wittgenstein's Concept of a Language Game http://www.california.com/~rathbone/word.htm


Versions of the "use theory"
The use theory of meaning is also called externalism (because of external factors), and functional role semantics.

Internalism and externalism, an annotated bibliography is: http://host.uniroma3.it/progetti/kant/field/voltoli.html

Semantics, functional role - The meaning of a representation is the role of that representation in the cognitive life of the agent.
It is an extension of the well known "use" theory of meaning as it supplements external use by including the role of a symbol inside a computer or a brain
Also known as conceptual role semantics. N. Block writes this:

"According to Conceptual Role Semantics ("CRS") [i.e. functional role semantics], the meaning of a representation is the role of that representation in the cognitive life of the agent, e.g. in perception, thought and decision-making. It is an extension of the well known "use" theory of meaning, according to which the meaning of a word is its use in communication and more generally, in social interaction. CRS supplements external use by including the role of a symbol inside a computer or a brain. The uses appealed to are not just actual, but also counterfactual: not only what effects a thought does have, but what effects it would have had if stimuli or other states had differed. The view has arisen separately in philosophy (where it is sometimes called "inferential," or "functional" role semantics) and in cognitive science (where it is sometimes called "procedural semantics"). The source of the view is Wittgenstein (1953) and Sellars, but the source in contemporary philosophy is a series of papers by Harman (see his 1987) and Field (1977). Other proponents in philosophy have included Block, Horwich, Loar, McGinn and Peacocke (1992). In cognitive science, they include Woods (1981) and Miller and Johnson-Laird (1976). (See references in Block, 1987.)"
(in Ned Block: Conceptual Role Semantics )


What the use theory is not
Note: the use theory of meaning is of course not a theory but a framework of thinking
theory = usually a systematic method, theory of meaning = method to find meaning
The use theory cannot give the meaning of a proposition, only specifies the general conditions among which meaning arises.

There exist several extensions of the use thery. Situational logic, situation calculus etc.
They do not solve the problem of meaning assignment and the nature of mental entitites, of course.


What the use theory is
The use theory is a general anti-representational framework of thinking
(and not just another semantic theory or truth theory!)

The full meaning is in Wittgenstein who expresses it radically (later uses try to to undo this, at least partially).



Meanings are not categories but families
L. Wittgenstein: Philosophical Investigations
Aphorism #65
Aphorism #23
Aphorism #11

For words and concepts, usually no conjunctive definition can be given (i.e. a list of common, defining features)
Family resemblance expresses a different concept of unity
    disjunction of conjunctions -
    sharing of traits
    but there is no trait common to all members of the class


A more detailed explanation is found here.

Family resemblance - The idea of family resemblance is Wittgenstein's answer to the idea of fixity of meaning. We tend to think of words as labels that we can apply to things, ideas, mental states, and so on. This leads to the notion that a word like "understanding" must have one fixed meaning, which we might identify as some sort of mental state or process. When we use the word "understanding" in different contexts, we think that both uses of the word share something in common.
In order to show the error in this way of thinking, Wittgenstein uses the metaphor of family resemblance. If we gather together five members of the same family, they probably look alike, although there is no distinctive feature that they all share in comm on. A brother and a sister might have the same dark eyes, while that sister and her father share a slightly turned-up nose. They have a group of shared features, some of which are more distinctly present in some members of the family, while some features are not present at all. Wittgenstein argues that the different uses of one word share the same family resemblance. There is no single defining characteristic of all uses of the word "understanding"; rather, these uses share a kind of family resemblance with one another.
http://www.sparknotes.com/philosophy/blueandbrown/themes.html


Wiitgenstein online defininition of key terms  http://www.geocities.com/wittgensteinonline/pg4.htm

Atomism, Logical Beetle in the box (analogy of) Criterion Contradiction Duck-rabbit (analogy of) Family resemblance Form of life Grammar, autonomy of Grammatical proposition Identity, the paradox of

Language-game Meaning Nonsense Ordinary language and philosophy Ostensive definition Picture theory of meaning Private language problem Rules Samples, explanation by

Saying and showing Sense and reference Solipsism Tautology Truth-Table Verifiability and Falsifiability Verification Principle

Family resemblance and the concept of 'species' in biology

Aristotle's categories are natural kinds, the example is species.
Darwin recognizes that "species" is a nominal concept, species are not categorical unities.
Categories have uniform membership, species have continuous variations.
As a consequence,
        Categories are immutable, sharp and permanent.
        Species are flexible, have dynamic boundary and are changeable.
Ever since Darwin the evolutionary concept of species is a general model for variation and variability.

Wittgenstein's family resemblance is a variation of this.
(In biology there are complications because of the difference in genetic and morphological similarity though).


The importance of the notion family resemblance
Again, not just a semantic theory. Who cares meaning?
It is a general model (like Darwin's) for entitites.
Text (in particular, printed text) is categorical.
Words of text have definitional, categorical properties.
Other objects tend to be more like members of families.

This is important if we want to imagine mental entities (or any other entities, of course).
In particular, pictures have family resemblance with each other and with what is pictured.



The Picture Theory of the Mind

Ch. Nyiri: The Picture Theory of Reason, http://www.uniworld.hu/nyiri/krb2000/tlk.htm
K. Cashell: An Attempt to Understand Wittgesntein's Picture Theory


The picture theory of the proposition
How to give meaning to a proposition?
The meaning is perhaps a picture.
In the Tractatus Logico-Philosophicus  1     2

"Ludwig Wittgenstein, supervising the searchlight of a captured Russian ship silently adrift on the Vistula in 1914, 9 was thinking about the mysteries of logic. And his quarry - to disclose the general, universal form of the proposition, that to which all empirical propositions of language (however complex) conform - would, he believed, solve at last those logical spectres haunting him since Cambridge, and which he alone, heroically, struggled to exorcise in Norway the year before.

During the first months of World War One he was working on the analysis of complex sentences into symbolic components; he believed that empirical language, if reduced to its elementary logical parts, could be shown to correspond to the irreducible (atomic) components that - taken in their totality - constitute reality. The direction of his thought was leading toward the conviction that only painstaking analysis could show the shared relational form of elementary proposition and associated portion of reality. In sum: if a proto-sign was discovered to represent the universal form of the general proposition, then such a sign would somehow also demonstrate the logical structure underlying language: that which enables language to describe (or at least give the appearance of logical correspondence with) a reality apparently indifferent to our description of it.

It was during his nocturnal supervision on the Vistula that he came to the conclusion that the relational form (logos) co-ordinating thought, language and the world was pictorial in nature. Prima facie, this says no more than that we picture facts to ourselves (2.1). Later the story was told how, while serving on the Eastern front, Wittgenstein read a report of a Parisian court case in which a model was used by witnesses to exhibit evidential facts (G.H. von Wright in Monk: 1990; 117).

No apocalyptic sign, or revelatory seal, merely the straightforward reconstruction of an accident used routinely in evidence: nevertheless it struck Wittgenstein (embroiled in his problematic) with the force of an epiphany. "
 

Wittgenstein believes that pictures are the essence of the relation between a proposition and the outside world.

    See http://www.uniworld.hu/egyetem/wittgenstein/irodalom/Tb.htm


The picture theory of the mind
Pictures are not only the sources of meaning for words.
Pictures are the medium of cognition.
Humans think direclty in pictures.

Historically
Already Plato...
Aristotle: De Anima (full text is here)     "the soul never thinks without an image" citation here.

Then the idea disappears publicly, because of the primacy of writing:
    this is well discussed in
                Spengler etc, see
                in Nyiri The Picture Theory of Reason, http://www.uniworld.hu/nyiri/krb2000/tlk.htm

But the "picture based mind" has always been present in philosophy, Locke, Russell, etc.
Nigel Thomas: Mental Imagery    (Stanford Encyclopedia)

Reappears in cognitive science ---- the imagery debate
e.g. Z. Pylyshyn    http://ruccs.rutgers.edu/pub/papers/ZPbbs98.ps

Most recently:
Merlin Donald, Origins of the Modern Mind,     "The most basic is pictorial..."
                                see Behavioral & Brain Sciences 16. 4 (Dec 1993): 737-791, full text
L. Barsalou (below).




Problems of the Picture Theory


The Problem of Generic Images
The triangle -- as opposed to a triangle

H.H.Price: pictures are "partially instantiative elements"
This means that they are strongly, but not completely, like a concrete example, an instance
        of what they represent.
(The word "instantiation" means: formation of an instance.)
Sober, E. 1976. Mental representations. Synthese 33: 101—148.


Taking concepts as embodiments and mental models - this will suggest a solution in Lecture x.
This is not a very diffcult problem.

(For instance, some theories claim that we do not store conrete memories but general ones only.
A famous example is "beatiful face" -- which is average face, as it turns out. Average in the statistical sense!).

        A famous study, in a new version: http://perception.st-and.ac.uk/
For embarrassment, here is more....
http://www.octapod.org.au/webdweller/averageface/
http://spotit.itc.it/doc/en/medio.html


The Problem of Logic and Conceptual Structure
A recording system --- and a conceptual system; these are different.

The issue is essentially Fodor's (1)-(3) in another form.
Inferential function; binding between reusable concept and individual; propositions; etc.

A recent solution:
L. Barsalou: Perceptual Symbol Systems, Behavioral and Brain Sciences 1998, full paper online

His "Perceptual symbols" are partial images, but also more complex, multi-modal.
Mental entities are simulators - perceptual symbols used for producing a variety of results.
Perhaps enable combination, aspect selection, type-token relations, categorical inference etc.
Plausible, but details are not filled in.
(when discussing mental models we return to here to discuss the concrete idea).
Remember that here we did not reach any conclusion (yet).


The problem of conventionality vs. "standing for itself"
L. Wittgenstein, L. Barsalou, N. Goodman believe pictures are empty without --- well, words.
Pictures are ambiguous without knowing how to use them - a use theory also for pictures.

Famous example (1): photograph of old man on a steep road - is he climbing or slipping backwards?
famous example (2).: duckrabbit   

Wittgenstein himself: thought that pictures carry meaning by convention, but nevertheless used them extensively for explanation.
If you can explain by pistures, it means they need no further explanation any more. They carry meaning. Except that.... (see next point).


How to solve the problem of conventionality

Pictures (and underlying mental entities) can have meaning for the mind, if
(1) agumented with e.g. animation (the temporal dimension fixes meaning - a movie is understandable where a picture not.
            how does the old man or the duckrabbit move?
(2) context and situation may supply last details (multimodality can be important).

Both are pointing outside from the domain of pictures only.


Summary of conclusion
The mind uses pictures (not language as folk intution has it)
Pictures are used in conjuction with other modalities.
Of these modalities, we will see that the motor-kinetic-kinesthetic-haptic system
    is of utmost importance, besides the visual of course.