6a. Harnad, S. (2005) To Cognize is to Categorize: Cognition is Categorization
That categorization is cognition makes sense, but what does "cognition is categorization" mean? (on the power and generality of categorization: doing the right thing with the right kind of thing.)
Reading: Harnad, S. (2005) To Cognize is to Categorize: Cognition is Categorization, in Lefebvre, C. and Cohen, H., Eds. Handbook of Categorization in Cognitive Science (2nd ed). Elsevier.
Instructions for commenting: Quote the passage on which you are commenting (use italics, indent). Comments can also be on the comments of others. Make sure you first edit your comment in another text processor, because if you do it directly in the blogger window you may lose it and have to write it all over again.
“To cognize is to categorize: cognition is the capacity to do the right thing with the right kind of thing.” (Harnad, 2005).
ReplyDeleteWhat I find interesting about Harnad's (2005) point is how generalized and basic categorization is. It is not just giving a label to an object (for example, saying that an object is a "cat"), it is forming categories that shape the behavior of that label, from basic sensorimotor interactions to higher level-like complex abstract thought. For example, even knowing if a sound is dangerous or safe is sorting sensory input into categories and then responding according to those categories. The framing lends to unifying cognitive processing and action under one mechanism (regardless of if its perceptual or pattern recognition). When we identify a pattern of sensory input/figure out an action to take, we are categorizing properties of things in the world and linking them to responses/propositions. Harnad emphasizes that categories can be learned in either a sensorimotor grounding way or verbal grounding, which links to the symbol grounding problem. Cognition isn’t some kind of magic power but rather, it’s just our flexible and layered ability to detect difference and respond to different kinds of things in the world.
***EVERYBODY PLEASE NOTE: I REDUCED THE MINIMUM NUMBER OF SKYWRITINGS. BUT THE READINGS ARE **ALL** RELEVANT TO AN OVERALL UNDERSTANDING OF THE COURSE. SO, EVEN IF YOU DO NOT DO A SKYWRITING ON ALL OF THEM, AT LEAST FEED EACH READING YOU DO NOT READ TO CHATGPT AND ASK IT FOR A SUMMARY, SO YOU KNOW WHAT THE READING SAID — OTHERWISE YOU WILL NOT HAVE A COMPLETE GRASP OF THE COURSE TO INTEGRATE AND INTERCONNECT FOR THE FINAL EXAM.***
DeleteLorena, yes, learning to categorize things —learning to do the right thing with the right kind of thing — comes far earlier than language: Learning what-to-do-with-what comes before learning what to call it.
ReplyDeleteOr, rather, “naming” (labelling) things is one of the later things you can learn to do with things — if you’re a human (and have evolved language: Weeks 6a-9b).
Other species can learn to do almost everything humans can do with things — they can even be trained to “name” things (that’s easy, it just means learning to make an arbitrary movement or sound). [Does that remind you of anything else in this course?]
But naming things is not language; it’s just labelling. Why would you want to label things?
Language itself begins with the capacity to produce (and understand true/false, subject/predicate propositions, composed of category-labels: each label a content-word referring to a category.
Propositions can define or describe things. Strings of content-words can also be used to request things, and ask about them.
And one of the most important questions you can ask about things can be asked before you even know what the thing or its label is!
You can ask what a label refers to. And the answer (if you ask someone who knows) is a proposition that defines or describes the features that distinguish the members from the non-members of the category to which the label refers: “An apple is a round, red fruit.” “Round” and “red” are (sensorimotor) features that distinguish apples from bananas (but not from billiard balls).
But not only does the person you ask need to already know those feature-names and their referents to be able to answer your question: You too need to already know them! What is going on here? and what does it have to do with symbol (i.e., label) grounding?
Stay tuned — but try to anticipate…
Thanks, that really clicked for me. I like that you pointed out how naming comes much later after actually learning to navigate through things. This relates back to our conversation earlier in the course, how animals can categorize and act on things in their environment (such as food or danger) without the need for language. That's categorization happening before words. The part about labeling just being an arbitrary sound or movement made me think about computation too, like a computer would operate on symbols without actually needing to "know" anything. A parrot can say "apple," but that doesn't mean it recognizes the category like we do when the symbols are grounded. I also found it interesting that for definitions to work, both people have to already be sharing a grounded knowledge of the features words that have been used. That really ties directly back to the symbol grounding problem, and words being grounded first in order for the rest of the language to be able to build off of them.
DeleteLorena, that's it...
Delete
ReplyDeleteThe thought experiment of Borges’s Funes was an interesting one that highlighted the importance of abstraction as necessary for categorization. Without abstraction, “he wouldn't have been able to grasp the concept of a dog… or anything else, whether an individual or a kind.” This inability to abstract stemmed from Funes’s inability to forget. It follows that forgetting is a crucial cognitive tool; if this is the case, then it implies that intelligent AI would need to be able to “forget” information as well to be able to think. As described in the paper, modern systems utilize selective “forgetting” mechanisms; for example, neural networks don’t memorize every single data point, but instead rely on pattern recognition through dimensionality reduction, ignoring irrelevancies, feature selection. But is this really the same mechanism at work that underlies our own human systems of forgetting and abstraction? Or is a rough approximation enough – it doesn’t matter if it reflects our internal systems as long as it is able to serve the functions?
Elle, let’s not forget that Borges was an artist. But, yes, selective attention, selective perception, distinctive feature-detection, and the ability to filter out irrelevant features are all important for cognition and learning. Funes was fiction, but there are real clinical cases of hypermnesia (ask ChatGPT about it, in relation to Borges’s Funes the Memorious, and Luria’s Mind of a Mnemonist, as well as to modern deep learning models and Dimension Reduction. You can even have GPT read the 2005 reading https://eprints.soton.ac.uk/261725/10/catconf.pdf
ReplyDeleteI'll go look into this! Thanks!
DeleteWhat struck me the most while I was reading this was the phrase that the ability to categorize cannot be hearsay “all the way down.” Even the most basic instructions that are communicated have an abstraction over some grounded aspect that we must perceive for ourselves and if we didn’t have the ability to experience them firsthand, we cannot truly grasp and learn from the instructions. It reminds me, strangely of the thought experiment of Zeno’s paradox, where to run a 100m race you must cross an infinite number of halves to reach the end, where each smaller half represents more basic instructions to but unless we have some way to ground the symbols in the instructions you can never quite reach the end (understanding).
ReplyDeleteLucy m, Good analogy to Zeno's Paradox, which makes it sound as if moving from point A to point B were an infinite regress and you could never get from A to B.
ReplyDeleteBut consult ChatGPT to learn why that's not a paradox but a fallacy (just as its twin -- Lewis Carroll's "What the tortoise said to Achilles" -- is a fallacy). Both of those are based on a misunderstanding (what is the misunderstanding?)
But the dictionary-go-round really would be an infinite regress if there weren't a way to ground dictionary look-up. Come back afterward and tell us why and how dictionary-look-up need not be an endless road from meaningless words to meaningless words.
A fallacy indicates that there is a logical misstep in reasoning that would undermine a whole argument. In this case the misunderstanding is equating an infinite number of steps with an infinite amount of time needed to do it. So, although there might be an infinite number of steps (trying to cross infinite halves) required to reach the finish line it can be done in a finite amount of time. And our dictionary look-up does not always have to be meaningless words to meaningless words because there is a subset that will be grounded in our sensorimotor experience, and we as humans have the ability to experience the reality that ties into those symbols to give them meaning ground up.
Deletehi Lucy, I like the way you framed dictionary look-up as a potential infinite regress, because it really highlights why grounding is necessary. What makes it different from Zeno’s paradox is that there’s a natural “stopping point”: some words are directly tied to sensorimotor categories that we can perceive and act on. Those grounded words anchor the rest of the symbolic chain, so it’s not endless word-to-word substitution but a network that’s pinned to reality at certain nodes. That means understanding isn’t about tracing back forever but about having enough of those anchors to make the whole system meaningful.
DeleteLucy M and Rena, correct (but it’s not so much about the “experience” but about learning the category-distinguishing features, whether directly by sensorimotor trial and error, or indirectly, by being told.
Delete“All of our categories consist in ways we behave differently toward different kinds of things … That is all that cognition is for, and about.”
ReplyDeleteI think this sentence really captures Harnad’s main point. Not only are categories the main function of cognition, but categories are all different and are not something innate. They consist of different stimuli that occur differently at different times and they form as needed in different unique situations based upon the need in that given situation. These categories are brought about by things such as recognition, decision, and perception, all of which need to be correctly mapped onto a category based on feature organization into members and non-members. Cognition then is not one of these things that is a function or variant (version, form, or type) of categorization, but categorization is the main function of cognition.
This work compellingly reconciles cognition as an ongoing categorization process, rather than a sequence of detached operations. Harnad’s emphasis on doing “the right thing with the right kind of thing” brings together sensorimotor experience and symbolic thought. It also highlights the foundational category formation which reframes that things such as symbolic grounding suggest that our mental architectures are thoroughly shaped by classification dynamics.
ReplyDeleteThe idea that “cognition is categorization” pushes further than saying we use categorization in thinking; it suggests that categorization is the core of cognition. Every act of recognizing, deciding, or responding is essentially sorting inputs into kind; doing the right thing with the right kind of thing. Whether it’s distinguishing edible from toxic mushrooms, or abstract categories like “prime numbers,” cognition always boils down to detecting invariants and acting accordingly. Without categorization, there would be only raw sensory flow, as in Borges’s “Funes the Memorious,” who could not abstract. In that sense, cognition isn’t built on categorization; categorization.
ReplyDeleteRena, I really like how you contrast concrete examples like edible mushrooms with abstract ones like prime numbers. It captures the range of what “cognition as categorization” can include. If I’m understanding Prof. Harnad correctly, though, even that distinction ultimately collapses. What seems like a difference in degree—between recognizing tangible things in the world and grasping intangible ideas—is actually a difference only in content, not in kind. All categories, no matter how concrete they seem, are abstract by nature. Whether we’re sorting mushrooms or numbers, we’re engaging in the same cognitive act: selectively attending to certain features and ignoring others. The “abstract” element isn’t reserved for concepts like mathematics—it’s built into the very act of recognizing anything as something. In that sense, even our most concrete perceptions are already abstractions. Every act of knowing is, at its core, an act of abstracting.
DeleteWhat stood out to me in Harnad’s argument is how categorization isn’t just about sorting the world into fixed boxes, but about adapting those boxes as contexts change. If cognition is categorization, then it’s also the ability to flexibly update our distinctions between “kinds of things” as our environment, goals, or experiences shift. This makes cognition an active, ongoing process rather than a static mapping between stimuli and responses. The same object might belong to different functional categories depending on circumstance. For instance, a rock can be a tool, a danger, or just part of the scenery. Harnad’s view implies that understanding depends on recognizing which features matter right now, not just in general. That seems especially relevant when thinking about how humans (and potentially AI systems) can generalize and transfer learning to new situations, we don’t just store categories, we re-categorize dynamically.
ReplyDeleteI agree with you, Rachael. The way Professor Harnad claims that "all of our categories consist in ways we behave differently toward different kinds of things" really frames cognition as an active sensorimotor process. The part in the paper that really stood out to me was how Harnad connects this to learning: categorization is not only reacting but adapting through feedback. In other words, we learn to “do the right thing with the right kind of thing” because experience and consequences shape our differential responses. Cognition, therefore, is not something separate from behavior but is the system's continuous process of refining its relation to the world.
DeleteThe section on supervised learning highlights that categorization is 'underdetermined', because any sensory input allows for infinitely many possible categorizations. This reminds me of the limits of self‑teaching, such as in music: without guided feedback, it is difficult to know which features of music matter most. Unsupervised learning is powerful only when the relevant affordances (the necessary features) are quite salient (for example, distinguishing a major chord from a minor chord by ear, because the difference in tone quality is obvious). But it fails when correct categorization depends on context, such as in jazz, where the same notes or chords can be interpreted differently depending on style, improvisation, or expression. Supervised learning provides corrective signals that highlight relevant features (affordances).
ReplyDeleteI like the way you tied music into this! You’re totally right that our sensory input is just a chaos of features, and without guidance, figuring out which features actually matter is nearly impossible. This whole situation is what they call underdetermination.
DeleteThe difference between a major and minor chord you mentioned is where unsupervised learning works perfectly. Our mind can sort those sounds based purely on their structural similarities because the essential features are obvious in the input itself. But in complex jazz, where the correct interpretation depends entirely on the context or the artist's style, relying only on internal analysis fails. The same sounds are throwing off an "infinity of potential shadows," and we need help resolving that confusion.
Our need for guiding signals is why we need supervised learning in order to find the invariant features to define a categeory. This ability to find these features is what allows us to compress differences within categories and helps us push differences farther apart
“All of our categories consist in ways we behave differently toward different kinds of things … That is all that cognition is for, and about.”
ReplyDeleteThis idea nicely links perception and action, but it also seems to make thinking just about being efficient, doing the right thing with the right kind of thing. Yet we sometimes act “wrongly” on purpose, to explore, play, or create. Art, humor, and moral choices often depend on breaking rules or mixing categories. If thinking is only categorizing, then where do curiosity and imagination belong. Are they new forms of categorizing, or something beyond it?
Hi Randala, I found your comment thought-provoking, especially in reference to how categorization relates to art and creativity.
Delete“When we recognize something, we see it as a kind of thing (or an individual) that we have seen before ... It requires the capacity to abstract.”
Considering categorization in the context of something like an abstract painting is an interesting case. If we look at an abstract painting, the viewer's perceptual machinery will abstract sensory inputs -- like colours, shapes, textures, etc -- and search for meaningful "kinds of things", but likely to no such luck in any concrete and specific sense of recognition, beyond the very basic features mentioned, like recognizing a "red thing" (which does not provide us with much satisfaction). Still, as one does with an abstract painting, the viewer interprets and tries to recognize and describe more specific "kinds of things" in the painting, even when there is none obviously present there. I think this sort of creative thinking involves the combination and/or stretching/transformation of categories in order to determine what "kind of thing" we are looking at and what to do with it. To make sense of it, we must pull vague features from the art and relate them to known "kinds", even if they do not immediately fit these "kinds" in an obvious way.
That’s such a thoughtful point Randala! I really like how you’re pushing beyond the idea of cognition as purely functional. I think you’re right that exploration, creativity, and even moral reflection involve playing with categories rather than just applying them. Maybe curiosity and imagination are still forms of categorization, but at a higher or more flexible level, where we intentionally test or blend categories to discover new “kinds of things.” In that sense, even breaking rules could be a cognitive process: it’s how we expand the boundaries of what counts as “the right thing” or “the right kind.” So maybe thinking isn’t only about being efficient, but also about revising and reinventing the categories that guide our actions in the first place.
Delete"If thinking is only categorizing, then where do curiosity and imagination belong. Are they new forms of categorizing, or something beyond it?"
DeleteWould imagination and curiosity be a form of categorization too since they are part of cognition? I think that curiosity is a mechanism for finding information to reduce uncertainty and help with categorization learning. Perhaps when we are too uncertain to properly categorize yet is when we are curious and once we have learned a new category enough to be able to confidently distinguish key features, we lose that curiosity. That could be why children are so curious compared to adults since they are much more uncertain and have yet to gather enough information for several categories. Imagination might just be jumbling categories together to try and imagine potential new categories and an attempt to gather information for potential categories that can be encountered in the person's life, but I am less confident on this idea.
Very interesting everyone! An object belongs to more than one category so I think that creativity and curiosity are just other ways of levering categorization. I remember from PSYC213-Cognition that when we talked about problem solving, we talked about creativity. You need to do the right thing with the right kinda thing to solve a problem. Imagine needing to escape from a room without a door but only a hole in the ceiling with a hook. On the floor there are books, a few pillows, two blankets, a bowl, but there is no ladder or “obvious” things to get up there. The salient features of the blanket might be “blue, fluffy, soft”. Somehow you think about tying both blankets and throw them so some fabric gets stuck on the hook. Then you get out. In this context, you needed something that was long enough to reach the ceiling. It is about abstracting the right features not to create a “new” category but to identify all the categories that an object could be part of in the context (that’s your right kinda thing). It requires creativity to be able to detect less salient features on the spot because the definition of blanket does not involves “long fabric you can use to create a rope and escape”. I believe that is might be how you get chain of thoughts and link things to others (thinking).
DeleteRandala, thank you for opening the floor to such an engaging discussion! You suggest that “thinking involves breaking categories,” as we see in art, humor, and moral choices. I wonder if art and humor might actually break existing categories only to form new ones — which still fits within Harnad’s view that “to cognize is to categorize." When we defy boundaries, we still engage in an act of categorization; "things" are part of multiple categories at all time. As for moral choices, I see them as the ultimate expression of categorization: deciding “the right thing to do with the right kind of thing.” Morality, then, might be less about going beyond categorization and more about applying it to human values and contexts.
DeleteRegarding creativity, I think it builds on existing categories, as the process of reorganizing features from what we have already categorized to form new categories, this links with our peers' comments on categorization being a dynamic and ongoing process. Curiosity, as Isabelle noted, is what allows us to ask : “What if this feature matters?”. I really liked the link between curiosity and the way it evolves from childhood to adulthood, it captures how our capacity to form and reform categories might develops over time.
Randala, I found your point really interesting! The argument that cognition is categorization from the reading made me think back to other psyc classes where we often discuss mental schemas. Everything clicked for me when I realized how closely these concepts relate. Schemas are mental frameworks that organize information about people, events, and concepts, and they are shaped by our sensorimotor experiences. This connects to the idea that “thinking involves breaking categories,” which Emmanuelle and you mentioned, though I see it a bit differently. Rather than breaking categories, I think thinking helps reshape and refine our schemas. Since each person’s sensorimotor experiences differ, so do their schemas. Moreover, when new information is processed to fit into existing schemas, it modifies their original structure. For this reason, I believe our schemas form the foundation of cognition—reinforcing the idea that cognition is, at its core, a process of categorization.
Delete“Funes’s infinite rote memory was hence a handicap, not an advantage. He was unable to forget -- yet selective forgetting, or at least selective ignoring, is what is required in order to recognize and name things”
ReplyDeleteThis phrase is powerful because it overturns the common assumption that more memory equals more intelligence. Professor Harnad uses Borges’s fictional character Funes to illustrate that cognition depends not on storing every detail, but on the capacity to abstract, to ignore irrelevant variation and detect invariants across experiences. This idea ties directly to the central claim that cognition is categorization, to know something is to be able to group different sensory inputs as belonging to the same kind.
The phrase reveals that abstraction, what Funes cannot do, is the essence of thought. Forgetting, here, is not a loss but a cognitive strategy. It enables the mind to compress the infinite variability of experience into manageable, meaningful categories. Without this selective filtering, there is no recognition, no naming, and ultimately no cognition at all.
Ayla, I really like your point about forgetting being a cognitive strategy rather than a weakness. What stands out to me is how this idea redefines intelligence. The implication here is that true intelligence comes from the ability to ignore unnecessary details and focus on what matters. I think that real understanding relies on the ability to simplify. It’s like trying to study for an exam by memorizing every single sentence of the textbook instead of focusing on the main ideas. You end up overwhelmed by facts but unable to explain what they mean. Without that selective filtering, meaning can’t really form at all.
DeleteI really like this discussion on intelligence as the ability to simplify, and Harnad’s idea of selective filtering adds another layer to that. He suggests that cognition depends on detecting the right invariants while filtering out irrelevant variation. This filtering isn’t passive, it’s an active process of shaping perception to focus on what matters for survival or understanding. Intelligence, then, isn’t about seeing everything, but about knowing what not to see so that meaning can emerge. As Jad mentioned, understanding involves simplifying a vast pool of information to fit our individual level of comprehension. That act of simplification is the result of feature detection, abstraction, and categorical perception working together.
DeleteBorge’s “Funes the Memorious” short story of a man who cannot abstract sparked a thought about a possible need for uncertainty for me as well. If a man who cannot categorize due to a lack of abstraction, and therefore sees every object and moment as a singular and unique representation of it in space and time, then a certain amount of uncertainty would be necessary to categorize properly in any given human. This leads to thinking that too much information retained is harmful to categorization, as a human or any being able of categorization needs only certain invariant features to be noticed to do the right thing with the right kind of thing. Following on that thought as well as yours, uncertainty might therefore be crucial for categorical perception, due to the infinitely thin categorical boundaries an infinite amount of information would cause. So yes, I do agree with you that forgetting is most probably a cognitive strategy and I want to add that I think it is used to clarify categorical boundaries through inhibition of unnecessary features.
DeleteWhat I find interesting about Professor Harnad’s paper is how he collapses the boundary between perception, thought, and action. His claim that “to cognize is to categorize” reframes cognition as something living and embodied, not abstract or detached. I was especially drawn to the idea that forgetting is necessary for understanding, that memory without abstraction, like Borges’s Funes, becomes useless. It made me think about how the brain filters information to create meaning. I also love how Harnad connects evolution, learning, and language, showing that even our most abstract ideas are built on physical experience.
ReplyDeleteI agree, central to Harnad’s argument is that cognition is embodied and interactive, relying on sensorimotor grounding. In the paper, it also emerges how categorization is not just a cognitive function and something that can be viewed as secondary but rather, the essence of cognition itself. The discussion on forgetting stood out to me as well, particularly as Harnad mentions “selective forgetting, or at least selective ignoring, is what is required in order to recognize and name things”. Essentially, categorization, processing and filtering information are fundamental to the operations of cognition. I am reminded of a concept “cognitive economy” from previous psychology courses referring to the mind's tendency to simplify, compress and generalize information in an attempt to conserve processing resources. Coupled with Harnad's claims, this creates a comprehensive understanding of how humans manage the high-load of sensory input by abstracting and grouping experiences to make thought and language possible.
DeleteHarnad’s argument that "to cognize is to categorize" powerfully reframes cognition as the ongoing process of sorting the world into actionable kinds. Many peers have highlighted the necessity of abstraction and selective forgetting for meaningful thought, drawing on Borges’s Funes as a cautionary tale against rote memory. However, I want to push further: Is there a risk in over-committing to stable categories?
ReplyDeleteWhile abstraction and filtering are essential, Harnad’s examples, such as the chicken-sexer’s evolving expertise, hint that true cognitive power lies in the flexibility to revise categories as contexts shift. In real life, categories are not static; what counts as “food,” or “art” can change with new information or goals. What I understand from this is that cognition is not just about compressing experience, but about maintaining a dynamic balance. I must abstract enough to act efficiently, yet remaining open to re-categorization when the world surprises us. Once we understand this we can explore how minds (and machines) can best manage this tension between stability and adaptability in their categories.
6.a. Dennett’s idea that all meaning is “derived” rather than “original” is interesting but confusing. He argues that even our own thoughts only seem meaningful because of how we interpret ourselves, not because they have intrinsic meaning. But if all intentionality is derived, it’s not clear from what it’s derived in the first place. The view avoids mystery by removing “real” meaning, but it risks making consciousness and understanding seem like illusions of interpretation rather than genuine experiences
ReplyDeleteI really like how you highlight the paradox in Dennett’s view that if all meaning is derived, it’s unclear what the original source is. Harnad’s work on the symbol grounding problem seems relevant here: I think he’d argue that meaning can’t be entirely derived from interpretation, because words/mental representations must be grounded directly in sensorimotor experience. Without that foundation, interpretation becomes circular. So maybe “original meaning” isn’t mystical but instead it’s experiential, rooted in how we interact with the world.
DeleteI’m interested in this section (10) of categories acquired through evolution and how it maps into the SGP. In Taddeo and Floridi’s piece on solutions to the SGP they discuss the importance of the zero semantical commitment condition. My understanding of this condition is that the interpretation of the symbols must be intrinsic and no prior framework can be programmed into the recipe that would guide the AA towards any type of categorical organization in particular. But we humans inherit categories through our genetics. For example, innate categorical perception in colour perception. Part of our visual perceptual process is opponent processing in retinal ganglion cells, which discern different wavelengths of light from one another, giving rise to our internal understanding of red vs green. This process is not learned nor taught (except for the names of the categories). But it is in some way “pre-programmed” into our system through evolution. So we do not satisfy that z condition. It seems inherent that by design, any sensorimotor system requires some sort of categorical organization just to be able to process stimuli into perception. How could you ever satisfy the z condition if the making of a sensorimotor system (even for an AA) necessarily required some rudimentary categorizing processes?
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThe idea that really stood out to me was that categorization is cognition and not just that categorization is part of cognition. Saying that “categorization is cognition” suggests that every cognitive process; perception, memory, decision-making, even reasoning, fundamentally depends on the ability to group things and respond appropriately to them. In other words, cognition itself is the continuous process of detecting what kind of situation you’re in and selecting the right behavior based on that. It ties back to the definition we talked about in class: “doing the right thing with the right kind of thing.” Everything from recognizing faces to understanding language depends on distinguishing what kind of thing you’re dealing with, so categorization isn’t just one cognitive skill.. it’s the underlying mechanism that makes all cognition possible.
ReplyDeleteThe idea behind categorization being innate is that as categories become more and more abstract, the overlap between the members’ features become less and less. For example a concrete category like “dogs” has features like “fur”, “barks”, etc. but an abstract category like “things” has almost no shared features so the intersection vanishes. So since there is no overlap in the sensory data then learning can’t have produced the categories, they have to be innate. However, not all categories come directly from sensory experience, some come from abstraction and language. Abstraction is the ability to focus on certain features and ignoring the irrelevant features. Language let’s people tell each other how to recognize or do something without having to have the sensory experience ourselves, but it still has to be grounded in some sensory motor experience. For example, to understand what a rose is you have to have firsthand experience with what a flower or a plant is. Therefore, categorization isn’t fully innate because even though some categories begin with sensory experience, others are built on top of those through abstraction and language. So our capacity for categorization is able to grow through interaction, communication, and layering new abstractions on top of prior grounded sensory experiences.
ReplyDelete"There would still be maximal confusion at the blue-green midpoint, but on either side of that boundary the correct choice of key and the amount of pressing would increase much more abruptly – one might even say “categorically”-- than with shades of gray."
ReplyDeleteI think what Professor Harnad means here is that even though colors change smoothly in nature, our brains don’t see them that way. Instead, we draw invisible lines that break the flow of colors into separate boxes. We said in class that anything that falls on a continuum is not categorization. That statement makes me wonder how even though the color spectrum itself is continuous, categorization happens when our mind draws boundaries on that continuum. What I find interesting is that these boundaries are not random, but are shared by most people. This means they’re built into how our brains process the world. So, even though the world gives us endless shades, our minds prefer to sort them into clear and meaningful kinds. That’s how we make sense of complexity (not by seeing everything, but by deciding what counts as different).
Jad, I like how you pointed out that the color spectrum itself is continuous while our perception isn’t. What Professor Harnad seems to suggest in this passage is that the mind doesn’t just notice differences, it creates sharper ones. The blue-green example shows that after a category boundary is established, perception near that line becomes discontinuous. The “jump” between blue and green feels bigger than it physically is.
DeleteWhat’s fascinating is how this effect isn’t limited to color. Professor Harnad connects it to how all categorical perception works, it compresses within-category differences and exaggerates between-category ones. That way, cognition gains precision at the cost of smoothness. So even though the sensory world is graded, our cognitive system reshapes it into discrete contrasts that are easier to name, remember, and act upon. In that sense, CP doesn’t just simplify reality, it transforms it into something the mind can think with.
In section 20, discrimination is described as being able to accurately describe the difference between two objects only when both are presented while categorization is being able to describe them in the absence of each other. While I understand this difference in definition, I am not sure I understand why this is relevant to our discussions. Firstly, we naturally live in a context, and nothing actually exists in a way where we have to categorize it absolutely. For evolutive purposes at least, the difference between discrimination and categorization might not be that important and may just demonstrate the limits of our biological hardware.
ReplyDeleteHi Matt! In my opinion, Harnard includes this discussion to emphasize why cognition can’t just be about noticing differences. It has to include recognizing kinds even when there isn’t something else to compare in front of us. I'm not sure if it's right to say we always “live in a context” because in real life, we often have to act on a single input (ex. Deciding if this mushroom is safe to eat. It’s not often we have mushrooms side by side for comparison). That’s where categorization or absolute discrimination, becomes crucial. It's the ability to store learned differences and being able to use them later, even without direct comparison. Miller’s 7+- 2 limit shows that categorization is where the brain starts compressing continuous sensory stimuli into a small manageable number of categories. So I think this is not just about biological constraints, but a point about how cognition works generally. It’s about turning sensory experience into meaningful kinds, which is a process that also underlies categorical perception, abstraction, and how we ground words to sensorimotor experience.
Delete"Within a very few generations the linguistics “thieves” outsurvive and out-reproduce the sensorimotor toilers"
ReplyDeleteThis is one aspect that I find incredibly interesting about the advantage of language. Here, the thieves are able to learn by overhearing what categories are about which is not dissimilar to how we get to learn categories for things we have not interacted with. On the other hand, the toilers have to learn about categories the hard way through direct trial and error. In this instance, the thieves outperform the toilers due to this ability. Creating and learning categories indirectly is better than the hard way because not only is there no way for you to assess all possible categories but the consequences of being wrong can be grave too (if you eat a poisonous mushroom you may die). Language allows us to get past these hurdles as long you are familiar with the features of some of these words as it can't just be overheard all the way down, you still need some physical grounding. Otherwise, you are back in the Chinese dictionary.
Sections 9 and 10 talked about categorical perception, explaining how there are some categories that we innately know, linked to specialized detectors. For colour, we have three specialized detectors selectively tuned to certain regions of the frequency spectrum (red, green, and blue) , which set us up with inborn boundaries between qualitative regions of the colour spectrum. The red/green contrast makes sense to me as an adaptive, evolutionary category for berry picking, but I’m curious why the blue/yellow contrast has equal weight in our perceptual system. What adaptive advantage does categorising these colours give us? Does there have to be one? Why else would this division be innate?
ReplyDeleteHey Kira, I think that what Harnad meant it section 10 is that we have these detectors because of the evolution processes. It has similarities with how we know categorization to function: through error-feedback and supervised learning. Evolution is just “shit happens”, but if the goal is to survive (right thing to do) then organisms who could categorize colours (right kinda thing) ― whether it is the boundaries between violet-blue-green-yellow-orange-red ― had a better chance of surviving and reproducing. In a way, evolution ”learned” from consequences through millions of years. Since organisms with better colour vision were surviving (positive outcome), they were the right kinda thing (members of a category who could do the right thing). There are probably more things to it, but I suppose that both contrasts are important or that it was just there in organisms who survived (just happened!) without other specific advantages.
Delete"Let us, too, set aside the “how” question for the moment, and note that so-called operant or instrumental learning -- in which, for example, a pigeon is trained to peck at one key whenever it sees a black circle and at another key whenever it sees a white circle (with food as the feedback for doing the right thing and no-food as the feedback for doing the wrong thing) -- is already a primitive case of categorization. It is a systematic differential response to different kinds of input, performed by an autonomous adaptive system that responded randomly at first, but learned to adapt its responses under the guidance of error-correcting feedback (thanks, presumably, to some sort of adaptive change in its internal state)."
ReplyDeleteI have a question on this quote. Would reinforcement learning count as supervised learning? Isn't supervised learning about trial and error in the same way reinforcement learning is? In supervised learning, we have to solve the credit assignment problem, which is finding what features matter most. In the same way, would having to find what features matter most for gaining a reward or avoiding punishment be considered supervised learning? I am just wondering what differentiates these two forms of learning or if reinforcement learning is a form of supervised learning. Is the difference that supervised learning is more explicit on what is the correct and incorrect answer while reinforcement learning is more about nudging someone in the vaguely right direction for the answer?
This comment has been removed by the author.
DeleteIsabelle, I had similar thoughts! I believe that reinforcement learning exists as its own category between unsupervised and supervised learning – that is, not purely one or the other. Unsupervised learning involves detecting structural patterns, as well as similarities and differences, directly from sensorimotor experiences. Contrastingly, supervised learning underlies explicit, error-corrective feedback that guides us when “the sensorimotor structure and its affordances* alone are not enough” (Harnad, 2005). For example, learning that the fuzzy creature in the park is a dog only after being explicitly told, “Yes, that is a dog”. Reinforcement learning, on the other hand, relies on trial and error through a reward and punishment system. Rather than receiving explicit correct or incorrect answers, the feedback from which learning occurs is based on the consequences of one’s behaviour. Notably, reinforcement learning would also not be considered unsupervised, as it depends on reward signals – a form of feedback.
Delete*For kid-sib, affordance refers to what the environment allows or offers one to do. For example, a chair allows you to sit on it. Therefore, we may say that a chair affords being sat upon.
This comment has been removed by the author.
ReplyDeleteThe reading argues that cognition is the process of categorization through sensorimotor systems’ systematic differential interaction with the world. For kid-sib, this means that we create categories based on invariant features or affordances from sensory input.
ReplyDeleteI found the Funes story helpful to my understanding of categorization, since we often think about it as being based on the presence of certain features. However, it also depends on abstraction, being able to ignore or forget others.
This is interesting in the context of ChatGPT, which I think falls in a bit of a grey zone. While, like Funes, it can remember everything, it is also able to ignore certain information (I tried to gaslight ChatGPT into telling me that Stevan Harnad wrote the Harry Potter series, and it rejected the statement). I think this is because it can cheat using the “Big Gulp” of information, so it weighs the validity/value of information it is given instead of remembering all information as truth.
Thinking about the fact that our ability to forget details is actually what allows us to think, communicate, and make sense of the world was interesting for me. If we remembered every single feature of every object or event we encountered, like Funes does, we would have so much information in our brains with no possible way to think and see any sort of patterns of similarities. Even though usually we see forgetting as a flaw of human cognition, it might actually be better labelled as a ‘feature’ of cognition? Because when we forget about specific details our scope broadens and then that is when we are able to do other important things like be creative and make predictions. This was interesting to me because now that I think about it seems so obvious, that forgetting is essential, but actually it usually is seen as something negative or a flaw of the human brain.
ReplyDeleteLauren, I agree. Evolutionarily, forgetting is just as important as remembering for survival. It allows the brain to prioritise relevant information and update outdated memories. Even how we learn is categorised to reflect this efficiency (motor skills are stored differently from explicit memories). These cognitive shortcuts reduce the need to recall every detail. To your point on how forgetting certain details broadens our scope, it can also narrow it by shaping what we perceive. Harnad (2003) suggests that perception is not only biologically grounded but also socially grounded: “Our perception of the world can also be warped, not just by how things are named but by what we are told about them.” For example, when societies erase or reinterpret parts of history, collective forgetting reshapes what people perceive as real or meaningful. This phenomenon shows that perception and meaning are not fixed or innate properties of the mind, but can change through shared narratives and the beliefs we inherit from others.
DeleteBiederman - who is mentioned in the article - proposed the Recognition by Component (RBC) theory. This made me think of his theory, which aims to explain visual perception, categorization and recognition among humans. He supposes that we categorize objects by breaking them down into simpler parts (or geons). For instance, one would be able to differentiate a mug from a glass due to their different combinations of geons. Biederman specifies some rules that justify factors such as angle of view, distortion, and visibility. While his theory is interesting and notions of it have been used in various domains including AI, it is limited; its scope fails to properly capture categorization and recognition. One of the main issues is discussed in this week's article: using this model, a specific combination of geons (or an object) could fall into more than one category. Would such a model be able to tell the difference between a car and a miniature car replica? As silly as this example sounds, the implications are important: one can be driven and one cannot. His model fails to capture the finer nuances that other features (and not mere combinations of basic shapes) provide. Would that model be fooled by the painting "La trahison des images"?
ReplyDeleteMaya and Laurens comments interested me -- I keep thinking about how what we call “bias” might just be the evolutionary cost of cognition itself. Forgetting operates as the compression tool that keeps experience manageable. Schemas are produced by that compression, and they serve as the foundation (or parameters) for prediction. They are efficient because they eliminate internal differences, but bias is a product of that same efficiency. In this way, prejudice and forgetting have a similar structure: they both arise when generalization outpaces revision. After learning invariants, the mind safeguards them. Stereotypes, as lenses that solidify faster and easier, are more cognitively stable than a fluid, flexible categorization system. Maybe understanding isn’t about escaping bias but requires recognizing where our schemas stop fitting reality. In that sense, with T3 in mind, cognition--human or artificial--only functions as long as it keeps unlearning itself.
ReplyDeleteI want to answer the following question posed in the reading about categories: Was I born knowing what is and is not in these categories, or did I have to learn it? I want to suggest that unsupervised learning can almost be thought of as having some form of innate knowledge. For example, if you showed a child a picture of a man named Harry and a picture of a dog with the same name, and asked whether they were the same, the child would likely say no. Even without being taught, there are clear differences between humans and dogs that we seem naturally able to recognize.
ReplyDeleteHowever, there is another version of this question concerning proper nouns: were we born knowing what each name referred to, or did we have to learn it? Here, the answer is clearly that we must learn it. If you presented the same child with two pictures of two different men and asked which one is Jerry and which one is Thomas, they would likely have no idea. This is because, unlike with unsupervised learning, they would need explicit feedback or instruction to know which name belongs to which person.
Therefore, I suggest that some basic or salient differences-like distinguishing between species-may be innate, helping us to differentiate categories naturally. Whereas other forms of categorization, such as associating names with specific individuals, require supervised learning, where guidance and correction play a crucial role.
The reading reminded me of an experiment I participated in some time ago. In that context, I had to learn to distinguish between “calamites” and “lacamites”, which were two different kinds of grating patterns. I received no verbal instruction to tell me exactly how to distinguish them, but rather I was successively presented with different patterns and was given feedback (“correct” or “incorrect”) every time I assigned them a label. At first, I thought the experimenters were messing with me, because they all looked somewhat the same. However, after a lot of trial-and-error, I was able to distinguish them almost perfectly. This demonstrates supervised categorical learning, where I learned to categorize (i.e. name) different patterns by abstracting the invariance from each category. That is, my brain was able to single out and give weight to the key feature that distinguished lacamites from calamites (ex. width of the bands) and to disregard other features (ex. shade of the bands) to be able to tell them apart. However, it probably would have taken me much less time to abstract the invariances if I had received explicit verbal instruction on how to tell apart lacamites from calamites.
ReplyDeleteThe reading explains that categorization involves identifying relevant invariants - features that distinguish members of a category from non-members - while ignoring irrelevant variance. I'm curious whether these relevant invariants remain stable or shift with experience. For example, a novice listener may categorize jazz by the following invariant cues: the presence of a saxophone and double bass, occasional chords that sound wrong, a general slow and soothing feel...etc. An experienced listener presented with the same sound however, may naturally recognize richer features to categorize the sound stimulus into something more particular than just "jazz". Example of these cues could be: bebop-style improvisation, use of certain trumpet-mutes, modal harmony...etc.
ReplyDeleteWith the same sound stimulus, one listener could be recognizing 'jazz' while the other could be recognizing a specific solo off of a Miles Davis recording from 1962. For the latter, broad features like “the presence of a saxophone and double bass” no longer serve as relevant invariants, since “jazz” is no longer the level at which they are categorizing. This seems to suggest that as expertise with a category grows, our perceptual system redefines what counts as relevant information.
I think I may be jumping a head a little bit relative to reading 6a, but the answer might just be in reading 6b, “Categorical Perception”. There are certain innate categories, and some other (most) that are learned. Additionally, according to the Whorf hypothesis, language changes the way in which we perceive and learn categories. Thus, the categories that aren’t innate are learned through language, and it is through language that we can name categories. So, I do too believe that the categories in the mind change as the individual learns through language and is exposed to increasingly more “things” to categorize (since categorization is doing the right thing with the right kind of “thing”). In the context of your jazz example, it makes sense that a trained jazz player would be capable of more rigorous categorization of music genres than someone who has neither been exposed to jazz music as much, nor been taught (through language) the associated content words (“bass”, “harmony”, etc.) that are used for categorization in the jazz player’s mind.
DeleteCognition is systematic categorization which is the differential interaction between an automonous, adaptive sensorimotor system and its world. To sort through we have to selectively extract invariant features called affordances, while activey forgetting the infinite variations of every sensory moment. This process is called abstraction and if we couldn't forget we would be unable to recognize kinds of things. Most categories are learned through adaptive supervised learning, using trial and error and corrective feedback to find key features. We use language to acquire abstract complex categories but these still have to be grounded in direct sensorimotor experiences to have meaning. Then i wonder, since abstraction requires selectively forgetting unique differences in sensory experience to form simple categories (like "red") how can we trust language to reliably convey complex abstract categories like good v bad, true v not, given that these words are recursively grounded in simpler experiences that were built precisely by ignoring crucial individual variations?
ReplyDelete