Okay, so our first speaker today is Hans Optebeek. Hans is a professor in the Faculty of Psychology and Educational Sciences at the University of Leuven. And he's the director of the Department of Brain and Cognition and a faculty member of the Laboratory of Biological Psychology. Hans's lab does some really interesting research investigating the neural basis of visual cognition and learning. And.
Today he's going to talk to us about the effect of expertise on behavioral, neural and computational representational space. Thanks very much. Let me share my screen.
Thank you Elaine for the very nice introduction. So I will indeed be talking about the effect of expertise and we did some fMRI to understand how expertise is changing brain function. And to investigate that, we did not look at faces because we are all experts at face recognition, so faces are not really special. And to study how expertise changes brain function, it's good to have people that don't have a certain expertise. So we went for more obscure domains of expertise that not all people have.
So these are special domains and only a minority of us master them. But many of us have such a specialty of some kind, either in a formal system, a formal hobby, for professional reasons, or even something that only you can do. And if you have an expertise of that kind, feel free to also put it in the question and answer part. And then other people can vote for it. And maybe you can be the champion expert of this session.
So feel free to do so. So what can we investigate then? Well. Like I said, we want to investigate how this changes brain function.
Now, previous neuroimaging studies have mostly looked at which regions are involved if you see the category of objects you're an expert for. And one big discussion, for example, has been whether the foot-to-front face area is involved in certain domains of expertise. And the answer seems to be that it depends.
So there are clearly differences between domains of expertise. And one such study was one we did in the lab a few years ago, where we... compared bird expertise with mineral expertise and found very different effects in how this expertise changes the neural processing of the objects of expertise, in particular in visual cortex, in high level visual cortex. In frontal cortex there were also very clear changes, but they seem to be more common between the different fields of expertise.
Now the problem is what really matters for expertise is not that you can recognize an object as being part of that domain. For example, a bird expert is not a bird expert because he can tell us that something is a bird, but he can tell us which type of bird it is, which species of bird it is. And that's the base of the behavioral definition of an expert. So that's what we should be looking for in the brain as well. And neuroimaging studies have not done that so much.
When they've done it, it's mostly through adaptation studies, where you use the fMRI adaptation as a way to get access to more finer representations. But you don't get much information about the representations themselves and the nature and the geometry of these representations through fMRI adaptation. So we have methods for that, representational methods like representation similarity analysis, using multivoxel pattern analysis, all of that.
But it's challenging to look at subordinate differences between stimuli because we know the signal is much weaker. And this is a study or a figure from a study from more than 10 years ago already showing that it's quite tricky to do. Nevertheless, we tried that.
We studied expertise in ornithology, in birdwatching. We scanned 20 experts and 20 controls. And we used a design with 24 stimuli, all birds, of course. And there was some attempt of us to dissociate the visual characteristics of the stimuli from the more semantic information, in the sense that there were eight triplets. And within each triplet, the B stimulus was visually quite different from the A stimulus, but yet belonged to the same species.
And it was... not more different from stimulus C in terms of the visual aspects, yet B and C are from a different species. We first did some behavior.
And if you look at the behavioral dissimilarity matrices, you get something like this. So you have experts above, controls below, and we have two tasks. And you can see that overall, these representations, these matrices look quite similar.
So overall, there's quite some similarity in this representational space. Nevertheless, in the experts, we see some difference depending on the task. So task difference, which we don't see in the controls.
And in particular, in the species task, you see certain kind of groupings that you don't see in the other people. During the scanning, actually, we went for an overall similarity task that doesn't really tap into this specific species aspect of the stimuli. Now, behaviorally, what we also see is that the experts are much more consistent among themselves in the representation. So if you correlate the matrix of an expert with the matrix of another expert, we find a higher correlation than when we do the same thing among controls.
So the green bars here are higher, and we see that particularly in the species task, even though it's also significant in the visual task. So let's now go to the neuroimaging part of this study. And then we get results like this one. So we present the 24 stimuli.
We look at the pattern of activity across foxholes. And we correlate this pattern of activity between conditions and themselves, a repeat of the same condition or between different conditions. And then you get a film matrix like this one. This is the one for frontal cortex, so a big region encompassing all frontal cortex.
And then you see that there is some structure in there, building controls and experts. But the structure seems to be more... pronounced in the experts. You see a bigger use of the color scale and the colors are, the color scale is matched between controls and experts. We did this in three large regions of interest, the same region that were used in that previous study on ornithology of 2018, and in early visual cortex there's a really good match between what we see in controls and experts.
In high visual cortex it doesn't look exactly the same anymore and an infrontal cortex is clearly different. So now I'm doing a Rorschach test and it's like looking at these pictures. and seeing what you see in it and what I see in it, of course we need to do some statistics in science, so let's see what actually drives this impression of a stronger representation in frontal cortex.
The first thing we looked at are again these intersubject correlations in the neural dissimilarity matrices, and we again see that these correlations are higher for the experts and for the controls. In particular in frontal cortex this effect is very strong, in the high level visual cortex it's also there. but it's weaker, but it's also significant. A second effect that we noted in frontal cortex is that the diagonal is most pronounced, and the diagonal means that you compare a condition with itself and you expect the diagonal to be more similar, less dissimilar than off-diagonal elements.
So that's a measure of the distinctiveness of the differences in the neural patterns, and you see that that is in frontal cortex stronger than for the express and for the composed. And then we also did some correlations with behavior, and there again in frontal cortex we see very clearly that the correlation with behavior is higher in the experts than in the controls. In higher visual cortex, whether you see it or not depends on where you put the statistical threshold, but it's clearly not so pronounced.
So these three effects are most clearly there in frontal cortex. We also looked a little bit in neural networks, whether we can see similar changes like that. We used the most common architecture, the LXNet, and we looked what kind of representation we see at the end of this network when a network has not been trained, then when it has been trained on ImageNet, and then third, when there's further training happening with a small bird database.
And what we see basically is that the more training the network gets, the clearer, the more distinctive these representations become, as the clearer the diagonal becomes, and also you start seeing correlations with human representations. Okay, so to summarize, expertise results in more robust neural representations. It comes for at least three changes, more consistency among individuals, more distinctive representations and more correlations with behavior. And these effects are more most clearly there in frontal cortex.
Okay, this was a very quick run through. If something was not clear, if there are questions, then please feel free to ask them now. Thank you very much Hans, that was a really interesting talk.
Let me see, we have a question here, sorry let me just bring up the, it's okay, we have a question here from Minya Zan, how big is the frontal region of interest? Do you think there are specific but smaller regions in the frontal cortex related to categorization? Yeah, it's a good question.
We used almost all the frontal cortex, like we also did in the earlier study, but we also looked in smaller regions, like for example, dorsolateral prefrontal cortex, and it seems that this is a general effect that you see throughout several regions. Now, we do not take all voxels, it's only the voxels that are active in the task that people were doing with the stema, so do a similarity rating. So, of course, there are certain regions in frontal cortex that are not active at all. But for example, dorsolateral prefrontal cortex phase specifically seems to show similar effects than when you take the whole big region together.
Great, thank you. And we have another question here from Jules Brochard. Would you have an idea of how neural circuit may adapt with respect to expertise?
I'm not sure what Jules means with the neural circuit. Does he mean the brain regions that are involved, or does he mean at the level of the microcircuitry, like the columnar structure of a region, and how... excitatory inhibitory cells connect to each other.
I don't know exactly at which level he sees that. Of course the neural circuitry at the most general level which regions are involved, like I said in my introduction, that has been the subject of quite some investigation already. And as studies have clearly shown that indeed the regions that are involved change when people acquire expertise.
But which way regions are getting more involved, that depends on the domain of expertise. Like I also mentioned, In at least some cases, you see that the fusiform face area seems to be involved, but that's clearly not the case in all areas of expertise. So it really depends on the expertise area where you see the changes. The effects that I'm showing now, you could see them as being more general in the sense that when a region is then changed to expertise, I would expect that those representations would be changed in ways that are similar to what I've shown here.
more robust representations. And more robust can then be the consequence of at least three things, the three things that I talked about. But I would expect that this effect would not necessarily be always in the same region.
So if you have another domain of expertise, other regions might be involved, and then you might see those effects in those regions. To show a very clear example, if it's an auditory level of expertise, like a musician listening to music, I would expect these kinds of effects in auditory codecs rather than in high-level visual codecs, to the extent that we had effects in high-level. visual cortex.
So I think it will depend on that. Yeah, we still have a few more questions. Here we have from Avital Hami.
Thanks for the talk. Why do you suppose there are group differences in the diagonals? Are the representations more distinct or less noisy in experts?
Yeah, why you can take that... in very different levels, but I would say that, I mean, what it exactly shows us is that it makes a difference whether you repeat the same condition or where you look for a difference between conditions. So the differences are more clearly represented if you have a clearer diagonal. So that's what the data show.
Why you then see it, well, I could rather just rephrase what I just said. I'm not claiming that these three effects that we see are totally independent of each other. To some degree, all three effects are somewhat also reflecting like noise or reliability of the data.
And so you could say you just have more reliable representations. So I used to more robust because reliable is a word that has some specific meaning, but yeah, they could also be somewhat related to each other, these three. And I would say in general, the representations are clearer. Yeah, that's for sure.
Great, thank you. So we have another question here from Haida. Do I understand it correctly that you start to see greater within category similarity in frontal cortex?
Do you think that high level visual cortex preserves the subordinate level differences between same category objects and that this is the reason why you see these clear patterns mostly in frontal? Or did I perhaps misunderstand the data? Yeah, I think that's a good question. I think we see greater reading category dissimilarity, so that's important.
It's a dissimilar, so it's a difference that becomes greater and better represented in frontal cortex. And indeed, we see it less clearly in high-level visual cortex, but you see some trends. In particular, the greater consistency between experts is also there in high-level visual cortex, and greater correlation with behavior, like I said, it's weaker than frontal cortex, but I wouldn't dare to say that it's not there.
So it's more a matter of degree than all or none, this difference between frontal cortex and a high level visual cortex. So it might also be a matter of us being able to measure the differences. But I also like the interpretation that Heidi gives to it and the solution or the hypothesis that she brings forward.
I think it's an interesting one and we could investigate that further. So thanks for the question. Okay, I think we maybe have time for one more question from Ineke. At the end stage of training the neural network, there is a correlation with the human representation.
Do you mean the representations from the frontal cortex or also with the representations from the higher visual areas? Yeah, I kept that a little bit fake because of time, but indeed it depends on how you look at it. If you do correlations with the behavior of people, then you see that these correlations are getting better. Yeah, once a network has gotten training, group difference in terms of finding better correlations with expert data, with novice data, that is something we most clearly saw for the...
frontal cortex data. So the frontal cortex of experts correlates better with what you see with a network that has been trained on birds compared to a network that has not been trained on birds. And the correlation is also higher than the correlation with controls. Great, well thank you very much Hans and thanks everyone for the wonderful questions, that was a great discussion.