The Influence of Language and Context on Sensorimotor Simulation in Concrete Concept Processing

SHI Rubin¹, XIE Jiushu², YANG Mengqing¹, WANG Ruiming¹
¹ Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents, Ministry of Education, South China Normal University, Guangzhou 510631, China
² School of Psychology, Nanjing Normal University, Nanjing 210097, China

Abstract

Sensorimotor simulation during concrete concept processing constitutes a core mechanism of conceptual representation. This study systematically investigated how linguistic factors (language type: first language vs. second language) and contextual factors (perceptual context: spatial perceptual context and semantic perceptual context) influence sensorimotor simulation. Experiment 1 employed a semantic relatedness judgment paradigm to examine whether sensorimotor simulation occurs during second language processing and whether it differs from that in first language processing. The results revealed that sensorimotor simulation persists in second language processing, though first language simulation demonstrates certain advantages. Experiment 2 utilized the semantic relatedness judgment paradigm and its variants, with two sub-experiments exploring how spatial perceptual information and semantic perceptual information affect sensorimotor simulation during conceptual representation. The findings indicated that sensorimotor simulation emerged during concrete concept processing under both weak spatial information conditions and shallow semantic information conditions. These results address the limitation that perceptual symbol theory has not offered specific predictions regarding sensorimotor simulation in second languages, demonstrating cross-linguistic stability of sensorimotor simulation. Moreover, the observation that sensorimotor simulation in concrete concept processing is not modulated by spatial or semantic information suggests that it can arise automatically to some degree.

Keywords: Concrete Concept, Conceptual Representation, Sensorimotor Simulation, Perceptual Symbol Theory

How concepts are represented in the human mind has long been a central question in cognitive science. Traditional propositional symbol theory posits that cognitive systems are composed of propositions that interconnect to form propositional networks (Harnad, 1990; Wang Ruiming et al., 2005). However, this theory faces several problems, including the lack of direct empirical evidence for propositional symbols and its inability to provide satisfactory explanations for the transformation process from perceptual objects to propositional symbols.

Received: July 16, 2021. *Supported by the National Social Science Fund Project (18BYY121). Corresponding author: Wang Ruiming, E-mail: wangrm@scnu.edu.cn

In response to these limitations, Barsalou (1999, 2008) proposed the innovative perceptual symbol system theory, which argues that conceptual representation involves sensorimotor simulation of the entities that concepts denote. During concept acquisition, lexical meanings and their associated perceptual contextual information form experiential traces that are stored together in the brain (Zwaan & Madden, 2005). The activation of these traces during conceptual representation is termed sensorimotor simulation (Strozyk et al., 2019). Different types of concepts activate different information during sensorimotor simulation: concrete concepts primarily activate object and scene information, whereas abstract concepts more strongly activate contextual and introspective information (Barsalou, 1999, 2008).

The primary innovation of perceptual symbol theory lies in its redefinition of the relationship between internal symbols and external stimulus prototypes. The theory maintains that this relationship is analogous and perceptual (Fodor, 1975; Pylyshyn, 1984; Barsalou, 1999, 2008; Mo Lei et al., 2006; Wang Ruiming et al., 2006). For instance, in representing the concept "table," perceptual symbol theory suggests that conceptual representation activates not only the abstract symbol for "table" but also perceptual information representing the table prototype, such as its shape, size, color, and orientation (Barsalou, 1999). Furthermore, the theory proposes that concepts may represent different information depending on semantic context—for example, "oval conference tables" in meeting room contexts versus "small round or square tables" in dining contexts.

Perceptual symbol theory considers the activation of sensorimotor information fundamental to conceptual representation. Classic studies supporting this view create conditions where conceptual information either matches or mismatches perceptual information to examine how perceptual information influences conceptual representation. Numerous findings demonstrate that matching conditions facilitate concept processing, indicating that conceptual representation is grounded in sensorimotor information and that the two share representational resources (Liu Wenjuan et al., 2015). For example, Zwaan and Yaxley (2003) employed pairs of concrete nouns with inherent vertical spatial relationships (e.g., "tree crown–tree root") presented vertically on screen. The spatial positions of the nouns either matched or mismatched their real-world spatial relationships. In matching trials, words appeared in positions consistent with their referents' actual spatial locations (e.g., "tree crown" above "tree root"), while mismatching trials presented them in inconsistent positions (e.g., "road" above "car"). Participants judged whether the two words were semantically related. According to perceptual symbol theory, processing these noun pairs involves sensorimotor simulation of their spatial information. When the presented spatial positions align with real-world positions, processing should be easier than in mismatching conditions. Results confirmed this prediction, showing significantly faster semantic relatedness judgments in matching conditions, thereby supporting perceptual symbol theory. Additional research has similarly revealed that conceptual representation processes activate sensorimotor information (Zwaan et al., 2002; Bergen et al., 2010).

Among perceptual information, spatial information exerts the most profound influence on concept processing. Humans inhabit physical space, and spatial concepts such as "up-down," "left-right," "high-low," and "inside-outside" constitute the most fundamental conceptual categories established through interaction with the objective world (Lakoff & Johnson, 1980). When acquiring concrete concepts containing implicit spatial information, individuals repeatedly associate abstract linguistic concepts with concrete objects, during which spatial information may be stored alongside conceptual information. Consequently, spatial information is simulated during conceptual representation. Previous studies using Stroop paradigms have found that processing concrete concepts with implicit spatial orientations activates spatial location information (Lachmair et al., 2011; Dudschig et al., 2012; Dudschig et al., 2013; Dudschig & Kaup, 2017). Therefore, the present study focuses on spatial information in concrete concepts, using the classic semantic relatedness judgment paradigm and its variants to investigate the activation of spatial information during representation of concrete concepts with implicit spatial features and its influencing factors.

Although substantial evidence supports that concrete concept representation activates sensorimotor information (Barsalou, 1999; Glenberg & Kaschak, 2002; Glenberg & Gallese, 2012; Zwaan & Madden, 2005), no consensus exists regarding the conditions under which sensorimotor information is activated during conceptual representation. This study systematically addresses this question by examining linguistic and contextual factors that influence sensorimotor simulation. First, we investigate language type (first vs. second language) as a factor affecting sensorimotor simulation in concrete concepts. Since fundamental qualitative differences exist between first and second language representations, these differences may lead to distinct patterns of sensorimotor simulation (Jared et al., 2013; Zhao et al., 2020). Therefore, we first examine this underlying factor of language type. Among contextual factors, spatial perceptual context and semantic perceptual context significantly influence sensorimotor simulation in concrete concepts. Sensorimotor simulation serves as a bridge and representational form connecting perception and semantics (Barsalou, 1999). Thus, investigating the external conditions for sensorimotor simulation requires examining both aspects. Spatial perception represents the most fundamental conceptual category humans acquire and holds significant importance for linguistic representation. Moreover, numerous studies on perceptual symbol representation have focused primarily on spatial perceptual contexts (Barsalou et al., 2018; Estes & Barsalou, 2018). Additionally, the depth of semantic processing directly affects the directionality of mutual influence between emotional concept processing and emotional face perception (Liu Wenjuan et al., 2016). Therefore, it is necessary to examine how semantic contextual information influences the occurrence of sensorimotor simulation.

Previous research on sensorimotor simulation in concrete concept representation has focused primarily on first language. However, substantial differences exist between first and second language acquisition. First language is typically acquired gradually in daily life environments, rich with personal experiences and emotional engagement, which may enable individuals to store relevant sensorimotor information during concept acquisition and consequently engage in sensorimotor simulation during conceptual representation. In contrast, second language acquisition (e.g., English in Chinese educational contexts) often emphasizes equivalent translation between Chinese and English concepts, potentially leading to the formation of abstract symbolic representations without corresponding perceptual information.

Prior research has also found that language type influences sensorimotor simulation during conceptual representation. Eilola and Havelka (2010) employed a Stroop color-word naming task and found that first language learners showed higher skin conductance responses to negative and taboo words compared to neutral and positive words, whereas this difference did not emerge in second language learners. However, many studies have found similar sensorimotor simulation in second language processing. Dudschig, de la Vega, and Kaup (2014) studied German-English bilinguals using a Stroop paradigm, presenting second language words with implicit spatial meanings (e.g., "star" and "mole") and emotional words (e.g., "happy" and "sad"), requiring forward or backward hand movements for color responses. Results showed that despite being task-irrelevant, word meanings elicited sensorimotor simulation similar to that in first language. Grauwe et al. (2014) investigated Dutch-German bilinguals using fMRI, revealing similar brain activation patterns between highly proficient bilinguals and monolinguals, with both first and second languages activating sensorimotor brain regions.

However, the bilingual participants in these second language studies were German-English or Dutch-English bilinguals, representing same-language-family pairs with minimal differences in cultural background and linguistic features. This similarity may facilitate cross-linguistic transfer of perceptual symbols. Chinese-English bilinguals (cross-language-family bilinguals) exhibit substantial differences in linguistic and cultural backgrounds. Do they also show perceptual symbol transfer? Costa et al. (2005) found that bilinguals' semantic access to second language depends on lexical similarity between first and second languages. Wang Ruiming et al. (2010) further demonstrated that late proficient Chinese-English bilinguals accessing third language (Japanese) did not activate English words dissimilar to Japanese, but did activate English words similar to French when accessing French. Recent research on bilingual orthographic transfer found that orthographic skills and experience can transfer between alphabetic scripts (e.g., English and French) to varying degrees, but not between alphabetic and logographic scripts (e.g., English and Chinese). This may result from higher overlap in orthographic rules and phonology-semantics structural relationships between alphabetic scripts compared to the lower overlap between alphabetic and logographic scripts (Xie Jiushu et al., 2022). The primary difference between Chinese-English bilinguals and bilinguals mastering two alphabetic scripts (e.g., German-English) lies in their distinct writing systems, with entirely different matching rules between graphic forms, phonology, and semantics (Lewis et al., 2015). Perceptual symbol representation formation depends on these matching relationships among orthography, phonology, and semantics. For instance, hostile speech sounds are sharper, while positive speech sounds are softer (Nielsen & Rendall, 2013). However, previous research has not systematically examined whether Chinese-English bilinguals exhibit similar perceptual symbol representations in both first and second languages. Therefore, Experiment 1 of this study employs the semantic relatedness judgment paradigm to investigate whether sensorimotor simulation in processing concrete concepts with relative spatial information is influenced by language type (first vs. second language).

Additionally, previous research has found that different perceived contexts affect sensorimotor simulation occurrence. In Lebois et al. (2014), participants read words with implicit spatial information and made upward or downward keypress responses. Spatial congruency effects did not always emerge; rather, participants activated sensorimotor information only when the context highlighted such information. However, prior research has rarely examined how contextual factors influence sensorimotor simulation during conceptual representation. Therefore, Experiment 2 investigates whether sensorimotor simulation during conceptual representation is automatic or strategic, examining both spatial perceptual context (strength of spatial perceptual cues) and semantic perceptual context (depth of semantic processing).

Previous research indicates that different perceptual information influences concept processing. Liu Wenjuan et al. (2016) presented emotional face pictures followed by emotional words to examine how perceptual information affects conceptual information. Results demonstrated mutual influence between perceptual and conceptual processing. According to perceptual symbol theory, sensorimotor information processing and conceptual information processing share overlapping neural mechanisms, with a bidirectional relationship: conceptual processing influences perceptual processing, and perceptual processing influences conceptual processing. Liu Wenjuan et al. (2016) provided support for this theory. Since perceptual information differences affect conceptual representation, a question remains unanswered: Does sensorimotor simulation induced by the semantic relatedness judgment paradigm result from automatic processing during conceptual representation or from strategic processing prompted by vertical spatial cues? This question has not been systematically investigated. Previous research found that adjusting perceptual salience influences strategic processing but has minimal impact on automatic processing (Lakens, 2012; Proctor & Xiong, 2015). Therefore, Experiment 2a uses the semantic relatedness judgment paradigm, creating different vertical spatial perceptual contexts to examine whether sensorimotor simulation during conceptual representation is automatic or strategic.

Does activation of sensorimotor information during conceptual representation also depend on semantic perceptual context? If individuals can still engage in sensorimotor simulation under conditions of very low semantic depth or when semantics are task-irrelevant, this would indicate high automaticity of sensorimotor information in conceptual processing. Conversely, it would suggest strategic processing. Previous research has focused primarily on concepts with typical absolute vertical spatial features, such as "sun" and "valley." Lachmair et al. (2011) presented location words strongly associated with upward or downward positions (e.g., "roof" or "root"), requiring participants to read words and make keypress responses. Across experiments, researchers varied the depth of semantic retrieval and whether responses required vertical movement or stationary keypresses. Results showed significantly faster responses when words' implicit spatial information matched response directions, indicating activation of spatial information during word processing. Dudschig, de la Vega, Filippis, and Kaup (2014) used a masking paradigm to obscure word awareness, yet observed reversed compatibility effects between space and action, suggesting highly automatic sensorimotor simulation during conceptual representation that persists even with weak semantic extraction.

However, these results may stem from using words with absolute vertical spatial relationships, which can automatically activate sensorimotor information. Under strong spatial information conditions, individuals may simulate vertical spatial information automatically without fully accessing semantics. In contrast, relative spatial concepts (e.g., "rose–vase") often lack strong absolute vertical relationships, possessing only relative vertical positions. This may require greater semantic reliance to determine spatial relationships and thereby trigger sensorimotor simulation. Therefore, Experiment 2b examines semantic perceptual context by manipulating semantic processing depth through semantic relatedness judgment (deeper semantic processing) and pseudoword judgment (shallower semantic processing) dual tasks to investigate whether sensorimotor simulation is strategic or automatic. In traditional semantic relatedness judgment tasks, participants may detect whether the vertical relationship of implicit spatial words matches real-world situations, potentially triggering strategic responses. The pseudoword judgment task, requiring identification of pseudowords without complete semantic access, better reveals whether sensorimotor simulation occurs automatically, thereby clarifying the automatic versus strategic nature of sensorimotor simulation.

In summary, this study systematically investigates the influence of language and context on sensorimotor simulation during concrete concept representation. Specifically, we first examine how language type (first vs. second language) affects sensorimotor simulation occurrence. Then, we investigate how spatial perceptual context and semantic perceptual context—both closely related to concrete concepts—influence sensorimotor simulation. To this end, we designed three experiments. Experiment 1 uses the semantic relatedness judgment paradigm to examine whether similar sensorimotor simulation occurs in first and second language concrete concept representation. Experiments 2a and 2b employ the semantic relatedness judgment paradigm and its variants to explore how spatial perceptual information and semantic perceptual information affect sensorimotor simulation during concrete concept representation.

Experiment 1

2.1 Purpose

This experiment investigates whether sensorimotor simulation occurs during second language conceptual representation and whether any differences exist between first and second language processing.

2.2 Methods

2.2.1 Participants

Forty-eight English major university students who had passed the Test for English Majors Band 4 (TEM-4) participated in the experiment (mean age = 22.60 years). All participants had normal or corrected-to-normal vision, were native Chinese speakers, and had no reading disabilities. They received modest compensation after completing the experiment.

2.2.2 Design

A 2 (spatial relation: congruent vs. incongruent) × 2 (language type: Chinese vs. English) within-subjects design was employed, with reaction time and accuracy as dependent variables.

2.2.3 Materials

Eighty English word pairs were created, including 20 semantically related experimental pairs, 20 semantically related filler pairs, and 40 semantically unrelated filler pairs. The 20 semantically related experimental pairs contained vertical spatial relationships (e.g., "rose–vase," "table–book") and were divided into two material sets matched for spatial relation consistency. Both sets contained these 20 pairs, with half presenting the word pairs in an order congruent with their real-world spatial relationships (e.g., "rose–vase") and half in an incongruent order (e.g., "table–book"). The 20 semantically related filler pairs were only semantically related without vertical spatial relationships (e.g., "campus–student"). Including these filler pairs ensured that 40 pairs required "yes" responses and 40 required "no" responses, balancing response types. More importantly, the filler pairs prevented participants from guessing the experimental purpose by diverting attention from the implicit vertical spatial relationships.

Fifteen university students who did not participate in the formal experiment rated the Chinese word pairs on semantic relatedness, familiarity, and concreteness using 7-point scales. For relatedness ratings, participants evaluated the degree of semantic association between words. For familiarity ratings, they assessed their familiarity with each word in the pair. For concreteness ratings, they evaluated how concrete each word's referent was. Results showed: (1) Significant differences among semantically related experimental pairs (M = 6.03, SD = 0.42), semantically related filler pairs (M = 5.92, SD = 0.39), and semantically unrelated filler pairs (M = 1.55, SD = 0.31), F(2, 79) = 1503.67, p < 0.001, with no significant difference between experimental and filler related pairs, F(1, 39) = 0.70, p = 0.409. (2) No significant difference in familiarity between experimental pairs (M = 6.22, SD = 0.39) and filler related pairs (M = 6.25, SD = 0.38), F(1, 39) = 0.05, p = 0.827. (3) No significant difference in concreteness between experimental pairs (M = 6.62, SD = 0.34) and filler related pairs (M = 6.61, SD = 0.25), F(1, 39) = 0.03, p = 0.874. (4) No significant difference in stroke count between experimental pairs (M = 17.43, SD = 3.61) and filler related pairs (M = 16.45, SD = 2.87), F(1, 39) = 0.89, p = 0.351.

English materials were translation equivalents of the Chinese materials. Seventeen university students who did not participate in the formal experiment rated the English materials. Results showed: (1) Significant differences among semantically related experimental pairs (M = 5.22, SD = 0.95), semantically related filler pairs (M = 5.42, SD = 0.63), and semantically unrelated filler pairs (M = 2.20, SD = 0.47), F(2, 79) = 228.24, p < 0.001, with no significant difference between experimental and filler related pairs, F(1, 39) = 0.66, p = 0.420. (2) No significant difference in familiarity between experimental pairs (M = 6.34, SD = 0.72) and filler related pairs (M = 6.54, SD = 0.24), F(1, 39) = 1.39, p = 0.245. (3) No significant difference in concreteness between experimental pairs (M = 6.28, SD = 0.38) and filler related pairs (M = 6.37, SD = 0.33), F(1, 39) = 0.56, p = 0.461. (4) No significant difference in letter count between experimental pairs (M = 4.70, SD = 0.88) and filler related pairs (M = 5.33, SD = 1.52), F(1, 39) = 2.54, p = 0.119.

2.2.4 Procedure

The experiment was programmed using E-Prime 1.2. It included Chinese and English blocks that all participants completed, with block order counterbalanced across participants. Each trial began with a 500 ms red fixation cross "+" at the center of a white screen to focus attention. After the fixation disappeared, two Chinese two-character words (Chinese block) or English words (English block) appeared simultaneously above and below the fixation point. Participants judged whether the words were semantically related, pressing the "J" key for related and the "F" key for unrelated. The mapping between keys and responses was counterbalanced across participants. Participants were instructed to respond as quickly and accurately as possible. After a response or 3000 ms without response, an 800 ms blank screen appeared before the next trial began. Participants completed practice trials with feedback before the formal experiment and had to reach a certain accuracy threshold to proceed. To ensure sufficient trial numbers, materials were repeated three times, with each participant completing 480 trials (half Chinese, half English). The experimental procedure is illustrated in Figure 1.

2.3 Results and Analysis

All participants achieved accuracy above 80%, so no data were excluded. Only data from the 20 semantically related experimental pairs were analyzed. Incorrect responses, reaction times shorter than 400 ms or longer than 2000 ms, and responses beyond 2.5 standard deviations were excluded, accounting for 9.9% of data. Reaction times for each condition are presented in Table 1.

Table 1 Reaction Times (M±SD) for Different Spatial Relations by Language Type

Language Type Spatial Congruent (ms) Spatial Incongruent (ms) Chinese 784.00±82.32 799.02±94.00 English 1004.66±110.66 1030.88±144.82

Analysis using SPSS 21.0 revealed a significant main effect of spatial relation, F(1, 47) = 4.28, p = 0.044, η²p = 0.08, with faster responses in spatially congruent than incongruent conditions. The main effect of language type was significant, F(1, 47) = 265.71, p < 0.001, η²p = 0.85, with faster responses in Chinese than English. The interaction between spatial relation and language type was not significant, F(1, 47) = 0.36, p = 0.551.

Accuracy data are presented in Table 2. Analysis showed no significant main effect of spatial relation, F(1, 47) = 0.10, p = 0.754. The main effect of language type was significant, F(1, 47) = 28.41, p < 0.001, η²p = 0.38. The interaction between spatial relation and language type was significant, F(1, 47) = 4.11, p = 0.048, η²p = 0.08. Simple effects analysis revealed no significant spatial congruency difference in English, F(1, 47) = 0.41, p = 0.404, but a significant difference in Chinese, F(1, 47) = 6.76, p = 0.012, η²p = 0.17, with higher accuracy for congruent than incongruent pairs.

Table 2 Accuracy Rates (M±SD) for Different Spatial Relations by Language Type

Language Type Spatial Congruent Spatial Incongruent Chinese 0.97±0.04 0.92±0.07 English 0.95±0.04 0.93±0.07

Experiment 1 demonstrated sensorimotor simulation in first language processing in both reaction time and accuracy, whereas second language processing showed simulation only in reaction time. Combined results indicate that sensorimotor simulation occurs in both Chinese-English bilingual conditions, but first language shows advantages in conceptual representation, possibly due to higher proficiency. Experiment 2 further investigates whether contextual factors influence sensorimotor simulation during conceptual representation, examining spatial perceptual context (Experiment 2a) and semantic perceptual context (Experiment 2b).

Experiment 2a

3.1 Purpose

This experiment investigates how different spatial perceptual contexts influence sensorimotor simulation during conceptual representation.

3.2 Methods

3.2.1 Participants

Forty-eight university students who did not participate in Experiment 1 were recruited (mean age = 20.00 years). All had normal or corrected-to-normal vision, were native Chinese speakers, and had no reading disabilities. They received modest compensation after the experiment.

3.2.2 Design

A 2 (spatial relation: congruent vs. incongruent) × 2 (spatial perception: strong vs. weak) within-subjects design was employed, with reaction time and accuracy as dependent variables.

3.2.3 Materials

The Chinese materials from Experiment 1 were used, consisting of 80 word pairs: 20 semantically related experimental pairs, 20 semantically related filler pairs, and 40 semantically unrelated filler pairs. Half of each material type was randomly assigned to the strong spatial perception condition and half to the weak spatial perception condition. Four material sets were created based on the four experimental conditions for each semantically related experimental pair. Each participant was randomly assigned to one material set.

3.2.4 Procedure

As illustrated in Figure 2, all participants completed two blocks differing in the vertical distance between words on screen. The experiment manipulated the salience of vertical spatial information by varying inter-word distance. In the long vertical distance block, words appeared farther apart vertically, making spatial relationships more salient. In the short vertical distance block, words appeared closer together, making spatial relationships less detectable. This manipulation of perceptual information strength tested its influence on sensorimotor simulation in concrete concept processing. Within each block, half the trials presented words in positions congruent with their real-world spatial relationships, and half presented them incongruently. Due to visual fatigue reported in Experiment 1 from prolonged viewing of white screens, Experiment 2 used black backgrounds with white text and white fixation crosses. To ensure sufficient trial numbers, each word pair was repeated five times, yielding 400 trials per participant. Response key mappings and block orders were counterbalanced.

3.3 Results and Analysis

Data were processed following the same procedure as Experiment 1. All participants achieved accuracy above 80%. Only data from the 20 semantically related experimental pairs were analyzed. Incorrect responses, reaction times shorter than 400 ms or longer than 2000 ms, and responses beyond 2.5 standard deviations were excluded, accounting for 7.6% of data. Reaction times for each condition are presented in Table 3.

Table 3 Reaction Times (M±SD) for Different Spatial Relations by Spatial Perceptual Context

Spatial Perceptual Context Spatial Congruent (ms) Spatial Incongruent (ms) Strong 732.53±97.06 744.35±88.56 Weak 672.34±78.68 679.83±75.84

Analysis revealed a significant main effect of spatial relation, F(1, 47) = 7.65, p = 0.008, η²p = 0.14, with faster responses in congruent than incongruent conditions. The main effect of spatial perception was significant, F(1, 47) = 58.61, p < 0.001, η²p = 0.56, with faster responses in weak than strong spatial perception conditions. The interaction between spatial relation and spatial perception was not significant, F(1, 47) = 0.21, p = 0.647.

Accuracy data are presented in Table 4. Analysis showed no significant main effect of spatial relation, F(1, 47) = 0.20, p = 0.659. The main effect of spatial perception was significant, F(1, 47) = 10.50, p = 0.002, η²p = 0.18, with higher accuracy in weak than strong spatial perception conditions. The interaction was not significant, F(1, 47) = 0.14, p = 0.712.

Table 4 Accuracy Rates (M±SD) for Different Spatial Relations by Spatial Perceptual Context

Spatial Perceptual Context Spatial Congruent Spatial Incongruent Strong 0.94±0.07 0.94±0.07 Weak 0.96±0.05 0.97±0.03

Experiment 2a results show significantly faster reaction times for spatially congruent than incongruent conditions, with no significant interaction between spatial relation and spatial perception. This indicates that processing implicit spatial information in word pairs is not affected by perceptual context factors. According to perceptual symbol theory, understanding implicit spatial information requires reactivating spatial perceptual information and activating semantic information. The lack of influence from spatial perceptual reactivation suggests that processing implicit spatial information in word pairs may be automatic.

Experiment 2b

4.1 Purpose

This experiment investigates how semantic processing depth influences sensorimotor simulation during conceptual representation.

4.2 Methods

4.2.1 Participants

Forty-eight university students who did not participate in previous experiments were recruited (mean age = 21.00 years). All had normal or corrected-to-normal vision, were native Chinese speakers, and had no reading disabilities. They received modest compensation after the experiment.

4.2.2 Design

A 2 (spatial relation: congruent vs. incongruent) × 2 (task type: semantic relatedness judgment vs. pseudoword judgment) within-subjects design was employed, with reaction time and accuracy as dependent variables.

4.2.3 Materials

The Chinese materials from Experiment 1 were used, consisting of 80 pairs: 20 semantically related experimental pairs, 20 semantically related filler pairs, and 40 semantically unrelated filler pairs. Half of each material type was randomly assigned to the semantic relatedness judgment task and half to the pseudoword judgment task. For the pseudoword judgment task, one word in each of the original 40 semantically unrelated filler pairs was replaced with a pseudoword. Pseudowords were created following Liu Wenjuan et al. (2016) by reversing character order or randomly combining two Chinese characters without real meaning. Other materials remained identical to Experiment 1. Four material sets were created based on the four experimental conditions for each semantically related experimental pair, with each participant randomly receiving one set.

4.2.4 Procedure

Similar to Experiment 2a, participants completed two blocks: semantic relatedness judgment and pseudoword judgment. In the semantic relatedness block, participants judged whether two words were semantically related. In the pseudoword block, they judged whether a pseudoword was present. Pseudoword positions were randomized: 50% appeared at the top of the screen and 50% at the bottom. To ensure sufficient trial numbers, each word pair was repeated five times, yielding 400 trials per participant. Response key mappings and block orders were counterbalanced.

4.3 Results and Analysis

Data were processed following the same procedure as Experiment 1. All participants achieved accuracy above 80%. Only data from the 20 semantically related experimental pairs were analyzed. Incorrect responses, reaction times shorter than 400 ms or longer than 2000 ms, and responses beyond 2.5 standard deviations were excluded, accounting for 7.2% of data. Reaction times for each condition are presented in Table 5.

Table 5 Reaction Times (M±SD) for Different Spatial Relations by Semantic Perceptual Context

Task Type Spatial Congruent (ms) Spatial Incongruent (ms) Semantic Relatedness Judgment 776.95±99.58 791.62±108.60 Pseudoword Judgment 839.05±116.82 853.53±101.66

Analysis revealed a significant main effect of spatial relation, F(1, 47) = 5.16, p = 0.028, η²p = 0.10, with faster responses in congruent than incongruent conditions. The main effect of task type was significant, F(1, 47) = 49.89, p < 0.001, η²p = 0.52, with faster responses in semantic relatedness than pseudoword judgment. The interaction was not significant, F(1, 47) < 0.001, p = 0.989.

Accuracy data are presented in Table 6. Analysis showed no significant main effect of spatial relation, F(1, 47) = 0.37, p = 0.548. The main effect of task type was significant, F(1, 47) = 6.00, p = 0.018, η²p = 0.11, with lower accuracy in semantic relatedness than pseudoword judgment. The interaction was not significant, F(1, 47) = 0.85, p = 0.362.

Table 6 Accuracy Rates (M±SD) for Different Spatial Relations by Semantic Perceptual Context

Task Type Spatial Congruent Spatial Incongruent Semantic Relatedness Judgment 0.93±0.10 0.95±0.08 Pseudoword Judgment 0.98±0.05 0.97±0.05

Experiment 2b results show significantly faster reaction times for spatially congruent than incongruent conditions, with no significant interaction between task type and spatial relation. This indicates that processing implicit spatial information in word pairs is not modulated by semantic perceptual context. According to perceptual symbol theory, understanding implicit spatial information requires semantic activation. The lack of influence from semantic processing depth suggests that processing implicit spatial information may be automatic. Additionally, pseudoword judgment produced longer reaction times than semantic relatedness judgment, possibly because semantic relatedness tasks involve semantic priming that facilitates processing (e.g., processing "car" facilitates processing "road"), whereas pseudoword judgment lacks such priming effects.

General Discussion

This series of experiments investigated how linguistic and contextual factors influence sensorimotor simulation during concrete concept representation. All three experiments found significant main effects of spatial relation, revealing sensorimotor simulation during concrete concept representation and supporting perceptual symbol theory.

Regarding factors influencing sensorimotor simulation, this study examined both linguistic (first vs. second language) and contextual factors (spatial perceptual context and semantic perceptual context). Results indicate that Chinese-English bilinguals produce sensorimotor simulation in both languages, but first language shows advantages: first language processing demonstrates both faster reaction times and higher accuracy in spatially congruent conditions, whereas second language processing shows only faster reaction times. However, our participants were English majors with high English proficiency. Whether less proficient second language learners would show similar sensorimotor simulation requires further investigation. This study also reveals that sensorimotor simulation during concrete concept processing is highly automatic, occurring even under weak spatial perceptual information conditions and shallow semantic retrieval conditions. In summary, this research demonstrates that sensorimotor simulation in concrete concept processing exhibits cross-linguistic stability and considerable automaticity.

Sensorimotor simulation in first language concrete concept processing has received substantial support. However, second language acquisition differs considerably from first language acquisition, raising questions about whether these differences affect sensorimotor simulation during conceptual representation. Previous research has not reached consistent conclusions. Some studies found weaker sensorimotor simulation in second language, such as Eilola and Havelka (2010) who reported stronger skin conductance responses to negative and taboo words in first language but not second language. Other studies found similar sensorimotor simulation in second language (Dudschig, de la Vega, & Kaup, 2014). Using concrete concepts with relative vertical spatial information, our study investigated whether Chinese-English bilinguals automatically activate spatial information during conceptual representation. Results showed sensorimotor simulation in both languages, but with accuracy advantages in first language, indicating superior perceptual symbol simulation in first language.

How then is second language sensorimotor information acquired? Is it transferred from first language concepts or reacquired during second language learning? Although our study did not directly address this question, reviewing previous research allows preliminary speculation to guide future studies. Prior research suggests that merely acquiring semantic connections between second language words and first language translations is insufficient for second language sensorimotor simulation capability; individuals must acquire corresponding perceptual experiences while learning second language vocabulary to generate sensorimotor simulation. Günther et al. (2017) presented participants with novel words, teaching them only semantic information through translation equivalents or embedding them in first language sentences. Despite accurate word meaning identification, sensorimotor simulation failed to emerge. In contrast, Öttl et al. (2017) taught participants artificial words while simultaneously presenting them in upper or lower visual fields, enabling acquisition of both word meaning and vertical spatial information. This resulted in vertical spatial simulation during conceptual representation. Comparing these studies suggests that second language perceptual information is reacquired during second language learning rather than transferred from first language. This explains our finding of sensorimotor simulation in second language processing: although English majors acquire second language vocabulary semantics primarily in classrooms, their attempts to use the language in real or simulated scenarios may enable them to develop sensorimotor simulation capabilities.

Experiment 1 also found first language advantages over second language in conceptual representation, manifesting as both faster reaction times and higher accuracy in spatially congruent conditions. Previous research has similarly demonstrated first language advantages. Baumeister et al. (2017) found weaker emotional experiences for second language emotion words compared to first language, possibly due to weaker emotional memory in second language. Chen et al. (2020) examined sensorimotor simulation during processing of spatially implicit sentences in Cantonese-Mandarin-English trilinguals, finding spatial simulation in all languages but more pronounced spatial congruency effects in first language than in second and third languages. Vukovic and Shtyrov (2014) used EEG to dynamically measure mu wave desynchronization in German-English bilinguals, finding motor cortex activation during both first and second language comprehension, but significantly weaker activation for second language. This imbalance may relate to second language proficiency. Highly proficient bilinguals have more stable connections between vocabulary and perceptual information, more similar to first language patterns, producing equivalent sensorimotor simulation. As second language proficiency increases, these connections strengthen, gradually balancing simulation across languages. Bergen et al. (2010) found that more proficient second language learners showed more native-like sensorimotor simulation in word-picture matching tasks. Harris et al. (2003) reported stronger autonomic responses to taboo and negative words in first language among late bilinguals, attributing this effect to second language proficiency. Thus, improved second language proficiency facilitates sensorimotor simulation during second language conceptual representation.

Perceptual symbol theory was primarily constructed based on first language representation and has not elaborated extensively on second language conceptual representation. However, as global communication intensifies, second language learning has become essential for adapting to globalization. To better understand second language conceptual representation and identify effective second language learning methods, systematically investigating sensorimotor simulation in second language is imperative. Experiment 1 of this study found similar perceptual symbol representations in first and second languages, supporting perceptual symbol theory. This is the first demonstration of sensorimotor simulation in second language among cross-language-family bilinguals, offering insights for resolving previous debates. The finding implies that both first and second language learning should be based on sensorimotor simulation. Combining our findings with existing research, we propose that second language sensorimotor simulation is not transferred from first language but rather re-established during second language learning. However, due to inherent first language advantages, second language sensorimotor simulation appears less fluent than first language simulation, representing an important extension of perceptual symbol theory.

Previous research demonstrates that different perceptual contextual information influences concept processing. Lebois et al. (2014) found that spatial congruency effects did not always emerge when participants read spatially implicit words and made upward or downward responses; rather, sensorimotor information was activated only when the context highlighted such information. However, few studies have examined how contextual factors influence sensorimotor simulation in concrete concept processing, and perceptual symbol theory has not detailed how contextual factors affect simulation. Liu Wenjuan et al. (2016) demonstrated mutual influence between perceptual and conceptual processing, providing support for perceptual symbol theory's claim that sensorimotor and conceptual information processing share overlapping neural mechanisms with bidirectional relationships. Experiment 2 of this study advanced this line of research by investigating whether these effects are automatic or strategic. Experiment 2a manipulated vertical spatial perceptual salience and found no effect on sensorimotor simulation, indicating high automaticity. Experiment 2b compared semantic relatedness judgment with pseudoword judgment to examine whether simulation occurs when semantic access is incomplete. Results showed no modulation by semantic processing depth, consistent with previous research. Studies using visual (Dudschig et al., 2013) or auditory (Dunn et al., 2014) presentation of absolute spatial concepts (e.g., "sun," "valley") in pseudoword judgment tasks found that participants' eyes saccaded toward congruent locations. However, these studies focused on concepts with strong absolute spatial information that might enable simulation without complete semantic access. Our study used relative spatial concepts (e.g., "rose–vase") lacking strong absolute spatial relationships. Thus, Experiment 2b reveals that concrete nouns with relative spatial relationships also show high automaticity and are not easily influenced by semantic processing depth.

In conclusion, this study preliminarily explored how linguistic and contextual factors influence sensorimotor simulation in concrete concept processing. Future research should examine non-proficient bilinguals to determine whether they produce similar sensorimotor simulation across languages. Additionally, this study focused on vertical spatial relationships; future research could investigate horizontal spatial relationships, such as associations between positive words and the right hand and negative words with the left hand.

This study systematically investigated how linguistic (language type) and contextual (perceptual context) factors influence sensorimotor simulation in concrete concept processing. Results demonstrate that proficient Chinese-English bilinguals produce sensorimotor simulation in both languages, though first language shows advantages. Moreover, sensorimotor simulation emerged automatically during concrete concept processing under both weak spatial information and shallow semantic information conditions. These findings support perceptual symbol theory, indicating that sensorimotor simulation in concrete concept processing exhibits both stability and automaticity. This discovery provides new empirical evidence for perceptual symbol theory and expands its application scope.

References

Barsalou, L. W. (1999). Perceptions of perceptual symbols. Behavioral and Brain Sciences, 22(04), 637–660.

Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59(1), 617–645.

Barsalou, L. W., Dutriaux, L., & Scheepers, C. (2018). Moving beyond the distinction between concrete and abstract concepts. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1752).

Baumeister, J. C., Foroni, F., Conrad, M., Rumiati, R. I., & Winkielman, P. (2017). Embodiment and emotional memory in first vs. second language. Frontiers in Psychology, 8(8), 394.

Bergen, B., Lau, T. T. C., Narayan, S., Stojanovic, D., & Wheeler, K. (2010). Body part representations in verbal semantics. Memory & Cognition, 38(7), 969–981.

Chen, D., Wang, R., Zhang, J., & Liu, C. (2020). Perceptual Representations in L1, L2 and L3 Comprehension: Delayed Sentence–Picture Verification. Journal of Psycholinguistic Research, 49(1), 41–57.

Costa, A., Santesteban, M., & Cano, A. (2005). On the facilitatory effect of cognate words in bilingual speech production. Brain and Language, 94(9), 94–103.

Dudschig, C., Lachmair, M., de la Vega, I., De Filippis, M., & Kaup, B. (2012). From top to bottom: Spatial shifts of attention caused by linguistic stimuli. Cognitive Processing, 13(1), 151–154.

Dudschig, C., Souman, J., Lachmair, M., de la Vega, I., & Kaup, B. (2013). Reading "sun" and looking up: The influence of language on saccadic eye movements in the vertical dimension. PloS One, 8(2), e56872.

Dudschig, C., de la Vega, I., De Filippis, M., & Kaup, B. (2014). Language and vertical space: On the automaticity of language-action interconnections. Cortex, 58, 151–160.

Dudschig, C., de la Vega, I., & Kaup, B. (2014). Embodiment and second-language: Automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical Stroop paradigm. Brain and Language, 132, 14–21.

Dudschig, C., & Kaup, B. (2017). Is it all task-specific? The role of binary responses, verbal mediation, and saliency for eliciting language-space associations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(2), 259–270.

Dunn, B., Kamide, Y., & Scheepers, C. (2014). Hearing "moon" and looking up: Word-related spatial associations facilitate saccades to congruent locations. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 36, No. 36).

Eilola, T. M., & Havelka, J. (2010). Behavioural and physiological responses to the emotional and taboo Stroop tasks in native and non-native speakers of English. International Journal of Bilingualism, 15(3), 353–379.

Estes, Z., & Barsalou, L. W. (2018). A Comprehensive Meta-Analysis of Spatial Interference From Linguistic Cues: Beyond Petrova et al. (2018). Psychological Science, 29(9), 1558–1564.

Fodor, J. A. (1975). The Language of Thought. Cambridge, MA: Harvard University Press.

Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565.

Glenberg, A. M., & Gallese, V. (2012). Action-based language: A theory of language acquisition, comprehension, and production. Cortex, 48(7), 905–922.

Grauwe, S. D., Willems, R. M., Rueschemeyer, S. A., Lemhöfer, K., & Schriefers, H. (2014). Embodied language in first- and second-language speakers: Neural correlates of processing motor verbs. Neuropsychologia, 56, 334–349.

Günther, F., Dudschig, C., & Kaup, B. (2018). Symbol grounding without direct experience: Do words inherit sensorimotor activation from purely linguistic context? Cognitive Science, 42, 336–374.

Harnad, S. (1990). The Symbol Grounding Problem. Physica D: Nonlinear Phenomena, 42(1-3), 335–346.

Harris, C. L., Ayçíçeğí, A., & Gleason, J. B. (2003). Taboo words and reprimands elicit greater autonomic reactivity in a first language than in a second language. Applied Psycholinguistics, 24(4), 561–579.

Jared, D., Pei Yun Poh, R., & Paivio, A. (2013). L1 and L2 picture naming in Mandarin–English bilinguals: A test of Bilingual Dual Coding Theory. Bilingualism: Language and Cognition, 16(2), 383–396.

Lachmair, M., Dudschig, C., Filippis, M. D., Vega, I., & Kaup, B. (2011). Root versus roof: Automatic activation of location information during word processing. Psychonomic Bulletin & Review, 18(6), 1180–1188.

Lakens, D. (2012). Polarity correspondence in metaphor congruency effects: Structural overlap predicts categorization times for bipolar concepts presented in vertical space. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(3), 726–736.

Lakoff, G., & Johnson, M. (1980). Metaphors We Live By. Chicago, IL: University of Chicago Press.

Lebois, L. A., Wilson-Mendenhall, C. D., & Barsalou, L. W. (2014). Are automatic conceptual cores the gold standard of semantic processing? The context-dependence of spatial meaning in grounded congruency effects. Cognitive Science, 39(8), 1764–1801.

Lewis, M. P., Simons, G. F., & Fennig, C. D. (2015). Ethnologue: Languages of the World (18th ed.). SIL International.

Liu, W. J., Li, Y., & Wang, R. M. (2015). The Role of Sensorimotor Information in Conceptual Representation. Journal of Psychological Science, 38(6), 1347–1352.

Liu, W. J., Shen, M. Q., Li, Y., & Wang, R. M. (2016). The interaction between emotional concept processing and emotional face perception. Acta Psychologica Sinica, 48(2), 163–173.

Mo, L., Wu, L. M., & Wang, R. M. (2006). Spatial Iconicity Affects Semantic Relatedness Judgment. Journal of Psychological Science, 29(4), 770–773.

Nielsen, A. K. S., & Rendall, D. (2013). Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon. Canadian Journal of Experimental Psychology, 67(2), 153–163.

Öttl, B., Dudschig, C., & Kaup, B. (2017). Forming associations between language and sensorimotor traces during novel word learning. Language and Cognition, 9(1), 156–171.

Proctor, R. W., & Xiong, A. (2015). Polarity Correspondence as a General Compatibility Principle. Current Directions in Psychological Science, 24(6), 446–451.

Pylyshyn, Z. W. (1984). Computation and Cognition. Cambridge, MA: MIT Press.

Strozyk, J. V., Dudschig, C., & Kaup, B. (2019). Do I need to have my hands free to understand hand-related language? Investigating the functional relevance of experiential simulations. Psychological Research, 83(3), 406–418.

Vukovic, N., & Shtyrov, Y. (2014). Cortical motor systems are involved in second-language comprehension: Evidence from rapid mu-rhythm desynchronization. NeuroImage, 102, 695–703.

Wang, R. M., Mo, L., Li, L., Wang, S. P., & Wu, J. (2005). Perceptual symbol representation and propositional symbol representation in language comprehension. Acta Psychologica Sinica, 37(2), 143–150.

Wang, R. M., Mo, L., Wu, L. M., & Li, L. (2006). The Influence of Spatial Information on Semantic Relatedness Judgment. Journal of Psychological Science, 29(6), 1320–1323.

Wang, R. M., Zhang, J. T., Li, L., & Mo, L. (2010). The Role of Bilinguals' Second Language in Semantic Access of Third Language. Journal of Psychological Science, 33(4), 853–856.

Xie, J. S., Shen, G. Y., & Huang, Y. L. (2022). Debates and integration of bilinguals' cross-language orthographic transfer. Journal of Psychological Science.

Zhao, T. Y., Huang, Y. L., Chen, D. G., Jiao, L., Ramos, F. M., Wang, R. M., & Xie, J. S. (2020). The modality switching costs of Chinese-English bilinguals in the processing of L1 and L2. Quarterly Journal of Experimental Psychology, 73(3), 396–412.

Zwaan, R. A., Stanfield, R., & Yaxley, R. H. (2002). Language comprehenders mentally represent the shapes of objects. Psychological Science, 13(2), 168–171.

Zwaan, R. A., & Yaxley, R. H. (2003). Spatial iconicity affects semantic relatedness judgments. Psychonomic Bulletin & Review, 10(4), 954–958.

Zwaan, R. A., & Madden, C. J. (2005). Embodied sentence comprehension. In Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thinking (pp. 224–245).

Appendix: Experimental Materials

Experiment 1 and 2 Materials

English Materials:
bridge, island, cloud, farmer, stage, desert, pants, table, ground, campus, water, lawyer, textbook, writer, fridge, mouse, chocolate, sheep, orange, noodle, cinema, river, stove, shoulder, field, actor, camel, raincoat, shirt, passenger, plant, student, judge, schoolbag, novel, television, candy, director, grape, dumpling, cloth, pizza, tiger, comedy, bedroom, dollar, camera, museum, dictionary, forest, clerk, secretary, calculator, knife, president, hotel, photo, snake, machine, channel, station, spokesman, highland, poster, gardener, attendant, chemist, subway, potato, stone, spider, vegetable, donkey, flood, housewife, lightning, onion, earth, journalist, radio, glove, brush, temple, passport, toilet, switch, hotpot, spoon, pepper, storm, whale, panda, bamboo, balloon

Note: Items 1-20 are semantically related experimental pairs, 21-40 are semantically related filler pairs, and 41-80 are semantically unrelated filler pairs.

Chinese Materials:
Note: Items 1-20 are semantically related experimental pairs, 21-40 are semantically related filler pairs, and 41-80 are semantically unrelated filler pairs.

Experiment 2b Materials

Note: Items 1-20 are semantically related experimental pairs, 21-40 are semantically related filler pairs, 41-60 are semantically unrelated filler pairs, and 61-80 are pairs containing pseudowords.