Does Linguistic Relativity Hypothesis Apply on ChatGPT Responses? Yes, It Does

Abstract

We present the first comprehensive, end-to-end quantitative evaluation of the linguistic relativity hypothesis in AI-generated text, using ChatGPT-4o mini to generate responses to 10 culturally salient prompts across 13 typologically diverse languages. Semantic shifts were quantified using pairwise cosine similarity scores computed from multilingual MiniLM sentence embeddings. A one-way analysis of variance (ANOVA) reveals statistically significant variation in semantic alignment across language pairs, with $F(77,702)=2.153$, $p=2.29\times 10^{-7}$, and effect size ${\eta }^2=0.191$. These results are further supported by a non-parametric Kruskal–Wallis test yielding $H=176.208$, $p=9.59\times 10^{-10}$, indicating robust differences in distribution. Prompt-specific semantic shifts also exhibit significant variation, as shown by ANOVA results $F(9,770)=24.239$, $p=1.00\times 10^{-36}$, and ${\eta }^2=0.221$. Sentiment polarity analysis using the Polyglot toolkit reveals significant effects of language on sentiment distribution, with $F(12,117)=2.637$, $p=0.0037$, and ${\eta }^2=0.213$. Disaggregated analysis shows that positivity ratios differ by prompt ($F(9,120)=3.621$, $p=0.0005$, ${\eta }^2=0.214$), while negativity scores display even greater divergence across prompts with $F(9,120)=12.755$, $p=4.59\times 10^{-14}$, and ${\eta }^2=0.489$. An unsupervised clustering procedure ($k=3$) classifies languages into three distinct groups based on semantic alignment: (i) high-alignment ($\text{mean similarity}\ge 0.90$), (ii) intermediate ($\text{mean similarity}\approx 0.75-0.85$), and (iii) neutral-tone clusters. Each group exhibits distinctive polarity profiles, with median sentiment polarity ranging from $-0.02$ to $0.11$. These results demonstrate that linguistic structures exert a measurable influence on AI-generated content, underscoring the need for culturally sensitive AI design practices. These results affirm that ChatGPT-4o mini's outputs align with the linguistic relativity hypothesis, clearly illustrating that language structures significantly shape AI-driven interpretation All associated code and data are available in the GitHub repository: https://github.com/ParthaPRay/Liguistic_Relativity_Chatgpt.