Daejeon, July 27–31
This paper is a continuation of the pilot study presented in Florescu (2025), comparing human written and AI-generated Kpop lyrics on English versions of Korean tracks and English original tracks.
Kpop is known for hybridization (Jin and Ryoo 2014) and for bending grammatical norms, especially of English (Willis 2014), as Kpop can sometimes be inspired by Western pop music (Li 2022; Barnes-Sadler et al. 2025).
Our contribution consists in introducing a manually curated dataset of human and AI-generated English Kpop song lyrics, and in comparing them by using both computational and manual close reading methods.
(Barnes-Sadler et al. 2025) focused on English usage within Korean language in Kpop tracks, rather than English-exclusive songs, treating mostly code-mixing or code-switching. (Frohmann et al. 2025) explored automated lyric generation and artistic assistance, without evaluating genre-specific stylistic fidelity or cultural nuance.
(Roe 2025; Klein et al. 2026) argued that generative AI outputs are cultural artifacts shaped by societal norms and biases, not neutral text, so they should be studied beyond their technical results. This directly applies to English lyric generation in Asian music contexts.
Although generative AI is heavily used in the music industry, few studies focus on their potential for song lyric production (Ding et al. 2025) and no research deals with English-exclusive Kpop songs, nor with LLMs’ ability to generate lyrics in such a specific context.
The corpus comprises 230 songs, 200 human written and 30 AI-generated, totalling over 100,000 words. The Kpop lyrics authored by popular Korean artists were extracted from a web platform database
https://genius.com/
We also experimented with (creativity) parameters, (maximum
temperature and
top p) for models that had this option available. We included publicly available LLMs on their websites or from Huggingface
https://huggingface.co/spaces/ https://deepinfra.com/chat/
Quantitative Analysis
We computed three basic metrics for both humans and machines: the average song length in words, the lexical diversity (unique words/total words), and the line repetition ratio (repeated lines/total lines), listed in Table 1. They reveal that human lyrics are longer, lexically richer, and contain more line repetitions (a typical K factor).
Language Style Matching score, which measures the use of function words, obtained with LIWC (Boyd et al. 2022), a tool designed to extract psycho-socio-linguistic categories, is rather high (0.82), confirming that AI matches the basic grammatical footprint of the human lyrics. Nevertheless, LLM-generated lyrics lack the structural complexity of Kpop, as shown by LIWC features given in Table 2. All the human scores are higher than the LLMs’ ones. Nonflu score, which represents the use of non-fluent grammar and wording, shows a spontaneity gap between Humans and LLMs, since human songs focus more on musical flow than on grammatical accuracy. The Filler scores (hooks like oh or uh) suggest that humans concentrate more on rhythmic padding to fit English words into fast-paced Korean melodies.
| Category | Avg words/song | Lexical diversity | Avg line repetition ratio |
| LLMs | 364 | 0.01283 | 0.2364 |
| Humans | 430 | 0.0437 | 0.3559 |
Table 1. Basic metrics
Humans also use Conversation (informal words like well, anyway), Netspeak (slang) and Assent (approval words like yeah, ok) more, denoting lyrics written for an audience, to create the typical Kpop "idol-style energy". The AI songs are more clean, even sterile and seem over-done. By stripping away the noise or energy, the LLMs fail to replicate the adlibs and rhythmic style that define human Kpop performance.
| Category | Conversation | Netspeak | Assent | Nonflu | Filler |
| LLMs | 1.96 | 0.9 | 0.89 | 0.12 | 0.05 |
| Humans | 5.61 | 2.06 | 1.54 | 1.35 | 0.76 |
Table 2. LIWC scores
Sentiment Analysis
To extract the sentiments from the lyrics we used Hartmann’s seven class emotion model DistilRoberta
https://huggingface.co/j-hartmann/emotion-english-distilroberta-base
Readability and Perplexity
Readability metrics indicated that the LLMs produce higher grade level lyrics, use longer words, more elevated vocabulary, and adhere to grammar rules, whereas humans employ wordplay and genre-specific deviations from English conventions, as noticed by (Schneider 2024), probably since human lyrics are optimized for auditory processing, while the LLMs tend to mimic written poetry.
Perplexity scores computed with GPT-Neo
https://huggingface.co/EleutherAI/gpt-neo-1.3B
Automatic Classification
Since the dataset is imbalanced, we built a frozen Distilbert
https://huggingface.co/docs/transformers/en/model doc/distilbert https://imbalanced-learn.org/stable/references/generated/imblearn.over sampling.SMOTE.html
Empirical Observations
We noticed that some models generated lyrics resembling more with generic pop instead of having "K" elements. LLMs used figurative language and elevated vocabulary but lacked the lyricism patterns characteristic of successful English Kpop songs.
LLMs demonstrated a moderate ability to generate English Kpop lyrics and struggled to authentically replicate the distinctive stylistic and cultural nuances of the genre. Their outputs either exaggerated certain lyrical tropes or underperformed in conveying the typical Kpop emotional depth, thematic consistency, rhythm, and playful wordplay.
Not only LLM-generated lyrics are of lower quality, but the overuse of generative AI in the creative industries poses major environmental sustainability risks. This technology requires massive amounts of resources, high carbon emissions which can negatively impact the climate, as well as society (Esho et al. 2026).
This work was supported by a grant of the Ministry of Education and Research, CCCDI - UEFISCDI, project number PN-IV-P6-6.1-CoEx-2024-0042, within PNCDI IV, and by a grant of the Ministry of Research, Innovation and Digitization, CNCS - UEFISCDI, project SIROLA, number PN-IV-P1-PCE-2023-1701, within PNCDI IV.