badge icon

This article was automatically translated from the original Turkish version.

Article

BERTurk

Developer

MDZ Digital Library (Bavarian State Library)

Stefan Schweter

Publication Date

2020

Language

Turkish

Architecture

BERT (Transformer-based)

Training Data

35 GB Turkish text

Access

Hugging Face (dbmdz)

Developer

MDZ Digital Library (Bavarian State Library)

Stefan Schweter

Publication Date

2020

Language

Turkish

Architecture

BERT (Transformer-based)

Training Data

35 GB Turkish text

Access

Hugging Face (dbmdz)

BERTurk is a language model pretrained on Turkish texts and based on the BERT (Bidirectional Encoder Representations from Transformers) architecture. It was developed by Stefan Schweter and is among the most widely used language models in Turkish natural language processing research.【1】 BERTurk is not an application in itself; it functions as a foundational model that can be adapted for tasks such as text classification, named entity recognition, and sentiment analysis.

History

Origins

BERT is a language model developed by Google researchers and based on the transformer architecture.【2】 The model represents a word not only by the words preceding it but by considering the context from both directions within a sentence. This approach achieved state-of-the-art results on many natural language processing tasks in English.【3】


From a Turkish perspective, two major challenges exist. The first is the structure of the language. Turkish is an agglutinative language in which a single word such as “gidemeyeceklerdi” can convey a meaning equivalent to an entire sentence in English. This rich morphological structure makes it difficult to directly apply methods developed for English to Turkish.【4】 The second challenge is the scarcity of resources; for a long time, labeled datasets and ready-made tools for Turkish natural language processing were limited.【5】


BERTurk is a BERT model developed specifically for Turkish and trained on Turkish text corpora.【6】

Development and Training Data

BERTurk was trained on approximately 35 GB of Turkish text data. The training corpus was constructed from the OSCAR dataset, Wikipedia dumps, and various OPUS collections. One version of the model features a vocabulary of 128,000 word pieces.【7】

Working Principle

The use of BERTurk consists of two stages. The model first acquires general language knowledge and is then adapted to a specific task.

Pretraining

During the pretraining stage, the model is trained on unlabeled texts. In this phase, some words in sentences are masked, and the model is expected to predict these masked words based on their surrounding context.【8】 Thus, the model learns to distinguish different meanings of a word like “yüz” in expressions such as “yüz lira” and “yüzünü yıkamak” based on context. Due to the high computational cost, pretraining is performed only once, and the resulting model is made available for use.

Fine-Tuning

In the fine-tuning stage, a classification layer is added on top of the model for a specific task. For example, when the goal is to classify reviews as positive or negative, the model is briefly retrained on a relatively small and labeled dataset specific to that task.【9】 This approach eliminates the need to train the model from scratch for each new task.

Use in Sentiment Analysis and Emotion Recognition

In Turkish literature, the term “sentiment analysis” is used to refer to two distinct tasks.【10】 The first is classifying a text as positive, negative, or neutral (in English: sentiment analysis). The second is identifying which emotion—such as happiness, anger, fear, or sadness—is most prominent in the text (in English: emotion recognition). BERTurk serves as a foundational model for both tasks.

Positive-Negative Classification

Studies in this area include those focused on film and product reviews. In one study using film reviews collected from the Beyazperde website, representations generated by the fine-tuned BERTurk model were combined with Support Vector Machines, achieving an accuracy rate of 98.4 percent.【11】 In another study using product reviews from Trendyol, BERTurk achieved an accuracy of 96 percent, while the traditional Support Vector Machine method reached 89 percent.【12】

Emotion Category Classification

One of the datasets used in Turkish emotion recognition studies is TREMO. The dataset, prepared by Toçoğlu and Alpkoçak, consists of 27,350 entries in which 4,709 individuals described memories and experiences related to emotions such as happiness, fear, anger, sadness, disgust, and surprise.【13】


Uçan, Dörterler, and Akçapınar Sezer fine-tuned multilingual BERT and BERTurk models on the TREMO dataset for emotion classification and compared these models with traditional machine learning and deep learning methods.【14】 In a more recent study, the BERTurk model fine-tuned on TREMO achieved an accuracy of 92.62 percent across six emotion categories and was used to examine the emotional aspects of anti-migrant posts on social media.【15】

Emotion Recognition in Speech

BERTurk operates solely on written text and cannot process speech directly. Therefore, to analyze emotions in speech, the speech must first be transcribed into text. However, the transcribed text does not contain information such as tone of voice, emphasis, or speaking rate. A systematic review of studies on negative emotions and stress in speech determined that anger is associated with an increase in fundamental frequency, increased vocal intensity, and faster speech rate, while stress is linked to increases in both fundamental frequency and vocal intensity.【16】

Limitations

BERTurk can process only textual data and cannot directly handle other data types such as audio or images. BERT-based models can process a maximum of 512 tokens in a single input; longer texts must be segmented before being fed into the model.【17】


BERTurk was trained on texts collected from general sources such as the OSCAR corpus and Wikipedia. For social media language, models such as TurkishBERTweet have been developed using over 894 million Turkish tweets; these models have a lighter architecture than BERTurk.【18】

Bibliographies

Açıkalın, Utku Umur, Benan Bardak, and Mücahid Kutlu. “Turkish Sentiment Analysis Using BERT.” *28th Signal Processing and Communications Applications Conference (SIU)*, (2020): 1–4. Accessed October 20, 2026.https://ieeexplore.ieee.org/document/9302492

Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies* (2019): 4171–4186. Minneapolis: Association for Computational Linguistics, 2019. Accessed October 20, 2026.https://aclanthology.org/N19-1423/

Google DeepMind. “3D render showcasing an abstract concept of artificial intelligence and technology with cube manipulation.” Pexels. Accessed October 20, 2026.https://www.pexels.com/photo/an-artist-s-illustration-of-artificial-intelligence-ai-this-piece-explores-the-prediction-method-used-in-large-language-models-it-was-created-by-artist-wes-cockx-as-part-of-the-visuali-18069813/

Gündüz, Hakan. “Sentiment Analysis on Turkish Movie Reviews with Deep Bidirectional Encoder Representations from Transformers (BERT) and Support Vector Machines.” *KSÜ Mühendislik Bilimleri Dergisi* 26, no. 2 (2023): 542–549. Accessed October 20, 2026.https://doi.org/10.17780/ksujes.1241043

Najafi, Ali. “Developing Turkish Language Models on Social Media.” Master's thesis, Sabancı Üniversitesi, 2024. Accessed October 20, 2026.

Schewski, Lilien, Mathew Magimai Doss, Guido Beldi, and Sandra Keller. “Measuring Negative Emotions and Stress through Acoustic Correlates in Speech: A Systematic Review.” *PLoS One* 20, no. 7 (2025): 1-28. Accessed October 20, 2026.https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0328833

Teke, B., S. N. Yazıcı, G. Zamir, A. B. Budak et al. “BERTurk-Based Sentiment Analysis on E-Commerce Multi Domain Product Reviews.” *Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi* 25, no. 3 (2025): 497–509. Accessed October 20, 2026.https://doi.org/10.35414/akufemubid.1537513

Toçoğlu, Mansur Alp, and Adil Alpkoçak. “Lexicon-Based Emotion Analysis in Turkish.” *Turkish Journal of Electrical Engineering and Computer Sciences* 27, no. 2 (2019): 1213–1227. Accessed October 20, 2026.https://journals.tubitak.gov.tr/elektrik/vol27/iss2/40/

Uçan, A., M. Dörterler, and E. Akçapınar Sezer. “A Study of Turkish Emotion Classification with Pretrained Language Models.” *Journal of Information Science* 48, no. 6 (2022): 857–865. Accessed October 20, 2026.https://doi.org/10.1177/0165551520985507

Wicaksono, Darmawan, Hasri Akbar Awal Rozaq, and Nevfel Boz. “Emotion Recognition for Low-Resource Turkish: Fine-Tuning BERTurk on TREMO and Testing on Xenophobic Political Discourse.” arXiv. (2025): 1-17. Accessed October 20, 2026.

Çelikten, Azer, and Hasan Bulut. “Turkish Medical Text Classification Using BERT.” *29th Signal Processing and Communications Applications Conference (SIU)* (2021): 1–4. Accessed October 20, 2026.https://ieeexplore.ieee.org/document/9477847

Çöltekin, Çağrı, A. Seza Doğruöz, and Özlem Çetinoğlu. “Resources for Turkish Natural Language Processing: A Critical Survey.” arXiv. (2022): 1-43. Accessed October 20, 2026.https://arxiv.org/abs/2204.05042

Citations

  • [1]

    Ali Najafi, “Developing Turkish Language Models on Social Media” (Yüksek lisans tezi, Sabancı Üniversitesi, 2024), iv, 2, Erişim Tarihi: 20 Ekim 2026, https://research.sabanciuniv.edu/id/eprint/51710/

  • [2]

    Azer Çelikten ve Hasan Bulut, “Turkish Medical Text Classification Using BERT,” 29th Signal Processing and Communications Applications Conference (SIU), (2021): 1, Erişim Tarihi: 20 Ekim 2026, https://ieeexplore.ieee.org/document/9477847

  • [3]

    Jacob Devlin vd., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Minneapolis: Association for Computational Linguistics), (2019): 4171, Erişim Tarihi: 20 Ekim 2026, https://aclanthology.org/N19-1423/

  • [4]

    Azer Çelikten ve Hasan Bulut, “Turkish Medical Text Classification Using BERT,” 29th Signal Processing and Communications Applications Conference (SIU), (2021): 1, Erişim Tarihi: 20 Ekim 2026, https://ieeexplore.ieee.org/document/9477847

  • [5]

    Utku Umur Açıkalın, Benan Bardak ve Mücahid Kutlu, “Turkish Sentiment Analysis Using BERT,” 28th Signal Processing and Communications Applications Conference (SIU), (2020): 1, Erişim Tarihi: 20 Ekim 2026, https://ieeexplore.ieee.org/document/9302492

  • [6]

    Ali Najafi, “Developing Turkish Language Models on Social Media,” Yüksek lisans tezi, Sabancı Üniversitesi, (2024): 2, Erişim Tarihi: 20 Ekim 2026, https://research.sabanciuniv.edu/id/eprint/51710/

  • [7]

    Ali Najafi, “Developing Turkish Language Models on Social Media,” Yüksek lisans tezi, Sabancı Üniversitesi, (2024): 10, Erişim Tarihi: 20 Ekim 2026, https://research.sabanciuniv.edu/id/eprint/51710/

  • [8]

    Jacob Devlin vd., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Minneapolis: Association for Computational Linguistics,), (2019): 4174, Erişim Tarihi: 20 Ekim 2026, https://aclanthology.org/N19-1423/

  • [9]

    Jacob Devlin vd., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Minneapolis: Association for Computational Linguistics,), (2019): 4175, Erişim Tarihi: 20 Ekim 2026, https://aclanthology.org/N19-1423/

  • [10]

    Mansur Alp Toçoğlu ve Adil Alpkoçak, “Lexicon-Based Emotion Analysis in Turkish,” Turkish Journal of Electrical Engineering and Computer Sciences 27, no. 2 (2019): 1213, Erişim Tarihi: 20 Ekim 2026, https://journals.tubitak.gov.tr/elektrik/vol27/iss2/40/

  • [11]

    Hakan Gündüz, “Sentiment Analysis on Turkish Movie Reviews with Deep Bidirectional Encoder Representations from Transformers (BERT) and Support Vector Machines,” KSÜ Mühendislik Bilimleri Dergisi 26, no. 2 (2023): 542, Erişim Tarihi: 20 Ekim 2026, https://doi.org/10.17780/ksujes.1241043

  • [12]

    Bekir Teke vd., “BERTurk-Based Sentiment Analysis on E-Commerce Multi Domain Product Reviews,” Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi 25, no. 3 (2025): 497, Erişim Tarihi: 20 Ekim 2026, https://doi.org/10.35414/akufemubid.1537513

  • [13]

    Çağrı Çöltekin, A. Seza Doğruöz ve Özlem Çetinoğlu, “Resources for Turkish Natural Language Processing: A Critical Survey,” Language Resources and Evaluation manuscript, (2022): 14, Erişim Tarihi: 20 Ekim 2026, https://arxiv.org/abs/2204.05042

  • [14]

    Alaettin Uçan, Murat Dörterler ve Ebru Akçapınar Sezer, “A Study of Turkish Emotion Classification with Pretrained Language Models,” Journal of Information Science 48, no. 6 (2022): 857, Erişim Tarihi: 20 Ekim 2026, https://doi.org/10.1177/0165551520985507

  • [15]

    Darmawan Wicaksono, Hasri Akbar Awal Rozaq ve Nevfel Boz, “Emotion Recognition for Low-Resource Turkish: Fine-Tuning BERTurk on TREMO and Testing on Xenophobic Political Discourse,” arXiv (2025): 1, Erişim Tarihi: 20 Ekim 2026, https://arxiv.org/abs/2505.12160

  • [16]

    Lilien Schewski vd., “Measuring Negative Emotions and Stress through Acoustic Correlates in Speech: A Systematic Review,” PLoS One 20, no. 7 (2025): 1, Erişim Tarihi: 20 Ekim 2026, https://doi.org/10.1371/journal.pone.0328833

  • [17]

    Jacob Devlin vd., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Minneapolis: Association for Computational Linguistics,), (2019): 4183, Erişim Tarihi: 20 Ekim 2026, https://aclanthology.org/N19-1423/

  • [18]

    Ali Najafi, “Developing Turkish Language Models on Social Media,” Yüksek lisans tezi, Sabancı Üniversitesi,), (2024): 2, Erişim Tarihi: 20 Ekim 2026, https://research.sabanciuniv.edu/id/eprint/51710/

Author Information

Avatar
AuthorTesnim StrazimiriOctober 25, 2026 at 12:34 PM

Discussions

No Discussion Added Yet

Start discussion for "BERTurk" article

View Discussions

Contents

  • History

    • Origins

    • Development and Training Data

  • Working Principle

    • Pretraining

    • Fine-Tuning

  • Use in Sentiment Analysis and Emotion Recognition

    • Positive-Negative Classification

    • Emotion Category Classification

    • Emotion Recognition in Speech

  • Limitations