Aspect-based sentiment in Turkish product reviews

An NLP study comparing lexicon, TF-IDF, BERTurk and KOZMOS approaches on 3,358 aspect-sentiment pairs.

Problem

“Shipping was fast, but the battery is poor” contains two different judgments. A single sentiment label merges them. Our three-person team compared methods that evaluate sentiment for individual aspects: shipping, quality, price, battery, audio/visual, packaging and general feedback.

Labeling 2,250 raw phone, headphone and computer reviews produced 3,358 aspect-sentiment pairs, each assigned positive, negative or neutral sentiment.

My contribution and methods

I worked on scraper development and data expansion, the phone/headphone labeling workflow and KOZMOS fine-tuning. The team also implemented rule-based ABSA, a TF-IDF hybrid and BERTurk.

Selenium collected reviews. The rule-based pipeline uses clause splitting, aspect dictionaries and negation rules. The hybrid replaces sentiment rules with TF-IDF and Logistic Regression/Linear SVM. Transformer models receive the review together with its target aspect.

KOZMOS uses ytu-ce-cosmos/turkish-small-bert-uncased. A customized Hugging Face Trainer applies weighted CrossEntropy, early stopping and macro-F1 model selection. Captum and attention visualizations support prediction analysis.

Evaluation

Because one review generates multiple rows, the saved split groups rows by review text. It contains 2,342 training, 506 validation and 510 test rows. Exact review-text overlap between the saved partitions is zero.

KOZMOS confusion matrix for positive, negative and neutral sentiment on 510 test examples

Recalculation from saved predictions gives KOZMOS macro F1 0.6568 and accuracy 80.59%. BERTurk produced the best team result at 0.6905 macro F1. Only 45 test examples are neutral, making that class particularly uncertain.

**Looking beyond the aggregate score** Positive and negative classes are stronger; the small neutral class has substantially weaker precision and recall.

Limitations

The Transformer experiments classify sentiment for a supplied aspect rather than jointly learning aspect extraction. The TF-IDF script also needs fold-local vectorizer fitting and grouped cross-validation, so the complete experimental pipeline should not be described as universally leakage-free.

**Which tokens receive attention?** Token attention in a shipping example helps inspect model behavior; attention alone is not a causal explanation.

Result

The project provides a review-analysis pipeline and a comparison of four approaches. My model result is KOZMOS; the BERTurk result belongs to the team comparison.