Problem
Thesis repositories contain years of research history, but their search pages do not reveal how topics change or overlap across disciplines. I built a reproducible pipeline to map themes in Turkish abstracts published between 2018 and 2026 in Software Engineering, Computer Engineering, and Electronics and Communication Engineering.
Data pipeline
The collection stage combined Selenium for year-by-year result pages with an HTTP/JSON endpoint for each thesis record. It kept raw and processed data separate, repaired 2,850 swapped Turkish and English title fields, removed abstracts shorter than 300 characters, and reduced the remaining English-looking abstracts to below 1% of the corpus.
After quality checks, the dataset contained 8,503 usable abstracts: 484 from Software Engineering, 7,016 from Computer Engineering, and 1,003 from Electronics and Communication Engineering.
Modeling and comparison
I trained a separate Gensim LDA model for each discipline and evaluated topic
counts from 5 to 30 using c_v coherence together with interpretability. The
selected models contained 10, 15, and 5 topics respectively. Document-topic
vectors were then analyzed with K-Means and Ward hierarchical clustering,
while Jaccard similarity compared the disciplines through their most
representative topic words.
Topic prevalence by year. The partial 2026 data was excluded from trend slopes.
What did not work
A stemming pilot over-trimmed Turkish words and removed useful meaning, so I
kept surface forms and designed suffix-tolerant stop-word matching instead.
Short or risky roots were restricted to exact matches to avoid deleting terms
such as veritabanı, yerel, and zamanlama.
Clustering also exposed a methodological trap. Its validity scores were best when the cluster count was close to the LDA topic count, because many document vectors were dominated by one topic. In that setting clustering mostly repeated the LDA assignment. I therefore fixed the final cluster count at six and treated it as a separate macro-theme view rather than a second topic-count estimator.
Average dominant-topic probabilities ranged from 0.59 to 0.78, explaining why unconstrained clustering tended to reproduce the topic model.
Result
Software and Computer Engineering showed the strongest vocabulary overlap with a Jaccard score of 0.373. Electronics and Communication Engineering was more distinct, scoring 0.112 against Software Engineering and 0.123 against Computer Engineering. From 2018 to 2025, deep learning and biomedical imaging themes increased while classical machine learning and pattern-recognition themes declined.
The analysis is limited by the uneven corpus sizes, LDA’s bag-of-words assumption, and word-level Jaccard similarity, which can miss semantically similar concepts expressed with different terminology.