Sentiment Classification of Instagram Poster Comments on the Documentary Film "Pesta Babi": A Comparative Study of SVM, Random Forest, and Naive Bayes with a Keyword-Based Exploration of Sociocultural Themes

Abstract

This study analyzes public sentiment in 704 Instagram comments responding to the poster of the Indonesian documentary "Pesta Babi: Kolonialisme di Zaman Kita," which triggered national debate in 2026. Comments were collected through total sampling and sentiment-labeled with AI assistance (partially via an Excel-integrated AI tool, partially via an AI-generated Python script executed in PyCharm), yielding an imbalanced distribution of 514 Positive and 190 Negative comments. After text preprocessing (regex-based tokenization and lowercasing) and binary Bag-of-Words feature extraction with L1 regularization, three algorithms Support Vector Machine, Random Forest, and Naive Bayes were compared using 10-fold cross-validation on the Orange platform. Random Forest achieved the highest overall accuracy (0.786) and AUC (0.858), while Naive Bayes showed the strongest balanced performance on the minority Negative class (MCC = 0.398). A supplementary keyword-based analysis found only 19.7% of comments referenced religious, artistic, or ethical-normative themes, indicating public discourse was dominated by other issues, notably political and funding-related accusations.