Beyond Noise: A BERT-Enhanced framework for Intelligent product optimization via online review Analytics

Online reviews provide valuable insights into customer preference and product attributes (PAs), enabling companies to formulate effective product improvement strategies. In addition to PAs, however, reviews often contain noise, such as information about logistics and marketing strategies. While managing this noise is crucial to improving PA extraction and categorization accuracy and efficiency, existing studies have largely overlooked or handled noise inadequately. To address this gap, this study proposes a hybrid AI-driven framework for identifying and categorizing PAs while filtering out noise from online reviews. First, we use bidirectional encoder representations from transformers (BERT) to identify informative reviews (i.e., those containing at least one PA). Then, we integrate latent Dirichlet allocation and Word2Vec to extract irrelevant information, aiming to isolate noise and extract PAs from the reviews. Furthermore, we categorize PAs using importance-performance analysis (IPA) and IPA-GAP1. In this process, the importance of attributes is calculated by fitting the relationship between customer sentiment toward attributes and customer satisfaction using random forest, and customer sentiments are determined using BERT-based sentiment analysis. Additionally, we use importance–performance competitor analysis to assess attribute performance and importance across different products. Finally, we propose an Improvement Priority Score (IPS), which integrates attribute importance, performance, and competitive performance gap to provide companies with actionable insights for product optimization under limited resources. The proposed framework is validated through a case study using phone reviews from JD.com.