Text-based sentiment analysis for evaluating the service provider professionalism (SPP) of macro work on online labor platforms (OLPs)

Abstract

Macro work is a significant Online Labor Platforms (OLPs) operation characterized by higher professionalism for service providers. Therefore, the professionalism assessment for providers of macro work is vital for OLPs. However, due to the high ambiguity of textual data, OLPs often overlook them when evaluating the Service Provider Professionalism (SPP) of macro work. Within OLPs, there is a large amount of textual data, which contains information reflecting their professionalism. Hence, this study proposes a method for evaluating the SPP of macro work on OLPs based on text sentiment analysis: (1) Select professional vocabulary related to a specific type of macro work as sentiment words; (2) Collect texts and score their professionalism values; (3) Calculate the sentiment word professionalism value based on the NBSP algorithm - an algorithm that combines the Naive Bayes and Semantic Orientation Pointwise Mutual Information (SO-PMI) algorithms; (4) Calculate the text professionalism value, namely the SPP value. Algorithm validation results show that compared to baseline algorithms, the NBSP algorithm achieves an increase in the accuracy of calculating text professionalism values by 4.45 - 27.75 percent points. To validate this method's effectiveness, this study conducted a comparative experiment on predicting the annual transaction amounts of IT service providers on a certain Chinese OLP under eight main-stream predictive models, incorporating the feature of SPP reduced MSE by 6% - 12%. This study contributes to expanding research in structuring textual data and text sentiment analysis in OLPs and enhances professionalism assessment for service providers of macro work on OLPs.