Web Service Composition Based on Reinforcement Learning

How we manage Web services depends on how we understand their variable parts and invariable parts. Studying them separately could make Web service research much easier and make our software architecture much more loose-coupled. We summarize two variable parts that affect Web service compositions: uncertain invocation results and uncertain quality of services. These uncertain factors affect success rate of service composition. Previous studies model the Web service problem as a planning problem, while this problem is considered as an uncertain planning problem in this paper. Specifically, we use Partially Observable Markov Decision Process to deal with the uncertain planning problem for service composition. According to the uncertain model, we propose a reinforcement learning method, which is an uncertainty planning method, to compose web services. The proposed method does not need to know complete information of services, instead it uses historical data and estimates the successful possibilities that services are composed together with respect to service outcomes and QoS. Simulation experiments verify the validity of the algorithm, and the results also show that our method improves the success rate of the service composition.