Giovanni De Gasperis, Stefania Costantini, Andrea Rafanelli, Patrizio Migliarini, Ivan Letteri, Abeer Dyoub
{"title":"深度q学习智能体训练和基准测试的约束程序逻辑生成环境的扩展","authors":"Giovanni De Gasperis, Stefania Costantini, Andrea Rafanelli, Patrizio Migliarini, Ivan Letteri, Abeer Dyoub","doi":"10.1093/logcom/exad032","DOIUrl":null,"url":null,"abstract":"Abstract Autonomous robots can be employed in exploring unknown environments and performing many tasks, such as, e.g. detecting areas of interest, collecting target objects, etc. Deep reinforcement learning (RL) is often used to train this kind of robot. However, concerning the artificial environments aimed at testing the robot, there is a lack of available data sets and a long time is needed to create them from scratch. A good data set is in fact usually produced with high effort in terms of cost and human work to satisfy the constraints imposed by the expected results. In the first part of this paper, we focus on the specification of the properties of the solutions needed to build a data set, making the case of environment exploration. In the proposed approach, rather than using imperative programming, we explore the possibility of generating data sets using constraint programming in Prolog. In this phase, geometric predicates describe a virtual environment according to inter-space requirements. The second part of the paper is focused on testing the generated data set in an AI gym via space search techniques. We developed a Neuro-Symbolic agent built from the following: (i) A deep Q-learning component implemented in Python, able to address via RL a search problem in the virtual space; the agent has the goal to explore a generated virtual environment to seek for a target, improving its performance through a RL process. (ii) A symbolic component able to re-address the search when the Q-learning component gets stuck in a part of the virtual environment; these components stimulate the agent to move to and explore other parts of the environment. Wide experimentation has been performed, with promising results, and is reported, to demonstrate the effectiveness of the approach.","PeriodicalId":50162,"journal":{"name":"Journal of Logic and Computation","volume":"11 1","pages":"0"},"PeriodicalIF":0.7000,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Extension of constraint-procedural logic-generated environments for deep Q-learning agent training and benchmarking\",\"authors\":\"Giovanni De Gasperis, Stefania Costantini, Andrea Rafanelli, Patrizio Migliarini, Ivan Letteri, Abeer Dyoub\",\"doi\":\"10.1093/logcom/exad032\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract Autonomous robots can be employed in exploring unknown environments and performing many tasks, such as, e.g. detecting areas of interest, collecting target objects, etc. Deep reinforcement learning (RL) is often used to train this kind of robot. However, concerning the artificial environments aimed at testing the robot, there is a lack of available data sets and a long time is needed to create them from scratch. A good data set is in fact usually produced with high effort in terms of cost and human work to satisfy the constraints imposed by the expected results. In the first part of this paper, we focus on the specification of the properties of the solutions needed to build a data set, making the case of environment exploration. In the proposed approach, rather than using imperative programming, we explore the possibility of generating data sets using constraint programming in Prolog. In this phase, geometric predicates describe a virtual environment according to inter-space requirements. The second part of the paper is focused on testing the generated data set in an AI gym via space search techniques. We developed a Neuro-Symbolic agent built from the following: (i) A deep Q-learning component implemented in Python, able to address via RL a search problem in the virtual space; the agent has the goal to explore a generated virtual environment to seek for a target, improving its performance through a RL process. (ii) A symbolic component able to re-address the search when the Q-learning component gets stuck in a part of the virtual environment; these components stimulate the agent to move to and explore other parts of the environment. Wide experimentation has been performed, with promising results, and is reported, to demonstrate the effectiveness of the approach.\",\"PeriodicalId\":50162,\"journal\":{\"name\":\"Journal of Logic and Computation\",\"volume\":\"11 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.7000,\"publicationDate\":\"2023-06-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Logic and Computation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1093/logcom/exad032\",\"RegionNum\":4,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, THEORY & METHODS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Logic and Computation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1093/logcom/exad032","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}
Extension of constraint-procedural logic-generated environments for deep Q-learning agent training and benchmarking
Abstract Autonomous robots can be employed in exploring unknown environments and performing many tasks, such as, e.g. detecting areas of interest, collecting target objects, etc. Deep reinforcement learning (RL) is often used to train this kind of robot. However, concerning the artificial environments aimed at testing the robot, there is a lack of available data sets and a long time is needed to create them from scratch. A good data set is in fact usually produced with high effort in terms of cost and human work to satisfy the constraints imposed by the expected results. In the first part of this paper, we focus on the specification of the properties of the solutions needed to build a data set, making the case of environment exploration. In the proposed approach, rather than using imperative programming, we explore the possibility of generating data sets using constraint programming in Prolog. In this phase, geometric predicates describe a virtual environment according to inter-space requirements. The second part of the paper is focused on testing the generated data set in an AI gym via space search techniques. We developed a Neuro-Symbolic agent built from the following: (i) A deep Q-learning component implemented in Python, able to address via RL a search problem in the virtual space; the agent has the goal to explore a generated virtual environment to seek for a target, improving its performance through a RL process. (ii) A symbolic component able to re-address the search when the Q-learning component gets stuck in a part of the virtual environment; these components stimulate the agent to move to and explore other parts of the environment. Wide experimentation has been performed, with promising results, and is reported, to demonstrate the effectiveness of the approach.
期刊介绍:
Logic has found application in virtually all aspects of Information Technology, from software engineering and hardware to programming and artificial intelligence. Indeed, logic, artificial intelligence and theoretical computing are influencing each other to the extent that a new interdisciplinary area of Logic and Computation is emerging.
The Journal of Logic and Computation aims to promote the growth of logic and computing, including, among others, the following areas of interest: Logical Systems, such as classical and non-classical logic, constructive logic, categorical logic, modal logic, type theory, feasible maths.... Logical issues in logic programming, knowledge-based systems and automated reasoning; logical issues in knowledge representation, such as non-monotonic reasoning and systems of knowledge and belief; logics and semantics of programming; specification and verification of programs and systems; applications of logic in hardware and VLSI, natural language, concurrent computation, planning, and databases. The bulk of the content is technical scientific papers, although letters, reviews, and discussions, as well as relevant conference reviews, are included.