Prompt engineering techniques for semantic enhancement in business process models

IF 4.5 3区 管理学 Q1 BUSINESS
Sarah Ayad, Fatimah Alsayoud
{"title":"Prompt engineering techniques for semantic enhancement in business process models","authors":"Sarah Ayad, Fatimah Alsayoud","doi":"10.1108/bpmj-02-2024-0108","DOIUrl":null,"url":null,"abstract":"<h3>Purpose</h3>\n<p>The term knowledge refers to the part of the world investigated by a specific discipline and that includes a specific taxonomy, vocabulary, concepts, theories, research methods and standards of justification. Our approach uses domain knowledge to improve the quality of business process models (BPMs) by exploiting the domain knowledge provided by large language models (LLMs). Among these models, ChatGPT stands out as a notable example of an LLM capable of providing in-depth domain knowledge. The lack of coverage presents a limitation in each approach, as it hinders the ability to fully capture and represent the domain’s knowledge. To solve such limitations, we aim to exploit GPT-3.5 knowledge. Our approach does not ask GPT-3.5 to create a visual representation; instead, it needs to suggest missing concepts, thus helping the modeler improve his/her model. The GPT-3.5 may need to refine its suggestions based on feedback from the modeler.</p><!--/ Abstract__block -->\n<h3>Design/methodology/approach</h3>\n<p>We initiate our semantic quality enhancement process of a BPM by first extracting crucial elements including pools, lanes, activities and artifacts, along with their corresponding relationships such as lanes being associated with pools, activities belonging to each lane and artifacts associated with each activity. These data are systematically gathered and structured into ArrayLists, a form of organized collection that allows for efficient data manipulation and retrieval. Once we have this structured data, our methodology involves creating a series of prompts based on each data element. We adopt three approaches to prompting: zero-shot, few-shot and chain of thoughts (CoT) prompts. Each type of prompting is specifically designed to interact with the OpenAI language model in a unique way, aiming to elicit a diverse array of suggestions. As we apply these prompting techniques, the OpenAI model processes each prompt and returns a list of suggestions tailored to that specific element of the BPM. Our approach operates independently of any specific notation and offers semi-automation, allowing modelers to select from a range of suggested options.</p><!--/ Abstract__block -->\n<h3>Findings</h3>\n<p>This study demonstrates the significant potential of prompt engineering techniques in enhancing the semantic quality of BPMs when integrated with LLMs like ChatGPT. Our analysis of model activity richness and model artifact richness across different prompt techniques and input configurations reveals that carefully tailored prompts can lead to more complete BPMs. This research is a step forward for further exploration into the optimization of LLMs in BPM development.</p><!--/ Abstract__block -->\n<h3>Research limitations/implications</h3>\n<p>The limitation is the domain ontology that we are relying on to evaluate the semantic completeness of the new BPM. In our future work, the modeler will have the option to ask for synonyms, hyponyms, hypernyms or keywords. This feature will facilitate the replacement of existing concepts to improve not only the completeness of the BPM but also the clarity and specificity of concepts in BPMs.</p><!--/ Abstract__block -->\n<h3>Practical implications</h3>\n<p>To demonstrate our methodology, we take the “Hospitalization” process as an illustrative example. In the scope of our research, we have presented a select set of instructions pertinent to the “chain of thought” and “few-shot prompting.” Due to constraints in presentation and the extensive nature of the instructions, we have not included every detail within the body of this paper. However, they can be found in the previous GitHub link. Two appendices are given at the end. Appendix 1 describes the different prompt instructions. Appendix 2 presents the application of the instructions in our example.</p><!--/ Abstract__block -->\n<h3>Originality/value</h3>\n<p>In our research, we rely on the domain application knowledge provided by ChatGPT-3 to enhance the semantic quality of BPMs. Typically, the semantic quality of BPMs may suffer due to the modeler's lack of domain knowledge. To address this issue, our approach employs three prompt engineering methods designed to extract accurate domain knowledge. By utilizing these methods, we can identify and propose missing concepts, such as activities and artifacts. This not only ensures a more comprehensive representation of the business process but also contributes to the overall improvement of the model's semantic quality, leading to more effective and accurate business process management.</p><!--/ Abstract__block -->","PeriodicalId":47964,"journal":{"name":"Business Process Management Journal","volume":"10 1","pages":""},"PeriodicalIF":4.5000,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Business Process Management Journal","FirstCategoryId":"91","ListUrlMain":"https://doi.org/10.1108/bpmj-02-2024-0108","RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"BUSINESS","Score":null,"Total":0}
引用次数: 0

Abstract

Purpose

The term knowledge refers to the part of the world investigated by a specific discipline and that includes a specific taxonomy, vocabulary, concepts, theories, research methods and standards of justification. Our approach uses domain knowledge to improve the quality of business process models (BPMs) by exploiting the domain knowledge provided by large language models (LLMs). Among these models, ChatGPT stands out as a notable example of an LLM capable of providing in-depth domain knowledge. The lack of coverage presents a limitation in each approach, as it hinders the ability to fully capture and represent the domain’s knowledge. To solve such limitations, we aim to exploit GPT-3.5 knowledge. Our approach does not ask GPT-3.5 to create a visual representation; instead, it needs to suggest missing concepts, thus helping the modeler improve his/her model. The GPT-3.5 may need to refine its suggestions based on feedback from the modeler.

Design/methodology/approach

We initiate our semantic quality enhancement process of a BPM by first extracting crucial elements including pools, lanes, activities and artifacts, along with their corresponding relationships such as lanes being associated with pools, activities belonging to each lane and artifacts associated with each activity. These data are systematically gathered and structured into ArrayLists, a form of organized collection that allows for efficient data manipulation and retrieval. Once we have this structured data, our methodology involves creating a series of prompts based on each data element. We adopt three approaches to prompting: zero-shot, few-shot and chain of thoughts (CoT) prompts. Each type of prompting is specifically designed to interact with the OpenAI language model in a unique way, aiming to elicit a diverse array of suggestions. As we apply these prompting techniques, the OpenAI model processes each prompt and returns a list of suggestions tailored to that specific element of the BPM. Our approach operates independently of any specific notation and offers semi-automation, allowing modelers to select from a range of suggested options.

Findings

This study demonstrates the significant potential of prompt engineering techniques in enhancing the semantic quality of BPMs when integrated with LLMs like ChatGPT. Our analysis of model activity richness and model artifact richness across different prompt techniques and input configurations reveals that carefully tailored prompts can lead to more complete BPMs. This research is a step forward for further exploration into the optimization of LLMs in BPM development.

Research limitations/implications

The limitation is the domain ontology that we are relying on to evaluate the semantic completeness of the new BPM. In our future work, the modeler will have the option to ask for synonyms, hyponyms, hypernyms or keywords. This feature will facilitate the replacement of existing concepts to improve not only the completeness of the BPM but also the clarity and specificity of concepts in BPMs.

Practical implications

To demonstrate our methodology, we take the “Hospitalization” process as an illustrative example. In the scope of our research, we have presented a select set of instructions pertinent to the “chain of thought” and “few-shot prompting.” Due to constraints in presentation and the extensive nature of the instructions, we have not included every detail within the body of this paper. However, they can be found in the previous GitHub link. Two appendices are given at the end. Appendix 1 describes the different prompt instructions. Appendix 2 presents the application of the instructions in our example.

Originality/value

In our research, we rely on the domain application knowledge provided by ChatGPT-3 to enhance the semantic quality of BPMs. Typically, the semantic quality of BPMs may suffer due to the modeler's lack of domain knowledge. To address this issue, our approach employs three prompt engineering methods designed to extract accurate domain knowledge. By utilizing these methods, we can identify and propose missing concepts, such as activities and artifacts. This not only ensures a more comprehensive representation of the business process but also contributes to the overall improvement of the model's semantic quality, leading to more effective and accurate business process management.

用于增强业务流程模型语义的即时工程技术
目的 "知识 "一词是指特定学科所研究的世界的一部分,包括特定的分类、词汇、概念、理论、研究方法和论证标准。我们的方法通过利用大型语言模型(LLM)提供的领域知识,使用领域知识来提高业务流程模型(BPM)的质量。在这些模型中,ChatGPT 是能够提供深入领域知识的 LLM 的杰出范例。覆盖面不足是每种方法的一个局限,因为它妨碍了全面捕捉和表述领域知识的能力。为了解决这种局限性,我们的目标是利用 GPT-3.5 知识。我们的方法并不要求 GPT-3.5 创建可视化表示;相反,它需要建议缺失的概念,从而帮助建模者改进其模型。GPT-3.5 可能需要根据建模者的反馈来完善其建议。设计/方法/途径我们首先提取关键元素,包括池、车道、活动和工件,以及它们之间的相应关系,如车道与池相关联、活动属于每个车道、工件与每个活动相关联,以此启动 BPM 的语义质量增强流程。这些数据会被系统地收集并结构化为 ArrayLists,这是一种有组织的集合形式,可实现高效的数据操作和检索。有了这些结构化数据后,我们的方法就是根据每个数据元素创建一系列提示。我们采用了三种提示方法:零镜头、少镜头和思维链(CoT)提示。每种类型的提示都是专门设计的,以独特的方式与 OpenAI 语言模型进行交互,旨在引出一系列不同的建议。当我们应用这些提示技术时,OpenAI 模型会处理每个提示,并返回一个针对 BPM 中特定元素的建议列表。我们的方法独立于任何特定的符号,并提供半自动化功能,允许建模者从一系列建议选项中进行选择。 研究结果这项研究表明,当与 ChatGPT 等 LLM 集成时,提示工程技术在提高 BPM 语义质量方面具有巨大的潜力。我们对不同提示技术和输入配置下的模型活动丰富度和模型工件丰富度进行的分析表明,精心定制的提示可以带来更完整的 BPM。这项研究为进一步探索如何在 BPM 开发中优化 LLM 迈出了一步。研究局限/影响局限在于我们评估新 BPM 语义完整性所依赖的领域本体。在我们未来的工作中,建模者将可以选择询问同义词、次同义词、超同义词或关键词。这一功能将有助于替换现有概念,从而不仅提高 BPM 的完整性,而且提高 BPM 中概念的清晰度和具体性。在我们的研究范围内,我们精选了一组与 "思维链 "和 "少量提示 "相关的指令。由于表述方式的限制和说明的广泛性,我们没有将每一个细节都包含在本文正文中。不过,这些细节可以在前面的 GitHub 链接中找到。最后给出了两个附录。附录 1 描述了不同的提示指令。在我们的研究中,我们依靠 ChatGPT-3 提供的领域应用知识来提高 BPM 的语义质量。通常情况下,由于建模者缺乏领域知识,BPM 的语义质量可能会受到影响。为了解决这个问题,我们的方法采用了三种旨在提取准确领域知识的提示工程方法。利用这些方法,我们可以识别并提出缺失的概念,如活动和工件。这不仅能确保更全面地表示业务流程,还有助于全面提高模型的语义质量,从而实现更有效、更准确的业务流程管理。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
8.60
自引率
9.80%
发文量
58
期刊介绍: Business processes are a fundamental building block of organizational success. Even though effectively managing business process is a key activity for business prosperity, there remain considerable gaps in understanding how to drive efficiency through a process approach. Building a clear and deep understanding of the range process, how they function, and how to manage them is the major challenge facing modern business. Business Process Management Journal (BPMJ) examines how a variety of business processes intrinsic to organizational efficiency and effectiveness are integrated and managed for competitive success. BPMJ builds a deep appreciation of how to manage business processes effectively by disseminating best practice. Coverage includes: BPM in eBusiness, eCommerce and eGovernment Web-based enterprise application integration eBPM, ERP, CRM, ASP & SCM Knowledge management and learning organization Methodologies, techniques and tools of business process modeling, analysis and design Techniques of moving from one-shot business process re-engineering to continuous improvement Best practices in BPM Performance management Tools and techniques of change management BPM case studies.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信