Assisting early-stage software startups with LLMs: Effective prompt engineering and system instruction design

IF 3.8 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information and Software Technology Pub Date : 2025-07-09 DOI:10.1016/j.infsof.2025.107832

Thea Lovise Ahlgren , Helene Fønstelien Sunde , Kai-Kristian Kemell , Anh Nguyen-Duc

{"title":"Assisting early-stage software startups with LLMs: Effective prompt engineering and system instruction design","authors":"Thea Lovise Ahlgren , Helene Fønstelien Sunde , Kai-Kristian Kemell , Anh Nguyen-Duc","doi":"10.1016/j.infsof.2025.107832","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>Early-stage software startups, despite their strong innovative potential, experience high failure rates due to factors such as inexperience, limited resources, and market uncertainty. Generative AI technologies, particularly Large Language Models (LLMs), offer promising support opportunities; however, effective strategies for their integration into startup practices remain underexplored.</div></div><div><h3>Objective:</h3><div>This study investigates how prompt engineering and system instruction design can enhance the utility of LLMs in addressing the specific needs and challenges faced by early-stage software startups.</div></div><div><h3>Methods:</h3><div>A Design Science Research (DSR) methodology was adopted, structured into three iterative cycles. In the first cycle, use cases for LLM adoption within the startup context were identified. The second cycle experimented with various prompt patterns to optimize LLM responses for the defined use cases. The third cycle developed “StartupGPT”, an LLM-based assistant tailored for startups, exploring system instruction designs. The solution was evaluated with 25 startup practitioners through a combination of qualitative feedback and quantitative metrics.</div></div><div><h3>Results:</h3><div>The findings show that tailored prompt patterns and system instructions significantly enhance user perceptions of LLM support in real-world startup scenarios. StartupGPT received strong evaluation scores across key dimensions: satisfaction (93.33%), effectiveness (80%), efficiency (80%), and reliability (86.67%). Nonetheless, areas for improvement were identified, particularly in context retention, personalization of suggestions, communication tone, and sourcing external references.</div></div><div><h3>Conclusion:</h3><div>This study empirically validates the applicability of LLMs in early-stage software startups. It offers actionable guidelines for prompt and system instruction design and contributes both theoretical insights and a practical artifact — StartupGPT — that supports startup operations without necessitating costly LLM retraining.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"187 ","pages":"Article 107832"},"PeriodicalIF":3.8000,"publicationDate":"2025-07-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information and Software Technology","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950584925001715","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Context:

Early-stage software startups, despite their strong innovative potential, experience high failure rates due to factors such as inexperience, limited resources, and market uncertainty. Generative AI technologies, particularly Large Language Models (LLMs), offer promising support opportunities; however, effective strategies for their integration into startup practices remain underexplored.

Objective:

This study investigates how prompt engineering and system instruction design can enhance the utility of LLMs in addressing the specific needs and challenges faced by early-stage software startups.

Methods:

A Design Science Research (DSR) methodology was adopted, structured into three iterative cycles. In the first cycle, use cases for LLM adoption within the startup context were identified. The second cycle experimented with various prompt patterns to optimize LLM responses for the defined use cases. The third cycle developed “StartupGPT”, an LLM-based assistant tailored for startups, exploring system instruction designs. The solution was evaluated with 25 startup practitioners through a combination of qualitative feedback and quantitative metrics.

Results:

The findings show that tailored prompt patterns and system instructions significantly enhance user perceptions of LLM support in real-world startup scenarios. StartupGPT received strong evaluation scores across key dimensions: satisfaction (93.33%), effectiveness (80%), efficiency (80%), and reliability (86.67%). Nonetheless, areas for improvement were identified, particularly in context retention, personalization of suggestions, communication tone, and sourcing external references.

Conclusion:

This study empirically validates the applicability of LLMs in early-stage software startups. It offers actionable guidelines for prompt and system instruction design and contributes both theoretical insights and a practical artifact — StartupGPT — that supports startup operations without necessitating costly LLM retraining.

查看原文本刊更多论文

协助早期软件创业公司使用llm：有效的即时工程和系统指令设计

背景：早期阶段的软件初创公司，尽管具有强大的创新潜力，但由于缺乏经验、有限的资源和市场不确定性等因素，经历了很高的失败率。生成式人工智能技术，特别是大型语言模型（llm），提供了有希望的支持机会；然而，将它们整合到创业实践中的有效策略仍未得到充分探索。目的：本研究探讨快速工程和系统指令设计如何提高llm在解决早期软件初创公司面临的特定需求和挑战方面的效用。方法：采用设计科学研究（DSR）方法，分为三个迭代周期。在第一个周期中，确定了在启动上下文中采用LLM的用例。第二个周期试验了各种提示模式，以优化已定义用例的LLM响应。第三个周期开发了“StartupGPT”，这是一个为创业公司量身定制的基于llm的助手，探索系统指令设计。通过定性反馈和定量指标的结合，25名初创企业从业人员对该解决方案进行了评估。结果：研究结果表明，定制的提示模式和系统指令显著增强了用户对现实世界启动场景中LLM支持的感知。StartupGPT在满意度（93.33%）、有效性（80%）、效率（80%）和可靠性（86.67%）等关键维度上获得了很高的评价分数。尽管如此，我们还是确定了需要改进的地方，特别是在上下文保留、个性化建议、沟通语气和外部参考资料方面。结论：本研究实证验证了llm在软件创业早期的适用性。它为提示和系统指令设计提供了可操作的指导方针，并提供了理论见解和实用工件- StartupGPT -支持启动操作，而无需昂贵的法学硕士再培训。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Information and Software Technology 工程技术-计算机：软件工程

CiteScore

9.10

自引率

7.70%

发文量

164

审稿时长

9.6 weeks

期刊介绍： Information and Software Technology is the international archival journal focusing on research and experience that contributes to the improvement of software development practices. The journal''s scope includes methods and techniques to better engineer software and manage its development. Articles submitted for review should have a clear component of software engineering or address ways to improve the engineering and management of software development. Areas covered by the journal include: • Software management, quality and metrics, • Software processes, • Software architecture, modelling, specification, design and programming • Functional and non-functional software requirements • Software testing and verification & validation • Empirical studies of all aspects of engineering and managing software development Short Communications is a new section dedicated to short papers addressing new ideas, controversial opinions, "Negative" results and much more. Read the Guide for authors for more information. The journal encourages and welcomes submissions of systematic literature studies (reviews and maps) within the scope of the journal. Information and Software Technology is the premiere outlet for systematic literature studies in software engineering.