Overcoming experimentation challenges in software ecosystems of large product and service organizations: A participatory action research study

Software ecosystems facilitate collaborative innovation and value co-creation among diverse actors through shared technological platforms. However, introducing experimentation practices, such as A/B testing, into these ecosystems within large organizations presents significant challenges due to complex structures, network effects, and complicated organizational dynamics. The challenge is more difficult when it comes to product and service organizations, especially in business-to-business (B2B) or industrial domains. This paper presents an action research study, aiming to overcome the barriers to adopting experimentation-based evolution approaches, conducted within a participating large software-intensive product and service organization, with a vast portfolio of software ecosystems in a wide spectrum of business domains.

Following a participatory action research methodology, the research team worked closely with the participating organization through three iterative cycles of planning, action, observation, and reflection. Data sources included a systematic literature review, expert interviews with 25 participants across 17 software ecosystems, and collaborative workshops with internal stakeholders. The study identifies key organizational, technical, and cultural challenges to introducing experimentation in software ecosystems of large organizations, particularly in business-to-business or industrial domains, and exemplifies a roadmap for iteratively addressing such challenges.