October 2019 Download this article as a PDFAbstract

This article provides new knowledge on the long-term use and value of a digital user involvement tool as a part of a living lab particularly in ICT, health and public service development contexts. Research has been carried out within the authentic living lab environment in 2011–2018. Empirical evidence is gathered from case living lab digital user involvement platform and activities conducted in multiple contexts. The primary source of information are the 70 in-depth interviews with the customer companies, public organizations and other stakeholders. The digital user community and user involvement tool-specific value for the development of products and services are a fast, easy and efficient user involvement regardless of time and location, tailored online methods based on the need of the customer, and the richness and quality of the end-user feedback.

1. Introduction

The significance of users in generating commercially feasible innovations has been recognized for decades, for example, von Hippel introduced the concept of User Innovation in the 80s (von Hippel, 1986; Herstatt and von Hippel, 1992). After the Open Innovation (OI) approach (Chesbrough, 2003) emerged in new service development, elaborate networks in which companies co-create to generate new products and services have been increasingly researched and established (Chesbrough and Appleyard, 2007; Chesbrough, Lettl and Ritter, 2018). The main shared thought in user innovation and open innovation approaches is an acknowledged need for external knowledge for innovation (West and Bogers, 2014). According to Wilkinson and De Angeli (2014) among others, the inclusion of users throughout the design process is crucial to the improved adoption of final solutions. They state that the examination of user needs has been prosperous in particular for the development of new products. The significance of open innovation and end-user involvement has been recognized also at the European Union level where the living labs strategy was established in the 2000s, and furthered with the promotion of open and collaborative innovation processes (Curley, 2016; Salmelin, 2016). According to a recent report from the European Union open innovation working group (ERAC, 2019), open innovation means that civil society, science, industry, and government work together in dynamic, diverse innovation ecosystems. The report suggests living labs as an example of innovation centres that are being established in universities and other public organizations.

The living lab approach, resting upon OI and user innovation paradigms, has been in the eye of scholars since the 2000s. According to Almirall et al. (2012), living labs are driven by two main ideas. Users are equal co-creators with other participants, and experimentation is conducted in real-world settings. Living labs are seen as an appropriate choice of innovation methodology when the fit of a particular technology or a set of technologies to a precise context is significant. A broad variety of slightly different living lab definitions can be found in the literature (see Leminen, 2015). In this article, a living lab refers to a network that integrates both user-centric research and open innovation (Leminen, Westerlund and Nyström, 2012), and where users and other relevant stakeholders are being involved in innovating and developing products and services in a real-life environment. Living labs are seen as a multidisciplinary research area with influences from innovation management studies, among other fields. The main elements of a living lab are co-creation, exploration, experimentation and evaluation (ERAC, 2019), characterised by a multi-method approach and active user involvement (ENoLL, 2019).

There is still a need for further studies of living lab processes and methods (Følstad, 2008; Dell’Era and Landoni, 2014), and of their implementation as well as value, which prior research has not paid enough attention. Due to the temporary, pragmatic and heterogeneous nature of living lab initiatives, their impact evaluation typically stays on the descriptive level (Ballon, van Hoed and Schuurman, 2018). However, according to researchers, there is growing demand for long-term living lab studies that serve to help practitioners succeed in their living lab activities (Rosado et al., 2015; Westerlund, Leminen and Rajahonka, 2018). Hence, in order to foster innovation and to facilitate responsible innovations, it is of utmost significance to understand the value of the living lab approach. As globalization, digitalization and competition drive the dynamic pace of change in the modern world, disciplines focussing on innovation, including living labs, are not left without influence, as the role of digital tools in open innovation activities has been emphasized. Thus, the long-term study of a digital user involvement tool as part of a living lab brings novel knowledge. It regards the value of this type of tool, as well as methods for user involvement in product and service development in several contexts.

2. Digital User Involvement in Living Lab Environment

As a multi-method approach is characteristic of living labs, a broad variety of user involvement methods have been utilized in living lab activities. A living lab is both a concept and a methodology. It combines different types of research methods including traditional and ICT enabled methods (Tang et al., 2012; Tang and Hämäläinen, 2014). According to a literature review by Følstad (2008), the user involvement methods in living labs typically consist of ethnographic methods like observation as well as other methods such as interviews, questionnaires and focus groups. Although traditional methods have been perceived as suitable for at least some living lab studies, they have not demonstrated any major methodological advances.

While the possibilities from ICT have emerged, new technology-enabled innovation methods have also received growing attention. A shift from user-centric towards community-centric involvement has taken place, but there are still only a few studies regarding the potential of, for example, a digital living lab's user communities. Community interaction, commitment and co-creation to achieve positive results in digital user communities for innovation purposes are essential (Brandtzaeg et al., 2010). Veeckman et al. (2013), recommend that a living lab should have access to a specific group of users, since there is a often a time-consuming need to recruit users for each living lab activity. Furthermore, strong community support is needed to keep users motivated to participate in living lab activities. Innovation taking place through open innovation communities (West and Bogers, 2014), and user communities with the help of collaborative digital tools has been connected to great disruptive potential through cost- and time-saving in research and innovation activities (Brandtzaeg et al., 2010; Curley, 2016). Piller, Ihl and Vossen (2010) used the term ‘customer community’ to refer to Internet-based communities or virtual meeting places that are based upon shared enthusiasm and knowledge concerning products or services. They divided customer communities into product-related discussion forums, and communities of creation where novel ideas and concepts are formed. Digital user involvement and collaborative digital tools have become part of a common method used in living labs, nevertheless, long-term research about it is missing (Leminen and Westerlund, 2017). According to Ståhlbröst and Holst (2013), IT based tools and methodologies in living labs can function as twin-world mediators that facilitate an interconnection between real-world devices and their virtual counterparts. The activities carried out in online contexts are thus both real and realistic to actors. However, the literature on innovation system value based on digital user communities is still scarce (Arnkil et al., 2010; De Moor et al., 2010; Xie and Jia, 2016; Huang et al., 2018).

The case of a digital user involvement tool and user community

A digital user community and user involvement tool PATIO with over 1000 voluntary registered users has been utilised in the activities of a local living lab since 2011 (Anttiroiko 2016; Huang et al, 2018; Haukipuro, 2019). PATIO provides companies, organizations and research institutes an opportunity to participate in the development of products and services through an easy-to-deploy digital tool. The aim is to bring together product or service developers and potential users for product or service development or co-creation. Since 2011, more than one hundred different test projects or activities have been carried out using PATIO. The activity spectrum has varied from idea generation to evaluation and testing of, for instance, mobile applications, devices, or diverse public services, as well as field-specific solutions. The main methods of PATIO include online discussion, surveys, user diaries and an evaluation jury.

A typical activity in PATIO starts with identifying customer needs. Customers usually need test users for some product or service development-related activity, which they can conduct by themselves, or specialised living lab l services and methods for user involvement. Sometimes, the best way to collect user experience is a survey, in cases when there is a need for a large amount of responses or quantitative data. In contrast, interactive online discussions offer a well-working qualitative method for a public (open) or a selected (closed) group of users. The PATIO user diary, based on the diary study research method that is used to collect qualitative data about user behaviour, activity and experiences over time (Flaherty, 2016), has been used to collect the user experiences of, for example, a home-tested product or report a user’s observations regarding a topic through user-sent pictures. User diary and survey contents are visible only to the user and PATIO moderator, whereas a forum discussion is visible to all accepted participants. PATIO activities can be set as public (anyone can see the content, but only registered users can comment on the forum), or private (only participants accepted by the moderator can see the content). After user studies have been implemented and data collected, the next step is to analyze and report the findings to the customer. Or, in case the customer will analyze the data themselves, the raw data are given to them.

Without a pool of registered users, PATIO would be a mere tool or collection of online methods. Thus, the importance of the user community cannot be overemphasized. The PATIO user community has been growing constantly from just a few active users in 2010 into an active community of more than one thousand users in 2018. The increase in the number of users has been recognized as a twofold phenomenon: attracting new users to register, arises from PATIO having interesting content (Laizane and Haukipuro, 2012; Huang et al., 2018). Thus, while the activities and topics in PATIO have been diverse, the user community is diverse as well.

The principle of PATIO is that users are anonymous to each other, and participate on a voluntary unpaid basis. Depending on activity, users can participate through an online discussion forum, where user identities are not revealed, but nick names are used, a survey, a user diary or various on-site activities such as user testing, focus group discussions or co-creation events. The online discussion in PATIO differs from a classic discussion forum. In PATIO, discussions are always moderated and led by a PATIO moderator(s), and preferably also by a customer representative. PATIO discussion topics can be opened by the moderators only, which makes the activities systematic and focused, yet enables interaction between users.

3. Methodology

The benefits of the case study approach have been recognized in different fields of qualitative research. Yin (1989, 2005) defines the case study as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context when the boundaries between phenomenon and context are not clearly evident and in which multiple sources of evidence are used”. The fundamental thought behind case research is the multifaceted view it can provide of a situation in its context (Halinen and Törnroos, 2005). The relation between a phenomenon and its context can be understood through the case study approach (Dubois and Gadde, 2002). Compared to the quantitative research approach, depth and comprehensiveness (Easton, 1995) are the defining characteristics of qualitative case research. Hence, the case study enables deep understanding of a specific phenomenon and is particularly suitable for exploration of a new or unique phenomenon (e.g. Eisenhardt, 1989).

Considering the nature of the living lab research environment from which the mainly qualitative research data were gathered, the case study design was regarded as an appropriate approach. The living lab network can be comparable to contemporary business networks, for which case study methods are recommended (Halinen and Törnroos, 2005). Furthermore, when aiming to increase understanding of a living lab environment and user-centric methods in different contexts, the study seeks to answer the “how” and “why” questions which are typical for case studies (Yin, 2005). Stake (1995) emphasized the advantages of case studies in terms of providing new insights for stakeholders, as a case study facilitates the investigation of a research subject in a real-life context.

The data collection methods utilized in this research consist of semi-structured in-depth interviews, discussions, meetings, meeting memos, workshop data, different documentation of activities, reports and a vast amount of data collected through the case's digital user involvement tool PATIO in 2011–2018. Additional project-specific data such as numerous meeting memos were available. Interviews (Arksey and Knight, 1999) were used as the primary data collection method, consisting of altogether 70 semi-structured in-depth interview sessions conducted in 2013–2018 with identified key informants such as customer company representatives, public sector service providers, researchers and other stakeholders. In several sessions, multiple informants were present. All interviews were recorded, transcribed, along with notes.

In the data analysis, triangulation (Golafshani, 2003), thematic analysis (e.g., Aronson, 1994), and categorization techniques were applied. Triangulation (Denzin, 1973), the use of multiple data collection and analysis methods to search for convergence (Golafshani, 2003), was applied. Data collection and data analysis were conducted concurrently as this helped identify gaps in the collected data (Miles and Huberman, 1994; Miles, Huberman and Saldaña, 2014).

4. Findings

The study contributes to the literature regarding citizen participation and living labs, and the development of efficient digital tools in this context. As previous research has not focused enough on citizen participation in innovation processes, the study contributes to this deficiency by showing how a digital user involvement tool and user community can involve end users in the needs of both the public sector and companies. The findings show that PATIO is an appropriate tool for reaching target-group specific users when compared with traditional user recruitment and involvement, which is often reported as time-consuming and costly. Thus, PATIO can be regarded as an effective tool for user involvement and citizen participation, that has proven to work well, in service and product development, and in city planning contexts. Overall, the findings and perceptions concerning PATIO have been mainly positive, hence supporting previous findings that regarded the feasibility of PATIO in the development of products and services. However, development ideas, such as new feature proposals for the PATIO system have also been brought up by customers.

Altogether, 1825 users have participated in the activities initiated or conducted in PATIO. In addition, for direct user recruitment types of activities, the exact number of participants recruited from the PATIO user community was not always known due to external contact points. Among the cases are 9 product-, 27 application-, and 35 service-related activities. The rest consist of non-categorized activities marked as “other”. The maturity of the products/services/application regarding 25 activities has been on the idea level, 35 on the concept level, 21 on the prototype level, and 18 on the market level, or otherwise ready or already existing solutions. In some cases, overlapping or multiple categories were applicable to these activities, for instance, in the eHealth user workshop PATIO activity, in which were involved the solutions of several companies, including both product and service ideas. The activities include 27 in which the customer was a startup or SME, 6 large enterprise owned activities, 46 research institute activities, and 31 public organization-driven activities. The relatively high number of research institute activities can be explained by the location of PATIO inside the University, as well as connection to several research projects. The duration of the activities ranged from a week to a year, however, the active phase was typically not more than two weeks. The activities' purpose is idea or feedback collection regarding a product, a service or an idea, user testing and user recruitment for varying purposes, typically a user study conducted by customer. There are also extensive user research activities that combine all the aforementioned purposes and utilize a broad variety of methods. Among the methods included in the PATIO system were online discussions in the PATIO forum (used in 60 activities), user diaries (used in 6 activities), an evaluation jury (3), and surveys (24), which can mean a survey implemented by PATIO or a survey implemented by a customer that was embedded in the PATIO survey page.

Table 1. PATIO activities in numbers

PATIO's context-specific use

The three main contexts in which PATIO has been used for digital user involvement are ICT, health and public service development. A large part of all activities conducted in PATIO have been ICT related, with user involvement in the development and testing of mobile applications and devices. PATIO has been tried to recruit users for testing, and also collecting user experience through surveys, user diaries and online discussions. For instance, ten local families tested a device in their homes and reported their use experience through the PATIO user diary. In another study, 25 selected participants used the user diary to report their use experience by mobile camera device. In both studies, surveys were also used. Hence, the customer companies received a large amount of rich data collected via multiple methods that could be used for further development of the products (Haukipuro, 2019).

Furthermore, in a 2018 activity conducted in PATIO, 36 users in total participated in the evaluation of a mobile application aimed at influencing public decision-making (Huang et al., 2018; Haukipuro, 2019). Characteristic for ICT-based development activities is that they are short-term and take place in certain phases of development, including user testing of a prototype or concept evaluation. According to customer feedback collected at the end of each activity, companies typically received improvement ideas, such as new feature proposals for their products, reports of bugs found in the software, usability issues, and overall feedback that has helped companies improve the quality of their products and solutions. In optimal cases, the user testing occurs before launch, when changes are still possible and cost-efficient to implement compared to after launching. This has been the case in most of the activities conducted, though there have been a few cases in which the results of user testing had a drastic and unwanted impact: a decision to terminate the tested product or solution (Haukipuro, 2019).

PATIO has been used for health-related user involvement in several activities such as developing eHealth product prototypes, and developing and testing health products, services and processes (Haukipuro, 2019). For example, PATIO was part of a new hospital innovation process where PATIO’s evaluation jury feature was used in the evaluation of companies’ development ideas and concepts within a hospital environment. Furthermore, PATIO was used for engaging health professionals and companies in digital co-creation through surveys and online discussion based on, for example, health product concepts and prototypes (Haukipuro, Väinämö and Hyrkäs, 2018). It was found that digital tools can be useful also in a traditional and hierarchy-based organization’s innovation activities, although compared to other use environments, successfully using them requires a lot of preparation and guidance. These activities initiated a new, long-term hospital innovation procedure in which digital tools have a significant role.

PATIO has been part of public service development activities in the context of smart city development of virtual services and urban planning (Haukipuro, 2019). The online discussion forum and surveys were used to collect citizen insights on public services in different occasions. Virtual services utilized the PATIO discussion forum for two different purposes. First, a collection of general citizens insights towards virtual services was featured in a public discussion. Second, a separate discussion for the employees and the authorities providing services was organized. In the case of the new city district development, the discussion forum and surveys were applied to engage citizens in urban planning. The citizen involvement process was repeated in several phases as the planning proceeded.

Summary of the main value of PATIO

The findings rely on the case studies and the empirical data collected in 2011–2018 in diverse living lab activities. The categories were decided after analysing the information collected from customers and other stakeholders, and through knowledge and know-how gained from the use of the PATIO tool for different purposes. To summarize the findings regarding the digital user community and user involvement tool PATIO in the development of products and services, the main value of PATIO for public and private sector customers can be summarised with eight categories:

Cost-efficiency refers to resource savings as customers make use of PATIO as a cost-effective tool for user recruitment, user testing, and moderation. For example, user recruitment is often regarded as a time-consuming and costly task, especially for small companies. The PATIO tool tackles this challenge by enabling easy and fast user recruitment. Target-group-specific users can be found without much effort from the database.

Customers also value PATIO’s timing & flexibility as it enables iterative product or service development in different phases, such as the idea phase, concept phase, and prototype phase. Methods can be tailored according to the needs of the customer, for example, in-depth online discussion or a user diary is perceived as valuable by some customers, whereas a survey is preferred on other occasions.

Ease of use; PATIO is perceived as easy to use by customer organization representatives who have actively participated in the moderation of online activities. The use of the tool does not require any specific technical skills.

Customers have been satisfied with the quality of the results obtained from PATIO activities. The choice of which methods to use in order to achieve good results requires expertise. As PATIO activities are mostly facilitated and planned by experts, the quality of results is perceived as good. One user diary by a researcher was perceived to help keep the activity focused. Multiple methods and a diverse user community tend to produce rich data. Especially moderated in-depth online discussions may provide valuable information regarding the everyday life of citizens. Fast and easy user involvement, user recruitment, user screening and feedback collection are PATIO's main asset. Users can be recruited for online activities, on-site user testing, or a combination of both. Users can be easily reached for discussion online after on-site user testing when needed. PATIO’s user community consists of people with diverse backgrounds: students, technology enthusiasts, elderly people, and professionals.

Open & closed participation is enabled in PATIO through open (public) activities, such as online discussion that anyone can view (even if not registered) and contribute to (when registered), and closed (private) activities to which users willing to participate are selected through certain criteria provided by customers. Each way of participating has its advantages: open activity can be seen to increase information and visibility of a certain theme such as city planning, whereas closed activity is perceived to increase the commitment of public sector employees, serving as a virtual meeting place for employees that might be located far away from each other, and enable easy in-depth data collection.

PATIO's multi-method approach is also valuable for customers. The main methods are online discussion (open or closed), surveys and user diaries. An evaluation jury as a method is tailored for easy and anonymous involvement of professionals and others for various evaluation purposes, irregardless time and place. Each method can be used individually, or all can be used together within the same activity. The methods are tailored based on customer needs, which ensures quality results. The use of multiple methods can also increase the reliability of results through parallel findings via different methods.

PATIO also supports sustainability through time and place independence, which is important for customers whose aim is to provide virtual meeting places, for employees or organizations to have an evaluation jury in PATIO, or to enable international user involvement encompassing environmental sustainability.

5. Discussion

Considering the under-researched area of digital user involvement within living lab environments, this article provides new knowledge that builds on long-term data from living lab activities in several contexts, such as ICT, health and public service development. The activities conducted in these areas show that there are some differences, for example, that ICT related user involvement activities typically are short-term, and usually take place in a certain phase of development such as prototype testing. In health and public service development contexts, the activities have been long-term user involvement repeated in different phases of development, for example, in city planning. The applicability of a digital user involvement tool and methods also differs within contexts. In particular, the health context differed from other contexts, in that the use of the digital tool and methods as a part of innovation activities required more preparation and guidance. However, regardless of the context, digital user involvement furthered product and service development.

The wide-ranging empirical data collected through PATIO from different types of customers and living lab activities has provided new knowledge about the use, applicability and value of this type of digital tool as part of a living lab. The findings show the value of PATIO as an easy, cost- and resource-effective way to involve users in various development activities, through a multi-method approach. From the local ecosystem perspective, PATIO has played an important role in the promotion of user-centric development practices among local businesses and the public sector because, among other reasons, PATIO has provided companies and organizations with a new, easy and efficient way to promote and carry out user-centric development activities facilitated by local living labs. Presumably, user testing, online discussions, surveys, and other user and citizen involvement activities conducted through PATIO have influenced the development of usable, desirable, and successful products and services. A combination of user community management and data collection through several methods differentiates PATIO from typical user involvement tools such as surveys posted to email lists. Figure 1 depicts the PATIO model for digital user involvement.

Based on the experience and numerous studies conducted using PATIO, the right timing has been recognized as important when conducting activities; the earlier the feedback is collected, the easier it is to consider end-user feedback and apply it, for example, with modifications to products or services. However, experience with PATIO activities conducted too early shows that in an activity where the aim was to collect feedback on healthcare product concepts from hospital professionals, feedback was not beneficial from the development point of view, as the product concept was not mature enough. One of PATIO’s identified strengths is the results quality as it enables collection of in-depth user experiences through a multi-method approach, facilitated by the living lab. Based on customer interviews, feedback and data collected from more than 100 activities conducted in PATIO, the few customers who were not pleased with the results had considered PATIO as merely a survey tool for reaching the masses, whereas customers who obtained qualitative in-depth data were most satisfied with the results. Hence, the change of attitude also requires clarifying this essential difference between basic online survey tools, discussion forums or social media. We discovered that PATIO's main asset, its user community, consists of diverse, motivated people who are willing to participate and devote time for evaluating and testing products and services. Fostering community vitality and development through providing and promoting diverse living lab activities is especially important as the diverse and active user community is crucial for the vitality of PATIO and on a larger scale, the whole living lab.

Figure 1. The PATIO digital user involvement model.

Considering the under-researched area of collaborative digital innovation tools (De Moor et al., 2010; West and Bogers, 2014; Leminen and Westerlund, 2017) that can have a significant role in living labs in the increasingly digitalized world, findings regarding the long-term use of this type of tool are significant for researchers as well as practice-oriented living lab stakeholders and customers. Novel long-term knowledge acquisition regarding user involvement in living lab activities through PATIO contributes to research on living lab practices (e.g. Veeckman et al., 2013; Schuurman et al., 2016) in terms of increasing our understanding of the value of these types of tools and methods in enhancing living lab practices. The findings show that early involvement of end-users and stakeholders can save resources and costs, and enhance the quality of products and services. Managers should utilize user-centric development services provided by living labs to ensure their products or services meet the needs of target group customers. The findings also provide information for managers regarding, online user involvement methods. According to the findings, online methods should be utilized for solutions mature enough (at least at the concept-level) in order to obtain best results. Furthermore, the facilitation of online involvement requires a kind of expertise that companies often do not possess. Thus, a living lab’s expertise is recommended to be utilized for online user involvement activities. To conclude, we believe that end-user and citizen involvement in different product and service development activities through the living lab approach should become rather a normal, common and continuous practice, rather than just a temporary experiment.

6. Conclusions

This article presented several benefits of digital user involvement in a living lab environment. Based on the findings, digital user community and user involvement tool-specific benefits for developing products and services can be summarized as fast, easy and efficient user involvement, regardless of time and location, tailored online methods based on the need of the customer, and rich quality of end-user feedback. In more detail, the PATIO-specific value categories are identified as Cost-efficiency, Timing & flexibility, Ease of use, Quality of the results, User involvement, Open & closed participation, Multi-method approach and Sustainability. According to the findings, online methods should be utilized for solutions mature enough (e.g. concept-level) in order to obtain best results. (Duplicated in previous paragraph)

As the need for user involvement knowledge and practices in product and service development activities has been raised among companies, public service developers and researchers, this article responds to this need by providing new knowledge on the long-term use and value of a digital user involvement tool as part of a living lab. The findings of the study encourage managers to utilize the services provided by living labs in order to ensure the use of appropriate living lab methods and tools to obtain the best results. Furthermore, the study stresses the value of end-user involvement for companies at the correct phases of product and service development.

The long-term experience and results of using this type of tool in product and service development activities shows that the combination of an active user community and tailored online methods makes user involvement smooth, easy and adaptable to a diverse context such as ICT, health and public service development activities. Findings of the study promote the use of digital user involvement mechanisms in daily living lab activities. However, as this study focused on the customer perspective, further research taking into account the end-user and facilitator perspective is also needed, regarding how to maintain, manage and motivate a user community, and how to select the most suitable online methods for each environment and activity to achieve the best results. Accordingly, the long-term impact of this type of tool and methods (e.g. for customer companies) should be researched further through follow-up studies within a certain time period.


This article is based on the first author’s PhD research (Haukipuro, 2019).


Key concepts

Context here means the interrelated conditions in which something exists or occurs (Merriam-Webster, 2018).

Customer value defined by Woodruff (1997) is “a customer's perceived preference for and evaluation of those product attributes, attribute performances, and consequences arising from use that facilitate (or block) achieving the customer's goals and purposes in use situations”.

ICT refers to Information and Communications Technology.

Innovation refers here to the definition by Skillicorn (2016): “Executing an idea, which addresses a specific challenge and achieves value for both the company and customer.”

Living lab is “a network that integrates both user-centric research and open innovation” (Leminen et al., 2012), and where users and other relevant stakeholders are involved to innovate and develop products and services in a real-life environment.

Living lab approach refers here to the use of living lab methods and tools in the development of products and services.

Open innovation (OI) refers to innovation in which a company’s outside innovation sources are taken advantage of: “The use of purposive inflows and outflows of knowledge to accelerate internal innovation, and expand the markets for external use of innovation, respectively” (Chesbrough et al., 2006, p.1).

SME (Small and Medium-sized Enterprise) is defined according to EU recommendation (European Commission, 2003), i.e., by the number of employees (<250), turnover (<50 € million), and balance sheet total (<43€million).

User involvement refers here to product or service development activities, in which end-users are considered co-developers through various methods. The central notion in the research of user involvement means moving users from being objects of research to become active participants.

User-centric development adopts the principles of user-centered / user-centred design (User-Centred Design, 2009), considering all phases of the product life cycle, including users and use contexts of activities, such as prototyping, implementation and testing. The objective of user-centric development can be both improving an existing product and developing new products.



Almirall, E., Lee, M. and Wareham, J. 2012. Mapping living labs in the landscape of innovation methodologies. Technology Innovation Management Review, 2(9): 12–18.

Anttiroiko, A. 2016. City-as-a-platform: The rise of participatory innovation platforms in Finnish cities. Sustainability, 8(9): 2–31.

Arksey, H. and Knight, P. T. 1999. Interviewing for social scientists. London: Sage.

Arnkil, R., Järvensivu, A., Koski, P. and Piirainen, T. 2010. Exploring quadruple helix outlining user-oriented innovation models. Tampere: Tampereen yliopistopaino Oy Juvenes Print.

Aronson, J. 1994. Pragmatic View of Thematic Analysis. The Qualitative Report, 2 (1). Available at: http://www.nova.edu/ssss/QR/BackIssues/QR2-1/aronson.html. [Accessed 23 Jan. 2018].

Ballon, P., Pierson, J. and Delaere, S. 2005. Test and experimentation platforms for broadband innovation: Examining European practice. Brussels: Vrije Universiteit Brussel.

Ballon, P., Van Hoed, M. and Schuurman, D. 2018. The effectiveness of involving users in digital innovation: Measuring the impact of living labs. Telematics and Informatics. doi: 10.1016/j.tele.2018.02.003.

Brandtzæg P.B., Følstad A., Obrist M., Geerts D., & Berg, R. 2010. Innovation in online communities – towards community-centric design. In Daras P., & Ibarra, O. M. (eds.), User centric media. UCMEDIA 2009. Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering Vol.40. Berlin, Heidelberg: Springer.

Chesbrough, H. 2003. Open Innovation: The New Imperative for Creating and Profiting from Technology. Boston, MA: Harvard Business School Press.

Chesbrough, H., Vanhaverbeke, W., & West, J. 2006. Open Innovation: Researching a New Paradigm, Oxford University Press, Oxford.

Chesbrough, H.W. and Appleyard, M. 2007. Open innovation and strategy. California Management Review, 50(1): 57–76.

Chesbrough, H., Lettl, C. and Ritter, T. 2018. Value creation and value capture in open innovation. Journal of Product Innovation Management, 35(6): 930–938.

Curley, M. 2016. Twelve principles for open innovation 2.0: evolve governance structures, practices and metrics to accelerate innovation in an era of digital connectivity. Nature, 533(7603): 314–316.

De Moor, K., Ketyko, I., Joseph, W., Deryckere, T., De Marez, L., Martens, L. and Verleye, G. 2010. Proposed framework for evaluating quality of experience in a mobile, testbed-oriented living lab setting. Mobile Networks and Applications, 15(3): 378–391.

Dell’Era, C. and Landoni, P. 2014. Living Lab: A Methodology between User-Centred Design and Participatory Design. Creativity and Innovation Management, 23(2): 137–154. [online] Available at: http://doi.org/10.1111/caim.12061 [Accessed 15 Jan. 2018].

Denzin, N. K. 1973. The research act (3rd ed.). Chicago: Aldine.

Dubois, A. and Gadde, L. 2002. Systematic combining: An abductive approach to case research. Journal of Business Research, 55(7): 553–560.

Eisenhardt, K.M. 1989. Building theories from case study research. The Academy of Management Review, 14(4): 532–550.

ENoLL. 2019. About Us. European Network of Living Labs (ENoLL). [online] Available at: https://enoll.org/about-us/ [Accessed 1 Apr. 2019].

ERAC. 2019. Opinion on Open Innovation. ERAC Standing Working Group on Open Science and Innovation. [online] Available at: https://data.consilium.europa.eu/doc/document/ST-1203-2019-INIT/en/pdf. [Accessed 1 Apr. 2019].

European Commission (EC). 2003. What is an SME? Retrieved from http://ec.europa.eu/growth/smes/business-friendly-environment/sme-definition_fi

European Commission (EC), 2016. The Open Innovation Yearbook 2016. Luxembourg: European Commission.

Flaherty, K. 2016. Diary Studies: Understanding Long-Term User Behavior and Experiences. Nielsen Norman Group. [online] Available at: https://www.nngroup.com/articles/diary-studies/ [Accessed 1 Apr. 2018].

Følstad, A. 2008. Living labs for innovation and development of information and communication technology: A literature review. The Electronic Journal for Virtual Organizations and Networks, 10: 99–131.

Golafshani, N. 2003. Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4): 597.

Halinen, A. and Törnroos, J. 2005. Using case methods in the study of contemporary business networks. Journal of Business Research 58(9): 1285–1297.

Haukipuro, L., Pakanen, M. and Väinämö, S. 2016. Online User Community for Efficient Citizen Participation. In: The 20th International Academic Mindtrek Conference (ACM): 78–85. [online] New York, NY: Academic Mindtrek. Available at: http://doi.org/10.1145/2994310.2994341 [Accessed 10 May 2016].

Haukipuro, L., Väinämö, S. and Hyrkäs, P. 2018. Innovation Instruments to Co-Create Needs-Based Solutions in a Living Lab. Technology Innovation Management Review, 8(5): 22-35. Available at: http://doi.org/10.22215/timreview/1156

Haukipuro, L. 2019. User-centric product and service development in a multi-context living lab environment: case OULLabs and PATIO. PhD. University of Oulu.

Herstatt, C. and von Hippel, E. 1992. From experience: Developing new product concepts via the lead user method: A case study in a “low-tech” field. The Journal of Product Innovation Management, 9(3): 213–221.

Huang, W., Pakanen, M., Haukipuro, L., Väinämö, S. and Arhippainen, L. 2018. Motivate Online Users by Moderating and Providing Tasty Testing Experiences. The 22nd Conference of Open Innovations Association FRUCT, Jyväskylä, Finland, 15–18 May 2018.

Laizane, S. and Haukipuro, L. 2012. Preliminary experiences with the online forum PATIO in a multi-contextual living lab environment. In: The ISM 2012 Workshop Innovation through Social Media. December 1–2, 2012, Oslo, Norway.

Leminen, S., Westerlund, M. and Nyström, A.-G. 2012. Living Labs as Open-Innovation Networks. Technology Innovation Management Review, 2(9): 6–11.

Leminen, S. 2015. Living labs as open innovation networks – networks, roles and innovation outcomes. PhD. Aalto University.

Leminen, S. and Westerlund, M. 2017. Categorization of Innovation Tools in Living Labs. Technology Innovation Management Review, (7)1: 15–25.

Miles M.B. and Huberman, A.M. 1994. Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage Publications.

Miles, M.B., Huberman, A.M. and Saldaña, S. 2014. Qualitative Data Analysis. A Methods Sourcebook. (3rd ed.). CA: Sage Publications.

PATIO 2018.  [online] Available at: http://www.PATIOlla.fi/en. [Accessed 10 Apr. 2018].

Piller, F.T., Ihl, C. and Vossen, A. 2010. A Typology of Customer Co-Creation in the Innovation Process. December 29, 2010. Rochester, NY.

Raunio, M., Nordling, N., Ketola, T., Saarinen, J.P. and Heinikangas, A. 2016. Open Innovation Platforms – an approach to city development. Handbook for developers. Tampere: 6Aika. Available at: https://avoimetinnovaatioalustat.files.wordpress.com/2016/02/kc3a4sikirja_eng.pdf [Accessed 2 Apr. 2019].

Rosado, L., Hagy, S., Kalmykova, Y., Morrison, G. and Ostermeyer, Y. 2015. A living lab co-creation environment exemplifying factor 10 improvements in a city district. Journal of Urban Regeneration & Renewal, 8(2): 171–185.

Salmelin, B. 2016. Living Labs and open Innovation in European Context. In: Mention, A., and Torkkeli, M. (Eds.), Open innovation: A multifaceted perspective. part I: 1–18. Imperial College Press, World Scientific.

Schuurman, D., De Marez, L. and Ballon, P. 2016. The impact of living lab methodology on open innovation contributions and outcomes. Technology Innovation Management Review, 6(1): 7–16.

Skillicorn, N. 2016. What is innovation? 15 experts share their innovation definition. Idea to Value. Retrieved from https://www.ideatovalue.com/inno/nickskillicorn/2016/03/innovation-15-experts-share-innovation-definition

Stake, R. E. 1995. The art of case study research. Thousand Oaks, CA: Sage.

Ståhlbröst, A. and Holst, M. 2013. The living lab methodology handbook. Social Informatics at Luleå University of Technology and CDT – Centre for Distance-spanning Technology, Luleå, Sweden: Vinnova.

Tang, T., Wu, Z., Karhu, K., Hämäläinen, M. and Ji, Y. 2012. Internationally distributed living labs and digital ecosystems for fostering local innovations in everyday life. Journal of Emerging Technologies in Web Intelligence, 4(1).

Tang, T. and Hämäläinen, M. 2014. Beyond Open Innovation: the Living Labs way of ICT Innovation. Interdisciplinary Studies Journal, 3(4): 15–23.

User-Centred Design. 2009. In: Liu, L. & Özsu, M.T. (Eds.), Encyclopedia of Database Systems. Springer, Boston, MA.

Veeckman, C., Schuurman, D., Leminen, S. and Westerlund, M. 2013. Linking living lab characteristics and their outcomes: Towards a conceptual framework. Technology Innovation Management Review, 3(12): 6–15.

Von Hippel, E. 1976. The dominant role of users in the scientific instrument innovation process. Research Policy, 5(3): 212-239.

Von Hippel, E. 1986. Lead users. Management Science, 32(7): 791–805.

West, J. and Bogers, M. 2014. Leveraging External Sources of Innovation: A Review of Research on Open Innovation. Journal of Product Innovation Management, 31(4): 814–831. Available at: https://doi.org/10.1111/jpim.12125

Westerlund, M., Leminen, S. and Rajahonka, M. 2018. A Topic Modelling Analysis of Living Labs Research. Technology Innovation Management Review, 8(7): 40–51. Available at: http://doi.org/10.22215/timreview/1170

Wilkinson, C.R. and De Angeli, A. 2014. Applying user centred and participatory design approaches to commercial product development. Design Studies, 35(6): 614–631.

Woodruff, R.B. 1997. Customer value: The next source for competitive advantage. Journal of the Academy of Marketing Science, 25: 139.

Xie, X. and Jia, Y. 2016. Consumer involvement in new product development: A case study from the online virtual community. Psychology & Marketing, 33(12): 1187–1194.

Yin, R. K. (ed.) 1989. Case study research—design and methods (2nd ed.). Newbury Park (CA): Sage Publications.

Yin, R. K. (2005). Case study research (3rd ed.). Thousand Oaks: Sage.


Share this article:

Cite this article:

Rate This Content: 
8 votes have been cast, with an average score of 4 stars

Keywords: Digital user involvement, living lab., user community