If the only tool you have is a hammer, you tend to see every problem as a nail.
Abraham Maslow (1908–1970)
This article examines the link between innovation processes and the use of innovation tools in living labs. So doing, it develops a conceptual framework based on the literature to analyze 40 living labs in different countries. The study contributes to the discussion on living labs by introducing a new typology of living labs based on their innovation process characteristics and usage of tools: linearizer, iterator, mass customizer, and tailor. Moreover, it proposes three ways to organize innovation activities in living labs. The article concludes by providing a set of implications to theory and practice, and suggesting directions for future research on living labs.
There is a need for deeper understanding of the characteristics, processes, and tools in living labs in order to integrate them with the innovation activities of organizations (Leminen & Westerlund, 2013; Niitamo et al., 2012; Sauer, 2013; Schuurman, 2015; Ståhlbröst, 2008; Tang, 2014). Although previous literature has studied living labs as a context, a methodology, or a conceptualization (Leminen, 2015), definitions of living labs commonly address the importance of the real-life environment, the involvement of multiple stakeholders, and the multiplicity of approaches (cf. Dutilleul et al., 2010; Følstad, 2008; Fulgencio et al., 2012; Leminen, 2015; Schuurman et al., 2012; Veeckman et al., 2013; Westerlund & Leminen, 2014; Leminen & Westerlund, 2016). Following our earlier definition, this study defines living labs as “physical regions or virtual realities, or interaction spaces, in which stakeholders form public–private–people partnerships (4Ps) of companies, public agencies, universities, users, and other stakeholders, all collaborating for creation, prototyping, validating, and testing of new technologies, services, products, and systems in real-life contexts” (Westerlund & Leminen, 2011).
In particular, methods and tools in living labs are underresearched (Bergvall-Kåreborn & Ståhlbröst, 2009; Ståhlbröst, 2008). Many prior studies on living labs focus on development approaches in which artefacts such as prototypes of products and services are developed, validated, and tested with users and multiple stakeholders. These approaches comprise: i) methods coupled to different contexts; ii) phased, processual methods; and iii) differentiation of living labs from other R&D methodologies (cf. Almirall et al., 2012; Bergvall-Kåreborn et al., 2009: Budweg et al., 2011; Coenen et al., 2014; Edvardsson et al., 2012; Eriksson et al., 2005; Følstad, 2008; Guzmán et al., 2013; Mulder, 2012; Mulder & Stappers, 2009; Ponce de Leon et al., 2006; Schaffers et al., 2009; Schumacher & Feurstein, 2007; Ståhlbröst, 2008; Tang et al., 2012). Moreover, there are only a few attempts to investigate tools in living labs. Äyväri and Jyrämä (2015) focus on management tools for living labs, whereas Ståhlbröst and Holst (2013) and Rits, Schuurman, and Ballon (2015) advance coordination tools for iterative, phased living labs.
Given that the literature on living labs that discusses innovation tools is scant, this study identifies and distinguishes the range of tools used to support innovation in living labs. The study classifies tools by the type of living lab in accordance with the categorizations by Leminen, Westerlund, and Nyström (2012) and Leminen (2013). We will also offer our empirical observations on the strategies employed for confronting the issue of tools in living labs. So doing, we address the following research questions: i) what innovation tools are used in living labs? and ii) how can they be categorized?
The remainder of this article is organized as follows. First, the article reviews prior literature on living lab, develops a framework, and presents the research design and dataset. Thereafter, the study reviews the key findings of qualitative case research resulting into four new types of living labs. Finally, the theoretical and managerial implications are discussed, and avenues for future research are provided.
Theoretical Background on Living Labs
Linear and iterative innovation process
Innovation activities typically follow a linear or an iterative process. Previous literature on living labs has made numerous attempts to illustrate linear innovation process by categorizing living lab activities into phases, typically starting from an early development phase and ending with initial market activities such as a market launch (Cleland et al., 2012; Lin, Lin, et al., 2012; Vicini et al., 2012). Literature focused on the linear innovation process also discusses the set-up of a living lab (Kang, 2012; Lin, Lin, et al., 2012), the management of phases in living labs (Gong et al., 2012), and various activities in different phases of living labs (Bendavid & Cassivi, 2012; Chen, 2012; Coenen et al., 2014; Ferrari et al., 2011; Katzy et al., 2012; Lin, Wang, & Yang, 2012; Schumacher & Feurstein, 2007; Shampsi, 2008). Katzy, Baltes, and Gard (2012) note that a linear innovation model systematically attempts to avoid or minimize interaction between the phases, whereas living labs attempt to better integrate phases by sharing knowledge. Hyysalo and Hakkarainen (2014) offer a comparison on two similar innovation projects – one a living lab project and another a conventional innovation project – and propose that the collaboration within the projects is very similar.
The iterative innovation model proposes that innovation activities are repeated rather than follow phases. Although Pierson and Lievens (2005) propose that living labs are cyclic by nature, other scholars (e.g., Bergvall-Kåreborn et al., 2009; Ståhlbröst, 2008) provide a guideline for the iterative living lab. Building on this guideline, Ståhlbröst and Bergvall-Kåreborn (2008) stress that iteration and interaction between phases foster innovation development, and Holst, Ståhlbröst, and Bergvall-Kåreborn (2010) add that openness improves and fastens innovation. Further, Tang and Hämäläinen (2014) propose a process model that has five iterative phases: requirements, co-design, prototyping, test and tracking, and commercialization. The iterative innovation model underlies the engagement of users and other stakeholders; thus, new experiences and knowledge are created by learning in innovation activities between stakeholders. Uncontrollable dynamics of everyday life are a source of complexities in real-life environments (Leminen, DeFillippi, & Westerlund, 2015), and such learning steers innovation in living labs.
Customized and standardized tools
A large body of literature on living labs refers to customized methods and tools, explaining a set of possible methods used for living labs innovation activities in different contexts (McNeese et al., 2000). Studies frequently couple different methods to real-life contexts, including e-environments, university research centres, everyday life, campuses, towns, districts, villages, rural areas, and industrial zones. For example, Bajgier and colleagues (1999) experimented with living labs in a city neighbourhood, while Benne and Fisk (2000) used a living lab as a temporary learning environment. In general, living labs focus on the development of methods and tools for innovation activities (cf. Intille, 2002; Kidd et al., 1999). Further, recent studies suggest that living labs pilot, develop, and experiment with different methods based on the results of innovation activities (Leminen & Westerlund, 2012) and that those living labs with little experience in particular attempt to develop their methods and tools for innovation (Leminen et al., 2016; Nyström et al., 2014). Mulder (2012) concludes that living labs use methods and tools heterogeneously and proposes that methods and tools should be harmonized across living labs to foster their usage.
Many studies on living labs refer to the usage of standardized, predefined set of tools in a variety of activities (Ponce de Leon et al., 2006) but do not explicitly describe them. Edvardsson and colleagues (2012) describe the living lab as a method containing many tools for customer involvement and as a context for user innovation. Furthermore, studies couple a set of methods and tools across innovation phases (Guzmán et al., 2013; Rits et al., 2015; Schaffers et al., 2009) and to five different views of living labs: user involvement, service creation, infrastructure, governance, innovation, and outcome (Mulder et al., 2008). Budweg and colleagues (2011) argue that living labs adapt the usage of methods depending on their maturity. Thus, more experienced living labs tend to pursue standardization of the usage methods. Literature of living labs aims to create taxonomy for methods used in living labs; for example, Fred, Leminen, and Kortelainen (2011) attempt to describe the applicability of methods in living labs, and Gray and colleagues (2014) document usefulness of methodologies in different contexts. Ståhlbröst and Holst (2013), in turn, introduce a handbook for living labs methodologies for iterative and cyclic innovation activities.
Given that methods and tools are embedded in living labs and their activities, any single study cannot offer a comprehensive, all-encompassing overview of their use in living lab activities and how innovation processes are integrated into living lab activities. Therefore, the present study builds a framework (Figure 1) that has two dimensions: i) innovation process (“predefined, linear” versus “iterative, nonlinear”) (cf. Schumacher & Feurstein, 2007; Ståhlbröst, 2008) and the usage of tools (“standardized” versus “customized”) (Ståhlbröst & Holst, 2013). The two-dimensional framework helps to identify how methods and tools support understanding of living lab innovation activities. Also, the framework categorizes living labs in relation to innovation process (linear/iterative). It attempts to explain innovation mechanisms and outcomes in living labs rather than claiming to show any causal links or correlations between dimensions and their ends. In this article, after introducing the created framework, we use it to map and categorize innovation processes and utilized tools, and to understand innovation processes in diverse living labs.
Figure 1. A conceptual framework for categorizing living labs based on their innovation process and tools
The study utilizes a qualitative, multiple case study approach (Yin, 2003) by analyzing an international data set of 150 interviews in 40 living labs in ten countries, namely Belgium, Canada, Denmark, Finland, France, Netherlands, South Africa, Spain, Sweden, and Turkey. More specifically, the study deploys snapshot studies as suggested by Jensen and Rodgers (2001). The case selection criteria required that the living labs must innovate in real-life environments, engage multiple stakeholders, and emphasize the role of users in innovation (cf. Almirall & Wareham, 2011; Bergvall-Kåreborn & Ståhlbröst, 2009; Leminen, 2013; Leminen et al., 2014; Leminen, Nyström, & Westerlund, 2015). Furthermore, the chosen cases reflect the diversity in living labs as they were clearly driven by different types of actors (Leminen, Westerlund, & Nyström, 2012).
The data were collected between 2007 and 2015 through face-to-face and phone interviews, which were audio recorded and then transcribed. We collected information on various themes following an interview guide (cf. Patton, 1990) and later conferred with the informants to verify the key findings. The informants include various stakeholders in living labs, including top and middle managers, scholars, project coordinators, technical specialists, and users. In addition to the interviews, we gathered secondary data from websites, bulletins, magazines, and case reports. To maintain confidentiality, the identities of the organizations and informants are withheld.
The empirical data were organized according to the informant, the date of interview, and the type of informant. Our study followed a multi-phased data analysis process, which consisted of open coding, focused coding, identification of innovation processes, and theorizing of codes. Table 1 gives an overview of the data analysis and the phases of the study.
In the first phase, we analyzed and coded the original transcribed interviews. The words associated with innovations processes, methods, methodologies, and tools were searched for using a content analysis technique. Following Roberts (1997) and Neuendorf (2002), the aim of the coding and content analysis was to understand the cases. The original, word-by-word transcribed empirical material was first independently coded by the authors, and later the results were jointly compared, discussed, and agreed by the authors.
Table 1. Data analysis process
Data Analysis Phase
1. Open coding
2. Focused coding
3. Identify innovation processes and usage of tools in living labs
4. Theorizing the codes
Then, a round of coding described the innovation activities, methods, and tools. This coding phase identified the standardized and customized tools in innovation activities of living labs.
In the third phase, innovation processes in the living labs were investigated. The innovation processes were coded and compared with the linear and iterative innovation processes suggested by Schumacher and Feurstein (2007) and Ståhlbröst (2008).
In the fourth and final phase, four archetypes of living labs were analyzed; in other words, we classified the cases based on their usage of tools and innovation processes (cf. Figure 1). We consider the four archetypes of living labs as one of our main findings.
This study analyzes and classifies the range of tools used to support innovation in living labs. Specific tools used for innovation in living labs include e.g. open communication and ideation tools for promoting, collecting, evaluating and disseminating contributions, as well as monitoring tools for tracking activity and individual contributions for possible legal reasons. These tools are fairly different from those used in traditional closed innovation model, in which project management tools are more efficient. The results from our analyses highlight the following four ways that tools are used in innovation activities of living labs: linearizers, iterators, tailors, and mass customizers (Figure 2).
Figure 2. Case living labs categorized using the conceptual framework
Linearizers focus on using both a linear innovation process and a standardized set of predefined tools. This categorization is in line with prior literature on living labs, which identifies testing activities with a predefined set of methods and tools (Ponce de Leon et al., 2006). Particularly, linearizers seem to lean on a structured but linear innovation process, where the usage of standardized tools can lead to incremental innovation. This finding is in line with extant studies, which confirm that a main body of innovations are incremental in living labs (e.g., Leminen, Nyström, & Westerlund, 2015). Linearizers often represent utilizer- and provider-driven living labs (cf. Leminen, Westerlund, & Nyström, 2012), which are run by companies or organizations pursuing efficiency of operations. More specifically, the organizations aim to improve efficiency of living lab operations with standardized tools and predefined linearized processes. Developed tools cover different innovation phases and a broad variety of customer-centric and customer-driven methods. Hence, living lab activities are often productized and commercialized, and the developed generalized tools in living labs are used for the different needs of customers. And, as an informant with design responsibility underlines, their living lab assumes a predefined linear innovation process with standardized tools in their living lab activities:
“The end of this process is testing with the users to have the experience. So, if we have the chance to have a company or public service who is coming at the beginning of the project, we could have all the steps from a real living lab, so exploration, co-creation, experimentation, evaluation. … But we have a different tool for each step. But in each tool … we have four steps; for the labour it’s the same, we have different steps. And, for example, for the labour, we create them for companies to test their product with the users. And the first step is to define with the company what they need to have as information, to define with them the protocol of the experimentation, to have some papers, some tools to collect this experience, and after to analyze this information for the companies.” (Case 32, Informant with design responsibility)
Similar to linearizers, iterators aim to find a solution with a standardized set of tools in innovation processes. However, in contrast to linearizers, iterators adapt the innovation process based on the experiences and learning in the innovation activities. This is in a line with Ståhlbröst and Holst (2013) and Rits and colleagues (2015), who document iterative innovation process with standardized tools. In some of our investigated living labs, standardized tools were combined with iterative innovation processes, which led to incremental innovation. Surprisingly, we only found few examples of iterators (provider-driven living labs) from the sample of 40 living labs. Thus, this study speculates that many living labs have not yet adjusted their prior knowledge on innovation activities to a predefined set of tools to be used. And, as the researcher in Case 25 proposes, a living lab may assume iterative innovation process with standardized tools in their living lab activities:
“I think 80 percent of the time we don’t follow the pre-determined pattern. And that’s because of the input you get. So, not only do the people that come up with the innovation have to iterate, but we as researchers have to iterate and think again, this is not the best step anymore. So maybe instead of a co-creation session, for example, we should do this step now, or we should do something completely different.” (Case 25, Researcher)
“We have today about 57 different methods to capture user feedback. We have the common tools like workshops and co-creation sessions and surveys and that kind of stuff. But you have also observation techniques, proxy-technology assessment tools, and whatsoever for capturing user feedback, both quantitative and qualitative.” (Case 25, Business Development Manager)
3. Mass customizers
Similar to linearizers, mass customizers adapt predefined linear innovation process; but, in contrast to linearizers, mass customizers customize their tools based on the needs in the innovation process rather than relying on the standardized tool set. The literature on living labs identifies several such types of living lab activities, where living labs aim to create a taxonomy and harmonize used methods and tools in living labs (cf. Fred et al., 2011; Gray et al., 2014). The majority of living labs seem to lean on mass customizers (Figure 2). The study identified many provider-driven, utilizer-driven, and enabler-driven living labs (cf. Leminen, Westerlund, & Nyström, 2012) as mass customizers. With mass customizers, the use of customized tools in a linear, standardized innovation process seems to lead to incremental innovation. Although the variety of tools opens up the possibility for radical innovation, the standardized innovation process restricts required innovation activities. Mass customizers reduce the variety of innovation processes by predefined linear innovation to improve the efficiency of innovation activities. However, the flexibility of innovation activities are possessed by customized tools in living labs. Mass customizers aim to find solutions for the needs of innovation activities by using predefined linear innovation process and the customized tools for innovation:
“We use methodology for user involvement and co-design, but we apply a lot of different methodology, depending on the topic of the research, for example.” (Case 13, Project Manager)
“I don’t like to apply the same method from one project to another. What we do in education with the remote network school, with the design experiment approach, with researchers who are familiar with research and design experiment and collecting data, sharing it, transfer it to the people on the ground to better improve the way they change their processes. That’s for me the best way to organize things with the academic world and the other stakeholders and users.” (Case 30, Director)
Similar to mass customizers, tailors customize the usage of tools and, similarly to iterators, tailors use iterative, non-linear innovation process. We identified two groups of tailors. The first group (representing three out of ten tailor organizations: Cases 1, 8, and 9) wish to explore the possibilities of living labs. Such organizations lack prior experience on living labs or tools needed for living labs. Thus, tools are particularly developed for their purpose(s). This grouping is in line with studies of Nyström and colleagues (2014) and Leminen and colleagues (2016). In the first group, we found tailors that included user-driven, utilizer-driven, and provider-driven living labs.
The second group (representing seven out of ten of the tailor organizations: Cases 7, 12, 14, 17, 18, 28, and 38) includes living labs that have prior living lab experience, but they wish to pilot, develop, and experiment with different methods, tools, and methodologies for their purposes. This grouping is in line with the studies of Kidd and colleagues (1999) and Intille (2002). Thus, iterations of innovation activities are needed in these living labs. Also, this finding is line with the earlier study of Eriksson, Niitamo, and Kulkki (2005), which highlights the need for iterations in living labs. However, a living lab as such is the not the main focus of the innovation activities; rather, a living lab supports other innovation and development activities of organizations. In particular, provider-driven living labs represent this second group. Also, in the second group of living labs, this study identified two living labs having the development and customization of tools for the purpose of the project at hand. The usage of these tools in innovative way(s) in an iterative nonlinear process supports the emergence of radical innovations. Tailors aim to find solutions for the needs of innovation activities by tailoring the innovation process and the usage of tools for innovation.
“We need to go that area, we need to find something. So, we thought, maybe we could combine this game with some kind of media (plus) bluetooth technology. So, we set up an experiment, we tried out combinations. Also, it was with the same user group, actually, as in the previous one. And then it is really important, we always think, how will it benefit the user group? We always try to connect new stuff to the user group so they can make sense of it, they can get [something] out of it. So, we never knew this when we started. When we contacted this company, we could never say, okay, we will do this with you. Because this happens on the way. The thing we can say to them is that we try to connect your technology and your project in new situations and new companies and so on.” (Case 8, Professor)
This study aimed to understand the range of innovation tools in the living lab context. It provides three theoretical contributions to the discussions of open innovation and living labs, and it presents new information about innovation processes and tools in living lab. First, the article presented a new conceptual framework for analyzing innovation processes and usage tools in living labs (Figure 1). Second, the framework distinguished four archetypes of living labs based on the innovation process and the usage of tools: linearizers, iterators, mass customizers, and tailors (Figure 2). Third, the study proposed three preliminary propositions based on the findings from the case studies (below).
This study takes a step forward by revealing tools used for supporting innovation in living labs. In contrast to other studies on innovation in living labs, which focus on explaining living labs as a methodology, this study argues that, by using appropriate tools, living labs can significantly foster the emergence of innovation. Therefore, this study contributes not only to the emerging literature on living labs by depicting fours ways tools are used in living labs, but also by proposing the ways such tools can be used to reorganize innovation more generally in the open innovation model.
The developed framework sheds light on innovation activities and how such innovation activities are coupled to the diverse living labs. The dimensions of the framework include the innovation process (“predefined, linear” versus “iterative, nonlinear”) and the usage of tools (“standardized” versus “customized”). Whereas the former is inherently related to the process views, for innovation activities in living labs, the latter is grounded on the usage of tools in living labs. This study concludes that emerging living labs probably start with a customized approach to investigating the possibilities of living lab activities. However, the results of our study show that living labs aim to reduce the complexity of their operations by the standardized tools, or by standardized innovation process, or both. The standardization is often documented to reduce costs and provides cost savings (cf. Kaufmann & Eroglu, 1999). However, the present study suggests that such standardized activities seem to lead on incremental innovations in living labs thus reducing the enthusiasm of living lab activities among stakeholders. This finding is in line with prior studies that highlight the importance of passion in addition to the resources and knowledge in living lab activities (Leminen & Westerlund, 2012; Leminen, Westerlund, & Kortelainen, 2012).
Four archetypes of living labs
The conceptual framework distinguishes four archetypes of living labs based on the innovation process and the usage of tools: linearizers, iterators, mass customizers, and tailors. Linearizers represent living labs with a predefined linear innovation process and a standardized set of tools. Such living labs aim at improving the efficiency of innovation activities and reducing costs both in the innovation process and in the usage of tools. Iterators, in turn, have a predefined set of tools but adapt themselves to the needs of customers through an adaptive and flexible innovation process. Mass customizers take the given predefined linear innovation process, but try to increase the flexibility by customizing the needed tools for innovation activities. Finally, tailors rely on both iterative, nonlinear innovation processes and customized tools. Tailors include living labs who have prior experience and knowledge for innovation activities but wish to keep the innovation activities flexible.
Three preliminary propositions
Given that there is a substantial need for research on specific innovation tools in living labs (cf. Almirall et al., 2012; Dell’Era & Landoni, 2014; Edwards-Schachter et al., 2012; Ståhlbröst, 2008), the study makes a contribution to the literature on living labs by identifying and explaining such tools. Similar to Oxford Dictionary, we view a tool as “a device or implement used to carry out a particular function”. Hence, the study proposes the following three propositions for the future living lab research and for managers and practitioners involved in living lab activities:
- Standardized tools decrease the complexity of innovation activities, and decreasing complexity leads to predefined incremental innovation outcomes in living labs.
- A predefined linear innovation process decreases the complexity of innovation activities, and decreasing complexity leads to predefined incremental innovation outcomes.
- Adopting an iterative, non-linear innovation process and customized tools for innovation activities increases the likelihood of an undefined and a novel innovation outcome.
The findings imply that not only scholars of innovation but also business managers and other stakeholders contemplating innovation development through living labs need to consider open innovation mechanisms and their underlying assumptions. In particular, understanding tools – especially the differences between different types of tools used to support innovation – helps stakeholders to decide what they want to achieve, and then to design or join living labs of a particular type to achieve those objectives. Hence, innovation tools aid managers of companies and organizations (public or private) to apply relevant innovation approaches as a part of their innovation management portfolio, particularly when innovation takes place through living labs. For this purpose, the study suggested preliminary implications as managerial implications, helping anyone interested in designing or participating in a living lab to better benefit from innovation tools. Finally, the study called for more research on innovation tools in living labs and other forms of open innovation.
The authors gratefully appreciate the two anonymous blind reviewers and the Editor-in-Chief, Chris McPhee, for their valuable comments, suggestions, and guidelines to improve this article. Special thanks are extended to all of the interviewees, who kindly shared their knowledge and experience about living labs. This paper is a revised and expanded version of a paper entitled “Innovation Tools in Living Labs”, which was presented at the 2016 ISPIM Innovation Conference in Porto, Portugal, June 19–22. ISPIM – the International Society for Professional Innovation Management – is a network of researchers, industrialists, consultants, and public bodies who share an interest in innovation management.
Almirall, E., Lee, M., & Wareham, J. 2012. Mapping Living Labs in the Landscape of Innovation Methodologies. Technology Innovation Management Review, 2(9): 12–18.
Almirall, E., & Wareham, J. 2011. Living Labs: Arbiters of Mid- and Ground-Level Innovation. Technology Analysis & Strategic Management, 23(1): 87–102.
Äyväri, A., & Jyrämä, A. 2015. Rethinking Value Proposition Tools for Living Labs. In E. Gummesson, C. Mele, & F. Polese (Eds.), Naples Forum on Service 2015, Naples, Italy, June 9–12. SIMAS Lab di Salerno for Naples Forum on Service.
Bajgier, S. M., Maragah, H. D., Saccucci, M. S., & Verzilli, A. 1991. Introducing Students to Community Operations Research by Using a City Neighborhood as Living Laboratory. Operations Research, 39(5): 701–709.
Bendavid, Y., & Cassivi, L. 2012. A ‘Living Laboratory’ Environment for Exploring Innovative RFID-Enabled Supply Chain Management Models. International Journal of Product Development, 17(½): 23–42.
Benne, M. R., & Fisk, A. D. 2000. Teaching Applied Experimental Psychology Using a Living Laboratory. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2000, 44: 2–75.
Bergvall-Kåreborn, B., Ihlström Eriksson, C. Ståhlbröst, A., & Svensson, J. 2009. A Milieu for Innovation: Defining Living Labs. In Proceedings of the 2nd ISPIM Innovation Symposium, New York, December 6–9.
Bergvall-Kåreborn, B., & Ståhlbröst, A. 2009. Living Lab: An Open and Citizen-Centric Approach for Innovation. International Journal of Innovation and Regional Development, 1(4): 356–370.
Budweg, S., Schaffers, H., Ruland, R., Kristensen, K., & Prinz, W. 2011. Enhancing Collaboration in Communities of Professional Using a Living Lab Approach. Production Planning & Control, 22(5-6): 594–609.
Chen, Y.-T. 2012. Sketch Industry Promotion Framework for Smart Living Services by Leveraging Living Lab Harmonization Cube. International Journal of Electronic Business Management, 10(2): 149–162.
Cleland, B., Mulvenna, M., Galbraith, B., Wallace, J., & Martin, S. 2012. Innovation of eParticipation Strategies Using Living Labs as Intermediaries. Electronic Journal of e-Government, 10(2): 120–132.
Coenen, T., van der Graaf, S,. & Walravens, N. 2014. Firing Up the City – A Smart City Living Lab Methodology. Interdisciplinary Studies Journal, 3(4): 118–128.
Dell´Era, C., & Landoni, P. 2014. Living Lab: A Methodology Between User-Centred Design and Participatory Design. Creativity and Innovation Management, 23(2): 137–154.
Dutilleul, B., Birrer, F. A. J., & Mensink, W. 2010. Unpacking European Living Labs: Analysing Innovation’s Social Dimensions. Central European Journal of Public Policy, 4(1): 60–85.
Edvardsson, B., Kristensson, P., Magnusson, P., & Sundström, E. 2012. Customer Integration within Service Development—A Review of Methods and an Analysis of Insitu and Exsitu Contributions. Technovation, 32: 419–429.
Edwards-Schachter, M. E., Matti, C. E., & Enrique Alcántara, E. 2012. Fostering Quality of Life through Social Innovation: A Living Lab Methodology Study Case. Review of Policy Research, 29(6): 672–692.
Eriksson, M., Niitamo, V.-P., & Kulkki. S. 2005. State-of-the-Art in Utilizing Living Labs Approach to User- Centric ICT Innovation – A European Approach. Luleå, Sweden: Luleå University of Technology.
Ferrari, V., Mion, L., & Molinari, R. F. 2011. Innovating ICT Innovation: Trentino as a Lab. In Proceedings of the ICEGOV2011: 329-332, Tallinn, Estonia, September 26–28.
Fred, M., Leminen, S., & Kortelainen, M. J. 2011. UDOI as a Part of Companies’ Product and Service Development Processes. In S. Leminen (Ed.), Co-creation with Users and Customers in Living Labs Integrating Users and Customers in Companies’ Business Processes, A76: 28–38. Vantaa, Finland: Laurea Publications.
Fulgencio, H., Le Fever, H., & Katzy, B. 2012. Living Lab: Innovation through Pastiche (a Research Linking Disparate and Discorded Ontology). In P. Cunningham, & M. Cunningham (Eds.), Proceedings of the eChallenges e-2012 Conference: 1-8. International Information Management Corporation.
Følstad, A. 2008. Living Labs for Innovation and Development of Communication Technology: A Literature Review. Electronic Journal for Virtual Organisations and Networks, 10: 99–131.
Gong, G., Hsiao, M., Hsieh, M.-D., Liu, L., Chiu, T., Lin, L.-C., Chen, K.-T., Chen, B., Lin, H.-H., Fang, E., Wang, M., & Wen, J.Y.-C. 2012. Application of the Living Lab Concept: Empirical Validation in Taiwan’s Minsheng Community. International Journal of Automation and Smart Technology, 2(3): 209–229.
Gray, M., Mangyoku, M., Serra, A., Sánchez, L., & Aragall, F. 2014. Integrating Design for All in Living Labs. Technology Innovation Management Review, 4(5): 50–59.
Guzmán, J. G., del Carpio, A. F., Colomo-Palacios, R., & de Diego, M. V. 2013. Living Labs for User-Driven Innovation: A Process Reference Model. Research-Technology Management, 56(3): 29–39.
Holst, M., Ståhlbröst, A., & Bergvall-Kåreborn, B. 2010. Openness in Living Labs: Facilitating Innovation. In Proceedings of the 33rd IRIS Seminar, Rebild, Denmark, August 20–24.
Hyysalo, S., & Hakkarainen, L. 2014. What Difference Does a Living Lab Make? Comparing Two Health Technology Innovation Projects. CoDesign, 10(3-4): 191–208.
Intille, S. S. 2002. Designing a Home of the Future. IEEE Pervasive Computing, 1(2): 76–82.
Jensen, J. L., & Rodgers, R. 2001. Cumulating the Intellectual Gold of Case Study Research. Public Administration Review, 61(2): 236–246.
Kang, S.-C. 2012. Initiation of the Suan-Lien Living Lab – A Living Lab with an Elderly Welfare Focus. International Journal of Automation and Smart Technology, 2(3): 189–199.
Katzy, B. R., Baltes, G. H., & Gard, J. 2012. Concurrent Process Coordination of New Product Development by Living Labs – An Exploratory Case Study. International Journal of Product Development, 17(½): 23–42.
Kaufmann, P. J., & Eroglu, S. 1999. Standardization and Adaptation in Business Format Franchising. Journal of Business Venturing, 14(1): 69–85.
Kidd, C., Orr, R., Abowd, G., Atkeson, C., Essa, I., MacIntyre, B., Mynatt, E., Starner, T., & Newstetter, W. 1999. The Aware Home: A Living Laboratory for Ubiquitous Computing Research. In Proceedings of the Second International Workshop on Cooperative Buildings (CoBuild ‘99), Pittsburgh, October 1–2.
Leminen, S. 2015. Living Labs as Open Innovation Networks - Networks, Roles, and Innovation Outcomes. Doctoral dissertation. Helsinki, Finland: Aalto University.
Leminen, S. 2013. Coordination and Participation in Living Lab Networks. Technology Innovation Management Review, 3(11): 5–14.
Leminen, S., DeFillippi, R., & Westerlund, M. 2015. Paradoxical Tensions in Living Labs. In Proceedings of the XXVI ISPIM Conference, Budapest, Hungary, June 14–17.
Leminen, S., Nyström, A.-G., & Westerlund, M. 2015. A Typology of Creative Consumers in Living Labs. Journal of Engineering and Technology Management, 37: 6–20.
Leminen, S., & Westerlund, M. 2012. Towards Innovation in Living Labs Network. International Journal of Product Development, 17(1/2): 43–59.
Leminen, S., & Westerlund, M. 2013. Incremental and Radical Service Innovation in Living Labs. In B. Christiansen, S. Yildiz, & E. Yildiz (Eds.), Transcultural Marketing for Incremental & Radical Innovation, Information Science Reference: 281–295 Hershey, Pennsylvania: IGI Global.
Leminen, S., & Westerlund, M. 2016. A Framework for Understanding the Different Research Avenues of Living Labs. International Journal of Technology Marketing, 11(4): 399–420.
Leminen, S., Westerlund, M., & Nyström, A.-G. 2014. On Becoming Creative Consumers – User Roles in Living Labs Networks. International Journal of Technology Marketing, 9(1): 33–52.
Leminen, S., Westerlund, M., & Kortelainen, M. 2012. A Recipe for Innovation through Living Lab Networks. In Proceedings of the XXIII ISPIM Conference, Barcelona, Spain, June 17–20.
Leminen, S., Westerlund, M., & Nyström A.-G. 2012. Living Labs as Open Innovation Networks. Technology Innovation Management Review, 2(9): 6–11.
Leminen, S., Westerlund, M., Nyström, A.-G., & Kortelainen, M. 2016. The Effect of Network Structure on Radical Innovation in Living Labs. Journal of Business & Industrial Marketing, 31(6): 743–757.
Lin, W.-Y., Lin, C.-T., Wang, Y.-H., & Chen, R.-T. 2012. The Transformation of Users in Living Lab Construction: The Case of Eco-City Living Lab. International Journal of Automation and Smart Technology, 2(3): 231–240.
Lin, C.-K., Wang, T.-H., & Yang, J.-F. 2012. TOUCH Doctor – A Nutrition Control Service System Developed under Living Lab Methodology. International Journal of Automation and Smart Technology, 2(3): 253–263.
McNeese, M. D., Perusich, K., & Rentsch, J. 2000. Advancing Socio-Technical Systems Design Via the Living Laboratory. In Proceedings of the IEA 2000/HFES 2000 Congress, 2:2-610-2-613
Mulder, I. 2012. Living Labbing the Rotterdam Way: Co-Creation as an Enabler for Urban Innovation. Technology Innovation Management Review, 2(9): 39–43.
Mulder, I., & Stappers, P. J. 2009. Co-creating in Practice: Results and Challenges. In Proceedings of the 15th International Conference on Concurrent Engineering (ICE 2009), Leiden, The Netherlands, June 22–24.
Neuendorf, K. A. 2002. The Content Analysis Guidebook. Thousand Oaks, CA: Sage.
Niitamo, V.-P., Westerlund, M., & Leminen, S. 2012. A Small-Firm Perspective on the Benefits of Living Labs. Technology Innovation Management Review, 2(9): 44–49.
Nyström, A.-G., Leminen, S., Westerlund, M., & Kortelainen, M. 2014. Actor Roles and Role Patterns Influencing Innovation in Living Labs. Industrial Marketing Management, 43(3): 483–495.
Patton, M. 1990. Qualitative Evaluation and Research Methods (2nd Ed.). Newbury Park, CA: Sage.
Pierson, J., & Lievens, B. 2005. Configuring Living Labs for a ‘Thick’ Understanding of Innovation. In Ethnographic Praxis in Industry Conference Proceedings, 2005(1): 114–127.
Ponce de Leon, M., Eriksson, M., Balasubramariam, S., & Donnelly, W. 2006. Creating A Distributed Mobile Networking Testbed Environment – Through The Living Labs Approach. Paper presented at Testbeds and Research Infrastructures for the Development of Networks and Communities, 2nd International Conference on TRIDENTCOM.
Rits, O., Schuurman, D., & Ballon, P. 2015. Exploring the Benefits of Integrating Business Model Research within Living Lab Projects. Technology Innovation Management Review, 5(12): 19–27.
Roberts, C. W. (Ed.) 1997. Text Analysis for the Social Sciences: Methods for Drawing Statistical Inferences from Texts and Transcripts. Mahwah, NJ: Lawrence Erlbaum.
Sauer, S. 2013. User Innovativeness in Living Laboratories: Everyday User Improvisations with ICTs as a Source of Innovation. Doctoral dissertation. University of Twente: The Netherlands.
Schaffers, H., Budweg, S., Kristensen, K., & Ruland, R. 2009. A Living Lab Approach for Enhancing Collaboration in Professional Communities. In K.-D. Thoben, (Ed.), Proceedings of the 15th International Conference on Concurrent Enterprising (ICE), Leiden, The Netherlands, June 22–24.
Shampsi, T. A. 2008. Living Labs: Good Practices in Europe. In J. Schumacher & V.-P. Niitamo (Eds.), European Living Labs: A New Approach for Human Centric Regional Innovation: 15–30. Berlin: Wissenschaftlicher Verlag Berlin.
Schumacher, J., & Feurstein, K. 2007. Living Labs – The User as Co-Creator. In Proceedings of the 13th International Conference on Concurrent Enterprising (ICE), Sophia Antipolis, France.
Schuurman, D. 2015. Bridging the Gap between Open and User Innovation? Exploring the Value of Living Labs as a Means to Structure User Contribution and Manage Distributed Innovation. Doctoral Thesis, University of Ghent: Belgium.
Schuurman, D., Lievens, B., De Marez, L., & Ballon, P. 2012. Towards Optimal User Involvement in Innovation Processes: A Panel-Centered Living Lab-Approach. In Proceedings of the Technology Management for Emerging Technologies PICMET ‘12 Conference: 2046–2054, Vancouver, July 29 – August 2.
Ståhlbröst, A. 2008. Forming Future IT – The Living Lab Way of User Involvement. Doctoral Thesis No. 62, Luleå University of Technology Social Informatics, Sweden.
Ståhlbröst, A., & Holst, M. 2013. The Living Lab Methodology Handbook. Danish Agency for Science Technology and Innovation, Lietuvos Mokslo Taryba, The Research Council of Norway, Norden NordForsk, Rannís and Vinnova.
Tang, T. 2014. Combining User and Context: Living Labs Innovation in Digital Services. Doctoral dissertation. Aalto University, Finland.
Tang, T., & Hämäläinen, M. 2014. Beyond Open Innovation: The Living Lab Way of ICT Innovation. Interdisciplinary Studies Journal, 3(4): 15–23.
Tang, T., Wu, Z., Karhu, K., Hämäläinen, M., & Ji, Y. 2012. Internationally Distributed Living Labs and Digital Ecosystems for Fostering Local Innovations in Everyday Life. Journal of Emerging Technologies in Web Intelligence, 4(1): 106–115.
Vicini, S., Bellini, S., & Sanna, A. 2012. The City of the Future Living Lab. International Journal of Automation and Smart Technology, 2(3): 201–208.
Veeckman C., Schuurman, D., Leminen, S., & Westerlund, M. 2013. Linking Living Lab Characteristics and Their Outcomes: Towards a Conceptual Framework. Technology Innovation Management Review, 3(11): 6–15.
Westerlund, M. & Leminen, S. 2014. The Multiplicity of Research on Innovation through Living Labs. In Proceedings of the XXV ISPIM Conference, Dublin, Ireland, June 8–11.
Westerlund, M., & Leminen, S. 2011. Managing the Challenges of Becoming an Open Innovation Company: Experiences from Living Labs. Technology Innovation Management Review, 1(1): 9–25.
Yin, R. K. 2003. Case Study Research Design and Methods. Thousand Oaks, CA: Sage Publications.
Keywords: innovation tool, Living lab, method, Open innovation, user innovation