Coming together is a beginning; keeping together is a progress; working together is success.
As society changes rapidly, there is a need to educate professionals who contribute to innovation and complex adaptations in organizations. As part of this education, companies, governmental bodies and other stakeholders have sought collaboration on complex issues in "living labs". Living labs are recognized as educational environments to prepare students in higher education for future roles. The aim of this article is to explore the nature and extent of the scientific literature about living labs in which actors in higher education actively participate. In total, 21 articles were included in this scoping review. Research into this topic appears to be relatively new. Definitions of living labs are mostly in line with the definition used by the European Network of Living Labs (ENoLL), with its large variation of lab locations. Few results about organization and governance were identified. Different lessons learned regarding processes, interaction and requirements for successful living labs were found. The design of learning is mostly described by learning activities. Learning outcomes are described in generic and specific competences and assessments in living labs are rarely described. The authors recommend more detailed studies into aspects of the successful participation of higher education to gain knowledge about enhancing learning outcomes, and the effects of educational activities within living lab environments.
The world is changing rapidly, leading to complex societal challenges. Continuous social changes affect the types of competences needed for professionals to contribute to innovation. Employers expect professionals to be lifelong learners and to constantly update their expertise in accordance with societal and professional demands. Complex societal challenges call for groups of collaborating experts with different backgrounds and contexts (Cremers et al., 2016). Consequently, there is a need to educate professionals who think and work in an interdisciplinary fashion, who contribute to innovation, and who achieve complex adaptations in organizations. Higher education prepares a substantial group of professionals for “real life”, although it is questionable if traditional classroom courses are preparing students sufficiently for the challenges of the future. According to Zitter, Hoeve, and De Bruijn (2016), the traditional and scholarly approach of higher education is too limited. It does not fit within the “Zeitgeist” of the current era, does not resonate with the preferences of students, and collides with the demands of professional practice (Zitter et al., 2016). For example, in the Netherlands, Zuyd University of Applied Sciences focuses specifically on developing students into professionals with skills that are relevant for the region. Thus, their main pillars include integrating research into education and embedding education in practice (Zuyd University of Applied Sciences, 2019).
Increasingly, companies, governmental bodies, civil societies, and other stakeholders seek collaboration on actual complex issues in so-called “living labs”. This concept offers opportunities for higher education to work closely with professional practice with the emphasis on innovation research in “real life”. In the literature, the “living lab” concept is increasingly gaining attention (Schuurman et al., 2015). The European Network of Living Labs (ENoLL) defines living labs as “user-centred, open innovation ecosystems based on systematic user co-creation approach, integrating research and innovation processes in real life communities and settings” (ENoLL, 2020). Real-life setting, co-creation, active user involvement, multi-stakeholder participation, and multi-method approach are the five major elements of a living lab (ENoLL, 2020). According to ENoLL, no single living lab methodology holds across the broad; all living labs combine and customize different user-centred, co-creation methodologies to best fit their purpose (multi-method approach). The building blocks of exploration, experimentation, and evaluation get performed in iterations, emphasizing the importance of coming to know the current state, designing possible future states of innovations, real-life testing, and assessing the experimental impact by means of user-feedback (Malmberg & Vaittinen, 2019).
Although the concept of living labs has been emerging in the scientific literature and the number of living labs in different areas is increasing, much greater understanding is needed about how to run a living lab successfully. Several aspects of living labs have been the subject of study in recent years, for example, studies into types of living labs and user roles within living labs (Leminen, 2015). In their systematic review, Schuurman et al. (2015) confirmed the increasing number of papers about living labs since 2006, meanwhile the theoretical foundation of the concept lags behind the increasing number of experiences people are having with living labs in practice around the world.
A study into living labs in the Netherlands highlighted the potential value of living labs, though also indicates the current early stage of living labs, and the need for further study (Maas et al., 2017). While the concept of a “living lab” is gaining recognition as an innovative approach for higher education to prepare students for their future roles (Maas et al., 2017), not enough is yet known regarding how to successfully integrate higher education and living labs. Interdisciplinary collaboration poses challenges to all stakeholders involved, such as dealing with differences in professional language and professional culture (Hummels & Vinke, 2010), or shaping the involvement of users in the innovation process (Grove, 2018). Embedding higher education into living labs has its own challenges, including how to merge the dynamics of education and innovation processes into real-life settings, and to match the competences of students with required expertise in the field. Insights gained from reported experiences and lessons learned about how to integrate higher education and living labs , how to facilitate students’ learning in living labs, and how to deal with the challenges it brings along, could provide guidance for future living labs. The aim of this article is therefore to explore the nature and extent of the scientific literature about living labs in which actors in higher education (for example, students and faculty) actively participate. To retrieve this information, a scoping review was conducted using the following research question for framing: What is known about the role of higher education and living labs in scientific literature and about the factors that influence integration of higher education and living labs?
We reviewed the literature on living labs by means of a scoping review. To accumulate as much information as possible about the concept, our main focus was on article relevance. We used the five-stage approach of Arksey and O’Malley (2003).
Identifying the research sub-questions
We formulated the following research sub-questions for background context:
- What kind of studies are conducted regarding living labs that include higher education?
- How are living labs defined and which models and approaches are used as theoretical underpinnings of the living labs?
- What are features of living labs in which higher education participates?
- What are lessons have been learned regarding integration of higher education in living labs?
- How is learning designed (for example, learning outcomes, learning activities and assessment) in the living labs?
Identifying relevant studies
Our study’s search included two concepts: “learning environment” and “living lab” (Figure 1). Using a literature discovery service from Ebsco Host, we searched 29 different databases simultaneously (including ScienceDirect, CINAHL, Psychology and Behavioral Sciences Collection, PsycARTICLES, Science Citation Index, IEEE Xplore Digital Library, Cochrane Database of Systematic Reviews, ERIC).
Figure 1. Search string
The search was limited to publications in Dutch and English published between 2000 and June 2021. In addition to searching electronic databases, we checked the reference lists of relevant articles. We also searched for journal articles as well as conference papers.
The selection of papers based on paper titles was done independently by two reviewers (RvdH and RD). Papers with titles referring to both “living labs” and “education” were given a score of 2, papers with titles referring either to “living labs” or “education” were scored with a 1, and papers on topics that were not relevant to our study were scored with a 0. When the score of the two reviewers together was >2, the abstracts were screened. Screening was performed by one reviewer (RvdH). Articles were included if both “living labs" and “education” were mentioned in the abstract. When the concept of “living labs” was not explicitly mentioned, the article was not included, because this study specifically focused on environments that are called a “living lab”. Other similar concepts may be partly comparable but were also not included because of small nuances between the concepts. Where there was doubt the full text was screened, and the reviewers discussed inclusion or exclusion of the remaining sources together.
Charting the data
A descriptive summary of each study was created in a spreadsheet to map the article’s citation information, general article information (type of publication, number of living labs discussed, domain of the study, subject of innovation, and aim of the study), definitions, key elements and theoretical underpinnings of living labs used by the authors, information on various features, lessons learned, and specific information about how learning is designed within the living labs. First, five articles were independently charted and discussed by two reviewers (RvdH and SB). The results were then discussed with a third researcher, RD. Subsequently, one of the reviewers (RdvH) continued with the other 15 articles.
Collating summarizing and reporting the results
Initial reading and preliminary content analysis led to the main categories described to structure the findings. After creating the table, the results were summarized, reported and discussed by the authors in order to cluster results and draw conclusions.
The search was performed on June 1st, 2021, resulting in 427 hits. After reading the titles, abstracts, and full texts, and correcting for duplicates, 21 full texts matching the inclusion criteria were selected. Figure 2 shows the selection flow chart of the inclusion and exclusion process. Excluded papers did not meet the inclusion criteria during the screening of the title or abstract because the topics “living lab” and “higher education” were not explicitly mentioned, as described in the Methods section.
Figure 2. Flow chart of included and excluded articles
Table 1 (supplementary material) gives an overview of all included articles. The first column shows the article reference (citation information). The second column provides general information about the type of study, the number of living labs involved in the study, the domain in which the living lab is situated, the subject of innovation and the aim of the study. The third column reports the definition of living labs as described in the article, often with additional key elements. The fourth column describes the theoretical underpinnings (models and/or approaches) of living labs as reported in the articles. The fifth column identifies the context and features of the living lab (environment), the stakeholders involved, and also the roles and governance structure within the living lab. Column six reports lessons learned involving initiation, evaluation, and sustainability of the living lab, including any success factors and challenges described in the papers. The seventh and last column shows specific information about educational aspects with an emphasis on learning outcomes, learning methods, and types of assessment, which is based on Biggs (2003).
General article information
The selected articles were published between 2007 and 2019. The living labs were situated in various domains, including ICT, education, healthcare (occupational therapy, gerontology), industrial design, sustainability, service business development, engineering, tourism, ambient intelligence, and architecture. For example, a living lab in architecture (Masseck, 2017) focused on renewable energy and nearly zero-energy buildings. An example from healthcare is a living lab to establish age-friendly services in co-creation with older adults (Van den Berg et al., 2019).
Aims of the studies ranged from evaluating the experiences of participants in living labs, to exploring or developing the theoretical foundations of living labs, or studying elements of living labs, for example, knowledge management (De Jager et al., 2012). Other aims included what is called “designed serendipity”, as well as the financial sustainability of living labs. For example, Santally et al. (2014) described the theoretical foundations needed to create a framework for a living lab that focuses on classroom education for the future. Van den Berg et al. (2019) studied the experiences of their living lab participants (older adults and undergraduate students) in a way that revealed the importance of equality and shared responsibility. Students were interpreted as “stakeholders” in all of the studies. Education was explicitly mentioned as the aim of the study in eleven of the articles. For example, Beecroft (2018) describes the interrelations between real world labs and higher education using a social practice perspective.
The types of studies varied. Five articles were qualitative studies or evaluations of living labs, another five described living lab cases/case studies, two were literature studies, and the majority were knowledge syntheses (n=9). Where articles combined several types of study, for example, a literature review together with one or more case studies, the main type of study is reported in Table 1. The number of living labs described in each article varied from one to five. However, most articles discussed a single living lab, often containing several different projects or educational courses related to this lab context.
Definitions, key elements, and theoretical underpinnings
Each article defined living labs differently, although they often used similar wording in their description. For example, “active user involvement” was referred to as “user-centred innovation”, “user-involvement”, “active participation”, or “collaborative development”. All five major elements as described by ENoLL frequently showed up: co-creation, real-life setting, multi-stakeholder participation, multi-method approach, and active user involvement, together with the accompanying building blocks: exploration, experimentation, and evaluation (Malmberg & Vaittinen, 2019). Callaghan and Herselman (2015) defined co-creation in living labs as input from users as co-creators utilized to research the context of ICT use (in this specific case), find new uses, and evaluate new solutions within everyday contexts. Masseck (2017) describes variation in real-life settings in architecture, which can range from small-scale knowledge dissemination and “experience homes”, up to city platforms for social innovation regarding sustainability, or a city itself with its buildings and inhabitants perceived as a supporting ecosystem for user-centred innovation. De Jager et al. (2012) highlighted the involvement of multiple stakeholders, describing a living lab as an “innovation platform” that engages all stakeholders, such as end users, researchers, industrialists, and policy makers at an early stage of the innovation process. Gualandi and Romme (2019) explained that a living lab can contribute to every phase of the innovation process by orchestrating and coordinating the activities of exploration, co-creation, experimentation, and evaluation. A living lab generates value to the entire supply chain and can explore and assess the environmental, social, and economic effects of new products or services created and tested in the living lab.
Two of the most distinct differences in the definitions of living labs concern specific references to research, learning, and education. Ten definitions explicitly include the element of research in their definition or key elements. An example of a definition explicitly mentioning research is the definition of Era and Landoni (2014) used by Grove (2018): “A Living Lab is a design research methodology aimed at co-creating innovation through the involvement of aware users in a real-life setting”. Additionally, the learning or educational aspect is described in six living lab definitions, for example, in the definition of Jernsand (2019) who describes living labs as spaces for open innovation, co-creation and experimentation in real-life settings with students. In their definition, Van den Berg et al. (2019) state that, “In an educational setting, a living lab enables different stakeholders, including students, to learn how to work on user-driven innovation”.
Because theoretical underpinnings can play an important role in a living lab’s operationalisation, and therefore influence the role of higher education, we searched for the theoretical foundations or approaches of each living lab. One article did not explicitly mention a theoretical foundation (Falk-Kessler et al., 2007). In the remaining twenty articles, a broad range of models were described as theoretical argumentation to start a living lab in the first place. Most of these models or approaches focused on processes such as social interaction, pedagogics, or design. Examples involving living labs and higher education include activity theory (Santally et al., 2014), appreciative inquiry (Callaghan & Herselman, 2015), design thinking (Jernsand, 2019), and service learning (Hansen, 2017). Some papers used a very detailed description of their approach, while others only mentioned the model, but did not elaborate on the application details.
Living lab features
All articles described the contexts in which living lab activities took place. Approximately half of the papers described living labs situated in a university department (n=9), sometimes combined with a virtual or web environment. Topics in living labs located at university departments included the future of teaching (Conruyt et al., 2014), as well as sustainability (climate change and urban sustainability, for example, at campus buildings (Evans et al., 2015)). In these examples, a clear relationship is visible between a living lab’s main topic and its location being a real-life environment close to users (in these cases students, lecturers, and others). Other contexts in which living labs were situated involve public spaces, community sites, and cities (n=9). Subjects of these living labs included sustainable tourism (Jernsand, 2019), library services (Kröse et al., 2012), and age-friendly services (Van den Berg et al, 2019). Two articles described a digital/web context, without a physical component, that is, a knowledge management application (De Jager et al., 2012; Grove, 2018). In one article, the living lab environment was labelled as a human-machine interactive environment (Peng, 2010).
Many different stakeholders were involved in the living labs. Two articles described collaboration in the form of a Public-Private-People-Partnership (De Jager et al., 2012; Santally et al., 2014). Hence, living labs can bring together diverse public and private actors. These partnerships include: companies, industry, associations, students (differing study levels and differing study programmes), academics, teachers, researchers, policy makers, end users (for example, older adults), citizens, service providers, and healthcare organizations. The roles and the composition of stakeholders can differ in each phase of exploration, experimentation, evaluation.
Some articles explicated the roles of the stakeholders. For example, students can play various roles, including learners, peer observers, project leaders, data collectors, analysts, and/or presenters. The roles of students can change over time (Falk-Kessler et al., 2007). Lecturers often provided guidance, coaching, and instruction, while end users were able to share their insights or function as mentors or trainers. Some articles emphasized the importance of social equality within the living labs (Van den Berg et al., 2019; Jernsand, 2019). Jernsand (2019) described “flat leadership” as a teaching style employed in their living lab of sustainable tourism, in which lecturers are mentors who listen and advise, rather than only giving directions.
Hardly any information was provided in the articles about the organization or governance of living labs, along with the conditions for sustainability in living labs. In their article, Gualandi and Romme (2019) addressed the financial sustainability of living labs by stressing the acquisition of funding and creation of value, as these are important conditions for living labs to become financially sustainable.
Design of learning
Our main interest was to ascertain if articles addressed the contribution of education to a living lab, and if so, how they designed learning in these labs. We searched for information on the following topics: learning outcomes of students and, where applicable, other stakeholders, if and how activities to enhance learning were described, and if and how articles reported on the assessment of learning in living lab contexts. 14 of the 21 articles mentioned learning outcomes of students in their study. The described outcomes can be divided into the disciplines of generic learning outcomes and specific learning outcomes. Generic learning outcomes were usually more broadly formulated and concerned topics such as professional development, clinical reasoning through lived experiences, reflection (learning-by-interaction), self-regulation of learning, taking responsibility, learning from experience, self-assessment, social awareness, innovation, and collaboration. Examples of discipline-specific outcomes were knowledge of and skills relevant to the development and implementation of age-friendly services (Van den berg et al., 2019), and specific sustainability development competences (Masseck, 2017).
Learning and teaching activities were not described in detail, however, examples of activities presented in this way included “fun learning”, which uses cartoons or story-telling cartoon movies, as well as gaming-to-learn, where learning-by-playing and serious gaming account for an important role in teaching and student learning (Santally et al., 2014). Doing research with others (not only students and teachers), rather than on others (van den Berg, 2019) are other examples of teaching and learning activities in a living lab. These include, developing creative innovations that answer the needs of users, teams working on parallel projects of their own choice, and observing and assessing assignments during lab activities (Falk-Kessler et al., 2007). Hummels and Vinke (2010) connect the term “individual curriculum” to their living lab, giving students an opportunity to select their own learning activities at the start of a semester, thus catering to their individual learning needs. Learning by doing, edutainment, using social media tools, placed-based learning, participatory methods, and workshops are other examples of teaching and learning activities in the context of living labs. Real-life environments that involve users engaging in co-creation are also essential elements in living lab learning activities.
The articles rarely discussed the assessment of learning in living labs. Only three articles described how students are assessed within the living labs; the forms of assessment used were presentations, qualitative assessments during and after activities (reflection seminars, group discussions, course evaluations), progress reports, student blogs, future-driven self-assessment (focus on utilising the programme of study to prepare students to develop sustainable self-assessment ability), exhibitions, and showcases in which students present their work, while coaches and fellow students act as peer reviewers (Hummels & Vinke, 2010). One article reported experiences regarding the assessment of learning in living labs, in which the authors concluded that “there seems to be less competitive pressure” in assessments in a living lab context than in regular assessments in the curricula (Hwawk et al., 2012).
The articles in our study often addressed lessons learned (including success factors and challenges) regarding initiation, evaluation, and sustainability of living labs. Generally, the lessons learned concern processes and interactions in living labs, as well as preconditions for successful living labs. Regarding the process, van den Berg et al. (2019) found a tension between what is beneficial for a user-driven living lab and what is appropriate for an educational system that focuses on control and prediction. It is deemed necessary in resolving this tension to find the right balance between “freedom and frameworks”. Furthermore, these authors learned about the value of investing time and effort in building relationships between co-creators. Hummels and Vinke (2010) indicate that an attitude of lifelong learning among all participants is essential for creating the right environment in a living lab. According to Grove (2018), “designed serendipity” (unexpectedness, insightfulness, and value added quality) is a success factor as it leads to useful findings and fits within a living lab approach that seeks to elicit unforeseen user ideas and behaviours to enhance product innovation.
Considering interaction in living labs, flat leadership and less competitive pressure amongst living lab participants tend to help to create a successful living lab (Hawk et al., 2012). Using social media tools such as blogs, wikis, Really Simple Syndication (RSS) feeds, sharing content, tagging and social networking were experienced to stimulate the success of a living lab (De Jager et al. 2012).
Requirements for successful living labs include a supportive logistic infrastructure. Falk-Kessler and colleagues (2007) describe the importance of coping with logistical barriers and establishing a community site willing and able to accommodate students during educational activities. Furthermore, building a sense of closeness between stakeholders, including firms and end-users, is seen as a precondition for a successful living lab. Jernsand (2019) also found “neutral places” to be of significance for living labs as they reduce the risk of participants being hampered by institutional “lock-in effects” such as incorporated norms, cultures, and working methods.
The aim of this article was to explore the scientific literature on living labs in which higher education actors (for example students and lecturers) actively participates. Potential results could guide higher education programmes and their networks in how to set up sustainable and meaningful collaborations for innovative educational courses, both together with and in the real world. Just as living labs are a relatively new phenomenon, this study also shows that research into living labs with the active participation of higher education appears to be new. The majority of the papers we studied were published recently, and the number of papers is limited. The kind of studies included were mainly descriptive and explorative in nature, reflecting the state of the art in living lab research. Schuurman et al. (2015) also found the number of empirical, quantitative, and comparative studies focusing on the added value of living labs as still rather limited. In our review, we found no studies that focused directly on the effects of learning in living labs.
Definitions of living labs generally involve the main aspects of ENoLL’s definition, meaning a real-life setting, co-creation, active user involvement, multi-stakeholder participation, and a multi-method approach (ENoLL, 2020). This might imply that a consensus exists about what constitutes the core of living labs. Some articles added terms related to education and research in their definition, which, from the perspective of universities, appears to be a logical addition. The fact that most articles do not explicitly mention research associated with living labs might be related to existing perceptions about the process of innovation that research is an inherent part of innovation. A similar assumption can be made about learning in living labs, since one cannot innovate without learning.
However, the inclusion of both students and teachers in living labs calls for active learning, and active learning is of importance for all stakeholders involved. Veeckman et al. (2013) linked living labs to "communities of interest" and "communities of practice", following the work of Wenger et al. (2002). In these communities, stakeholders are informally connected by what they do together and by what they have learned through their mutual engagement in these activities (Veeckman et al., 2013). This perspective calls for discussion about incorporating learning as one of the core elements of future living lab definitions. Consequently, we can see how giving attention to learning in real life contexts might also impact the theoretical underpinnings of living labs. The available body of knowledge about communities (Wenger et al., 2002) and hybrid learning environments (Bouw et al., 2019) support the embeddedness of higher education actors in living labs. Wals, Lans and Kupper (2012) defined a hybrid learning environment as a social practice around ill-defined, authentic tasks or issues, whose resolution requires transboundary learning. For example, available knowledge exists about how to assess students in hybrid learning environments. Zitter et al. (2016) emphasized the crucial role of participants from practice or business in the assessment of students. The selected papers addressed various features of living labs sometimes in detail, and other times generally.
This study revealed that almost half of living labs are situated at universities. This could well reflect the state of the art in the development of living labs in higher education, with universities themselves as both founders and organizers. This circumstance also provokes discussion about the real-life element of living labs versus the merely “academic”. However, the topics of the living labs situated at universities, for example, IT, sustainability, and education, all link to topics studied in which the users of the living lab are themselves users at universities. From an educational perspective, the real-life element of living labs is an innovative aspect for education, offering students experiences outside the classroom.
In her comment about neutral places, Jernsand (2019) emphasized the impact of the location on the success of living labs. Thus, conducting further research into real-life aspects of living labs, including their location, the intensity of interaction between students and users, and the learning experiences of students would be a useful line of approach. It is notable that this study found few research results about the organization and governance of living labs involving higher education. One article concerning innovation networks implies that collaboration in these networks requires clear and SMART goals from the beginning, as well as continuous management of the main elements of the network, and investment in information and communication technology to improve information sharing and formal coordination (van Tomme et al., 2011). In an article on innovation management, the authors stress the importance of a strategy to guide the approach that steers the innovation, the processes, the portfolio, and the projects in the innovation funnel, as well as leadership, resources, and the competences of staff (Igartua & Albors, 2011). The lack of information ascertained by this study might relate to the locations of living labs at universities. In short, the attempt to embed higher education within living labs situated outside of universities may lead to other challenges than those faced by living labs embedded in higher education institutions.
We found several lessons learned regarding processes and interactions in living labs, as well as and requirements for successful living labs, including the importance of balance between freedom and frameworks. Furthermore, the literature shows it is crucial to invest in relationships between co-creators. Likewise, less competition and flat leadership with a living lab help to create a successful environment. Moreover, a living lab needs a supportive logistic infrastructure and closeness between stakeholders.
When focusing on how learning is designed in living labs, a distinction has been made between generic competences and specific competences. As expected, the specific competences differ between labs depending on their domains and subjects studied. Commonalities among generic living lab competences include co-creation, cooperation, clinical reasoning, and reflection, along with innovativeness and the ability to learn from experience. These competences match the key elements of living labs according to ENoLL, which are a real-life setting, co-creation, active user involvement, multi-stakeholder participation, and a multi-method approach (ENoLL, 2020). The learning activities identified in living labs seemed to be more innovative and interactive in contrast with more classical learning activities. Only three papers reported on assessments in living labs. Although education was part of most research objectives found in our study, we also discovered that none of the studies focused on the effects of educational activities in living labs on the competences of students.
Our aim was to explore the nature and extent of the scientific literature about living labs in which higher education actors actively participate. Other non-scientific papers that discuss this subject were not part of the selection, therefore this review does not capture the full body of knowledge in this domain. It is possible scientific studies that may be relevant could have been missed because of our selection of databases and use of search terms. Our search and selection specifically focused on articles addressing the concept of the “living lab”, as it seems to be an internationally accepted concept, and other reviews of living labs literature have already been conducted as referred to in our introduction. Our finding that all of the articles referred mostly to the same or similar aspects of living labs (as described by ENoLL) supports the assumption that this review capture the concept we were searching for.
Based on this scoping review, we conclude that research on embedding higher education in living labs is still at an early stage. More detailed studies into the participatory aspects of higher education are recommended in order to gain knowledge about enhancing learning outcomes, and the effects of educational activities including assessments within living lab environments.
In addition, knowledge appears to be lacking about conditions, organization, and governance of living labs, and further study would certainly be worthwhile. More emphasis on learning as a crucial aspect of living labs may steer the research and the theoretical foundations that support the embeddedness of higher education in living labs.
A previous version of this paper was presented in the 2021 Living Lab Days Conference.
Arksey, H., & O’Malley, L. 2005. Scoping Studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1): 19-32. DOI: https://doi.org/10.1080/1364557032000119616
Beecroft, R. 2018. Embedding Higher Education into a Real-World Lab: A Process-Oriented Analysis of Six Transdisciplinary Project Courses. Sustainability, 10(10): 3798. DOI: https://doi.org/10.3390/su10103798
van den Berg, A., Dewar, B., Smits, C., & Jukema, J.S. 2019. Experiences of Older Adults and Undergraduate Students in Co-creating Age-friendly Services in an Educational Living Lab. International Practice Development Journal, 9(2): 2. DOI: https://doi.org/10.19043/ipdj.92.002
Biggs, J. 2003. Constructing Learning by Aligning Teaching: Constructive alignment. Teaching for Quality Learning at University: 11-33.
Bouw, E., Zitter, I., & de Bruijn, E. 2019. Characteristics of Learning Environments at the Boundary between School and Work- A Literature Review. Educational Research Review, 26(1): 1-15. DOI: https://doi.org/10.1016/j.edurev.2018.12.002
Callaghan, R., & Herselman, M. 2015. Applying a Living Lab methodology to Support Innovation in Education at a University in South Africa. TD: The Journal for Transdisciplinary Research in Southern Africa, 11(1): 21-38. DOI: https://doi.org/10.4102/td.v11i1.30
Conruyt, N., Sébastien, V., Sébastien, O., Sébastien, D. & Grosser, D. 2014. Sign Management for the Future of e-Education: Examples of Collaborative e-Services in a Living Lab. In IFIP International Workshop on Artificial Intelligence for Knowledge Management: 1-20. Springer, Cham. DOI: https://doi.org/10.1007/978-3-319-28868-0_1
Cremers, P.H., Wals, A.E., Wesselink, R., Nieveen, N., & Mulder, M. 2014. Self-directed Lifelong Learning in Hybrid Learning Configurations. International Journal of Lifelong Education, 33(2): 207-232. DOI: https://doi.org/10.1080/02601370.2013.838704
Cremers, P.H., Wals, A.E., Wesselink, R., & Mulder, M. 2016. Design Principles for Hybrid Learning Configurations at the Interface Between School and Workplace. Learning Environments Research, 19(3): 309-334. DOI: https://doi.org/10.1007/s10984-016-9209-6
Cremers, P.H. 2016. Guidebook Living Labs: Tool for designing and evaluating living labs at the interface between school and workplace. Hanze Hogeschool.
ENoLL. 2020. What are Living Labs. https://enoll.org/about-us/
Era, C.D. & Landoni, P. 2014. Living Lab: A Methodology between User-Centred Design and Participatory Design. 23(2): 137-154. https://doi.org/10.1111/caim.12061
Evans, J., Jones, R., Karvonen, A., Millard, L., & Wendler, J. 2015. Living Labs and Co-Production: university campuses as platforms for sustainability science. Current Opinion in Environmental Sustainability, 16: 1-6. DOI: https://doi.org/10.1016/j.cosust.2015.06.005
Falk-Kessler, J., Benson, J.D., & Witchger Hansen, A.M. 2007. Moving the Classroom to the Clinic: The Experiences of Occupational Therapy Students During a “Living Lab”, Occupational Therapy In Health Care, 21(3): 79-91. DOI: https://doi.org/10.1080/J003v21n03_05
Goodlad, K., & Leonard, A.E. 2018. Place-Based Learning across the Disciplines: A Living Laboratory Approach to Pedagogy. Insight: A Journal of Scholarly Teaching, 13: 150-164. DOI: https://doi.org/10.46504/14201808go
Grove, W. 2018. Living Labs and Designed Serendipity: collaboratively discovering the UDUBSit & Mfunzi emerging platforms. Afrika Focus, 31(1). DOI: https://doi.org/10.21825/af.v31i1.9041
Gualandi, E., & Romme, A.G.L. 2019. How to Make Living Labs More Financially Sustainable? Case studies in Italy and the Netherlands. Engineering Management Research, 8(1): 11-19. DOI: https://doi.org/10.5539/emr.v8n1p11
Hansen, S.S. 2017. The Campus as a Living Laboratory: Macalester college case study. In Handbook of theory and practice of sustainable development in higher education: 223-239 Springer, Cham.
Hawk, N., Romine, M. & Bartle, G. 2012. The Living Labs: innovation in real-life settings. Quarterly Review of Distance Education, 13(4): 225.
Hummels, C.C.M., & Vinke, A.A. 2010. Community Building Through a Theme-Based Living Learning Lab. In ConnectED2010 2nd International Conference on Design Education; 28 June – 1 July 2010, Sydney: 1-4.
Igartua, J.I., & Albors, J. 2011. The Implementation Process of Innovation Management in a Medium-Sized Company. Projectics/Proyectica/Projectique, (1): 105-121.
De Jager, L., Buitendag, A.A., & Van der Walt, J.S. 2012. Presenting a Framework for Knowledge Management within a Web-enabled Living Lab. South African Journal of Information Management, 14(1): 1-13. DOI: https://doi.org/10.4102/sajim.v14i1.506
Jernsand, E.M. 2019. Student Living Labs as Innovation Arenas for Sustainable Tourism. Tourism Recreation Research, 44(3): 337-347. DOI: https://doi.org/10.1080/02508281.2019.1613299
Jukema, J.S., Veerman, M., Van Alphen, J., Visser, G., Smits, C., & Kingma, T. 2017. Nurturing Gerontology Students’ Intrinsic Motivation to Cocreate: The design of a powerful learning environment. Gerontology & geriatrics education, 40(4): 432-441. DOI: https://doi.org/10.1080/02701960.2017.1377702
Kröse, B., Veenstra, M., Robben, S., & Kanis, M. 2012. Living Labs as Educational Tool for Ambient Intelligence. In International Joint Conference on Ambient Intelligence: 356-363. Springer, Berlin, Heidelberg. DOI: https://doi.org/10.1007/978-3-642-34898-3_27
Leminen, S. 2015. Living Labs as Open Innovation Networks-Networks, Roles and Innovation Outcomes. Doctoral Dissertation. Altoo University, Finland
Maas, T., van den Broek, J. & Deuten, J. 2017. Living labs in Nederland: van open testfaciliteit tot levend lab. Rathenau Instituut.
Malmberg, K., & Vaittinen, I. 2019. Living Lab Methodology Handbook. https://u4iot.eu/pdf/U4IoT_LivingLabMethodology_Handbook.pdfhttps://u4iot.eu/pdf/U4IoT_LivingLabMethodology_Handbook.pdf
Masseck, T. 2017. Living labs in architecture as innovation arenas within higher education institutions. Energy Procedia, 115: 383-389. DOI: https://doi.org/10.1016/j.egypro.2017.05.035
McCormick, K., & Kiss, B. 2015. Learning Through Renovations for Urban Sustainability: The case of the Malmö Innovation Platform. Current Opinion in Environmental Sustainability, 16: 44-50. DOI: https://doi.org/10.1016/j.cosust.2015.06.011
Peng, Y., Lu, Z., Wang, J., Xu, J., & Li, H. 2010. Experiential Teaching Based on Living Lab: A study on the telecommunication service experience. In 2010 International Conference on E-Health Networking Digital Ecosystems and Technologies (EDT) 2: 340-343. IEEE. DOI: https://doi.org/10.1109/EDT.2010.5496353
Santally, M.I., Cooshna-Naik, D., & Conruyt, N. 2014. A Model for the Transformation of the Mauritian Classroom based on the Living Lab Concept. In 2014 IST-Africa Conference Proceedings: 1-10. IEEE. DOI: https://doi.org/10.1109/ISTAFRICA.2014.6880603
Schuurman, D., De Marez, L., & Ballon, P. 2015. Living Labs: a systematic literature review. Open Living Lab Days 2015, Proceedings. Presented at the Open Living Lab Days 2015.
Van Tomme, N., Voets, J., & Verhoest, K. 2011. Samenwerking in Ketens en Netwerken: praktijkervaringen uit de zorg-en welzijnssector. SWVG-Rapporten.
Veeckman, C., Schuurman, D., Leminen, S., & Westerlund, M. 2013. Linking Living Lab Characteristics and their Outcomes: Towards a conceptual framework. Technology Innovation Management Review, 3(12): 6-15. DOI: http://dx.doi.org/10.22215/timreview/748
Wals, A.E.J., Lans, T., & Kupper, H. 2012. Blurring the Boundaries between Vocational Education, business and research in the agri-food domain. Journal of Vocational Education, and Training, 64(1): 3-23. DOI: https://doi.org/10.1080/13636820.2011.586129
Wenger, E., McDermott, R. A., & Snyder, W. 2002. Cultivating communities of practice: A guide to managing knowledge. Cambridge, MA: Harvard Business Press.
Zitter, I., Hoeve, A., & de Bruijn, E. 2016. A Design Perspective on the School-Work Boundary: A hybrid curriculum model. Vocations and Learning, 9(1): 111-131. DOI: https://doi.org/10.1007/s12186-016-9150-y
Zuyd University of Applied Sciences. 2019. Zuyd Strategy. Passion for development. Heerlen: Zuyd University of Applied Sciences.
Appendix 1. Supplementary Materials
Keywords: approaches, definitions, higher education, key elements, Living lab, scoping review