September 2012 Download this article as a PDFAbstract

A shift towards open innovation approaches with systematic user involvement has occurred within media and ICT. One of the emerging frameworks structuring these initiatives is the "living lab" approach. Despite the growing evidence of the beneficial nature of customer involvement in product development, research into specific user characteristics for innovation is still scarce, particularly in living labs, with the notable exception of literature on lead users. Especially within the context of living labs for ICT and media innovation, an application of the lead-user framework looks promising as a way to structure and facilitate user involvement. This article is based on the experiences of three Flemish living lab initiatives with a panel-based approach and provides a customer characteristics framework that guides user involvement in living labs.


Although the quotation topping this article may sound dated, this line of reasoning is closely associated with the technology-push paradigm and has dominated the view on innovation for a long time. The market-pull paradigm, dating back to the 1960s, shifted the focus from pure invention and development of technology towards the eventual adopter and user of the innovation. With the recent advent of the open-innovation paradigm, end-users have reclaimed their place within innovation processes, particularly in new media and ICT. This is reflected in popular concepts such as open source, crowdsourcing, and user generated content. One of the most recent methodologies for user-centered innovation is the living lab approach, which has gained momentum especially in Europe through the support of EU-policy and various international joint initiatives, such as the European Network of Living Labs (ENoLL), which together consist of over 500 living labs worldwide.

Although living labs provide a way to structure and facilitate user involvement in new media and ICT innovation (Almirall and Wareham, 2009), few attempts have been made to couple the user and customer involvement literature with living labs. This article sheds light on the question of which users to involve in a living lab project by providing a framework for customer characteristics in innovation. It builds upon lead-user characteristics and the concrete application will be demonstrated by means of multiple cases from three Flemish ICT Living Labs: LeYLAB, Vlaams Proeftuin Platform, and Mediatuin. It is suggested that a panel-based living lab approach might facilitate and optimize this kind of user involvement and some key lessons are abstracted out of concrete practice.

Customer Characteristics for User Involvement

For quite some time, studies have been investigating characteristics for user involvement. Eric von Hippel came up with his influential lead-user concept in the 1970s. He considered using lead users as a counter weight for traditional market research, which focuses on users at the centre of the market. Instead, the lead-user approach looks for users at the leading edge of the target market or even at users from other markets, who face similar problems as the target market, but in a more extreme form. According to von Hippel, these lead users display two main characteristics: i) they face needs long before the others in the market and ii) they expect to have a significant benefit when they obtain a solution to these needs.

Recently, this dichotomy has been challenged by pleading for a more collaborative mode of user participation in innovation processes. This has been given names such as "design by customers", "open innovation with customers" or "collaborative new product development" (e.g., Piller and Ihl, 2009). Instead of looking for lead users and lead-user innovation or simply surveying users from the centre of the target market, collaboration with users or customers during various stages of the new product development process is put forward as best practice. This has led to studies investigating customer characteristics for involvement in innovation processes. Out of these studies, we deduct four main dimensions on which we will build our framework and which will be further explored in the section on user characteristics:

  1. User expertise: consists of product-related knowledge and user knowledge. This dimension is abstracted from research by Lüthje and Herstatt (2004), who demonstrated that the ability of lead users to be effective contributors to the innovation process is related to two major characteristics: adequate technological expertise and superior knowledge of the user domain and "use experience". User expertise thus implies that the user has specific knowledge or expertise with regards to the innovation or the domain in which a company wants to innovate.
  2. Usage intensity: measures the experience of the user, including both the duration and diversity of the usage (Shih and Venkatesh, 2004). Research has showed that extreme usage and use innovativeness aids the innovation process by foreshadowing changing and emerging usage behaviour (Pichyangkul et al., 2012).
  3. New needs: refers to the fact that the user has emerging needs that cannot be fulfilled by the current market offering. This dimension is abstracted from the classical lead-user definition and can be detected by two proxies. A first proxy is dissatisfaction with the current offering, which leads the user to become an ex-customer. Research by Duverger and Hassan (2011) mentioned the innovative capacities of these "defectors" and demonstrated that this kind of "ex-user" is able to generate radical new product or service ideas. A second proxy is user innovation, because studies have shown that lead users are likely to solve their unmet needs by innovating themselves (e.g., Lüthje and Herstatt, 2004).
  4. User innovativeness: can be measured through the rate of adoption of technology and innovations in a certain domain. This is based on the diffusion-of-innovations framework by Rogers (1962), which illustrated that users show unique characteristics based upon time of adoption.

Panel-Based Living Labs

Living labs have been defined from different angles and with different outcomes in the literature. Schuurman and colleagues (2012) extensively discussed various bottom-up and top-down conceptualizations out of concrete living labs practices with various results. However, we chose the following definition, inspired by Almirall and Wareham (2008): living labs can be seen as innovation arenas or "innovation intermediaries" because they build a multi-stakeholder ecosystem where users are subjected to a combination of research methodologies while they test new technologies that are still in development with the focus on accessing the ideas and knowledge of the users regarding the tested technology. Therefore, living labs are capable of providing structure to user participation in innovation processes.

Living labs are seen as separate from other innovation approaches by means of two dimensions: a high degree of realism and a high degree of (user) involvement (Table 1). Living labs offer both realism and an active user involvement, because the user is regarded as a partner in the innovation process during which the needs, aspirations, and motives of users emerge in their everyday context in an active and iterative manner. Living lab settings are used to perform quantitative and qualitative research methods on the users’ ideas, skills, knowledge, and experiences.

Table 1. Living Labs versus other research methods


High Realism

Low Realism

Active user involvement

Living Labs

Prototyping/testing platforms

Passive user involvement

Field trials

Market pilots

Voice of the customer methods (e.g., surveys, focus groups)

A panel-based approach is yet another element that can be added to living labs. This approach differentiates itself from "traditional" living labs because it implies a more permanent living lab infrastructure, as opposed to one-shot living lab applications, in which the most important and central "infrastructure" consists of a thematically recruited and profiled panel of users. It can be argued that instead of putting "the user" or " the customer" at the centre of the innovation process, a well-described and thematically focused panel is put in the centre of the process. In terms of the stages in the setting-up of a living lab, as defined by Pierson and Lievens (2005), a panel-based living lab approach yields many benefits; these benefits are listed in Table 2.

Table 2. Added value of a panel-based living lab approach


"Traditional" Living Labs

Panel-Based Living Labs


  • an exploration of the technological and social implications of the technology or service under investigation; technological scan and state-of-the-art study that identifies the relevant concepts
  • through the longitudinal data the panel generates, a permanent "contextualization" is taking place for the surveyed topics. The recurring surveys also allow easy integration of a "new" topic for contextualizing purposes.


  • identifying potential users or user groups; this can be done on a socio-demographic level, based on selective or criterion sampling; this allows for theoretical variation of previously defined concepts
  • the identification of potential users or user groups is only a matter of selecting the right profiles out of the panel database. This avoids the time-consuming and costly surveying and recruiting of relevant user profiles.


  • an initial measurement of the selected users on current characteristics, behaviour, and perceptions regarding the research focus in order to enable an ex post measurement after the implementation phase
  • the initial measurement of the selected users on current characteristics, behaviour, and perceptions regarding the research focus is in most cases (partly) already present within the panel data, so only a brief extra survey needs to be filled out by a selected set of panel members. This consists mostly of an estimation of the attitude and adoption intention towards the innovation concept (see text).


  • the operationally running test phase of the living lab; research methods: direct analysis of usage by means of remote data collection techniques (e.g., logging), indirect analysis based on, for example, focus groups, interviews, self-reporting techniques
  • the operationally running test phase of the living lab is also made a lot easier by the panel approach because accurate and up-to-date data is available, which allows for an optimal selection and setting up of the trial phase with less worries regarding privacy and related concerns, because all panel members have "opted in". A better collaboration is also guaranteed because users have already agreed to be part of a research panel.


  • an ex post measurement of the users (same techniques of initial measurement) and a set of technological recommendations from the analysis of data gathered during the implementation phase
  • an ex post measurement of the users can possibly be integrated with the existing surveys sent out to the panel members. The fact that, after a living lab project, the users remain part of the panel and the research data is added to the existing data ensures a broader picture over the different living lab projects.

We will illustrate this panel-based approach by means of iLab.o, the living lab division of the Interdisciplinary Institute for BroadBand Technology (IBBT), which is a founding member and secretary for ENoLL. In practice, iLab.o sets up medium to large-scale trials outside the lab environment involving different stakeholders. Within these trials, representative users have the chance to test ICT innovations over a longer period of time in their daily professional and private environments. This allows for researchers to assemble user feedback and to systematically observe, monitor, and analyze user behaviour in a natural environment. iLab.o’s panel-based approach consists of recurring recruitment activities to gather panel members who are willing to cooperate in living lab research. The recruitment consists of a large intake survey that looks at the respondent’s usage and adoption of (media) technologies, adjusted to the thematic focus of the specific living lab. This way, there is a constant inflow of panel members with up-to-date data regarding their habits, usage, and adoption of specific products, technologies, and services. For the recruitment of respondents, quota samples are used to ensure the representativeness of the survey population. All this data is stored and managed by the Living Lab Integrated Data Collection and Aggregation Model (LLADA), which is a piece of software specially created for living lab panel management. Besides data from the recurring intake surveys (for an example, see Digimeter), all data from living lab research is collected with this tool. This way, the user profiles of the living lab panel members are updated every time they participate in living lab research. A necessary prerequisite for this panel approach to function optimally is rigorous panel management. iLab.o is the research partner in three ICT-related living labs in the geographical area of Flanders, all of them partly being financed by the Flemish government. Table 3 compares these three living labs, which are further described in Boxes 1 to 3.

Box 1. LeYLab

LeYLab was set up in September 2010 following a public call in Flanders for living labs with "converged broadband access networks" as the central theme. LeYLab was operational by July 2011 and its fibre network is located in two geographically restricted areas (Buda and Overleie) in the City of Kortrijk. The goal of LeYLab is to stimulate innovation and to measure the relevance of new services for the personal lifestyle and living environment of the test users. The consortium of LeYLab consists of 11 industrial partners and the research partner IBBT-iLab.o. The living lab focuses on three thematic domains: e-care, multimedia, and gaming. The fibre internet connection functions as a facilitator for the testing of innovative services and products. In January 2011, a large communication and recruitment action was set up to motivate people living in the selected areas to participate in the living lab. Eventually, 115 addresses were connected to the fibre network; the addresses are mostly residential but also include cultural organizations, schools, and companies. In order to facilitate testing of different services for different devices, the consortium decided to provide some of the connected homes with extra devices (e.g., Android tablets, mini-PCs connected to flatscreen TVs) besides the fibre connection. All connected addresses received multiple surveys in order to allow profiling of the test users for the relevant thematic domains and all data and actions running on the LeYLab fibre network were monitored and logged.

Box 2. Vlaams Proeftuin Platform

The Vlaams Proeftuin Platform (Flemish Living Lab Platform) officially started in October 2010 to support the development of innovative information, communication, and entertainment (ICE) products and services. Its mission is to boost the valorisation of ICE research and development in Flanders and to support joint value creation for all stakeholders. Vlaams Proeftuin Platform is a consortium of four industrial partners and the research department IBBT-iLab.o. The living lab focuses on three domains: Smart Cities, Smart Grids, and Smart Media. A large panel of 2015 users has been built up and has been thoroughly profiled within the three domains through bi-monthly domain-specific surveys.

Box 3. Mediatuin

Mediatuin (or media garden) started in October 2010 to optimize, co-create, and validate media innovation with a cross-media focus. The Mediatuin consortium consists of three industrial partners (SonicAngel, Netlog, and Telenet), the research department IBBT-iLab.o, and REC Radiocentrum (a non-profit organization aimed at stimulating and educating young media talents). The thematic focus of Mediatuin is media, with special attention given to radio and music. By means of a large intake survey, a dataset of more than 7000 respondents was collected with more than 2000 people willing to be involved in living lab projects as test users. This survey was very detailed and focused on the thematic domains of Mediatuin, thus offering a lot of relevant data for the projects that were set up.

Table 3. Characteristics of the three panel-based living labs

Living Lab




Number of Test Users


  • multimedia
  • e-homecare
  • gaming
  • experimenting with new applications that request a fast Internet connection
  • digital inclusion, bringing any multimedia service to everyone on any device
  • optimizing the life of care-demanding citizens through new technologies
  • fibre internet connection
  • 43 Android tablets
  • 36 mini-PCs connected to flatscreen TVs
  • project-based infrastructure
  • core panel of 115 connected addresses with +/- 200 profiled panel members

Vlaams Proeftuin Platform

  • smart grids
  • smart media
  • smart cities
  • energy management and energy reduction
  • user research on smart media applications and services to evaluate innovative media experience
  • increasing the self-sustain of elderly people
  • no permanent infrastructure, only project based
  • 2015 profiled panel members


  • cross-media innovation
  • co-creating and validating cross media innovations and formats
  • no permanent infrastructure, only project based
  • dataset of 7000 respondents of which 2057 consented to be panel members

Through the profiling of the test users for the relevant domains and for the chosen focus, the panels from the three living labs can be considered as an essential part of the "living" infrastructure of these labs. This makes it easy and quick to gather a relevant set of respondents or test users for a concrete living lab project being carried out in the living lab. Also, by running different projects, further data and knowledge regarding these panel members are generated, which refreshes and updates the database, thereby adding even more depth to the profiles.

User Characteristics in Living Labs

We will now provide some examples from living lab projects where user characteristics, abstracted from the living lab panels’ user profiles, were used to select and recruit users for involvement in different research steps. These examples will illustrate the added value of employing our framework for user selection over random or general user selection or recruitment by means of practice-based evidence.

Dimension 1: User expertise

Within a LeYLab project, users were recruited for a co-creation and co-design session in order to develop a second-screen tablet application for a regional broadcaster’s popular quiz program. The selection of users was based upon their experience with second-screen applications and social media and their interest in quiz programs. This data was captured during the intake survey that had to be filled out by every LeYLab panel member. This way, we were able to quickly gather a relevant group of people for the co-creation session, moderated by a researcher but also with active participation of the application developer and a representative from the quiz program. The developer and the quiz program’s representative already had basic ideas for how the application would function, but after the co-creation session, these ideas were changed quite radically and a paper mock-up was developed, from which the actual application was developed later in the project. Because of their user expertise, the participants were able to confront the developer’s ideas with their own usage experience and provide concrete suggestions and comments that were directly implementable. Their knowledge also aided in co-designing the actual user interface of the application.

Dimension 2: Usage intensity

With Mediatuin, usage data from a beta version of an online application was gathered through log files. This way, test users could be segmented based upon their usage intensity. This segmentation was enriched with survey data before the actual usage of the innovative application, which allowed for a comparison between usage intention and actual usage. Users matching the different segments were assembled within a co-creation session that resulted in a lot of very specific feedback ranging from enthusiastic users that were disappointed with the actual beta version to skeptical users that were positively surprised by the functionality of the application. This way, a broad range of feedback could be captured with only a limited set of divergent test users. Usage intensity was in this case used as a criterion for a co-creation session after the actual testing of the innovation and referred to the usage of the innovation itself. By comparing actual behaviour with intended behaviour before the field trial (see below), captured during the contextualization stage, discrepancies can be detected. Test users with a positive discrepancy (low usage intention, high actual usage) are key to discovering certain drivers for adoption and usage by user groups that at first sight did not find much appeal in the innovation. The other way round, a negative discrepancy (high use intention, low actual usage) can highlight the barriers that can impede adoption and usage by possible earlier adopters.

Dimension 3: New needs

With Vlaams Proeftuin Platform, a sample of youngsters was selected for participation in a live field trial of an online advertising platform for youngsters. Through logging, the usage of these test users could be assessed. Some of the youngsters only logged in to the platform once and never came back after their first usage, although some of them had showed interest in it during the pre-testing evaluation of the concept.. We considered this to be an indicator of dissatisfaction and thus of new or unmet needs with regards to the innovation. These users were contacted for participation in the co-creation sessions. Apparently, during the actual testing, some aspects of the platform left them dissatisfied, which led to their abandonment after one usage session. These users provided valuable feedback to the platform developers. Dissatisfied users are especially able to provide information regarding certain needs that are currently unsatisfied. After a field trial, the dissatisfied test users are able to identify the barriers or flaws resulting in their dissatisfaction. This finding establishes a connection with the previous dimension, because low usage intensity might be an indicator of dissatisfaction.

In Mediatuin, we used the detection of "user innovation" as a proxy to identify new needs within a project for an online radio recording service. We included an open question regarding users’ current habits and practices for recording radio within the recruitment survey to assess interest in the concept. Besides some general answers, we identified one user who had programmed his own online recording solution for Linux. He simply wrote down the lines of code he had used to create his own solution. This user innovator was used later on in the development process of the online radio recording service. User innovators can provide relevant input to the innovation process because these users clearly have new needs and also user expertise. User innovation can thus be seen as a proxy to identify users with new needs and with high user expertise, in other words the so-called lead users that can generate valuable information during the whole innovation trajectory and that also can be engaged in more profound and technical co-creation activities.

Dimension 4: User innovativeness

This final dimension is utilized in nearly all living lab projects, as within the "concretization stage" (see Table 2), the adoption intention of the innovation in development is surveyed by means of the product-specific adoption intention (PSAP) method (De Marez and Verleye, 2004). For the selection of test users, a variation in terms of user innovativeness guarantees a broader picture because users identified as potential later adopters are likely to show different usage patterns than potential early adopters or visionaries. With Mediatuin and Vlaams Proeftuin Platform, discrepancies between innovativeness and actual usage intensity identified dissatisfaction. However, user innovativeness towards technologies in a given target domain can also be used, for example, in the LeYLab case. Early adoption of tablets and second-screen services was used as an indicator of user expertise with regards to the innovation in development.


Living labs are being used to structure user participation in real-life settings. However, to optimize this participation, we firmly believe the customer and user characteristics of test users should be taken into account. This article suggested four different dimensions for user involvement in innovation processes in light of the panel-based living lab approach. We will now formulate key lessons that can aid innovation managers and living lab organizers when setting up a living lab infrastructure or a concrete living lab innovation project. These findings are also of interest to companies willing to engage end-users in their innovation efforts, because they provide some insight into how this can be done.

In order to use the user expertise criterion for user selection, it is necessary to recruit or utilize thematic panels with a specific focus. When the profiling is not sufficient, or not enough panel members have the right criteria or characteristics, extra intake is needed. However, this extra intake is also an opportunity to refresh and enlarge your existing living lab panel. When your panel has a mismatch with the living lab project and there is an insufficient number of users with relevant user expertise available, it is better not to use the living lab for that particular project.

For the usage intensity dimension, it is necessary to capture user behaviour. This can be done through self-assessment of panel members (e.g., surveys), but this should be complemented with unobtrusive logging data registering usage behaviour. A permanent infrastructure with logging facilities, such as in the case of LeYLab, provides the best opportunities to gather and utilize the data in order to recruit test users based on usage intensity.

In order to use the dissatisfaction criterion, which is associated with the user type, surveys are the most obvious technique, but there is also the possibility to look for and analyze indicators of dissatisfaction, such as a decreasing usage intensity. Again, logging can be used successfully here, because one or a few usage moments in the beginning of the test phase within a living lab followed by no activity at all might be an indicator of dissatisfaction with the tested product or service.

The new-needs criterion is most closely associated with the classic lead-user concept. Dissatisfaction with the current offering can be an indicator of new needs, which makes it necessary to measure the degree of satisfaction in order to identify possible "defectors". Scanning for user innovation is another way to detect new needs. This can be done by simply asking for examples of user innovation in a survey or during interviews because a lot of innovating users are happy to share their innovation with you. Home or site visits can also reveal user innovation.

Finally, the user innovativeness dimension, which was associated with the diffusion-of-innovations framework, allows for user segmentation when the time of adoption is predicted for the innovation concept in development. This predicted adoption potential also allows researchers to identify discrepancies with actual usage behaviour during the live phase of the living lab. A detailed profiling of the panel in terms of innovativeness with regards to a certain thematic domain is also advised. The speed and number of adoptions with regards to relevant technologies and services already available in the target market domain should be surveyed.

In sum, a panel-based living lab facilitates user recruitment based on specific characteristics related to the innovation being developed and tested in the living lab. However, recruiting and managing this panel requires a lot of time and effort and should be done with careful consideration. Living lab projects should fit the scope of the panel; otherwise the added value of the living lab will be lost. However, when projects fit the scope, it will keep the panel alive and up to date, and it will improve the added value of the living lab through the continuous data generation. It is also apparent that the four identified dimensions of the user-characteristics framework are far from independent . A lot of the criteria and proxy measures are mostly interrelated, so the framework should be used in a dynamic way, adapting it to the specific target domain in which the living lab activities will run and carefully selecting variables and proxies to identify the different characteristics. Further exploration and implementation of this framework is definitely a subject for future research.

Share this article:

Cite this article:

Rate This Content: 
3 votes have been cast, with an average score of 5 stars

Keywords: customer characteristics, living labs, Open innovation, user panels, user-driven innovation