Introduction

Knowledge mobilization relies on evidence of sufficient quality to enable appropriate decision making. Although many established sectors produce well-controlled research with sufficient sample sizes to guide decision making confidently, the knowledge mobilization process is challenged in sectors where the pace of change outpaces the ability to complete high-quality research methodologies. For example, technology-sector research involving products and humans can take much longer to complete that the annual or faster product-revision cycles. Therefore, the outcomes from a large multi-site, randomized control study may not apply to a new product that replaces the tested technology. Although well-controlled research with appropriate sample sizes is needed, this approach must be balanced with other evidence sources to address the knowledge-immediacy requirements. The assistive device sector is an interesting group for exploring this human–technology knowledge mobilization issue.

Assistive devices that improve mobility for people with disabilities cover a range of technologies, from wheelchairs to prosthetics and orthotics to robotic exoskeletons. Although the target populations and core technologies vary between these mobility assistive devices, a common element is the rapid pace of innovation in relatively small markets (Baljko & Hamidi, 2014). Modern assistive devices are on annual or biannual revision cycles, often with faster revision cycles for devices controlled by high-end microprocessors. While this pace of innovation should be encouraged to provide the best technology for people with mobility disabilities, constant change presents challenges for evidence-based practice and knowledge mobilization.

In the field of physical rehabilitation, several challenges have been identified for knowledge development and mobilization (Johnston et al., 2009). These challenges include:

• the complexity of interactions between people with a disability and their environment (WHO, 2016)
• the need to evaluate people as they participate in their daily lives
• small sample sizes
• the difficulty or impossibility of implementing blinding or placebos
• ethical issues for control groups who do not receive treatment
• integrating assistive technology evaluation with other research methods and design factors
• the lack of funding for large effectiveness studies
• the difficulty of addressing issues between large social systems and physical, economic, and environmental factors

These challenges are directly related to knowledge mobilization for assistive devices. In fact, poor consideration of assistive technology use in rehabilitation research can be a confounding factor when interpreting research results (Rust & Smith, 2005).

To positively affect practice, the appropriate evidence with the appropriate quality must exist. For mobility assistive devices, appropriate evidence may be in forms other than peer reviewed academic publications. Appropriate assistive device research for knowledge mobilization can be classified as technical, biomechanical, and clinical. An aspect that is often overlooked on the technical category are standards-based evaluations that may be conducted by the manufacturer, independent laboratories, or government laboratories. Although countries may implement additional test procedures, the base for assistive device technical evaluations are the International Standards Organization (ISO, 2016; Rust & Smith, 2005) and Rehabilitation Engineering Society and Assistive Technology Society for North America (RESNA, 2016). Standard tests typically cover structural capacity (load tests, etc.), device function (wheelchair stability, etc.), and other factors such as flammability.

The study by Gebrosky and colleagues (2013) demonstrated the usefulness of these standard tests by performing the ANSI/RESNA standard tests on a series of lightweight wheelchairs. Most of the wheelchairs tested did not meet durability standards and a recommendation was made for stronger regulations and testing by independent and certified facilities. Ensuring that standard tests are appropriately administered is essential, but access to these test results is also lacking given that companies, test facilities, and regulators are not required to share test outcomes. Even if the intent to share this information existed, a reference and indexing source that can easily be used by clinicians is not available. For knowledge mobilization, systems such as clinical trial registries (such as ClinicalTrials.gov) are needed to provide an access gateway to the standard evaluation outcomes for assistive devices. This would enable direct access by the healthcare community and provide a base for knowledge synthesis by researchers and other stakeholders.

Assistive devices are typically categorized under Class 1 or Class 2 for regulatory approval. In Canada, Class 1 devices have the lowest medical risk and do not require a Medical Device License. Therefore, much of the background evidence is not provided through the regulatory process. Even with regulatory information, independent research evidence to guide clinical decision making is usually lacking when new or modified devices appear in the market.

From a biomechanical perspective, a modest amount of literature exists for the effect on movement and body structures from assistive device use. A search using the Scopus database with keywords “wheelchair biomechanics” returned 953 results, and 151 results since 2012. A search using “(prosthetics or orthotics) and biomechanics : excluding implants (surgical), Animal, Arthroplasty, Bone Cements, Biomedical Engineering – Surgical Implants” returned 2,689 publications, with 406 results since 2012. In comparison, a search using “physical rehabilitation and biomechanics” returned 55,042 publications. Most of the biomechanics literature involves motion laboratory studies, with the majority of locomotor research on level ground. However, the emergence of wearable sensor systems and room-scale virtual reality systems (Sinitski et al., 2015) are providing research tools to generate device-function evidence that better reflects movement environments encountered in daily living.

Clinical research on assistive technology includes questionnaire-based tools for evaluating user perceptions of using an assistive device for daily living and function-based tasks. On a base level, classification tools can be used to facilitate collection of data related to assistive devices and disability and to define categories related to disability and assistive technology (e.g., function, body structures, activities, participation, environmental factors) (Jutai et al., 2005; Lenker & Paquet, 2003). Other tools measure psycho-social aspects of assistive technology use (Jutai & Day, 2002; Scherer et al., 2011), user satisfaction (Demers et al., 2002; Samuelsson & Wressle, 2008), factors for assistive technology selection (Bernd et al., 2009), and device specific tools (Condie et al., 2006; Miller & Swanson, 2009; Wright, 2009).

Ideally, a continuum of evidence can be amassed that covers the need for immediate information on new technologies, broader research that quantifies how a device compares with similar assistive devices in the market or literature, and high-quality evidence for assistive device concepts that exist throughout a device class. To explore this continuum concept, a case study involving carbon-fibre ankle–foot orthoses is presented here.

Evaluating the Current State of Carbon-Fibre Ankle–Foot Orthoses

A carbon-fibre ankle–foot orthosis (CF-AFO) is a lower limb brace that controls movement of the foot relative to the shank for people with lower limb weakness, paralysis, or excessive tone/spasm. Carbon-fibre materials produce a device that is thin and light, stiff, or capable of storing and returning energy when walking. A market survey conducted by the investigator returned 68 distinct CF-AFO products from 13 manufacturers (i.e., devices with distinct designs or functions, not including different sizes). Most CF-AFOs consist of a carbon-fibre foot and shank section that inserts into the shoe and a strap at the proximal end to anchor the orthosis to the shank (Figure 1). The energy storage design consists of separate foot and shank sections that both connect to a centre energy storing part (rods or rectangular section) that stores energy after foot strike and then releases stored energy at toe off. An expiring patent on this technology should lead to many more devices entering the market place in the next few years.

Figure 1. One piece prefabricated ankle–foot orthosis (A: Ossur AFO Dynamic) and custom ankle–foot orthosis with carbon-fibre strut (B: Fabtech Posterior Dynamic Element).

The first level of evidence for device function and unique contributions is patents. The investigator identified 25 relevant patents with a Google Patent search using the keywords “carbon fibre ankle foot orthosis AFO”. This information provides claims and design concepts for many of the commercial devices. Peer review is from patent office experts. Although this level of evidence is insufficient to guide clinical practice, it does provide information to help understand the objectives, concepts, and novel contributions that differ between devices.

From a regulatory perspective, CF-AFOs are Class 1 medical devices and therefore the manufacturer is not required to provide evidence on device safety or function. Although regional regulatory requirements may provide a level of evidence, this information cannot be expected for this class of assistive device. In general, when assembling knowledge for mobilization, regulatory testing results are typically not considered, but could be a useful source of information that would be available as a technology enters the market.

From the literature, searches of Scopus and Google Scholar databases with the keywords “ankle foot orthosis afo carbon fibre” produced 29 relevant articles published between 2006 and 2016. Of these, 9 were related to AFO mechanical testing, 15 using biomechanical and physiological analysis, and 4 using clinical tests.

For mechanical testing, a series of experimental protocols were used to apply loads and measure the resulting AFO movement. These methods evaluated AFO stiffness, engineering modelling and structural analysis, and repeated loading to determine how the device could fail. However, the lack of a standard measurement method prohibits valid comparisons between studies and meta-analysis. The orthosis–limb interaction is also a consistent deficiency. Orthoses are designed to intimately interact with the foot and lower limb; therefore, the limb is part of the structural system and therefore must be considered when testing device performance. Most studies did not consider the limb and, of the three studies that tested the AFO with a limb surrogate, a rigid plaster limb shape with a single axis joint was employed. For carbon-fibre AFO with multiple axes of possible movement, a solid single-axis foot–leg model is insufficient for replicating AFO movement under daily activity loads. Future research should develop a more appropriate foot–leg surrogate to enable mechanical testing so that the results can be more directly translated to the real world. For knowledge mobilization, this example applies broadly to knowledge obtained through simulation and modelling. Insufficient or inappropriate identification of the person within the system being evaluated could lead to poor evidence that would not be easily identified by the knowledge user following knowledge translation.

Biomechanical testing can be grouped by design: custom AFO with a posterior strut that stores and releases energy during gait, or lighter one-piece carbon-fibre AFO that is typically prefabricated. 3D quantitative motion analysis demonstrated that devices with a posterior strut typically stored and returned more energy, thereby helping the person walk more naturally and with less effort, and fine control of strut stiffness may not be required for prescription but can affect comfort and range of movement. Although the biomechanical evidence is adequate, most papers originated from one study at one laboratory and many of the other studies did not provide appropriate comparative data. Biomechanical research on the one-piece carbon-fibre designs included different populations in each study, such as able-bodied, Charcot-Marie-Tooth, polio, multiple sclerosis, peroneal nerve injury, and stroke. Given that each population had different walking characteristics and requirements, comparisons across populations become difficult. Analyses also vary, from full 3D motion analysis to simple stride analysis to physiological energy expenditure. Although the available biomechanical information provides a basis for understanding how various CF-AFOs perform during walking, the level of evidence remains insufficient to support clinical decisions when choosing between the many designs available in the marketplace.

Biomechanical laboratory-based analyses have difficulty meeting the immediacy requirement for knowledge mobilization, where barriers for providing timely outcomes include costs and time for completing the study (fitting devices for each person), laboratory access, and recruitment fatigue from a regional participant pool that are approached for multiple evaluation studies. Economically, funding these studies may be difficult for small manufacturers where profits from individual products are much less than other health sectors, such as pharmaceuticals. Motion-laboratory analyses may be better positioned for assessing broader classes of devices and first occurrence of disruptive technologies.

Methodologically, many biomechanical studies compared AFO gait to walking without a device. Given that the literature has previously established that any appropriately prescribed AFO improves gait over walking without the orthosis (Tyson et al., 2013), this comparison does not inform the clinical community as to the device’s relevance in the marketplace. This is in contradiction to the typical placebo methodology where the placebo is a harmless intervention prescribed for the psychological benefit of the participant. For knowledge mobilization, comparisons between the assistive technology and the current standard of practice for the target population are required to ensure that a clinical decision maker can use the translated knowledge to recommend the most appropriate device for the patient.

Clinical testing also varied between the four studies. Across all studies, different combinations of 14 tests were employed to evaluate balance, muscle strength, functional status, walking and movement capacity, mobility capacity, quality of life, and disability. As with the biomechanical studies, different populations participated in each study, and each population had different movement characteristics. These clinical tests provided functional performance measures that helped to complete the knowledge base for CF-AFO; however, the quantity of information is insufficient for making evidence-based decisions for CF-AFO prescription and fitting. The advantage of these clinical tests are relatively quick implementation (minutes), portability, quick reporting, and standardized protocols. However, these tests only provide a high-level assessment of the parameter (i.e., we could learn that a person walks faster with a new assistive device, but not why they walk faster).

As with much of the assistive technology literature, participants were not well described in reference to how their disability could affect CF-AFO performance. Detailed research participant descriptions are required to enable the reader to assess the study and make decisions about the relevance of outcomes to a specific clinical population (e.g., deciding whether a study is relevant to a clinician's practice). Similar guidelines to the “ISPO Recommendation for Defining Participants in Prosthetics Research” (ISPO, 2016) would benefit knowledge mobilization for CF-AFO. Also, CF-AFOs are often not well described, which is essential for study replication and applying the results in practice. This is especially relevant for custom manufactured orthoses. With many academic publishers being able to include additional documents online, detailed orthotic device descriptions should be included with publications in this area. For prefabricated devices, the model and date are essential because a CF-AFO brand name may differ annually as device characteristics evolve at the manufacturer level. For knowledge mobilization involving technology, insufficient description of the tested technology occurs often. For example, the technology’s model number may be provided but the software/firmware version that is essential to the technology’s performance is typically not provided. A consistent effort from multiple stakeholders would be required to affect change and solve these reporting deficiencies.

Conclusion

As demonstrated by the CF-AFO case study, it is apparent that typical medical research methods are insufficient to generate timely evidence to guide assistive device prescription and consumer/clinician decision making and enable effective knowledge mobilization (Johnston et al., 2009). Immediacy is a core requirement that is contrary to the time required to complete controlled research with a sufficient sample size. Overall, a series of recommendations can be considered.

First, as outlined by Ottenbacher and Hinderer (2001), single-participant studies should be considered for evidence-based practice when quick reports on new devices or practices are needed. The lack of generalizability from single-participant studies can be partially addressed by adding additional replications of the measured treatment effect, systematic replication of the program or treatment over a wider range of situations, and meta-analysis. However, consistent data collection and reporting in the field are needed to provide the appropriate information.

Second, beyond single-participant studies, the assistive device field requires a multisite, multinational clinical database approach to support data assembly, knowledge extraction, and knowledge mobilization (Fuhrer, 2001). Secure international data networks are now available for collecting de-identified information on the user, assistive device characteristics, and outcome measures (e.g., SwedeAmp quality registry [Emilsson et al., 2015]). By engaging multinational clinical, non-government, and government organizations, the infrastructure to participate efficiently in a global registry can be created, thereby providing timely data on device outcomes that can be synthesized computationally to reveal patterns, trends, and associations related to assistive-technology utilization (i.e., Big Data approaches). The obstacles to achieve such an objective include data-security requirements across jurisdictions, engagement at the clinician level to collect high-quality standardized outcome measures, engagement at the local administration level to enable de-identified data sharing, and partnerships to enable timely and relevant analysis and reporting from the system to the appropriate recipients (i.e., knowledge mobilization). Differences in clinical practices between sites is also a potential problem because the local therapy methods may influence outcomes beyond factors attributed to the assistive device.

Third, in addition to clinician-generated information, assistive-device consumers may also participate by sharing their wearable mobility device information (smart watch, smart phone, etc.) or stored data from their intelligent assistive devices (microprocessor controlled prosthesis, powered wheelchair, etc.). Industry must be engaged by multiple stakeholders (including end users, healthcare providers, professional associations, and privacy regulators) to make the transition from only using device sensor data for device control and product development to making this device-user-centric data an integral part of the evidence base.

The research process cannot keep pace with innovation, so evidence to guide prescription is lacking; however, the continuous knowledge generation era is upon us and the assistive technology field is well positioned to take advantage and bring immediacy to knowledge mobilization. For assistive devices, partnerships and action are required to “mobilize” knowledge mobilization.

 

---

References

Baljko, M., & Hamidi, F. 2014. Knowledge Co-Creation and Assistive Technology. Scholarly and Research Communication, 5(3).

Bernd, T., Van Der Pijl, D., & De Witte, L. P. 2009. Existing Models and Instruments for the Selection of Assistive Technology in Rehabilitation Practice. Scandinavian Journal of Occupational Therapy, 16(3): 146–158.
http://dx.doi.org/10.1080/11038120802449362

Condie, E., Scott, H., & Treweek, S. 2006. Lower Limb Prosthetic Outcome Measures: A Review of the Literature 1995 to 2005. Journal of Prosthetics and Orthotics, 18(6): 13–45.

Demers, L., Weiss-Lambrou, R., & Ska, B. 2002. The Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0): An Overview and Recent Progress. Technology and Disability, 14(3): 101–105.

Emilsson, L., Lindahl, B., KöSter, M., Lambe, M., & Ludvigsson, J. F. 2015. Review of 103 Swedish Healthcare Quality Registries. Journal of Internal Medicine, 277(1): 94–136.
http://dx.doi.org/10.1111/joim.12303

Fuhrer, M. J. 2001. Assistive Technology Outcomes Research: Challenges Met and Yet Unmet. American Journal of Physical Medicine & Rehabilitation, 80(7): 528–535.

Gebrosky, B., Pearlman, J., Cooper, R. A., Cooper, R., & Kelleher, A. 2013. Evaluation of Lightweight Wheelchairs Using ANSI/RESNA Testing Standards. Journal of Rehabilitation Research and Development, 50(10): 1373–1390.
http://dx.doi.org/10.1682/JRRD.2012.08.0155

ISO. 2016. ISO Standard 11.180: Aids for Disabled or Handicapped Persons. Geneva: International Organization for Standardization (ISO).
http://www.iso.org/Iso/Home/Store/catalogue_ics/catalogue_ics_browse.Htm...

ISPO. 2016. Defining Participants in Prosthetics Research. International Society for Prosthetics and Orthotics. Accessed August 11, 2016:
http://www.ispoint.org/Defining-Participants-Prosthetics-Research

Johnston, M. V., Vanderheiden, G. C., Farkas, M. D., Rogers, E. S., Summers, J. A., & Westbrook, J. D. 2009. The Challenge of Evidence in Disability and Rehabilitation Research and Practice: A Position Paper. Austin, TX: The National Center for the Dissemination of Disability Research (NCDDR).

Jutai, J., & Day, H. 2002. Psychosocial Impact of Assistive Devices Scale (PIADS). Technology and Disability, 14(3): 107–111.

Jutai, J. W., Fuhrer, M. J., Demers, L., Scherer, M. J., & DeRuyter, F. 2005. Toward a Taxonomy of Assistive Technology Device Outcomes. American Journal of Physical Medicine & Rehabilitation, 84(4): 294–302.

Lenker, J. A., & Paquet, v. L. 2003. A Review of Conceptual Models for Assistive Technology Outcomes Research and Practice. Assistive Technology, 15(1): 1–15.
http://dx.doi.org/10.1080/10400435.2003.10131885

Miller, L. A., & Swanson, S. 2009. Summary and Recommendations of the Academy's State of the Science Conference on Upper Limb Prosthetic Outcome Measures. Journal of Prosthetics and Orthotics, 21(9): 83–89.

Ottenbacher, K. J., & Hinderer, S. R. 2001. Evidence-Based Practice: Methods to Evaluate Individual Patient Improvement. American Journal of Physical Medicine & Rehabilitation, 80(10): 786–796.

RESNA. 2016. AT Standards. Rehabilitation Engineering & Assistive Technology Society of North America (RESNA). Accessed May 11, 2016:
http://www.resna.org/At-Standards

Rust, K. L., & Smith, R. O. 2005. Assistive Technology in the Measurement of Rehabilitation and Health Outcomes: A Review and Analysis of Instruments. American Journal of Physical Medicine & Rehabilitation, 84(10): 780–793.

Samuelsson, K., & Wressle, E. 2008. User Satisfaction with Mobility Assistive Devices: An Important Element in the Rehabilitation Process. Disability and Rehabilitation, 30(7): 551–558.
http://dx.doi.org/10.1080/09638280701355777

Scherer, M. J., Craddock, G., & Mackeogh, T. 2011. The Relationship of Personal Factors and Subjective Well-Being to the Use of Assistive Technology Devices. Disability and Rehabilitation, 33(10): 811–817.
http://dx.doi.org/10.3109/09638288.2010.511418

Sinitski, E. H., Lemaire, E. D., & Baddour, N. 2015. Evaluation of Motion Platform Embedded with Dual Belt Treadmill Instrumented with Two Force Plates. Journal of Rehabilitation Research and Development, 52(2): 221–234.
http://dx.doi.org/10.1682/JRRD.2013.11.0244

Tyson, S. F., Sadeghi-Demneh, E., & Nester, C. J. 2013. A Systematic Review and Meta-Analysis of the Effect of an Ankle-Foot Orthosis on Gait Biomechanics after Stroke. Clinical Rehabilitation, 27(10): 879–891.
http://dx.doi.org/10.1177/0269215513486497

WHO. 2016. International Classification of Functioning, Disability and Health (ICF). World Health Organization (WHO). Accessed May 17, 2016.
http://www.who.int/classifications/icf/en/

Wright, V. 2009. Prosthetic Outcome Measures for Use with Upper Limb Amputees: A Systematic Review of the Peer-Reviewed Literature, 1970 to 2009. Journal of Prosthetics and Orthotics, 21(9): 3–63.