Saturday, March 10, 2012

Human factors research and practice - Part 2: bridging the gap


This article by Steve Shorrock and Amy Chung was published in The Ergonomist (newsletter of the Institute of Ergonomics and Human Factors), March 2010. It is reproduced here to give more open access and to encourage further conversation on this issue.  Part 1 is here. The full article in the journal Ergonomics is at http://www.tandfonline.com/doi/abs/10.1080/00140139.2011.568636 (ask me for a preprint).

In the last article we outlined some of the quantitative findings from our survey of 587 practitioners regarding the application of ergonomics and human factors (E/HF) research. We concluded that the research practice gaps could be reduced by addressing a number of factors associated with the research, the organisation, and individual practitioners and researchers. In this Part 2, we report on the respondents’ top suggestions for how practitioners and researchers could improve research application. Respondents made 733 suggestions for practitioners and 781 suggestions for researchers. Below, we identify the top 5 suggestions for practitioners and researchers.

How practitioners can improve research application
  1. Increase collaboration and communication between E/HF researchers and/or practitioners and increase contact, conferences, networking and association. Many of the suggestions concerned the relevance of the research focus to the ‘real world’. Typical comments included: “need to communicate real-world problems to researchers so that solutions can be researched”, “communicate the voids in research (as applied to the real world), perhaps in Letters to the Editors”. Other suggestions concerned a more collaborative working relationship, with suggestions such as “practitioners in research laboratories” and “researchers on the project team to provide strategic oversight”. One suggested “links with a research group that share your basic approach and philosophies”, while another suggested that practitioners “consult researchers using helpdesks”. Finally in this category were suggestions for more crossover workshops, dissemination clubs, and forums for discussion of current research.
  2. Read more and be more aware of the research, including more time to read research at work. Many respondents suggested that time to search literature should be built into projects. But several commercial conflicts were noted: “it’s hard to find the time if you work in consultancy; time is money and so if you commit to searching out and then reading papers they had better be good and useful (and a lot are not)!” Some respondents suggested seeking articles for a specific task/ job, but others hinted that this did not meet their needs (e.g. “figure out an easier way to monitor and filter research as it is published instead of searching for it when specifically required”). Some respondents suggested linking reading to personal development planning. 
  3. Promote or obtain better awareness, support, cooperation, permission and authority for research application from decision makers, stakeholders, colleagues, organisation, Societies, other disciplines/ professions and public. One respondent suggested that “practitioners need to educate their customers and ‘translate’ research into everyday English for their customers to aid buy-in of its application”. Another noted that “more industry applications should be based on specific literature; clients should start looking for more than just a guru that says he knows what to do and should insist on evidence”. Other suggestions concerned allied professions (“feed HF into the tools and techniques other technical specialists use and own; translate our discipline into their language!”), and users (“make sure you have a strong link with operational people”). 
  4. Provide more appropriate application of research findings to real problems. Several respondents argued for an evidence based approach (“seek evidence for their interventions – don’t just follow fads”). Another set of suggestions promoted more publishing of application (“provide real world application results in published work”). Other respondents suggested more collaborative application, such as “a mentor coaching relationship with researchers to help find solutions and test ideas”, or “researchers involved in the application environment.” However, several comments indicated that the horse should come before the cart. One respondent warned that practitioners should “address real world problems rather than invented abstractions.” Another stated they should “identify the problem, then identify the solution. Don’t go looking for places to make the research fit.” 
  5. Carry out more research and publication, including replicating or extending research, addressing conflicting results and methodologies and specific themes or topics of research. Several of these emphasised relevance (“write more about ‘real/applied’ problems in practitioner/professional literature, so that researchers can keep up to date with current applied challenges.”) Many of the respondents suggested that practitioners publish case studies showing successful application. However, several highlighted the lack of outlets to do this (“this is a huge problem as practitioners don’t have a venue to do this”).
How researchers can improve research application
  1. Ensure the research focus and methodology is more applied, real-world, relevant, germane, practical and generalisable to organisational environment. Many comments concerned the research focus: “focus research efforts on immediate HF problems relevant to industry”; “understand the needs of practitioners in real industrial environments and aim research at solving real problems”. Respondents frequently mentioned the external validity of methodology, especially the need for more field studies and more representative participants (“more in-situ studies; there is a difference in how people behave at work and in a laboratory”; “stop using simulators as the only means of data gathering”). 
  2. Increase collaboration and communication between E/HF researchers and/or practitioners; increased contact, conferences, networking and association. Many of these had a dual focus on realism (“listen to industry about what research is required instead of deciding what is interesting!”; “by asking practitioners where they think the gaps are in research to back up their practice”). Others focussed on the need to collaborate with regard to outputs (“engage with industry working groups to tailor publications that educate without patronising, i.e. understand that the applied world is not the research world”; “researchers should work together with practitioners on ergonomics projects and stop sitting in their offices publishing loads of theoretical papers”). 
  3. Form more definitive conclusions, and clearer implications, recommendations or applications when reporting. These suggestions were more focussed on the need to provide implications (“describe the outcomes in terms of tangible implications for practitioners”; “provide pragmatic recommendations, guidelines or applications when summarising their research findings”). One focused on the need for “better abstracts” and “clear articulation of ‘so what’ in human language”. Other respondents lamented “silly one-paragraph impact statements” and conclusions indicating that “further research is necessary”. 
  4. Report text and statistical analyses in a different format (e.g. web, magazines, practitioner journals, society publications, summaries, database, email alerts) or in native language. These respondents suggested such alternative non-journal media should be used for “snippets” or “practitioner-friendly versions”. Coupled with the barrier that many practitioners don’t have time to read academic articles, such alterative formats may help uptake of research application if presented “in a form that industry can use”. 
  5. Report text and statistical analyses in a more understandable, clear and readable manner in journal articles. Respondents emphasised the need for writing “in plain English that can be understood outside of their industry” and “dropping the academic jargon”. One responded starkly that academics should “get out of their ivory towers and speak a language that HF people and non-HF people can understand instead of psychobabble”. Another advised researchers to practise what they preach: “Write to your audience, apply the principles of HF to the writing of the article.”

Research and practice are sometimes painted as cosier bedfellows than appears to be the reality in E/HF. This work has highlighted the gaps and bridges between research and practice, but it is up to us to build and cross the bridges. We hope that the data derived from this study may spur motivation for more applicable research and more research application.
This article reported just the top suggestions for improved research application. What more can researchers do to improve application? What can practitioners do to turn theory or research findings into real world application? 

Human factors research and practice - Part 1: surveying the gap

This article by Steve Shorrock and Amy Chung was published in The Ergonomist (newsletter of the Institute of Ergonomics and Human Factors), February 2010. It is reproduced here to give more open access and to encourage further conversation on this issue. The full article in the journal Ergonomics is at http://www.tandfonline.com/doi/abs/10.1080/00140139.2011.568636 (ask me for a preprint).

As a “scientific discipline concerned with the understanding of interactions among humans and other elements of a system”, and a profession that “applies theory, principles, data and methods to design” (according to the International Ergonomics Association), ergonomics and human factors (E/HF) practice should arguably be built upon a solid research base. But do practitioners actually apply research? Do they find it useful and usable? Despite a historical schism between researchers and practitioners, such questions have, perhaps unsurprisingly, been almost completely overlooked in the literature. Yet they are questions that have profound implications for the discipline and profession.

Photo credit: futureshape http://flic.kr/p/S2VLM CC BY 2.0
A widening gap between research and practice has been observed in many disciplines, and especially concerns research published in academic journals. Researchers have found that, while academics place a high value on research, practitioners often do not. Practitioners are often largely unaware of relevant research, or do not implement research findings, or lament that the field is not practical or relevant enough. Understanding whether such a gap exists in E/HF is relevant to all researchers and practitioners. So we conducted a study to examine E/HF practitioners’ application of research findings published in scientific journals, in terms of levels of readership, usefulness, and barriers and facilitators to application.

A questionnaire was developed comprising questions about the practitioner, their access and attitude to published research, and possible barriers and facilitators to using research findings in practice. The questionnaire was targeted at practitioners who apply an understanding of E/HF to the design of interactions in real settings. We wrote to IEA federated E/HF societies to request that they ask for potential respondents. We also advertised the survey on E/HF LinkedIn groups, and emailed LinkedIn members and other professional contacts.

A total of 587 usable questionnaires were submitted from respondents working in 46countries. The top five countries in which respondents worked were US, UK, Australia, Netherlands and Canada, which also approximately reflected national ergonomics society membership. The respondents represented a wide range of organisational types, application areas, qualifications and industries. Respondents’ averaged estimated work time over the most recent 12 months as an active E/HF professional was split fairly evenly between research, application and education.

Respondents read an average of one scientific journal article per month concerning E/HF or related fields over their most recent12 months as an active E/HF professional. The three journals most frequently named as most useful to respondents as practitioners were Applied Ergonomics, Human Factors and Ergonomics. These were also the three most highly rated of the 13 IEA-endorsed journals (see www.tinyurl.com/IEAjournals), and only these journals, plus International Journal of Human-Computer Interaction and International Journal of Industrial Ergonomics were marked as having been read by 50% or more respondents.

Respondents rated E/HF or related scientific journals overall as moderately useful to them as an E/HF practitioner. Statistical analyses showed that reading frequency and overall journal usefulness were significantly higher for:
  • respondents who worked in academic/research institutions, against other organisations; 
  • society members, against non-members; 
  • respondents who spent less than half their time applying E/HF, against those that spent more than half; 
  • respondents who spent more than half their time engaged in research, against those that spent less than half; and 
  • respondents who spent more than half their time engaged in research, against those who spent more than half their time on application. There were no meaningful correlations between years of professional experience in E/HF and reading frequency, or between experience and overall journal usefulness.
The top five barriers to research application for respondents working in academic/research institutions were:
  • “Decision makers won’t allow application of research” 
  • “The research has methodological inadequacies” 
  • “Key stakeholders won’t cooperate with application of research findings” 
  • “I don’t feel I have enough authority to change current organisational processes” 
  • “There’s insufficient time on the job to apply new ideas” 
The top five barriers to research application for respondents working in other organisations were:
  • “The relevant literature isn’t compiled in one place”
  • “Implications for practice aren’t made clear in the article”
  • “I don’t have time to read research”
  • “The research isn’t relevant to my practice”
  • “Journal articles aren’t readily available/accessible”
Four of the five top barriers for those in academic/research organisations were related to characteristics of the organisation, with only one related to the research. However, for non-academics the situation is reversed; four of the top five barriers were related to research, with only one related to the organisation. 

The findings of the survey provide valuable insights into E/HF practitioners’ reported access to, and attitudes toward, research published in journals, and factors that hinder research application. The dominance of barriers related to the research itself for practitioners may parallel the drift from pragmatic to pedantic science in industrial, work and organisational psychology. The secondary organisational barriers may reflect the pressures in organisations to produce rapid results, contributing to a shift from pragmatic to popularist science in practice. 

This research highlighted a number of gaps between research and practice associated with organisational affiliation, research/application involvement, experience, and ergonomics society membership. Those practitioners with the most power to effect change in organisations may, ironically, be benefiting least from E/HF research.

Further research is underway to examine the issues raised in more depth. However, the research-practice gaps could be reduced by addressing a number of factors associated with the research, the organisation, and individual practitioners and researchers. In Part 2, we report on the respondents’ suggestions for ‘bridging the gap’. We hope that this work may encourage a conversation within our discipline about the research-practice relationship.

So, what are the barriers to research application for you? How do you find and access useful and applicable research among the hundreds of HF articles published each month? What value does published human factors/ergonomics research have for humanistic practitioners? What others genres of research might be beneficial to a humanistic approach of human factors?