Computer Supported Design Tools Distributed Communication Tools Design Knowledge

AMIA Annu Symp Proc. 2003; 2003: 559–563.

Designing a Knowledge Management System for Distributed Activities: A Human Centered Approach

This article has been cited by other articles in PMC.

Abstract

In this study we use the principles of distributed cognition and the methodology of human-centered distributed information design to analyze a complex distributed human-computer system, identify its problems, and generate design requirements and implementation specifications of a replacement prototype for effective organizational memory and knowledge management. We argue that a distributed human-computer information system has unique properties, structures and processes that are best described in the language of distributed cognition. Distributed cognition provides researchers a richer theoretical understanding of human-computer interactions and enables researchers to capture the phenomenon that emerges in social interactions as well as the interactions between people and structures in their environment.

INTRODUCTION

Distributed cognition provides a unique framework to describe the interactions, processes, and knowledge structures critical to the design of distributed information systems. This paper will describe the process of how applying a methodology, grounded in the principles of distributed cognition, generated the products necessary for the design and implementation of a human-centered distributed information system to address the organizational memory and knowledge management needs of a complex, distributed environment at Mission Control Center (MCC), NASA Johnson Space Center, Houston, Texas. We will focus on complex social, organizational and cognitive issues involved in designing information technologies within a distributed, collaborative environment

Designing information systems infrastructures for the capture of organizational memory and the distribution of this knowledge across an organization requires not only an in-depth understanding of the numerous technical knowledge management activities, but also, more importantly and often omitted, an understanding and inclusion of the social, cultural, organizational and cognitive aspects that occur not only within an individual or group of individuals but also occur across individuals and artificial agents.

Typically, information systems are designed according to traditional engineering and technology principles. Although these principles have provided a sound, technical basis for understanding the fundamental process of information management and retrieval, it does not address the unique social, organizational and cognitive issues inherent within a distributed work environment.

The study of human-computer interaction (HCI) has made significant contributions to the designing of information systems, however, the primary focus has been on the design of the interfaces between systems and users and not the deeper structures that are fundamental for the design of truly human-centered systems [¹]. In the mid-to-late 80's, research in the field of Computer Supported Cooperative Work (CSCW) produced many interesting findings about the considerations required in the design of computer systems to support group communication and collaboration. While CSCW's predominate research agenda tended to focus on social and organizational concerns, which is appropriate for the study of social interactions of people working together with computer systems, critical cognitive aspects were omitted in these analyses [²].

There is a need within the field of CSCW to develop a methodology that provides systematic principles, guidelines and procedures for designing human-centered computing systems. For many years, researchers have used cognitive, social, organizational or anthropological approaches to study both individual and collective work activities. Each approach, having their own theoretical and methodological framework, has attempted to analyze and explain characterizations of work activities in order to possess the necessary prerequisites for the design and use of technological artifacts. These frameworks usually focused on how an individual is the unit of analysis and performs a certain task within a given period of time thus freeing the task from the context. Information is collected on how to perform a task most efficiently, however, neither the task nor the individual are isolated from the environment's surroundings. Frameworks which have been developed separately from the various disciplines "do not present an adequate means of studying the dynamics of collaborative activity in situ" [²].

Distributed cognition offers a new framework to describe, represent, and study organizational memory and knowledge management. It looks for cognitive processes, wherever they may occur, on the basis of the functional relationship of elements that participate in this process [³]. The core unit of analysis is the functional system which is composed of human and artificial agents and their relations which are distributed across time and space dimensions [⁴ ^, ⁵ ^, ⁶ ^, ⁷].

According to the distributed cognition approach, cognition can be distributed across members of a group, across internal and external representations, and across space and time. Distributed cognition can lead to interesting discoveries concerning how interactions between agents are coordinated and how artifacts and tools are really used. It also has useful application in determining which features of the activities or artifacts are relevant for the efficiency of task performance and which are necessary for the activity to continue to perform well.

DOMAIN

The task domain for the current study is the Biomedical Engineer (BME) console at the Mission Control Center at NASA Johnson Space Center. In this domain, Flight Surgeons (FS) have the primary authority for the health and safety of all crewmembers assigned to a particular expedition; Console Bio medical Engineers (BME) are responsible for providing the technical and operational support for medical operations activities; and Liaisons are responsible for tracking and working on issues, and helping to reduce the interruptions to the Console BME.

The process of providing and maintaining support for medical operations for human space flight requires complex interactions and exchanges of information both synchronously and asynchronously and across space and time. It involves acquisition, transmission, distribution, integration, retrieval and archiving of large amounts of data stored within disparate systems in a variety of formats [⁸]. Personnel involved in this information flow process typically function in a multi-tasking, interruption-oriented workplace with access to unwieldy information sharing tools. Sharing and communication of individual and group knowledge are essential components of this process and become even more critical during collaborative group problem solving. The very nature of this environment makes the task of knowledge management daunting.

METHODS

Data Collection

Video Observation.

The intense, time-pressure, data-intensive nature of the BME's shift handover was chosen for a live two-hour observation. The observation was recorded using a video camera. After conducting the live observation, the tapes were repeatedly reviewed by several of the researchers over a two month period. MacSHAPA [⁹], a Macintosh based Exploratory Sequential Data Analysis software application, was used to perform both qualitative and quantitative analyses. The tape content was initially chunked into broad activities and was then further decomposed into smaller, defined basic units of activities. Focusing on the information flow and the components of the activities, the data was then coded. Various analyses were performed and information on durations, transitions, timelines, content analysis, cycles, lags, comparisons were obtained.

Document review.

Documents which are routinely used and accessible by the BME's were reviewed by the researchers. These documents were available in paper and electronic formats. The researchers conducted various information retrieving searches in the electronic documents

Interviews.

Cumulatively ten face-to-face, telephone and email interviews were conducted with the Console BME's, Lead Expedition BME's, BME Liaisons and other NASA personnel associated with this domain.

Information collected by these various methods was used for conducting distributed functional, representational, user and task analyses [¹] In addition, this information was used to support the results from an earlier distributed user and task analysis of this domain conducted by one of the researchers earlier in the study [⁸]. Two methodologies commonly employed in distributed functional analysis, work domain analysis (which focuses on the structures of a work domain) and cognitive work analysis (which focuses on cognitive activities in the work domain) were also performed.

Design Methodology

The methodology of Human Centered Distributed Information Design (HCDID) [¹] was used to guide the data collection, data analysis, and subsequent design and evaluation of a human-centered distributed information system to address the organizational memory and knowledge management needs of a complex, distributed environment prototype. Based on distributed cognition, it considers human-centered computing not just at the levels of representations but also at the levels of users, functions and tasks.

RESULTS

Observation and Analyses

During the two hour video observation, the Console BME's worked through multiple issues with various priority levels. Observed events related to one specific issue with six related sub-issues will be highlighted for discussion throughout this section. The main characteristics of this issue are:

Console BME's thought that a similar problem had previously occurred but they were uncertain about the nature of the problem, who, how or if the problem had ever been resolved.
Minimal attempts by the Console BME's to search intranet documents for information.
Search of hard copy manuals did not produce desired results and retrieval of information was labor intensive.
Time that the Console BME's allotted to this retrieval task was contingent upon the priority of the request and whether they were operating in a routine or emergency mode.
Problem solving was dependent upon receiving and relaying information between and across BME, Flight Surgeon and other related domains.
Requests for standard, routine information resulted in numerous phone call/voice loop interactions with the BME's which then generated additional activities for the BME's.
Console BME's encountered numerous phone or voice loop interruptions which caused the BME's not to return to the original activity the majority of the time.

Communication Flow and Information Exchange

BME Log Notes.

Information obtained from the BME log notes revealed that during a 24 hour period, the Console BME's interacted with 12 different personnel/departments and had a total of 21 phone call/voice loop interactions regarding sub-issue #3. (see Figures 1 and 2).

An external file that holds a picture, illustration, etc. Object name is 114f1.jpg

Communication flow sub-issue #3 during a 24 hour period.

An external file that holds a picture, illustration, etc. Object name is 114f2.jpg

Information exchange sub-issue #3 during 24 hour period.

Application of the HCDID methodology to the collection of data produced information that provided the researchers with the knowledge of a deeper understanding of the dynamic social interactions and cognitive processes which occur within, among and across the BME's and their environment. This key knowledge provided the foundation which allowed the researchers to begin the design process of the distributed information system prototype.

DESIGN CONSIDERATIONS

Information gathered from the data collection and analyses exposed many organizational memory and knowledge management issues. These issues were classified into four categories: organizational memory, problem solving, communication, and work environment (Table 1).

Table 1

Identified issues in organizational memory and knowledge management

Organizational memory	Problem solving	Communication	Work Environment
Informal knowledge not adequately captured Formal knowledge not framed within context Limited successful searchability of information	Interaction and exchange of information across multiple agents distributed across time and space dimensions Contingent upon receiving information from multiple sources Multiple sub-issues nested within a main issue Repeated problem solving for routine tasks	Information relayed across various mediums Minimal information sharing with a significant number of the group uniformed Most frequent communication pattern is one-to-one to a large distributed group	Numerous interactions requiring a high-level of multitasking Highly interruption-oriented work environment

The application of the HCDID methodology highlighted the complex interdependencies between human and artificial agents that occur within this distributed, collaborative environment. The identification of these issues was the unique contribution of distributed cognition principles and the HCDID methodology. Identification of these issues allowed the researchers to then state the design requirements and implementation specifications necessary for the information systems prototype. This analytic methodology allowed the researchers to understand and describe this environment at the system level rather than at the individual level.

DESIGN REQUIREMENTS AND IMPLEMENTATION SPECIFICATIONS

While the basic structure and function of the BME console may be described as a distributed and collaborative environment, the stressful, highly interruptive, event driven nature of this environment does not foster a user friendly, efficient means to capture, distribute and retrieve critical information for group problem solving.

Data from the various distributed cognitive analyses identified the need to create a collaborative, asynchronous, and spatially distributed workspace to capture both formal and informal organizational knowledge in the BME domain.

The organizational memory and knowledge management issues shown in Table 1 provided the basis for identifying the necessary requirements for the design and the specifications for implementation. The design requirements and implementation specifications are shown in Table 2.

Table 2

The process from data analysis and issue identification to the generation of design requirements and implementation specifications

Observational Data Analysis Results	Organizational Memory and Knowledge Management Issue Identifications	Information Systems Prototype Design Requirements	Implementation Specifications
Thought similar problem had previously occurred; uncertain about the nature of the problem, and who, how or if the problem was resolved Minimal attempts to search intranet documents for information Search of hard copy manuals did not produce expected results Retrieval of information was labor intensive, contingent current BME alert mode Problem solving was dependent upon receiving and relaying information between and across BME and other domains Requests for standard, routine information resulted in numerous phone call/voice loop interactions over a 24 hour period Phone or voice loop interruptions caused BME's not to return to the original activity the majority of the time	Informal knowledge not adequately captured Formal knowledge not framed within context Limited successful searchability of information Interaction and exchange of information occurs across multiple agents, multiple mediums distributed across time and space dimensions Problem solving contingent upon receiving information from multiple sources Multiple sub-issues nested within a main issue Repeated problem solving for routine tasks Minimal information sharing with significant number of group uninformed One-to-one to large distributed group communication pattern Numerous interactions requiring a high-level of multitasking Highly interruption-oriented work environment	Provide a means for collaborative communication Capture informal know ledge Organize knowledge as searchable data Frame formal know ledge within context Increase search and retrieval capabilities Increase information sharing across groups Minimize repeated problem solving with routine tasks Decrease interruptions Redirect one-to-one to group communication patterns	Issue specific workspace to collect items related to an issue which is accessible in one place to multiple agents Workspaces that hold files, links, actions permanently Group access at different security levels to files, links, etc Asynchronous communication to decrease interruptions & increase collaboration Navigation that puts users in reasonable places after completing item Keep navigation to a minimum of clicks Create fields with drop-down menus or other terminologically sound methods for entering data Data organized into logical structures (concept map) Search capability that allows specific search criteria Status and task logs to capture running "at a glance" information as well as more informal information

DISCUSSION

In this study we demonstrated a methodology rooted in a distributed cognition perspective to generate the products necessary for the design and implementation of human-centered distributed information systems.

Unlike the traditional approaches to cognitive task analyses, the HCDID methodology allowed for the identification and close examination of how interactions across humans and artificial agents were coordinated and which artifacts were used for each interaction. This aided the researchers in uncovering phenomenon that might not have been seen with traditional HCI methodologies.

Examining data from multiple sources obtained from a complex, distributed work environment and a methodology grounded in the principles of distributed cognition provided the researchers an in-depth understanding of the domain and revealed some interesting findings about the organizational memory and knowledge management of the BME domain.

The BME domain and its information systems may be described as a highly complex, asynchronous environment distributed across several time and space dimensions. The HCDID methodology was particularly useful in capturing the social cognitive nature of the BME domain. The individuals in this domain each possess different types of knowledge and routinely engage in problem-solving collaboration, necessitating the pooling and sharing of information across various media.

Distributed cognition principles and the HCDID methodology provides a unique language to capture and describe the cognitive phenomenon distributed between and across human and artificial agents.

CONCLUSIONS

This study demonstrates how the HCDID methodology can provide researchers with a richer theoretical understanding of human-computer interactions and enable researchers to capture the phenomenon that emerges in complex, social interactions as well as the interactions between people and structures in their environment.

A distributed human-computer information system such as the BME console has unique properties, structures and processes that are best described in the language of distributed cognition. These properties, structures and processes determine the performance level of the distributed system. The HCDID methodology guides the design of these properties, structures and processes to maximize the performance level of the distributed system.

Acknowledgments

This study was supported by a grant from the Human-Centered Computing and Intelligent Systems Program, National Aeronautics and Space Administration, Grant No. NCC 2-1234.

REFERENCES

1. Zhang J, Patel VL, Johnson KA, Malin J, Smith JW. Designing human-centered distributed information systems. IEEE Intelligent Systems. 2002;17(5):42–47. [Google Scholar]

2. Rogers R, Ellis J. Distributed Cognition: an alternative framework for analyzing and explaining collaborative working. Journal of Information Technology, 1994. 1994;9(2):119–128. [Google Scholar]

3. Hollan, J.D., Hutchins, E., Kirsh, D. (1999). Distributed cognition: A new theoretical foundation for human-computer interaction research. ACM Transactions on Human-Computer Interaction

4. Hutchins, E. (1995a). Cognition in the wild Cambridge, MA: MIT Press.

5. Wright PC, Fields RE, Harrison MD. Analyzing human-computer interaction as distributed cognition: The resources model. Human-Computer Interaction. 2000;15(1–41) [Google Scholar]

6. Zhang, J. A distributed representation approach to group problem solving. Journal of American Society of Information Science. 1998;49(9):801–809. [Google Scholar]

7. Flor, N. V., & Hutchins, E. L. (1992). Analyzing distributed cognition in software teams: A case study of team programming during perfective software maintenance. In J. Joenemann-Belliveau & T. G. Moher & S. P. Robertson (Eds.), Empirical Studies of programmers: Ablex Publishing.

8. Johnson, K.A., Shek, M. & McGinnis, P. (2000). User and task analysis of the flight surgeon console at the mission control center of the NASA Johnson Space Center. Unpublished paper.

9. Sanderson PM, Scott JJP, Johnston T, Mainzer J, Watanabe LM, James JM. MacSHAPA and the enterprise of Exploratory Sequential Data Analysis (ESDA) International Journal of Human-Computer Studies. 1994;41:633–68. [Google Scholar]