Descriptions of Grant-Funded Projects
Assessing Patient-Provider Communication Using Computational Linguistics, Machine Learning, and LLMs
This work uses advanced computational linguistics in the analysis of secure messaging between patients and providers to enhance patient-centered outcomes. Secure messaging, a common form of communication in clinical practice, presents an opportunity to analyze rich linguistic data to improve patient decision-making effectiveness. We will leverage large language models for improved predictive analysis, integrating them with traditional machine learning methods to better identify effective deliberations in clinical exchanges. This approach aims to enhance the depth and breadth of insights from data analytics, improve the translation of comparative effectiveness research into clinical practice, and assist health systems in employing AI to support effective clinical communication.
Award Number: PCORI (Role: PI)
Interpersonal Coordination and Co-regulation During Collaborative Problem Solving
Collaborative problem solving (CPS) is an essential 21st century skill in our increasingly connected and globalized world. Yet, there is a paucity of knowledge on how to define, measure, and develop this skill, especially in the context of STEM learning. Thus, the goal of this project is to gain fundamental knowledge on how interpersonal interactions influence CPS processes and outcomes in digital STEM learning environments. The hypothesis is that CPS is fundamentally about communication and coordination among people who have thoughts, feelings, and behaviors and who both react to and influence each other’s thoughts, feelings, and behaviors. Increasing basic understanding on how these interpersonal processes emerge and influence outcomes is a critical step towards designing next-generation STEM learning environments that make CPS more enjoyable, engaging, and effective.
Award Number: NSF 1660894 (Role: PI)
Multimodal Coordination Dynamics During Conversation
When two people interact, complex patterns of behavior emerge quite spontaneously. These patterns are organized around multiple types of movement that simultaneously co-occur with little to no conscious awareness. Nevertheless, they form a stable network of associations that guide how people converge on meaning and respond to higher-level communicative goals. In this research, we take a dynamical systems approach to understand how various components of behavior, including acoustic, linguistic, and movement components, interact over time.
This focus on the interdependence of behavior can also be characterized as a focus on behavioral coordination, whereby individual’s cognitive processes and behavioral patterns are integrated into a coupled system. How this coordination changes under various high-level conditions, such as when people lie to each other, persuade each other, or attempt to understand each other, is currently being explored in the lab. To do so, we use automated motion analysis from video, motion tracking tools like the Microsoft Kinect, and high-powered infrared motion tracking cameras.
Award Number: NSF 1103356 (Role: PI)
Assessing Physician and Patients’ Linguistic Concordance
The aim in this project is to use natural language processing and machine learning techniques to quantify linguistic concordance and its prevalence among physician-patient dyads and to examine the extent to which it is associated with patients’ self-reported understanding of their physicians’ communication. The dataset involves ~5,000 ethnically diverse patients with diabetes and ~1,100 of their physicians, contributing an unprecedented number of physician-patient secure communication exchanges (>250,000). So far, we have discovered that physicians’ ability to adapt the complexity of their language to better match the health literacy of their patients is an interactive communication competency that can promote shared meaning – a proximal communication outcome in the pathway to both intermediate and distal health outcomes. This work is in collaboration with the University of California, San Francisco and Kaiser Permanente Health System.
Award Number: NIH R01 LM012355 (Role: Co-I)
Falling Under the Lens of Suspicion: Identifying Behavioral Factors that Generate Suspicion.
In every false conviction case, there came a point in the investigation when an innocent person became a suspect. Despite the countless accounts of how this presumed guilt can start a chain reaction of confirmation bias that taints all stages of the criminal justice process, little is known about what triggers suspicion about the innocent person in the first place. Drawing from existing and to-be-created databases, the overarching goal is to identify and understand what demographic, behavioral, and linguistic factors initially spark suspicion and trigger dangerous confirmation bias processes leading to false convictions—with a particular focus on one of the earliest moments in which a person might become a suspect: when they call 911. A key hypothesis is that when witnesses’ behaviors violate expectations, people “morally typecast” the witness as less capable of being a victim and more capable of being a perpetrator. Increasing basic understanding on how these expectations are formed, how they are violated, and how they interact with witnesses’ gender and race, is a critical step towards predicting whether callers are ultimately charged with the crime they are reporting. Moreover, understanding how these processes emerge will be critical in developing state-of-the-art curriculum to educate police about their expectations’ accuracy and consequences, and to educate attorneys on defending clients they believe to be innocent.
Award Number: NSF Law & Science/NSF Human Networks & Data Science 2146834 (Role: Co-PI)
Action Dynamics of False Responding and Deception
A convincing deceiver must act in discordance with their knowledge of the truth. To do so requires the deceiver to resolve competition between what is known to be true and what is intended to be false. In this line of research, we investigate the temporal signatures of this competition by examining the action dynamics of arm movement while participants respond falsely or truthfully to various sources of information. To do so, we use a simple computer mouse and Nintendo Wiimote to track a continuous stream of x,y arm movement coordinates. This information not only extends traditional approaches that have collapsed deceptive behavior into ballistic reaction time movements, but provides a more sensitive measure to detect deception in the real-world.
Award Number: NSF (Role: PI)
Other Projects
The Dynamics of Perspective-taking
This research explores how people represent and comprehend another’s perspective in social communication. This line of inquiry has contributed to many insights in the areas of pragmatics, embodiment, and social cognition. We are attempting to extend this research in two major ways. The first is to interpret perspective-taking behavior as a nonlinear dynamical process. Here we argue that the fundamental dynamical properties that exist throughout nature also exist at the level of high-level, human cognition. The second way is to use action dynamics techniques, as in capturing arm movement trajectories, to better understand the factors and cognitive processes that underlie egocentric and allocentric (“other-centric”) response choices.
Previous Projects
Language and Deception
Even amongst the most talented of deceivers, their ability to conceal verbal lies can be compromised by subtle linguistic features that have been shown to reveal hidden cognitive states. These features include the literal meaning of the words used, as well on how words are arranged and structured in discourse. For this project, we take advantage of advances in technology and linguistic theory to analyze hundreds of features of language using natural language processing tools, ranging from word information variables to sentence and discourse level variables. Thus, language serves as a window into cognition, addressing a diverse set of interrelated questions, including a) What are the qualitative and quantitative linguistic features of deception, b) How does the relationship between language and cognition vary from individual monologue to conversational deception?, and c) Are there consistent patterns of language that cut across context?
Dynamic Psycholinguistics
One of my primary interests has always been in the area of psycholinguistics. Specifically, I am most interested in how nonlinear dynamical systems and cognitive dynamics can offer new insights into the study of human language. Both approaches are unique strands of research, but are nevertheless closely related. In dynamical systems, the general thesis is that language operates as a self-organizing system, and thus signatures of nonlinear dynamics should be evident in language representation and use. In cognitive dynamics, the emphasis is on the covariance of cognition and action as it unfolds in real-time. Both strands offer powerful methodological tools, including arm trajectory analysis, cross-recurrence analysis, 1/f analysis, and neural networks. In collaboration with, and using techniques pioneered by Dr. Rick Dale, I explore the psychological study of language as a dynamic psycholinguistics.
Discourse Analysis for Educational Practices
With Drs. Danielle McNamara and Phil McCarthy, I have been involved in several projects (and papers) that are aimed at better understanding reading and writing comprehension through discourse analysis. In addition to gaining basic insights on cognitive processes, the ultimate goal is to apply our findings to the design of educational technologies.
Distributional Statistics / Semantic Models
Much of my early undergraduate training was on assessing high dimensional semantic models like HAL (Hyperspace Analogue to Language) and LSA (Latent Semantic Models). I have not lost any of my early appreciation for their explanatory power.