Automated Scoring of Integrative Complexity from Text using Discourse Relations and Stance

Research Proposal (in progress)

Conceptual/Integrative complexity is a construct developed in political psychology and clinical psychology to measure an individual’s ability to consider different perspectives on a particular issue and reach a justifiable conclusion after consideration of said perspectives. The construct is composed of two dimensions, Differentiation and Integration. Differentiation relates to the capacity of individuals to adopt and to apply a variety of perspectives on an issue. On the other hand, Integration refers to the capacity of individuals to recognise interweaving connections and contrasts across these perspectives. Integrative complexity (IC) is usually determined from text through manual scoring, which is time-consuming, laborious and expensive. Consequently, there is a demand for automating the scoring, which could significantly reduce the time, expense and cognitive resources spent in the process. Any algorithm that could achieve the above with a reasonable accuracy could assist in the development of intervention systems for reducing the potential for aggression, systems for recruitment processes and even training personnel for improving group complexity in the corporate world. It is of outmost importance that the construct be studied and automated, not only to allow researchers in psychology to use it more freely and frequently, but also to attain a deeper understanding of its working, to allow researchers in computational linguistics and artificial intelligence to obtain a deeper understanding of how our minds consider different information sources during decision-making.

Previous work on the automation of the scoring process was implemented through a purely statistical approach (Ambili & Rasheed, 2014). The automation of scoring of Integrative Complexity is a particularly tough problem, as the problem of detection of differentiation and integration is virtually unsolved in current literature. The problem of detecting differentiation in text boils down to the detection of differentiated statements on a particular issue. The concept of differentiation makes sense to a human reader because of the ability of humans to comprehend the thesis of the issue at hand and the ‘extent of differentiation’ expressed by the subsequent/preceding statements in the text. Integration refers to the capacity of individuals to recognize interweaving connections and similarities across perspectives. Hence, when integrative complexity is low, individuals tend to form simple and rigid attitudes and perceptions and are often unable to appreciate or absorb the views of other individuals (Suedfeld, Tetlock & Streufert, 1992).

One such field that could significantly contribute to this area is the work on discourse computation and discourse relations. Discourse relations are low-level structures of discourse that connect between the semantic content of adjacent units of discourse (Webber, Egg & Kordoni, 2011). This inter-connecting semantic content is an abstract object — a proposition, a fact, an event, a situation, etc. Other features of interest that could augment the automation of the scoring process are models of stance and engagement (Hayland, 2005). Stance is usually defined as an attitudinal dimension that involves the author’s judgments, opinions and commitments, whereas Engagement is considered an alignment dimension that recognizes the implicit relationship held by the authors for their readers. Recent work on the automated classification of stance in essays could reveal insights that could be utilized in the development of features (Faulkner, 2014). The contribution of these linguistic markers in the prediction of the construct is significant and needs to be studied.

The aim of this proposal is to delineate the possibilities of the use of these features to assist in the automation of the scoring process of Integrative Complexity. I strongly believe that the resulting project would grow far beyond the basic scope of this proposal. The realization of this work could assist in building an intervention system to predict potential acts of aggression and subsequently plan interventions and resolve conflicts, since IC is also used to predict forms of aggression. Future work should amass large amounts of data to enable automated integrative complexity to be a foreseeable reality.

References

Ambili, A. K., & Rasheed, K. M. (2014, December). Automated Scoring of the Level of Integrative Complexity from Text Using Machine Learning. In Machine Learning and Applications (ICMLA), 2014 13th International Conference on (pp. 300–305). IEEE.

Faulkner, A. (2014, March). Automated Classification of Stance in Student Essays: An Approach Using Stance Target Information and the Wikipedia Link-Based Measure. In The Twenty-Seventh International Flairs Conference.

Hyland, K. (2005). Stance and engagement: A model of interaction in academic discourse. Discourse studies, 7(2), 173–192.

Suedfeld, P., Tetlock, P. E., & Streufert, S. (1992). Conceptual/integrative complexity. In C. P. Smith, J. W. Atkinson, D. C. McClelland, and J. Veroff (Eds.), Motivation and personality: Handbook of thematic content analysis, 393–400. New York: Cambridge University Press

Webber, B., Egg, M., & Kordoni, V. (2011). Discourse structure and language technology. Natural Language Engineering, 18(4), 437–490.