Friday, August 9, 2013

Assessing Complex Systems and Sonnet 73

As my own views about education continue to emerge, I understand them best within the context of the conversation about complexity—complexity as a large, transdisciplinary conversation that has been emerging for centuries, but that was made unavoidable by the emergence of relativity and quantum physics at the beginning of the 20th century. The fact that I just used a form of the term emerge three times in a single sentence suggests how much Complexity has informed my thinking. As I am so very fond of following rabbit holes, it helps me from time to time to gather my thoughts to see if something coherent emerges. See?

I've been reading through a series of articles about complexity and the limits of knowledge from a 2005 special edition of Futures. I recommend it to anyone interested in either complexity or knowledge or the knowledge of complexity or the complexity of knowledge. You can really get tangled up, or at least I can. So I want to do a bit of untangling.

At the largest scale I can think about, complexity is that zone of engagement between the open-ended future and the closed past. We call that zone of engagement the now or the present. I could refer to it as The Now and perhaps win an honorable mention in the next Eckhart Tolle book or a few minutes on Oprah, but I'm feeling sober this morning, so I'll just stick with the now. Complexity is the activity that emerges between the juxtaposition of the hot, open-ended potential of the future and the cold, fixed certainty of the past. Complexity is the result of the tension between hot and cold, or to borrow a phrase from David Foster Wallace, it is the result of the miscegenation between a hot air mass and a cold air mass. That image works for me: we exist in the thunderstorm of the now, and though we may long for the potential of the future or the certainty of the past, we cannot live in either place. Life, and by extension knowledge, cannot exist in the chaotic order of the future or the fixed order of the past, but only in the dynamic, emerging order of the now as the heat of the future slides by and is transformed into the cold of the past (I'm perfectly willing to believe that the transition from hot activity to cold fixity only gives us the illusion of movement, but the visual metaphor is appealing to me). The complexity of the now is all we get, all we have, but because the now is a complex system, it is profoundly affected by and interacts with both the future and the past. Both the future and the past inform the now, and the dynamism of the now informs both the future and past in turn.

So for me, complexity is about as big an idea as I can have—sort of a God idea, but I don't intend to talk about God in this post; rather, I want to talk about knowledge and education and what the overarching concept of complexity has to do with them. How does it inform my ideas of knowledge and education? That's the question.

In their Introduction: Complexity and Knowledge (Futures, 2005, Vol. 37, pp. 581-584), Peter Allen and Paul Torrens note that the study of open systems proved to be very problematic for scientific knowledge in both the hard and soft sciences. They say:
For isolated and closed systems classical thermodynamics gave us the knowledge to predict the transformations and final equilibrium states of a system. Obviously, for frictionless systems such as those involved in planetary motion, Newton’s Laws allowed the prediction of orbits and eclipses, both forwards and backwards in time. Knowledge was complete and related directly to prediction. But, open systems were much more problematic. (581-582)
Closed systems, it seems, function in the simple and complicated domains, as defined in the Cynefin Framework. The simple and complicated domains afford us "the knowledge to predict the transformations and final equilibrium states of a system … both forwards and backwards in time." In closed systems, we can arrive at complete knowledge with reliable—testable and verifiable—predictions. Open systems do not allow such affordances.

This is a big problem, as Allen and Torrens note. So what's wrong with open, complex systems? First, we have a boundary issue. Open systems do not have discrete boundaries. I can see this quite clearly when I try to imagine the boundary between now and the future. The boundary has a thickness. I can feel the future coming and the past slipping, sometimes quite strongly, but I can never quite put my finger on the exact line between the future and now, and as soon as I fix my finger to a line, it slips into the past (the line, not my finger, which fortunately stays with me in the now). So the boundary also has an incredible thinness. So which is it—thick or thin? Well, both, of course. The boundary is open, and the exchanges between the system inside (now, for instance) and the systems outside (future and past, for instance) modify all systems. I really am speaking universally here; thus, I include those systems within the simple and complicated domains. From my point of view, everything belongs to the complex domain, and the simple and complicated are but temporary arrangements that we form for our convenience—like a sock drawer, or a classroom. We can pretend for a moment that our classrooms belong to the simple or complicated domains, but they don't. The classroom is a complex system of complex systems, and to treat them otherwise is to risk complete misunderstanding.

The dynamic interaction at the boundaries among complex, open systems means that it is very difficult to limit ourselves to local causality. In other words, the events in any one classroom are the result of remote causes (familial, social, economic, political, etc.) just as much, sometimes more so, as local causes (say, a classroom lecture or demonstration), and we are unlikely to be able to assess exactly what caused any given behavior in our students. Nor can we predict reliably the effects of any applied intervention or instructional design. As Allen and Torrens put it:
The simplest definition of a complex system is one that can respond in more than one way to its environment. … So, ‘knowledge’ about the future trajectory of the system can be both quantitatively and qualitatively wrong. … Innovation can occur, and it may have untold implications for the future evolution of both the ‘inside’ and the ‘outside’ the system. Similarly, the same ‘intervention’ may produce two different results on what were believed to be similar systems, since a single complex system can respond to an intervention in different possible ways. The outcomes could differ qualitatively and this surely must therefore introduce some doubt into the ethical basis for the intervention. … These new ideas force us to accept a significant reduction in our powers of prediction, and even in our ability to frame a useful question.
I find myself, here, slipping into considerations about evaluation and assessment in education, and I'm reminded of the recent words by Christina Hendricks, Stephen Downes, and Keith Brennan about how to assess a MOOC. I won't go into the details of their discussion, but I will say that from my vantage point measuring a MOOC, or any other classroom, is more like measuring a thunderstorm than measuring an automobile. That being said, I think we are beginning to develop some useful metrics for measuring open, complex systems. I may need to complete one of Siemens' learning analytics MOOCs to learn what some of those metrics.

I want to add, as well, that I think we literary scholars have been confronting open, complex systems for a long time. Consider a Shakespearian sonnet—Sonnet 73 will do. Almost all the data that I can gather from traditional measurement (meter, rhyme, number of lines, number of feet, etc.) says so very little about the poem. That data can enrich my understanding and appreciation of the poem, but by itself, that data reduces the poem to a closed system, a handy sock drawer, some trivia to answer on a test, and I would never read the poem again if that's all I had. Only when I open the poem to its environment, allow it to breathe, allow it to help me make connections to grandma, winter freezes, and dying embers, to my hopes and fears, only then do I find value and meaning. I find that value and meaning difficult to measure and assess, but I'm hopeful that we are developing the tools that will help us do so. Some very interesting things are happening in the digital humanities that point this way. I'll have to read some more.

No comments: