RAA4: Assisting Instructional Designers on the Model Driven Architecture in Technology Enhanced Learning Systems

Drira, R., Laroussi, M., Le Pallec, X., & Warin, B. (2012). Contextualizing learning scenarios according to different Learning Management Systems. Learning Technologies, IEEE Transactions On, Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6104031

 

Background: 

Technology Enhanced Learning (TEL): This paper defines TEL as a complex system formed by a set of interdependent and heterogeneous components (i.e., actors, tools, and learning objects) organized in space and time in order to satisfy a learning goal.

Learning Management Systems (LMS): This paper defines LMS as a software system that supports distance teaching and learning. “An LMS provides much relevant functionality for collaborative learning, assessment, and communication using extremely powerful tools such as forums, chats, wikis, blogs, quizzes, etc.”

This paper is in the context of instructional design in the Technology Enhanced Learning (TEL) systems. The authors state that there are many different Learning Management Systems (LMS), in order to achieve interoperability, organizations have developed Learning Technology Standards (LTS), but using standards have some drawbacks. For examples, it is too generic, and also the instructional designers must use a LTS compliant and thus cannot flexibly tailor the instructional design to the specific needs of the specific learning contexts. The instructional design thus lost the pedagogic expressiveness and contextual expressiveness.

One solution is to use the Model Drive Architecture (MDA) in the instructional design process to deal with the problem of system interoperability across different execution platforms. However, novice designers can have some technical difficulties to use this approach.

The Model Drive Architecture (MDA) approach in instructional design follow the following three steps:

1. A model of the intended system with a specific meta-model is defined. This meta-model allows an accurate description of specific needs.

2. A model transformation engine with specific rules is used to transform the preceding model into an LMS-specific model.

3. The specific model can be deployed on the LMS using an automatic generator/deployer.

Purpose of the Research: 

The focus of this paper is on the step 2 above. Novice designers usually have technical difficulties in transforming the models, and the purpose of this paper is to design a tool to help instructional designers in this process.

Methods:

The authors propose an novel approach called ACoMoD (Assistance for Contextualized Modeling of Learning Systems), and therefore developed a graphical and interactive tool called Gen-COM. The Gen-COM integrates some best practices instructional designs recommended to the designers. Then the authors did a user study on this tool among 44 instructional designers. They asked for the users’ feedbacks on the usefulness of assistance for tailoring pedagogy with technical tools, usefulness of good practice recommendations, and the usability of Gen-COM.

Main Findings:

1. Designers found that Gen-COM was useful in tailoring pedagogy with LMS tools. Designers who are skilled in model transformation emphasized that Gen-COM offers a powerful transformation mechanism.

2. Although the integration of best practices in the design process is useful for novices, it is less so for designers who are very familiar with their institutional context.

3. Gen-COM clearly separates the work space for matching pedagogy and technology from the best practice reminders.

4. Most designers state that they are more likely to use a model-driven approach with tools like Gen-COM, which hide technical difficulties while allowing them to benefit from many advantages.  These advantage include interoperability, reuse, and personalization.

Analysis: 

Some parts of this paper get a bit technical and difficult to read, and the authors use a lot of acronyms. This paper is not directly related to user experiences, but I found it implies an issue between the UX team and the developers team, that is, the UX team strives to do user-centered design, and they want the design to tailor to the specific contexts of the users. However, the developers want standards, interoperability, and reuse.  They do not want to redesign everything for new user contexts and needs, they want to use the frameworks they developed before. This might be even true for novice developers, and for a small developer team, because designing for specific contexts takes time and expertise. When the time is limited, and the development team is small and full of novice developers, the developers will feel their situations are not understood by the UX researchers. And here comes the biggest issues of communication, and imperfection in the final products. This paper is trying to fill this gap by designing a tool for the designers to reduce the technical requirement on them.

RAA3: Analyzing #insomnia on Twitter

Jamison-Powell, S., Linehan, C., Daley, L., Garbett, A., & Lawson, S. (2012). “I can’t get no sleep”: Discussing #insomnia on Twitter. Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (pp. 1501–1510). Retrieved from http://dl.acm.org/citation.cfm?id=2208612
Purpose of the Research: 
(1) To explore the role of social media in the discussion of mental health issues, and with particular reference to insomnia and sleep disorders: whether individuals are disclosing their insomnia related information online, if so, what they are disclosing.
(2) To provide analytical results to those in the interaction design community interested in integrating online social media systems in health interventions.
Methods:
A mixed-method approach is used to analyze 18,901 tweets using hashtag #insomnia. First, an automatic content analysis is used to confirm “whether individuals are disclosing their insomnia related information online”. Second, a qualitative thematic analysis is performed to answer the question “what individuals are disclosing about their insomnia”.
(1) Content analysis using a software named Linguistic Inquiry and Word Count (LIWC) . Simply put, this software contains a pre-defined English dictionary which includes categories of negative words and positive words, as well as word roots related to health, time, anger, anxiety, sadness, etc. The software performs frequency counting and statistical analysis of the input text based on this dictionary. In order to compare these #insomnia tweets with the general posts on Twitter, 17,532 random general non-specific tweets on Twitter were used as a control group. LIWC is used to compare the #insomnia tweets and the non-specific tweets, and produces table 1 below.
(2) Inductive thematic analysis. The authors acknowledge that the LIWC software analysis results maybe inaccurate because of the subjectivity and ambiguity of  informal human language, as well as the large amount of abbreviations used in tweets due to the character limit. So they perform inductive thematic analysis. A sample of 749 is taken from every 25th tweet of the original 18,901 tweets. One researcher first read the 749 tweets for several times, and comes up with 49 categories, and gives the results to two other researchers. The three of them discuss and decide on 45 categories. Then they take another sample of 514 tweets from every 35th tweet of the original 18,901 tweets, and code the 514 tweets into the 45 categories independently.  Inter-rater reliability is also examined.  All the researchers finally refine the categories into a hierarchy in figure 1.
Main Findings: 
(1) Results in table 1 below confirms that individuals do disclose insomnia related information online.
(2) Inductive thematic analysis categorize the content of #insomnia tweets into two major categories: describing the insomnia experience and coping with insomnia.
Analysis: 
The authors claim that this paper, in a generic sense, contributes to a growing body of literature regarding online self-disclosure. Based on the analysis results, they come up with a brief design requirement for a social network structure allowing insomnia therapy to be delivered in an online group. “The structure would be a useful format for users to exchange support and practical information. To allow users to exercise catharsis,  a space could be provided to disclose symptoms and frustrations together with an area where the users can “communicate with” insomnia. To comply with the principles of sleep hygiene, a digital barrier should be in place in any interactive platform, this would reduce the amount of access a user has to an online support network at times when they should be refraining from stimulating environments. However, rather than simply restrict all access a “life-line” should be kept open, providing information on good sleep hygiene and progressive muscle relaxation exercises to aid sleep. “
This paper relates to what we learned in CGT512 how to analysis qualitative data and thus produce design requirements.
This paper also relates to my research interests in social media analytics for instrumenting the social and emotional aspects of college students’ life. It provides information on useful software and methods.

RAA2: Adaptive Agents Monitor and Increase Students’ Engagement

Szafir, D., & Mutlu, B. (2012). Pay attention!: designing adaptive agents that monitor and improve user engagement. Proceedings of the SIGCHI conference on Human factors in computing systems, CHI  ’12 (pp. 11–20). New York, NY, USA: ACM. doi:10.1145/2207676.2207679
Purpose of the Research:
The authors of this paper set to answer the question: How to design computer-based educational tools that monitor student attention and employ attention-inducing strategies to improve learning in the way human teachers do?
Methods: 
The authors design a human like social robot as an instructor. This robot can monitor students’ attention level by detecting their unconscious brain signals. They use a wireless electroencephalography (EEG) headset to detect the students’ brain signals. This method is often used in Brain-Computer Interfaces (BCI). When detecting a drop in engagement level, the robot then will recapture the students’ attention level using verbal or nonverbal cues (immediacy cues).
The authors then propose two hypotheses: (1) The system can increase students’ attention, and therefor increase learning performance (specific to this experiment, increase students’ recall of narrative stories); (2) The system can increase students’ motivation and rapport to the robotic instructor. The first hypothese is developed based on the arousal-attention theory, and the second is developed according to motivational theory.
The authors then test the two hypotheses with 30 college students (15 male, 15 female).
Main Findings: 
Experiments with the 30 college students suggest that the first hypothesis is fully supported, while for the second hypothesis, only female students demonstrate increased motivation and rapport to the robot. The authors then discuss the possible reasons for the result, for example, the robot is a small stature with a child-like voice, which may have made it more difficult for males to connect with the robot than females.
Analysis:
(1) This paper has nice review of educational psychology theories related to students emotion, attention, and motivation, and shows how these theories can be used to improve computer based education systems. These are useful information for my own research in engineering education.
(2) The paper explains the difference between active Brain-Computer Interfaces (BCI) and passive BCI. In active BCI, the users have to consciously control their brain activities in order to control the computers. This usually requires extensive tuning and training on both the parts of the system and the users, thus is difficult to become generalizable. In passive BCI, the systems detect users’ conscious and unconscious brain signals, and combine these with other user inputs such as voice and gestures to determine the actions. I am thus seeing the future of user interface a combination of BCI and NUI (natural user interface), where the users use both brain activities and natural body movements, tactile, and verbal actions to interact with the computers.

RAA1: Visual Difficulties to Enhance Engagement and Learning

Hullman, J., Adar, E., & Shah, P. (2011). Benefitting InfoVis with Visual Difficulties. Visualization and Computer Graphics, IEEE Transactions on, 17(12), 2213–2222.
Purpose of the Research: 
To provide a counterpoint to efficiency-based design theory with guidelines that describe how visual difficulties can be introduced to benefit comprehension and recall.
Methods:
This is an essay-style paper. The method uses is to synthesize empirical results from cross-disciplinary research on visual information representations.
Main Findings:
The dominant visual design and evaluation guidelines are based on the cognitive efficiency model, which refraining from using distracting visual elements, irrelevant information, leveraging labeling, and graphical formats that reduce cognitive processing by the users. However, empirical studies from various sub-fields of psychology and education support that desirable visual difficulties may induce active processing and engagement of the users thus enhance deep reflection and long term recall. Visualization effectiveness is better characterized as a trade-off between efficient processing and desirable visual difficulties to stimulate learning. 
Analysis:
One analogy to the authors’ argument I can think of is watching movies vs. reading books. Just like one folk mentioned in class about the Harry Potter movies vs. books, watching movie is effortless, the users do not need to actively construct the details in their minds. However, according to the logic of the authors’ argument, reading books make the readers work, think, and thus more engaged in learning. Leaving space for the users to think and reflect is one thing, another thing is that it maybe okay to add some distracting or seemingly irrelevant information to just attract the users’ interests so they could be more engaged. A large part of this paper is actually talking about engagement, rather than visual difficulties, but the authors use the contradictory and eye-catching phrase  “visual difficulties” in the title. This paper reminds us that we have almost passed the stage of designing only for efficiency, supporting for reflection and learning need to be taken into consideration.