Chapter 15: The Open Learning Initiative: Enacting Instruction Online

Abstract

A chapter from Game Changers

Ross Strader and Candace Thille

This chapter is licensed under a Creative Commons Attribution 3.0 Unported License

New technologies are often used to replicate current systems, without much thought given to how the affordances of the technology can help design a better system. Higher education has been particularly guilty of this lack of imagination. Since the days of "distance education" delivered over closed-circuit television, we have too often sought to use technology simply to replicate the traditional lecture-based classroom model. Technology has brought about significant change in many sectors of our economy, yet the primary delivery system for knowledge in our country has largely remained unchanged.

Why is this? Is it that the traditional lecture-based model works so well that we have no need to look for anything better, and that we are best served by using technology merely to replicate and augment this system that has been in place for hundreds of years? Our process for higher education worked well in the context for which it was constructed—when we could safely assume that we were teaching small classes of students with fairly homogeneous background knowledge, relevant skills, and future goals. However, that context has changed. We now teach vastly larger numbers of students who have a much greater diversity of background knowledge, relevant skills, and future goals.

Technology has clearly provided us with some benefits. Students today do not always have to be physically present in the classroom—instead, we use technology to provide them with "anytime, anywhere" access to video-recorded lectures, electronic textbooks, or audio-based podcasts. We set up online discussion forums so that our students can communicate and collaborate more easily and efficiently. Because of technology, we now have the ability to create elaborate computer simulations of phenomena that are too large or too small to physically observe. However, at the core of all of these uses of technology is still the same underlying model: the primary mode of knowledge transfer is that of a student sitting and listening to an instructor giving a lecture.

Limitations of Traditional Instruction

Problem No. 1: Many Instructors Teach to a Certain Percentile of the Class

As the diversity of the student population increases, instructors are forced to make increasingly tough choices about the level at which their teaching is targeted. Some instructors teach to the top students in the class, and most of the students in the class struggle to keep up. Others strive to ensure that the needs of every student are met, which unfortunately can result in an uninspiring, even tiresome experience for many of the students in the class. Others seek to hit a middle ground, which can result in both of these problems—half the class is lost and half the class is bored.

Problem No. 2: Students Frequently Do Not Receive Immediate Feedback Crucial to the Learning Process

In order to learn the material, most students listen to instruction in the classroom and read textbooks. To demonstrate their level of competence, they turn in homework and take quizzes. By the time they receive feedback on their work, the chance to correct any misunderstandings or reinforce correct responses has often passed. The main reason that human tutors can be so effective in working with students is that they are able to provide immediate, targeted feedback at the right points in the learning process.¹

Problem No. 3: In All but the Smallest Classes, the Student's Knowledge State Is a Black Box to the Instructor

Instructors might have brief glimpses into this black box through homework and quizzes, but again, this information typically arrives too late to be of use while the instruction is being given—the point at which it would be most valuable. Depending on the instructor's ability to interpret students' facial expressions, he or she may have little or no understanding of how well students are grasping the concepts being presented. In large lecture halls, the instructor is deprived of the benefit of even this level of feedback beyond the first few rows.

Problem No. 4: Degrees Favor Time Spent in a Classroom over Demonstration of Competency

Students come to our colleges and universities with a wide range of backgrounds and abilities. Yet we force them into a one-size-fits-all four-year plan toward a degree, with very little flexibility. In addition, degrees do not so much certify that the student has mastered a given set of competencies and is now proficient in a particular field as they certify that he or she was in the right place at the right time for four years running.

Problem No. 5: There Is Great Inefficiency in Creating Instruction within Higher Education

By and large, instructors create their own course materials for their own lectures. Knowledge gained by an experienced instructor about how best to teach the material is typically lost when that instructor retires or moves on and a new instructor begins the cycle again. This model is not as problematic in specialized graduate courses where faculty can bring their expertise on a given topic to bear in a way that few others could. However, it is terribly wasteful in large undergraduate courses, where there are thousands of instructors across the country—all developing what are essentially the same materials year after year.

Unfortunately, most of these limitations persist even with the use of new technologies. The Open Learning Initiative (OLI) was created in 2002 with a grant from the William and Flora Hewlett Foundation to address these challenges: Rather than simply moving artifacts of the traditional classroom-based model to the web, how can we use technology to enact instruction in this new online environment, and what benefits can we derive from this new use of technology?

The OLI Approach

To answer these questions, OLI put together a team of content experts, learning scientists, human-computer interaction experts, and software engineers. We started with one of the most widely used undergraduate courses: Introductory Statistics. We took advantage of work that had been done in the field to identify a common set of learning outcomes that students should be able to achieve after taking the course. We set out to create an online course environment that would bring to bear not only the affordances of the new technologies, but also everything that the state of the science had to tell us about human learning.

The collaboration among this diverse group of experts led us to a method of instruction that is repeated throughout an OLI course for each concept. Instruction on every concept starts with one or more student-centered, observable learning objectives. We then present expository content in the form of text, images, simulations, short (3–5 minutes) videos, and worked examples where appropriate (Figure 1).

Interspersed with the exposition are interactive tasks that support students to engage in authentic practice with the concepts and skills they are learning. The tasks are presented in a supported environment with hints available to the students if they are struggling. They receive feedback that reinforces correct responses and targets common student misconceptions (Figure 2).

Finally, we offer students a chance to do a quick self-assessment and reflect on what they have learned, so that they can decide whether they should move on or whether they need additional practice (Figure 3).

A key attribute of the OLI environment is that while students are working through the course, we are collecting analytics data and using those data to drive multiple feedback loops (Figure 4):

• Feedback to students: We provide the student with timely and targeted support throughout the learning process. This support is in the form of corrections, suggestions, and cues that are tailored to the individual's current performance and that encourage revision and refinement.
• Feedback to instructors: The richness of the data we are collecting about student use and learning provides an unprecedented opportunity to give instructors a clear picture of the student's current knowledge state. As a result, instructors are able to spend less classroom time lecturing and more time interacting with students in ways that take advantage of the instructor's unique expertise and interests targeted to student needs.
• Feedback to course designers: Analysis of these interaction-level data allows us to observe how students are using the material in the course and assess the impact of their use patterns on learning outcomes. We are then able to take advantage of that analysis to iteratively refine and improve the course for the next group of students.
• Feedback to learning science researchers: Finally, there is a feedback loop for learning science researchers who use information gathered by the OLI environment to create and refine theories of human learning. In addition to building on what we know about learning, our courses serve as a platform in which new knowledge about human learning can be developed.

Effectiveness

To evaluate the effectiveness of the OLI approach, evaluators inside and external to the OLI project have conducted studies comparing students using OLI in hybrid mode (instructor-led class using OLI) to students in a traditional classroom environment. In an accelerated learning study using the OLI statistics course, students learned a full semester's worth of material in half the time and performed as well as or better than students learning from traditional instruction over a full semester. Furthermore, there was no significant difference in information retention between OLI students and traditional students in tests given more than a semester later.²

Similar results were achieved in a community-college accelerated learning study with a different course (OLI Logic & Proofs). Students in the OLI course learned 33 percent more material than students in traditional instruction and performed at higher levels on shared material.³ In a study conducted on the OLI chemistry course at Carnegie Mellon University, the number of student interactions with the virtual lab was shown to outweigh all other factors—including gender and SAT score—as the predictor of positive learning outcomes.⁴

How the OLI Approach Can Address the Limitations of Traditional Instruction

Problem No. 1: Many Instructors Teach to a Certain Percentile of the Class

One of the advantages of online education is that students are able to move through the material at their own pace. Technology also makes it easy to provide multiple levels of instruction and many pathways through the same material. Students who come into the course with background knowledge or who are quick learners are able to achieve a given learning objective and move on. Students who need more time with the material are able to work through it at a slower pace and are able to access additional content—alternate explanations, more worked examples, and more practice activities. Depending on his background, a given student may move quickly through one part of the course but need to spend more time in another part.

Since we maintain a model of the student's knowledge state behind the scenes—a model driven by the student's work in the course—one can imagine that we could deliver course content in a completely adaptive manner. We could keep giving the student more instruction on a concept until she has achieved the learning objective, and then allow her to move on to the next topic. However, we feel that to do so would actually be a disservice to the student, as we would not be helping her to develop the metacognitive skills necessary for guiding her own learning. Thus, one of our goals at OLI is for students to become able to assess for themselves when they need more practice and when they are ready to move on—in essence, we would like them not only to learn statistics, or biology, or psychology, but also to learn how to become better learners.

Problem No. 2: Students Frequently Do Not Receive Immediate Feedback Crucial to the Learning Process

One of our goals in creating an OLI course is that the interactions the student has with the course should, to the extent possible, model the types of interactions that student would have with a human tutor helping him work through the material. Rather than presenting the student with a large amount of content to listen to or read through as he would with a traditional lecture or textbook, we introduce him to a concept with a limited amount of expository content. We then let him move immediately into activities where he is working with the concept.

In these "Learn by Doing" activities, the student is not expected to have mastered the material—indeed, we expect her to make mistakes. During the authoring process, we work to capture the common mistakes that students make when learning a given concept. This can be done through conversations with faculty who have extensive experience teaching the course, or by analyzing artifacts of student work, e.g., homework and quizzes. We then write feedback targeted toward those common student mistakes. When a student chooses an incorrect answer, the feedback explains why the answer is incorrect and corrects that misconception—just as a human tutor would if he were helping the student work through the material. When done correctly, this can be very powerful. One student who received such feedback while working through an OLI course was overheard to say, "How did the computer know what I was thinking?" The experience that student had is what we strive for when authoring feedback.

Problem No. 3: In All but the Smallest Classes, the Student's Knowledge State Is a Black Box to the Instructor

We use the model of the student's knowledge state that we maintain behind the scenes to drive a dashboard-style display for the instructor.⁵ This display gives a high-level overview of how students in a class are performing on the learning objectives for each module in the course (Figure 5). This gives instructors using OLI a rich view into what has always been a black box. Before going into class, instructors can see quickly the concepts students are grasping and the concepts with which they are struggling. This enables instructors to spend their time with students in a way that better utilizes their expertise. Instead of spending valuable class time going over concepts that students were able to learn outside of class, they can address problems students are having. They can also focus on richer aspects of the material that they might not have had time to cover in a traditional instruction model.

Problem No. 4: Degrees Favor Time Spent in a Classroom over Demonstration of Competency

A trend with online education in general—not just OLI—is a focus on demonstration of competency as opposed to the more traditional "seat time" measure. In OLI, learning objectives are at the core of everything we do. They are the focus of the instruction that is presented and are what students are asked to evaluate themselves against when measuring their learning. These objectives are the primary way in which information about students' knowledge states is presented to instructors. They are the driving force in our course-development teams as new content is authored. While OLI is not a credit-awarding entity, we strive to create the most efficient process for our students in much the same way that Western Governors University does. Students who have had previous exposure to the material or who are able to learn it quickly can demonstrate competency and move on. Students who need to spend more time on a concept are afforded that option. Our goal is not for every student to complete every activity in an OLI course. Rather, our goal is for students to work enough with a given concept that they achieve that learning objective and then move on.

Problem No. 5: There Is Great Inefficiency in Creating Instruction within Higher Education

Technology can help us solve this problem and maximize productivity by enabling us to create courses that meet the needs of students and faculty across many institutions. This will allow us to eliminate the current redundancy in creating instruction that is rampant across higher education. For example, introductory statistics courses are taught at almost every community college and university across the country. Hundreds of instructors each year spend time creating instruction similar to that created last year at other institutions and that will be re-created next year at still other institutions. Instead of spending resources on thousands of separate introductory statistics courses, we think it makes more sense to bring content experts from a wide range of colleges and universities together to create a small number of statistics courses that will meet the needs of students at all of those colleges and universities.

Challenges

Moving to new outcomes in higher education is not without its challenges. The main challenge we find is that when we combine the affordances of technology with what we know about human learning, the opportunity we have for changing the way we approach instruction is significant enough that it necessitates a fundamental shift for both students and instructors. For students, the focus on learning objectives rather than on simply completing a certain amount of assigned work makes them more responsible for their own learning. They are not always comfortable with this new competency-based model. While there is much to be gained in helping students improve their metacognitive skills and become better learners, this represents a fundamental shift for them, and they need support in making the change. The key to this change is the underlying contract that we have with the student; we will help them to avoid wasting time by enabling them to maximize their productivity in achieving their goals.

For instructors, the challenge is in moving away from the historical activity-based model ("students who do more, learn more"). The new model is one in which the amount of work students must do depends on their diverse backgrounds and skill sets, with each student doing as little or as much as necessary to achieve the learning objectives. Instructors are accustomed to using participation as a proxy for learning. Technology enables us to move beyond that system and actually measure and report learning. However, it is sometimes difficult for many instructors to fully embrace this new approach, as it represents a dramatic change in the way they teach. We have found that instructor training and participation in communities of use are very helpful to instructors in making this shift.

Another challenge we face with instructors is the "expert blind spot" problem, wherein expertise in a subject area may make educators blind to the learning processes and instructional needs of novice students. The educators themselves often are entirely unaware of having such a blind spot.⁶ Even when provided with evidence that a given set of instruction and activities on a concept does result in achievement of the learning objective, instructors will sometimes be reluctant to use the material because it is not explained in the same way that they would explain it. This problem is also rooted in the existing structure of higher education, where instructors develop the material for their courses more or less on their own. Our challenge is to help them to understand that OLI really focuses on what students do outside of class, and that we can give instructors better information to help them design their in-class instruction.

Conclusion

At OLI, we believe that technology can be harnessed to make significant improvements in higher education in terms of cost, productivity, and learning, and we believe that we are on the right path toward making that change happen. In 1991, Herb Simon, a Nobel laureate from Carnegie Mellon University, said that, "Improvement in post-secondary education will require converting teaching from a 'solo sport' to a community-based research activity." Information and communication technologies can now be used to provide meaningful, actionable feedback to students, instructors, instructional designers, and learning scientists. This information is not available in the traditional "teaching as a solo sport" model. To date, these technologies have not been widely used for such purposes. Once they are, the long-hoped-for transformational impact of technology on education becomes a reality.

Notes

1. B. Bloom, "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring," Educational Researcher 13, no. 6 (1984): 4–16.
2. M. Lovett, O. Meyer, and C. Thille, "The Open Learning Initiative: Measuring the Effectiveness of the OLI Statistics Course in Accelerating Student Learning," Journal of Interactive Media in Education (2008), retrieved from http://jime.open.ac.uk/2008/14/.
3. C. D. Schunn and M. Patchan, An Evaluation of Accelerated Learning in the CMU Open Learning Initiative Course Logic & Proofs (technical report by Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA, 2009).
4. K. Evans, D. Yaron, and G. Leinhardt, "Learning Stoichiometry: A Comparison of Text and Multimedia Formats," Chemistry Education Research and Practice 9 (2008): 208–18.
5. M. Lovett, J. Rinderle, J. Brooks, C. Thille, and R. Strader, "Creating a Learning Dashboard for Instructors in OLI" (unpublished).
6. M. J. Nathan and A. Petrosino, "Expert Blind Spot among Preservice Teachers," American Educational Research Journal 40, no. 4 (2003): 905–28.

Ross Strader is Associate Director of the Open Learning Initiative (OLI) at Carnegie Mellon University. His work focuses on the intersection of technology and the learning sciences, looking at ways to bring learning research to bear on technology-based, student-centered learning environments. Strader has won the Carnegie Mellon Graduate Student Teaching Award and received Carnegie Mellon's Andy Award for Innovation. Candace Thille is the Founding Director of Carnegie Mellon OLI and co-director of OLnet. She serves as a redesign scholar for NCAT; as a Fellow of ISDDE; on the Executive Advisory board for HP Catalyst; as co-author of the National Education Technology Plan; and on the PCAST working group to improve STEM education.

This chapter is licensed under a Creative Commons Attribution 3.0 Unported License

© 2012 Ross Strader and Candace Thille