the Internet as Part of Project 25 in the Fall Semester, 1997

January, 1998

 

J. Joseph Hoey, Ed.D.
Assistant Director for Planning and Policy Studies
 John M. Pettitt, Ed.D.
Assistant Professor
 Catherine E. Brawner, Ph.D.
 Shelley P. Mull, B.S.
Graduate Assistant
 

Project 25: First Semester Assessment

Executive Summary

 

Available evidence points to Project 25 as a successful venture:

 

21

 

Table of Contents

In January, 1997, the decision was made to launch a project to place 25 existing courses at NC State on the Internet for fall 1997. The purpose of this project was to make a clear demonstration of the degree to which the institution is ready to enter the distance education market on the Internet in a meaningful way, pending appropriate funding, while providing support and encouragement for enhancements important to resident instruction.

In addition to the 29 participating faculty members who contributed extraordinary amounts of time, energy, and creativity to the success of Project 25, the professional staff in Computing Services and in the new Learning Technologies Center (housed in D.H. Hill Library) were extensively involved in training, technical and logistical support, server maintenance, and other aspects of providing the necessary infrastructure to ensure project success.

Top of Section

Table of Contents

This section focuses on comparing students enrolled in the six Project 25 courses being assessed using qualitative data obtained through student focus groups and interviews, again using the four conceptual areas outlined in "Assessing Web-Based Courses at NC State." These four conceptual areas are:

Five qualitative student interview projects were undertaken for this assessment. Students in web and classroom sections of CS 021 were interviewed in two separate focus groups. For undergraduate students in BUS 370, MB 409, ARE 306 and MA 114, a focus group was conducted for those in web sections, and seven separate telephone interviews were conducted for those in classroom sections. In a separate project whose results are included here for comparison, another focus group was held of students in PRT 462.

The reader should be cautioned that participants in focus groups and interviews such as these are not necessarily representative of the population of web-based and classroom students respectively. However, their opinions may prove informative for decision making purposes and formative program evaluation.

CS 021: Students in both sections of this course tended to wait to study for tests until learning objectives had been handed out in class or posted on the web site. Both sections reported using notes to study more so than the class text. Web students expressed a clear desire for the material to be posted to the web site as early as possible in the semester, yet two students indicated that they had only visited the site 4-6 times the entire semester and had printed out all the materials they needed. Web site enhancements they requested included (1) color graphics of what they were studying; (2) an article on the site to go along with and enhance what they were studying; and (3) having the professor record the lectures and post them on the site, either as audio or video.

Self-motivation emerged as an issue, especially with the web students. Several of them noted that more regularly-paced assignments were needed, so that they would know where they were supposed to be in terms of the course materials. In the absence of these regular assignments, they tended to wait until the last minute to do assignments or to study. The end result, they indicated, was that they ended up putting less time into the course than they felt they should have. Focus group participants in the classroom section agreed unanimously that they had spent more time on this class than on their other classes.

Students in the classroom section were enthusiastic about the solid basis the course had given them for future coursework in this curriculum, for future employment, and for understanding the theory behind common turfgrass management practices. On the other hand, students in the web section expressed less confidence about their ability to use what they had learned in this course to continue with their next classes, or to go out into the real world. They felt that they were benefiting less from this course than if they had taken the classroom-based section.

Regarding future registration for web-based courses at NC State, several students in both sections indicated that they would be more interested in taking a web-based course in future if it were not so directly related to their major and if it did not have such an important hands-on component. However, two-thirds of the focus group participants in the classroom section said they would just not be interested in taking a web-based course at NC State. Two commuter students in the classroom section voiced the opinion that it was worthwhile driving the distance to attend the lectures rather than take a web-based section.

BUS 370, MB 409, ARE 306 and MA 114: Students in both groups reported that they tended to study and prepare for these courses the same way that they prepared for other courses. If they are inclined to procrastinate they will procrastinate in these courses, and if they are inclined to keep up with the material they will do so in these courses as well. Some students in the web sections indicated that they were doing better in that course than they were doing in other courses because they were able to work on class material when they were ready rather than at the designated class time, when they might have other things on their minds. The professors noted that the web students, particularly the underclassmen, seemed to think about the material more than their traditional classroom students before posing questions.

Students who chose the web sections were generally pleased with their choices but had nagging concerns about missing out on something in class. The feeling of missing out was confirmed for many on early tests where material that they felt was not adequately covered in the web materials appeared on the tests. Correspondingly, students who chose not to take the web section cited fear of missing out on something as one of the primary reasons for choosing the classroom section.

PRT 462: Focus group participants in this class felt that class lectures should be put on the web site. They reported getting impatient listening to audio over the web, and indicated that it might be better just to omit lectures altogether and find another medium for transmitting course content. Like focus group participants from CS 021, these participants noted that PRT 462 might not be the kind of course that is most amenable to being put on the web due to its hands-on nature.

CS 021: Focus group participants in both sections of CS 021 displayed somewhat similar learning style preferences. Participants in the web section reported feeling like they were missing what was being emphasized in class, and that they had less guidance than they would have liked. They felt that a lot of resources were provided, but no one was there to lead them through those resources. Students in the classroom section agreed that "I can learn better when he [the professor] talks to me and gives examples in class." Web section participants pointed out that having a teacher and attending lectures has been ingrained since childhood and is a difficult habit to break.

Getting questions answered in a timely fashion was another point of similarity between the two CS 021 focus groups. The web group disliked having to wait to have their questions answered. As one participant put it, "active learning is interrupted" when he had to e-mail the professor with his questions or contact him via Netforum.

BUS 370, MB 409, ARE 306 and MA 114: The amount of interaction between students and professors varied with the nature of the course. In general, students in the undergraduate classes only had as much interaction with the professors as was required or was needed for them to understand the material.

The classroom sections of two of the courses had in excess of 200 students, so less interaction occurred or was expected than in smaller classes. Thus, these large classes seemed most suited to the web method of instruction by the professors who taught them. Students in the traditional classes tend to confine their interaction with the professors to the time before and after class. The web students in these courses corresponded with the professors (especially via e-mail) more than the traditional students.

One course had a lab section which all students attended and which the professor led. In this case, the students had direct weekly contact with the professor and could resolve any issues at that time. In another class, web students spent more face to face time with their professor than in the other sections. This is partly due to the fact that they were required to go to his office to pick up graded assignments (as a method of staying in contact) but also due to the fact that those students needed conceptual help with the material. The lack of understanding may be due to the fact that the material was presented in somewhat abbreviated form on the web site and required students to make more use of their texts.

PRT 462: Issues that arose in this focus group were common to other groups as well. Students expressed concerns that by enrolling in an online course section, they would be missing out on the important classroom interactions and would not be able to hear other students’ questions. They reported a general feeling of isolation and lack of feedback. Moreover, they pointed out, students in online sections may miss the benefit of interacting with visiting professionals who come to speak to the class. Problems with testing procedures were also raised. Students felt that an electronic testing format should be considered.

CS 021: Focus group participants in both sections of this course were decidedly independent learners. All participants in the web section studied by themselves; only one focus group participant in the classroom section had formed a two-person study group. Furthermore, students in the web section did not like Netforum due to the time lag involved in getting replies. They tended not to use it above the required minimum. Students in the web section recommended the inclusion of a chat room as a site enhancement that would build student-student interaction.

BUS 370, MB 409, ARE 306 and MA 114: Participants interviewed from the classroom-based sections were more likely to form study groups than students in the web sections. This is not surprising given that web students tended not to know their classmates personally. In one section where the web students did know each other, they did form a study group. On-line focus group participants in general did not express a need to interact with each other on-line as part of the class. While some of the professors would have liked to see more interaction, they did not tend to require students to interact with each other any more than they would in a traditional section. Students only used Net Forum if it was required and then only participated enough to meet the minimum requirements. The exception to this was EAC 586 which had a strong emphasis on using Net Forum as an interaction mechanism. Unlike the other courses under study, those students were geographically dispersed and unlikely to have any other contact off line with their classmates or with the professor.

PRT 462: Participants in the focus group for this course were not enthusiastic about the use of Netforum as a learning tool: they felt that it lacked timeliness, was too busy, and lacked a good graphical interface.

CS 021: Focus group participants in both sections reported that the CS 021 web site was user-friendly and well thought out. No significant technical problems with the site were reported. Both groups appeared to include a number of fairly sophisticated computer users. Students in the web section did not report any noticeable gain in Internet skills as a result of this course, since they had already gained the requisite computing skills in high school or elsewhere. Half of the focus group participants in the classroom section considered themselves computer literate, but two others indicated they had improved their computer skills as a result of taking this course.

Many of the students enrolled in CS 021 were older, non-traditional learners who worked during the day. Web section participants tended to access the site late at night, and were enthusiastic about this convenience. Several students in the classroom section also accessed the web site regularly, and reported that the lecture notes on the web were good to use when they missed a class, or to check for anything they might have missed in class. Both sections applauded the use of learning objectives on the web site, but also said that better graphics and animation would help improve the site. Web participants wanted the lectures on video and included in the web site. Both sections recommended that a course web site be used as a tool to enhance learning, but not the only way to learn in a course.

BUS 370, MB 409, ARE 306 and MA 114: Focus group participants taking the courses on the web were highly computer literate. A few had their own web pages and all reported having considerable experience using the Internet for study and recreational purposes. Interestingly, the amount of Internet familiarity required for taking these courses in a web-based format was relatively low, and all of the classroom based students who were interviewed appeared to have the Internet skills to successfully interact with the course materials. The difference between the groups of students was primarily in their willingness to take the risk of taking a class in a web-based format as well as the degree to which computer use was a part of the students’ lifestyles. All of the web students interviewed had their own personal computers, even if that was not the primary way they accessed course material. Only one of the classroom-based students interviewed indicated that he had a computer. Although not asked directly about PC ownership, most students indicated that they would go to a computer lab on campus if they needed to access course materials on the web. Classroom participants in general wanted to be in the classroom environment and did not trust that they could learn the material adequately if they had taken the class on the web. Most classroom-based participants indicated they would not take another course on the web unless they were required to or had some pressing personal reason to do so.

When students in either group needed help with the material or had technical problems, they would tend to turn to their friends for help rather than any formal organization. Classroom students who had missed lectures would get the notes from friends even though the web materials were accessible to them. Web students who had technical questions would first ask friends and then turn to their professor, usually sending e-mail to indicate that there was a problem, rather than use any formal help resources at the university.

PRT 462: Focus group participants in this course noted that system problems stemming from UNITY file size limitations were the most pressing problem they encountered when attempting to transfer the large graphic files they needed to create as part of their course assignments. They also voiced some concerns about getting technical help when needed, and indicated that the hours posted for technical support were all scheduling conflicts for them.

Top of Section

Table of Contents

This section of the report deals with student performance in web-based and classroom-based sections in Project 25. In this section, comparisons include all twelve courses where both web-based and classroom-based sections were offered in fall 1997 as part of Project 25 and were taught by the same instructor. A total of 1278 students were enrolled in these courses, 247 in the web-based sections and 1031 in the classroom-based sections. Unless noted, all courses carried 3 hours of credit. The courses used for analysis included: