A Rubric to Identify Online Course Plans for a Community of Inquiry

This blog was originally posted on the AACE Review (Rogers, 2018).

Community of Inquiry

A community of inquiry (COI) is what it sounds like—people gather to learn from each other. I argue that a COI can be preplanned to engender a robust learning environment. What that entails is under investigation. For instance, a query of COI educational research on the EdTechLib database garnered 6500 articles. “The ‘community’ in “community of inquiry” is not defined by time or space. A common question, problem, or interest helps to forge the connection” (Shields, 1999, para. 2).

Historically, interdisciplinary scholarly communities have been around since the time of Theagenes of Rhegium who orally interpreted texts to pupils in the 6th century B.C.E. (Hornblower & Spawforth, 1998). Those ancient Greek gatherings were generally teacher-centered in an unidirectional flow of information between the teacher and listening participants until eventually taking on the Socratic method of shaping pupils’ understanding through questioning for critical thinking in the 3rd century B.C.E.

As for the American educational setting, the foundations of a COI can be found in John Dewey’s writing and reform efforts, which were influenced by Charles Sanders Pierce’s logic of inquiry for scientific methods and Jane Addams’ pragmatic approach to social analysis (Shields, 1999). For example, Dewey strongly believed that through experience-based learning, students could intellectually address the subject matter with the assistance of their teachers (Dewey, 1938).

Fast forward to computer-mediated instruction, Garrison, Anderson, and Archer (2000) proposed a COI framework for distance education. It includes the following elements they deem essential: social presence (SP), cognitive presence (CP), and teaching presence (TP). According to Google Scholar, their COI framework has been cited academically 4817 times. Based on their research and related literature, my interpretation of the COI presences is as follows:

  • SP is the co-construction of meaning through shared learning experiences to engender student agency from connectedness.
  • CP is the engagement in learning activities that demand higher-order thinking skills.
  • TP refers to feedback and instruction and can be presented through the instructor or student-led activities.

Online Community of Inquiry Syllabus Rubric ©

The online course syllabus serves as a plan of action that can be utilized for discussing continuous improvement between course design collaborators (i.e., instructional designers, course developers, instructors). To that end, I developed a rubric to evaluate online instructors’ planned interactions for delivering computer-mediated instruction based on their syllabi. It is used to analyze proposed interaction treatments (ITs) such as student-student opportunities for discussion, not the actual course. Our purpose was to determine the inclusion and strength of ITs to provide instructional design (ID) feedback to online instructors regarding their course plans. The underlying theoretical premise being the more interactive the course, the higher the level of student satisfaction and course achievement. Cummins, Bonk, and Jacobs (2002) conducted a similar syllabi study that looked at formats and levels of communication of online courses from colleges of education.

The rubric’s purpose is to provide a pragmatic solution to prevent problematic teacher-led (passive knowledge) online courses with little student interaction nor rigorous academic challenges. The Online Community of Inquiry Syllabus Rubric© is based on general concepts from Garrison, Anderson, and Archer’s (2000) COI framework, quality distance education rubrics (California State University-Chico, 2009; Johnson, 2007; Quality Matters™, 2014; & Roblyer & Wiencke, 2004), and significant literature. It consists of the following categories: ID for CP, technology tools for COI, COI loop for SP, support for learner characteristics, and instruction and feedback for TP. The 5-point rubric has the following scale for the criteria: low, basic, moderate, above average, and exemplary. Points awarded determine the course’s potential level of engendering an online COI (i.e., low, moderate, or high). See rubric.

Content Analysis Research of Online Course Syllabi

Rogers and Van Haneghan (2016) conducted the initial research utilizing the rubric with two raters. Good interrater-reliability agreement was obtained in the review of 23 undergraduate and graduate education online course syllabi, intraclass correlation coefficient (ICC) = .754, p < .001 and 95% CI [.514, .892]. Results indicated the potential for above-average CP (M = 4.7); however, SP (M = 3.1) was moderate, and TP (M = 2.7) was basic. Rogers and Khoury (2018) replicated the study at a different institution across disciplines with 31 syllabi; those findings mirrored the previous study’s levels of COI presences indicating a weakness in TP. For action research, the rubric criteria and results can serve as talking points between instructional designers and course developers to address gaps. Table 1 provides common ID feedback based on our 2018 syllabi analysis.

Table 1: Common Feedback Based on the Online Community of Inquiry Syllabus Rubric Analysis

Rubric Category Instructional Design Recommendations
Instructional Design for Cognitive Presence Include higher order thinking activities such as case analysis, papers that require synthesis or evaluation of peer, self, and/or product. See the list of cognitive activities in the Online Course Design Guide in Table 3.
Education Technology for COI · Add group work for collaborating on projects with Google Hangouts or Skype, so students can learn from each other.

· Use Schoology’s Media Album for students to share their projects and obtain peer feedback. For example, students could narrate PowerPoint project and save as MP4 to create a video presentation to add to a digital portfolio.

COI Loop for Social Presence · Provide a rubric for discussions to make the criteria clear.

· Provide discussions on readings to enhance learning from each other.

Support for Learner Characteristics

 

· Add the College’s accommodation statement.

· Provide links to academic tutoring services.

· Provide strategies for remediation and/or resources for building background knowledge.

Instruction and Feedback for Teaching Presence · Add specific online virtual office hours and format options. For example, use Skype, Google Hangouts, or FaceTime with your smartphone for human interaction.

· Describe direct instruction. Will there be narrated PowerPoints, audio summaries, lecture notes, or commercial program?

· Add information on feedback response time and format.

References

Cummings, J. A., Bonk, C. J., & Jacobs, F. (2002). Twenty-first century college syllabi: Options for online communication and interactivity. Internet & Higher Education, 5(1), 1.

Dewey, J. (1938). Experience and education. The Kappa Delta Pi Lecture Series. New York: NY: Collier Books.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education 2(2-3), 87-105. doi:10.1016/s1096-7516(00)00016-6

Hornblower, S., & Spawforth, A. (1998). The Oxford companion to classical civilization. New York, NY: Oxford University Press.

Johnson, E. S. (2007). Promoting learner-learner interactions through ecological assessments of the online environment. Journal of Online Learning and Teaching, 3(2). Retrieved from http://jolt.merlot.org/vol3no2/johnson.htm

QM Higher Education Rubric Fifth Edition. (2014). Quality Matters. Retrieved from https://www.qualitymatters.org/sites/default/files/PDFs/StandardsfromtheQMHigherEducationRubric.pdf

Roblyer, M., & Wiencke, W. (2004). Exploring the interaction equation: Validating a rubric to assess and encourage interaction in distance courses. Journal of Asynchronous Learning Networks, 8(4).

Rogers, S., & Khoury, S. (2018, October). Rubric to evaluate online course syllabi plans for engendering a community of inquiry: Round II. Paper presented at the meeting of the Association of Educational Technology & Communications, Kansas City, MO.

Rogers, S., & Van Haneghan, J. (2016). Rubric to evaluate online course syllabi plans for engendering a community of inquiry. Proceedings of Society for Information Technology & Teacher Education International Conference, 349-357. Chesapeake, VA: AACE.

Rubric for Online Instruction. (2009). Center for Excellence in Learning and Teaching. California State University-Chico. Retrieved from http://www.csuchico.edu/tlp/resources/rubric/rubric.pdf

Shields, P. M. (1999). The community of inquiry: Insights for public administration from Jane Addams, John Dewy and Charles S. Pierce. Archives of the Digital Collections at Texas State University. Retrieved from https://digital.library.txstate.edu/bitstream/handle/10877/3979/fulltext.pdf?sequence=1&isAllowed=y

Join me at AERA 2019 in Toronto

Sandra Rogers standing near AERA conference sign celebrating 100 years

I’ll be attending my second conference of the American Educational Research Association (#AERA19) this year. The theme is ‘Leveraging Education Research in a Post-Truth Era: Multimodal Narratives to Democratize Evidence.’  It will be held in Toronto, Canada from April 5-9th at the Metro Toronto Conference Centre. I was impressed with last year’s conference but a bit overwhelmed. Hopefully, with the help of their conference app, I’ll find the sessions I need.

View this link to see the poster for Dr. Khoury and my session: Rubric to Analyze Online Course Syllabi Plan for Engendering a Community of Inquiry: Round II. Come join me on Saturday morning, April 6, from 8:00 to 9:30am in the Metro Toronto Convention Centre, 300 Level, Hall C. It’s hosted by the Division C – Section 3b: Technology-Based Environments in the subunit for Distance and Online Education. I’ll be sharing copies of my Online Community of Inquiry Syllabus Rubric.

I’ve shared our research paper on the AERA online Repository.  Read this blog page to learn more about our study. My hope is that it will be replicated to validate the rubric and improve not only instructors’ syllabi but teaching and learning in distance education. Let me know if you’re interested in replicating our study.

Are you going to AERA? Let’s connect in Toronto!

Sandra Annette Rogers, PhD

Teacherrogers Products
Pre-K, Kindergarten, First, Second, Third, Fourth, Fifth, Sixth, Seventh, Eighth, Ninth, Tenth, Eleventh, Twelfth, Higher Education, Adult Education, Homeschooler, Staff, Not Grade Specific - TeachersPayTeachers.com

Join me at AECT in Kansas City, MO!

Photo of Sandra Annette Rogers
Say hello if you see me.

The Association for Educational Communications & Technology (AECT) is, in my humble opinion, the premier association for instructional designers. My professors in my doctoral studies had been promoting this professional organization and their educational technology standards to their students. I finally attended the AECT conference last year and was blown away by the professional level of everyone I met and how cordial they were to newcomers. This year, their 2018 conference will be held in Kansas City, MO from October 23-27 at the Kansas City Marriott Downtown. I’ll be there, so let me know if you plan to attend. For AECT members, I placed my slides and research paper on the new conference online portal.

This time around, I’ll be presenting on my latest research and giving a workshop on the Online Community of Inquiry Syllabus Rubric that  I co-developed with Dr. Van Haneghan. It serves as a great collaboration tool to provide feedback to instructors and for syllabi content analysis for action research. Here’s my schedule:

Wed, Oct 24, 9:00am to 12:00pm, Marriott, Room-Bennie Morten B

Use of Online Community of Inquiry Syllabus Rubric for Course Developers and Collaborators, Drs. Rogers & Khalsa

Workshop – Registration Required
The syllabus serves as an action plan, which can be used as a resource for collaboration with instructional designers. In this session, participants will discuss how the Online Community of Inquiry Syllabus Rubric© (Rogers & Van Haneghan, 2016) can be used to pinpoint course development discussions on cognitive, social, and teaching presence for distance education instructors. Research and development of the rubric, a worked sample, commonly shared feedback, and rubric rater training will be provided.


Division of Distance Learning

Thu, Oct 25, 9:40 to 10:05am, Marriott, Room-Julia Lee A

Rubric to Evaluate Online Course Syllabi Plans for Engendering a Community of Inquiry: Round II, Drs. Rogers & Khoury

We replicated a research study that analyzed online course syllabi with the Online Community of Inquiry (COI) Syllabus Rubric© (Rogers & Van Haneghan, 2016). The rubric consists of the following elements: instructional design for cognitive presence, technology tools for COI, COI loop for social presence, support for learner characteristics, and instruction and feedback for teaching presence. We reviewed 31 syllabi across disciplines and found above average cognitive presence, average social presence, and basic teaching presence.

#AECT2018 #elearning #communityof inquiry #edtech

Quality Matters for Online Instruction

Quality Matters (QM) logo

What is it?

Quality Matters™ (QM) is a peer-review process for providing feedback and guidance for online course design.  According to the QM website, it originated from the MarylandOnline Consortium project in 2003. They received a grant from the US Department of Education to create a rubric and review process based on research and best practices.  In 2014, it became its own nonprofit organization.  Through a subscription service, the organization now provides training, resources, conference events, and research collaborations.  They currently have 5000 QM certified reviewers to assist subscribers with the peer review process of their online courses.

Who uses it?

QM provides specific rubrics and guidelines for the quality assurance review process for K-12, higher education, publishers, and continuing education programs that offer distance education.  QM has a new program to bring the rubric and process to students.  The QM process is specifically for hybrid and fully online courses; it’s not for web-enhanced face-to-face courses.  QM currently has 900 subscribers.  Subscription prices are adjusted to the size of your online programs.

How does it work?

A subscribing institution (or individual) requests a QM review of their course and submits an application.  QM recommends that you familiarize yourself with the rubric through the training process in advance of the review.  They also recommend that the course for review not be new—that it has been through a few semesters to work out the bugs.  A QM coordinator for your course assigns you a team of reviewers consisting of a team leader and two other certified peer reviewers, one of which is a subject matter expert.  They read your self-report about the course and review your course using the rubric and guidelines.  The rubric covers these general standards: 1. Course Overview & Introduction, 2. Learning Objectives (Competencies), 3. Assessment & Measurement, 4. Instructional Materials, 5. Course Activities & Learner Interaction, 6. Course Technology, 7. Learner Support, and 8. Accessibility & Usability.  The team contacts you with questions throughout the 4-6 week process.  Then they present you with your evaluation with time to address any major issues before finalizing the report.

What are the benefits?

Those courses that pass the review process receive recognition on the QM website.  Even if you meet the standards, the peer reviewers provide you with recommendations for further improvements.  Instructors can use this feedback for other courses they teach or debrief with colleagues about it.  This serves as an ongoing continuous improvement process.  This is something that institutions can promote to their clients and instructors can add to the curriculum vitae.  From personal experience in becoming a QM certified peer reviewer, I can attest to the benefits of knowing the best practices and accessibility requirements for online course design.  It has helped me to become a better online instructor and provided me with a wealth of knowledge for my work as an instructional designer.  I’m grateful to the Innovation in Learning Center at the University of South Alabama for training me on the QM process and providing the opportunity to become a certified peer reviewer.

Join me at SITE 2016 in Savannah, GA!

Photo of Sandra Annette Rogers
Say hello if you see me.

Two of my proposals were accepted for presentation at the Society for Information Technology and Teacher Education (SITE) International Conference in Savannah, GA.  I’d love to connect with any of my readers who are also going to SITE. This will be my second time to attend this conference and my first time in the city of Savannah.  I can’t wait!

Here’s my current schedule for the conference: (All times are Eastern Standard Time.)

1. Brief Paper: Rubric to Evaluate Online Course Syllabi Plans for Engendering a Community of Inquiry, March 22, 2016 at 11:50- 12:10 P.M., in the Hyatt Regency F.

2.  Poster Session: Saudi ELLs’ Digital Gameplay Habits and Effects on SLA: A Case Study,  March 23, 2016 at 5:30-7:00 P.M. in the Hyatt Regency Harborside Center. See my poster below.

Rubric to Evaluate Online Course Syllabi

Online Cognitive Activities that Engender a Community of Inquiry

Tag words from my blog

What does your syllabus say about your online course?  I just completed a research project developing a rubric  to identify the potential for a community of inquiry in online college courses.  Then I used the rubric to review 23 online course syllabi from my university’s College of Education.  I found a high amount of cognitive presence in the instructional activities and extensive and varied learner support.  Overall, the syllabi met, or exceeded, a moderate level of planned activities to engender a community of inquiry in their online courses.  As you may surmise, the online course syllabi were very detailed.  I did not review the actual courses, only the syllabi.

Here are the examples of cognitive online activities used in the undergraduate and graduate level courses:  developing questionnaires, peer review of papers, pre- and post-assessments, analysis of case studies, critically review an article, development of a personal instructional design model, student-created multiple-choice questions, hyper inquiry team project, academic controversy assignment, instructional design project, peer evaluations of project, simulation project, develop a creativity workshop, developing an online course, developing course evaluations, creating a welcome video, creating an academic contract, creating a course checklist, writing a literature review, completing CITI module, evaluating a program, completing a meta-evaluation of a program evaluation, develop an autobiography, conduct child observations, weekly self-evaluation of own learning, create a professional development plan, essay exams, develop a book trailer, develop a podcast, develop lesson plans, develop a how-to video, write a blog, develop a personal learning network, develop a digital story, develop a wiki, curate digital books and other electronic resources, and participate in monitored teacher education field experience.

Sandra Rogers