Blackboard Test Generator Converts MS Word Formatted Tests into LMS Quizzes

The Blackboard (Bb) Test Generator converts your electronic file tests (i.e., MS Word or Text) into the learning management system (LMS) test questions. Bb Test Generator is an open educational resource. This will save time from building a multiple choice or T/F test online one question at a time IF you already have it prepared. The College of Southern Idaho provides a Bb test generator website where you can copy-and-paste your test to the Bb test generator to convert it into a zip file that can be uploaded into Schoology. The directions on their Website are fairly straightforward. After you convert the text, you’ll obtain a bbquiz zip file.

For Schoology LMS, follow these steps after you log into your  course to upload the test:

  1. Create a blank test in Schoology.
  2. Then select Add Question.
  3. From the drop-down menu, select Import Test/Quiz.

  1. Select Blackboard 7.1-9.0 button for import type from the pop-up window. Select next and locate your bbquiz zip file for import from your computer.

5. Then provide the appropriate test settings within Schoology.

  1. Save!

Note. In Schoology, the default points awarded for test questions is 1. To change them all to something else without having to manually do this one-by-one for long tests, follow these steps:

  1. Select the Options tab within the Quiz Questions view, and save your test to the Schoology Question Bank. Make a new Question Bank if you don’t have one already. Save Question Bank to your personal Resources (Home) on Schoology.

Schoology Test creation options include a drop-down menu for Add Questions to Bank

  1. Then delete your quiz after you’ve saved it to the Schoology Question Bank. Yes, start all over!
  2. Create a new test (Add Quiz), and use questions from your test bank.

  3. Select From Question Banks from the drop-down menu to Add Questions.

Quiz creation tool includes a tab to Add Questions with a option in the drop-down menu for From Question Bank5. Open the Question Bank in your Resources to add Set Points BEFORE you copy it over. This is the only material you can actually edit within your personal resources in Schoology.

In your personal resources, within your saved Question Bank, you can set points for all test items by typing in the value

 

Use Gwet’s AC1 instead of Cohen’s Kappa for Inter-rater Reliability

Last year, I attended a lecture by my former assessment and measurement professor, Dr. Van Haneghan, at the University of South Alabama. He addressed the paradox of using Cohen’s Kappa (k) for inter-rater reliability and acknowledged that it was identified in the literature two decades ago but has been mainly overlooked. The problem is that Cohen’sskews data when there is a high agreement between raters or an imbalance in the margins of the data tables (Cicchetti & Feinstein, 1990; Gwet, 2008).  This is contradictory to the statistical technique’s purpose, as researchers want to obtain an accurate degree of agreement. So if you’ve ever used Cohen’s Kappa for inter-rater reliability in your research studies, I recommend recalculating it with Gwet’s first-order agreement coefficient (AC1).

I decided to rerun the stats for my research study involving two raters analyzing the content of 23 online syllabi with the Online Community of Inquiry Syllabus Rubric for my presentation at AERA. AgreeStat was used to obtain Cohen’s k and Gwet’s AC1 to determine inter-rater reliability per category. Tables 1A-B show how the k statistic was affected by high agreement in the category of instructional design (ID) for cognitive presence (CP), while Gwet’s AC1 was not. Overall, Gwet’s AC1 values ranged from .102 to .675 (Mean SD = .135 ± .128). Interrater-reliability for scoring this category was good according to Altman’s (1991) benchmark, Gwet’s AC1 = .675, p < .001, and 95% CI [.04, .617].

Table 1A

Distribution of Scores by Rater and Category (Instructional Design for Cognitive Presence)

Rater2CP
Rater1CP 3 4 5 Missing Total
3 0 0 0 0 0 [0%]
4 1 2 1 0 4 [17.4%]
5 4 0 15 0 19 [82.6%]
Missing 0 0 0 0 0 [0%]
Total 5 2 16 0 23 [100%]
[21.7%] [8.7%] [69.6%] [0%] [100%]
 

Table 1B

 

Inter-rater Coefficients and Associated Parameters for ID for CP

METHOD Coeff. StdErr 95% C.I. p-Value
Cohen’s Kappa 0.36406 0.172287 0.007 to 0.721 4.617E-02
Gwet’s AC1 0.67549 0.128882 0.408 to 0.943 2.944E-05
Scott’s Pi 0.33494 0.195401 -0.07 to 0.74 1.006E-01
Krippendorff’s Alpha 0.34940 0.195401 -0.056 to 0.755 8.754E-02
Brenann-Prediger 0.60870 0.140428 0.317 to 0.9 2.664E-04
Percent Agreement 0.73913 0.093618 0.545 to 0.933 7.344E-08

Note. Unweighted Agreement Coefficients (Coeff.). Standard Error (StdErr) is the standard deviation. CI= confidence interval.

Gwet’s AgreeStat, Version 2015.6.1 (Advanced Analytics, Gaithersburg, MD, USA) currently costs $40. It’s fairly easy to use. See Kilem Gwet’s blog to learn more.

#AgreeStat #GwetAC1 #CohenKappa #Interrater-reliability

References

Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.

Cicchetti, D.V., & Feinstein, A.R. (1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43(6), 551-558. doi:10.1016/0895-4356(90)90159-m

Gwet, K. (2008). Computing inter-rater reliability in the presence of high agreement. British Journal of Mathematical & Statistical Methodology, 61(1), 29-48. doi:10.1348/000711006×126600

Join me at AERA 2018 in NYC

Photo of Sandra Annette Rogers
Say hello if you see me.

I’m so excited about attending my first conference of the American Educational Research Association (#AERA18) this year. This year’s theme is the dreams, possibilities, and necessity of public education. It will be held in New York City from April 13-17th at various participating hotels. There are 17,000 registrants!

My first event at the conference is to meet my second language research mentor on Friday! The Second Language Research special interest group (SIG) offered mentorship from volunteers in their group, and I signed up.

On Tuesday the 17th, I’ll be participating in a roundtable to discuss the research study with the Online Community of Inquiry Syllabus Rubric(c) that Dr. Van Haneghan and I conducted. It will be held in conjunction with other roundtables on the topic of Quality Assurance of Online Teaching & Learning, which is hosted by the Online Teaching & Learning SIG.  Join my roundtable at 10:35am to 12:05pm at the New York Marriott Marquis, Fifth Floor, Westside Ballroom Salon 4. If you can’t make it, the paper will be provided in the AERA Online Repository.

Lastly, I’d like to thank the Spring Hill College Friends of the Library for helping fund this professional development activity!

Cool Tool | Kahoot!

I subscribe to this blog and thought this was something my readers would appreciate learning.

EdTech Digest

CREDIT Kahoot!.pngKahoot!, the game-based learning platform, makes it easy to create, discover, play, and share fun learning games in minutes. Games can be found for any subject, in any language, and are accessible on any device. With Kahoot!, you can introduce new topics, test knowledge, revise, connect with others all over the world, survey opinions, gather insights, facilitate discussion, spark up professional development, or just have a little fun. The social, pedagogical games can be used in the classroom for formative assessments, to survey student opinions, and as rewards. Kahoot! will be launching its new mobile app, which it previewed at ISTE this year, in the fall. Teachers can use the app to send after-class challenges to students as homework, and learners can continue playing for revision and fun wherever they are. The game platform now hosts more than 50 million monthly active users and a public library of more…

View original post 33 more words

Join me at SITE 2016 in Savannah, GA!

Photo of Sandra Annette Rogers
Say hello if you see me.

Two of my proposals were accepted for presentation at the Society for Information Technology and Teacher Education (SITE) International Conference in Savannah, GA.  I’d love to connect with any of my readers who are also going to SITE. This will be my second time to attend this conference and my first time in the city of Savannah.  I can’t wait!

Here’s my current schedule for the conference: (All times are Eastern Standard Time.)

1. Brief Paper: Rubric to Evaluate Online Course Syllabi Plans for Engendering a Community of Inquiry, March 22, 2016 at 11:50- 12:10 P.M., in the Hyatt Regency F.

2.  Poster Session: Saudi ELLs’ Digital Gameplay Habits and Effects on SLA: A Case Study,  March 23, 2016 at 5:30-7:00 P.M. in the Hyatt Regency Harborside Center. See my poster below.

Rubric to Evaluate Online Course Syllabi