Experience with large class testing under Ecampus (Blackboard)

Alex:


a quick summary on the my first large-class test under Ecampus using QW3


a) things went well overall. We had some difficulties with the lock-down browser (Respondus), not the least of which they released a required update 2 days before the exam that <had> to be installed to run Ecampus robustly let alone QW3 scorms. Even with that installed however, occasionally (5% of the time) the SCORM in ecampus would hang up while loading. This required shutting down the browser and restarting it, after which the QW3 scorm ran fine. In short, the Ecampus-Respondus environment that we are stuck with to run QW3 modules is considerably less robust that QW3 itself. (!!) Proctors with some training are needed


b) student operation of the software was overall good. We did have a problem with the minimized SWF window causing some of the graphics to be somewhat illegible but students could maximize manually. I did not work this problem out in the index.html javascript, but and I may not have been able to -- Ecampus seems to control launched windows. But loading speed was OK (3 megabyte SWF versions). Latency not an issue


c) colleagues and staff (grad students) were formidably impressed by the overall quality of the exam. We are doing things no one else on campus is.


d) what went wrong: some students (as expected) accidentally closed the exam window by mistake in the first few minutes of the test (usually trying to maxim ize the window). The "code" restart patch you and I cooked up was a lifesaver. Without that, we would have been scheduling makeup exams, etc. I counted 7 of these. One enterprising student even finished his exam and asked us to restart it for him after he saw his grade, when he saw others doing so. You see why I wanted the exams to be maximized.....


e) security: was not an issue. we checked people into groups and that handled the lack of IP restriction issue. Took a little extra manpower but worked fine. Plagiarism was in no way an issue (18 different sections with randomized selection of one question from each)


f) student response: favorable and unfavorable. they like the exam but didn't do well on it! Part of that is normal for freshman, though I'll take some of the blame for this --- developing new testing materials always involves some trial-and-error


g) results: still working on that. some minor issues: (1) importing into EXCEL works a little poorly because I had so many questions (over 100) that the number of columns in the CSV exceeded Excel 2005 maxima for column import. also, clearly what is needed is a "decoder" to take results and extract the relevant questions for an individual so they can review exam results after the fact. That being said, the results are FABULOUS for documentation/backup. Its all there.


In short, it was a successful day. I think all the issues I have been working on (and getting your help on) were vital ones. Disasters could have happened. The things you supported me on are real live issues which will affect others using LMS for large-class testing. I can't say I would suggest seeing all them go into future production runs of QW3 but I do know that they should be available via the manual and user network. A lot of these things need to be worked out on a site-by-site basis


I do know that the page-control issues I mentioned yesterday do need to be addressed, on a production basis. I visualize QW3 attracting users who want more than just multiple choice pages (many packages do that). Ultimately, users will want a little more page control that QW3 currently owns. That's my take at least.


Alex -- I'm as excited as ever about QW3! The problems we have had are all LMS support issues. Its the cards we as teaching faculty are dealt. With your help, I was able to overcome some pretty nasty obstacles. I applaud your innovation and your motivation. I am a diehard QW3 fan! My heartfelt thanks for making this product available and as good as it is.....


Joe




Edited 1 times. Last edit at 09/11/08 02:35PM by Alexander McCabe.

Thanks for posting this Joe, it's a great resource for others who might want to use Question Writer with Ecampus and some great ideas for future features and development. (btw, I added some linebreaks to help with the formatting.)

One immediate thing I can suggest is an upgrade of Excel - Excel 2007 can support more columns than earlier versions -
[msdn.microsoft.com]

More info from Joe on large result sets -

the Open Office spreadsheet ("calc") , even fairly old versions, handles the results CSV fine.