I still grade homework assignments the same way, except that nowadays I upload the documents into an LMS (which then emails the docs to the students) instead of clear-cutting trees.  Which means that after spending many, many hours laboriously grading my students’ assignments (creating Word documents as I go) I then upload all the documents into the LMS at once.  The first time I did this I was surprised at the amount of time I spent following links and uploading files.  The next couple of times I timed myself, and was found myself spending more than 15 minutes minutes doing nothing but uploading finished documents into my LMS for a small class.  And that one term that I sent back feedback via Outlook Web Access?  With all the clicking on links, cutting and pasting boilerplate messages, and attaching files I spent more than 30 minutes just sending out feedback, for a small class of 20 or so students.  It might not sound like much, but it’s 30 minutes of pure overhead.

Which brings me to this post’s topic: the excessive overhead many LMS’s impose on teachers when uploading large numbers of documents, and how this overhead can be eliminated.

The fundamental problem is the number page-loads (and mouse clicks) per document (per student) the LMS forces on the instructor.  Typically, the web app presents the instructor with a list of all the students, with each student’s name rendered as a hyperlink.  In order to upload the feedback the instructor has to click on a student’s name.  And then wait for the "Upload Feedback" page to load.  Next, the instructor clicks several more times to upload the feedback file and/or fill in a TEXTAREA with a message for the student, and then finally clicks on the ‘Ok’ button.  And then waits for the next page to load.  If the instructor is lucky, the app will send the instructor directly back to the list of students and clearly display a confirmation/failure message at the top of that page.  If the instructor is unlucky, then clicking the ‘Ok’ button will be followed by a brief wait, then a page that displays the confirmation/failure message.  This confirmation page comes complete with ANOTHER link to follow in order to get back to the ‘List of all Students’ page (after another wait, of course).  This second approach was used by BlackBoard 5 pretty much everywhere – after every action the instructor did, Bb 5 sent the teacher to a page that basically said ‘Hey, that worked!’, and contained a link to go back and do the next useful action.

What’s bad about this approach is that it forces the teacher to send feedback to a class of 20, 30, 40, or more students one student at a time.  Let’s say that there’s 30 students in the class, and that each page-load requires 5 seconds of time.  Consider the following workflow:

Download the ‘List All Students’ page, then enter a loop:

1. Download a specific student’s page
2. Upload the feedback for that student
3. Re-download the ‘List All Students’ page

Using the above workflow, it’s clear that the instructor be spending ( 5 + 30 x ( 5 + 5 ) ~= 300  seconds / 60 = ) 5 minutes just waiting for things to load, not counting the time to upload the files from the teacher’s computer.

BlackBoard V5 adds a step 2.5 into the loop: "Download a page telling you everything went fine".  In that case, the teacher will spend ( 5 + 30 x (5 + 5 + 5 ) ~= 450 seconds / 60 = ) 7.5 minutes just waiting for stuff to load. 

As the overhead-per-page number increases the number of minutes of pure overhead increases.  A short list of possible slowdowns include a slow server, a slow connection to the server, deciding to include the time to upload the feedback file (which can easily take 5-10 seconds, depending on the file’s size), deciding to include the time required to scroll to back the right place in the list, deciding to include the time required to re-orient yourself after the page loads, etc.  With 10 seconds/load using the first workflow, we’re looking at about 10 minutes of overhead.  Increase the number of students, and these numbers all go up, too.  Of course, if these numbers go down (fast server, fast connection to the server, small class size, or an instructor that uploads homework feedback with the frantic speed of a kid playing video games, etc), then you suffer through less overhead.  But the underlying point remains – this style of user interface doesn’t scale to medium or large class sizes.

When dealing with the large volume of individual students that teachers normally do, some sort of ‘consolidated interface’ is critical to maintaining productivity.  Instructors should be presented with a single page that lets contains a list of file upload widgets, so that the instructor can upload each and every student’s feedback on a single page.  Instead of forcing the instructor to repeatedly visit one page per student, all relevant work is consolidated onto a single page, thereby eliminating unneeded overhead.  What’s left is the necessary work of actually uploading the feedback files.  Unfortunately there aren’t too many examples of this that I’m aware of.  The Moodle gradebook is one of the few exceptions, and is pictured here:

(This image was taken from http://www2.oakland.edu/elis/traindocs/Moodle/WeightedGrades/editgrades.jpg). 

The above image has a lot stuff going on in it, so let’s focus on the important parts using the image below:

Notice that each student goes into a separate row; in the above picture each of the two students’ rows are highlighted with the yellow boxes.  For each student, ALL of the graded items are listed, one item per column; in the above picture there’s a blue box around the first three items for "Test Student13".  While the above gradebook doesn’t allow for file upload specifically, it does neatly demonstrate what I mean by a ‘consolidated interface’ – the teacher can now enter the grades for "Homework 1" for all the students in the class on a single page. 

I hope that you, gentle reader, can see that consolidating all the relevant work onto a single page enables greater productivity by NOT forcing teachers to spend substantial chunks of their grading time fighting to upload feedback and grades to students.

What about grading student-by-student online?

You might be wondering why I still grade on my computer by saving feedback into Word documents when I could just type my feedback into an LMS directly.  Here’s a couple of reasons:

1. MS Word is more reliable than a website.  Browser sessions time-out (losing the work that’s been done)(especially if you take a long time to grade an individual item), my ISP not-infrequently drops my connection, and so on.  By saving my work to a local file I know that I won’t lose the work I’ve done.
2. It’s easy to make backup copies.  By uploading the documents to the LMS I’ve effectively made a backup copy, since the documents now exist in the LMS AND on my local hard drive.  But if I want more copies I just need to copy the directory (manually, or in a script using something like XCOPY)
3. MS Word has more features than LMS’s.  Several LMS’s allow the teacher to write feedback directly to the student, using a Rich Text edit box.  But these rarely have features like tables, or autocorrect.
4. Writing feedback directly into the LMS still incurs the overhead, even if it’s less visible.  One might not notice the overhead as much, since one spends a bunch of time grading between loading pages, but it’s still there.
5. Once the students have seen their feedback, it can’t be (easily) changed.  What if you grade Aaron Aardvarkson directly in the LMS at 1:10pm, which he can then immediately see.  Suppose that when you get to Mike Mason at 3:30pm you realize both Aaron and Mike made the same mistake.  Let’s say that you furthermore realize that the point penalty you leveled on Aaron was too high.  Clearly you must grade both Mike and Aaron consistently, so you can’t just give Mike the smaller penalty without changing Aaron’s penalty.   But if Aaron can see his grade at 1:10pm, then by 3:30pm it’s too late to go back and change it.  This assumes that the system even lets the teacher change the original grade, and doesn’t just force the instructor to upload a fresh copy.
6. The instructor might not want to grade all the students in one sitting.  What if you grade Aaron Aardvarkson at 8:10pm, Mike Mason at 11:30pm, then go to sleep so that you can get up in time for your early class the next day?  You can bet that you’ll receive emails from ungraded students asking when they’ll be graded.