A Tour of Gradiance

This demo will walk you through a student experience, first visiting a homework and then a SQL laboratory. At each important step, you may see the page that the student sees by clicking on a link. After viewing each of these pages, click the Back Button on your browser to return to this tour.

Logging In

The student begins at the Gradiance Login Page. It is easy to create a new account the first time you visit. Otherwise, enter your name and password.

Student Home Page

Our demo student, Calvin Hobbes, has signed in and sees his Student Home Page. He can sign up for new classes if he has been given a class token by his instructor. We require a code to avoid a situation where an instructor finds many random people are taking the class assignments and cluttering up the grade sheet.

In this case, Calvin selects Demo-class, the only class in which he is registered, from the upper screen.

Class Home Page

Now, Calvin sees the Class Home Page for Demo-class. On the left menu, he can select Homeworks or Labs to work on; let's choose Homeworks first.

Homeworks

Calvin will now see a Table of Homeworks that he must work on, along with any homeworks that he has completed in the past.

There is an important difference between Gradiance homeworks and typical homeworks. The difference is not in the kind of problem students are asked to solve. As with conventional homework, the student is asked to solve a problem and write down their solution. However, to automate the process, students to do hand the work in to a TA to be graded a week later and returned. Rather, Gradiance samples their work via a random multiple-choice question. If they make the wrong choice, the student is given a Choice Explanation, which is typically either a hint or an explanation of why their choice was wrong. Then, the student is allowed to retake the homework, and is given new choices for the same question. In this way, homework is changed from a quiz, with a long feedback loop into a learning opportunity, with immediate feedback!

In this demo, there are four homeworks on different topics. Two are from our Database offerings: on relational algebra and the E/R model. The other two are variations of the same assignment, for demonstration purposes. Item (3) is a homework composed of sample questions on physics, chemistry, and math. It is in the ordinary style, where students who try the homework and make mistakes are given choice explanations. Item (4) is the same homework, but when a student submits the work, the complete problem solutions are shown. In practice, students would only see the full solution after the homework deadline has passed.

Let's pick homework (3) and do the general science/math problems. Calvin chooses Open Homework next to item (3).

Answering a Sample Homework

The Homework opens for Calvin. He sees the four questions of this homework in random order (so it is harder for students to say "what did you get for question 3?"), and the choices for each question have been randomized as well. One correct answer has been chosen from a pool of correct answers, and three incorrect answers have been chosen from another pool. These four choices are presented in random order.

Let's focus on the second question: computing a trajectory. The projectile has an x-position of 1000t, where t is the time since firing. It's y-position is 500t-16t²; the "16" is one half the acceleration of gravity. If we replace t by x/1000, we get y = x/2 - 16(x/1000)². Now, we can take any (x,y) pair and see if it is on the trajectory. The four choices happen all to have x=3000 or x=4000 (there are also choices with x=1000 and x=2000, as we shall see when we open the homework a second time). We can calculate that when x=3000, y = 1500 - 16*9 = 1356, and when x=4000, y = 2000 - 16*16 = 1744. Thus, (d) is correct. However, we're going to choose (c), just to see what happens.

Calvin tries to solve each of the problems and chooses his answers. He clicks Submit at the bottom of the assignment and is given immediate feedback.

Student Feedback

The Gradiance Response is the homework just submitted, with a total score and with choice explanations, in red, for all questions. Sometimes, no choice explanation is provided for correct answers, but we observe a comment for the correctly answered question (1).

If we look at question (2), we see the choice explanation that tries to explain our error. Since the point (c) chosen, (4000, 2000), lies on the straight line y = x/2, there is a good reason to suppose that the student forgot that gravity reduces the height, and had used the formula y=500t instead of the correct y=500t-16t².

Trying Again

Calvin may now realize that the formula for y is not y=500t, but 500t minus something. Let's suppose that Calvin makes another mistake, and thinks that the acceleration -gt, when integrated over time, is gt², rather than the correct gt²/2. He thus gets the equation of motion y = x/2 - 32(x/1000)², rather than the correct formula with 16 in place of 32.

Calvin now navigates back to the homework he was working on and Opens the Homework Again. We can see several changes. The order of questions has changed, and the trajectory problem happens now to be first. We also see a new set of four choices for this question. Since Calvin's solution is still in error, he might choose (d) instead of the correct (a).

Again, Calvin makes his choices for all four questions and clicks Submit. Note that if Calvin has worked out the formula for the error in a trapezoidal approximation to a quadratic, and solved the problem about the cow pasture, he will have no trouble finding the correct choices for these problems. Likewise, if he understands chemical bonds, he will be able to answer the chemistry problem without difficulty. Thus, the burden of selecting choices for problems he knows how to solve is minimal. Note that it is essential that the student make choices for all problems. If they got credit for a problem the first time they made a correct choice, they could easily repeat the homework and answer at random until by chance they got each problem right once. That would negate the Gradiance goal of assuring that students truly learn the material.

Second Feedback

The system again makes a Response to the Student, with a score and choice explanations. If we look at the response for question (1), we see the suggestion that the formula may be almost right, but the effect of gravity may be off by a factor of 2. It is not explained in which direction the error is made, so the student still has to figure out the details of what is wrong.

A Laboratory Assignment

Let us leave Calvin to work out the correct formula, and take a look at how Gradiance manages programming laboratories. The particular lab we'll work through is in SQL, the database programming language, but the same technology can be applied to learning any programming language, or any other system that involves programming.

Calvin returns to his home page for Demo-class and selects Laboratories from the left menu. He will see the labs he needs to work on, and from this list he selects the "KingsAndQueens" lab and opens it. He sees the Lab Problem Description and boxes in which to enter six SQL queries. Unlike homeworks, labs allow students to submit work for any subset of the problems and get feedback. The reason is that it is normal for programs not to work the first time, and it is too confusing for students to try to solve more than one problem at a time.

Submitting a Query

We shall concentrate on the first query: "Who was king in the year 1000?". In particular, suppose Calvin writes the erroneous query:

     SELECT name FROM Kings
     WHERE beginReign <= 1000 AND endReign >= 1000;

Notice that the first query asks for both the name and nickname, and this query omits the latter. If Calvin clicks Submit at the bottom of the lab, he gets the following Table of Outcomes.

There, we see that the first query was syntactically correct. Had there been a syntax error caught by the DBMS, that error would have been reported to the student on this screen. However, we also see that there was a semantic error --- the query produced the wrong result. The Help Link invites the student to see a sample database (which in this case is the historically true data), and to compare his result with the result of a correct query.

The Example Database

Following the link Results From Example Database takes us to the following Comparison of Results. We see the result of the student's query, and the result of the reference (correct) query. From this comparison, it should be clear that the mistake has to do with the absence of a nickname column, since the correct king, and only that king, seems to have been produced by the student's query. The student still has to change the query by hitting the back button twice, but they should have a good idea what their mistake was, by this point. If the error is more complex, note that the comparison screen also shows the complete set of tuples for each relevant relation, so the student can undertake a serious study of the problem, if needed.

Catching Fakes

The student should next enter a correct query, such as:

     SELECT name, nickname FROM Kings
     WHERE beginReign <= 1000 AND endReign >= 1000;

However, suppose the student wants to avoid thinking hard, and realizes that they now know what result should be produced by the query. Any programming language allows someone to write a piece of code that produces a particular result regardless of input, and SQL is no exception. The clever, but lazy student can now write:

SELECT DISTINCT 'Aethelred II' AS name,
    'The Unready' AS nickname
FROM Kings;

Surprisingly, this query is not graded correct. If they repeat the process of submitting and getting a results table, they find that the Results From Example Database link produces This Message. Gradiance was unable to show them a difference between their result and the correct result, because they had contrived explicitly to produce the same result as the correct query on the sample data. However, the internal test for correctness is not revealed, and Gradiance is able to determine that, had the historical facts been different, their "query" would not have worked.