Monday, April 19, 2010

Should Harvard's Intro CS class do away with grades?

We've been having some discussion amongst the CS faculty over the last few weeks about whether CS50, the intro computer science class at Harvard, should do away with letter grades and instead switch to SAT/UNSAT. Recently, the Harvard Crimson reported that CS50 is going to do away with letter grades, but this is not true -- the issue is still being debated by the faculty, and has yet to have any formal approval. (I don't know what led the Crimson to report it as though it were a fait accompli.) Given that my opinion seems to differ from most of the CS faculty, I thought I'd put my two cents forth here. Of course, this only represents my own thoughts on the matter, not the CS faculty as a whole (so if you are a Crimson reporter, don't go around reporting this as the final word).

For the record, I used to teach CS61, one of the two follow-on courses to CS50, so the output of CS50 directly feed into my course, and I have a vested interest in the intro course maintaining a certain standard. (My colleague Steve Chong is taking over CS61 next fall.)

David Malan, who teaches CS50, has done a fantastic job at increasing enrollments since taking it over 3 years ago -- I believe the enrollment has more than doubled (possibly tripled) under his watch. CS50 is going great guns and attracting students from all over the University, including many students who would have otherwise never taken a CS class. (Unlike a lot of places, CS is not required for Harvard undergrads as a whole, although it is required for a number of majors, including engineering, applied math, etc.) The course culminates in the CS50 fair where students show off their final projects. It is a great course and David has done amazing things to raise the profile of CS at Harvard.

So, right off the bat, I question the need to change the grading option for CS50, which potentially creates more problems than it solves.

David says that he has been wanting to get rid of letter grades entirely in CS50 for some time, and this year decided to raise the issue for discussion. David's claim is that there are still a lot of Harvard students who are intimidated by CS50 (despite the massive trend in increased enrollment) and that doing away with grades will make those students feel more comfortable trying out CS. While this may be true, I am not sure that we should be designing our foundational intro courses for those students. Harvard already has a "CS for non-majors" course, CS1, to fill that need.

A number of faculty believe that more women will be attracted to CS if they can take the course without worrying about their grade. I can't say whether that is true, but I find it somewhat implausible that what is standing between women and CS is just letter grades.

Even if you believe that doing away with letter grades will increase enrollment, consider the possible downsides. The first is that students who complete CS50 with a "SAT" grade won't have a good idea of their readiness to take more advanced CS courses. Grades do have informational value, and eliminating them makes it much harder to plan one's future course of study. The second is that the course will be less attractive to students who intend to major in CS or engineering, or who are simply used to getting all A's (as many incoming freshmen at Harvard are), so this change could actually decrease enrollment at the top end of the course. It would be fine if the hardcore students could just skip CS50 and take one of the later courses instead, but those courses are not currently designed to supplant CS50 for the better-prepared students. Also, skipping CS50 means missing out on the community experience that I think is important for student cohesion.

The third, and most severe, problem with this proposal is that it will make CS50 no longer suitable for the ABET-accredited engineering degree (which requires letter-graded courses) and the University-wide General Education requirement. So by trying to cast a wider net with CS50, the course no longer satisfies important requirements for certain students, so this approach seems self-defeating.

The compromise proposal currently on the table is to offer CS50 in two flavors: a letter-graded version (call it CS50g) and a SAT/UNSAT version (call it CS50su). It would be the same exact course, just with different grading options. This way students who need the letter grade can take CS50g, and everyone else can take CS50su. It seems clear that this will backfire in several ways. First, many students who take CS50su will later realize they needed a letter grade to satisfy a requirement down the line, but it will be too late. Second, it will create a class distinction between the CS50g and CS50su students. Students taking the course for a letter grade will demand much more detailed feedback on their assignments and more rigor in the grading process (since they will be competing for A's) whereas the course staff, used to dealing with mostly SAT/UNSAT students, will not feel the need to make such fine distinctions.

I have not looked into what other universities do and whether there is any agreement that doing away with letter grades for an intro CS course is a good idea. I know that MIT grades all freshmen pass/fail to alleviate stress associated with grades, which might make sense if done across the board, but it is unclear that changing one class is the right way to do this.

Personally, I'd rather not mess with CS50 right now. It's doing great things and I am concerned that doing away with letter grades would do more harm than good. Of course, many of my colleagues here at Harvard disagree with me, and they are encouraged to weigh in on the comments section below.

Update 4/20/10: The Crimson published a follow-on article this morning discussing the CS50 controversy.

Update #2 4/26/10: Given the many concerns raised, there was simply not enough time for the CS50 SAT/UNSAT proposal to be discussed by the faculty before the deadline for changes to the course catalog for next fall. As a result the proposal has been tabled for now; I expect it will be revisited next year.

21 comments:

  1. Is Harvard contemplating ABET accreditation for Computer Science? I know that Engineering Science is already accredited.

    Most (if not all) of the Engineering programs at UC Berkeley are accredited by ABET and I am unconvinced of its value. It is an enormous amount of overhead, and I have yet to see indications that it has improved our program in any significant way.

    ReplyDelete
  2. Doug - not that I have heard. We have an ABET accredited Engineering S.B. degree and based on what I have seen, I would tend to agree with your concerns about the overhead.

    ReplyDelete
  3. You can imagine being even more flexible -- at the end of the quarter every student decides whether they want their grade or not. Those who enjoyed the class and want to continue taking CS classes will elect the grade option. Those who gave it a try and didn't like it/decided it wasn't for them/didn't do too well will choose the SAT/UNSAT option. Then you are really marketing the class as a 'trial with a money back guarantee' :)

    ReplyDelete
  4. Maria, that is an interesting idea, but I'm pretty sure it's unworkable. First of all, students would have to register for one or the other course and I don't know of a mechanism for changing which course you are in past the add/drop deadline (early in the semester). Also, once you complete the course I don't believe there's any way to change how it's listed on the transcript.

    Finally, another concern with CS50g vs CS50su is perhaps that only the best students will take the graded option, and then they will all get A's -- thereby defeating the purpose of grades, or at least contributing to grade inflation.

    ReplyDelete
  5. If only the best students take 50g, might it not be the case that they don't all get A's, because the curve is so harsh? Or would they be compared against all the "slackers" who are not taking it for a grade?

    Anyway, I think the mix-and-match idea is quite scary. It would be hard to pull off. I'm with you that grades serve a purpose.

    ReplyDelete
  6. So actually it wasn't the crimson that first reported the change as a fait accompli, but rather David Malan himself via an email to the course staff, which he encouraged be distributed publicly. According to his email the change was not something still up for debate but rather already approved by the authorities.

    ReplyDelete
  7. I don't know how one would compare grades for the 50g and 50su students. David says that they would be internally graded the same, but the only difference would come down to whether they only get a SAT/UNSAT or a real letter grade in the end. If that is true *and* the best students take 50g, then presumably most students taking 50g will get high marks. If on the other hand you grade them separately, then you are creating a two-class system and that seems unfair.

    Actually, students can already take CS50 pass/fail (not the same as SAT/UNSAT). But since "opt out" is a lot less effective than "opt in", we wanted to encourage most students to take the 50su option and leave 50g there for the "few" students that would want a letter grade. My belief is that if both options are offered, nearly all students will take the letter graded option.

    ReplyDelete
  8. Michael MitzenmacherApril 19, 2010 at 6:55 PM

    Matt --

    Just to clarify, although the option of switching entirely to SAT/UNSAT was discussed, I think there was widespread concern for many reasons, including those you pointed out, with that option. I don't think that would have even been viable, without a lot more time for preparation and planning for the consequences. I think the main reason to even discuss it now is just because the Crimson article incorrectly seemed to suggest that was what was going on, and I think you did an excellent job pointing out why it would be quite problematic.

    The "compromise" plan -- or g/su plan -- is really the only short-term viable plan if we want to make a change. The downsides you list are pretty focused on the complete switchover. That's not to say that there aren't downsides for the g/su plan, but they're of a different flavor.... that I expect we'll continue to discuss.

    ReplyDelete
  9. Right, this post is mainly about the hypothetical idea of going entirely SAT/UNSAT, but this was David's first proposal to the faculty, and the one reported on by the Crimson. As for the g/su option, to me it seems unnecessary given that students can already take CS50 pass/fail. I don't see the potential advantages of that approach outweighing the potential disadvantages.

    ReplyDelete
  10. "First of all, students would have to register for one or the other course and I don't know of a mechanism for changing which course you are in past the add/drop deadline (early in the semester). Also, once you complete the course I don't believe there's any way to change how it's listed on the transcript."

    This is a political issue, not a technical one. Princeton has had Pass/D/F election (all grades C-1 or above are converted to a "P", with the option to switch to regular grading until the ninth week of the term) for some years now.

    ReplyDelete
  11. Addendum: at Princeton, the "C- or higher converts to a P" grading option can only be exercised at most four times during your four years. If you rescind P/D/F between the fifth and ninth week, it counts against your P/D/F budget as if you had actually exercised the option. At their option, faculty members can exclude P/D/F grading.

    ReplyDelete
  12. The reasons against the switch don't sound that convincing to me. Is the concern that someone will get a "Pass" and feel overconfident about taking a "harder" course? This seems far-fetched - they still have homework and test grades to give them feedback after all. And even if they do take a "hard" class, they can either just drop it or work their socks off to learn something which is challenging for them. After a short while the student expectations will calibrate: "If you just barely passed CS50 don't think you're Linus Torvalds or anything."

    So:

    Advantages: Potentially increase enrollment, raising the profile of CS and getting many new recruits (including potentially more underrepresented groups).

    Disadvantages: Students might have to take an extra course for accreditation purposes. Some students might get overconfident and take a class which is hard for them.

    If this is the situation it seems like changing might be worth it.

    But one thing I would like to hear is the opinion of the current students about the proposed change.

    ReplyDelete
  13. Anon re: technical vs. political issue: I would argue that *all* problems at universities are political :-)

    Anon re: reasons against the switch: I have no doubt that the switch would have some advantages, don't get me wrong. I'm concerned about unintended consequences, such as the course getting so watered down over time that it no longer serves its current pedagogical role.

    ReplyDelete
  14. It's worth being clear about the MIT grading system: Grades *are* assigned during both fall semester (pass/no record) and spring semester (ABC/no record). They're not visible on the transcript, but they are available to the student's academic advisor, which addresses one of your points about the lack of feedback to the student (and their advisor) about their readiness to continue. I think that providing detailed performance feedback is critical for the intro classes, though there's no real need to make it evident on the official transcript.

    ReplyDelete
  15. It sounds like you already have a great thing going. Don't change a winning strategy. If you want to make the course more friendly or attractive than it already is, don't muck with grading, which has all the negative effects you listed. Rather, keep improving the human side of the course: really friendly TAs, group study sessions, social activities, no-risk tune-up exercises people can do to boost their self-confidence and skill. Ideally, the course should have the reputation that Anyone Who Tries Will Do Well And Everyone Will Have Fun. WWRPD? (RP = Randy Pausch) Your students are certainly capable. The idea of the CS50 Fair is brilliant.

    ReplyDelete
  16. To clarify, ABET has no policy regarding whether courses are graded or not. In fact, under ABET guidelines, a program is prohibited from using course grades as a measure of student outcomes.

    ReplyDelete
  17. That's true, but I never said that ABET (as an organization) required courses to be graded. It's Harvard that requires letter grades for those courses. Hence, students wishing to do the ABET engineering degree here can't get credit for taking CS50 unless they get a letter grade. See: http://webdocs.registrar.fas.harvard.edu/ugrad_handbook/current/chapter3/SB_eng.html

    ReplyDelete
  18. A 2002(!) USA Today editorial reported that "eight out of every 10 Harvard students graduate with honors and nearly half receive A's in their courses" so it seems the grade inflation cat has already been out of the bag for some time.

    I googled "harvard grade inflation", not sure if direct links are appreciated here.

    ReplyDelete
  19. Dartmouth has a system called NRO, standing for Non-Recording Option. At the start of the term, a student may set a target grade for a course. If the student gets the target grade or better, then the grade the student got goes on the transcript. If the student passes the course but with a grade below the target grade, then NR goes on the transcript. If the student fails the course, then the failing grade goes on the transcript. So it's like pass/fail, but with the ability to report a good grade. The student can set whatever grade target he or she wishes (apparently, some students have even set targets of C-), but there are a couple of catches: (1) you can't have more than three NR grades (though you can make or exceed your target grade more often), (2) you can't NRO a course that counts for your major, and (3) an NR grade never satisfies any distributive requirements.

    This system seems to work pretty well. Not that Harvard would ever copy something Dartmouth does, mind you...

    ReplyDelete
  20. I have always thought that tests and grades do not work. The reason is that many (if not most) students just prepare themselves for certain test or tests, which then they pass with some grades.

    If you do a follow-up to the students in a few months after the test (if not weeks or days later) you will see that much of the contents of the test are already forgotten, and hopefully only the most important ideas remain.

    I have observed this behaviour on people with good grades and bad grades. The only students that consistently keep a good level of understanding, are those that have studied because of they are truly interested in the material of a course, not because of getting a good grade.

    In my opinion, grades distract students from the actual learning process, by setting up goals that do not really measure learning. On the other hand, the 'best' learners are the one that are really involved with the material of a course.

    So even if there is a good correlation between good grades and preparation for the next course, I disagree with the idea that good grades are the key for producing better prepared students. Instead, involving the students in group projects and the product of their efforts is the way to go. Students that enroll in a course because of their own interest are more likely to study with more interest, and hopefully learn more.

    Sorry for posting so late in the discussion!

    ReplyDelete

Startup Life: Three Months In

I've posted a story to Medium on what it's been like to work at a startup, after years at Google. Check it out here.