One of the issues that I always struggled with as an academic -- and I know many other faculty struggle with -- is keeping grad students on track and giving them useful feedback to help them along in their careers. PhD students often get lost in the weeds at some point (or many points!) during grad school. Of course, part of doing a PhD is figuring out what you want to work on and doing things that might seem to be "unproductive" to the untrained eye. On the other hand, many PhD students grind to a halt, spending months or even years on side projects or simply doing nothing at all. One problem my own students often had was working super hard to submit a paper and then doing almost no new work for 2-3 months while waiting to get the reviews back.
When a student gets stuck in a rut, how do you help them out of it? How do you help students clear a path to productivity?
One thing that many PhD programs lack is any regular and formal evaluation of a student's progress. Harvard never did anything formal, although I tried to get something going it did not last beyond one year -- not enough faculty cared to participate, and we couldn't agree on the process or desired outcomes. At Berkeley, every PhD student got a formal letter every year with a "letter grade" indicating how well you were doing and with some optional comments from your advisor on your overall progress. Although that feedback could have been delivered informally, there was a psychological impact to the formal letter and the idea that all of the professors were meeting in a smoky room to talk about your standing. CMU has its infamous "Black Friday" where all the profs literally do get together to score the grad students. Not having been to CMU, I wonder how this was viewed by the students -- did they find this feedback valuable, stressful, or just plain annoying?
Although this kind of feedback can be useful, for many students it goes in one ear and out the other. I think that part of the reason is that there is often no penalty for doing poorly on a review -- about the only thing a PhD program can threaten you with is kicking you out, and most programs that I know of avoid that unless there's a case where a student has been totally unproductive for a period of several years. It's hard to get kicked out of grad school. By the same token there's little incentive to do well on a review: you're not going to graduate any sooner or get paid more. (Sidebar - should PhD programs pay high-performing grad students bonuses?)
The other issue is that these mechanisms are somewhat open loop in the sense that the student is not expected to lay out a plan and stick to it. Most PhD programs expect students to file some kind of formal plan of study or research leading towards their degree, but it is usually a matter of paperwork and is done just once, or maybe twice, during the course of the program. This has almost no value to the student and is just a matter of paperwork. My feeling is that students would benefit tremendously from a more frequent and formal planning process.
At Google, the approach we use for planning is based on OKRs, or "objectives and key results." Every employee and team is expected to come up with their OKRs for the coming quarter, and score the OKRs from the previous quarter in terms of how much progress was made towards each goal. This is extremely useful process since it gets you thinking about what you need to do over the next 3 months (which seems to be about the right planning horizon for most activities) and you have the chance to reflect on your successes and failures of the previous quarter. It's not expected that you achieve 100% of your goals -- if you are doing so, then your OKRs were not ambitious enough -- you should be shooting for a grade of 70-80%.
I wonder if grad students wouldn't benefit from using something like OKRs for planning their research. A student should be able to say what they are doing over the next 3 months. Looking back on the previous 3 months and grading your progress tells you whether you are generally on track. Having quarterly OKR scores can also help advisors point out where the student needs to improve and documents clear-cut cases where a student has been unproductive (something that both students and advisors are often in denial about). Thoughts?