Rubrics are nice to provide a quantifying measure to otherwise hand-wavy evaluations, though consistency across time, both within the same interviewer and across multiple ones is important as well.
For example, there might be factors like Monday Grouch, Friday Outta Here, Hangry Before Lunch, and Food Coma After.
The Electrical Engineering PhD Qualifying Exam at Stanford feels more like a quiz show than a test to gauge preparedness for research, where in the course of a M-F week, 100+ students have 10-minute one-on-one Q&A sessions with professors, who make up their own questions that could be anything vaguely related to their areas of research.
Aside from the brainteaser nature of this process, the consistency is a major issue. Say a professor thought her question was easy on Monday and graded all Monday students by that level, but by Friday realized it was actually hard. (Less often, vice versa.) All the Monday and Friday scores would need to be adjusted. Not to mention having many of these sessions in a day is rather taxing (both for the profs and students) so there's the "early morning vs. late afternoon" also.
Tangentially, an example of such a quals question would be: You see an opaque box on a table. How would you deduce what's in the box without moving or breaking it, like drilling a hole?
Rubrics are nice to provide a quantifying measure to otherwise hand-wavy evaluations, though consistency across time, both within the same interviewer and across multiple ones is important as well.
For example, there might be factors like Monday Grouch, Friday Outta Here, Hangry Before Lunch, and Food Coma After.
Case study: Judges' parole rate before and after lunch https://www.wired.com/2011/04/judges-mental-fatigue/
Another example TL;DR below...
The Electrical Engineering PhD Qualifying Exam at Stanford feels more like a quiz show than a test to gauge preparedness for research, where in the course of a M-F week, 100+ students have 10-minute one-on-one Q&A sessions with professors, who make up their own questions that could be anything vaguely related to their areas of research.
Aside from the brainteaser nature of this process, the consistency is a major issue. Say a professor thought her question was easy on Monday and graded all Monday students by that level, but by Friday realized it was actually hard. (Less often, vice versa.) All the Monday and Friday scores would need to be adjusted. Not to mention having many of these sessions in a day is rather taxing (both for the profs and students) so there's the "early morning vs. late afternoon" also.
Tangentially, an example of such a quals question would be: You see an opaque box on a table. How would you deduce what's in the box without moving or breaking it, like drilling a hole?