A new professional chapter

I am thrilled to announce that starting in the fall, I will be working as an assistant professor for the Minerva Schools! I want to resurrect this blog with a post about my decision, and what it means for my career as well as for higher education more broadly.

What is Minerva? Good question. While they don’t yet have the name recognition of some of our older institutions, they aim to be "the first elite American university to be launched in a century". Starting from the question of “what does it mean to be an educated person in our time?” they designed a new type of university, stripped of unnecessary sports teams or facilities. In fact the only physical buildings are rented dormitory buildings in San Francisco and other world cities. Classes are held online, but unlike the MOOC model of video lectures, the pedagogical philosophy is geared towards fully active learning. The teaching platform allows instructors to rapidly give polls and quizzes, and to create small groups with a touch of a button. The students live together and move to a different international city each year.

Why I’m excited about this:
If you’ve read this blog before, you will know that I have been uncomfortable with a number of aspects of a Typical Academic Career. I worry about the consequences of grade inflation, and of how to maintain academic rigor without getting crushed in teaching evaluations. I think about what I call “the ever-accelerating hamster wheel” problem: our research impact is often measured in terms of number of publications, and we are putting out more work than can be read, forcing us to aggressively market our own research just to be heard through the noise. In my field, about 1/3 of papers are never cited! And we hire far more graduate students and postdocs to do this ever increasing amount of work while the number of academic positions for them is dwindling through the adjunctification of the professorship. Funding levels are so low that economists have questions whether spending time on grant proposals is even worth it.

So, is it better to change an institution from within or to blaze a different path? This is the question I have been wrestling with over the course of this job season. Academia (though not academics, generally) is conservative, and its wheels turn slowly. It’s deeply hierarchical, and can lull people with the sweet siren song of the status quo. That said, it’s what I’ve been working single-mindedly towards for the last 15 years. The argument for being at a traditional university can be best summed up by one of my mentors, who argues that while R1 life is not perfect, it's the best of the available alternatives. But what if it can be made better?

In the end, the question I kept asking myself is "what do I want out of an academic career? What makes for a good academic life?" For a long time, I've been uncomfortable with institutions that do not value excellence in teaching. Any one professor's research program, no matter how high profile, is still a small slice in the big pie of human knowledge, while the impact that one can have in the life of a student through teaching and mentorship can last a lifetime. But what about research? Am I shutting myself out from this world? I don't think so. I am testing the bold hypothesis that I can do great research outside of the normal paradigm.

Paradoxically, I think that my research might have more impact when freed from the pressures of "bean counting". The academics whose work I read most are not the ones publishing the most papers, but the ones publishing papers with the most depth of thought. I hope to maintain and develop a number of collaborations, tapping into the best minds and free from the need to be in any one location. And I am working to have a home base where I can mentor the research projects of my Minerva students, sparking the same passion for research in them as I had as an undergraduate.

Stay tuned for the future, I think it's going to be bright.

Who takes the responsibility for quality higher education?

This gives me chills: a professor denied tenure for using the Socratic method of teaching. Of course, there are two sides to every story and this article is rather one sided - I have been in classes where so-called Socratic methods are thinly veiled excuses for hurling insults at students - but if we are to take the article at face value, this is another story in a disturbing educational trend.

The Socratic method is challenging for students and requires preparation and engagement with the material. It requires being able to effectively communicate under pressure. However, I feel that learning involves a certain amount of discomfort. Learning means pushing past the boundaries of what we already know, and what we can already do. Most undergraduate courses I took were of lecture-style, teaching students to expect to be a passive audience in class. It's a much easier route and the student can hide lack of preparation, misunderstanding or having a bad day. However, these students cannot hide forever, and this under-preparation often comes back to haunt them at exam time.

As a TA in graduate school, I saw many freshmen having harsh wake-up calls when the first midterms came back. The typical story was "But I came to all the classes, and I read the book chapters twice! How could I have gotten a C on the exam???" The unfortunate answer is that the student mistakes being able to parrot back a section of textbook or lecture for understanding the material. When an exam forces the student to use this information in an analytic or synthetic way, the facade of learning crumbles. 

I don't know any instructor who wants to give a student a poor grade, but the integrity of the educational system depends on accurate assessment of mastery. If an instructor is fired, demoted or denied tenure due to the rigors of his/her course, this could spell the end of education. Sadly, this story is reminiscent of this case: a professor denied tenure for not passing enough students. I highly recommend reading this page because, if we are to take the author at his words, he took every reasonable action to enable his students to succeed.

Who is responsible for student success in higher education? Professors, of course need to be responsible for presenting learning opportunities to students in a clear manner, and to be available for advice and guidance at office hours. However, university students are adults and need to take responsibility for the ultimate learning outcomes. I am concerned by a culture of entitlement that has conditioned students to expect top marks for simply showing up. The expectations of the "self-esteem generation" and the incentives of professors to earn high student evaluations both play a role, I suspect.

I wonder sometimes whether the cost of attendance at American colleges and universities partially drives this phenomenon. Paying for education turns students and their families into customers, and "customers are always right". Perhaps subsidizing higher education would create a culture that divorces education from "service", leading to more honest evaluations and better learning.

Bayesian truth serum, grading and student evaluations

In one of my last posts, I examined some proposals for making university grading more equitable and less prone to grade inflation. Currently, professors are motivated to inflate grades because high grades correlate with high student evaluations, and these are often the only metrics of teaching effectiveness available. Is there a way to assess professors' teaching abilities independent of the subjective views of students? Similarly, is there a way to get students to provide more objective evaluation responses?

It turns out that one technique may be able to do both. Drazen Prelec, a behavioral economist at MIT, has a very interesting proposal for motivating a person to give truthful opinions in face of knowledge that his opinion is a minority view. In this technique, awesomely named "Bayesian truth serum"*, people give two pieces of information: the first is their honest opinion on the issue at hand, and the second is an estimate of how the respondent thinks other people will answer the first question.

How can this method tell if you are giving a truthful response? The algorithm assigns more points to responses to answers that are "surprisingly common", that is, answers that are more common that collectively predicted. For example, let's say you are being asked about which political candidate you support. A candidate who is chosen (in the first question) by 10% of the respondents, but only predicted as being chosen (the second question) by 5% of the respondents is a surprisingly common answer. This technique gets more true opinions because it is believed that people systematically believe that their own views are unique, and hence will underestimate the degree to which other people will predict their own true views.

But, you might reasonably say, people also believe that they represent reasonable and popular views. They are narcissists and believe that people will tend to believe what they themselves believe. It turns out that this is a corollary to the Bayesian truth serum. Let's say that you are evaluating beer (as I like to do), and let's also say that you're a big fan of Coors (I don't know why you would be, but for the sake of argument....) As a lover of Coors, you believe that most people like Coors, but feel you also recognize that you like Coors more than most people. Therefore, you adjust your actual estimate of Coors' popularity according to this belief, therefore underestimating the popularity of Coors in the population.

It also turns out that this same method can be used to identify experts. It turns out that people who have more meta-knowledge are also the people who provide the most reliable, unbiased ratings. Let's again go back to the beer tasting example. Let's say that there are certain characteristics of beer that might taste very good, but show poor beer brewing technique, say a lot of sweetness. Conversely, there can be some properties of a beer that are normal for a particular process, but seem strange to a novice, such as yeast sediment. An expert will know that too much sweetness is bad and the sediment is fine, and will also know that a novice won't know this. Hence, while the novice will believe that most people agree with his opinion, the expert will accurately predict the novice opinion.

So, what does this all have to do with grades and grade inflation? Glad you asked. Here, I propose two independent uses of BTS to help the grading problem:

1. Student work is evaluated by multiple graders, and the grade the student gets is the "surprisingly common" answer. This motivates graders to be more objective about the piece of work. We can also find the graders who are most expert by sorting them according to meta-knowledge. Of course, this is throwing more resources after grading in an already strained system.

2. When students evaluate the professor, they are also given BTS in an attempt to elicit an objective evaluation.

* When I become a rock star, this will be my band name.

A solution to grade inflation?

In the somewhat limited teaching experience I've had, I have found grading to be particularly difficult. The grade a student receives in my class can determine whether he'll get or keep scholarships and will play a role in determining what kinds of opportunities he'll have after my class. This is a huge responsibility. As a psychophysicist, I worry about my grade-regrade reliability (will I grade the same paper the same way twice), order effects in my grading (if I read a particularly good paper, do all papers after it seem to not measure up?), and whether personal bias is affecting my scoring (Sally is always attentive and asks good questions in class, while Jane, if present, is pugnacious and disruptive).

Of course, the easiest thing is to give everyone generally good grades. The students won't argue that they don't deserve them, and in fact, there is evidence that they'll evaluate me better for it in the end.



And while many institutions have (implicitly or explicitly) adopted this strategy, the problem with grade inflation is that it hurts students who are performing at the top level, and removes accountability from our educational system. So, what do we do about grading?



The Chronicle of Higher Education has an interesting article showing two possible solutions. The second solution involves AI-based grading, which sounds intriguing. Unfortunately, no details were provided for how (or how well) it works, so I remain skeptical. However, the first proposed solution merits some discussion: outsource grading to adjunct professors who are independent of the course, professor and students. The article follows an online university that has enacted this strategy.


Pros of this idea:
- As the grader is not attached to either the professor or the student, bias based on personal feelings towards a student can be eliminated.
- In this instantiation, graders are required to submit detailed justifications for their grades, are provided extensive training and are periodically calibrated for consistency. This can provide far more objective grading than what we do in the traditional classroom.


However, the idea is not perfect. Here are some cons that I see:
- The graders' grades get translated into pass or fail. A pass/fail system does not encourage excellence, original thinking, or going beyond the material given.
- Much of traditional grading is based on improvement and growth over a semester, and this is necessarily absent in this system. Honestly, I only passed the second semester of introductory chemistry in college (after failing the first test) because the professor made an agreement with me that if I improved on subsequent tests, she would drop the first grade.
- Similarly, the relationship between professor and student is made personal through individualized feedback on assignments. Outsourcing grading means that there cannot be a deep, intellectual relationship between parties, which I believe is essential to learning and personal growth.


While not perfect, this is an interesting idea. What are your ideas for improving on it (or grading in general)?

The value of teaching at the university level

The Neuroskeptic has a particularly insightful post on the uncomfortable disconnect between how universities, academics and politicians see the role of teaching. I've written occasionally on some of the broken aspects of the academy, and I think Neuroskeptic's piece adds a couple of crucial thoughts to the discussion:

"And academics have no incentive to teach well and, in most cases, no incentive to make sure that their university has a reputation for good teaching."

(Emphasis mine.)

Indeed, if anything, being involved in excellent teaching is viewed as the "kiss of death" for one's tenure at many American research universities. And, as Neuroskeptic points out, the nomadic lives of young researchers prevents strong ties to a particular university:

"Until you get to the level of tenured professor, if ever, you cannot assume that you'll be working in the same place for very long. Many academics will go to one university for their undergraduate degrees, another for their masters, another for their doctorate, and then another two or three as junior faculty member before they "settle down" - and the majority don't make it that far."

Perhaps the solution is to tenure faculty more often and earlier.  Imagine young, energetic, passionate academics, unafraid to teach with excellence and filled with a sense of place in their institution. Maybe this is what we need for excellent undergraduate education.