Harvard Office of Institutional Research models: explicit racial penalty required to reproduce actual admit rates for Asian-Americans

This is my third post discussing the Students For Fair Admissions lawsuit against Harvard over discrimination against Asian-American applicants. Earlier posts here and here discussed, among other things, the tendency of the Admissions Office to assign low personal ratings to A-A applicants. A-As received, on average, the lowest such ratings among all ethnic groups from the Admissions Office. In contrast, alumni interviewers (who actually met the candidates) gave A-A applicants scores comparable to white applicants, and higher than other ethnic groups.

Harvard's Office of Institutional Research (OIR) produced a series of internal reports on discrimination against Asian-American applicants, beginning in 2013. They attempted to model the admissions process, and concluded there was outright penalization of A-A applicants:

Mark Hansen, the (now former) OIR employee, remembers far more. He remembers working with others in OIR on the project. He remembers gathering data, conducting the regression analysis, collaborating with colleagues, coordinating with the Admissions Office, and discussing the results of OIR’s investigation with Fitzsimmons and others on multiple occasions.  Hansen expressed no concerns with the quality and thoroughness of OIR’s statistical work. Moreover, he has a clear understanding of the implications of OIR’s findings. Hansen testified that the reports show that Asian Americans “are disadvantaged in the admissions process at Harvard.” And when asked: “Do you have any explanation other than intentional discrimination for your conclusions regarding the negative association between Asians and the Harvard admissions process?” Hansen responded: “I don’t.”

The figures below show several OIR models which try to fit the observed admit rates for various groups. The only model that comes close (Model 4) is one which assigns outright penalties to A-A applicants (using "demographic" -- i.e., explicitly racial -- factors). IIUC, this is *after* the low Personal Rating scores from the Admissions Office have already been accounted for!

In the decades leading up to the data discovery forced by the SFFA lawsuit, we heard many claims that legacy / recruited athlete status, or leadership characteristics, or extracurriculars, were the reasons for A-As having such a low acceptance rate (despite their strong academic records). The OIR analysis shows that these effects, while perhaps real, are only part of the story. In Model 4, pure racial bias reduces the A-A percentage of the entering class from 26% (after accounting for all the factors listed above) to the actual 18-19%!

Harvard Office of Institutional Research on Discrimination Against Asian-American Applicants

Harvard's Office of Institutional Research (OIR) produced a series of internal reports on discrimination against Asian-American applicants, beginning in 2013. I believe this was in response to Ron Unz's late 2012 article The Myth of American Meritocracy. These reports were shared with, among others, William Fitzsimmons (Dean of Admissions and Financial Aid) and Rakesh Khurana (Dean of Harvard College). Faced with an internal investigation showing systemic discrimination against Asian-American applicants, Harvard killed the study and quietly buried the reports. The Students For Fair Admissions (SFFA) supporting memo for Summary Judgment contains excerpts from depositions of these and other Harvard leaders concerning the internal reports. (Starting p.15 -- SAD!)

The second report included the figure below. Differences are in SDs, Asian = Asian-American (International applicants are distinct category), and Legacy and Recruited Athlete candidates have been excluded for this calculation.

As discussed in the previous post: When it comes to the score assigned by the Admissions Office, Asian-American applicants are given the lowest scores of any racial group. ... By contrast, alumni interviewers (who actually meet the applicants) rate Asian-Americans, on average, at the top with respect to personal ratings—comparable to white applicants ...

From the SFFA (Students For Fair Admissions) supporting memo for summary judgement:

OIR found that Asian-American admit rates were lower than white admit rates every year over a ten-year period even though, as the first of these two charts shows, white applicants materially outperformed Asian-American applicants only in the personal rating. Indeed, OIR found that the white applicants were admitted at a higher rate than their Asian-American counterparts at every level of academic-index level. But it is even worse than that. As the second chart shows, being Asian American actually decreases the chances of admissions. Like Professor Arcidiacono, OIR found that preferences for African American and Hispanic applicants could not explain the disproportionately negative effect Harvard’s admission system has on Asian Americans.

On David Card's obfuscatory analysis: the claim is that within the pool of "unhooked" applicants (excluding recruited athletes, legacies, children of major donors, etc.), Asian-Americans are discriminated against. Card's analysis obscures this point.

The task here is to determine whether “similarly situated” applicants have been treated differently on the basis of race; “apples should be compared to apples.” SBT Holdings, LLC v. Town of Westminster, 547 F.3d 28, 34 (1st Cir. 2008). Because certain applicants are in a special category, it is important to analyze the effect of race without them included. Excluding them allows for the effect of race to be tested on the bulk of the applicant pool (more than 95% of applicants and more than two-thirds of admitted students) that do not fall into one of these categories, i.e., the similarly situated applicants. For special-category applicants, race either does not play a meaningful role in their chances of admission or the discrimination is offset by the “significant advantage” they receive. Either way, they are not apples.

Professor Card’s inclusion of these applicants reflects his position that “there is no penalty against Asian-American applicants unless Harvard imposes a penalty on every Asian-American applicant.” But he is not a lawyer and he is wrong. It is illegal to discriminate against any Asian-American applicant or subset of applicants on the basis of race. Professor Card cannot escape that reality by trying to dilute the dataset. The claim here is not that Harvard, for example, “penalizes recruited athletes who are Asian-American because of their race.” The claim “is that the effects of Harvard’s use of race occur outside these special categories.” Professor Arcidiacono thus correctly excluded special-category applicants to isolate and highlight Harvard’s discrimination against Asian Americans. Professor Card, by contrast, includes “special recruiting categories in his models” to “obscure the extent to which race is affecting admissions decisions for those not fortunate enough to belong to one of these groups.” At bottom, SFFA’s claim is that Harvard penalizes Asian-American applicants who are not legacies or recruited athletes. Professor Card has shown that he is unwilling and unable to contest that claim.

This is an email from an alumni interviewer:

[M]y feelings towards Harvard have been slowly changing over the years. I’ve been interviewing for the college for almost 10 years now, and in those ten years, none of the Asian American students I’ve interviewed has been accepted (or even wait-listed). I’m 0 for about 20. This is the case despite the fact that their resumes are unbelievable and often superior to those of the non-Asian students I’ve interviewed who are admitted. I’ve also attended interviewer meetings where Asian candidates are summarily dismissed as “typical” or “not doing anything anyone else isn’t doing” while white or other minority candidates with similar resumes are lauded.

From p.18 of the SFFA memo:

Mark Hansen, the (now former) OIR employee, remembers far more. He remembers working with others in OIR on the project. He remembers gathering data, conducting the regression analysis, collaborating with colleagues, coordinating with the Admissions Office, and discussing the results of OIR’s investigation with Fitzsimmons and others on multiple occasions.  Hansen expressed no concerns with the quality and thoroughness of OIR’s statistical work. Moreover, he has a clear understanding of the implications of OIR’s findings. Hansen testified that the reports show that Asian Americans “are disadvantaged in the admissions process at Harvard.” And when asked: “Do you have any explanation other than intentional discrimination for your conclusions regarding the negative association between Asians and the Harvard admissions process?” Hansen responded: “I don’t.”

A very sad tweet:

Here's an exchange in the Harvard lawsuit in which a a teacher at Stuyvesant started to cry when she was shown data that showed her Asian students were *half* as likely to be admitted to Harvard as her white students in 2014 pic.twitter.com/EfjBeBFgfy

— Molly Hensley-Clancy (@mollyhc) June 15, 2018

Harvard discrimination lawsuit: data show penalization of Asian-Americans on subjective personality evaluation

Harvard and Students For Fair Admissions (SFFA), which is suing Harvard over discrimination against Asian-American applicants, have released a large set of documents related to the case, including statistical analysis of records of more than 160,000 applicants who applied for admission over six cycles from 2000 to 2015.

Documents here and here. NYTimes coverage.

The following point does not require any sophisticated modeling (with inherent assumptions) or statistical expertise to understand.

Harvard admissions evaluators -- staffers who are likely under pressure to deliver a target mix of ethnicities each year -- rate Asian-American applicants far lower on subjective personality traits than do alumni interviewers who actually meet the applicants. The easiest way to limit the number of A-A admits each year would be to penalize them on the most subjective aspects of the evaluation...

As stated further below: When it comes to the score assigned by the Admissions Office, Asian-American applicants are assigned the lowest scores of any racial group. ... By contrast, alumni interviewers (who actually meet the applicants) rate Asian-Americans, on average, at the top with respect to personal ratings—comparable to white applicants...

SFFA Memorandum: Professor Arcidiacono found that Harvard’s admissions system discriminates against Asian-American applicants in at least three respects. First, he found discrimination in the personal rating. Asian-American applicants are significantly stronger than all other racial groups in academic performance. They also perform very well in non-academic categories and have higher extracurricular scores than any other racial group. Asian-American applicants (unsurprisingly, therefore) receive higher overall scores from alumni interviewers than all other racial groups. And they receive strong scores from teachers and guidance counselors—scores that are nearly identical to white applicants (and higher than African-American and Hispanic applicants). In sum, Professor Arcidiacono found that “Asian-American applicants as a whole are stronger on many objective measures than any other racial/ethnic group including test scores, academic achievement, and extracurricular activities.”

Yet Harvard’s admissions officials assign Asian Americans the lowest score of any racial group on the personal rating—a “subjective” assessment of such traits as whether the student has a “positive personality” and “others like to be around him or her,” has “character traits” such as “likability ... helpfulness, courage, [and] kindness,” is an “attractive person to be with,” is “widely respected,” is a “good person,” and has good “human qualities.” Importantly, Harvard tracks two different personal ratings: one assigned by the Admissions Office and another by alumni interviewers. When it comes to the score assigned by the Admissions Office, Asian-American applicants are assigned the lowest scores of any racial group. ... By contrast, alumni interviewers (who actually meet the applicants) rate Asian Americans, on average, at the top with respect to personal ratings—comparable to white applicants and higher than African-American and Hispanic applicants.

From the Crimson:

The report found that Asian American applicants performed significantly better in rankings of test scores, academics, and overall scores from alumni interviews. Of 10 characteristics, white students performed significantly better in only one—rankings of personal qualities, which are assigned by the Admissions Office. [italics added]

See also Too Many Asian Americans: Affirmative Discrimination in Elite College Admissions. (Source of figure at top; the peak in A-A representation at Harvard, in the early 1990s, coincides with external pressure from an earlier DOJ investigation of the university for discrimination.)

A very sad tweet:

Here's an exchange in the Harvard lawsuit in which a a teacher at Stuyvesant started to cry when she was shown data that showed her Asian students were *half* as likely to be admitted to Harvard as her white students in 2014 pic.twitter.com/EfjBeBFgfy

— Molly Hensley-Clancy (@mollyhc) June 15, 2018

For the statistically sophisticated, see Duke Professor Arcidiacono's rebuttal to David Card's analysis for Harvard. If these entirely factual and easily verified characterizations of Card's modeling (see below) are correct, the work is laughable.

Professor Card’s models are distorted by his inclusion of applicants for whom there is no reason to believe race plays any role.

As my opening report noted, there are several categories of applicants to whom Harvard extends preferences for reasons other than race: recruited athletes, children of faculty and staff, those who are on the Dean’s List or Director’s List [i.e., Big Donors], legacies, and those who apply for early admission.1 Because of the significant advantage that each of these categories confers on applicants, my report analyzed the effect of race on an applicant pool without these special categories of applicants (the baseline dataset), which allowed me to test for the effect of race on the bulk of the applicant pool that did not fall into one of these categories.2

Professor Card, however, includes all of these applicants in his model, taking the remarkable position that there is no penalty against Asian-American applicants unless Harvard imposes a penalty on every Asian-American applicant. But this is an untenable position. I do not assert that Harvard uses race to penalize Asian-American applicants who are recruited athletes, children of donors (or others identified on the Dean’s List), legacies, or other preferred categories. By including these special recruiting categories in his models, Professor Card obscures the extent to which race is affecting admissions decisions for all other applicants.

Professor Card further exacerbates this problem by including in his calculations the large majority of applicants whose characteristics guarantee rejection regardless of their race. Harvard admits a tiny fraction of applicants – only five or six percent in recent years. This means that a huge proportion of applicants have no realistic chance of admission. If an applicant has no chance of admission, regardless of his race, then Harvard obviously does not “discriminate” based on race in rejecting that applicant. Professor Card uses this obvious fact to assert that Harvard does not consider race at all in most of its admissions decisions. Further, he constructs his models in ways that give great weight to these applicants, again watering down the effect of race in Harvard’s decisions where it clearly does matter. (To put it in simple terms, it is akin to reducing the value of a fraction by substantially increasing the size of its denominator.)


Professor Card removes interaction terms, which has the effect of understating the penalty Harvard imposes on Asian-American applicants.

As Professor Card notes, his model differs from mine in that he removes the interaction terms. An interaction term allows the effects of a particular factor to vary with another distinct factor. In the context of racial discrimination, interaction terms are especially helpful (and often necessary) in revealing where certain factors operate differently for subgroups within a particular racial or ethnic group. For example, if a law firm singled out African-American women for discriminatory treatment but treated African-American males and other women fairly, a regression model would probably not pick up the discrimination unless it included an interaction between African-American and female.

Professor Card rightly recognizes that interaction terms should be included in a model when there is evidence that racial preferences operate differently for particular groups of applicants; yet he nonetheless removes interaction terms for variables that satisfy this condition. The most egregious instance of this is Professor Card’s decision not to interact race with disadvantaged status—even though the data clearly indicate that Harvard treats disadvantaged students differently by race.

...

Professor Card’s report changes none of my conclusions; to the contrary, given how easy it is to alter the results of his models and that my own models report the same results even incorporating a number of his controls, my opinions in this case have only been strengthened: Harvard penalizes Asian-American applicants; Harvard imposes heavy racial preferences in favor of Hispanic and African-American applicants; and Harvard has been manipulating its admission of single-race African-American applicants to ensure their admission rate approximates or exceeds the overall admission rate. Professor Card has demonstrated that it is possible to mask the true effects of race in Harvard’s admission process by changing the scope of the analysis in incorrect ways and choosing inappropriate combinations of control variables. But Professor Card cannot reach these results by applying accepted statistical methods and treating the data fairly.

Big Ed on Classical and Quantum Information Theory

I'll have to carve out some time this summer to look at these :-) Perhaps on an airplane...

When I visited IAS earlier in the year, Witten was sorting out Lieb's (nontrivial) proof of strong subadditivity. See also Big Ed.

A Mini-Introduction To Information Theory
https://arxiv.org/abs/1805.11965

This article consists of a very short introduction to classical and quantum information theory. Basic properties of the classical Shannon entropy and the quantum von Neumann entropy are described, along with related concepts such as classical and quantum relative entropy, conditional entropy, and mutual information. A few more detailed topics are considered in the quantum case.

Notes On Some Entanglement Properties Of Quantum Field Theory
https://arxiv.org/abs/1803.04993

These are notes on some entanglement properties of quantum field theory, aiming to make accessible a variety of ideas that are known in the literature. The main goal is to explain how to deal with entanglement when – as in quantum field theory – it is a property of the algebra of observables and not just of the states.

Years ago at Caltech, walking back to Lauritsen after a talk on quantum information, with John Preskill and a famous string theorist not to be named. When I asked the latter what he thought of the talk, he laughed and said Well, after all, it's just linear algebra :-)

The Life of this World

From this 2011 post:

I've been a fan of the writer James Salter (see also here) since discovering his masterpiece A Sport and a Pastime. Salter evokes Americans in France as no one since Hemingway in A Moveable Feast. The title comes from the Koran: Remember that the life of this world is but a sport and a pastime ... :-)

I can't think of higher praise than to say I've read every bit of Salter's work I could get my hands on.

For true Salter fans, a new (2017; he passed in 2015) collection of previously uncollected nonfiction: Don't Save Anything: Uncollected Essays, Articles, and Profiles. I especially liked the essay Younger Women, Older Men, originally published in Esquire in 1992.

From A Sport and a Pastime.

“When did you get out of Yale?”
“I didn’t,” he says. “I quit.”
“Oh.”

He describes it casually, without stooping to explain, but the authority of the act overwhelms me. If I had been an underclassman he would have become my hero, the rebel who, if I had only had the courage, I might have also become. ... Now, looking at him, I am convinced of all I missed. I am envious. Somehow his life seems more truthful than mine, stronger, even able to draw mine to it like the pull of a dark star.

He quit. It was too easy for him, his sister told me, and so he refused it. He had always been extraordinary in math. He had a scholarship. He knew he was exceptional. Once he took the anthropology final when he hadn’t taken the course. He wrote that at the top of the page. His paper was so brilliant the professor fell in love with him. Dean was disappointed, of course. It only proved how ridiculous everything was. ... He lived with various friends in New York and began to develop a style. ... in the end he quit altogether. Then he began educating himself.

...

She stoops with the match, inserts it, and the heater softly explodes. A blue flame rushes across the jets, then burns with a steady sound. There’s no other light in the room but this, which reflects from the floor. She stands up again. She drops the burnt match on the table and begins to arrange clothing on the grill of the heater, pajamas, spreading them out so they can be warmed. Dean helps her a bit. The silk, if it’s that, is quite cold. And there, back from the Vox opposite the Citroen garage, its glass doors now closed, they stand in the roaring dark. In a fond, almost brotherly gesture, he puts his arms around her. They hardly know one another. She accepts it without a word, without a movement, and they wait in a pure silence, the faint sweetness of gas in the air. After a while she turns the pajamas over. Her back is towards him. In a single move she pulls off her sweater and then, reaching behind herself in that elbow-awkward way, unfastens her brassiere. Slowly he turns her around.

...

From a message to a friend, who knew Salter, and asked me to articulate what I most admire about his work.

About 5 years ago I became friends with the writer Richard Ford, who offered to introduce me to his friend Salter. I was less enthusiastic to meet him than I would have been when he was younger. I did not go out of my way, and we never met.

Since he lived in Aspen, and I was often there in the summers at the Physics institute, I have sometimes imagined that we crossed paths without knowing it.

I admire, of course, his prose style. Sentence for sentence, he is the master.

But perhaps even more I admire his view of the world -- of courage, honor, daring to attempt the impossible, men and women, what is important in life.

The Rise of AI (Bloomberg Hello World documentary)

Great profile of Geoff Hinton, Yoshua Bengio, etc., but covers many other topics.

Note to readers: I'll be at the 35th International Conference on Machine Learning (ICML 2018) in Stockholm, Sweden (July 10-15, 2018), giving a talk at the Reproducibility in ML Workshop.

Let me know if you want to meet up!