Moore's Law and AI

By now you've probably heard that Moore's Law is really dead. So dead that the semiconductor industry roadmap for keeping it on track has more or less been abandoned: see, e.g., here, here or here. (Reported on this blog 2 years ago!)

What I have not yet seen discussed is how a significantly reduced rate of improvement in hardware capability will affect AI and the arrival of the dreaded (in some quarters) Singularity. The fundamental physical problems associated with ~ nm scale feature size could take decades or more to overcome. How much faster are today's cars and airplanes than those of 50 years ago?

Hint to technocratic planners: invest more in physicists, chemists, and materials scientists. The recent explosion in value from technology has been driven by physical science -- software gets way too much credit. From the former we got a factor of a million or more in compute power, data storage, and bandwidth. From the latter, we gained (perhaps) an order of magnitude or two in effectiveness: how much better are current OSes and programming languages than Unix and C, both of which are ~50 years old now?

HLMI = ‘high–level machine intelligence’ = one that can carry out most human professions at least as well as a typical human. (From Minds and Machines.)

Of relevance to this discussion: a big chunk of AlphaGo's performance improvement over other Go programs is due to raw compute power (link via Jess Riedel). The vertical axis is ELO score. You can see that without multi-GPU compute, AlphaGo has relatively pedestrian strength.

ELO range 2000-3000 spans amateur to lower professional Go ranks. The compute power certainly affects depth of Monte Carlo Tree Search. The initial training of the value and policy neural networks using KGS Go server positions might have still been possible with slower machines, but would have taken a long time.

Trump on Trump

Someone sent me a link to this 1990 Playboy interview with Donald Trump. There's much more in the interview than what I have excerpted below. See also Crazy like a fox.

What satisfaction, exactly, do you get out of doing a deal? I love the creative process. I do what I do out of pure enjoyment. Hopefully, nobody does it better. There's a beauty to making a great deal. It's my canvas. And I like painting it. I like the challenge and tell the story of the coal miner's son. The coal miner gets black-lung disease, his son gets it, then his son. If I had been the son of a coal miner, I would have left the damn mines. But most people don't have the imagination-or whatever-to leave their mine. They don’t have "it."

Which is? "It" is an ability to become an entrepreneur, a great athlete, a great writer. You're either born with it or you're not. Ability can be honed, perfected or neglected. The day Jack Nicklaus came into this world, he had more innate ability to play golf than anybody else.

Do you suppose your children inherited "it" from you? Statistically, my children have a very bad shot. Children of successful people are generally very, very troubled, not successful. They don't have the right shtick. You never know until they're tested. But I do well with my children.

Do you think they will have to make it? I would love them to be in business with me, but ninety-five percent of those children fail in a sophisticated big business. It takes confidence, intelligence, shtick. If any one of these traits is missing, you're not going to make it.

Your older brother, Fred, who died from heart failure brought on by acute alcoholism, had a more difficult time with him [Trump's father], didn't he? Take one environment and it will work completely differently on different children. Our family environment, the competitiveness, was a negative for Fred. It wasn't easy for him being cast in a very tough environment, and I think it played havoc on him. I was very close to him and it was very sad when he died . . . toughest situation I've had ...

What did you learn from his experience? [Pauses] Nobody has ever asked me that. But his death affected everything that has come after it. ... I think constantly that I never really gave him thanks for it. He was the first Trump boy out there, and I subconsciously watched his moves.

And the lesson? I saw people really taking advantage of Fred and the lesson I learned was always to keep up my guard one hundred percent, whereas he didn't. He didn't feel that there was really reason for that, which is a fatal mistake in life. People are too trusting. I'm a very untrusting guy. I study people all the time, automatically; it's my way of life, for better or worse.

Why? I am very skeptical about people; that's self-preservation at work. I believe that, unfortunately, people are out for themselves. At this point, it's to many people’s advantage to like me. Would the phone stop ringing, would these people kissing ass disappear if things were not going well? I enjoy testing friendship .... Everything in life to me is a psychological game, a series of challenges you either meet or don't. I am always testing people who work for me.

How? I will send people around to my buyers to test their honesty by offering them trips and other things. I've been surprised that some people least likely to accept a trip from a contractor did and some of the most likely did not. You can never tell until you test; the human species is interesting in that way. So to me, friendship can be really tested only in bad times. I instinctively mistrust many people. It is not a negative in my life but a positive. Playboy wouldn't be talking to me today if I weren't a cynic. So I learned that from Fred, and I owe him a lot. . . . He could have ultimately been a happy guy, but things just went the unhappy way.

How large a role does pure ego play in your deal making and enjoyment of publicity? Every successful person has a very large ego.

Every successful person? Mother Teresa? Jesus Christ? Far greater egos than you will ever understand.

A favorite word of yours, tough. How do you define it? Tough is being mentally capable of winning battles against an opponent and doing it with a smile. Tough is winning systematically.

Life? Or death? Both. We're here and we live our sixty, seventy or eighty years and we’re gone. You win, you win, and in the end, it doesn't mean a hell of a lot. But it is something to do-to keep you interested.

Do you agree with the T-shirt that says, WHOEVER HAS THE MOST TOYS WINS? Depends on your definition of winning. Some of my friends are unbelievably successful and miserable people. I truly believe that someone successful is never really happy, because dissatisfaction is what drives him. I've never met a successful person who wasn't neurotic. It's not a terrible thing ... it's controlled neuroses.

Do you think George Bush is soft? I like George Bush very much and support him and always will. But I disagree with him when he talks of a kinder, gentler America. I think if this country gets any kinder or gentler, it's literally going to cease to exist. I think if we had people from the business community-the Carl Icahns, the Ross Perots-negotiating some of our foreign policy, we'd have respect around the world.

You categorically don't want to be President? I don't want to be President. I’m one hundred percent sure. I'd change my mind only if I saw this country continue to go down the tubes.

Postdoc Position

Please help me fill this position! This search is a bit out of sync with the regular postdoc application process, so I need some help spreading the word.

Theoretical Physics Postdoc at Michigan State University 

Stephen Hsu, Vice-President for Research and Professor of Physics at MSU, anticipates filling a postdoctoral position to start in the summer or fall of 2016. The successful applicant will have broad interests in theoretical physics and good computational skills. In addition to research in particle physics and cosmology, there will be opportunities to work on problems in machine learning and computational genomics.

The High Energy Theory group at MSU currently consists of eight faculty members: Sekhar Chivukula, Jon Pumplin, Wayne Repko, Carl Schmidt, Elizabeth Simmons, Dan Stump, C.-P. Yuan and Stephen Hsu, as well as postdoctoral fellows and several graduate students. Ongoing research encompasses QCD theory and phenomenology, electroweak symmetry breaking mechanisms, supersymmetry and other beyond-the-standard-model scenarios, cosmology, and collider phenomenology. Recently, a new group of 3 theorists have been hired in the area of lattice QCD. The Physics/Astronomy Department at MSU has 60 faculty members; it has strong research programs in Condensed Matter Physics, Nuclear Physics, and Astronomy, in addition to High Energy Physics (http://www.pa.msu.edu/hep/hept.html).

See MSU Applicant Page http://www.hr.msu.edu/hiring/msujobs.htm , posting 2859 (PA). Applications should be uploaded to MSU’s online job application site, https://jobs.msu.edu and should include a CV, research plan and publication list. In addition, three letters of recommendation should be submitted electronically by the recommenders through this application system. Review of applications will begin immediately and will continue until the position is filled. MSU is an affirmative action, equal opportunity employer MSU is committed to achieving excellence through cultural diversity. The university actively encourages applications and/or nominations of women, persons of color, veterans and persons with disabilities.

Crazy like a fox

Trump Unstoppable? Matt Taibbi writes in Rolling Stone. (Does Rolling Stone still exist? It's only a matter of time as the lawsuit(s) over the UVA fraternity rape hoax work their way through the system.)

See also Trump: the Master Persuader and American and Chinese Oligarchies.

How America Made Donald Trump Unstoppable: ... It turns out we let our electoral process devolve into something so fake and dysfunctional that any half-bright con man with the stones to try it could walk right through the front door and tear it to shreds on the first go.

And Trump is no half-bright con man, either. He's way better than average.

... in an insane twist of fate, this bloated billionaire scion has hobbies that have given him insight into the presidential electoral process. He likes women, which got him into beauty pageants. And he likes being famous, which got him into reality TV. He knows show business.

That put him in position to understand that the presidential election campaign is really just a badly acted, billion-dollar TV show whose production costs ludicrously include the political disenfranchisement of its audience. Trump is making a mockery of the show, and the Wolf Blitzers and Anderson Coopers of the world seem appalled. How dare he demean the presidency with his antics?

But they've all got it backward. The presidency is serious. The presidential electoral process, however, is a sick joke, in which everyone loses except the people behind the rope line. And every time some pundit or party spokesman tries to deny it, Trump picks up another vote.

... Trump's basic argument is the same one every successful authoritarian movement in recent Western history has made: that the regular guy has been screwed by a conspiracy of incestuous elites. The Bushes are half that conspiratorial picture, fronts for a Republican Party establishment and whose sum total of accomplishments, dating back nearly 30 years, are two failed presidencies, the sweeping loss of manufacturing jobs, and a pair of pitiable Middle Eastern military adventures – the second one achieving nothing but dead American kids and Junior's re-election.

[ CAN YOU BLAME WORKING CLASS AMERICANS FOR FEELING THIS WAY? When a majority of citizens disagrees with economic elites and/or with organised interests, they generally lose.  ]

... Trump picked on Jeb because Jeb is a symbol. The Bushes are a dissolute monarchy, down to offering their last genetic screw-up to the throne.

Jeb took the high road for most of the past calendar year, but Trump used his gentlemanly dignity against him. What Trump understands better than his opponents is that NASCAR America, WWE America, always loves seeing the preening self-proclaimed good guy get whacked with a chair. In Greenville, Trump went after Jeb this time on the issue of his brother's invasion of Iraq.

"The war in Iraq was a big f ... fat mistake, all right?" he snorted. He nearly said, "A big fucking mistake." He added that the George W. Bush administration lied before the war about Iraq having WMDs and that we spent $2 trillion basically for nothing.

[ WHO CAN DISAGREE? ]

... Reporters have focused quite a lot on the crazy/race-baiting/nativist themes in Trump's campaign, but these comprise a very small part of his usual presentation. His speeches increasingly are strikingly populist in their content.

His pitch is: He's rich, he won't owe anyone anything upon election, and therefore he won't do what both Democratic and Republican politicians unfailingly do upon taking office, i.e., approve rotten/regressive policies that screw ordinary people.

[ VOTERS ARE NOT RATIONAL, AND TRUMP KNOWS IT. ]

...Cheryl Donlon says she heard the tariff message loud and clear and she's fine with it, despite the fact that it clashes with traditional conservatism.

"We need someone who is just going to look at what's best for us," she says.

I mention that Trump's plan is virtually identical to Dick Gephardt's idea from way back in the 1988 Democratic presidential race, to fight the Korean Hyundai import wave with retaliatory tariffs.

Donlon says she didn't like that idea then.

Why not?

"I didn't like him," she says.

[ TRUMP WILL EAT HILLARY ALIVE. ]

... At a Democratic town hall in Derry, New Hampshire, Hillary's strangely pathetic answer about why she accepted $675,000 from Goldman to give speeches – "That's what they offered" – seemed doomed to become a touchstone for the general-election contest. Trump would go out on Day One of that race and blow $675,000 on a pair of sable underwear, or a solid-gold happy-face necktie. And he'd wear it 24 hours a day, just to remind voters that his opponent sold out for the Trump equivalent of lunch money.

... The triumvirate of big media, big donors and big political parties has until now successfully excluded every challenge to its authority. But like every aristocracy, it eventually got lazy and profligate, too sure it was loved by the people. It's now shocked that voters in depressed ex-factory towns won't keep pulling the lever for "conservative principles," or that union members bitten a dozen times over by a trade deal won't just keep voting Democratic on cue.

Trump isn't the first rich guy to run for office. But he is the first to realize the weakness in the system, which is that the watchdogs in the political media can't resist a car wreck. The more he insults the press, the more they cover him: He's pulling 33 times as much coverage on the major networks as his next-closest GOP competitor, and twice as much as Hillary.

Trump found the flaw in the American Death Star. It doesn't know how to turn the cameras off, even when it's filming its own demise. ...

Behold, Atlas

5 foot 9 180 lbs of robot genius :-)

DeepMind and Demis Hassabis

Recent profile in the Guardian; 15 facts about Hassabis. The mastery of Atari games through reinforcement learning deep neural nets is described here (Nature). See also Deep Neural Nets and Go: AlphaGo beats European champion.

Guardian: ... “We’re really lucky,” says Hassabis, who compares his company to the Apollo programme and Manhattan Project for both the breathtaking scale of its ambition and the quality of the minds he is assembling at an ever increasing rate. “We are able to literally get the best scientists from each country each year. So we’ll have, say, the person that won the Physics Olympiad in Poland, the person who got the top maths PhD of the year in France. We’ve got more ideas than we’ve got researchers, but at the same time, there are more great people coming to our door than we can take on. So we’re in a very fortunate position. The only limitation is how many people we can absorb without damaging the culture.”

That culture goes much deeper than beanbags, free snacks and rooftop beers. Insisting that the Google acquisition has not in any way forced him to deviate from his own research path, Hassabis reckons he spends “at least as much time thinking about the efficiency of DeepMind as the algorithms“ and describes the company as “a blend of the best of academia with the most exciting start-ups, which have this incredible energy and buzz that fuels creativity and progress.” He mentions “creativity” a lot, and observes that although his formal training has all been in the sciences, he is “naturally on the creative or intuitive” side. “I’m not, sort of, a standard scientist,” he remarks, apparently without irony. Vital to the fabric of DeepMind are what he calls his “glue minds”: fellow polymaths who can sufficiently grasp myriad scientific areas to “find the join points and quickly identify where promising interdisciplinary connections might be, in a sort of left-field way.” Applying the right benchmarks, these glue people can then check in on working groups every few weeks and swiftly, flexibly, move around resources and engineers where required. “So you’ll have one incredible, genius researcher and almost immediately, unlike in academia, three or four other people from a different area can pick up that baton and add to it with their own brilliance,” he describes. “That can result in incredible results happening very quickly.” The AlphaGo project, launched just 18 months ago, is a perfect case in point.

... “just thinking time. Until three or four in the morning, that’s when I do my thinking: on research, on our next challenge, or I’ll write up an algorithmic design document.” ... It’s not so much actual AI coding, he admits, “because my maths is too rusty now.  [ Quel dommage! ]  It’s more about intuitive thinking. Or maybe strategic thinking about the company: how to scale it and manage that. Or it might just be something I read in an article or saw on the news that day, wondering how our research could connect to that.”

See also Don’t Worry, Smart Machines Will Take Us With Them: Why human intelligence and AI will co-evolve.

Missing Heritability and GCTA: Update on PNAS dispute

GCTA is a statistical method for estimating the heritability of a complex trait using (phenotype | genotype) data from unrelated individuals. It has been applied to many human phenotypes, including disease conditions and behavioral traits. GCTA results tend to be consistent with earlier twin and family studies of heritability, and suggest that significant heritability is due to common genetic variants that will be identified in the future through increased statistical power (sample size).

A recent PNAS paper by researchers at Stanford claims to identify many problems with GCTA. The conclusions of this paper have been hotly contested by the GCTA authors and others.

Earlier post (January 1, 2016) on PNAS paper Limitations of GCTA as a solution to the missing heritability problem. (See also: many posts on this blog which mention GCTA.)

Detailed comments and analysis here and here by Sasha Gusev. Gusev claims that the problems identified in Figs 4,7 are the result of incorrect calculation of the SE (4) and failure to exclude related individuals in the Framingham data (7).

GCTA authors Visscher, Yang, et al. respond to PNAS paper -- they accept none of the criticisms (February 13, 2016 biorxiv).

PNAS authors reply to Visscher, Yang, et al. comments (February 16, 2016 bioarxiv). They claim that relatedness thresholding used with GCTA analysis is flawed and that residual standard errors are much larger than claimed.

Gamazon and Park (February 18, 2016 bioarxiv) question spectral analysis and random matrix theory results in the PNAS paper. (I believe this is the first critique which looks at the mathematics of the PNAS paper, as opposed to simulation results.)

This dispute shows the utility of blogs (Gusev) and biorxiv for rapid scientific discussion. Some of the commentaries listed above are 20+ pages long with figures and equations. This discussion would not have been possible (or would have taken months or years) in a journal setting.

The next step should be a mini-workshop conducted online, with each group allowed 30 min to present their results, followed by questions :-)


I've always felt that the real weakness of GCTA is the assumption of random effects. A consequence of this assumption is that if the true causal variants are atypical (e.g., in terms of linkage disequilibrium) among common SNPs, the results could be biased. It is impossible to evaluate this uncertainty at the moment because we do not yet know the genetic architectures of any complex traits. See Why does GCTA work? for more discussion and a link to work by Lee and Chow examining this issue.

Recently, a promising new method (Heritability Estimates from Summary Statistics) has been proposed which does not make assumptions about the effect size distribution -- it uses GWAS estimates of effect size to directly estimate variance accounted for by each region of the genome. The initial application of this method also suggests significant heritability due to common variants.

The broader debate over whether common variants will eventually account for significant heritability in many complex traits has been going on for years now. The centrality of GCTA results to this question decreases by the year as more and more heritability is accounted for by specific loci identified at genome wide significance in well-powered GWAS. For example, this slide (see talk on genomic prediction I gave in 2015 at NIH and HLI) shows that GWAS hits on height now account for 16% of total variance. That means a predictor could be constructed with correlation ~0.4 to the actual trait. I think the argument is basically over, unless you have some ulterior motive for denying the potential of genomic prediction.

Elite schools, birthright, and credentials

In mixing together the truly talented with the rich and powerful, elite US universities perform a useful service to both groups.

Khan is discussing themes related to his book Privilege: The Making of an Adolescent Elite at St. Paul's School.

See also Credentialism and elite employment , Credentialism and elite performance, and Defining Merit.

Bloomberg View: Save Us From The Ivy League Oligarchy.

NIH peer review percentile scores are poorly predictive of grant productivity

The impacts of studies ranked in the 3rd to 20th percentile are more or less statistically indistinguishable. With current funding lines as low as 10th percentile, this means that many unfunded proposals are better than funded studies.

NIH peer review percentile scores are poorly predictive of grant productivity
DOI: 10.7554/eLife.13323.001

Peer review is widely used to assess grant applications so that the highest ranked applications can be funded. A number of studies have questioned the ability of peer review panels to predict the productivity of applications, but a recent analysis of grants funded by the National Institutes of Health (NIH) in the US found that the percentile scores awarded by peer review panels correlated with productivity as measured by citations of grant-supported publications. Here, based on a re-analysis of these data for the 102,740 funded grants with percentile scores of 20 or better, we report that these percentile scores are a poor discriminator of productivity. This underscores the limitations of peer review as a means of assessing grant applications in an era when typical success rates are often as low as about 10%.

Free Harvard, Fair Harvard: Enrollment Trends

The graph below shows changes in the number of Harvard students by ethnic group, relative to the total college age US population of that group (Harvard Enrollment Per Capita = HEPC). Over the last 20 years, the Asian American HEPC has declined by almost 60%. Unless Asian American applicants to Harvard have, on average, declined significantly in relative quality (anecdotal evidence suggests that is far from true), we are left with a mystery: Why has Asian American HEPC declined so precipitously?

Only the innumerate can fail to be intrigued (alarmed? offended?) by this simple observation.

Some caveats: both numerator and denominator for HPEC are difficult to determine. The former comes from NCES data (National Center for Education Statistics), self-reported by universities. The denominator comes from Census bureau Current Population Survey data, and is a bit noisy year to year. I doubt we can trust the HEPC number from one year to the next, but the 20 year trend is probably roughly reliable.

The Economist covered this topic recently in an article entitled The model minority is losing patience. From their chart, one can see that the HEPC mystery extends to the rest of the Ivy League: Asian American enrollment at the Ivies has mysteriously converged at around 15-20%, despite the huge growth in college age Asian American population over the last 20 years.

Caltech is the one school among those in the graph which explicitly declines to use race as a preference (or penalty) in admissions. Caltech has the academically strongest student body and its alumni win more Nobel Prizes and major science and technology awards per capita than any other school.

If you are a Harvard degree holder, I urge you to vote for the Free Harvard, Fair Harvard slate in the coming Overseer elections this spring. We are asking for greater transparency in Harvard admissions, which would help resolve the HEPC mystery discussed above.

See also 20 years @15 percent: does Harvard discriminate against Asian-Americans?

The historical parallels with anti-semitic practices of the early 20th century are reviewed in detail:

... In a letter to the chairman of the committee, President Lowell wrote that “questions of race,” though “delicate and disagreeable,” were not solved by ignoring them. The solution was a new admissions system giving the school wide discretion to limit the admission of Jewish applicants: “To prevent a dangerous increase in the proportion of Jews, I know at present only one way which is at the same time straightforward and effective, and that is a selection by a personal estimate of character on the part of the Admissions authorities ... The only way to make a selection is to limit the numbers, accepting those who appear to be the best.”

... The reduction in Jewish enrollment at Harvard was immediate. The Jewish portion of Harvard’s entering class dropped from over 27 percent in 1925 to 15 percent the following year. For the next 20 years, this percentage (15 percent) remained virtually unchanged.

... The new policy permitted the rejection of scholastically brilliant students considered “undesirable,” and it granted the director of admissions broad latitude to admit those of good background with weaker academic records. The key code word used was “character” — a quality thought to be frequently lacking among Jewish applicants, but present congenitally among affluent Protestants.

Epistasis and Complex Traits

Short summary: To first approximation we can ignore gene-gene interactions in the prediction of complex traits. This paper examines specifically how non-additive variance is driven to zero as the number of loci involved becomes large, assuming some dispersion in allele frequencies.

Earlier paper of Hill, Goddard, and Visscher and the simpler 2 locus case is discussed here.

Influence of Gene Interaction on Complex Trait Variation with Multilocus Models

Asko Mäki-Tanila, William G. Hill
GENETICS September 18, 2014 vol. 198 no. 1 355-367; DOI: 10.1534/genetics.114.165282

Although research effort is being expended into determining the importance of epistasis and epistatic variance for complex traits, there is considerable controversy about their importance. Here we undertake an analysis for quantitative traits utilizing a range of multilocus quantitative genetic models and gene frequency distributions, focusing on the potential magnitude of the epistatic variance. All the epistatic terms involving a particular locus appear in its average effect, with the number of two-locus interaction terms increasing in proportion to the square of the number of loci and that of third order as the cube and so on. Hence multilocus epistasis makes substantial contributions to the additive variance and does not, per se, lead to large increases in the nonadditive part of the genotypic variance. Even though this proportion can be high where epistasis is antagonistic to direct effects, it reduces with multiple loci. As the magnitude of the epistatic variance depends critically on the heterozygosity, for models where frequencies are widely dispersed, such as for selectively neutral mutations, contributions of epistatic variance are always small. Epistasis may be important in understanding the genetic architecture, for example, of function or human disease, but that does not imply that loci exhibiting it will contribute much genetic variance. Overall we conclude that theoretical predictions and experimental observations of low amounts of epistatic variance in outbred populations are concordant. It is not a likely source of missing heritability, for example, or major influence on predictions of rates of evolution.

See also Determination of Nonlinear Genetic Architecture using Compressed Sensing, and related posts on epistasis and additivity.

LIGO detects gravity waves

Live-blogging the LIGO announcement of detection of gravity waves. Detection of an event in 2015 (initial science run of advanced LIGO) is good news for the future use of gravity waves as an astrophysical probe -- it suggests a fairly high density of NS-NS, NS-BH, and BH-BH binaries in the universe. Each time astronomers have developed a new probe (radio waves, x-rays, etc.) they have discovered new cosmic phenomena. The future is promising!

Techno-pessimists should note that detecting gravity waves is much, much harder than landing on the moon. LIGO measured a displacement 1/1000 of a neutron radius, in a noisy terrestrial background, accounting even for quantum noise.

https://www.ligo.caltech.edu/: 9/14/15 detection of BH-BH (~ 30 solar masses) merger at distance 1.3 Gy. The energy in the gravitational wave signal was ~3 solar masses!

Here is the paper  http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.116.061102

When I was an undergraduate, I toured the early LIGO prototype, which was using little car shaped rubber erasers as shock absorbers. Technology has improved since then, and the real device is much bigger.

Kip Thorne (from whom I learned General Relativity) has been one of the driving forces behind the effort to detect gravity waves for over 40 years. The picture below was taken during a conference in Eugene back in 2005.

Free Harvard, Fair Harvard: Freeharvard.org

We received confirmation from Harvard today that our petitions have been accepted and that our slate will appear on the Overseer ballots. For more information see FreeHarvard.org.

Our slate: Ralph Nader, Ron Unz, Lee Cheng, Stuart Taylor Jr., and myself.

Our platform:
1. More transparency in Harvard admissions
2. Increased use of endowment income to make Harvard more accessible

We are NOT conservative extremists: I voted twice for Obama, as did others on our slate.

We are NOT running against Affirmative Action. I support moderate admissions preferences, and I support diversity on campus. However, I am against preferences which are so large that they make it unlikely that the recipient of the preference can succeed in challenging courses at the university. Any admissions system has to be studied carefully to understand its consequences, and Harvard's is no exception.

Slate Star Codex on Superforecasting

Scott Alexander (Slate Star Codex) on Philip Tetlock's Superforecasting:

Book review
Highlighted passages.

I especially liked this passage that Scott highlights:

When hospitals created cardiac care units to treat patients recovering from heart attacks, Cochrane proposed a randomized trial to determine whether the new units delivered better results than the old treatment, which was to send the patient home for monitoring and bed rest. Physicians balked. It was obvious the cardiac care units were superior, they said, and denying patients the best care would be unethical. But Cochrane was not a man to back down…he got his trial: some patients, randomly selected, were sent to the cardiac care units while others were sent home for monitoring and bed rest. Partway through the trial, Cochrane met with a group of the cardiologists who had tried to stop his experiment. He told them that he had preliminary results. The difference in outcomes between the two treatments was not statistically signficant, he emphasized, but it appeared that patients might do slightly better in the cardiac care units. “They were vociferous in their abuse: ‘Archie,’ they said, ‘we always thought you were unethical. You must stop the trial at once.'” But then Cochrane revealed he had played a little trick. He had reversed the results: home care had done slightly better than the cardiac units. “There was dead silence and I felt rather sick because they were, after all, my medical colleagues.”

[ Cochrane Collaboration ]    [ Bounded Cognition ]

See also Medical Science?

Black Mirror

The best bit, with Jon Hamm, starts @29min :-)




Recommended by Auren Hoffman.

Trump: the Master Persuader

Dilbert creator Scott Adams on Trump -- the Master Persuader :-)

Donald Trump has a way with words—and with people. Yet despite his popularity, he has been a mystery to the media, which have mostly derided his campaign as consisting of nothing more than random insults and ignorant bluster.

Scott Adams, prolific author, blogger, and creator of the massively popular comic strip Dilbert, has a different theory. He tells Reason TV's Zach Weissmueller that the media are being trolled by a skilled manipulator, or in Adams's parlance, a Master Wizard. So exquisite does Adams believe Trump's skills to be that he predicts The Donald will go on to win the presidency.

Improved CRISPR–Cas9: Safe and Effective?

Two groups (Zhang lab at MIT and Joung lab at Harvard) announce improved "engineered" Cas9 variants with reduced off-target editing rates while maintaining on-target effectiveness. I had heard rumors about this but now the papers are out. See CRISPR: Safe and Effective?

Nature commentary Genome Editing: The domestication of Cas9.

High-fidelity CRISPR–Cas9 nucleases with no detectable genome-wide off-target effects

Nature 529, 490–495 (28 January 2016) doi:10.1038/nature16526

CRISPR–Cas9 nucleases are widely used for genome editing but can induce unwanted off-target mutations. Existing strategies for reducing genome-wide off-target effects of the widely used Streptococcus pyogenes Cas9 (SpCas9) are imperfect, possessing only partial or unproven efficacies and other limitations that constrain their use. Here we describe SpCas9-HF1, a high-fidelity variant harbouring alterations designed to reduce non-specific DNA contacts. SpCas9-HF1 retains on-target activities comparable to wild-type SpCas9 with >85% of single-guide RNAs (sgRNAs) tested in human cells. Notably, with sgRNAs targeted to standard non-repetitive sequences, SpCas9-HF1 rendered all or nearly all off-target events undetectable by genome-wide break capture and targeted sequencing methods. Even for atypical, repetitive target sites, the vast majority of off-target mutations induced by wild-type SpCas9 were not detected with SpCas9-HF1. With its exceptional precision, SpCas9-HF1 provides an alternative to wild-type SpCas9 for research and therapeutic applications. More broadly, our results suggest a general strategy for optimizing genome-wide specificities of other CRISPR-RNA-guided nucleases.


Rationally engineered Cas9 nucleases with improved specificity

Science 01 Jan 2016: Vol. 351, Issue 6268, pp. 84-88
DOI: 10.1126/science.aad5227

The RNA-guided endonuclease Cas9 is a versatile genome-editing tool with a broad range of applications from therapeutics to functional annotation of genes. Cas9 creates double-strand breaks (DSBs) at targeted genomic loci complementary to a short RNA guide. However, Cas9 can cleave off-target sites that are not fully complementary to the guide, which poses a major challenge for genome editing. Here, we use structure-guided protein engineering to improve the specificity of Streptococcus pyogenes Cas9 (SpCas9). Using targeted deep sequencing and unbiased whole-genome off-target analysis to assess Cas9-mediated DNA cleavage in human cells, we demonstrate that “enhanced specificity” SpCas9 (eSpCas9) variants reduce off-target effects and maintain robust on-target cleavage. Thus, eSpCas9 could be broadly useful for genome-editing applications requiring a high level of specificity.

These are the days of miracle and wonder!

Deep Neural Nets and Go: AlphaGo beats European champion

I'm surprised that this happened so fast. I guess I need to update some priors :-)

AlphaGo uses two neural nets: one for move selection ("policy") and the other for position evaluation ("value"), but also uses MC search trees. Its strength is roughly top 1000 or so among all human players. In a few months it is scheduled to play one of the very best players in the world.

For training they used a 30 million position Go database of expert games (KGS Go Server). I have no intuition as to whether this is enough data to train the policy and value NNs. The quality of these NNs must be relatively good, as the MC tree search used was much smaller than for DeepBlue and its hand-crafted evaluation function.

Some grandmasters who reviewed AlphaGo's games were impressed by the "humanlike" quality of its play. More discussion: HNN, Reddit.

Mastering the game of Go with deep neural networks and tree search

Nature 529, 484–489 (28 January 2016) doi:10.1038/nature16961

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Schematic representation of the neural network architecture used in AlphaGo. The policy network takes a representation of the board position s as its input, passes it through many convolutional layers with parameters σ (SL policy network) or ρ (RL policy network), and outputs a probability distribution p (a|s) or p (a|s) over legal moves a, represented by a σρ probability map over the board. The value network similarly uses many convolutional layers with parameters θ, but outputs a scalar value vθ(s′) that predicts the expected outcome in position s′.


Related News: commenter STS points me to some work showing the equivalence of Deep Learning to the Renormalization Group in physics. See also Quanta magazine. The key aspect of RG here is the identification of important degrees of freedom in the process of coarse graining. These degrees of freedom make up so-called Effective Field Theories in particle physics.


These are the days of miracle and wonder!

Free Harvard, Fair Harvard: Let the Voters Decide!

The petitions are in! Thanks to everyone who worked so hard to collect ~300 signatures in just the past two weeks. It appears we'll be on the Overseer ballots sent to 300k Harvard degree holders.

Our slate: Ralph Nader, Ron Unz, Lee Cheng, Stuart Taylor Jr., and myself.

Our platform:
1. More transparency in Harvard admissions
2. Increased use of endowment income to make Harvard more accessible

We are NOT conservative extremists: I voted twice for Obama, as did others on our slate.

We are NOT running against Affirmative Action. I support moderate admissions preferences, and I support diversity on campus. However, I am against preferences which are so large that they make it unlikely that the recipient of the preference can succeed in challenging courses at the university. Any admissions system has to be studied carefully to understand its consequences, and Harvard's is no exception.

Barack Obama (1991 Harvard Overseer petition candidate): He [Steve] looks like someone who can do great things!

Press coverage:

New York Times 1/14/16 (front page)

Harvard Magazine 1/27/16 (9000 words by editor John Rosenberg)

Discussion with CNN contributor and prominent Asian American writer Jeff Yang:

SMPY at 50: Research Associate position

I'm posting the job ad below for David Lubinski. The Study of Mathematically Precocious Youth (SMPY) is the most systematic long term study of individuals of high cognitive ability since the Terman Study.

SMPY helps to establish a number of important facts about individuals of high ability:

1. We can (at least crudely) differentiate between individuals at the 99th, 99.9th and 99.99th percentiles. Exceptional talent can be identified through testing, even at age 13.

2. Probability of significant accomplishment, such as STEM PhD, patents awarded, tenure at leading research university, exceptional income, etc. continues to rise as ability level increases, even within the top 1%.

3. There are systematic differences in cognitive abilities and profiles in different fields (business, medicine, engineering, physics, etc.)

4. Men and women of exceptional ability differ in life aspirations and preferences.

No one can claim to understand high level human capital, technological innovation, scientific progress, or exceptional achievement without first familiarizing themselves with these results.

Needless to say, I think this Research Associate position will entail important and fascinating work.

Research Associate:

The Study of Mathematically Precocious Youth (SMPY) seeks a full-time post-doctoral Research Associate for study oversight, conducting research, writing articles, laboratory management, and statistical analyses using the vast SMPY data base. SMPY is a four-decade longitudinal study consisting of 5 cohorts and over 5,000 intellectually talented participants. One chief responsibility of this position will be to manage laboratory details associated with launching an age-50 follow-up of two of SMPY’s most exceptional cohorts: a cohort of 500 profoundly gifted participants initially identified by age 13 in the early 1980s, and a second cohort of over 700 top STEM graduate students identified and psychologically profiled in 1992 as first- and second-year graduate students. Candidates with interests in assessing individual differences, talent development, and particularly strong statistical-technical skills are preferred. Send vitae, cover letter stating interests, (pre)reprints, and three letters of recommendation to: Dean Camilla P. Benbow, Department of Psychology & Human Development, 0552 Peabody College, Vanderbilt University, Nashville, TN, 37203. The position will remain open until a qualified applicant is selected. For additional information, please contact either co-director: Camilla P. Benbow, camilla.benbow@vanderbilt.edu, or David Lubinski, david.lubinski@vanderbilt.edu.

http://www.vanderbilt.edu/Peabody/SMPY/. Vanderbilt University is an Equal Opportunity/Affirmative Action Employer.

We are aiming for a June 30th start date but that’s flexible.

Some relevant figures based on SMPY results of Lubinski, Benbow, and collaborators. See links above for more discussion of the data displayed.

Free Harvard, Fair Harvard: Harvard Magazine and CNN coverage

We are rapidly approaching the February 1 deadline for petition signatures supporting our Free Harvard, Fair Harvard (FHFH) campaign.

Two articles just appeared concerning the campaign. Harvard Magazine's Overseers Petitioners Challenge Harvard Policies contains a thorough and lengthy review of the issues. After a quick read, I have two specific comments.

1. The author seems unaware of research showing that SAT scores are predictive not just of freshman grades, but also of performance in upper division courses and beyond (for example, they predict GRE and LSAT scores obtained by seniors applying to graduate school). These are important facts underlying the mismatch hypothesis that should be clarified.

2. The article points out that

... Faculty of Arts and Sciences [FAS] ... owned $15.4 billion of the endowment, which was valued at $37.6 billion last June 30: about 41 percent. Approximately $2.5 billion (slightly less than 7 percent) of the endowment is presidential funds—the income from some of which may be directed to FAS and the College. But the remaining majority of endowment assets is owned by other schools or units, and presumably the income distributed from them is largely or completely unavailable to pay for undergraduate tuition ...

My understanding, as someone who has endowed a scholarship at Caltech, is that although gifts are typically made with specific restrictions, universities often take liberties when necessary with the returns on these gifts. For example: after the 2008 financial crisis the Caltech President unilaterally (without informing donors like myself) imposed a tax on endowment income in order to cover a shortfall in operating budget. Indeed, I cannot identify any legal restriction in the gift agreement that prevents Caltech from imposing such a tax. I imagine most of Harvard's endowment is in a similar situation.

The second article is a CNN opinion piece by noted author and journalist Jeff Yang. I had a long discussion with Jeff about FHFH, which we recorded just for fun.

For the record, my position on Affirmative Action: I am not against moderate preferences based on ethnicity. However, I am against preferences which are so large that they make it unlikely that the recipient of the preference can succeed in challenging courses at the university.

Autistic Monkeys via gene editing

[[ Flash video here ]]


See also De novo mutations and autism and Disruptive mutations and the genetic architecture of autism.

Nature News: The laboratory monkeys run obsessively in circles, largely ignore their peers and grunt anxiously when stared at. Engineered to have a gene that is related to autism spectrum disorder in people, the monkeys are the most realistic animal model of the condition yet, say their creators. Researchers hope that the animals will open up new ways to test treatments and investigate the biology of autism. But the jury is still out on how well the monkeys’ condition matches that of people with autism. 

Autism has a vast array of symptoms and types, but researchers think that at least 100 genes play a part. The scientists who led the latest work, which is published on 25 January in Nature (Z. Liu et al. Nature http://dx.doi.org/10.1038/nature16533; 2016), turned to the autism-related gene MECP2: both people who have extra copies of the gene (MECP2-duplication syndrome) and people who have certain mutations in this gene (Rett’s syndrome) share many of the symptoms of autism. Previously researchers have engineered monkeys to have autism-related genes (H. Liu et al. Cell Stem Cell 14, 323–328; 2014), but this is the first published demonstration of a link between those genes and the animals’ behaviour. 

... Qiu, meanwhile, is excited by the prospect of using the model to identify exactly where in the brain the MECP2 overexpression causes trouble. His team is already using brain-imaging technology on the monkeys to pinpoint such areas. Next, the researchers plan to use the CRISPR gene-editing technique to knock out the extra MECP2 copies in cells in those regions and then check whether the autisim-like symptoms stop.

Here's the paper:

Autism-like behaviours and germline transmission in transgenic monkeys overexpressing MeCP2 

Nature (2016) doi:10.1038/nature16533 

Methyl-CpG binding protein 2 (MeCP2) has crucial roles in transcriptional regulation and microRNA processing1, 2, 3, 4. Mutations in the MECP2 gene are found in 90% of patients with Rett syndrome, a severe developmental disorder with autistic phenotypes5. Duplications of MECP2-containing genomic segments cause the MECP2 duplication syndrome, which shares core symptoms with autism spectrum disorders6. Although Mecp2-null mice recapitulate most developmental and behavioural defects seen in patients with Rett syndrome, it has been difficult to identify autism-like behaviours in the mouse model of MeCP2 overexpression7, 8. Here we report that lentivirus-based transgenic cynomolgus monkeys (Macaca fascicularis) expressing human MeCP2 in the brain exhibit autism-like behaviours and show germline transmission of the transgene. Expression of the MECP2 transgene was confirmed by western blotting and immunostaining of brain tissues of transgenic monkeys. Genomic integration sites of the transgenes were characterized by a deep-sequencing-based method. As compared to wild-type monkeys, MECP2 transgenic monkeys exhibited a higher frequency of repetitive circular locomotion and increased stress responses, as measured by the threat-related anxiety and defensive test9. The transgenic monkeys showed less interaction with wild-type monkeys within the same group, and also a reduced interaction time when paired with other transgenic monkeys in social interaction tests. The cognitive functions of the transgenic monkeys were largely normal in the Wisconsin general test apparatus, although some showed signs of stereotypic cognitive behaviours. Notably, we succeeded in generating five F1 offspring of MECP2 transgenic monkeys by intracytoplasmic sperm injection with sperm from one F0 transgenic monkey, showing germline transmission and Mendelian segregation of several MECP2 transgenes in the F1 progeny. Moreover, F1 transgenic monkeys also showed reduced social interactions when tested in pairs, as compared to wild-type monkeys of similar age. Together, these results indicate the feasibility and reliability of using genetically engineered non-human primates to study brain disorders.

Black Hole Memory and Soft Hair

A recent paper by Hawking, Perry, and Strominger (Soft Hair on Black Holes) proposes a new kind of soft hair (i.e., soft gravitons or photons) on the black hole horizon. This hair is related to recent results on BMS symmetries and soft (zero) modes by Strominger and collaborators. The existence of an infinite number of additional symmetries and conserved charges in gravity (which can be measured using the gravitational memory results of Braginsky and Thorne) is uncontroversial. The subtle question (discussed by Jacques Distler and Lubos Motl) is whether one can think of black holes as carrying these charges, or whether they are only a property of the asymptotic vacuum of Minkowski space. (I suppose that the fact that one can measure these charges for a localized gravitational wave suggests that they are not just due to choice of boundary conditions at null infinity.)

I found this talk (video below) by Malcolm Perry to be a very pedagogical introduction to BMS symmetries. If I understood correctly, the infinite number of gravitational vacuum states can be thought of as a choice of boundary condition on perturbations to the Minkowski metric (i.e., on gravitational radiation at infinity). The BH horizon also has a BMS symmetry and one can think of the arbitrary choice of function there as a condition on the outgoing Hawking radiation (i.e., soft photons or gravitons) -- see @50min in the video. Perry claims (rather quickly) that one could measure the charges on / near the horizon using methods analogous to Braginsky and Thorne (i.e., inertial detectors). If this is true then it seems reasonable to think of the charges as actually having something to do with the horizon. That is, there are many more "types" of soft radiation coming out of the BH than originally thought -- the radiation can carry not just M, Q, J, but an infinite number of quantum numbers.

I admit to still being confused about this, but it is clear that Perry et al. have thought specifically about it. Whether this helps solve the BH information puzzle is not clear to me at the moment, but it does raise many issues.





Two talks by Strominger on BMS, soft modes, and memory. The first is longer and includes quite a lot of discussion. The second is at Strings 2015 and is more polished / compressed.

Hitler doesn't get a postdoc in High Energy Theory

Via Peter Woit at Not Even Wrong. I think it's hysterical and also incisive.



See also A Tale of Two Geeks and Voting and Weighing.

You might think science is a weighing machine, with experiments determining which theories survive and which ones perish. Healthy sciences certainly are weighing machines, and the imminence of weighing forces honesty in the voting. However, in particle physics the timescale over which voting is superseded by weighing has become decades -- the length of a person's entire scientific career.

Oh, and also: Frauds!

American and Chinese Oligarchies

Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens

Martin Gilens and Benjamin I. Page

Each of four theoretical traditions in the study of American politics—which can be characterized as theories of Majoritarian Electoral Democracy, Economic-Elite Domination, and two types of interest-group pluralism, Majoritarian Pluralism and Biased Pluralism—offers different predictions about which sets of actors have how much influence over public policy: average citizens; economic elites; and organized interest groups, mass-based or business-oriented.

A great deal of empirical research speaks to the policy influence of one or another set of actors, but until recently it has not been possible to test these contrasting theoretical predictions against each other within a single statistical model. We report on an effort to do so, using a unique data set that includes measures of the key variables for 1,779 policy issues.

Multivariate analysis indicates that economic elites and organised groups representing business interests have substantial independent impacts on US government policy, while average citizens and mass-based interest groups have little or no independent influence.

From the paper:

... When a majority of citizens disagrees with economic elites and/or with organised interests, they generally lose. Moreover, because of the strong status quo bias built into the US political system, even when fairly large majorities of Americans favour policy change, they generally do not get it.

... Americans do enjoy many features central to democratic governance, such as regular elections, freedom of speech and association and a widespread (if still contested) franchise. But we believe that if policymaking is dominated by powerful business organisations and a small number of affluent Americans, then America's claims to being a democratic society are seriously threatened.

Interview with Gilens:

Let's talk about the study. If you had 30 seconds to sum up the main conclusion of your study for the average person, how would you do so?

I'd say that contrary to what decades of political science research might lead you to believe, ordinary citizens have virtually no influence over what their government does in the United States. And economic elites and interest groups, especially those representing business, have a substantial degree of influence. Government policy-making over the last few decades reflects the preferences of those groups -- of economic elites and of organized interests.

You say the United States is more like a system of "Economic Elite Domination" and "Biased Pluralism" as opposed to a majoritarian democracy. What do those terms mean? Is that not just a scholarly way of saying it's closer to oligarchy than democracy if not literally an oligarchy?

People mean different things by the term oligarchy. One reason why I shy away from it is it brings to mind this image of a very small number of very wealthy people who are pulling strings behind the scenes to determine what government does. And I think it's more complicated than that. It's not only Sheldon Adelson or the Koch brothers or Bill Gates or George Soros who are shaping government policy-making. So that's my concern with what at least many people would understand oligarchy to mean. What "Economic Elite Domination" and "Biased Pluralism" mean is that rather than average citizens of moderate means having an important role in determining policy, ability to shape outcomes is restricted to people at the top of the income distribution and to organized groups that represent primarily -- although not exclusively -- business.

See also Fukuyama and Zhang on the China Model, and Daniel Bell's The China Model: political meritocracy and limits of democracy.