Alone on a Mountain

Reminds me of my years in Oregon :-)

See also Off the Grid in British Columbia.

CNGB: China National Gene Bank

Unbeknownst to me I've been skyping with a collaborator who has been working from this location.

SCMP: China opens first national gene bank, aiming to house hundreds of millions of samples

China’s first national gene bank, claimed to be the largest of its kind in the world, officially opened on Thursday to store and carry out research on hundreds of millions of genetic samples. The centre, dubbed China’s Noah’s Ark by mainland media, aims to collect 300 million genetic samples at its base in Shenzhen when all two phases are complete

... Cao said it was in China’s national interest to have its own gene bank rather than storing samples in other countries. “Cooperation is the global trend but it is more secure to preserve data in China since the variations among the genomes from different races could be used in both good and evil ways,” Cao said. ...

Amazon unstoppable?

Over 50% of Americans, and 70% with household income over $100k, are Amazon Prime users.

Amazon is killing malls, retailers, brands, and more.

Von Neumann and Realpolitik

"Right, as the world goes, is only in question between equals in power. The strong do what they will and the weak suffer what they must." -- Thucydides, Melian Dialogue.

 

Von Neumann, Feynman, and Ulam.

Adventures of a Mathematician (Ulam): ... Once at Christmas time in 1937, we drove from Princeton to Duke University to a meeting of the American Mathematical Society. ...

As we passed the battlefields of the Civil War, Johnny recounted the smallest details of the battles. His knowledge of history was really encyclopedic, but what he liked and knew best was ancient history. He was a great admirer of the concise and wonderful way the Greek historians wrote. His knowledge of Greek enabled him to read Thucydides, Herodotus, and others in the original; his knowledge of Latin was even better.

The story of the Athenian expedition to the island of Melos, the atrocities and killings that followed, and the lengthy debates between the opposing parties fascinated him for reasons which I never quite understood. He seemed to take a perverse pleasure in the brutality of a civilized people like the ancient Greeks. For him, I think it threw a certain not-too-complimentary light on human nature in general. Perhaps he thought it illustrated the fact that once embarked on a certain course, it is fated that ambition and pride will prevent a people from swerving from a chosen course and inexorably it may lead to awful ends, as in the Greek tragedies. Needless to say this prophetically anticipated the vaster and more terrible madness of the Nazis. Johnny was very much aware of the worsening political situation. In a Pythian manner, he foresaw the coming catastrophe. ...

... I will never forget the scene a few days before he died. I was reading to him in Greek, from his worn copy of Thucydides, a story he liked especially about the Athenians' attack on Melos, and also the speech of Pericles. He remembered enough to correct an occasional mistake or mispronunciation on my part.

... there was nothing small about his interests ... He was unique in this respect. Unique, too, were his overall intelligence, breadth of interest, and absolute feeling for the difference between the momentary technical work and the great lines of the life of the mathematical tree itself and its role in human thought.

Michael Anton: Inside the Trump White House

Michael Anton is head of strategic communications for the National Security Council. See related Politico article.

The Atlantic: Michael Anton warned last year that 2016 was the Flight 93 election: “Charge the cockpit or you die.”

Americans charged. Donald Trump became president of the United States. And Anton, the author of that now-notorious essay, is helping to fly the plane—running communications for the National Security Council.

Anton cuts a curious figure through the Trump White House. A thoroughly educated dandy, his writings are at the core of an effort to construct an intellectual framework around the movement that elected a president who has shown no inclination to read books and who speaks in an unpretentious New York vernacular. ...

Yann LeCun on Unsupervised Learning

This is a recent Yann LeCun talk at CMU. Toward the end he discusses recent breakthroughs using GANs (Generative Adversarial Networks, see also Ian Goodfellow here and here).

LeCun tells an anecdote about the discovery of backpropagation. The first implementation of the algorithm didn't work, probably because of a bug in the program. But they convinced themselves that the reason for the failure was that the network could easily get caught in a local minimum which prevents further improvement. It turns out that this is very improbable in high dimensional spaces, which is part of the reason behind the great success of deep learning. As I wrote here:

In the limit of high dimensionality a critical point is overwhelmingly likely to be a saddlepoint (have at least one negative eigenvalue). This means that even though the surface is not strictly convex the optimization is tractable.

This (free version!) new textbook on deep learning by Goodfellow, Bengio, and Courville looks very good. See also Michael Nielsen's book.

If I were a young person I would be working in this area (perhaps with an eye toward applications in genomics, or perhaps working directly on the big problem of AGI). I hope after I retire they will let me hang out at one of the good AI places like Google Brain or Deep Mind :-)

History of Bayesian Neural Networks

This talk gives the history of neural networks in the framework of Bayesian inference. Deep learning is (so far) quite empirical in nature: things work, but we lack a good theoretical framework for understanding why or even how. The Bayesian approach offers some progress in these directions, and also toward quantifying prediction uncertainty.

I was sad to learn from this talk that David Mackay passed last year, from cancer. I recommended his book Information theory, inference and learning algorithms back in 2007.

Yarin Gal's dissertation Uncertainty in Deep Learning, mentioned in the talk.

I suppose I can thank my Caltech education for a quasi-subconscious understanding of neural nets despite never having worked on them. They were in the air when I was on campus, due to the presence of John Hopfield (he co-founded the Computation and Neural Systems PhD program at Caltech in 1986). See also Hopfield on physics and biology.

Amusingly, I discovered this talk via deep learning: YouTube's recommendation engine, powered by deep neural nets, suggested it to me this Saturday afternoon :-)

The Rise and Fall of the Meritocracy (BBC podcast)

British socialist Michael Young coined the term meritocracy in his 1958 dystopian satire The Rise of the Meritocracy. He realized even then that a system which rewarded individuals fairly, based on ability and effort, would likely lead to genetic class stratification, due to the heritability of traits. His son Toby, a journalist for the Spectator, explores this topic in an excellent BBC podcast, featuring researchers such as Robert Plomin and me.

In the future, will we redistribute genetic wealth as well as material wealth?

Toby Young's six problems with meritocracy

The Rise of the Meritocracy is a dystopian satire written almost sixty years ago by pioneering sociologist Michael Young. It imagined a modern society uncannily like our own and coined the term meritocracy.

Michael Young's son Toby, a journalist for the Spectator, has been asking if his asks if his father's dark prophesy is correct. Here are Toby Young's six problems with a meritocracy.

1. Where meritocracy came from

The word ‘meritocracy’ was coined by my father, a left-wing sociologist called Michael Young, to describe a dystopian society of the future. In his 1958 book The Rise of the Meritocracy, he imagines a 21st Century Britain in which status is determined by a combination of IQ and effort. He acknowledged that this was fairer than an aristocratic society in which status is simply passed on from parents to their children, but it was precisely because meritocracy gave a patina of legitimacy to the inequalities thrown up by free market capitalism that he disapproved of it.

2. Is a meritocratic society fairer?

The political philosopher John Rawls pointed out that a meritocratic society isn’t necessarily fairer than an aristocratic one. After all, the qualities that meritocracy rewards – exceptional intelligence and drive – are, for the most part, natural gifts that people are born with. Since successful people have done nothing to deserve those talents, they don’t deserve the rewards they bring any more than they deserve to inherit a fortune.

3. Complete equality of opportunity

For a society to be 100% meritocratic, you need complete equality of opportunity. But the only way to guarantee that is to remove children from their parents at birth and raise them in identical circumstances. If you don’t do that, the socio-economic status of a child’s parents will inevitably affect that child’s life chances.

4. Is it in the genetics?

According to the political scientist Charles Murray, meritocracy inevitably leads to a genetically-based caste system. Why? Because the traits selected for by the meritocratic sorting principle are genetically-based and, as such, likely to be passed on from parents to their children. Genetic variation means some highly able children will be born to people of average and below average intelligence, but the children of the meritocratic elite will, in aggregate, always have a competitive advantage and over several generations that leads to social ossification.

5. Noblesse oblige

One of the things my father disliked about meritocracy was that it engendered a sense of entitlement amongst the most successful. Because they regard their elevated status as thoroughly deserved, they’re not burdened by a sense of noblesse oblige. At least in an aristocratic society, members of the lucky sperm club are afflicted by guilt and self-doubt and, as such, tend to be a bit nicer to those below them.

6. Is America the most meritocratic country?

Americans like to tell themselves that they live in the most meritocratic country in the world but, in fact, it may be one of the least. In most international league tables of inter-generational social mobility, which measure the chances a child born into one class has of moving into another over the course of their lifetime, America is at the bottom.

[ Of course, lack of mobility could also result in a society that is both meritocratic and already somewhat stratified by genetics. See Income, Wealth, and IQ and US economic mobility data. ]

I wonder whether even Michael Young was aware that the British implementation of meritocracy through competitive civil service and university entrance examinations (a century before his book The Rise of the Meritocracy) was a deliberate adoption of Chinese ways.

Les Grandes Ecoles Chinoises: ... the British and French based their civil service and educational examination systems on the much older Chinese model ...

... French education was really based on the Chinese system of competitive literary examinations, and ... the idea of a civil service recruited by competitive examinations undoubtedly owed its origins to the Chinese system which was popularized in France by the philosophers, especially Voltaire. ...

Summary of the case of Britain and colonial India can be found here. Amusingly, 19th century British writers opposed to the new system of exams referred to it as "... an adopted Chinese culture" (p. 304-305).

Penalized regression from summary statistics

One of the difficulties in genomics is that when DNA donors are consented for a study, the agreements generally do not allow sharing (aggregation) of genomic data across multiple studies. This leads to isolated silos of data that can't be fully shared. However, computations can be performed on one silo at a time, with the results ("summary statistics") shared within a larger collaboration. Most of the leading GWAS collaborations (e.g., GIANT for height, SSGAC for cognitive ability) rely on shared statistics. Simple regression analysis (one SNP at a time) can be conducted using just summary statistics, but more sophisticated algorithms cannot. These more sophisticated methods can generate a better phenotype predictor, using less data, than a SNP by SNP analysis.

For example, the objective function used in LASSO (L1-penalized regression) is of the form

where, for the genomics problem, y is the phenotype vector, X the matrix of genomes, beta the vector of effect sizes, and lambda the penalization. Optimization of this function seems to require access to the full matrix X and vector y -- i.e., requires access to potentially all the genomes and phenotypes at once. Is there a modified version of the algorithm that works on summary statistics, where only subsets of X and y are available? Carson Chow has advocated this approach to me for some time. If one can separately estimate X'X (LD matrix of genomic correlations), and gather X'y (phenotype-SNP correlations) from summary statistics, then LASSO over silo-ed data may become a reality. Of course, the devil is in the details. The paper below describes an approach to this problem.

Polygenic scores via penalized regression on summary statistics

Timothy Mak, Robert Milan Porsch, Shing Wan Choi, Xueya Zhou, Pak Chung Sham
doi: https://doi.org/10.1101/058214

Polygenic scores (PGS) summarize the genetic contribution of a person's genotype to a disease or phenotype. They can be used to group participants into different risk categories for diseases, and are also used as covariates in epidemiological analyses. A number of possible ways of calculating polygenic scores have been proposed, and recently there is much interest in methods that incorporate information available in published summary statistics. As there is no inherent information on linkage disequilibrium (LD) in summary statistics, a pertinent question is how we can make use of LD information available elsewhere to supplement such analyses. To answer this question we propose a method for constructing PGS using summary statistics and a reference panel in a penalized regression framework, which we call lassosum. We also propose a general method for choosing the value of the tuning parameter in the absence of validation data. In our simulations, we showed that pseudovalidation often resulted in prediction accuracy that is comparable to using a dataset with validation phenotype and was clearly superior to the conservative option of setting the tuning parameter of lassosum to its lowest value. We also showed that lassosum achieved better prediction accuracy than simple clumping and p-value thresholding in almost all scenarios. It was also substantially faster and more accurate than the recently proposed LDpred.

See also Bayesian large-scale multiple regression with summary statistics from genome-wide association studies.

National Geographic: How Humans Are Shaping Our Own Evolution

See also A Brief History of the Future, as told to the Masters of the Universe and Super-Intelligent Humans are Coming.

National Geographic: How Humans Are Shaping Our Own Evolution:

... Unlike our forebears, we may soon not need to wait for evolution to fix the problem. In 2013 Nick Bostrom and Carl Shulman, two researchers at the Future of Humanity Institute, at Oxford University, set out to investigate the social impact of enhancing intelligence, in a paper for Global Policy. They focused on embryo selection via in vitro fertilization. With IVF, parents can choose which embryo to implant. By their calculations, choosing the “most intelligent embryo” out of any given 10 would increase a baby’s IQ roughly 11.5 points above chance. If a woman were willing to undergo more intensive hormone treatments to produce eggs faster—“expensive and burdensome,” as the study notes with understatement—the value could grow.

The real benefit, though, would be in the compound gain to the recipient’s descendants: After 10 generations, according to Shulman, a descendant might enjoy an IQ as much as 115 points higher than his or her great-great-great-great-great-great-great-great-grandmother’s. As he pointed out to me, such a benefit is built on extremely optimistic assumptions, but at the least the average recipient of this genetic massaging would have the intelligence equal to a genius today. Using embryonic stem cells, which could be converted into sperm or ova in just six months, the paper notes, might yield far faster results. Who wants to wait two centuries to be the scion of a race of geniuses? Shulman also mentioned that the paper omitted one obvious fact: “In 10 generations there will likely be computer programs that outperform even the most enhanced human across the board.”

There’s a more immediate objection to this scenario, though: We don’t yet know enough about the genetic basis for intelligence to select for it. One embryo doesn’t do advanced calculus while another is stuck on whole numbers. Acknowledging the problem, the authors claim that the ability to select for “modest cognitive enhancement” may be only five to 10 years off.

At first glance this would seem improbable. The genetic basis of intelligence is very complex. Intelligence has multiple components, and even individual aspects—computational ability, spatial awareness, analytic reasoning, not to mention empathy—are clearly multigenetic, and all are influenced by environmental factors as well. Stephen Hsu, vice president for research at Michigan State University, who co-founded the Cognitive Genomics Lab at BGI (formerly Beijing Genomics Institute), estimated in a 2014 article that there are roughly 10,000 genetic variants likely to have an influence on intelligence. That may seem intimidating, but he sees the ability to handle that many variants as nearly here—“in the next 10 years,” he writes—and others don’t think you’d need to know all the genes involved to start selecting smarter embryos. “The question isn’t how much we know or don’t know,” Church says. “It’s how much we need to know to make an impact. ..."

On the genetic architecture of intelligence and other quantitative traits
https://arxiv.org/abs/1408.3421

Somewhere ... It's happening...

Chomsky: Russia conspiracy theories "a joke"

Chomsky on the current media / left obsession with anti-Russia conspiracy theories. I guess he and Trump are both Putin puppets... Oops, except Trump just attacked Assad and risked killing Russian soldiers. So that just leaves Chomsky.

NOAM CHOMSKY: It’s a pretty remarkable fact that—first of all, it is a joke. Half the world is cracking up in laughter. The United States doesn’t just interfere in elections. It overthrows governments it doesn’t like, institutes military dictatorships. Simply in the case of Russia alone—it’s the least of it—the U.S. government, under Clinton, intervened quite blatantly and openly, then tried to conceal it, to get their man Yeltsin in, in all sorts of ways. So, this, as I say, it’s considered—it’s turning the United States, again, into a laughingstock in the world.

So why are the Democrats focusing on this? In fact, why are they focusing so much attention on the one element of Trump’s programs which is fairly reasonable, the one ray of light in this gloom: trying to reduce tensions with Russia? That’s—the tensions on the Russian border are extremely serious. They could escalate to a major terminal war. Efforts to try to reduce them should be welcomed. ...

So, meanwhile, this one topic is the primary locus of concern and critique, while, meanwhile, the policies are proceeding step by step, which are extremely destructive and harmful. So, you know, yeah, maybe the Russians tried to interfere in the election. That’s not a major issue. Maybe the people in the Trump campaign were talking to the Russians. Well, OK, not a major point, certainly less than is being done constantly. And it is a kind of a paradox, I think, that the one issue that seems to inflame the Democratic opposition is the one thing that has some justification and reasonable aspects to it.

... Well, you can understand why the Democratic Party managers want to try to find some blame for the fact—for the way they utterly mishandled the election and blew a perfect opportunity to win, handed it over to the opposition

... NATO maneuvers are taking place hundreds of yards from the Russian border. The Russian jet planes are buzzing American planes. This—something could get out of hand very easily. ... people like William Perry, who has a distinguished career and is a nuclear strategist and is no alarmist at all, is saying that we’re back to the—this is one of the worst moments of the Cold War, if not worse.

See also Trump: Give Peace a Chance, Obama: "Don't do stupid sh*t", and Trump, Putin, Stephen Cohen, Brawndo, and Electrolytes.

Sex Differences In The Adult Human Brain: UK Biobank data

Male brains exhibit larger variance across all morphological measures. (Don't tell Larry Summers! ;-)

Note, as far as I can tell the authors don't normalize the SD by mean value for each gender to obtain the typical percentage fluctuation (a dimensionless quantity). The male brain is about 10% larger and each of the subregions is roughly that much bigger as well. If you divide the larger male SD by the larger male mean for each morphology the effect is much smaller than tabulated in the second figure below.

Sex Differences In The Adult Human Brain: Evidence From 5,216 UK Biobank Participants

doi: https://doi.org/10.1101/123729

Sex differences in human brain structure and function are of substantial scientific interest because of sex-differential susceptibility to psychiatric disorders and because of the potential to explain sex differences in psychological traits. Males are known to have larger brain volumes, though the patterns of differences across brain subregions have typically only been examined in small, inconsistent studies. In addition, despite common findings of greater male variability in traits like intelligence, personality, and physical performance, variance differences in the brain have received little attention. Here we report the largest single-sample study of structural and functional sex differences in the human brain to date (2,750 female and 2,466 male participants aged 44-77 years). Males had higher cortical and sub-cortical volumes, cortical surface areas, and white matter diffusion directionality; females had thicker cortices and higher white matter tract complexity. Considerable overlap between the distributions for males and females was common, and subregional differences were smaller after accounting for global differences. There was generally greater male variance across structural measures. The modestly higher male score on two cognitive tests was partly mediated via structural differences. Functional connectome organization showed stronger connectivity for males in unimodal sensorimotor cortices, and stronger connectivity for females in the default mode network. This large-scale characterisation of neurobiological sex differences provides a foundation for attempts to understand the causes of sex differences in brain structure and function, and their associated psychological and psychiatric consequences.

Susan Rice and U.S. person information "derived solely from raw SIGINT"

I hope this scandal will focus additional attention on massive bulk collection and preservation of private communications of US citizens by NSA.

Media discussion continues to focus on "unmasking" = dissemination of identities of US individuals. However, I have yet to see discussion of whether someone like Rice could order specific database searches (e.g., by NSA, of preserved records) on a specific individual to acquire intercepts such as voice transcripts, emails, etc. It doesn't seem to become an unmasking until that information is distributed in the form of an intelligence report (or, is such a request automatically an unmasking?). The search results alone constitute an invasion of individual privacy. It is unclear to me who has access to such results, and under what conditions the searches can be requested. There are well known instances of NSA employees abusing these powers: see LOVEINT. Could the White House order something similar without a record trail? (See excerpt added below.)

Bloomberg: Susan Rice Sought Names in Trump Intel, Says Eli Lake

Former national security adviser Susan Rice made multiple requests for the identities of people connected to the transition team of Donald Trump contained in raw intelligence reports, according to U.S. officials familiar with the matter. Bloomberg View columnist Eli Lake has the details.

From Nunes, Trump, Obama and Who Watches the Watchers?, this is the legal standard that the Susan Rice unmaskings will be judged by:

Section VI: ... An IC element may disseminate U.S. person information "derived solely from raw SIGINT" under these procedures ... if ... the information is “necessary to understand the foreign intelligence or counterintelligence information,”

Richard Haas notes that this kind of activity on the part of Susan Rice and NSC staff is only justifiable under "extraordinary circumstances"!



Added (from comments):

The Observer: ... In addition, Rice didn’t like to play by the rules, including the top-secret ones. On multiple occasions, she asked the NSA to do things they regarded as unethical and perhaps illegal. When she was turned down — the NSA fears breaking laws for any White House, since they know they will be left holding the bag in the end — Rice kept pushing.

As a longtime NSA official who experienced Rice’s wrath more than once told me, “We tried to tell her to pound sand on some things, but it wasn’t allowed—we were always overruled.” On multiple occasions, Rice got top Agency leadership to approve things which NSA personnel on the front end of the spy business refused. This means there may be something Congress and the FBI need to investigate here.

...

John Schindler is a security expert and former National Security Agency analyst and counterintelligence officer. A specialist in espionage and terrorism, he’s also been a Navy officer and a War College professor. He’s published four books and is on Twitter at @20committee.