Information Processing: 06/2021

Wednesday, June 30, 2021

Six Ways From Sunday: Tucker vs NSA

Chuck Schumer: You take on the intelligence community, they have six ways from Sunday to get back at you.

Tucker Carlson has potential as a politician -- there is at least a small chance that someday he'll be POTUS. The intelligence services are, I am sure, very interested in any kompromat they can acquire on him for future use. You mean foreign intel services? No, I mean our intel services :-(

Clarification, from comments:

The post is not primarily about Tucker. It's about intel services spying on American citizens.

Most importantly, Tucker's story is credible: some whistleblower saw intercepted Tucker emails and contacted him to let him know he is under surveillance. But as anyone paying attention knows, we are ALL under surveillance due to "bulk collection" revealed many years ago, e.g., by Snowden. The Rogers saga and FISC report show that this bulk-collected data is not very well protected from intel agency types who want to have a peek at it...

Re: bulk collection, non-denial denials ("not an intelligence target of the Agency" ha ha), see

Wikipedia: According to a report in The Washington Post in July 2014, relying on information furnished by Snowden, 90% of those placed under surveillance in the U.S. are ordinary Americans, and are not the intended targets. The newspaper said it had examined documents including emails, message texts, and online accounts, that support the claim.

See Twilight Struggles in a Wilderness of Mirrors: Admiral Mike Rogers, the NSA, and Obama-era Political Spying.

Below is a Rogers timeline covering illegal spying using NSA data. This illegal use of data is a matter of record -- undisputed, but also largely unreported. The FISC (FISA court) report on this illegal use of data appeared in April 2017; the author is Rosemary Collyer, the head FISA judge. The report was originally classified Top Secret but was later declassified and released with redactions. Collyer uses the phrase "institutional lack of candor" when referring to behavior of federal agencies in their dealings with FISC over this issue. ...

The court learned in October 2016 that analysts ... were conducting prohibited database searches “with much greater frequency than had previously been disclosed to the court.” The forbidden queries were searches of Upstream Data using US-person identifiers. The report makes clear that as of early 2017 NSA Inspector General did not even have a good handle on all the ways that improper queries could be made to the system. ...

March 2016 – NSA Director Rogers becomes aware of improper access to raw FISA data.

April 2016 – Rogers orders the NSA compliance officer to run a full audit on 702 NSA compliance.

April 18 2016 – Rogers shuts down FBI/NSD contractor access to the FISA Search System.

Mid-October 2016 – DNI Clapper submits a recommendation to the White House that Director Rogers be removed from the NSA.

October 20 2016 – Rogers is briefed by the NSA compliance officer on the Section 702 NSA compliance audit and “About” query violations.

October 21 2016 – Rogers shuts down all “About" query activity. Rogers reports the activity to DOJ and prepares to go before the FISA Court.

October 21 2016 – DOJ & FBI seek and receive a Title I FISA probable cause order authorizing electronic surveillance on Carter Page from the FISC. At this point, the FISA Court is unaware of the Section 702 violations.

October 24 2016 – Rogers verbally informs the FISA Court of Section 702(17) violations.

October 26 2016 – Rogers formally informs the FISA Court of 702(17) violations in writing.

November 17 2016 (morning) – Rogers travels to meet President-Elect Trump and his Transition Team in Trump Tower. Rogers does not inform DNI James Clapper.

November 17 2016 (evening) – Trump Transition Team announces they are moving all transition activity to Trump National Golf Club in New Jersey.

I was recently in a Zoom meeting on geopolitics that included Admiral Rogers. I wanted to ask him privately about the above. Perhaps someday I'll get the chance.

Caption: NSA Director Rogers describes to Congress how little privacy Americans have from government surveillance.

Alternate Caption: NSA Director Rogers tells Congress how much legal oversight remains over the activities of intel services.

Tuesday, June 29, 2021

Machine Learning Prediction of Biomarkers from SNPs and of Disease Risk from Biomarkers in the UK Biobank (published version)

This is the published version of our MedRxiv preprint discussed back in April 2021. It is in the special issue Application of Genomic Technology in Disease Outcome Prediction of the journal Genes.

There is a lot in this paper: genomic prediction of important biomarkers (e.g., lipoprotein A, mean platelet (thrombocyte) volume, bilirubin, platelet count), prediction of important disease risks from biomarkers (novel ML in a ~65 dimensional space) with potential clinical applications. As is typical, genomic predictors trained in a European ancestry population perform less well in distant populations (e.g., S. Asians, E. Asians, Africans). This is probably due to different SNP LD (correlation) structure across populations. However predictors of disease risk using directly measured biomarkers do not show this behavior -- they can be applied even to distant ancestry groups.

The referees did not like our conditional probability notation:

( biomarkers | SNPs ) and ( disease risk | biomarkers )

So we ended up with lots of acronyms to refer to the various predictors.

Some of the biomarkers identified by ML as important for predicting specific disease risk are not familiar to practitioners and have not been previously discussed (as far as we could tell from the literature) as relevant to that specific disease. One medical school professor and practitioner, upon seeing our results, said he would in future add several new biomarkers to routine blood tests ordered for his patients.

Machine Learning Prediction of Biomarkers from SNPs and of Disease Risk from Biomarkers in the UK Biobank

Erik Widen 1,*,Timothy G. Raben 1, Louis Lello 1,2,* and Stephen D. H. Hsu 1,2

1 Department of Physics and Astronomy, Michigan State University, 567 Wilson Rd, East Lansing, MI 48824, USA

2 Genomic Prediction, Inc., 675 US Highway One, North Brunswick, NJ 08902, USA

*Authors to whom correspondence should be addressed.

Academic Editor: Sulev Koks

Genes 2021, 12(7), 991; https://doi.org/10.3390/genes12070991 (registering DOI)

Received: 30 March 2021 / Revised: 22 June 2021 / Accepted: 23 June 2021 / Published: 29 June 2021

(This article belongs to the Special Issue Application of Genomic Technology in Disease Outcome Prediction)

Abstract

We use UK Biobank data to train predictors for 65 blood and urine markers such as HDL, LDL, lipoprotein A, glycated haemoglobin, etc. from SNP genotype. For example, our Polygenic Score (PGS) predictor correlates ∼0.76 with lipoprotein A level, which is highly heritable and an independent risk factor for heart disease. This may be the most accurate genomic prediction of a quantitative trait that has yet been produced (specifically, for European ancestry groups). We also train predictors of common disease risk using blood and urine biomarkers alone (no DNA information); we call these predictors biomarker risk scores, BMRS. Individuals who are at high risk (e.g., odds ratio of >5× population average) can be identified for conditions such as coronary artery disease (AUC∼0.75), diabetes (AUC∼0.95), hypertension, liver and kidney problems, and cancer using biomarkers alone. Our atherosclerotic cardiovascular disease (ASCVD) predictor uses ∼10 biomarkers and performs in UKB evaluation as well as or better than the American College of Cardiology ASCVD Risk Estimator, which uses quite different inputs (age, diagnostic history, BMI, smoking status, statin usage, etc.). We compare polygenic risk scores (risk conditional on genotype: PRS) for common diseases to the risk predictors which result from the concatenation of learned functions BMRS and PGS, i.e., applying the BMRS predictors to the PGS output.

Figure 11. The ASCVD BMRS and the ASCVD Risk Estimator both make accurate risk predictions but with partially complementary information. (Upper left): Predicted risk by BMRS, the ASCVD Risk Estimator and a PRS predictor were binned and compared to the actual disease prevalence within each bin. The gray 1:1 line indicates perfect prediction. ... The ASCVD Risk Estimator was applied to 340k UKB samples while the others were applied to an evaluation set of 28k samples, all of European ancestry. (Upper right) shows a scatter plot and distributions of the risk predicted by BMRS versus the risk predicted by the ASCVD Risk Estimator for the 28k Europeans in the evaluation set. The BMRS distribution has a longer tail of high predicted risk, providing the tighter confidence interval in this region. The left plot y-axis is the actual prevalence within the horizontal and vertical cross-sections, as illustrated with the shaded bands corresponding to the hollow squares to the left. Notably, both predictors perform well despite the differences in assigned stratification. The hexagons are an overlay of the (lower center) heat map of actual risk within each bin (numbers are bin sizes). Both high risk edges have varying actual prevalence but with a very strong enrichment when the two predictors agree.

Sunday, June 27, 2021

Othram CEO David Mittelman interview with Razib Khan

Razib Khan interviews Othram CEO David Mittelman.

See Othram: the future of DNA forensics for some lab photos and analysis of relative identification via DNA database, the legal status of large databases (23andMe, Ancestry), etc.

Fun fact: David is a serious powerlifter :-)

I've done a lot of things in science, but I get a unique kind of personal satisfaction every time Othram solves a case! Of course, this is happening so often now that I can barely keep track.

Saturday, June 19, 2021

LEO SAR, hypersonics, and the death of the naval surface ship

Duh... Let's spend ~$10B each for new aircraft carriers that can be easily monitored from space and attacked using hypersonic missiles.

Sure, in a real war with a peer competitor we'll have to hide them far from the conflict zone. But they're great for intimidating small countries...

Monday, June 14, 2021

Japan and The Quad (Red Line geostrategy podcast)

I recommend this episode of The Red Line geostrategy podcast: Japan and The Quad.

Serious analysts from the Asia-Pacific region (e.g., Australia, India, Japan, etc.) are often much better than their US counterparts. US analysts tend to misperceive local political and economic realities, and can be captives of Washington DC groupthink (e.g., about weapons systems like aircraft carriers or the F35 or missile defense).

For example, Australian analysts acknowledged the vulnerability of US aircraft carriers to PRC ASBM and cruise missiles well before it became common for US analysts to openly admit the problem. The earliest technical analysis I could find of PRC satellite capability to track US surface ships in the Pacific came from an Indian military think tank (see maps below), at a time when many US "experts" denied that it was possible.

In this podcast Japan's reliance on sea lanes for energy, food, and raw materials is given proper emphasis. Japan imports ~60% of its food calories and essentially all of its oil. The stituation is similar for S. Korea and Taiwan. It is important to note that blocking sea transport to Taiwan and Japan does not require PLAN blue water dominance. ASBM and cruise missiles which threaten aircraft carriers can also hold oil tankers and global shipping at risk from launch sites which are on or near the Asian mainland. Missile + drone + AI/ML technology completely alters the nature of sea blockade, but most strategic planners do not yet realize this. Serious conflict in this region would likely wreak havoc on the economies of Taiwan, S. Korea, and Japan.

Red Line: ... In seeking to counter an ever-expanding China, Tokyo is turning abroad in search of allies. Key to this is the recent revival of "The Quad", a strategic dialogue between the US, Australia, Japan and India. Will it be enough to counter their rising neighbour across the East China Sea? Is this the first step to creating an "Asian NATO", and how will China respond?

Guests:

Owen Swift

Geopolitics and defence analyst specialising in Australian & East Asian Defence Written with organisations including The Australian Strategic Policy Institute and Monash University. Senior Producer and resident Asia-Pacific expert at The Red Line

John Nilsson-Wright

Senior Lecturer on Japanese Geopolitics and International Relations for Cambridge University. Senior Research Fellow for Northeast Asia for Chatham House. Author of the book Unequal Allies about the post-war relationship between Japan and the United States.

John Coyne

Head of the Northern Australia Strategic Policy Centre at the Australian Strategic Policy Institute (ASPI). Head of Strategic Policing and Law Enforcement at ASPI. One of the most trusted experts when it comes to the dynamics of East Asia for Australia and the United States.

Part 1: The Return from Armageddon (02:52) Owen Swift overviews Japan's place in East Asia and the fundamental geographic challenges that inform its geopolitics. We tackle Japan's inability to domestically provide the resources and food that its population needs, and how it has historically dealt with this insecurity. The consequences of World War 2 wreaked havoc on Japan's economy, political system and territorial holdings. We analyse the short and long term consequences of this, and seek to understand why it was that Japan took a positive view of the US occupation, comparing it to the option of a possible partial USSR occupation. In the 1980s some thought Japan was on a path to overtake the US economically. While that hasn't come to pass, we look at what it was that made Japan's economic miracle, and the effect that US involvement has today. We look at domestic issues in Japan, including the drastic demographic decline, their ongoing 'defensive only' posture, and the policy options on the table for balancing against the rise of China. Finally we overview Japan's involvement in the Asia-Pacific region as a whole, analysing who it has the best relations with. We look at the extensive investments and infrastructure development Japan is undertaking in ASEAN states, and its cooperation with India and Australia in recent years.

Part 2: The Grand Dilemma (17:56) John Nilsson-Wright helps us understand the fundamental shifts in Japanese politics and foreign policy, including Article 9, the tension between the Yoshida doctrine, public opinion and US pressure within Japan, and the country's re-entry into the sphere of great power competition. We examine the extent of Japan's military presence in the Indo-Pacific; looking at its exercises with other powers and the concerns its neighbours have, some of whom still bear significant scars from World War 2. South Korea's relationship with Japan is one that, on the surface, seems like it should be closer than it is. We analyse why it is that despite their mutual interest in countering North Korea and China, their close geography and both being under the US umbrella, the two states have been unable to overcome enormous domestic resentment and historic scars. Japan's constitution has very tight constraints on what it can do militarily. Nilsson-Wright helps us understand the details of these restrictions and their history over the past few decades. We look at how the legal interpretation of the article has changed as Japan's needs have changed. We also look at Japan's expanding concept of national interest, which began as a purely defensive, geographically limited concept, but that has continued to expand in recent years. We contrast that with the difficulty the government has had with domestic views of Japan's role on the global stage. We tackle territorial issues including the Kuril and Sakhalin Islands, and look at Japan's role in a potential Taiwanese conflict.

Part 3: A United Front? (43:48) John Coyne a takes us through the details of the Quad, and the roles that its constituent members play. We look at Japan's re-examination of their supply chains, their development of strategic depth and the recent news that they are considering abolishing the 1% of GDP cap on military spending. Coyne helps us understand what the Quad actually is - It is not and does not seek to be an Asian NATO - just as ASEAN is not an Asian EU. We look at what the limitations are for each of the states involved, particularly India. We look at the actual relationships and cooperation that has been seen between Quad members. With Japan's newfound willingness to be involved in military operations, we examine how closely they will work with Australia, India and the United States, and the extent to which the Quad is more than just symbolic. We then turn to China's response. Is China likely to seek a grouping like the Quad in opposition to it? Can the Quad actually contain China or its navy in any practical sense? What will China do if cooperation tightens? We look at how China has already sought to hit back, targeting Australia in particular with the "14 Grievances", which were delivered as a consequence of the deterioration of their relationship. Australia's membership and participation in the Quad is a key part of this deterioration. Finally we look at how the Quad members have worked to strategically separate from China, such as Japan's work to defeat China's monopoly on the rare earths industry.

The map below appeared in the 2017 blog post On the military balance of power in the Western Pacific.

Here is another map:

Authors: Professor S. Chandrashekar and Professor Soma Perumal

We can state with confidence that the Yaogan satellite constellation and its associated ASBM system provide visible proof of Chinese intentions and capabilities to keep ACG strike groups well away from the Chinese mainland.

Though the immediate purpose of the system is to deter the entry of a hostile aircraft carrier fleet into waters that directly threatens its security interests especially during a possible conflict over Taiwan, the same approach can be adopted to deter entry into other areas of strategic interest.

Viewed from this perspective the Chinese do seem to have in place an operational capability for denying or deterring access into areas which it sees as crucial for preserving its sovereignty and security.

Bonus: This political cartoon about the G7 meeting has been widely shared in the sinosphere. Some of the esoteric meaning may be lost on a US audience, but see here for an explanation. Note the device on the table which turns toilet paper into US dollars. The Japanese dog is serving radioactive water to the co-conspirators. Italy, the BRI participant, refuses the drink. France and Germany seem to be thinking about it carefully. Who is the little frog? (Hint: NTD)

Sunday, June 13, 2021

An Inconvenient Minority: The Attack on Asian American Excellence and the Fight for Meritocracy (Kenny Xu)

Kenny Xu is a brave young man. His new book An Inconvenient Minority: The Attack on Asian American Excellence and the Fight for Meritocracy expertly documents a number of unpleasant facts about American society that most major media outlets, education leaders, and social justice advocates have been obfuscating or outright suppressing for decades.

1. Asian Americans (not foreign students from Asia, but individuals of Asian heritage who are US citizens or permanent residents) have been discriminated against in admission to elite institutions of higher education for over 30 years.

To put it bluntly, Asian Americans must, on average, outperform all other groups in order to have an equal chance of admission to universities like Harvard or Yale. If one were to replace Asian Americans with Jews in the previous sentence, it would describe the situation in the early 20th century. Looking back, we are rightfully ashamed and outraged at the conduct of elite universities during this period. Future Americans, and observers all over the world, will eventually have the same reaction to how Asian Americans are treated today by these same institutions.

2. Asian American success, e.g., as measured using metrics such as income, wealth, or education, is problematic for simplistic narratives that emphasize race and "white supremacy" over a more realistic and multifaceted analysis of American society.

3. Efforts to guarantee equal outcomes, as opposed to equal opportunities, are anti-meritocratic and corrosive to social cohesion, undermine basic notions of fairness, and handicap the United States in scientific and technological competition with other nations.

The Table of Contents, reproduced below, gives an idea of the important topics covered. Xu had an insider's view of the Students for Fair Admission v. Harvard trial, now awaiting appeal to the Supreme Court. He also describes the successful effort by a grass roots coalition of Asian Americans to defeat CA Proposition 16, which would have reinstated racial preferences in the public sector (including college admissions) which were prohibited by Proposition 209 in 1996.

Over the years I have had many conversations on this topic with well-meaning (but often poorly informed) parents of all ethnic and cultural backgrounds. I cannot help but ask these people

Are you OK with discrimination against your child? What did they do to deserve it?

Are you going to let virtue-signaling administrators at the university devalue the hard work and hard-won accomplishments of your son or daughter? Are you going to do anything about it?

and I cannot help but think

If you won't do anything about it, then f*ck you. Your kids deserve better parents.

Kenny calls it a Fight for Meritocracy. That's what it is -- a fight. Don't forget that Meritocracy is just a fancy word for fairness. It's a fight for your kid, and all kids, to be treated fairly.

I highly recommend the book. These issues are of special concern to Asian Americans, but should be of interest to anyone who wants to know what is really happening in American education today.

Related posts: discrimination against Asian Americans at elite US universities, on meritocracy, and UC faculty report on the use of SAT in admissions.

Thursday, June 03, 2021

Macroscopic Superpositions in Isolated Systems (talk video + slides)

This is video of a talk based on the paper

Macroscopic Superpositions in Isolated Systems

R. Buniy and S. Hsu

arXiv:2011.11661, to appear in Foundations of Physics

For any choice of initial state and weak assumptions about the Hamiltonian, large isolated quantum systems undergoing Schrodinger evolution spend most of their time in macroscopic superposition states. The result follows from von Neumann's 1929 Quantum Ergodic Theorem. As a specific example, we consider a box containing a solid ball and some gas molecules. Regardless of the initial state, the system will evolve into a quantum superposition of states with the ball in macroscopically different positions. Thus, despite their seeming fragility, macroscopic superposition states are ubiquitous consequences of quantum evolution. We discuss the connection to many worlds quantum mechanics.

Slides for the talk.

See this earlier post about the paper:

It may come as a surprise to many physicists that Schrodinger evolution in large isolated quantum systems leads generically to macroscopic superposition states. For example, in the familiar Brownian motion setup of a ball interacting with a gas of particles, after sufficient time the system evolves into a superposition state with the ball in macroscopically different locations. We use von Neumann's 1929 Quantum Ergodic Theorem as a tool to deduce this dynamical result.

The natural state of a complex quantum system is a superposition ("Schrodinger cat state"!), absent mysterious wavefunction collapse, which has yet to be fully defined either in logical terms or explicit dynamics. Indeed wavefunction collapse may not be necessary to explain the phenomenology of quantum mechanics. This is the underappreciated meaning of work on decoherence dating back to Zeh and Everett. See talk slides linked here, or the introduction of this paper.

We also derive some new (sharper) concentration of measure bounds that can be applied to small systems (e.g., fewer than 10 qubits).

Related posts:

Classics on the arxiv: von Neumann and the foundations of quantum statistical mechanics

Pure State Quantum Thermalization: from von Neumann to the Lab

Entanglement and fast thermalization in heavy ion collisions

Information Processing

About Me