Genomic prediction: no bull

This Atlantic article discusses the application of genomic prediction to cattle breeding. Breeders have recently started switching from pedigree based methods to statistical models incorporating SNP genotypes. We can now make good predictions of phenotypes like milk and meat production using genetic data alone. Cows are easier than people because, as domesticated animals, they have smaller effective breeding population and less genetic diversity. Nevertheless, I expect very similar methods to be applied to humans within the next 5-10 years.

The Atlantic: ... the semen that Badger-Bluff Fanny Freddie produces has become such a hot commodity in what one artificial-insemination company calls "today's fast paced cattle semen market." In January of 2009, before he had a single daughter producing milk, the United States Department of Agriculture took a look at his lineage and more than 50,000 markers on his genome and declared him the best bull in the land. And, three years and 346 milk- and data-providing daughters later, it turns out that they were right. "When Freddie [as he is known] had no daughter records our equations predicted from his DNA that he would be the best bull," USDA research geneticist Paul VanRaden emailed me with a detectable hint of pride. "Now he is the best progeny tested bull (as predicted)." 

Data-driven predictions are responsible for a massive transformation of America's dairy cows. While other industries are just catching on to this whole "big data" thing, the animal sciences -- and dairy breeding in particular -- have been using large amounts of data since long before VanRaden was calculating the outsized genetic impact of the most sought-after bulls with a pencil and paper in the 1980s. 

Dairy breeding is perfect for quantitative analysis. Pedigree records have been assiduously kept; relatively easy artificial insemination has helped centralized genetic information in a small number of key bulls since the 1960s; there are a relatively small and easily measurable number of traits -- milk production, fat in the milk, protein in the milk, longevity, udder quality -- that breeders want to optimize; each cow works for three or four years, which means that farmers invest thousands of dollars into each animal, so it's worth it to get the best semen money can buy. The economics push breeders to use the genetics. 

The bull market (heh) can be reduced to one key statistic, lifetime net merit, though there are many nuances that the single number cannot capture. Net merit denotes the likely additive value of a bull's genetics. The number is actually denominated in dollars because it is an estimate of how much a bull's genetic material will likely improve the revenue from a given cow. A very complicated equation weights all of the factors that go into dairy breeding and -- voila -- you come out with this single number. For example, a bull that could help a cow make an extra 1000 pounds of milk over her lifetime only gets an increase of $1 in net merit while a bull who will help that same cow produce a pound more protein will get $3.41 more in net merit. An increase of a single month of predicted productive life yields $35 more. 

When you add it all up, Badger-Fluff Fanny Freddie has a net merit of $792. No other proven sire ranks above $750 and only seven bulls in the country rank above $700.

See below -- theoretical calculations suggest that even outliers with net merit of $700-800 will be eclipsed by specimens with 10x higher merit that can be produced by further selection on existing genetic variation. Similar results apply to humans.

... It turned out they were in the perfect spot to look for statistical rules. They had databases of old and new bull semen. They had old and new production data. In essence, it wasn't that difficult to generate rules fortransforming genomic data into real-world predictions. Despite -- or because of -- the effectiveness of traditional breeding techniques, molecular biology has been applied in the field for years in different ways. Given that breeders were trying to discover bulls' hidden genetic profiles by evaluating the traits in their offspring that could be measured, it just made sense to start generating direct data about the animals' genomes. 

"Each of the bulls on the sire list, we have 50,000 genetic markers. Most of those, we have 700,000," the USDA's VanRaden said. "Every month we get another 12,000 new calves, the DNA readings come in and we send the predictions out. We have a total of 200,000 animals with DNA analysis. That's why it's been so easy. We had such a good phenotype file and we had DNA stored on all these bulls."

... Nowadays breeders can choose between "genomic bulls," which have been evaluated based purely on their genes and "proven bulls," for which real world data is available. Discussions among dairy breeders show that many are beginning to mix in younger bulls with good-looking genomic data into the breeding regimens. How well has it gone? The first of the bulls who were bred from their genetic profiles alone, are receiving their initial production data. So far, it seems as if the genomic estimates were a little high, but more accurate than traditional methods alone.

The unique dataset and success of dairy breeders now has other scientists sniffing around their findings. Leonid Kruglyak, a genomics professor at Princeton, told me that "a lot of the statistical techniques and methodology" that connect phenotype and genotype were developed by animal breeders. In a sense, they are like codebreakers. If you know the rules of encoding. it's not difficult to put information in one end and have it pop out the other as a code. But if you're starting with the code, that's a brutally difficult problem. And it's the one that diary geneticists have been working on.

(Kruglyak was a graduate student in biophysics at Berkeley under Bill Bialek when I was there.)

... John Cole, yet another USDA animal improvement scientist, generated an estimate of the perfect bull by choosing the optimal observed genetic sequences and hypothetically combining them. He found that the optimal bull would have a net merit value of $7,515, which absolutely blows any current bull out of the water. In other words, we're nowhere near creating the perfect milk machine.  

Here's a recent paper on the big data aspects of genomic selection applied to animal breeding.

Applied genomics: the genetic "super cow"

This figure and the excerpt below come from the genomics column of The Bullvine, a publication aimed at dairy breeders (via Carl Shulman). In the highly-developed world of genomic cattle breeding, it has been speculated that the theoretical "maximal type" bull could have ten times the merit (defined in terms of increased earnings of its progeny, see below) of the top breeding bulls that exist today. Sound familiar? See This is for PZ Myers for related discussion concerning human cognition.

While the "maximal type" may never be achieved, it is interesting that each new top sire surpasses the previous record holder, and previous generations, by a substantial margin. In the human context, this would correspond to a steady stream of individuals, each of greater ability, all of whom surpass the greatest historical geniuses.

THE GENETIC “SUPER COW” – MYTH VS REALITY

By Andrew Hunt

During the recent “Advancing Dairy Cattle Genetics: Genomics and Beyond”, Paul VanRaden with USDA’s Animal Improvement Programs Laboratory pointed out that “If we took the best haplotypes (genes) from all the cows genomic tested to date, we would have a cow at $7515 Net Merit”. That’s pretty spectacular considering the current top Sire on the $NM list is DE VOLMER DG SUPERSHOT at 1000 $NM.

Now to put that into perspective the current rate of gain is $80 per year. So in order to breed that $7515 animal it would take us 81 years to actually breed that animal. Therefore it raises the question whether such an animal is actually achievable and is there technology out there that could accelerate the process of getting that Super Cow.

The interesting fact here is the greatly accelerated rate of genetic gain since the introduction of genomics. This results for the most part from the greatly shortened generation interval. Females are now being used as bull mothers 18 months sooner than in the past (as yearlings vs mid 1st lactation), and sires of sons are now being used 24 and sometime 36 months sooner than they were in the past. (Genomic indexes vs waiting for proven sires). The almost 40% increase in reliability of estimated transmitting ability has breeders and AI companies contracting and working with these elite animals at a significantly younger age. ...

See also this post from 2012: Genomic prediction: no bull

The Atlantic: ... Data-driven predictions are responsible for a massive transformation of America's dairy cows. While other industries are just catching on to this whole "big data" thing, the animal sciences -- and dairy breeding in particular -- have been using large amounts of data since long before VanRaden was calculating the outsized genetic impact of the most sought-after bulls with a pencil and paper in the 1980s. Dairy breeding is perfect for quantitative analysis. Pedigree records have been assiduously kept; relatively easy artificial insemination has helped centralized genetic information in a small number of key bulls since the 1960s; there are a relatively small and easily measurable number of traits -- milk production, fat in the milk, protein in the milk, longevity, udder quality -- that breeders want to optimize; each cow works for three or four years, which means that farmers invest thousands of dollars into each animal, so it's worth it to get the best semen money can buy. The economics push breeders to use the genetics.

The bull market (heh) can be reduced to one key statistic, lifetime net merit, though there are many nuances that the single number cannot capture. Net merit denotes the likely additive value of a bull's genetics. The number is actually denominated in dollars because it is an estimate of how much a bull's genetic material will likely improve the revenue from a given cow. A very complicated equation weights all of the factors that go into dairy breeding and -- voila -- you come out with this single number. For example, a bull that could help a cow make an extra 1000 pounds of milk over her lifetime only gets an increase of $1 in net merit while a bull who will help that same cow produce a pound more protein will get $3.41 more in net merit. An increase of a single month of predicted productive life yields $35 more. When you add it all up, Badger-Fluff Fanny Freddie has a net merit of $792. No other proven sire ranks above $750 and only seven bulls in the country rank above $700.

... theoretical calculations suggest that even outliers with net merit of $700-800 will be eclipsed by specimens with 10x higher merit that can be produced by further selection on existing genetic variation. Similar results apply to humans. ...

Frontiers in cattle genomics

A correspondent updates us on advances in genomic cattle breeding. See also Genomic Prediction: No Bull and It's all in the gene: cows.

More than a million cattle in the USDA dairy GWAS system (updated with new breeding value predictions weekly), as cost per marker drops exponentially: https://www.cdcb.us/Genotype/cur_freq.html

The NM$ (Net Merit in units of dollars) utility function for selection is more and more sophisticated (able to avoid bad trade-offs from genetic correlations): http://www.ars.usda.gov/research/publications/publications.htm?SEQ_NO_115=310013

Cheap genotyping has allowed mass testing of cows, and made it practical to use dominance in models and to match up semen and cow for dominance synergies and heterosis (the dominance component is small compared to the additive one, as usual: for milk yield 5-7% dominance variance, 21-35% additive): http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0103934

[Note: additive heritability for the traits cattle breeders work on is significantly lower than for cognitive ability.]

Matching mates to reduce inbreeding (without specific markers for dominance effects) by looking at predicted ROH: http://www.ars.usda.gov/research/publications/publications.htm?SEQ_NO_115=294115

Identifying recessive lethals and severe diseases: http://aipl.arsusda.gov/reference/recessive_haplotypes_ARR-G3.html http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0054872

For humans, see Genetic architecture and predictive modeling of quantitative traits.

It's all in the gene: cows

Some years ago a German driver took me from the Perimeter Institute to the Toronto airport. He was an immigrant to Canada and had a background in dairy farming. During the ride he told me all about driving German farmers to buy units of semen produced by highly prized Canadian bulls. The use of linear polygenic models in cattle breeding is already widespread, and the review article below gives some idea as to the accuracy.

See also Genomic Prediction: No Bull and Plenty of room at the top.

Invited Review: Reliability of genomic predictions for North American Holstein bulls

Journal of Dairy Science Volume 92, Issue 1, Pages 16–24, January 2009.
DOI: http://dx.doi.org/10.3168/jds.2008-1514

Genetic progress will increase when breeders examine genotypes in addition to pedigrees and phenotypes. Genotypes for 38,416 markers and August 2003 genetic evaluations for 3,576 Holstein bulls born before 1999 were used to predict January 2008 daughter deviations for 1,759 bulls born from 1999 through 2002. Genotypes were generated using the Illumina BovineSNP50 BeadChip and DNA from semen contributed by US and Canadian artificial-insemination organizations to the Cooperative Dairy DNA Repository. Genomic predictions for 5 yield traits, 5 fitness traits, 16 conformation traits, and net merit were computed using a linear model with an assumed normal distribution for marker effects and also using a nonlinear model with a heavier tailed prior distribution to account for major genes. The official parent average from 2003 and a 2003 parent average computed from only the subset of genotyped ancestors were combined with genomic predictions using a selection index. Combined predictions were more accurate than official parent averages for all 27 traits. The coefficients of determination (R2) were 0.05 to 0.38 greater with nonlinear genomic predictions included compared with those from parent average alone. Linear genomic predictions had R2 values similar to those from nonlinear predictions but averaged just 0.01 lower. The greatest benefits of genomic prediction were for fat percentage because of a known gene with a large effect. The R2 values were converted to realized reliabilities by dividing by mean reliability of 2008 daughter deviations and then adding the difference between published and observed reliabilities of 2003 parent averages. When averaged across all traits, combined genomic predictions had realized reliabilities that were 23% greater than reliabilities of parent averages (50 vs. 27%), and gains in information were equivalent to 11 additional daughter records. Reliability increased more by doubling the number of bulls genotyped than the number of markers genotyped. Genomic prediction improves reliability by tracing the inheritance of genes even with small effects.


Results and Discussion: ... Marker effects for most other traits were evenly distributed across all chromosomes with only a few regions having larger effects, which may explain why the infinitesimal model and standard quantitative genetic theories have worked well. The distribution of marker effects indicates primarily polygenic rather than simple inheritance and suggests that the favorable alleles will not become homozygous quickly, and genetic variation will remain even after intense selection. Thus, dairy cattle breeders may expect genetic progress to continue for many generations.

... Most animal breeders will conclude that these gains in reliability are sufficient to make genotyping profitable before breeders invest in progeny testing or embryo transfer. Rates of genetic progress should increase substantially as breeders take advantage of these new tools for improving animals (Schaeffer, 2008). Further increases in number of genotyped bulls, revisions to the statistical methods, and additional edits should increase the precision of future genomic predictions.

Table 3

Trait	Parent average		Genomic prediction			Gain from nonlinear genomic prediction compared with published parent average
	Published	Observed	Expected	Linear	Nonlinear
Net merit	30	14	67	53	53	23
Milk yield	35	32	69	56	58	23
Fat yield	35	17	69	65	68	33
Protein yield	35	31	69	58	57	22
Fat percentage	35	29	69	69	78	43
Protein percentage	35	32	69	62	69	34
Productive life	27	28	55	42	45	18

"Horses ain't like people, man. They can't make themselves better than they're born. See, with a horse, it's all in the gene. It's the fucking gene that does the running. The horse has got absolutely nothing to do with it." --- Paulie (Eric Roberts) in The Pope of Greenwich Village

Fast fox evolution

This Times article describes controlled (should I say directed?) breeding experiments on small mammals such as foxes and rats performed by a Siberian scientist Dmitri Belyaev. Belyaev has managed to produce nice (tame, domesticated) and nasty (aggressive) versions of each animal in a surprisingly short time by simply breeding according to exhibited trait. The tame foxes can even perform difficult cognitive tasks like reading humans well enough to determine what they are looking at -- dogs can do this, but generally smarter chimps cannot. It's a great demonstration of how fast evolution can proceed when selection pressure is strong enough. One of the fascinating side effects of selection based on behavior is that certain physical traits (in foxes: floppy ears, white patches of fur and differently shaped skulls) were also altered, so the tame foxes can be readily identified versus wild or aggressive ones. (I don't see why this would be a priori implausible: the genes for tameness might have superficial physical as well as behavioral effects.)

These results rule out any simple-minded conclusions about whether superficial physical traits might be correlated to cognitive or behavioral traits. Sub-populations that look alike might actually behave alike. Sometimes you can judge a book by its cover :-)

NYTimes: Belyaev chose to test his theory on the silver fox, a variant of the common red fox, because it is a social animal and is related to the dog. Though fur farmers had kept silver foxes for about 50 years, the foxes remained quite wild. Belyaev began his experiment in 1959 with 130 farm-bred silver foxes, using their tolerance of human contact as the sole criterion for choosing the parents of the next generation.

“The audacity of this experiment is difficult to overestimate,” Dr. Fitch has written. “The selection process on dogs, horses, cattle or other species had occurred, mostly unconsciously, over thousands of years, and the idea that Belyaev’s experiment might succeed in a human lifetime must have seemed bold indeed.”

In fact, after only eight generations, foxes that would tolerate human presence became common in Belyaev’s stock. Belyaev died in 1985, but his experiment was continued by his successor, Lyudmila N. Trut. The experiment did not become widely known outside Russia until 1999, when Dr. Trut published an article in American Scientist. She reported that after 40 years of the experiment, and the breeding of 45,000 foxes, a group of animals had emerged that were as tame and as eager to please as a dog.

As Belyaev had predicted, other changes appeared along with the tameness, even though they had not been selected for. The tame silver foxes had begun to show white patches on their fur, floppy ears, rolled tails and smaller skulls.

...There was far more to Belyaev’s experiment than the production of tame foxes. He developed a parallel colony of vicious foxes, and he started domesticating other animals, like river otters and mink. Realizing that genetics can be better studied in smaller animals, Belyaev also started a study of rats, beginning with wild rats caught locally. His rat experiment was continued after his death by Irina Plyusnina. Siberian gray rats caught in the wild, bred separately for tameness and for ferocity, have developed these entirely different behaviors in only 60 or so generations.

The collection of species bred by Belyaev and his successors form an unparalleled resource for studying the process and genetics of domestication. In a recent visit to Novosibirsk, Dr. Brian Hare of the Planck Institute used the silver foxes to probe the unusual ability of dogs to understand human gestures.

If a person hides food and then points to the location with a steady gaze, dogs will instantly pick up on the cue, while animals like chimpanzees, with considerably larger brains, will not. Dr. Hare wanted to know if dogs’ powerful rapport with humans was a quality that the original domesticators of the dog had selected for, or whether it had just come along with the tameness, as implied by Belyaev’s hypothesis.

He found that the fox kits from Belyaev’s domesticated stock did just as well as puppies in picking up cues from people about hidden food, even though they had almost no previous experience with humans. The tame kits performed much better at this task than the wild kits did. When dogs were developed from wolves, selection against fear and aggression “may have been sufficient to produce the unusual ability of dogs to use human communicative gestures,” Dr. Hare wrote last year in the journal Current Biology.

Genius (Nautilus Magazine)

The article excerpted below, in the science magazine Nautilus, is an introduction to certain ideas from my paper On the genetic architecture of intelligence and other quantitative traits.

Super-Intelligent Humans Are Coming (Nautilus, special issue: Genius)

Genetic engineering will one day create the smartest humans who have ever lived.


Lev Landau, a Nobelist and one of the fathers of a great school of Soviet physics, had a logarithmic scale for ranking theorists, from 1 to 5. A physicist in the first class had ten times the impact of someone in the second class, and so on. He modestly ranked himself as 2.5 until late in life, when he became a 2. In the first class were Heisenberg, Bohr, and Dirac among a few others. Einstein was a 0.5!

My friends in the humanities, or other areas of science like biology, are astonished and disturbed that physicists and mathematicians (substitute the polymathic von Neumann for Einstein) might think in this essentially hierarchical way. Apparently, differences in ability are not manifested so clearly in those fields. But I find Landau’s scheme appropriate: There are many physicists whose contributions I cannot imagine having made.

I have even come to believe that Landau’s scale could, in principle, be extended well below Einstein’s 0.5. The genetic study of cognitive ability suggests that there exist today variations in human DNA which, if combined in an ideal fashion, could lead to individuals with intelligence that is qualitatively higher than has ever existed on Earth: Crudely speaking, IQs of order 1,000, if the scale were to continue to have meaning.

... Does g predict genius? Consider the Study of Mathematically Precocious Youth, a longitudinal study of gifted children identified by testing (using the SAT, which is highly correlated with g) before age 13. All participants were in the top percentile of ability, but the top quintile of that group was at the one in 10,000 level or higher. When surveyed in middle age, it was found that even within this group of gifted individuals, the probability of achievement increased drastically with early test scores. For example, the top quintile group was six times as likely to have been awarded a patent than the lowest quintile. Probability of a STEM doctorate was 18 times larger, and probability of STEM tenure at a top-50 research university was almost eight times larger. It is reasonable to conclude that g represents a meaningful single-number measure of intelligence, allowing for crude but useful apples-to-apples comparisons.

... Once predictive models are available, they can be used in reproductive applications, ranging from embryo selection (choosing which IVF zygote to implant) to active genetic editing (for example, using CRISPR techniques). In the former case, parents choosing between 10 or so zygotes could improve the IQ of their child by 15 or more IQ points. This might mean the difference between a child who struggles in school, and one who is able to complete a good college degree. Zygote genotyping from single cell extraction is already technically well developed, so the last remaining capability required for embryo selection is complex phenotype prediction. The cost of these procedures would be less than tuition at many private kindergartens, and of course the consequences will extend over a lifetime and beyond.

The corresponding ethical issues are complex and deserve serious attention in what may be a relatively short interval before these capabilities become a reality. Each society will decide for itself where to draw the line on human genetic engineering, but we can expect a diversity of perspectives. Almost certainly, some countries will allow genetic engineering, thereby opening the door for global elites who can afford to travel for access to reproductive technology. As with most technologies, the rich and powerful will be the first beneficiaries. Eventually, though, I believe many countries will not only legalize human genetic engineering, but even make it a (voluntary) part of their national healthcare systems.

The alternative would be inequality of a kind never before experienced in human history.

Note Added: I posted the following in the comments at the Nautilus site and also on Hacker News (ycombinator), which has a big thread.

The question of additivity of genetic effects is discussed in more detail in reference [1] above (sections 3.1 and also 4): http://arxiv.org/pdf/1408.3421...

In plant and animal genetics it is well established that the majority of phenotype variance (in complex traits) which is under genetic control is additive. (Linear models work well in species ranging from corn to cows; cattle breeding is now done using SNP genotypes and linear models to estimate phenotypes.) There are also direct estimates of the additive / non-additive components of variance for human height and IQ, from twin and sibling studies. Again, the conclusion is the majority of variance is due to additive effects.

There is a deep evolutionary reason behind additivity: nonlinear mechanisms are fragile and often "break" due to DNA recombination in sexual reproduction. Effects which are only controlled by a single locus are more robustly passed on to offspring. Fisher's fundamental theorem of natural selection says that the rate of change of fitness is controlled by additive variance in sexually reproducing species under relatively weak selection.

Many people confuse the following statements:

"The brain is complex and nonlinear and many genes interact in its construction and operation."

"Differences in brain performance between two individuals of the same species must be due to nonlinear effects of genes."

The first statement is true, but the second does not appear to be true across a range of species and quantitative traits.

Final technical comment: even the nonlinear part of the genetic architecture can be deduced using advanced methods in high dimensional statistics (see section 4.2 in [1] and also http://arxiv.org/abs/1408.6583....

##################

I just realized I've said all of this already in http://arxiv.org/pdf/1408.3421... (p.16):

... The preceding discussion is not intended to convey an overly simplistic view of genetics or systems biology. Complex nonlinear genetic systems certainly exist and are realized in every organism. However, quantitative differences between individuals within a species may be largely due to independent linear effects of specific genetic variants. As noted, linear effects are the most readily evolvable in response to selection, whereas nonlinear gadgets are more likely to be fragile to small changes. (Evolutionary adaptations requiring significant changes to nonlinear gadgets are improbable and therefore require exponentially more time than simple adjustment of frequencies of alleles of linear effect.) One might say that, to first approximation, Biology = linear combinations of nonlinear gadgets, and most of the variation between individuals is in the (linear) way gadgets are combined, rather than in the realization of different gadgets in different individuals.

Linear models work well in practice, allowing, for example, SNP-based prediction of quantitative traits (milk yield, fat and protein content, productive life, etc.) in dairy cattle. ...