Election 2020: quant analysis of new party registrations vs actual votes

I think we should ascribe very high uncertainty to polling results in this election, for a number of reasons including the shy Trump voter effect as well as the sampling corrections applied which depend heavily on assumptions about likely turnout. 

Graphs below are from a JP Morgan quant analysis of changes in number of registered voters by party and state, and the correlation with actual votes in subsequent election. Of course it is possible that negative covid impact has largely counteracted the effect discussed below (which is an integrated effect over the last 4 years) -- i.e., Trump was in a strong position at the beginning of 2020 but has declined since then. 

This is an unusual election for a number of reasons so it's quite hard to call the outcome. There's also a good chance the results on election night will be heavily contested.

The author of this analysis is Marko Kolanovic, Global Head of Macro Quantitative and Derivatives Strategy at J.P. Morgan. He graduated from New York University with a PhD in theoretical high-energy physics.

Anyone with high conviction about the election is welcome to post their analysis in the comments.

Chaos Monkeys: physics to Goldman to Y Combinator to Twitter to Facebook

Highly recommended! I blogged about this guy 5 years ago here: From physics to Goldman to Y Combinator. The book is hilarious and pretty accurate, AFAICT. I don't know much about Facebook corporate culture or that particular era of ad monetization, but the finance and startup stuff all rings true.

The reality is, Silicon Valley capitalism is very simple:
Investors are people with more money than time.
Employees are people with more time than money.
Entrepreneurs are the seductive go-between.
Startups are business experiments performed with other people's money.

I was a Berkeley PhD student in physics when the first dot-com bubble grew to bursting and popped around 2001. Between the month-long backpacking trips and the telenovela-esque romances, I switched thesis topic three times, and felt my twenty-something vitality slipping away in academic wankery. Inspired by Michael Lewis’ Liar’s Poker and the prior example of many a failed physicist, I looked for a Wall Street gig as a way out. Very improbably, I landed a job on the trading desk of Goldman Sachs, earning twice what my tenured professor made, pricing and modeling credit derivatives at ground-zero of the credit bubble. I may have owned one pair of lace-up shoes at the time, but I got used to speaking in quantities of hundreds of millions of dollars, and thinking a million was a ‘buck’, i.e., a rounding error for most purposes. I was very far away indeed from Berkeley.

Right around 2008, when Lehman Brothers and Bear Stearns blew up, I knew the financial jig would be up for a while (and possibly forever), unlike most of my colleagues, who seemed to think orgies of rapacious greed lasted forever. The only piece of the US economy that would be spared the apocalypse was clear in my mind: the Bay Area tech of my languid grad school days, and all that VC money that (hopefully) hadn’t touched the mortgage bubble..

Two weeks later, I started as employee number seventy-something at a venture-backed advertising startup so incompetent and vile I’ll save the historical distaste for later. Bookended as it was by experiences at Facebook and Goldman, my time there was instructive in its awfulness and how not to run a company. But there was one piece of upside: I learned how online advertising worked, specifically its ad exchange variants. As a ‘research scientist’ I tortured every piece of data until it confessed, and used it to predict user behavior, value of media purchased, and optimal bids in the largest ad auctions in the world. Dull stuff you might say, but it’s what pays for the Internet, and it would set me light-years ahead of anyone inside Facebook Ads, when the time came.

But we’re jumping ahead.

Along with the two best engineers at Shitty Unnamed Company, I applied and was accepted to Y Combinator, the Valley’s leading startup incubator. We pitched some wild, ridiculous idea around local businesses that was doomed from the start, which eventually morphed into a novel tool for managing Google search campaigns for small businesses. The tool was beautiful, innovative, and didn’t make us a dime. More bad news: We got vindictively and frivolously sued by Shitty Company and fought an existential legal battle we narrowly won by being lying, ruthless little shits. We couldn’t raise money. We had co- founder and morale issues. Every ill that plagues early-stage startups visited us in turn, like some admonitory biblical tale about what happens if you fuck with the Israelites. ...

... Every startup entrepreneur faces the immense disadvantage of playing a crooked, complex game for the first time, against a world composed mostly of masters. Arrayed against you is an army of wily, self-interested venture capitalists who know term sheets better than their wife’s ass. Or seductive sales execs who could make pedophilia and genocide enforceable via a legal contract. Or petulant co-founders with hidden agendas and momentarily suppressed grievances. Or ungrateful employees who are exploiting your startup until they can start their own. Or thick-headed journalists with urgent deadlines who just want you for a misleading quote. You get trounced again and again, and the only hope is that you learn something of the game before expiring. This is your principal challenge as a first-time entrepreneur: to learn the game faster than you burn cash and relationships.

Theory, Money, and Learning

After 25+ years in theoretical physics research, the pattern has become familiar to me. Talented postdoc has difficulty finding a permanent position (professorship), and ends up leaving the field for finance or Silicon Valley. The final phase of the physics career entails study of entirely new subjects, such as finance theory or machine learning, and developing new skills, such as coding.

My most recent postdoc interviewed with big hedge funds in Manhattan and also in the bay area. He has accepted a position in AI -- working on Deep Learning -- at the Silicon Valley research lab of a large technology company. His compensation is good (significantly higher than most full professors!) and future prospects in this area of research are exciting. With some luck, great things are possible.

He returned the books in the picture last week.

James Simons: Mathematics, Common Sense, and Good Luck

A great MIT colloquium by Jim Simons (intro by I. Singer). Interesting discussion @28 min about how Simons (after leaving mathematics at 38) became an investor. Initially, he relied both on fundamental / event-driven analysis (reading the newspaper ;-) as well as computer models. But Simons eventually decided on a completely model-driven approach, and the rest is history.

@38 min: on RenTech's secret, We start with first rate scientists ... Great infrastructure ... New ideas shared and discussed as soon as possible in an open environment ... Compensation based on overall firm performance ...

@44 min: Be guided by beauty ... Try to do it RIGHT ... Don't give up and hope for some good luck!

@48 min: a defense of HFT ... the cost of liquidity?

@55 min: world's greatest investor is a Keynesian :-)

@58 min: brief precis of financial crisis ... See also here.

See also Jim Simons is my hero.

Michael Lewis, Longform, and HFT

I recommend this podcast interview with Michael Lewis (Longform.org). As with other Longform interviews, it focuses on his work/career as a writer and journalist. Flash Boys is only mentioned in passing.

Re: Flash Boys and high frequency trading (HFT): some years ago, post-financial crisis, I was in touch with some people at the SEC (Securities Exchange Commission, not the football conference) who were trying to build a quant group. Traditionally the SEC hires mostly lawyers, but the financial crisis and the Madoff scandal convinced them they needed to improve their analytical horsepower. I posted this job ad on the blog for them (no idea whether it was useful). At the time I also suggested that the SEC investigate HFT practices, but they didn't seem to know what I was talking about! 8-(

Goldman v. Aleynikov

Michael Lewis on the Aleynikov-Goldman HFT matter. The article mentions that Goldman trailed other players like Citadel when Aleynikov was hired. The head of HFT at Citadel was (I believe) a contemporary of mine in grad school at Berkeley, who did his dissertation in string theory. Malyshev, the guy who hired Aleynikov from Goldman, had his own legal problems when he left Citadel.

Vanity Fair: ... Serge knew nothing about Wall Street. The headhunter sent him a bunch of books about writing software on Wall Street, plus a primer on how to make it through a Wall Street job interview, and told him he could make a lot more than the $220,000 a year he was making at the telecom. Serge felt flattered, and liked the headhunter, but he read the books and decided Wall Street wasn’t for him. He enjoyed the technical challenges at the giant telecom and didn’t really feel the need to earn more money. A year later the headhunter called him again. By 2007, IDT was in financial trouble. His wife, Elina, was carrying their third child, and they would need to buy a bigger house. Serge agreed to interview with the Wall Street firm that especially wanted to meet him: Goldman Sachs.

... And then Wall Street called. Goldman Sachs put Serge through a series of telephone interviews, then brought him in for a long day of face-to-face interviews. These he found extremely tense, even a bit weird. “I was not used to seeing people put so much energy into evaluating other people,” he said. One after another, a dozen Goldman employees tried to stump him with brainteasers, computer puzzles, math problems, and even some light physics. It must have become clear to Goldman (as it was to Serge) that he knew more about most of the things he was being asked than did his interviewers. At the end of the first day, Goldman invited him back for a second day. He went home and thought it over: he wasn’t all that sure he wanted to work at Goldman Sachs. “But the next morning I had a competitive feeling,” he says. “I should conclude it and try to pass it because it’s a big challenge.”

... He returned for another round of Goldman’s grilling, which ended in the office of one of the high-frequency traders, another Russian, named Alexander Davidovich. A managing director, he had just two final questions for Serge, both designed to test his ability to solve problems.

The first: Is 3,599 a prime number?

Serge quickly saw there was something strange about 3,599: it was very close to 3,600. He jotted down the following equations: 3599 = (3600 – 1) = (602 – 12) = (60 – 1) (60 + 1) = 59 times 61. Not a prime number.

The problem wasn’t that difficult, but, as he put it, “it was harder to solve the problem when you are anticipated to solve it quickly.” It might have taken him as long as two minutes to finish. The second question the Goldman managing director asked him was more involved—and involving. He described for Serge a room, a rectangular box, and gave him its three dimensions. “He says there is a spider on the floor and gives me its coordinates. There is also a fly on the ceiling, and he gives me its coordinates as well. Then he asked the question: Calculate the shortest distance the spider can take to reach the fly.” The spider can’t fly or swing; it can only walk on surfaces. The shortest path between two points was a straight line, and so, Serge figured, it was a matter of unfolding the box, turning a three-dimensional object into a one-dimensional surface, then using the Pythagorean theorem to calculate the distances. It took him several minutes to work it all out; when he was done, Davidovich offered him a job at Goldman Sachs. His starting salary plus bonus came to $270,000.

... One small example of the kind of problems Serge found: Goldman’s trading on the NASDAQ exchange. Goldman owned the lone (unmarked) building directly across the street from NASDAQ in Carteret, New Jersey. The building housed Goldman’s dark pool. When Serge arrived, 40,000 messages per second were flying back and forth between computers inside the two buildings. Proximity, he assumed, must offer Goldman Sachs some advantage—after all, why else buy the only building anywhere near the exchange? But when he looked into it he found that, to cross the street from Goldman to NASDAQ, a signal took five milliseconds, or nearly as much time as it took a signal to travel on the fastest network from Chicago to New York. “The theoretical limit [of sending a signal] from Chicago to New York is something like seven milliseconds,” says Serge. “Everything more than that is the friction caused by man.” The friction could be caused by physical distance—say, if the signal moving across the street in Carteret, New Jersey, traveled in something less direct than a straight line. It could be caused by computer hardware. (The top high-frequency-trading firms chuck out their old gear and buy new stuff every few months.) But it could also be caused by slow, clunky software—and that was Goldman’s problem. Their high-frequency-trading platform was designed, in typical Goldman style, as a centralized hub-and-spoke system. Every signal sent was required to pass through the mother ship in Manhattan before it went back out into the marketplace. “But the latency [the five milliseconds] wasn’t mainly due to the physical distance,” says Serge. “It was because the traffic was going through layers and layers of corporate switching equipment.” ...

In this last paragraph Aleynikov sounds more like Sakharov than a millionaire quant ;-)

“If the incarceration experience doesn’t break your spirit, it changes you in a way that you lose many fears. You begin to realize that your life is not ruled by your ego and ambition and that it can end any day at any time. So why worry? You learn that, just like on the street, there is life in prison, and random people get there based on the jeopardy of the system. The prisons are filled with people who crossed the law, as well as by those who were incidentally and circumstantially picked and crushed by somebody else’s agenda. On the other hand, as a vivid benefit, you become very much independent of material property and learn to appreciate very simple pleasures in life such as the sunlight and morning breeze.”

Hail to the quants, pundit fail

Pundit idiocracy: "Close race", "Too close to call", "Neck and neck". (I heard this all day long.)

Quants and data geeks: "Obama will win. Unlikely to be close."

From an earlier post High V, Low M:

high verbal ability ... is useful for appearing to be smart, or for winning arguments and impressing other people, but it's really high math ability that is useful for discovering things about the world -- that is, discovering truth or reasoning rigorously.

... The statistical techniques used to analyze data obtained in a messy, complex world require mathematical ability to practice correctly. In almost all realistic circumstances hypothesis testing is intrinsically mathematical.

See also Obama wins! and Expert Prediction. Scorecard of predictions here (accuracy highly correlated with M, not V ;-)

Who is this guy?

Xu Cheng, Moodys’ Analytics: Obama 303, Romney 235 (Note that this prediction was made back in February) “This prediction is tied to the Moody’s Analytics current baseline forecast for U.S. growth, which assumes that most states will continue to recover at slow to moderate speeds.”

Obama wins!

At least, according to the quants who performed the mind boggling, incomprehensible, mysterious, nearly impossible task of averaging state poll numbers to estimate likely electoral vote totals.

Pundit and non-quant reactions evidence of Idiocracy. See earlier post Bounded Cognition.

Chronicle: ... While it may not seem likely, poll aggregation is a threat to the supremacy of the punditocracy. In the past week, you could sense that some high-profile media types were being made slightly uncomfortable by the bespectacled quants, with their confusing mathematical models and zippy computer programs. The New York Times columnist David Brooks said pollsters who offered projections were citizens of “sillyland.”

Maybe, but the recent track record in sillyland is awfully solid. In the 2008 presidential election, Silver correctly predicted 49 of 50 states. Wang was off by only one electoral vote. Meanwhile, as Silver writes in his book, numerous pundits confidently predicted a John McCain victory based on little more than intestinal twinges.

... Most journalists are ill equipped to interpret data, he says (and few journalists would disagree), so they view statistics with skepticism and occasionally, in the case of Brooks, disdain. “The data-driven people are going to win in the long run,” Jackman says.

He sees it as part of the rise of what’s being called Big Data—that is, using actual information to make decisions. As Jackman points out, Big Data is already changing sports and business, and it may be that pundits are the equivalents of the baseball scouts in Michael Lewis’s book Moneyball, caring more about the naturalness of a hitter’s swing than whether he gets on base.

“Why,” Jackman wonders, “should political commentary be exempt from this movement?”

... Last week the professional pundit and MSNBC host Joe Scarborough ranted that people like Silver, Wang, Linzer, and Jackman—who think the presidential race is “anything but a tossup”—should be kept away from their computers “because they’re jokes.” Silver responded by challenging Scarborough to bet $1,000 on Romney (in the form of a donation to the American Red Cross) if he was so sure. This led to hand-wringing about whether it was appropriate for someone affiliated with The New York Times to make crass public wagers.

But the bet seemed like an important symbolic moment. The poll aggregators have skin in the game. They’ve made statistical forecasts and published them, not just gut-feeling guesses on Sunday-morning talk shows. And, in Silver’s case, as a former professional poker player, he is willing to back it up with something tangible.

Alex Tabarrok, an economist and blogger for Marginal Revolution, applauded, calling such bets a “tax on bullshit.” ...

Shout out to Sam Wang, Caltech '86 :-)

Quants and campaigns

Big data, analytics, randomized experiments and modern political campaigns. "... the most advanced political marketers are ahead of commercial marketers." Turnout efforts targeted at voters whose preference is predictable are more efficient (in votes per dollar spent) than attempts at persuasion.

Sasha Issenberg shows how cutting-edge social science and analytics are reshaping the modern political campaign, upending the way political campaigns are run in the 21st century. In The Victory Lab: The Secret Science of Winning Campaigns Issenberg writes about the techniques—including persuasion experiments, innovative ways to mobilize voters, heavily researched electioneering methods—and shows how they’re being used.

$440M in 45 minutes

Does this qualify as the most expensive software bug of all time? Raises concerns about our future as passengers in driverless vehicles ;-)

I suppose it's a positive that Knight had to recognize the losses immediately, instead of sweeping them under the rug by adjusting a parameter in a risk model (see, e.g., JP Morgan whale + a million other recent examples). Would Knight have lost even more money if the exchange hadn't shut down trading in the affected names?

NYTimes: $10 million a minute. 

That’s about how much the trading problem that set off turmoil on the stock market on Wednesday morning is already costing the trading firm. 

The Knight Capital Group announced on Thursday that it lost $440 million when it sold all the stocks it accidentally bought Wednesday morning because a computer glitch. ... 

The problem on Wednesday led the firm’s computers to rapidly buy and sell millions of shares in over a hundred stocks for about 45 minutes after the markets opened. Those trades pushed the value of many stocks up, and the company’s losses appear to have occurred when it had to sell the overvalued shares back into the market at a lower price.   

The company said the problems happened because of new trading software that had been installed. The event was the latest to draw attention to the potentially destabilizing affect of the computerized trading that has increasingly dominated the nation’s stock markets.

This says it all. Previous posts on high frequency trading.

Update: My representative is on the job!

NYTimes: ... Some critics of the current market structure have said that much bolder reform is needed. One change that has been contemplated is a financial transaction tax, which would force firms to pay a small levy on each trade. At the right level, this could pare back high-frequency trading without undermining other types, supporters say. 

“It would benefit investors because there would be less volatility in the market,” said Representative Peter DeFazio, a Democrat of Oregon. He introduced a bill containing a financial transaction tax last year. 

Opponents of such a levy say that it could hurt the markets and even make it more expensive for companies to raise capital. 

“I would be very concerned about unintended consequences,” said Mr. Sauter. 

But Representative DeFazio, who favors a levy of three-hundredths of a percentage point on each trade, says he thinks the benefits of high-frequency trading are overstated. “Some people say it’s necessary for liquidity, but somehow we built the strongest industrial nation on earth without algorithmic trading,” he said.

Finance and the allocation of human capital

How finance sucks human capital from more productive activities. FT Alphaville:

... a bloated financial sector can also suck in more than its share of talent, hampering the development of other sectors.8

That last sentence is a smack in the face, isn’t it? FT Alphaville was dying to know what Footnote 8 would contain.

Here it is:

8 See S Cecchetti and E Kharroubi, “Reassessing the impact of finance on growth”, BIS, January 2012, mimeo

So we looked it up. From the introduction:

… in our examination of industry-level data, we find that industries that are in competition for resources with finance are particularly damaged by financial booms. Specifically, we show that manufacturing sectors that are either R&D-intensive or dependent on external finance suffer disproportionate reductions in productivity growth when finance booms.

At first, these results may seem surprising. After all, a more developed financial system is supposed to reduce transaction costs, raising investment directly, as well as improve the distribution of capital and risk across the economy. 1 These two channels, through the level and composition of investment, are the mechanisms by which financial development improves growth. 2 But the financial industry competes for resources with the rest of the economy. It requires not only physical capital, in the form of buildings, computers and the like, but highly skilled workers as well. Finance literally bids rocket scientists away from the satellite industry. The result is that erstwhile scientists, people who in another age dreamt of curing cancer or flying to Mars, today dream of becoming hedge fund managers.

See also my earlier post A reallocation of human capital (pre-financial crisis).

Quants at the SEC

Two years ago I was asked by someone at the Securities Exchange Commission to post a job ad for quant risk analysts. During some of my conversations with the people there, I raised the issue of high frequency trading, which wasn't yet on their radar! Hopefully it is now.

This NYTimes article reports that things have improved a bit recently there.

NYTimes: ... Embarrassed after missing the warning signs of the financial crisis and the Ponzi scheme of Bernard L. Madoff, the agency’s enforcement division has adopted several new — if somewhat unconventional — strategies to restore its credibility. The S.E.C. is taking its cue from criminal authorities, studying statistical formulas to trace connections, creating a powerful unit to cull tips and assign cases and even striking a deal with the Federal Bureau of Investigation to have agents embedded with the regulator. 

... Mr. Sporkin has built a team of more than 40 former traders, exchange experts, accountants and securities lawyers to sift through roughly 200 pieces of intelligence a day, distilling the hottest tips into a daily “intelligence report.” “It’s the central intelligence office for the whole agency,” Mr. Sporkin said. 

The overhaul came with an upgrade in technology. The hub of Mr. Sporkin’s outfit is a “market watch room,” replete with Bloomberg terminals and real-time stock pricing monitors that keep an eye on the markets. 

... Rather than examining questionable trades in specific stocks, Mr. Hawke and his team now analyze suspicious traders and their network of connections on Wall Street. The investigators have turned to statistics, using tools like “cluster analysis” and “fuzzy matching,” to identify relationships and trading patterns that sometimes go undetected.

I, quant

A commenter linked to this Guardian interview with a UK quant. I found a number of his comments interesting enough to post here. See the original for more detail about the software he develops. I always felt that if I went into finance it would be as a trader, but with quant skills ;-)

"My parents discovered that I was of a mathematical bent aged three when I was apparently lining up my toys in order of size and then colour. I was one of these terrible, precocious kids who did their mathematics O-level aged 12. After a long academic career I ended up doing theoretical physics for my PhD, and spent a couple of years at Cern in Geneva. Many people I know from back then are still at universities, doing research and climbing the slippery slope to professorships and fellowships. They work the same astonishing long hours as I do, yet get paid a fraction and, from a purely scientific perspective, get to do some really, really interesting science. I often say (only half jokingly) that I "sold my soul" – I make a little over £200,000 a year, including my bonus."


"I said I was a quant, derived from the word 'quantitative'. We're the people of a definite mathematical bent, and if you're looking for a warrior-like analogy, we are perhaps the "armourers" of the financial industry, or, let me think … Traders are the warriors of our world; they go out and fight. I think of them as 'egos on legs'. Sharp suits, looking very smart… We quants are the trader's brain. It's our model that defines not only the risks the trader can take, the model also calculates how much risk he is taking with his particular trades at any given moment and we also predict future movements in valuation, pricing and the like."


"I have been in banking for over 20 years, and for several years I was with one of the major international investment banks. I discovered that I am just not enough of an arsehole to make it there. Why the top people at investment banks are like that? Well you have a thousand vice-presidents vying for 10 managing director posts. What do you think will happen? People will do anything to get ahead, back-biting, back-stabbing, the whole nine yards. For those of us who find life surrounded by other people difficult enough as it is, the requirement to network is hellish."


"Not sure though that I'd voluntarily swap IQ points for EQ – even though I'm certain that I'm going to end up as one of the single old blokes that you might occasionally come across – nice, big house in the country, lots of dogs, materially comfortable and yet utterly alone and mad as a fish.

Later, when asked to elaborate on that final point, he responds via email:

"I've long been aware of the prospect (with some 'tongue-in-cheek') of becoming mad as a fish, and the attractiveness of the current imbalance between EQ and IQ is that I know that my biggest, deepest fear is failure. With the current imbalance, I know that the risk of failure is reduced to its current level: eg, small but still real. That fear of failure drives me and means that I know I'm giving up anything approaching EQ in pursuit of avoidance of failure."

The new gatekeepers

The Chronicle has a nice article on what it takes to run admissions at a selective college. The requirements of communication skills (e.g., to address concerns of multiple stake holders) and ability to develop "data-informed strategies" apply to almost all high level leadership positions these days, whether in finance, business or academia. Perhaps we'll eventually see a squeezing out of High V, Low M types as leaders. (Fat chance!)

I suspect these numerate admissions deans know many things that can't be discussed in public ;-)

For how SAT scores predict college performance, see here and here.

For a description of Harvard's "holistic admissions" policies (adopted in the 1950s), which have become a model for other elite schools, see here. Then dean of admissions Wilbur Bender may not have been very quantitative, but he had excellent intuition about How the World Works. (See also here.)

'Numerical and Verbal'

A profession that once relied on anecdotes and descriptive data now runs on complex statistical analyses and market research. Knowing how to decipher enrollment outcomes is a given; knowing how to forecast the future is a must. Which students are most likely to apply, submit deposits, and matriculate? At what cost to the college? How likely will they be to graduate? Such questions echo in the modern enrollment office, which is often supported by one or more institutional researchers, as well as consulting firms that sell recruitment strategies in various flavors.

Search the job listings for top-level admissions and enrollment openings, and you will find that many colleges seek a "data-driven" leader, someone who will develop "data-informed" strategies. This past winter, for instance, Pomona College, in California, began a national search to replace Bruce J. Poch, who had stepped down after 23 years as vice president and dean of admissions. Among the qualifications listed in the job advertisement: "an ability to analyze and use data to guide decision-making and measure results."

David W. Oxtoby, Pomona's president, led the college's search committee. The modern admissions dean, he says, must have a "technical, quantitative facility," the ability to delve into the relationship between a student's SAT score and her subsequent performance in college, or why some kinds of students are more likely to enroll than others. Moreover, Pomona had decided to merge its admissions and financial-aid offices (a change many colleges have made already). So the new dean would need to speak the language of costs.

That's not to say anyone wanted to hire an accountant. Numbers, Mr. Oxtoby says, have not diminished the importance of communication skills. Pomona's search committee sought someone who could articulate the value of a liberal-arts education, and relate to faculty members. The "ideal" candidate, the job listing said, would also know how to talk to the news media; today's admissions leaders are also public-relations specialists with loud microphones.

"The job is numerical and verbal," Mr. Oxtoby says. "It's still all about relationships. You still need a sense of that person on the other end of the admissions process. If you lose that, you just become another technocrat, and you've lost the reason why you're doing this job."

Pomona interviewed a dozen candidates before hiring Seth Allen, dean of admission and financial aid at Grinnell College, in Iowa. Soon to occupy one of the premier jobs in admissions, Mr. Allen, 43, represents the next generation of enrollment chiefs. They've ascended during an era of high competition, learning how to market their colleges and massage the metrics that define success in admissions.

Although idealism may inform their work, they are clear-eyed realists. They are not introverts, for they must collaborate constantly with faculty members and other campus offices. They are diplomats who must manage competing desires: those of administrators who want to enroll more first-generation and low-income applicants, professors who want more students with high SAT scores, trustees who want to lower the tuition-discount rate. "Twenty years ago," Mr. Allen says, "there were not as many wants."

Drawn to statistics at an early age, Mr. Allen earned a bachelor's degree in economics at the Johns Hopkins University in 1990. He first worked as an admissions counselor for his alma mater, a cutting-edge laboratory in the then-burgeoning science of enrollment management. Mr. Allen learned how predictive modeling could project net tuition revenue, how many biology majors would enroll, and a hundred other outcomes.

From physics to Goldman to Y Combinator

Better to die free than live as a slave, or something like that. The author describes his path from physics grad school to Goldman to a bay area startup. Via maoxian.

Why founding a three-person startup with zero revenue is better than working for Goldman Sachs

I joined Goldman Sachs in 2005, after five flailing years in a physics Ph.D. program at Berkeley.

The average salary at Goldman Sachs in 2005 was $521,000, and that’s counting each and every trader, salesperson, investment banker, secretary, mail boy, shoe shine, and window cleaner on the payroll. In 2006, it was more like $633,000.

In the summer of 2005, I took one look at my offer letter and the Goldman Sachs logo above it, another look at my sordid grad student pad, and I got on a plane to New York within the week. I packed my copy of Liar’s Poker for reference.

My job on arrival? I was a pricing quant on the Goldman Sachs corporate credit trading desk1. We traded credit-default swaps, both distressed and investment-grade credit, and in the bizarre trading experiment assigned to me, the equity part of the corporate capital structure as well.

There were other characters in this drama. The sales guys were complete tools, with a total IQ, summing over all of them, still safely in the double digits. The traders were crafty and quick-witted, but technically unsophisticated and with the attention span of an ADHD kid hopped up on meth and Jolly Ranchers. And the quants (strategists in Goldman speak)? Mostly failed scientists (like me) who had sold out to the man and suddenly found themselves, after making it through two years of graduate quantum mechanics, with a bat-wielding gorilla peering over their shoulder (that would be the trader) asking them where their risk report was.

... The sad truth is: quants were the eunuchs at the orgy. We were the ever-present British guy in every Hollywood WWII film: there to add a touch of class and exotic sophistication, but not really matter much to the plot (and maybe even conveniently take some bad guy’s bullet).

But things weren’t all bad! At its best, when the markets presented an apocalyptic Boschian landscape of damned souls torn asunder by hellish tortures, every Goldman grunt, sergeant, or general would close ranks and form a Greek phalanx of greed. Unlike almost every other bank on the street, Goldman could actually calculate its risk across desks and asset classes, out to five decimals [see footnote].

... What’s work like now? Writing code. Worrying about everything from our credit card billing to the pile of dirty dishes in the sink that will give us all diptheria some day. Writing linkbait blog posts to get us free PR (like the one you’re reading now). Schmoozing with investors, and playing the junior high school popularity contest that is startup funding. Keeping jealous tabs on other startups to see how they’re doing compared to us. Trying to put myself in the mind of our users to make something they’d want. Oh, and launching…finally, good God…launching.

You see, starting a product from an empty text buffer is very different from keeping a well-oiled money-machine running8. I’ve had apocalyptic fights with the other founders that almost ended in fisticuffs. I’m watching my four-month-old daughter grow up via Skype. These jeans I’m wearing will likely fuse with my skin at some point if I don’t take them off. I haven’t seen a paycheck or a loving woman in much too long.

You know what I regret most though, going from Goldman to this?

Not having made the switch earlier. ...


[Some footnotes]

It’s a somewhat different story on structured credit desks, where numerical modeling is perceived to be more important. Also, on algorithmic trading desks, where the quant writes the code that does the trading, and the sometimes blurry line between quant and trader basically disappear.

At the risk of getting sued, let me throw you geeks a bone and part the Goldman veil a bit. The Goldman Sachs risk system is called SecDB (securities database), and everything at Goldman that matters is run out of it. The GUI itself looks like a settings screen from DOS 3.0, but no one cares about UI cosmetics on the Street. The language itself was called SLANG (securities language) and was a Python/Perl like thing, with OOP and the ORM layer baked in. Database replication was near-instant, and pushing to production was two keystrokes. You pushed, and London and Tokyo saw the change as fast as your neighbor on the desk did (and yes, if you fucked things up, you got 4AM phone calls from some British dude telling you to fix it). Regtests ran nightly, and no one could trade a model without thorough testing (that might sound like standard practice, but you have no idea how primitive the development culture is on the Street). The whole thing was so good, I didn’t even know what an ORM really was until I started using Rails and had to wrestle with ActiveRecord. The codebase was roughly 15MM lines when I left, and growing. I suspect my retinas are still scarred by the weird color blue SecDB was by default.

If doing a startup is like rolling a boulder up a hill, then working at Goldman Sachs is like rolling it down the hill: you just have to stay out of the way of the boulder.

How the other half lives: quants

I was asked to write a review of Scott Patterson's book The Quants for Physics World, a UK magazine. You can read the whole thing at the link below. My review went through some British copy editing, which changed the writing style slightly :-)

Physics World: This past summer I spent the long US Independence Day weekend at a reunion with three of my undergraduate classmates. All of us went on to earn PhDs in physics, but I am the only practising physicist. The others work in finance – one at a hedge fund, the other two at major banks. They all seem to be enjoying their work, and they have obviously been very successful: our reunion took place near a famous ski resort, where one of the financiers has built a 1200 m^2 retreat complete with gym, indoor pool and wine cellar.

Even the brightest graduate students in maths and physics know that their chances of assuming a position like that of their PhD supervisors are slim. For the last 20 years the cream of the crop of physicists leaving the field has gone on to positions in finance, typically in places such as New York or London. Once there, these individuals become hedge-fund managers, derivatives traders and risk managers, to take just a few examples from my own cohort.

So what do these people do? I have always been surprised that scientists in academia are not more curious about the lives of their former peers working in the "real world". For those who are interested, Scott Patterson's The Quants does an admirable job of exploring the increasingly mathematical and technological world of high finance, and the activities of the many physicists, mathematicians and engineers who inhabit it.

Patterson writes for the Wall Street Journal, and regular readers of the newspaper will recognize him as an insightful reporter who has covered a number of important topics over the years, most recently the rise of high-frequency trading. In researching the book, Patterson had access to a Who's Who of prominent "quants", a colourful group of characters with backgrounds and personalities that will be strangely familiar to anyone who has spent time among physicists. The term "quant" is short for "quantitative" and refers to those who apply mathematical or computational methods to finance. ...

... It is a pity that although Patterson gives us a broad survey of quant finance, he devotes little space to the bigger question: are developments such as the mathematization of markets and the flow of top brains to financial activities good for society? On this matter, perhaps it is best to leave the last word to Charlie Munger, the long-time investment partner of financial guru-in-chief Warren Buffet. Writing in 2006, Munger had the following to say about the "brain drain" of top talent into finance.

"I regard the amount of brainpower going into money management as a national scandal. We have armies of people with advanced degrees in physics and math in various hedge funds and private-equity funds trying to outsmart the market. A lot of…older people…can remember when none of these people existed…At Samsung, their engineers meet at 11 p.m. Our meetings of engineers [meaning our smartest citizens] are also at 11 p.m., but they're working on pricing derivatives. I think it's crazy to have incentives that drive your most intelligent people into a very sophisticated gaming system."

Physicist(s) and the flash crash

Flash crash -- mystery solved?

This article profiles former physicist Gregg Berman and his investigation of the flash crash of May 6. Berman works for the SEC now after a long career in finance as a quant and risk manager. Earlier I posted a job ad for his group.

I predict when the dust settles Gregg won't be the only ex-physicist appearing in the story ;-)

NYTimes: ... In investigating the crash, Mr. Berman says he finds himself in a position similar to his physics work 20 years ago, when he was collecting huge amounts of data and comparing the competing views of many laboratories on a question dividing particle physics — whether the neutrino, one of the least known and most common elementary particles, actually had mass.

Today he finds himself in familiar territory, sifting through huge amounts of messy and disjointed data, and at the same time reading blogs and e-mails from a wide range of observers, each with a theory about what happened on May 6.

Despite his formal training as a physicist, Mr. Berman is no stranger to stock markets. After academia, he spent 16 years on Wall Street, first devising algorithmic trading strategies for hedge funds, then working for RiskMetrics Group, where he created software and dispensed risk-management advice to asset managers, banks and hedge funds.

Having worked with hedge funds and high-frequency traders, Mr. Berman came to his current job a year ago with practical market knowledge and a familiarity with the world of stock trading. Several prominent market players say they found Mr. Berman’s rare combination of experiences refreshing — and reassuring.

... “Many market participants told us, ‘We’re not quite sure what happened over all, but this is what my firm saw and the actions we took,’ ” Mr. Berman said. “It was like ‘C.S.I.’ We wanted to interview everyone around.”

Mr. Berman said the level of detail gleaned from his investigation will help provide the explanation for what occurred on May 6, even if it may not delivery the simple answer that many people would like.

“This level of fact proved to be very, very telling,” he said. “We started to build up a complete picture.”

Physics Envy

A reader referred me to this excellent paper by Andy Lo and Mark Mueller. Mark and I were both Harvard postdocs at the same time. I seem to remember long conversations about both physics and finance in the Dunster dining room :-)

Warning: Physics Envy May be Hazardous to Your Wealth!

The quantitative aspirations of economists and financial analysts have for many years been based on the belief that it should be possible to build models of economic systems - and financial markets in particular - that are as predictive as those in physics. While this perspective has led to a number of important breakthroughs in economics, physics envy has also created a false sense of mathematical precision in some cases. We speculate on the origins of physics envy, and then describe an alternate perspective of economic behavior based on a new taxonomy of uncertainty. We illustrate the relevance of this taxonomy with two concrete examples: the classical harmonic oscillator with some new twists that make physics look more like economics, and a quantitative equity market-neutral strategy. We conclude by offering a new interpretation of tail events, proposing an uncertainty checklist with which our taxonomy can be implemented, and considering the role that quants played in the current financial crisis.

While physics envy might be a problem for economists or theoretical biologists, making physics your career (as opposed to becoming a quant, as Mark did) is certainly hazardous to your net worth!

Quants!

This book seems to be mainly about the alpha-seeking variety of quant, as opposed to the risk managing or derivatives pricing kind. A little gee-whizzy for me, but might be good for some insight into the activities of groups like PDT, AQR, etc.

WSJ: ... At Morgan Stanley's investing powerhouse Process Driven Trading on Monday, Aug. 6, founder Peter Muller was AWOL, visiting a friend near Boston. Mike Reed and Amy Wong manned the helm, PDT veterans from the days when the group was nothing more than a thought experiment, its traders a small band of young math whizzes tinkering with computers like brainy teenagers in a cluttered garage.

On Wall Street, they were all known as "quants," traders and financial engineers who used brain-twisting math and superpowered computers to pluck billions in fleeting dollars out of the market. Instead of looking at individual companies and their performance, management and competitors, they use math formulas to make bets on which stocks were going up or down. By the early 2000s, such tech-savvy investors had come to dominate Wall Street, helped by theoretical breakthroughs in the application of mathematics to financial markets, advances that had earned their discoverers several shelves of Nobel Prizes. ...

Money geeks

The Times has a long piece on quants by Dennis Overbye. The usual suspects are interviewed, but Overbye doesn't really engage with the details of the financial crisis. No mention of CDOs, CDS or securitization; no mention of where quant careers are headed in the wake of the disaster. I agree with Andy Lo's sentiments at the end of the article, but it remains to be seen what portion of blame consensus will apportion to the geeks, and what degree of "math-risk-premia" will be assigned to complicated trading activities in the future.

They Tried to Outsmart Wall Street

...They are known as “quants” because they do quantitative finance. Seduced by a vision of mathematical elegance underlying some of the messiest of human activities, they apply skills they once hoped to use to untangle string theory or the nervous system to making money.

This flood seems to be continuing, unabated by the ongoing economic collapse in this country and abroad. Last fall students filled a giant classroom at M.I.T. to overflowing for an evening workshop called “So You Want to Be a Quant.” Some quants analyze the stock market. Others churn out the computer models that analyze otherwise unmeasurable risks and profits of arcane deals, or run their own hedge funds and sift through vast universes of data for the slight disparities that can give them an edge.

Still others have opened an academic front, using complexity theory or artificial intelligence to better understand the behavior of humans in markets. In December the physics Web site arXiv.org, where physicists post their papers, added a section for papers on finance. Submissions on subjects like “the superstatistics of labor productivity” and “stochastic volatility models” have been streaming in.

Quants occupy a revealing niche in modern capitalism. They make a lot of money but not as much as the traders who tease them and treat them like geeks. Until recently they rarely made partner at places like Goldman Sachs. In some quarters they get blamed for the current breakdown — “All I can say is, beware of geeks bearing formulas,” Warren Buffett said on “The Charlie Rose Show” last fall. Even the quants tend to agree that what they do is not quite science.

As Dr. Derman put it in his book “My Life as a Quant: Reflections on Physics and Finance,” “In physics there may one day be a Theory of Everything; in finance and the social sciences, you’re lucky if there is a useable theory of anything.”

...Physicists began to follow the jobs from academia to Wall Street in the late 1970s, when the post-Sputnik boom in science spending had tapered off and the college teaching ranks had been filled with graduates from the 1960s. The result, as Dr. Derman said, was a pipeline with no jobs at the end. Things got even worse after the cold war ended and Congress canceled the Superconducting Supercollider, which would have been the world’s biggest particle accelerator, in 1993.

...Dr. Derman said, “Nobody ever took these models as playing chess with God.”

Do some people take the models too seriously? “Not the smart people,” he said.

Quants say that they should not be blamed for the actions of traders. They say they have been in the forefront of pointing out the models’ shortcomings.

“I regard quants to be the good guys,” said Eric R. Weinstein, a mathematical physicist who helps run the Natron Group, a hedge fund in Manhattan. “We did try to warn people,” he said. “This is a crisis caused by business decisions. This isn’t the result of pointy-headed guys from fancy schools who didn’t understand volatility or correlation.”

...The recent debacle has only increased the hunger for scientists on Wall Street, according to Andrew Lo, an M.I.T. professor of financial engineering who organized the workshop there, with a panel of veteran quants.

The problem is not that there are too many physicists on Wall Street, he said, but that there are not enough. A graduate, he told the young recruits, can make $75,000 to $250,000 a year as a quant but can also be fired if things go sour. He said an investment banker had told him that Wall Street was not looking for Ph.D.’s, but what he called “P.S.D.s — poor, smart and a deep desire to get rich.”

He ended his presentation with a joke that has been told around M.I.T. for a long time, but seemed newly relevant; “What do you call a nerd in 10 years? Boss.”