Scifoo 2016

Photos from Palo Alto and Scifoo 2016. We weren't allowed to take photos inside the Googleplex.

Foo Camp 2016

I was at Foo Camp the last few days. This year they kept the size a bit lower (last year was kind of a zoo) and I thought the vibe was a lot more relaxed and fun. Many thanks to the O'Reilly folks for running this wonderful meeting and for inviting me. My first time was 9 years ago!

I ran a session TRUMP 2016? CAN IT HAPPEN HERE? (a few people in the session caught the Sinclair Lewis reference) to get a feel for whether the tech community understands what's happening in our country. At another meeting earlier in the year I concluded

Everything at this meeting is off the record, so I can't say much about it. The one comment I'll make is that among this group of elites almost no one I've spoken to groks Trump or his appeal to a large number of Americans.

The other session I co-ran (with Othman Laraki of Color Genomics) was on genomics.

Lots of good stuff at hashtag #foocamp.

Here's a list of book recommendations from one of the sessions (James Cham).

First day of FOO Camp 2016. Whisky tasting in the Design Village soon. #foocamp pic.twitter.com/PyuNY1mG7t

— Andrew Crow (@AndrewCrow) June 11, 2016

The Future of Machine Intelligence

See you at Foo Camp in June! Get a free copy of this book at the link.

The Future of Machine Intelligence 

Perspectives from Leading Practitioners
By David Beyer

Publisher: O'Reilly
Released: March 2016

Advances in both theory and practice are throwing the promise of machine learning into sharp relief. The field has the potential to transform a range of industries, from self-driving cars to intelligent business applications. Yet machine learning is so complex and wide-ranging that even its definition can change from one person to the next.

The series of interviews in this exclusive report unpack concepts and innovations that represent the frontiers of ever-smarter machines. You’ll get a rare glimpse into this exciting field through the eyes of some of its leading minds.

In these interviews, these ten practitioners and theoreticians cover the following topics:

Anima Anandkumar: high-dimensional problems and non-convex optimization
Yoshua Bengio: Natural Language Processing and deep learning
Brendan Frey: deep learning meets genomic medicine
Risto Miikkulainen: the startling creativity of evolutionary algorithms
Ben Recht: a synthesis of machine learning and control theory
Daniela Rus: the autonomous car as a driving partner
Gurjeet Singh: using topology to uncover the shape of your data
Ilya Sutskever: the promise of unsupervised learning and attention models
Oriol Vinyals: sequence-to-sequence machine learning
Reza Zadeh: the evolution of machine learning and the role of Spark

About the editor: David Beyer is an investor with Amplify Partners, an early-stage VC focused on the next generation of infrastructure IT, data, and information security companies. Part of the founding team at Patients Know Best, one of the world’s leading cloud-based Personal Health Record (PHR) companies, he was also the co-founder and CEO of Chartio.com, a pioneering provider of cloud-based data visualization and analytics.

The story of the Monte Carlo Algorithm

George Dyson is Freeman's son. I believe this talk was given at SciFoo or Foo Camp.

More Ulam (neither he nor von Neumann were really logicians, at least not primarily).

Wikipedia on Monte Carlo Methods. I first learned these in Caltech's Physics 129: Mathematical Methods, which used the textbook by Mathews and Walker. This book was based on lectures taught by Feynman, emphasizing practical techniques developed at Los Alamos during the war. The students in the class were about half undergraduates and half graduate students. For example, Martin Savage was a first year graduate student that year. Martin is now a heavy user of Monte Carlo in lattice gauge theory :-)

Crossing the Pacific

So long, Hong Kong...

Foo Camp!

Someone is mining ether!

Sci Foo 2015

I'm in Palo Alto for this annual meeting of scientists and entrepreneurs at Google. If you read this blog, come over and say hello!

Action photos! Note most of the sessions were in smaller conference rooms, but we weren't allowed to take photographs there.

Creation, Myths and Twitter

Great article by Nick Bilton on the creation myth (and true story) behind Twitter. To see that luck plays an unimaginably huge role in life you just need to look carefully at the story behind any successful company or entrepreneur.

NYTimes: ... Soon, the question of a name came up. Williams jokingly suggested calling the project “Friendstalker,” which was ruled out as too creepy. Glass became obsessive, flipping through a physical dictionary, almost word by word, looking for the right name. One late afternoon, alone in his apartment, he reached over to his cellphone and turned it to silent, which caused it to vibrate. He quickly considered the name “Vibrate,” which he nixed, but it led him to the word “twitch.” He dismissed that too, but he continued through the “Tw” section of the dictionary: twist, twit, twitch, twitcher, twitchy . . . and then, there it was. He read the definition aloud. “The light chirping sound made by certain birds.” This is it, he thought. “Agitation or excitement; flutter.” Twitter.

... Whatever his reasons, Dorsey had recently met with Williams and threatened to quit if Glass wasn’t let go. And for Williams, the decision was easy. Dorsey had become the lead engineer on Twitter, and Glass’s personal problems were affecting his judgment. (For a while, portions of the company existed entirely on Glass’s I.B.M. laptop.) After conferring with the Odeo board, around 6 p.m. on Wednesday, July 26, 2006, Williams asked Glass to join him for a walk to South Park. Sitting on a green bench, Williams gave his old friend an ultimatum: six months’ severance and six months’ vesting of his Odeo stock, or he would be publicly fired. Williams said the decision was his alone.

... Williams and Dorsey started meeting for weekly dinners to discuss the problems, but one night Dorsey became defensive. “Do you want to be C.E.O.?” he said abruptly. Williams tried to evade the question, but eventually replied: “Yes, I want to be C.E.O. I have experience running a company, and that’s what Twitter needs right now.” ... told him that they were replacing him as C.E.O. with Williams. Dorsey sat before a bowl of uneaten yogurt and granola as he was offered stock, a $200,000 severance and a face-saving role as the company’s “silent” chairman. No one in the industry had to know that he was fired. (Investors would not want to be seen as pitting one founder against another anyway.) But Dorsey had no voting rights at the company. He was, essentially, out.

... Access to the tech blogosphere and press can help percolate a fledgling start-up into a multibillion-dollar business. But this access often relies on having a narrative — being an entrepreneur with just the right creation story. ... After he was stripped of his power at Twitter, Dorsey went on a media campaign to promote the idea that he and Williams had switched roles. He also began telling a more elaborate story about the founding of Twitter. In dozens of interviews, Dorsey completely erased Glass from any involvement in the genesis of the company. He changed his biography on Twitter to “inventor”; before long, he started to exclude Williams and Stone too.

... Without Williams and Stone influencing its development with the lessons they learned from Blogger, it still would not have taken off. Making it a company required Williams’s money, then Wilson, Sabet and Fenton’s and dozens of other investors, not to mention Costolo, who turned it into viable business, and 2,000 employees who helped shape it into one of the biggest social networks on the planet. Such is the case with every company in Silicon Valley, though you never hear it in their creation myth. Dorsey will make $400 million to $500 million when Twitter goes public. Glass stands to make about as much as Dorsey’s secretary at Square. ...

This is from a 2009 post Me and Twitter. I had met Williams at Foo Camp, probably in 2007. In the 2009 post I didn't mention that most of the conversation was about Odeo, Williams' podcasting startup from which Twitter sprang as an almost accidental creation. His description to me of how the Twitter idea originated was a bit different than what Bilton reports.

I met Twitter founder Evan Williams a few years ago, before Twitter was anywhere near a big thing. He told me about Blogger, which he sold to Google, and then the inevitable "So what are you working on now?" question came up.

He described Twitter to me, and two thoughts entered my mind. The first shows I am old, or out of touch, or have no feel for Web 2.0 consumer startups: "Who would use that?" I said to myself.

The second thought, which I actually verbalized, turns out to be a good question (still unanswered) and shows I may have VC potential: "How are you going to monetize that?" :-)

Andrew Ng and Coursera

Great profile of Coursera co-founder Andrew Ng. I first met Andrew a few years ago at Foo Camp. He attended a talk I gave on psychometrics, and we had several discussions afterwards. The formulation of g as a means of compressing cognitive data was not lost on him.

Little did I know at the time he was interested in technology applied to teaching.

Here are his popular lectures on machine learning. See also Whither higher education?

Chronicle of Higher Education: ... Today Mr. Ng is an innovator in an entirely different setting: online education. He is a founder of the start-up Coursera, which works with 33 colleges to help them deliver free online courses. After less than a year of operation, the company already claims more students—1.3 million—than just about any educational institution on the planet. Mr. Ng likes to say that Coursera arrived at an "inflection point" for the idea of massive open online courses, or MOOC's, which are designed so a single professor can teach tens of thousands of students at a time.

... "There's a certain way of thinking that many AI researchers have—it's the idea of automation," Mr. Ng explains, his lanky frame folded onto a couch in a conference room. He speaks in a quiet voice colored by a British accent—he was born in Britain and grew up in Hong Kong and Singapore—and his understated manner makes you forget that his teaching videos have been viewed hundreds of thousands of times. He is sometimes recognized as a kind of celebrity on the streets near Stanford.

"A lot of AI successes have been about automating the routine things that do not call on the highest levels of human creativity," he says, noting that spam filtering and recognizing faces in photographs can now be done deftly by software.

After teaching at Stanford for several years (he's now on leave), Mr. Ng felt that grading was eating up too much of teaching time. Computers, he thought, could step in and grade complex assignments, not just multiple-choice exams.

"I actually enjoy working through problems with students," Mr. Ng says. "What I don't enjoy is grading 400 homeworks. And so our thinking was to automate some of the grading so it frees up more faculty time for the interactions."

He put his ideas into practice about five years ago, when he started Stanford Engineering Everywhere, which offered MOOC's before anyone had heard of them.

Foo: exuberant geeks

Making the future.



CTOs and CEOs and Founders and Inventors and Creators.



Never before have the economic and creative prospects been so good for a young person with quantitative or technical abilities. In my generation we had fewer options: defense, academia or big stodgy corporations. Today you can code or analyze data or build mathematical models for a hedge fund, a bank, a social networking startup, a web publisher, an e-commerce site, a video game company, Google, etc., etc. If you can manage a team and communicate your vision to investors and partners, all the better. The sky is the limit.




"The Times gets a billion impressions a day. How do they optimize their ad revenue? To get real-time or nearly real-time analysis of, say, how many people in Wisconsin who were on Zappos within the last hour also viewed a Style section article, we need a big hadoop deployment and in-memory database that let's the user slice through a 100 dimensional parameter space. It's business intelligence on a supercomputer. Remnant ad space is auctioned on AdSense to competing bots, with the whole thing -- cookie analysis, automated bidding, ad fetching and placement -- taking place over 500 milliseconds."

"We're empowering people in rural India by giving them access to piece-rate work over the internet."

"90% of mechanical turk work is web spam." :-)

"They tried to buy us with shares priced on a secondary market, but I wouldn't trade my execution risk against that valuation risk."

"Yes, it's a flashlight but it's LED with a lithium ion battery. 3000 lumens shined into a burglar's eyes will blind them for 20 minutes. Those fins are for heat dissipation; the tip reaches 180 degrees."

"Computer vision is next. Much easier than NLP, but it'll be hacks strung together like everything else."


A few more Foo Camp 2011 photos here. Scott Berkun shares his insights here.

Foo 2011 photos

Opening dinner.




The scheduling scrum.



Werewolf?



Make zone.





A schedule board.



Face board.



A session on Bitcoin. There seems to have been a bit of a crash today -- the dollar exchange rate went down to $8 from $28. Was there more rhetoric from Senator Schumer?



Outside camping.



Inside camping.



Geeks and booze.



Favorite quote of the day: "Startup CEOs are warm sociopaths."

Foo Camp 2011

I'm off today to Sebastapol, CA for another Foo Camp :-)



Here is what I wrote about it last year.

My overview of psychometrics

I used these slides in two talks given at Foo Camp 2010 and Sci Foo 2010. At these "self-organized" meetings attendees are encouraged to talk about whatever they find interesting, and I usually choose to talk about wacky stuff rather than my main research, which tends to be a bit too specialized for the audience. In previous years I've talked about ultimate fighting, internet security, startups, etc.

At Foo, which has a Silicon Valley flavor, I had several CEOs and a bunch of technologists in the audience, and didn't receive any objections to the material. One CEO (an IIT grad with a PhD in engineering from Princeton) who runs a software company employing 1000 developers in India, was very interested in my results and has since agreed to run some experiments (stay tuned!) related to personnel selection and the relation between g and coding ability. At Sci Foo, which has a more scientific or academic flavor, the audience consisted of science writers, Google engineers, physical and computer scientists, a neuroscientist, and (I think) a social scientist. Only the last two voiced objections -- the social scientist actually got up and walked out after 15 minutes. Others in the audience found these objections rather amusing -- How could they argue? All you did was show data; the conclusions are obvious!

See additional comments (elaboration from the talks) here.

More SciFoo 2010 notes

Some quick notes. I don't know if I'll have the energy to put in a full set of links, but you can pursue any of these topics with your own google searches. I'm sure I missed a lot of good sessions -- too much good stuff going on :-)

Great talk on the Pirahã and Chomsky's universal grammar by Dan Everett. (The Pirahã have no words for distinct numbers and no recursion!)

Long discussion with Erik Verlinde about his idea that gravity is an emergent entropic force.

Dinosaurs and reptiles cannot gallop, with the exception of one weird crocodile -- Paul Sereno.

Ed Felton can uniquely identify individual physical objects (e.g., a particular sheet of paper) using an optical scanner, and, using cryptography, produce a digital signature (i.e., printed directly on the object) that proves that the particular object isn't counterfeit. This has tons of interesting applications.

Frank Wilczek got me excited about graphene. Max Tegmark got me intrigued about 21 cm radio waves and a funny FFT telescope.

Tsutomu Shimomura's startup may drastically reduce the cost of LED lighting.

Sendhil Mullainathan's research explores the psychological consequences of scarcity.


A few more photos from the closing session. Sorry my photos are so boring but we got a warning from one of the Google organizers right at the beginning not to take photos outside of certain restricted areas. Earlier post here.

SciFoo 2010 notes

There seem to be a lot of physicists here this year. A partial list of theorists: Adi Stern, Chetan Nayak, Frank Wilczek, David Gross, David Tong, Eva Silverstein, Lee Smolin, Erik Verlinde, Alan Guth, Max Tegmark, Paul Davies, Giovanni Amelino-Camelia. Do I count Ed Lu? He was an astronaut for a long time. Guth, who is a very level-headed guy, told me he's now 99 percent confident that inflation is correct, given the CMB results from the last decade. I think I convinced Chetan and maybe Adi that they are actually many worlders ("... if you do a decoherence calculation, and at the end don't insist on throwing away all the parts of the wavefunction except one of the decoherent parts, then you're a many worlder" ;-) Max claims to have a way to get the Born rule from many worlds, but I don't believe him :-) Guth is a many worlder.

I could easily spend all my time at the physics talks, but I think it's better use of this kind of meeting to attend talks outside my specialty. At dinner I met a guy who does fMRI on psychopaths and the guy who built a wind-powered car that goes faster than the wind.




Larry Page addressing the campers.




The campers introducing themselves.




Goofing around with a super croc fossil.




Two Caltechers of my vintage: Tsutomu Shimomura and Ed Felton. We talked about the huge cognitive surplus in physics -- both of these guys were trained in physics before going on to other things.

SciFoo 2010

See you there on Friday! This video is from last year.

What's special about Foo?

On my way home I thought a bit about what is so special about Tim O'Reilly's Foo Camp. If I recall correctly, I've now attended 3 times: 2007, 2008 and 2010. Probably what I am about to say is not new, although it does come from my atypical physicist / entrepreneur / amateur social scientist perspective.

There are famous and influential people at Foo, but I imagine other gatherings (that I don't get invited to, such as Davos :-) have even more.

There are smart people at Foo, but average IQs at meetings in certain subfields are even higher.

There are really creative and energetic people at Foo, and on these criteria I doubt it can be surpassed.

But, in my mind, the two things that make the meeting truly unique are:

1. The diversity of talents and viewpoints, with participants ranging from hackers to social activists to scientists to grizzled CEOs and investors. The people at Foo are deliberately trying to create the future and are engaged with all that that entails: technology, ideas, organizations, capital.

2. The unique *social* environment. Tim has managed to create a "social reality distortion field" which enables interactions that are passionate but simultaneously friendly and open. People are genuinely happy to be at Foo Camp, and they are generous, intellectually and otherwise, with others. If I hear an argument at Foo that I don't agree with, I will try to give the speaker the benefit of the doubt, and factor in the unique knowledge and experience they bring to the issue. My questions will be friendly and non-aggressive. This is quite different from what happens at specialized meetings (e.g., among physicists) or in academia in general.

What can other meetings learn from Foo? If organizers are brave enough and set the social tone from the beginning, they can positively affect the quality of the event. Also, since interesting people are often multi-faceted, it might be worth setting aside some time in the evening for short demos or discussions on topics outside the main focus of the meeting. Sometimes at physics workshops I am amazed at how boring the dinner conversations are, given the special brains at the table. Some of the best talks I've attended at Foo are on "life topics'' such as dealing with success and failure, Paleo fitness, work/life balance, living green, etc.

Here is a great post by Scott Berkun, with similar thoughts about what makes Foo special. It seems that Scott and I went to almost entirely different sessions, which is probably why I barely had a chance to say hello to him this year.