Random matrix and phase shifts

I just stumbled across this great article on Tracy-Widom distribution. It talks about random matrices and phase shifts.  This reminded me of some work I did on resource allocation in network interconnection. We derived routing matrix conditions for "peering" and "dis-peering" (the latter a new term) to be equilibria in the decentralized resource allocation game. I wonder what a probablisitic approach with the routing matrix randomized would add to the game theoretic results. Large scale self organizing interconnections (or failure thereof).

Don't buy the second item on the menu

Yesterday a door to door salesman from Time Warner Cable came to our door. He was a nice guy so I listened to him even though I've been avoiding cable for a long time. Anyway here are the current options for Internet access from Time Warner Cable:

"If Internet is all you need, however, TWC offers its “Everyday Low Price” plan for just $15 per month. This includes 2 Mbps download speeds, 5 emails accounts and 100 MB of email storage. Need to go faster? Try Basic (3 Mbps and $30 per month) or Standard ($35 per month with 15 Mbps). If you’re an online gamer or download large files on a regular basis, the 20Mpbs of TWC’s Turbo plan ($45 per month) may be the best option, while home business users may want to try out the Extreme plan, which offers 30 Mbps download speeds at $55 per month. Finally, if you have a large family or Internet users or connect multiple devices on a daily basis, you may need the Ultimate plan, which provides download speeds of 50 Mbps, 30 email accounts and 10 gigabytes (GB) of email storage for $65 each month."

OK now consider the slope or marginal prices:

• The first 2 Mbps costs $7.50 per. Fair enough. 
• But then the next 1Mbps costs $15!
• The next 12Mbps cost  just $0.42 per!!!
• Then the next 5Mbps cost $2, 
• And the following 10Mpbs are $1,
• And then 20Mbps more at $0.50 each.

The first and the last three are totally reasonable. But notice the second item on the list is a horrifically bad deal.  Why does it exist, who in their right mind would pick that? It's like  a trap. Maybe some people will just ignore the Mbps amidst all the verbiage about storage and email etc (extras which really are insignificant in terms of cost) and think to themselves: "Hey, I'm not poor and "Everyday Low Price" that sounds like the plan for poor people.  And I don't understand the high end stuff, so let me get Basic, that sounds reasonable." And boom, they are paying an astronomical price. Even if not many people fall for it, it's very profitable.  It's also kind of unethical in my opinion. (This is not the first time I'm finding fault with Time Warner in these pages, and I don't even use them. )

A more innocent version of the same thing is wine lists at restaurants. Never order the second item! It's for suckers. To see why, imagine a naive and status-conscious customer who doesn't know much about wine: he will skip the first one to avoid looking cheap, but will hesitate to go to far down the list because they can't justify buying the expensive ones. So he will settle for the second one. The restaurateur willing to exploit this can profit by putting the cheapest wine at the second cheapest price.  Thus gouging the suckers without affecting others. The moral of the story is, even if you don't know about wine, you can still have a wine list strategy.

Net neutrality

"Net neutrality" a hot topic again these days. Plus ça change, plus ça reste pareil. Given the amount of confusion out there it seems like it won't be the last time.

When people say "neutrality" they could mean any combination of:
1) dominant access or backbone providers should not discriminate between customers, they should offer a similar prices to any buyer
2) all / most / many networks must exchange traffic free of charge with each other
3) all traffic must be treated the same regardless of application
4) all end users must pay a flat price for unlimited usage

My view, as regular readers... < crickets > ... can guess, is that 1) is the only good version.  2) I've written quite a bit about before, and I still think it's wrong,  but thankfully 2) is rapidly joining  3) which has been obsolete for years.  4) is fine when feasible but demanding it be a requirement of all forms of access is just silly.

But 1) is really important! I hope that somehow emerges as the dominant focus this time but I'm not holding my breath.

For example, in the US right now there's a real danger with Comcast: local access monopoly x continental scale + vertical integration with content. Huge issue. This is all about 1), but the general public thinks the issue is 4), which means "net neutrality" will be defeated as irrational whining.

Optimism

A few months ago, I stumbled across a business card left behind on a table. Under the name of the company, it had three words, each followed by a period,  representing, I guess, the three pillars of their "corporate values".  One word struck me: Optimism.   I must have snickered,  because someone asked what was up. I instinctively thought "optimism" was a silly value, but it dawned on me that I had never thought about it explicitly.

Is optimism good?

The question sounds strange because Optimism, today, in American culture, is automatically assumed to be A Good Thing. Like "pro-active". People use that word as if it's synonymous with "good".  E.g. Person A: "Don't do this bad thing." Person B: "It's not bad, it's pro-active!"  Noooo....  Just like sometimes, being pro-active is evil, being optimistic is not automatically good.

Let's define optimism as follows: Having high expectations for a positive outcome. That is to say, compared to most "normal" people's probability distribution of outcomes, yours has more weight on the positive side. Say we both bet on the same horse, and one of us thinks we'll lose and one thinks we'll win.  So when is it good to be the optimist?  I would slice it on three levels:

1. Of course if you turn out to be right, then great... But that just means you got lucky.
2. What if you had to make the same choice over and over again, and on average the optimistic view is more accurate? Great, but that's not really optimism, it's having a better probabilistic model, better foresight.
3. Now what if you believe the same probabilities as everyone else, but you are more willing to take the risky choices and eventually you're better off? You are good at taking the right amount of risk for reward, and if in the long run you are better off (technically i.e. if you are on the efficiency frontier in the risk, reward plane),  then ... well that's good judgement. 

But in all these forms, the optimism is situational! The are plenty of situations where the wise person would take the "pessimistic" position.   Thus, as a fundamental value to live by, "optimism" is actually orthogonal to the things we consider good, truthful etc.

People (including Corporations!) of the world, listen to me: Value luck, foresight or judgement.  Not optimism.  That's just silly.

 

Time lapse #2

This one had a couple of adjustments.

1. To get maximum depth of focus: set the aperture to be as small as possible (f/22) and let the shutter speed be automatically determined. By the last few frames, the exposure time is up to 3-4 secs. 
2. To compensate for the lack of trains: added a hurricane!

Time lapse

Here's a little experiment I did this weekend. My first ever time-lapse video:

It consists of:

• one photo every 5 minutes, 
• 101 images which become
• a movie of 10 frames per second, 
• for a speedup of 3000 x real life.  

Setup: 

1. Plug camera into laptop via USB. 
2. To remotely control camera: a program called EOS Utility which comes included with most Canon DSLRs. I'd never even looked at those discs that came with the camera, which is more than 3 years old.  Good thing I never threw them away! 
3. To make the HD movie out of the JPEG images: Movie Maker tool in Picasa.

I'd assumed it was all going to be much more complicated but as soon as I started looking into it, it became obvious that everything I needed was already right here at my fingertips.  This reminds me of a previous little weekend project -- 100% wireless...   There ought to be a word for this, when things turn out delightfully easier than expected. The opposite of a bug. Let's call it a butterfly.

Defending America?

Here are three facts every American should know:

1. Americans pay $1 Trillion a year for War and ‘Security’.   In fact, that understates it a bit, for 2012 the US is spending over a trillion dollars on national security. 
2. The US spends more than the next 10 biggest military spending countries... combined! Taking #2 China and #3 Russia together, the US still spends 3 times more. 
3. Military spending doubled in the last decade. The US spends more today than it ever did during the "arms race" of the cold war against the Soviet Union, and that's adjusted for inflation.  More than at the peak of the cold war!

Now regardless of your opinion, if you live in the US, simply test a few friends on any one of these facts. I bet most of them are not even close. In the mainstream debate on military spending, reducing it is considered the crazy thing!  The above facts rarely get mentioned, neither by the politicians nor the reporters. 

Defending America? From what or whom? Military-industrial complex anyone? If you're feeling courageous, here's a fascinating piece entitled Stormtroopin' USA. 

Five books: Non-fiction

A few months ago, the New York Times had an article on the greatest non-fiction books. They asked 33 people to list their top 5.  (To my dismay, Kapuscinski's "The Emperor", one of the worst books of all time in my opinion, came out tied for the best of the best. That book is so bad that, when I tried reading it, I couldn't help but physically throw it away in disgust. My hatred for that book deserves a whole blog post of it's own. But this is not that post.)  Inspired by that article, I thought I'd try to remember my some of my favorite non-fiction books of all time. It wouldn't be fair to call it a "top 5", since I am not being that thorough.  If I sat down to do this ten times, I'd probably end up with ten slightly different lists.

Rather here are 5 non-fiction books I really enjoyed, that had a deep influence on my world view, and that just happen to bubble to the surface at the moment:

• Gödel, Escher, Bach: An Eternal Golden Braid, by Douglas Hofstadter. Absolutely brilliant exploration of logic.  Recursion is just so... fundamental. The human mind must be an instrument of God. Made me love J.S. Bach (and "Alice in Wonderland").  And a proof of Gödel's incompleteness theorem. 
• Guns, Germs and Steel: The Fates of Human Societies, by Jared Diamond. Very ambitious, basically the history of civilization organized around a few very compelling ideas. Geography is very very important. Innovation diffuses latitudinally.
• Seven Pillars of Wisdom, by T. E. Lawrence. Supremely elegant writing. Story of the middle east during the first world war. Essential reading if you want to understand a lot of international politics today.
• Life is long if you know how to use it, by Seneca. Beautifully concise stoic philosophy. Don't sweat status. 
• An Autobiography, or The Story of My Experiments with Truth, by M. K. Gandhi. The great man's early life and journey to satyagraha, truth through non-violence.

Hmm, clearly 5 is not enough. Here are a few more that would be included in different iterations of the "top 5".

• The Discoverers, by Daniel Boorstin. A masterful, sweeping history of science. Extremely educational.
• An anthropologist on Mars, by Oliver Sacks. The human brain is really really interesting.  It's easy to not realize how amazing our vision, memory, etc. are ....  until something goes wrong.
• The Autobiography of Malcolm X as told to Alex Haley, by Malcolm X and Alex Haley. Powerful life story. I learned a lot about the USA in this one.

Oh what the hell, let's make it an even 10:

• Zen and the Art of Motorcycle Maintenance, by Robert Pirsig.  Not the best writer of the bunch, but this book had a profound influence on me. Key word: Quality. What is good? What is not good? Also, I'm not sure if I learned it in this book, but I believe that you should never get upset by something that was predictable.
• Titan: The Life of John D. Rockefeller, by Ron Chernow. Very thorough biography of a fascinating man in a fascinating time. Very relevant today. If you are thinking of pursuing an MBA, don't. Instead, read books like this one.

Political bugs: six fixes to the US budget deficit

A couple of months ago, in the midst of the hype around the US government debt crisis, someone on a mailing list posed the following question: what are some of the "real" things that can be done to solve the problem?

The premise of the question is that the politicians were not proposing rational solutions but rather, were speaking tactically, with ulterior motives, in other words being craven, pandering, demagoguing, etc. Indeed, the US deficit is one of those things where there are a few clear "right things" that the majority of rational informed people would agree with, and yet, wrong things keep happening over and over again. Much like the "design bugs" I like to occasionally rant about, these are political bugs. I don't mean the fundamentally difficult issues where reasonable people can sincerely disagree: a carbon tax, medicare for all, a flat-tax, consumption tax, etc. etc. Those aren't bugs, rather they are unsolved problems, or new "features" to be designed. But for this post, I'm only taking about no-brainers, the bugs.

So coming to the US budget deficit specifically, here was my list of bugs (originally written on July 1, 2011):

1. Eliminate mortgage interest deductions. Why should renters subsidize owners? No good reason.
2. Eliminate corporate healthcare deduction i.e. count employer provided benefits as income. Why should employees of larger companies that provide health benefits be subsidized by people who get their health insurance in other ways? Why would society want to encourage linking health insurance to employment, as opposed to say children's education? No good reason. 
3. Abolish payroll taxes and shift that all into income tax. Payroll taxes ostensibly pay for social security and medicare but really are just a regressive extra income tax, why not roll it into the overall progressive income tax? 
4. Reduce social security by making it means tested. Why should anyone above the median income get any additional income from the government? If the purpose of a social safety net is to have the fortunate help the less fortunate, what's the point of helping the rich? 
5. Reduce social security by raising the threshold age. The retirement ages were set a long time ago, when people aged earlier. Sixty five years old in 1930 was like being 75 or 80 today. If the population distribution shifts older, retirement age should rise. The ratio of retired people to working people has to be kept more or less constant otherwise a) it obviously won't work, and b) it will be clearly unfair. 
6. Reduce military spending to a level where it can be called a "defense budget" as opposed to an "offense budget": 50% would still leave it twice as big as the next biggest military budget, and still greater than the next 5 countries combined. 

The combination of the above would easily eliminate the structural deficit. Not only that, none of these points fundamentally challenging any principles or stated societal goals, nor are they suggesting qualitative change These are straight quantitative adjustments that, in principle, libertarians, leftists, fiscal conservatives, and most mainstream economists would probably agree with. They should be no-brainers.

And yet, they can't happen. The reality is that regulatory capture makes these things almost impossible. So the debate goes on and on about tiny meaningless slices of spending that are easy to demonize, but have no impact on the real problem. Maybe something will bump the political economic system out of this miserable Pareto optimum. Unfortunately it looks like for now, the main lesson of the last three years of economic crises remains that the first casualty is causality.

Cognomics: what is the market price of your mind?

Last year, I came across a post about a very interesting study of CAPTCHA economics (in fact, this post is a slightly expanded re-post of an email I sent to a mailing list in Agust 2010).  CAPTCHAs are the little boxes you see on login pages across the web. They are meant to prevent software bots, run by spammers, crackers and other Internet bad guys, from pretending to be real users and abusing web services.

The key feature of a CAPTCHA is that it's a puzzle that is very easy for humans to solve but very hard for computers. So to defeat the CAPTCHAs, the bad guys have created online "farms", platforms where people are payed to solve the CAPTCHAs for them.

Leaving aside the nefarious nature of the application, it is fascinating to note that this is one of the first large scale instances of humans renting out their brains for a few seconds at a time. It raises the question: what is the market value of your mind? Or more precisely, of your ability to think? In a sense, of course, billions of people are renting out their brains every day by doing cognitive work in exchange for a salary. But normal employee/employer relationships are complex in ways that are not fully captured by price alone, in other words it's not a commodity. But with the CAPTCHA farm, you get as close as possible to cognition as a commodity.

Note that I said "one of the first" instances... There's another bigger one that's been around much longer: advertising. When advertisers pay for ad placement, they are paying for your attention, so in a sense they are renting your mind. Particularly with Internet advertising, the advertiser literally buys a few seconds of a person's attention, one person at a time. This was a big meme during the dot-com era, the "attention economy".

So, if indeed there's a market for cognition, how do the prices compare across these two sub-markets?

• Spammers* pay O($1) per thousand CAPTCHAs solved.
• Advertisers pay O($1) per thousand impressions

First observation: In both cases, they are paying to get your mind for O(10seconds). So the mind is really a commodity worth the same whether you are renting it out to store and propagate a commercial message or to do some computation!

Second, if you draw a little diagram of the flow of value, the cognitive supply chain as it were, the two are like mirror images of each other, with the same values circulating in opposite directions. Like electrons and anti-electrons in the same circuit! Here's what we get clockwise for the advertising business:

Advertiser → $1 → Publisher → (payload) → User → (profit from product) → Advertiser

And here's what we get, anti-clockwise for the web-spam business:

Spammer ← (profit from scam) ← Publisher ← (payload) ← User ← ($1) ← Spammer

Funny stuff. Spammers are like the anti-matter to the matter of advertisers, the evil twin from the underworld.   Fortunately, they are not quite symmetric in size. It's a lot harder for the bad guys to grow the same size as legal ads/content ecosystem!

Note: of course, the main difference between the minds rented for ads and the minds rented for solving CAPTCHAs is wealth... Rich people's minds are valuable to advertisers,  poor people's minds are useful to spammers. But despite this split along wealth lines, the prices are of the same order of magnitude.