From the batcomputer to Quora: the quest for the perfect answering machine

1 08 2011

Note: I’m resuscitating this blog one more time, but slowly: copying my posts from Biznology and other places to here and applying minor edits. Naturally, they lost their freshness, but I want to make this WordPress blog an archive of all my posts.

As previously seen in Biznology:

Cartoon Maze Card

Image by andertoons via Flickr

When Quora announced in May that they were eliminating their policy against self-promoting questions and answers, some analysts wondered if that was opening the gates for spammers to dominate the conversation. The reality is that the whole evolution of Q&A services is not much different from what Google and other search engines have been experiencing throughout the years. It’s a battle to separate the wheat from the chaff, where the chaff keeps finding creative ways to look like the wheat. Keep reading, and you’ll find why developing the perfect Q&A engine should not be our real objective here.

As a kid, I spent my fair number of hours watching re-runs of camp TV shows, including the classic Batman TV series from the 60’s. I remember how the batcomputer was able to answer any question you asked it, no matter how weird or convoluted they were. For those of you who never had the privilege (?) to see the precursor of IBM’s Watson, here it is, courtesy of YouTube (it’s a long video, so I’m taking you directly to the 2:20 mark):

Yes, you saw it right. The bat-computer was fed a bunch of alphabet soup letters and gave the dynamic duo the answer they were looking for, where they should go next to complete their mission. However, as a sign of things to come, Batman then tries to go extreme and feeds the bat-computer with the Yellow Pages directory book, but—oh the horror—the batcomputer fails miserably trying to get them a more precise answer for their subsequent question.

More than 40 years later, our quest for the infallible computer has not changed much. Watson could easily answer Jeopardy! questions about song lyrics and book topics, but choked when facing more nuanced themes. That was not very different from the 18th century “Mechanical Turk”, which was capable of winning chess games, solving puzzles, conversing in English, French and German and even answering questions about people’s age and marital status, but had its fair share of defeats.

I concede that services like Wolfram Alpha, ChaCha and Quora raised the bar compared to early players such as Yahoo! Answers and WikiAnswers, but they all come short to address complex, subtle or fringe questions.

If you don’t believe me, just try yourself. Use your favorite online Q&A service to ask a question that you can’t easily find in Wikipedia or via a quick Google search and let me know if you get anything meaningful back.

Quora gave many of us the hope that we would be finally getting a high-quality, well-curated Q&A service. It’s becoming increasingly clear now that, albeit a step forward, Quora is not the know-all oracle that we were looking for.

Are we going to ever find the perfect Q&A service, where more nuanced questions will get satisfactory responses? My guess is “no”, but not even Adam West’s noodle-eating batcomputer would know the answer for that.

In fact, at the end of the day, that answer is not relevant at all. As we make strides in the information technology journey, our fundamental objective is not to replace people with machines. Our real target is to free us all from as many mundane and “automatable” tasks as possible, so that we can focus our efforts and energy more and more on the tasks that only humans can do. Having increasingly smarter systems that can answer most of our trivial questions are not a sign of our defeat to “our new computer overlords.” It’s rather a great opportunity to re-define what being human actually means.

Marketing segmentation and the game of averages

26 07 2011

Note: I’m resuscitating this blog one more time, but slowly: copying my posts from Biznology and other places to here and applying minor edits. Naturally, they lost their freshness, but I want to make this WordPress blog an archive of all my posts.

As previously seen in Biznology:

Back in March, a Hunch Blog post (“You’ve got mail: What your email domain says about you”) made some noise around the net, courtesy of Gizmodo, swissmiss, and hundreds of tweets and retweets, most likely by Gmail users, who are depicted very favorably compared to Yahoo!, Hotmail and poor AOL users. But how much of that is really “utterly fascinating psychographic analysis” – as described by some of the retweeters – and how much is just over-extending the concept of marketing segmentation? Is the average Gmail user significantly different from the average Yahoo! user?

This is how that post summarized its findings:

  • AOL users are most likely to be overweight women ages 35-64 who have a high school diploma and are spiritual, but not religious. They tend to be politically middle of the road, in a relationship of 10+ years, and have children. AOL users live in the suburbs and haven’t traveled outside their own country. Family is their first priority. AOL users mostly read magazines, have a desktop computer, listen to the radio, and watch TV on 1-3 DVRs in their home. At home, they lounge around in sweats. AOL users are optimistic extroverts who prefer sweet snacks and like working on a team.
  • Gmail users are most likely to be thin young men ages 18-34 who are college-educated and not religious. Like other young Hunch users, they tend to be politically liberal, single (and ready to mingle), and childless. Gmail users live in cities and have traveled to five or more countries. They’re career-focused and plugged in — they mostly read blogs, have an iPhone and laptop, and listen to music via MP3s and computers (but they don’t have a DVR). At home, they lounge around in a t-shirt and jeans. Gmail users prefer salty snacks and are introverted and entrepreneurial. They are optimistic or pessimistic, depending on the situation.
  • Hotmail users are most likely to be young women of average build ages 18-34 (and younger) who have a high school diploma and are not religious. They tend to be politically middle of the road, single, and childless. Hotmail users live in the suburbs, perhaps still with their parents, and have traveled to up to five countries. They mostly read magazines and contemporary fiction, have a laptop, and listen to music via MP3s and computers (but they don’t have a DVR). At home, Hotmail users lounge around in a t-shirt and jeans. They’re introverts who prefer sweet snacks and like working on a team. They consider themselves more pessimistic, but sometimes it depends on the situation.
  • Yahoo! users are most likely to be overweight women ages 18-49 who have a high school diploma and are spiritual, but not religious. They tend to be politically middle of the road, in a relationship of 1-5 years, and have children. Yahoo! users live in the suburbs or in rural areas and haven’t traveled outside their own country. Family is their first priority. They mostly read magazines, are almost equally likely to have a laptop or desktop computer, listen to the radio and cds, and watch TV on 1-2 DVRs in their home. At home, Yahoo! users lounge around in pajamas. They’re extroverts who prefer sweet snacks and like working on a team. Yahoo! users are optimistic or pessimistic, depending on the situation.

I’m primarily a Gmail user, and definitely not a young man under 34, not single, not thin. Maybe I’m the exception that confirms the rule, but looking at how the data was collected and how it’s analyzed, you start wondering about what they mean by “margin of error is +/- 1%”. First of all, the sample is composed of Hunchers (people who bothered to answer their 20 questions to build a taste profile). The majority of respondents use Gmail, and Yahoo! is not even the second largest contingent. Contrast that with other data showing that Yahoo! Mail may have three times more visits that Gmail, even though that advantage seems to be shrinking.

Of course, this is nothing new. A year ago, as the Hunch post acknowledges, the Oatmeal has done a similar, tongue-in-cheek, analogy (please do visit their site for a more readable image):

Oatmeal: Email Domains

Similar to the Mac guy vs. the PC guy, and all the generational stereotyping, these shallow interpretations of market segmentation carry some degree of prejudice behind their light-hearted approach. Of course, there’s no such a thing as the average person, which would be Asian, Christian, a Mandarin speaker, with no access to computers or the Internet and no University degree. Most of us would not fit that profile.

That’s not to say that market segmentation is not a useful tool, but a bare minimum quality to it is needed. The text book example of inappropriate use of that tool is to divide table salt buyers into blond and brunette customers and mistake the differences between those two groups as indicators of purchasing behavior. Useful market segments need to be measurable, substantial, accessible, differentiable and actionable (Kotler et al.).

Of course, there is probably some merit in the domain comparisons with regard to AOL users. Because AOL was extremely popular as an Internet service provider in the 1990s and almost insignificant now, it does serve as a marker of a given segment of the population who remained loyal to it. Other than that, most of the attributes linked to the other major mail domains are likely to not be substantial, differentiable and actionable. Discarding Yahoo! Mail and Hotmail users as not being computer savvy or career-focused or “plugged-in” may be a major mistake in one’s online marketing strategy.

A solution for handling accents on a Mac

17 01 2010

In the never-ending Mac versus PC discussion, I often find myself slightly favouring the Apple side for better hardware-software integration and for consistently bringing innovative and elegant solutions for some of the personal computing usability pain points. But it has never been a slam dunk for his Steveness, as Macs are not perfect either. By the way, my MacBook freezing problem is still there, making me likely the person who had the most problems with Mac computers in the world: video card, keyboard, trackpad, battery, optical drive, cooling fan, hard drive, and LCD screen (this one was my own fault).

My number one pet peeve on the Mac world ***WAS*** the way Mac OS handles accents when you have a US keyboard and want to write in one of the European languages that need accents and other special characters. I often write things in Portuguese, and in Windows, after you configure it properly, the keyboard just mimics the way typewriters handled accents: for example, to type the <é> in “Pelé” or “café”, you just type the key <‘> and then the letter <e>. On a Mac, once you configure it to use “Brazilian” as the Input Source, to type <é> you have to do something very awkward: type <Option>+<e>, then type <e>. Basically, <Option>+<e> is the acute accent, <Option>+<i> is the circumflex accent, <Option>+<n> is the tilde and so on.

Of course, the point here is not to mimic a typewriter – a large part, if not the majority, of people using computers today never used a typewriter. The point is that the US keyboard has keys with the acute accent, the circumflex accent and the tilde, so why not use them? <Option>+<e> is not intuitive at all, and you have to use three keystrokes instead of two, slowing you down enough to break your typing rhythm.

But there is a solution and I just found it yesterday! If you want your Mac to handle accents the Windows / typewriter way, try the following:

  1. Google “brasileiro.bundle teclado” (the original file in Geocities is no longer available). I found it here.
  2. Unzip the file and double-click the .dmg to mount it
  3. Copy the file Brasileiro.bundle to the folder /Library/Keyboard Layouts
  4. Log out and log in
  5. In System Preferences, go to Language & Text and then select Input Sources
  6. On the left panel, you should now have Brazil and Brazilian, with round Brazilian flag icons
  7. Enable Brazil if you are using a US keyboard, or Brazilian if you are using a Portuguese keyboard
  8. Enable Show input menu in menu bar
  9. In the menu bar at the top of your Mac, you should now see a flag with your default input language (in my case, it’s Canadian English). Just switch that to Brazil or Brazilian when typing in Portuguese. It also should work with all accented characters in Spanish and most in French (except the ligature ones: ae and oe; for those, I suspect you still have to resort to <Option>+<‘> and <Option>+<q>, or get a French.bundle file).

Input Menu with "Brazil" and "Canadian English"

I hope this helps the 0.01% of you for whom this is a Mac annoyance. And I hope that Apple fixes that in the near future to come as a default input source instead of hack.

Best Buy Flyer is my muse (No, that’s not a papaya in his hands)

10 01 2010
First work week in January was super-busy, it felt like 3 weeks in 1. My MacBook also started acting up, freezing every couple of hours, preventing me from backing up my disk. I moved all the important files to my cheap HP Pavillion, repaired the MB hard drive and I’m hopefully back to normal now.

End result is that I did not have much time for anything other than keeping my nose above water level. Got a Pogo Style for my iPhone (thanks Béné for the tip) and only had a chance to play with it today.

This is my first drawing using the stylus: no photo overlay, just observation of a photo on a Best Buy flyer. I don’t understand why nobody can come up with an iPhone stylus with a pencil-like tip. The Pogo is better than using my fingers, but I still feel like I’m drawing with a fossilized cigarrette butt.

Sent from my iPhone

Posted via email from The Green T

The orange bucket at Ikea

2 01 2010

If I could, I’ll do drawing or music for a living. There’s a huge mismatch between the things I love doing and things people would pay me for doing. As a matter of fact, in the case of music, I can easily find people who would pay me for not doing it 🙂

Latest iPhone drawing based on one of my fave L’s pictures:

Sent from my iPhone

Posted via email from The Green T

Happy New Year! Feliz Ano Novo!

31 12 2009

A great 2010 to all of you!

Sent from my iPhone

Posted via email from The Green T

Drawing with SketchBook Mobile on the iPhone

29 12 2009

For a limited time, Autodesk SketchBook Mobile is on sale for $1.99. I just bought it, and I’m really impressed on how easy it is to draw with SBM on the small iPhone screen. This is my second drawing (the first one was horrible, so I promptly deleted it). Of course, I’m cheating a bit here: I took a picture of a magazine ad and draw over it using the layer feature. But for a 10-minute, it’s not too bad: it took me longer to type this on the phone than the whole sketching process.

Sent from my iPhone

Posted via email from The Green T

Top Web 2.0 Expo Keynote Videos: Irene Greif (IBM) and ManyEyes

25 09 2008

Irene Greif’s presentation on ManyEyes was actually a great success. You know that, as I work for IBM, my opinion on this is biased, so I’ll be short and just point you to the feedback other attendees have provided at Crowdvine. The live example visualizing Attorney General Alberto Gonzales’s testimony to members of the Senate Judiciary Committee was very compelling.

Here’s the video:

The Olympics and Power Law Distributions

22 08 2008

I find amusing all the discussion around ranking countries in the Olympics Medal Standings based on the overall total or the number of golds. This may be relevant for China or the US, as holding the top position is a strong statement in world sports dominance. In the case of Brazil and Canada, as of this writing, it may mean a jump from #26 to #16 and from #17 to #13, respectively, on the stands, which may look like a big deal, but in a cold analysis, you’re just seeing a Power Law distribution effect, the math pattern behind the long tail.

When you are in the long tail, you’re merely comparing peanuts. One extra gold medal may make you go up several positions, but a jump from 40th place to 20th does not mean that you improved 100%. Using the gold-medal-first rank, Brazil was #52 in Sydney (2000) and #16 in Athens (2004) and Canada #21 and #24. The variation there does not mean that those countries became much better or worse in a 4-year span. It just means that they both are in that majority where sports excellence is the exception, not the rule. Nothing to be ashamed of.

Our brains are used to normal distributions and linear relationships and we tend to interpret logarithmic relationships in a linear way. I remember a speaker making a joke about a supposedly dumb statement by a US presidential candidate around the lines of “silly person was astonished to learn that half of the US population was below average in performance criteria X”. The underlying assumption was that “average” always marks the middle point of a distribution. Of course, that only occurs in perfectly normal distributions, with mirrored tails on both ends.

Inspired by Clay Shirky in his excellent book “Here comes everybody”, I plotted the medal stands and got the following curve:

The speaker above was probably thinking about median, not average. Start paying attention to published stats around you, and you will notice how often numbers are over-extended, converting subtle differences in absolute rankings. I think I mentioned this in a previous post: numbers don’t lie, but they can easily mislead.

Flickr or Picasa Web Albums?

21 01 2008

While I like Flickr for its Web 2.0 approach to pictures, the limitation of 200 photos for the free account is a real hassle. So, for the time being, I’m switching to Picasa Web Albums. See my albums here: