Signals of Noise

Since I did my engineering major in Instrumentation and Control, I have been reading all about measurements. Though my work has never been directly related to using or designing measurement devices, my work has always been, now that I think about it, related to measurements.

At my first job I was writing configuration software for power quality measurement devices. Post ISB I have been working on search, web and marketing analytics. All related to measurement. Which brings me to this post on the the suitability of accuracy vs. precision in the field of measurement.

Before proceeding further lets see what each means though in regular usage we are wont to use them interchangeably. Wikipedia has the following to say about accuracy and precision.

In the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to its actual (true) value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which repeated measurements under unchanged conditions show the same results.

Since we generally use each of these terms as synonyms we are always striving for accuracy.

First we must agree that there can be no measurement without error. All measurement systems physical, on the web or data is prone to systemic errors. Whether due to imperfect calibration, external influences or imperfect methods of observation. We simply must live with error.

So how do we get around this omnipresent error?

Lets think of the true value of a variable as a point in space. And the measured value is a region around/some distance from this point. The distance and size of this region from the true value depends on the measurement error in the system. The distance (from the true value to the mean of the measured value) represents the accuracy of the system while the size is the precision of the system. So if we focus on accuracy the distance is lower, but the size could be big. Which essentially means that knowing one measured value, it is very difficult to predict the next value even though that may be pretty close to the true value.

On the other hand, for a high precision system, even if the measured value is very far away from the true value, it is much easier to predict the next measured value. In other words, accurate systems have lower errors which are more random in nature, while precise systems (could) have higher errors which are less random. Of course a system could be either accurate or precise or both or none.

As you see while striving for accuracy can reduce the absolute error, it increases the randomness of the error value as well. While it is just the opposite when striving for precision. The decreased randomness also renders itself more favorably towards predictive ability of the system.

Hence, analytics systems should really be chasing precision and not accuracy. Knowing that a system always has some definite error is surely better than knowing that a system has a small but relatively indefinite error.

All said, the pursuit for an accurate AND precise system continues.

Some more on the topic by Avinash Kaushik.

[image: Wikipedia]

I have decided to move the trivia posts away from this blog to my tumblog. Primarily because those posts were not quite in line with the primary objective of this blog.

I will still continue to post interesting trivia, photos, videos and links at the new blog. I have also syndicated my Delicious shares and Flickr images to the blog. There is also a widget on the sidebar of this blog to show recent updates over there.

Here are some quick links:

• entropy dot me: http://entropy.sumande.com
• RSS Feed: http://feeds.feedburner.com/entropydotme

Do you like it? Let me know your thoughts.

When you are in early stage product development, a user story is one of the most important items that you should consider. A user story is a description of a goal/scenario a (typical) user would want to achieve with the product.

Why is a user story so important?

1. Whether you are building a product within an organization or you are a start-up, you have to sell your product to those who are willing to sponsor the development. If your product addresses a very clear or common problem and there is a demonstrable market out there, your job becomes easy. But still it may not be suitably clear to everyone how your product will be of help.
2. You want to partner with someone to co-create the product or have them tweak a product of theirs to work with your product.
3. Explain clearly to the engineering and quality teams a typical use case and the relationship between the product modules.
4. Pitch your product to potential customers.
5. Consistency when different members of the product team evangelize the product at separate locations at different times.

Here are some steps that you could consider when coming up with a user story. 

1. List the product modules and their features (on a whiteboard or anyplace the team can see easily) so it is easier to visualize the relationship between them when writing the story. This is important to ensure that your story includes only what your product offers currently while leaving scope for expansion.
2. Identify the user who is going to use the product and make the story compelling for her. It makes no sense to write a story with the CEO in mind when the CFO is the one who will be using the product.
3. Identify one problem in an area that the user will be responsible for. Else she’ll not be interested.
4. Then show how the product helps identify that problem and solve it. It is important here to cover as much of the product features as possible without reaching novel length. Not all features may be useful for a particular person you are pitching to, but will at least include those that matter to her.
5. Link it back to a beneficial outcome solving the original problem to close the loop. If it doesn’t help the user, it doesn’t help you. Period. And the user must realize the benefits.

What are some other considerations for writing user stories? What are some other reasons for wiritng a user story? Do you have an example where a user story failed and what did you learn from it?

[image: Flickr/Anna Gutermuth]

Forrester analyst Doug Williams writes about his research on the use of social media to co-create with consumers.  An opportunity for consumer product designers that, he feels, is underutilized to a large extent.

… that 83% of companies use social media, but fewer than half of those have product teams that are currently using social media to influence product design, creation, or strategy. In that report, I also divulge that 72% of consumer product strategy (CPS) professionals claim that social media will enhance their existing capabilities of using customer input to shape product strategy.

The root of the problem is that most of those responsible for consumer product strategy are not sure if their consumers are willing to participate in co-creation. A survey was conducted to find out the willingness of US consumers to co-create with their favorite brands. More than half of those surveyed said that they would.

Here are some highlights from the survey published in the post.

•Sixty-one percent of US online adults are what I call “willing co-creators:” they would consider providing input to help companies design and build new products or improve existing products.

•A majority of willing co-creators would participate in a co-creation engagement regardless of the product, brand, or service involved — although we tend to think that the 30% who would only participate in co-creation efforts with their favorite products and brands might be more engaged and thus produce a better co-creation experience.

•In 12 of the 17 industries we specified, more than half of the willing co-creators expressed interest in co-creation engagements.

(Full report here.)

Co-creation is an extension of what is popularly known as crowd sourcing. While crowd sourcing tend to include large community driven programs like Wikipedia and Linux, co-creation is a much more niche business strategy to derive mutual benefit for the company and its consumers. A successful co-creation program can be extremely beneficial as it allows the company to come up with the product its consumers want without a large outlay for market research and product development.

A co-creation program was recently launched by Lays in India where they partnered with its consumers to create new flavors and opened up to the public to vote for the winning flavor. A perfect blend of social media and co-creation.

For more of co-creation, collaboration and crowd sourcing you should definitely read WIkinomics.

Update: A new post on the Forrester blog provides some more details from the research. Personal technology, home entertainment and small appliances and consumer packaged goods lead the interest industries for co-creation. Read more here.

[image: Flickr/Jacob Boetter]

There is no dearth of data today. Every Internet channel of worth – from Google to Twitter to Facebook – is collecting sh*t loads of it. Then there are all these analytics vendors who are doing their own crawling. But what good is all that data if there are no insights from it?

Zach Hofer-Shall from Forrester considers these questions on his blog. He rightly suggests that before diving into the ocean of data consider what you might be able to do with all that data. Will it turn up insights that are valuable? Are those who intend to use this data even prepared to be bombarded with the never-ending flow of real time data?

But there’s a bigger question at hand: what will these vendors do with the firehose? Back to my rhetorical question before: what good is a plethora of data if you can’t find what you’re looking for? With more data comes increased importance of text analysis. Being able to pull in 65 million tweets a day is great in theory, but requires vast amounts of processing and analysis to actually turn it into something of value. When someone tweets about what they had for lunch or I tweet about a soccer game I’m watching, these irrelevant items build into the countless others that make up that firehose.

Having worked in the area of analytics I can vouch for what he says. Whatever channel you choose, there is an overwhelming amount of data that is available. But all that is loaded with lots of noise. Extracting meaningful analysis out from it is the big challenge. Furthermore if you are analytics vendor there is the added challenge of presenting all the insights and enabling your customer to use those insights in a way that is trivial.

A recent article on CrunchGear dwells on the thought that if you are mad at Google what would you do? Would you stop using it? Can you even stop using it? The post was largely influenced by a report in the Wall Street Journal on Google’s policy decisions on using personal data for its ad targeting network.

The WSJ report is fodder enough for a full dissertation. However, I was more wondering on whether it is really possible to stop using Google? Let’s see what Google services I use and what are the alternatives out there.

• GMail: This one’s the most difficult of the lot. I started using GMail right around the time it was launched. Loved it the first time. There really wasn’t any other better alternative back then. The interface and features were just perfect. Of course some computers were reading my mail. But it was ok.
  Anyway. With 23k+ mails in my mailbox. It will be difficult. Besides my ISB alumni mail uses GMail with Google Apps. And I can’t really influence them to change.
• Google Search: This really would be expected to have come first, but then my default browser search is Bing. So essentially I can live without it. Though I still fall back on Google sometimes for their better search refinement options.
• Blogger: Had my first blog with Blogger. But as you can see have shifted since then.
• Picasa: I have some photos up there, but I can easily move them to Flickr. And Windows Live Photo Gallery is a decent free alternative to the desktop version.
• AdSense and AdWords: Yeah, I have some adsense ads on this blog, but then monetization is not a real necessity for me. And definitely I am not buying keywords to drive traffic.
• Chrome: I am a Firefox user.
• News: Hmm. Not found a better news aggregator yet. This one’s difficult to let go.
• Reader: Same as News above.
• Orkut: Deleted my account a few months back. Facebook and Twitter rules here.
• Docs: I still prefer Microsoft Office.
• YouTube: Hell no. Can’t let go of YouTube whatever happens. There is just no alternative. Even if the long tail of internet video is really long. (52% of all video views on the web happen outside the top 25 web video sites. But YouTube has more than 20%.)
• Google Analytics: This is another one difficult to let go. Don’t think there is a better alternative. That too free.
• Android: I am still using a Nokia, i.e. Symbian. But my next phone won’t be. It could be Android or it could be iOS. Who knows.

These are the main offerings. Other minor ones anyway I don’t use. So all in all, love it or hate it, it is just difficult to let go of Google. And they might come up with a better social network soon. And it may be a whole lot difficult for others hooked to Google Search.

What I am more interested is in knowing if there is anyone out there who can live without Google? Steve Ballmer could be one.

PS: I also want to keep seeing these lovely Doodles.

Facebook is rolling out changes to its services. Some of them are announced on their blog. Just a couple of days back they announced that they were changing the ways photos are viewed. Moving all photos into the same page like it is displayed in Bing and Google image search.

While some of the other changes you just discover. They are unannounced and very subtle.

Today they announced the roll-out of new tools for the Notes application. They have made it easier to format the text, albeit very limited controls. Tagging and adding photos is also now easier. But they seem to be ignoring the import feed part of Notes. It just doesn’t update for me. I am using it on this blog’s page to import my Delicious bookmarks. But the only bookmarks I ever see are those that were imported when I added the feed. No updates show up at all.

The other subtle change they have rolled out applies to the News Feed. Previously you could easily hide a person or application from your feed that you did not wish to see by clicking on the small ‘Hide’ button. Those options are still there but the ‘Hide’ button is replaced by a small ‘x’. But the most significant change is the ability to mark a feed item as spam.

Facebook didn’t announce this change on their blog. Or did I miss it? So I have no idea what happens when you mark something as spam. If it hides a particular spammy story floating across the network without affecting updates from the person who posted it, it would be great. Especially with all the iPad and iPhone 4′s being given away for free.

…And ReadWriteWeb is wrong.

Just a few days before I posted the video of Seth Godin arguing the necessity of pursuing innovators and early adopters, Audrey Watters of the ReadWriteWeb wrote a story on “Why We Shouldn’t Get Too Excited About Early Adopters“. I had marked it for reading and managed to catch up only today.

In that article she argues, drawing evidence from a Clive Thompson article for the Wired magazine, that it is not right for companies to target early adopters. After all they constitute just 13.5% of the market compared to the majority 85%. This based on the argument that early adopters are anyway going to buy the product. So why bother.

All this is clearly prompted by the demise of Google Wave. Which despite efforts of the early adopter community failed to gain wide spread adoption. However, the point she misses here is that Google Wave failed not because it was not targeted towards the late majority. It failed because it was not a product that would ever appeal to the majority in its current form. It was way too complicated. And the use cases were not clear. No argument against the fact that it was indeed revolutionary. But, it suffered from bad timing?

The idea of pitching a product to the innovators and early adopter is to gauge market response to the product and establish some sort of a co-creation platform where the early adopter community helps in the evolution of the early product. If they love it they will spread the word to the late majority and adoption will spread. Whether the late majority adopts is or not will ultimately depend on how well thought out the product is to target that segment of the population.

There is no doubt that taking a product from the early innovators and early adopters to the majority is a major challenge. And to say that just start at the majority is trying to solve this problem too simplistically. There is a significant difference in the wants and needs of these groups of people. This topic is very well covered by Geoffrey A. Moore in his book Crossing the Chasm. The focus should be on choosing the product features wisely, targeting the right group of customers, positioning and marketing strategy among others.

There is a separate problem with bypassing the early adopters and targeting the majority. This group (the majority) of the population is generally risk averse. So no matter how good or bad a product is they are not going to experiment with the unknown. (Unless it is a product from a brand with a reputation for well designed products spanning all segments. Such as Apple.) They depend on others (the innovators and the early adopters) to lead them towards adoption. They are unwilling to listen to what marketers have to say. Status-quo is good enough. As Seth said, if you want to make average products then for sure make them their target. Otherwise it is wiser to ignore them at the beginning.

The failings of a single product (here Google Wave, and Audrey’s opinion is clearly biased by her love of Wave) cannot be a solid reason to start ignoring the early adopters. We can then say good-bye to innovation.

[image: Wikipedia]

Twitter announced a while back on their official blog the launch of the official ‘Tweet Button’. It will generally replace the, till now, popular TweetMeme button. But Twitter says that they are working closely with TweetMeme on this. The resultant tweet will use Twitter’s own ‘t.co’ URL shortener and not bit.ly. But since the button itself displays the number of clicks, there won’t be a pressing need for bit.ly’s analytics.

Here is the video release by Twitter on the same.

I have already added it to this blog. You can get yours here. First use is always appreciated. On this post or any other.

Bing has updated its search results to include Wolfram | Alpha results. With this new collaboration you can easily type in questions and the top listing will show results from Wolfram | Alpha.

For example, maybe your child is doing a report on France and they need to find the country’s GDPor the capital, go to Bing and type in “gdp of France” or “what is the capital of France” or how about those science homework questions that you think you know but you just need to make sure, like what is the  boiling point of water? You can also use Bing to instantly compare geographical information, like comparing the current population in India to that of China or even how their literacy rates compare. Bing and Wolfram | Alpha make finding the answers to these common questions that much easier by proving this information to you right in the search results.

You could also query about dates for upcoming holidays and Bing will give you the answer.

So Bing continues rolling out rich features in its effort to take market share away from Google. Now will Google go ahead and copy Bing? Once again.