So much of the public discussion of social media monitoring and analysis focus on commercially available platforms that are more or less ready to go to work. Somebody has to set up the queries, and there may be some dashboard configuration, but the tools are generally pay-and-go propositions. A recent article points out that some companies are going for something much more customized to their needs.

In its big list issue on companies doing IT well, Information Week mentioned Toyota's new social media and CRM tool:

The tool took 60 hours to develop, largely using software Toyota already had. Oracle Endeca Discovery handles data discovery and search analytics, WiseWindow and DataSift aggregate social data, and Lexalytics analyzes sentiment.

Toyota is using the tool to improve customer service, product forecasting and quality, and lead generation. It plans to feed information to dealers in the future.

If the usual model is off the rack—software-as-a-service from a single vendor, up and running with minimal configuration in minutes, hours, or days—this is

Before you can analyze, you need data. In thinking of what you can do with social media data, I find it helpful to think about three buckets of social media data: content, activity, and people data. Let's talk about content. If you look at social media from one angle, that's what it is: lots of content. What do you do with that?

What is Content Data?
When we talk about listening and how people express their opinions, we're talking about working with content data. From the text of tweets, blog posts, and product reviews to pictures, videos, and audio recordings, content is everything that people are posting and sharing online. When people ask about sentiment, opinion, and complaints, they're asking about content.

Analyzing Content Data

Before you can pull insights from your data, you need data, but I'm hearing more concerns about data quality in social media analysis lately. Before, people asked about the traditional tradeoff in text queries: finding relevant content while excluding off-topic content. Lately, I'm hearing more about social data that's intentionally tainted. If you're looking for meaning in social media data, you may have to deal with adversaries.

Yes, and you've been playing without an opponent, which is, as you may have guessed, against the rules.
— "Anton Ego," Ratatouille

Quiz: A government agency wants to monitor social media in the course of performing its function. Is that an obvious use of public information, or further evidence of a dark conspiracy? Oh, good, I see lots of hands for both answers. Let's look at what's really going on here.

You have zero privacy anyway. Get over it.
—Scott McNealy (1999)

When people hear about social media monitoring by a government agency—such as the recent news of FBI, DHS, and CIA programs—the usual response is outrage about the perceived violationof privacy. People are living their lives online, and they don't want the government listening in.

Superficially, that's completely understandable. Most of us don't want people eavesdropping on us, even if we aren't hiding anything and don't harbor conspiracy theories. We just like our conversations to be kept within the group we think we're talking to. The usual response makes intuitive sense, even if we realize that these online conversations are, technically, public.

I think I've figured out the source of the difficulty—and controversy—in some of the measurement discussions around social media. It all starts when we talk aboutmeasuring things that can't reallybe measured, because they can't be observed. If we called it what it is—modeling—we'd see that differences in opinion are unavoidable.

Take influence. As a concept, it's not all that hard to define, and I don't think there's a lot of disagreement on what it means. But have you ever seen a unit of influence?

What did it look like? A lot like persuasion? What does that look like?

It started with a simple challenge: if I were to draw a big circle around the things I find interesting enough to follow and declare them to be one thing, how would I label it? To avoid flying completely off into pointless musing, assume that it's relevant professionally. Considering that the circle included social media, analytics, intelligence, geopolitics, and natural disasters—to pick a few—the label wasn't obvious. By declaring them to be one thing, though, it soon became clear that the theme was the importance—the value—ofknowledge.

The label was Omniscience.

"That's pretty ambitious."

In a finite world, individuals specialize, but organizations don't have the same limitations. Given enough specialists, you can do it all. The challenge is in managing them. Somebody has to get on top of all these silos.

In my ten-minute pretend-keynote at last year's Defragconference, I asked people to look beyond the existing silos of data and analytics to consider what more we could do. I challenged them with this simple idea:

Analytics + Intelligence –> Strategic Value of Information