Feelings on Buzz Tracking Services
Michael Arrington of, well, you know, has an article covering a new buzz tracking service called Scout Labs. Here’s the idea: Businesses (brand managers in particular) create an agent called a “scout” for their brands. Scout Labs then looks back to find mentions of your brand (blog posts, videos, etc) and scores the mentions based on a sentiment continuum (positive, neutral, negative).
In his post, Arrington noted that the software is still fuzzy when trying to decide whether a post is complimentary or negative based on language and context:
The company created a Scout for both TechCrunch and Michael Arrington and let me review it last week. For the most part they got the hard part right - determining if a blog post was positive, negative or neutral. They don’t deal with irony well, though. One blog post about me called me “high and mighty.†Scout Labs thought it was positive when it was in fact highly critical. But I hit a button and switched the post to negative in a click. The company uses this feedback to train its system towards better future results.
Scout Labs is one of a number of companies (ex: BuzzMetrics, BuzzLogic) that are trying to automate and measure blog coverage when it comes to companies and brands.
Technical Questions
I’m very curious about any service that tries to quantitatively measure tone, so whenever I hear about a service like this, I immediately wonder:
- How do they measure influence? For instance, a mention of Scout Labs on blogstring obviously has a much lower influencer rating than, well, TechCrunch. So how does the service rank a blog’s influence?
- Piggybacking on the first question: How often does the service “re-rate” blog influence? Let’s say this blog gains popularity quickly (hey, it could happen). How does the service reevaluate the influence and importance of postings?
- Speaking of re-evaluating, what role does the freshness of a post play? Does the service factor in frequency of posting?
- Does the service piggyback on existing ratings (ie Compete, PageRank, Technorati Authoriry, etc.)?
- How does the service spider blogs? Does it obey robots.txt? How often does it spider?
- Does the service hide itself from bloggers (does it show up as a bot or referrer)?
Gut Level Questions
Now that I’ve got the “under the hood” questions out of my system, I have one big question:
What do companies do with the data?
Obviously there is a difference between providing data to a company and what the company actually does with it. And since I’ve been on a personal responsibility kick lately, I’m going to spare you that discussion……
Instead, let me go the other way and ask:
As a blogger:
- How do you feel knowing that companies are monitoring blogs to see if you are talking about their products?
- If you looked at your referrer stats and saw that you were getting a large number of new visitors from a buzz monitoring service, how would that make you feel?
I know that someone reading this will say “um, Nathan, I hate to break it to you, but companies are already monitoring blogs for mentions.” That person would be right, and I know that. I’m not presenting this as news. I’m not breaking new ground here. Instead, I just want to know how this makes you feel.
I could see different people having different reactions ranging from validation (this blogging thing must be paying off if Coca Cola cares about me) to paranoia (hey there evil corporation, stay outta my blog, maaaaaan).
So how does this make you feel?
Filed under: blogging, statistics, transparency
PS- If you’re here from Scout Labs, can I have my beta invite?
If you can find 10 PR people who are happy with this after a year or two, i’ll be impressed. I constantly get the “what do you do with the data question.” You just can’t do this stuff entirely with computers, you need humans.. Reliable, trained, efficient knowledgeable and economically viable humans. (Okay, that’s a plug for our folks up in Berlin, NH) But If microsoft doesn’t trust computers to do this stuff why should anyone else?
Nathan! We love you. SUCH good thoughts and questions. (Do you want a job ;-))
We are not launching for a few months, while we do nothing but listen to customers, so I apologize in advance for being more vague about algorithms and the technical stuff than you would clearly like. But we can say a bit.
We have an influence algorithm which applies to publishers which DOES change and morph over time and that IS specific to the topic you are scouting. (It’s not a generic influence score, a la Technorati “authority”). We use data that’s generally available (you listed some great ones) and some proprietary data that only we have our hands on.
Then we have a Significance algorithm which tells a user what mentions (blog post, user review, article, etc.) to pay attention to. It takes into account the traffic to and level of influence of the publisher, certainly, but post-specific “activity”, especially as its measured as a rate (velocity), is most important here. We want companies to know what consumers are paying attention to RIGHT NOW so that companies can tune in when and where its needed.
But your last commentary about “SO WHAT”? is the best. Influence and Significance and Entity-specific sentiment are just raw ingredients. By themselves, they are quite un-interesting. (It’s funny that this is where much of the comments and “buzz” has been since Arrington’s post.)
We have marketing in our DNA and have had the honor of having over 40 major brands advising us since day 1, so we actually set out to see if we could answer, in an automated way, the questions that most every marketer wants to ask her customers. We quickly realized that we would need things like sentiment detection, influence and significance to answer some of those more complex questions, but we see them as eggs, flour and salt. We’re busy making pastries.
As for your beta invitation, we are rolling companies on a few a week, and after today, there is quite a backlog
We’ve got you on the sneak peek list in any case. But you sound like a GREAT beta advisor / tester and I’m sure we will speak more soon.
Thanks again.
–Jenny and the Scout Labs team.
Good post, Nathan.
Can’t help but feel that technology still has a long way to go before content analysis like this can be done effectively by software. Especially for the blogosphere, which is full of sarcastic and humourous comment and must present a significant challenge to a piece of code. But I hope I’m wrong!
On your last question, I don’t think that anyone should feel any different knowing that their blogs are being read by companies. Bloggers know that when they write something, it is now in the public domain and there for ALL to read. And that includes the evil corporations.
Just as every blogger has (or should have) a right to freedom of expression, it is everyone else’s right to be able to view that content and pass their own judgement on it.
As an individual, I may read several blogs on a topic to build a perspective on a subject… companies are entitled to do the same to help build a picture of how the market views them. As long as we don’t see an uptick in bloggers having mysterious accidents, I think it’s safe to say buzz tracking doesn’t pose much of a threat.
Let’s be honest, if people feel uncomfortable about their blogs being monitored, then why are they writing them to begin with?
Cheers,
Danny
Thanks for the comment, Danny. I totally agree with the sentiment that if I felt uncomfortable having my blog monitored, I wouldn’t write it. In fact, I WANT people to find it and do all I can to promote it! So to me, if someone finds my blog because of a monitoring service, I’m happy.
I guess I’m really just asking how bloggers feel about this because even if people shouldn’t feel any different, I think they might.
I think there’s a difference between having your writing available to all to discover and having automated software looking for keyword mentions. I’m not saying that the software is bad, and I’m not saying it’s good. Just a different way to discover your stuff.
To me, there’s a natural association between brand monitoring through blogs and PR pitches. And as you know, there has been a significant backlash against those who pitch blindly to bloggers. So it’s not the fact that these monitoring services exist that is noteworthy to me. It’s what the companies will do with the information that I see as really, really interesting.
So in an effort to stop my senseless blathering on the subject, let me just say this: we’re all being monitored in one way or another. How we’re being monitored (ranked, scored, etc.) and how companies will act on that data are the things I find fascinating.
Thanks again!