There’s been a lot of discussion lately about the qualifications of self-proclaimed social media experts who market to lawyers.  People like Brian Tannebaum and myself are convinced that many (if not most) of these self-proclaimed experts are snake oil salesmen (or as I more colorfully put it, “bullshit peddlers”) hoping to take advantage of lawyers.  But surely not every one of these people is a charlatan.  I therefore began to wish that there was a way for lawyers to determine whether or not an “expert” is actually qualified to actually opine about a specific subject.  And then it hit me…. lawyers…. experts… opine… Daubert! 

In 1993, the U.S. Supreme Court decided Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993) and articulated a new standard for determining the admissibility of expert testimony.  Now, any lawyer wanting to hire an expert witness for his or her case is likely to make sure that expert can survive a Daubert challenge.  (Except of course in those jurisdictions that use Frye.  Since Daubert is considered to be the harsher standard, I’ll use it here today.  If any social media expert tells me that he or she doesn’t qualify under Daubert but does under Frye, I promise I’ll run him through Frye.)

Daubert offered a list of nonexhaustive and nonexclusive factors to determine whether expert testimony is sufficiently reliable to go before a jury.  The four major Daubert factors are: (1) Testing and testability; (2) Peer review and publication; (3) Error Rate; and (4) General acceptance in the scientific community.

Testing and testability

The Daubert court quoted philosopher Karl Popper, who stated that “the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability.” Falsifiability simply means that something is capable of being proven false.  Testability is a more specific kind of falsifiability that means that a specific theory can be falsified through testing. 

The areas of question for a social media expert here are twofold.  First, are his or her theories falsifiable, and second, has he or she tested them?  I suspect that most self-proclaimed experts will offer theories that are not falsifiable because they will be predicated upon vague and ambiguous concepts.  Consider an expert who tells you that you need to write trustworthy and authoritative material to gain incoming links to your site.  Trustworthy and authoritative to whom?  These terms lack objective definitions, so such advice cannot be falsified.  But if we do agree on the proper definitions, can we falsify such a statement?  Yes, through testing.  And here’s a test for you: Write an article about the law that is so compelling that the Harvard Law Review dedicates an entire issue to your article alone.  Post it in its entirety on your web site.  Then, create a web site dedicated to advancing the most hokey, unscientific theory you can imagine about the cause of some childhood illness.  Make sure the theory also relies heavily on conspiracies between the government and multibillion-dollar corporations.  Hire Jenny McCarthy to promote the web site, and see which site gets more incoming links.

The next area of questioning is whether the purported expert has actually tested his or her theories.  Does he or she maintain a series of discrete social media profiles (blogs, web sites, twitter accounts, etc.) to test different theories?  If not, how can the expert test to determine what is and is not effective?  Using his or her own profile isn’t effective because his or her own profile is (a) not similar enough to a practicing attorney’s to offer useful comparisons, and (b) is probably riddled with the results of old, discarded theories.  Testing based upon his or her own profile would be the equivalent of a pharmaceutical seeking approval for a drug for children by running tests on elderly people who already have a dozen other drugs in their system.

If an expert is offering you vague and ambiguous theories that he or she has not tested, he or she would fail Daubert.  If you wouldn’t hire such an expert for your clients, why would you hire him or her for your law firm?

Peer review and publication

Justice Blackmun explained in Daubert both why peer-review should be considered, and why it shouldn’t be dispositive:

Another pertinent consideration is whether the theory or technique has been subjected to peer review and publication. Publication (which is but one element of peer review) is not a sine qua non of admissibility; it does not necessarily correlate with reliability, and in some instances well-grounded but innovative theories will not have been published. Some propositions, moreover, are too particular, too new, or of too limited interest to be published. But submission to the scrutiny of the scientific community is a component of “good science,” in part because it increases the likelihood that substantive flaws in methodology will be detected. The fact of publication (or lack thereof) in a peer reviewed journal thus will be a relevant, though not dispositive, consideration in assessing the scientific validity of a particular technique or methodology on which an opinion is premised.

When interviewing a self-described social media expert, ask him for a list of the peer-reviewed publications he’s been in.  You may very well be told that he hasn’t been in any peer-reviewed journals, but has instead done one-better by writing a book.  Getting a book published is impressive, but does not qualify one to be an expert.  Recall of course that Suzanne Somers has written books about womens’ health when in fact she’s a quack. 

The social media expert you’re interviewing may suggest that social networking is “too new” to have dedicated journals.  Well, what is social media?  It’s communicating through computers.  And the Journal of Computer-Mediated Communication has been online since 1995.  In 2007, that journal published Social Network Sites: Definition, History, and Scholarship, an article with a good history of social media.  It also provides a good starting point for further research in its lengthy bibliography.  The handful of references below come from the bibliography:

Backstrom, L., Huttenlocher, D., Kleinberg, J., & Lan, X. (2006). Group formation in large social networks: Membership, growth, and evolution. Proceedings of 12th International Conference on Knowledge Discovery in Data Mining (pp. 44-54). New York: ACM Press.

Donath, J., & boyd, d. (2004). Public displays of connection. BT Technology Journal, 22 (4), 71-82.

Golder, S. A., Wilkinson, D., & Huberman, B. A. (2007, June). Rhythms of social interaction: Messaging within a massive online network. In C. Steinfield, B. Pentland, M. Ackerman, & N. Contractor (Eds.), Proceedings of Third International Conference on Communities and Technologies (pp. 41-66). London: Springer.

Hodge, M. J. (2006). The Fourth Amendment and privacy issues on the "new" Internet: Facebook.com and MySpace.com. Southern Illinois University Law Journal, 31, 95-122.

Lampe, C., Ellison, N., & Steinfeld, C. (2007). A familiar Face(book): Profile elements as signals in an online social network. Proceedings of Conference on Human Factors in Computing Systems (pp. 435-444). New York: ACM Press.

The references above should tell you that (a) there are peer-reviewed publications that deal with the science of social networking, and (b) if a self-proclaimed expert tells you that there aren’t, he’d get his ass handed to him in a Daubert hearing.

Error rate

One of the shortcomings of Daubert is that it declares that there is an acceptable rate of error for a scientific theory, but doesn’t declare what that error rate is.  The best we’re given is a citation in Daubert to United States v. Williams, 583 F.2d 1194 (2d Cir. 1978), cert. denied, 439 U.S. 1117 (1979).  In Williams, the Second Circuit held that a false positive rate of 6.3% was admissible.  I suspect that most social media experts have no idea about whether their theories have an error rate below 6.3%, because they likely have no idea what their error rate is.  If you ask, you might be told that it’s not possible to compute an error rate for something like social media.  That’s incorrect.  It’s simple, really: Ask them how many times they’ve given their clients advice.  Then ask them how many times that advice turned out to be wrong.  If they can’t do the math to figure out the error rate, it’s yet another sign that the social media expert would not survive Daubert scrutiny. 

General Acceptance

The final Daubert prong is actually the main prong of Frye, and that is whether or not the expert’s methodology and opinion have achieved any level of general acceptance in the appropriate scientific community.  Let me save you some time and trouble here.  If any expert disagrees with these principles, he not only would fail Daubert, but fails at the Internet:

• Be honest about yourself and your experience.  (Don’t bill yourself as an expert in an area of law if you’ve only handled one case in that area, for example.)
• At a minimum, have a website, a blog, a LinkedIN profile, and a Twitter account.  Consider Facebook.
• Try to blog regularly – at least once a week.
• If you build a reputation as a great lawyer, it will be easier to build a reputation online.  (Consider Gerry Spence’s blog.  If he weren’t Gerry Spence, it would get no traffic.)
• Every now and then, write a post from the heart.
• Link to people you know and trust, and don’t be afraid to ask them to link to you.
• Participate in online discussions about topics you’re knowledgeable about.
• Don’t expect success overnight.

Conclusion

If people are going to not only sell themselves as experts, but try and sell themselves as experts to lawyers, isn’t it fair to expect them to undergo the same scrutiny you’d subject an expert to in a lawsuit?  I know that many of you are probably a little intimidated about social media and technology in general.  So I’ll tell you what: If you’ve got a question or two about social media, shoot me an email.  I am not an expert!  But I’ve been blogging since 2003 and I probably know enough to prevent you from wasting money on someone who hopes to exploit your ignorance.