Martin Atkins (mart) wrote in apparentlymart,
Martin Atkins

Social Whitelisting and Swarming with OpenID

Simon Willison proposes the sharing of whitelists as a way of countering the spam problem. His approach of just dumping out the whitelist is a nice, simple implementation, but it has scalability problems as the whitelist grows larger.

By coincidence I wrote the other day a draft of a very simple Group Membership Protocol which lists as one of its use-cases whitelisting. While it solves Simon's scalability problem, it creates another one: it's simply not feasible to query a large number of groups when accepting a blog comment or forum post, as the overhead of doing more than a few queries would be enormous. So how can social networking be deployed in a manner that does scale?

I think one answer to this is to exploit network effects and use swarming to propogate whitelists. Each site has a small set of trusted whitelists, and each site exposes its whitelist as a queryable set. When we do a whitelist lookup against our trusted neighbours, we cache the outcome and expose that in our own whitelist.

We don't trust everyone in the world, though; we trust our friends, and we probably trust our friends to have trustworthy friends. We need a way to determine how many levels removed a given whitelist assertion to determine its trustiness, so we can augment the Group Membership Protocol with a TTL to prevent request loops and an extra response parameter in the XML that tells you how “far away” the response was.

I'll probably not trust and forward assersions that are made more than two or three hops away. However, if I get a whitelist recommendation that I agree with, I can then add them to my own whitelist and thus make the assersion stronger for those sites that depend on me.

Bloggers tend to read and post in small communities of other weblogs. The whitelist network is likely to form into mesh-like clusters in particular communities; someone who makes good posts in one identity-related blog is likely to be whitelisted on most other identity-related blogs quite quickly, and people who are active in more than one community will act as bridges between communities. Whitelist entries can then fan out from specific communities like OpenID out into general Identity and eventually out into general web development and beyond.

I guess it's time to write a Group Membership Protocol library!


  • Moved to TypePad is now hosted on TypePad rather than LiveJournal. All of the old content remains over here in LiveJournal land, but those who are…

  • Moving the Goalposts

    In the few weeks since I published the first drafts of AtomActivity, ActivitySchema and friends several things have come about: FriendFeed is…

  • Activity Streams and Comment Aggregation

    One pain point that exists for activity streams right now is the dispersal of responses over various networks. When I post a blog entry like this…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.