This blog can't be viewed on LiveJournal. Instead see

Social Whitelisting and Swarming with OpenID

22nd Jan 2007

Simon Willison proposes the sharing of whitelists as a way of countering the spam problem. His approach of just dumping out the whitelist is a nice, simple implementation, but it has scalability problems as the whitelist grows larger.

By coincidence I wrote the other day a draft of a very simple Group Membership Protocol which lists as one of its use-cases whitelisting. While it solves Simon's scalability problem, it creates another one: it's simply not feasible to query a large number of groups when accepting a blog comment or forum post, as the overhead of doing more than a few queries would be enormous. So how can social networking be deployed in a manner that does scale?

I think one answer to this is to exploit network effects and use swarming to propogate whitelists. Each site has a small set of trusted whitelists, and each site exposes its whitelist as a queryable set. When we do a whitelist lookup against our trusted neighbours, we cache the outcome and expose that in our own whitelist.

We don't trust everyone in the world, though; we trust our friends, and we probably trust our friends to have trustworthy friends. We need a way to determine how many levels removed a given whitelist assertion to determine its trustiness, so we can augment the Group Membership Protocol with a TTL to prevent request loops and an extra response parameter in the XML that tells you how “far away” the response was.

I'll probably not trust and forward assersions that are made more than two or three hops away. However, if I get a whitelist recommendation that I agree with, I can then add them to my own whitelist and thus make the assersion stronger for those sites that depend on me.

Bloggers tend to read and post in small communities of other weblogs. The whitelist network is likely to form into mesh-like clusters in particular communities; someone who makes good posts in one identity-related blog is likely to be whitelisted on most other identity-related blogs quite quickly, and people who are active in more than one community will act as bridges between communities. Whitelist entries can then fan out from specific communities like OpenID out into general Identity and eventually out into general web development and beyond.

I guess it's time to write a Group Membership Protocol library!


  • What about signed messages?

    Well if you want a system for building a web of trust between users and sites, what about cryptographic messages? PGP ( is built around establishing trust and verifying identities. So when someone posts you just have to a) check that the poster is a trusted person and b) that the message is authentic?

    This is, however, well outside the domain of OpenID. But isn't that as it should be? Shouldn't my OpenID be associated with my PGP key at a higher level? Anything we can do to avoid the management problems of blacklist/whitelist "solutions" is a step in the right direction IMHO.
    By ext_28522 at 11:35 pm on 22nd Jan 2007
    • Re: What about signed messages?

      Hash: SHA1

      Sorry, I should try to lead by example.

      As I was trying to say, messages should be cryptographically signed. Sure,
      there needs to be an easy mechanism to do this, but it is the right solution in
      my opinion.

      -----BEGIN PGP SIGNATURE-----
      Version: GnuPG v1.4.3 (GNU/Linux)

      -----END PGP SIGNATURE-----
      By ext_28522 at 11:45 pm on 22nd Jan 2007
    • Re: What about signed messages?

      One of the virtues of OpenID is that it takes all of that complicated cryptography stuff out of the user's hands and lets some other entity (the OpenID Provider) control it on the user's behalf. Of course, the user can choose to do it himself if he wants by running his own personal OpenID Provider.

      Technologies like PGP are doomed so long as I have to be at my own PC to use them. Perhaps with some help from protocols and client software the usability can be improved, but as long as I'm identifying myself with something I can't remember I'm tied to using my identity only in locations where that something (my PGP key) is already stored.

      By Martin Atkins at 08:32 am on 24th Jan 2007
  • i'm working on it...

    hi. first, thanks for mentioning stuffopolis. second, i've checked in some code (php, rails coming soon) for group membership. i stayed consistent with your parameters but using POST instead of GET:

    shoot me an email. i would love to talk to you.

    herestomwiththeweather [at] gmail
    By ext_10012 at 10:20 am on 8th Feb 2007