the tools which currently consume it are:

• mjolnir (for moderators of rooms/spaces): https://github.com/matrix-org/mjolnir#mjolnir
• synapse-antispam (for server admins, hooked up to mjolnir): https://github.com/matrix-org/mjolnir/tree/main/synapse_antispam
• varunastra (our internal server anti-abuse systme)

I just read the blog post — https://blueskyweb.xyz/blog/5-4-2022-working-in-public

It feels like the world is in similar shoes…

We’re in R&D mode at the moment, experimenting with pieces that point in the right direction. We don’t have a finished product or a fully-specified protocol, but we’re putting together components

At least in my own personal work I have the same feeling. All the pieces more or less exist; it's a matter of finding some time to put things together in the right way.

ADX is an excellent collaboration of protocols. Has there been any discussion towards Encrypted Data Vaults, from the Confidential Data Storage concept Local First or Verifiable Credentials?

https://identity.foundation/confidential-storage/

Verifiable Credentials is an interesting idea to verify real people from real world 3rd party entities https://www.w3.org/TR/vc-data-model/

and last but not least there is FedCM
http://wicg.github.io/FedCM
to think about for browser adoption of identity hubs and SSO. That’s everything I have at the moment.

I've tried to build this big-tent in matrix as the #decentralised-social-networks:codelutin.com matrix space. It gather tens of rooms: we have all the official #activitypub-community:codelutin.com projects, same with #scuttlebutt-community:codelutin.com where pretty much all main projects are present and some other official rooms for movim, OHN, OpenEngiadina, SOLID, #bsky:matrix.org.

This space is part of a bigger community name #next-internet:codelutin.com which also gather P2P/decentralization project like GunDB, #yggdrasil-network:matrix.org, #ipfs-space:ipfs.io, #hypercore-community:matrix.org...

I know that Discord is still popular but I really hope we can make matrix the place to be.

Anyway thank you arcalinea. You have built the dream-team. I don't have time to follow everything that occurs in this space but I've silently followed Whyrus, Paul and others for years now. I'm sure you will publish mind-blowing things (I've read you ecosystem review x times 🤯). Thank you for letting us be part of that experiment, it's such a luck to read from great mind at work. I feel like an amateur philisopher having a diner with both Aristotle and Descartes. I really hope the moderation will work so you'll keep building in the open without being trolled.

there are a couple of constraints that lead to this:

1. We need keypairs to be rotatable, because people lose their private keys;
2. We want DIDs to be stable, and with rotation support that means we cant use public keys as DIDs - instead we use a hash of the first DID Document;
3. If using a hash DID then we need a strong & auditable guarantee of a specific history of the document;
4. We also need DIDs to be discoverable as you say -- aka we always need to be able to resolve the document from the DID

pfrazee: KERI is from what I can tell a kind of snowflake as it basically creates a "micro-ledger" (their words not mine hehe) for each DID in order to deal with key rotation etc.
That means that the UDIDC (unnamed blah blah) would need to not keep the document and it's history per-se, but the events that make it up.
I was mainly wondering if you had considered it to be honest, nothing more, thanks. :)

ps. It's pretty nice as a concept, there are some pretty interesting presentations from the authors you might enjoy. ie https://www.youtube.com/watch?v=izNZ20XSXR0

That will surely be true, but forgive me: what it is convenient or profitable for Google for “the web” to be, is probably not what the web should be.

I mean, of course, yes, Sign In With Google will use Google’s thing and it’ll mostly be fine, but the iOS app will be ugly and annoying and not work like a proper iOS app and Apple’s thing will work incredibly smoothly for SIWA sites with Apple devices and be janky for everything else, and the other 70% of the web will use something else. So now I have yet another piece of cognitive load. And the problem (“I, as a user, want to not have to think about logging in to anything and still be secure”) will not be solved.

but the system is very flexible, we can define the schemas however we want to, and then its up to the indexers from there

as whyrusleeping said here, the schema system is very flexible. Different apps should be able to define different schemas in different manners. So at the abstract level: Posts, likes, comments, labels, etc are all the same sort of object - a content-addressed document that adheres to a schema published on the schema network. Some schema systems will likely allow the (somewhat) ridiculous chain of "post < comment < like< label < like < comment" that @geoah:nimona.io points out. Others will be a bit more restrained in the sort of interactions they allow.

My opinion is we should call things what they are: Likes aren't posts. Labels aren't posts. But likes, posts, and labels are all social media documents that do share many things in common. So let's handle the shared abstraction at one level, but don't introduce complexity at the application layer by pretending that every object is the same "type" of thing. Currently, we split up the documents in a namespace by "posts" & "interactions". As Why said "A like is not a post", & as Aaron Goldman said "It is not yet clear how or even if reactions need special treatment." At the moment we are giving them special treatment, but we've had discussions about removing that divide & treating them all as the same "type" of object

Fancy stuff is great but hopefully the baseline least-common-denominator requirement would be something like just having in your Personal IPFS Repository stuff like this:

12345{
   type: "Note",
   title: "My First Post",
   content: "Hello Everyone"
}

I think if we focus on designing "types" (the properties), and don't create a ton of complexity right off the bat (at least not as a requirement) then this is going to catch on fast, and apps that can "browse" the content will spring up overnight. My platform is already poised and waiting. :)

Really happy to see this first launch from this awesome team. 😎 Congrats and best wishes! I think you are uniquely positioned to unify the somewhat fragmented decentralized social network ecosystem.

Really cool that you’re using IPLD, such a powerful tech — which didn’t seem ready when I tried it via JS a year ago, so great to see it is working in ADX now.

I dropped a PR to run tests and builds in CI, about which I am passionate, as some may remember from early IPFS days.

Hey Bluesky team, I spent some time yesterday playing with the demo. I’m going to put together a few notes from my experience once I finish going through all the documentation. One thing I thought worth bringing up now though: I was unable to install ADX with node 15.

Looks like the @typescript-eslint/eslint-plugin errors out if you try to use node 15. Is this a known issue? Worth creating a ticket for?

In reply to

arcalinea
I would say you should work out how reputation feeds would work in a generalizable way without going to the effort of building on ADX right now, because it's still an experiment -- everything can and will change. Wouldn't want you to have to rewrite everything later

Greetings Bluesky :-)
Anton Kent, p2p engineer from

I am going to do a review of ADX soon.

p.s.
If you are interested: we are using heavily patched and optimized Go-Threads/Textile + our own CRDTs + p2plib.

(first try reading the spec 😓) Making speech orthogonal to reach is 👍️. There is freedom for indexer to chose content. But is there something for authors to chose/consent to indexers?

Three cases of me wanting to control indexing of my content

• Google using my content as a clue to page-rank the web
• things like ThreadReaderApp unroll (i don't want my content to be printed on a website I don't trust)
• ClearView feeding it's database

Mastodon "solve" that by having search only on hashtags so people can choose which term their content can be found. There is also a #nobot convention (you can add #nobot to your bio and a benevolent bot won't follow you) but it lacks ability to chose which bot I'm ok with or not.

One thing I was still confused on after reading through the docs: Who hosts the DID documents? At first I thought it might be the consortium, but y'all mentioned the consortium potentially only keeping track of the keypair changes of DID documents. Does the consortium host them or do they serve purely as an auditable archive?

I'm assuming understanding this might also require me to understand how the did:web method (and its potential replacement) works. I'm still wrapping my head around DIDs in general.

DID documents that its responsible for

Which DID documents is it responsible for? Any DID document referenced in the did:bluesky method? And following that logic, could apps/servers potentially support other DID methods that define different hosting methods (e.g. ion) and bypass the DID consortium?

One of the things that we learned back in the early days of blogs is that, since blogs were scattered all over the web, it was very hard to discover what blogs existed. The same lesson had been learned for the web itself in earlier days and lead to TBL's list of new websites as well as to the use of Usenet's comp.infosystems.www.announce to announce new websites.

You might have all the crawling and indexing capacity in the world, but if you can't discover Personal Data Servers, you've got nothing to crawl. For blogs, we once attempted to address this problem with FeedMesh,

How will an ADX crawler discover what Personal Data Servers exist?

I see that the architecture says that "Personal Data Servers can actively push updates to Crawling Indexers." So, as with WebSub, etc. a PDS can "announce" its presence. But, while a PDS "can" push updates, is it expected or required that they will do so? Also, if PDS's are notifying specific crawlers of their updates, doesn't that build in a systemic "first-mover" or "dominant provider" advantage since PDS's will tend to only update the most well-known crawler(s), but not new entrants, or more specialized crawlers? To avoid such system bias, and the resulting barrier to entry, I suggest that it would make sense to look at the development of shared receivers for updates that can then redistribute those updates to a variety of crawlers.

In essence, the suggestion here is that the aggregation of raw content streams should be considered as something distinct from crawling or indexing. (The current architecture seems to consider them to be the same thing.) An aggregation server might receive and redistribute updates, or, it might simply notify subscribing crawlers of which PDS's have been updated. An aggregation service might also provide a list of PDS's that exist (e.g. All PDS's that have ever pushed an update, or perhaps just the subset that have updated "recently," etc.).

How will an ADX crawler discover what Personal Data Servers exist?

Bob Wyman I'm guessing the indexers could maybe crawl the consortium for this information (since the DID document also stores all of a user's services)?

I think a robots.txt equivalent wouldn't offer the semantic richness necessary to support the kind of control that n@numero6:codelutin.com requests in his post.

A robots.txt file is limited to saying only whether or not crawling is permitted, either in general or by an identified crawler. However, it seems that n@numero6:codelutin.com 's request might be more accurately described as wanting to also limit the privilege to use the fruits of crawling. (e.g. He might allow Google to crawl and index, but wish to block Google from using the crawled data as "a clue to page-rank the web." Or, he might permit crawling in general, but seek to prohibit the construction and publishing of an "unroll.")

Certainly, one could define a semantically richer "robots.txt" file and make it visible to crawlers, but then you'd have to trust crawlers to respect it -- just as you do with robots.txt. Also, unless that data were carried along with crawled data and passed on to indexers, etc, the use of aggregators or crawlers that served more than one indexer would be a problem since one indexer might do something forbidden but some other one would not. Alternatively, you require that crawlers announce their intended use of data while crawling. Having the PDS evaluate crawlers' statements of intent would allow them to reject crawlers based on their intent. (If you didn't trust crawlers to accurately describe their intent, you could buid a "Crawler Reputation" system that attempted to describe what various crawlers did with what they find.) But, once again, the use of shared crawlers would complicate matters. (You'd still want the "usage restrictions" to be passed on, by a crawler to its indexers, etc. But, then you'd have to trust those downstream indexers to respect the rights limitations.)

It's an interesting problem.

Pinging an indexer, like Google, would only be useful if that indexer was willing to notify potential "competitors" about what pings it had received. Google did, in fact, do this for FeedMesh, back in the early days of the web. So did PubSub and a couple other blog services. However, some blog services felt that they "owned" the data they had discovered and refused to share their work (although they did read from FeedMesh for their own purposes... Grumble...)

If it becomes understood that pinging some specific dominant indexer is the way to be discovered by other less-dominant crawlers, then that makes that indexer an essential and privileged component of the entire system's infrastructure and creates a barrier to entry for anyone else. Hopefully, those offering competitive services will compete based on the quality of the service they provide, not based on unearned privileges due to their merely having been first movers.

This would only work if all DID documents could, in fact, be crawled as a collection. Is it expected that all DID docs would, in fact, be kept in one place? Why? The normal expectation for a DID document is that while it can be dereferenced via the DID, its actual storage location can be just about anywhere. Would ADX want to change this? Also, some DID methods allow for privacy in that they require authentication and authorization in order to retrieve the DID doc. Would that not be permitted for DID docs used with ADX? In any case, if there was a way to discover all DID docs, you'd want to have the ability to do things like fetch only DID docs updated since some date, etc.

A method more in line with the way DIDs normally work would be for someone somewhere, perhaps the consortium, to offer a list of DIDs that identified PDS's. Then, the list might be fetched and each DID dereferenced to discover the location of the PDS. But, that is likely to be a pretty expensive operation -- that's a great deal of DID doc dereferencing.

I thank about that. We could make a "decentralized cryptographicaly-signed proof of consent": the indexer may publish a document explaining is intent (= legal text). User may be asked its consent and a message (cryptographically signed by the user and containing a reference / DID / Hash to the signed "contract") will be stored somewhere (both by the indexer and the user data host). An indexer giving / selling / offering a service / API on a dataset can than proof that consent was obtained (hash merkle tree of all the consent documents as a verifiable small size proof that x thousands of consent have been obtained to build the dataset). Of course, this does not reduce the risk of an indexer obtaining consent to access data and make a secondary opaque non-consented usage. But, on the legal side, a legal entity will be able force an indexer to provide proof of consent on a dataset. Since, the "contract" of consent (signed by the user) is an immutable public document (content-indexed): one can proof that a data have been misused / consent has been given and the indexer activity is legit.

I'm just dropping my brain output. Please ignore if it's stupid / off-topic 😅

in the words of Alan Kay

For important negotiations we don't send telegrams, we send ambassadors. This is what objects are all about, and it continues to be amazing to me that the real necessities and practical necessities are still not at all understood. Bundling an interpreter for messages doesn't prevent the message from being submitted for other possible interpretations, but there simply has to be a process that can extract signal from noise.

schemas and free JSON, or data, are both noise in the dark. It's useless if I make an extension if I also can't send the means to present itself.