Sun, Nov 6, 2022
- whyrusleeping14:07Deciding what that means is worth some thought
- 14:07Probably makes sense to write a few like “RFC” type issues describing things the community could be helpful on
- mikestaub14:50
In reply to this message
That sounds good. There are 600+ people in this room now, perhaps we should do a poll and ask "What level of commitment to contribution are you willing to make per week? ( 40 hours, 20, 10, 3 )" If there are less than 10 devs at this time perhaps this can wait. I just wanted to do a pulse check and see where things are at. - 14:58blackfreethinkers joined the room
- bnewbold15:11re: the earlier pfrazee comments earlier about DMs and private messages, as a thought experiment I would propose that atproto just not support any form of non-public messaging. keep the protocol small and focused. leave private messaging (DM or group) to existing applications/protocols like Signal and Matrix, don't try to compete with them. an application (like a mobile app) could potentially support these other protocols with shared contacts. some glue to bridge DIDs as persistent contacts/identifiers for those other mediums would be helpful
- 15:13I guess i'm not a particularly active user of any modern social media tools, so maybe i'm missing things, but it always feels like the private messaging feature is bolted on and undesirable in many cases. what people do want is to have reasonably strong/safe linking between identities, so you know if you are contacting XYZ on twiter or bsky via email or matrix, you would actually be getting the same person. but that is trivial with a simple link or profile entry.
- 15:15the important privacy properties for private messaging, like minimizing side-channel metadata leakage and deniability (refutability, perfect future secrecy) are kind of hard to get right, I feel like
- bnewbold15:21a docs request (which I get is sort of low priority at this stage as things are still in flux) would be extending the existing identity docs to explicitly talk through how rewriting history would (or would not) work. aka, if a user or PDS starts publishing a new history chain, with a valid signature (verified by current DID doc verification key), are other PDS servers immediately drop all prior histories? how are other servers supposed to verify if a new "HEAD" is the now-canonical HEAD? eg, if there are multiple branching HEADs. the kind of thing i'm thinking about is replicating from multiple third parties (maybe aggregators, or caches, or "high-availablilty persistence services" like P's old hashbase for dat) and getting conflicting responses. or receiving a random repo PUT request which is valid but maybe out of date. should receiving PDS servers always do a quick sync root request to the DID's canonical PDS?
- 15:22^ this is ecosystem-impactful
- 15:27somebody above was discussing trying to block indexers/aggregators. my current feeling is that a distinction between atproto and a large subset of the mastadon community, is that atproto is welcoming of "big world" aggregation (in addition to "small world" hosting stuff), while in contrast some think that "big world" stuff is potentially undesirable on principle, and are skeptical of random aggregation. eg, I think of atproto as being closer to a service like tumblr, which as I remember/understand was totally open to being crawled and indexed by things like Google, as compared to Twitter which was originally open but ended up having very restricted ($$$ or grandfathered) access to the firehose, lots of expectations about not storing user content that may have been deleted, etc. I think there are legitimate concerns and arguments in both directions; the way I have been thinking of atproto is in the "sure anybody can aggregate, user content is public and users need to keep that in mind" direction
- 15:30
some tangential references to the indexing/crawling issue are:
- this drew devault post about bot load on sourcehut (sr.ht), a git repo hosting service: https://drewdevault.com/2022/05/25/Google-has-been-DDoSing-sourcehut.html
- the fine non-profit work that https://www.softwareheritage.org/ has been doing crawling and preserving pretty much "all" git repositories they can find (eg, large hosting platforms, personal repos, as well as things like converting older non-git repos to git format)
- the old technorati service which crawled and indexed blogs specifically (as opposed to general web content): https://en.wikipedia.org/wiki/Technorati
- this drew devault post about bot load on sourcehut (
- 15:34auraelion joined the room
- 15:47@cmoog:matrix.org joined the room
- 16:00Charlie Moog changed their display name to cmoog
- @wclayferguson:matrix.org16:12bnewbold: I agree, the "Minimum Viable Protocol" should definitely not include support for private and secure encrypted/secret messages, in version 1.0, because that adds an order of magnitude to complexity, and what people around the world were kinda hoping for is a simple protocol for sending messages, that's somehow or other superior to ActivityPub.(edited)
- 16:15It needs to be simple enough that a developer can understand it after 30min of reading. Not 30 hours.(edited)
- @wclayferguson:matrix.org16:20IMO, all that was really needed, from my perspective, was an ActivityPub-variant that used a PublicKey as identity rather than a DNS-based name.
- @planetoryd:matrix.org
- bnewbold16:56
In reply to this message
The way I think about it is that atproto talks about the canonical representation of user content, and how to replicate it, like git repositories (any checkout is as good as any other; there is some awkwardness about "completeness" with large blobs/media and sub-repos). While ActivityPub is about notifications and feeds, like RSS (original content is often simplified or truncated in RSS feeds, though sometimes it does not matter).
DID and account migration are another big difference, but feels more up in the air with the DID "placeholder" mechanism not being a robust proven tech. did:web is a good backup, any extremely trivial (I guess a good thing?), but does not deliver on the interesting new account migration stuff (just falls back to DNS)(edited) - whyrusleeping17:02Adding new did methods will definitely happen at some point, PLC is just the only supported method for now to make all the development smoother
- @wclayferguson:matrix.org
- whyrusleeping17:08Im not really interested in having more activitypub discussions tbh
- 17:08Weve had those
- 17:08weve moved on
- @wclayferguson:matrix.org17:09Also having names built into URLs it makes it hard for someone who doesn't know how to setup a DNS name (SSH) and run their own complete server is a problem. So using a PublicKey like ATP is doing is good.
- whyrusleeping17:10Yeah, usernames/handles need to be extremely simple for people to use and setup
- bnewbold17:10
In reply to this message
huh, I haven't seen much interesting atproto/ActivityPub discussion here or elsewhere, just a lot of noise. but if it isn't wanted here will take it elsewhere! - @wclayferguson:matrix.org17:10I'd like to see something where a non technical person (my mom even) can just run a piece of software and start participating in social media conversations. Personally that's my ideal scenario.
- George Antoniadis17:10I can see a bot in this channel's future that auto replies to anything with "activity pub" with a link to the faq xD https://atproto.com/guides/faq#why-not-use-activitypub
- whyrusleeping
- 17:11If a make a broad statement like that i don’t really mean to call anyone out
- @wclayferguson:matrix.org17:12By pointing out the flaws in AP, that people don't like about it, that will help make ATP at least be aware of what users need and want.
- whyrusleeping17:14Yeah, i want to find the right balance
- 17:15Too much AP comparison feels weird, none also feels weird
- 17:17On the private messages discussion, i tend to agree that it should be a separate protocol
- 17:18The way to do identity should allow people to build messaging applications that integrate with ATP identities
- 17:18But we shouldnt focus on it up front
- 17:18We’re about the public conversation afterall
- Aaron Goldman17:21
In reply to this message
#general-bsky:matrix.orgfor dSocialCommons is meant as lobby for many approaches to compare and hopefully copy each other as distinct from the Dev channel that is focused on building this project. - bnewbold17:23here is a nitty-gritty question to try and keep the conversation back at the "dev" layer:
why are thepublicKeyMultibase
keys in did:plc docs so much longer than the did:key-encoded keys? I suspect this is a DID thing not specifically an atproto or did:plc thing, but nothing obvious turned up, I think they are both supposed to be base58btc encoded
eg, looking at: https://atproto.com/specs/did-plc#example - mikestaub17:23
In reply to this message
for PMs I don't think it makes sense to invent something new but rather use signal or peergos - @wclayferguson:matrix.org17:30I say there's huge benifit to also focusing on "Public Append-Only" content for the first MVP. Merkle trees make the syncing perfect and simple. No one knows that around here better than whyrusleeping because of his IPFS expertise! (oh, and me too). Once you start trying to do editable data a vast array of difficult problems come up instantly and multiply complexity 100x.(edited)
- 17:32And if it's done with PublicKey identities you can tell other "unnamed protocols" they're just obsolete now. hahah. :)
- 17:34one final point, it can be the simplicity that makes ATP "better". Be simpler than Matrix for example. Simple is indeed a feature in itself.
- Aaron Goldman
- 17:38`did:key:zDnaeSezF2TgCD71b5DiiFyhHQwKAfsBVqTTHRMvP597Z5Ztn` Is an entire key and is better compared to the object rather than the key bytes. ``` { id: 'did:plc:7iza6de2dwap2sbkpav7c6c6#signingKey', type: 'EcdsaSecp256r1VerificationKey2019', controller: 'did:plc:7iza6de2dwap2sbkpav7c6c6', publicKeyMultibase: 'zSSa7w8s5aApu6td45gWTAAFkqCnaWY6ZsJ8DpyzDdYmVy4fARKqbn5F1UYBUMeVvYTBsoSoLvZnPdjd3pVHbmAHP' }, ```
- 17:39Definitely get the points for a technical question for that one 😁
- whyrusleeping17:56
In reply to this message
Yeah, we need something thats easy to integrate, works with our identity system, and that is scalable to the user load we are expecting to see - 17:58
In reply to this message
Yeah, we've realllly been trying to keep things as simple as we can make them. A lot of time was spent trying to toe the line between 'as simple as possible' and 'not being obsolete in the next year' - 17:58trying to maximize the pair of (simplicity, flexibility)
- 17:59onlytangerine joined the room
- whyrusleeping18:00oonlytangerine: why not grapefruit or kumquats?
- Dean18:42
In reply to this message
Well it's working so far. I've been able to use the provided APIs fine with the dev-env server! - 18:43It's actually super fun to work with the protocol and not have to worry about the back-end.
- 18:45jmcasey joined the room
- jmcasey18:45Did the createRecord parameters change? Getting "{"message":"Unknown parameter: did"}" and can't see where in the code it's doing that check
- Steven Franssen
- Aaron Goldman19:01"Append only" is just one of the persistent data structures.
- @wclayferguson:matrix.org19:04
In reply to this message
It could just wrap each "month's" posts into a separate ZIP/CAR file and each one can point back to the merkel root of the previous. Just like how Blockchain works and is trustworthy even if you're not running a full node. - 19:05Everything in the world that's got large data usually has some kinda paging. Letting months or weeks be the paging is super clean and simple.
- 19:06especially since this is definitely a time-series of data if it's append only.
- Aaron Goldman20:20
In reply to this message
The problem with an append only block chain is that purge requests become difficult to honor. - 20:22The tree is better if you may want to remove something much later you just recalculate the path from root to leaf instead of all subsequent pages.
- 20:26At least in my mind this is the main win from separating the DID Document deltas from the repo. You need the full history of the DID Document to validate the current state. You don't need the history of the repo just the latest commit and tick. Then it's a Merkel tree of the current state.
- mikuhl20:57I am going to spontaneously combust because I am so excited to use this protocol!!
- Aaron Goldman
- mikuhl21:01lol
- @wclayferguson:matrix.org21:35
In reply to this message
Two things:
- you jumped to "mutable data" when I was talkin' about immutable. :P
- but if we look at the mutable case. The master "record" be a single list of all the root merkels for every month of a person's life. What that means is each "rolled up" monthly chunk of data can be updated and it simply results in a new root merkle hash for that one month ZIP/CAR file.
The benefit if the "by month" approach is that when I want to follow someone, I don't get ALL their data unless I ask for it...but I do always get that "master record" which is essentially a list of monthly hashes.
- 21:35One merkel tree per month.
- 21:36the only thing that makes this stuff "big" really is the images/attachments. You could probably store someone's full social media history of typed text in a single ZIP and not even break it up.
- 21:37IF these decisions have already been finalized let me know and I'll shut up. :)
- @wclayferguson:matrix.org22:00BTW. mutable data doesn't work well with Social Media anyway. Somebody can say "Do you like puppies" and 100 people reply yes, then they go back and edit it to ask "Do you kick puppies". It just doesn't work for pure Social Media use cases.(edited)
- 22:01You really just need a permanent record of who said what and when to have conversations where people aren't driven insane by "revisions of history"
- 22:02Even if no one has replied to a post yet people can share links to it and say "Look at this thing I agree with"....and then somebody changes said thing to some form of evil. It just cannot work.
- 22:03Sorry to flood the room with text. I'm done I hope. :)
- whyrusleeping22:09Bruh
- mikuhl22:18now that was a TED talk :)
- @wclayferguson:matrix.org22:27I didn't wanna get too persnickety about how things need to be done....but I failed.
- whyrusleeping22:28Okay, ill read the text wall, hang on
- @wclayferguson:matrix.org22:29It's about puppies. :) TL;DR....the issue about the "time window" in which editing should be allowed.(edited)
- whyrusleeping22:31So, in terms of “things that have been decided”, posts are all in a big merkletree keyed by their “TID” which is basically just a timestamp. This makes it really easy to filter the tree by a specific time range
- 22:31How blobs get stored and linked in isn’t necessarily figured out yet, its pretty tricky tbh
- @wclayferguson:matrix.org22:31gotcha. thanks.
- whyrusleeping22:32Posts are inevitably mutable, but replies (IIUC) reference both TID and CID, which means you can detect if a post has changed, and application logic can figure out what to do
- @wclayferguson:matrix.org22:33so anyway if the content changes the CID changes too. so...there's that. In a way it means everything is immutable only in the sense that the CID chagnes if an edit happens.
- whyrusleeping22:33Yeah, just confirmed, replies reference both cid and a “mutable reference “
- @wclayferguson:matrix.org22:34ah, ok, that extra level of 'indirection' is allowing the edit then. got it.
- 22:34reference==indirection
- whyrusleeping22:34Yeah, allows it but makes it detectable
- @wclayferguson:matrix.org22:35that's how my own platform works too. If you edit then the digital signature changes. But my platform is more of a CMS than Social Media. In social media really the simplest challenge to solve in MVP is the "Immutable Public Data" case.
- 22:36What i meant was I have a "permanent" record ID that isn't a hash.
- 22:36but that is almost like a wrench in the gears of Social Media if people can "revise history" in the middle of a discussion for example.
- 22:38(in other words the "puppy problem"....of revising history)
- whyrusleeping22:39Only if the application doesnt do anything about it
- @wclayferguson:matrix.org22:40I guess the issue is once something is posted it has a URL, that can be shared, and people can say "look I agree with ABC", and then ABC changes.
- whyrusleeping22:40Same problem as replies really
- @wclayferguson:matrix.org22:40I haven't "Solved it" in my own app either. I'm just saying it's a justification for immutability.
- 22:41But yes, it's same exact problem as replies.
- whyrusleeping22:41Urls could contain both references too, and the application can display a warning
- @wclayferguson:matrix.org22:42yep.
- 22:42or just require a delete an repost new message.
- 22:42but that's same as 'immutable' so....i retract that. lol.(edited)
- 22:43anyway thanks for listening. just wanted to raise that issue.
- whyrusleeping22:48Yeah, its really important, and one of the first things i was thinking about when proposing the repo architecture
- 23:43gonzo_marx joined the room
- Steven Franssen
Mon, Nov 7, 2022
- @wclayferguson:matrix.org01:18In my mind I'd say an append-only edit is not a mutation. correct. You're not editing your original words (which could the change meaning even if you only change one letter of the text) It's impossible for software to know if you're just correcting a typo or possibly intentionally 'inverting' the meaning in some clever way.(edited)
- 03:03valere joined the room
- 03:25plywooder joined the room
- 03:37@kf.walkow:converser.eu left the room
- 06:19@jaklt:tchncs.de joined the room
- 07:17mimonelu joined the room
- 09:13Jorge Eduardo da Silva Figueiredo joined the room
- Jorge Eduardo da Silva Figueiredo09:14Olá
- Aaron Goldman10:06The question of wether the repo is mutable or immutable comes down to the reference URI not the repo itself. Let's take git as an example: The git hub url `https://github.com/bluesky-social/atproto/blob/d44cee71139af2267996f8b4c9412cde099ea37b/README.md` contains the hash of the commit. If I want to validate the blob at that url I can pull the Commit, root tree object, and the blob. The sha1s prove the contents of the blob are the blob referenced in the URL. Even if GitHub disappeared tomorrow If I have the URL and the repo I can still prove this. On the other hand the URL `https://github.com/bluesky-social/atproto/blob/main/README.md` references `main` this is mutatable as it is referencing whatevergithub.combelieves is the head of main. If GitHub disappeared tomorrow this URL is meaningless or at best means anything that was ever served from it. So is a git repo mutable or immutable? If we were designing it today which type of reference should we use? To get from mutable to immutable We need to know the commit hash. To get from immutable to mutable We need to know the branch name. Both references feel somehow incomplete. The immutable reference doesn't tell me a authority to get the latest value and the mutable reference doesn't give me a cryptographic hash to get the bytes as they existed at the time of the reference. `at://repo_did/collection/record_tid` doesn't give you a hash for the commit. `at://commit_hash/collection/record_tid` doesn't give you the DID to find the latest version of the repo. (Note: Not the actual ATP URL just making a point. Still well defined URL parsing since https://username@host/path is a valid url) `at://repo_did@commit_hash/collection/record_tid` would give you both. It is an immutable reference because you can use the commit hash to know exactly what the bytes were but it is also a mutable reference since you have the DID to find the latest version of that record.(edited)
- Dean10:09Something I'd like to see is native support for horizontal scaling on ATP indexer servers considering these are most likely to draw the majority of bandwidth. It would help when there's a sudden influx of users to a specific indexer/name provider. I'm sure this is already in consideration, I'm just writing down thoughts as I'm coding with the current 0.0.1 protocol.
- Aaron Goldman10:10Not pick: slightly more accurate to refer to hash identified content as semi-immutable having only two states the content and don't know. The repo model lets you redact data from the repo by deleting the blob without invalidating the rest of the content proofs. It's just that that one record @ that commit is now "don't know"
- 10:11marios joined the room
- @wclayferguson:matrix.org10:24Aaron Goldman: I think you're right. My statement of immutable v.s. mutable was highly specific. The terminology fails us in this case I think, too. The problem to be solved is the "puppy problem" where somebody says, do you "like" puppies, gets a bunch of replies, and then edits it to say "kick" instead of "like". I guess the replies need to point to the original post and not be carried forward to the "edited" post?(edited)
- 10:26I think whyrusleeping is aware and has is solved probably. Thanks for clarifying so thoroughly..(edited)
- Aaron Goldman10:50at://repo_did@commit_hash/collection/record_tid at://handle@commit_hash/collection/record_tid Is slightly different from the current implementation which has the URL and CID are different fields but yes including the CID is good for preventing the puppy attacks you describe. Not yet clear how we should get both the repo ID and commit CID into the same URI.(edited)
- 10:54@lisko:matrix.org left the room
- @wclayferguson:matrix.org10:55perhaps leaving out the 'commit_hash' can make it pull up the 'latest commit'?
- 10:56(not saying it solves 'puppy problem' tho)
- 10:56BUT including the commit_hash would basically "solve it", by being commit specific.
- 10:58
another approach possibility:
Every node has a "type"
- "Type=Comment" nodes are be immutable. You cannot change a comment. Required to delete and repost new one (resulting in new record ID).
- "Type=Paragraph" (like some chunk of "long form" content/document), should be mutable, because it's like a blog, or document.
- bnewbold12:43
In reply to this message
yeah, it would be nice to have a way to encode the commit CID in at:// URIs. or, just add as a field in references, along with the URI and the content CID.
otherwise the CID reference in mentions/links are a bit "deeplink" and require a big index of record-CID to commit for verification. AKA, it is the commit that has a signature and can be validated.
not a huge issue, there is a way to validate with the way things are today, just takes a bit of extra work - 12:45in implementing a PDS, i'm leaning towards retaining all the commits and MST nodes Forever, immutably. record contents are what could get purged/redacted/hidden in response to abuse or legal issues. that way things like references can be confirmed to have actually linked to an actual thing, even if the thing isn't returned
- Daniel Holmgren13:41hey just saw this one: the reason is that did:key is encoded with the compressed 33byte public key and publicKeyMultibase is meant to be the uncompressed 65 byte public key
- 13:44ope aaron actually got to it already 😛
- bnewbold13:48I'm curious what the "scene" vs. "user" repo types are. "scene" is like... a group?
- pfrazee13:53we're leaking alpha!
- 13:54scenes are like a group, yeah
- George Antoniadis
- Daniel Holmgren13:55but better 😜
- pfrazee
- George Antoniadis
- pfrazee13:56hah oh to be clear, "leaking alpha" means revealing details of the unfinished app
- George Antoniadis
- pfrazee13:57;)
- 13:58I'm working on the onboarding copy now...
- 13:58the details have been blurred to protect the ~innocent~ alpha
- 13:59not exactly easy to surprise people with features when our backend is foss
- 13:59but a couple technical details of those recent updates, since I'm here!
- 14:00an "account" -- as in, some entity in the network with a repo and an identity -- is now generically referred to as an actor
- 14:00actors are typed. "User" and "Scene" are the two kinds of actors we have atm, but you can bet that'll grow
- 14:01this means that scenes behave much in the same way as regular users. They have their own handle, their own profile page, their own feed, etc
- 14:01the actor type gets declared in the
app.bsky.system.declaration
record - 14:02you'll notice
declarationCid
variously in the APIs, eg on follows. This is the CID of that declaration record. It's so software can detect if the actor type changes -- so if you follow a scene, it doesnt become a user in a surprising way - 14:02the declaration is meant to be immutable but there's no way to enforce that other than this kind of referential integrity check (the declarationCid)
- 14:04you may also notice two new record types,
app.bsky.graph.assertion
andapp.bsky.graph.confirmation
. This is a generic utility for asserting social-graph relationships bidirectionally - 14:05we'll use this in a number of things, but scenes are membership-based and require both sides to agree with the membership. (And either can revoke it)
- 14:05so in essence the scene publishes an assertion of membership, and the user publishes a confirmation of that assertion
- 14:06so the "follow" record type represents a uni-directional graph relationship, while the "assertion/confirmation" types represent bi-directional graph relationships
- 14:06if you like this, it was all my idea. If it seems too complex, this was all Daniel Holmgren's idea
- 14:07(jk obviously, I'm pretty confident this will work well and have a lot of utility)
- 14:08this all represents some of the interesting, more sophisticated data modeling you have to do with independent actors publishing
- 14:08the CID usage for referential integrity == "how do we handle information reliability in a distributed setting"
- 14:09the bidirectional record linkages == "how do we coordinate action between actors in a distributed setting"
- 14:09it's a different mindset than centralized services to be sure
- 14:10iafarouk joined the room
- George Antoniadis14:12Are users and scenes different types? I can see
app.bsky.system.actorScene
but notapp.bsky.system.actorUser
.
If different, do the schemas support some way of either inheritance for relationships (ie you can "like" all "actor" types) or maybe just allow for unions (ie "actorUser" or "actorScene") - pfrazee14:13they're different "actor types." We dont have formal inheritance in lexicon, but that's the implied relationship
- 14:14the way to think about ATP's records is more akin to a database, and lexicon is providing the schemas for the database. We don't formalize OO-style inheritance in lexicon for the same reason, say, SQL doesnt
- 14:14that said, lexicon is also used to generate code/apis, and you can produce APIs which represent the implied concepts in the way you're describing
- @wclayferguson:matrix.org14:29Makes sense that a User and a Scene are kind of both at the top level (root) of a sort of hierarchy, that can contain all of the details inside it. Sounds good. Thanks for the update!(edited)
- 14:39justthisguyatx joined the room
- Alejandro_14:54I'm curious about how a closed beta can work for a decentralized, open-source thing? Will there be only one instance for testing? will a limited amount of people be able to create an instance and make a federation?
- pfrazee14:55yeah it's a good question. It's specifically: access to our mobile app, and access to our instance
- 14:56and while a lot of the foundations for federation are done (see above for all the ways we model the data in preparation for it) the actual server-to-server sync protocols haven't been implemented yet
- 14:57we know exactly how the sync protocols will work, but saving it for the last task before the open beta allows us to understand all of our requirements before implementing it
- 14:59I really enjoy building fully in the open and normally would -- I live streamed my last project's development and I'm kind of dying to do that here -- but bluesky is in a kind of weird position where there's a lot of press attention, and we really didn't want "bluesky unveils a hot box of garbage" headlines
- pfrazee15:00plus it's really hard to communicate with a community while primary development is happening (notice some of the enormous change in recent PRs) and this is meant to avoid miscommunicating how stable things are
- Alejandro_
- pfrazee
- George Antoniadis15:02
In reply to this message
lurked mostly, loved having them on while working, they were really good fun. - pfrazee15:02nice. Those were good times
- 15:02I'm practically begging jay to let me livestream again so ... hopefully that can happen
- George Antoniadis15:04Once the cat's out of the bag I guess it might be easier to find some time to work on secondary projects around bsky. Or is the bsky app going to be open source as well?
- pfrazee15:12we're definitely going to FOSS a reference client and all the tooling we use. Not sure yet whether every piece of the client will be though
- Dean15:41
In reply to this message
Is the current spec (as described in the docs) stable enough to work on front-ends and not expect a complete rewrite from the existing ATP protocol spec? I noticed some things already differ (e.g. session helpers and did/cid arguments for lookups) but the method calls are the same. - pfrazee15:43the core concepts are pretty darn stable, but I imagine building a frontend right now would suck
- 15:44for instance, last week we changed the NSIDs of every schema, which would break everything
- Dean15:56
In reply to this message
I started playing around with it this weekend and the experience has been great so far! - Aaron Goldman15:56
In reply to this message
for
at://repo_did@commit_hash/collection/record_tid
orat://handle@commit_hash/collection/record_tid
the "big index of record-CID to commit" is not like searching IPFS the RepoID (DID or handle) is also there so you can look up the home server and then pass the url with the commit_hash to it.
The home server is the one that keeps the old commits, path nodes, and records to return for a uri request. Or it can respond that this record has been deleted. Probably we would want it to return not only the exact record but the current record for the same path in the latest repo.This also serves as a guard agents the handle being reassigned by the DNS system. If the commit hash matches then it is the content that was the handle at the time the link was generated. Irrespective of wether the publisher later decided to stop paying their DNS name bill.
- 15:59Dean changed their profile picture
- 16:14atnorl changed their display name to xonampo
- 16:14xonampo changed their display name to atnorl
- whyrusleeping16:29
Fun things you can do already, once you export your repo to a car file (using
com.atproto.sync.getRepo
) you can import that into ipfs and explore the data:123456789
why@sirius ~/c/gosky (master)> ipfs dag import repo.car Pinned root bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe success why@sirius ~/c/gosky (master)> ipfs dag get bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe {"root":{"/":"bafyreifslfzyxiaqeaeeoydnb7fwmxq7cwjqrqtbvjw2rbq7ibarnl4fui"},"sig":{"/":{"bytes":"aaRPVJa/QRbhCOuUTJS8M7x/KEgpBW25HHfLVZvB9bTqrfDNMuESuVdxdT1SbV0p5FfVFpYLsL5KyXejjTP2dA"}}} why@sirius ~/c/gosky (master)> ipfs dag get bafyreifslfzyxiaqeaeeoydnb7fwmxq7cwjqrqtbvjw2rbq7ibarnl4fui {"auth_token":null,"data":{"/":"bafyreife3hhlk6ukkpjjvp3fmn4eocqfczlf472kgjgzfygz3bjsw43l3u"},"meta":{"/":"bafyreidevl4oqvw4gfoxnyctsr373zi7dnnj4pjkbh4nclubkqaf37vtvq"},"prev":{"/":"bafyreigc5zp7rzfuuxoylmbsdj4meog6esxuqgpehty6g466f2flxnj4ra"}}⏎ why@sirius ~/c/gosky (master)> ipfs dag get bafyreife3hhlk6ukkpjjvp3fmn4eocqfczlf472kgjgzfygz3bjsw43l3u {"e":[{"k":"app.bsky.feed.post/3jgwsf5q64c2z","p":0,"t":null,"v":{"/":"bafyreig4s2bn7yjo2i2r2gut7uvxvvcgsyf4jcyndcbnjdm5jkjx35o6yu"}},{"k":"tjxhkc2z","p":24,"t":null,"v":{"/":"bafyreiduauw76xk4gz3srdhdnkwzmwuisviqp72nyia3crklgxwhddv7ga"}},{"k":"system.declaration/self","p":9,"t":null,"v":{"/":"bafyreid27zk7lbis4zw5fz4podbvbs4fc5ivwji3dmrwa6zggnj4bnd57u"}}],"l":null} why@sirius ~/c/gosky (master)> ipfs dag get bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe/root/data/e/0/v {"$type":"app.bsky.feed.post","createdAt":"2022-11-07T11:04:03-08:00","text":"i like cats and dogs too"}
- 16:50Scott Edwards joined the room
- 16:57sables joined the room
- Aaron Goldman17:19Message deleted
- Aaron Goldman17:19it is a UX chalenge that there are three ways to follow
- 17:20do you want what was linked to
- 17:20latest from that publisher
- 17:20or the latest with that name
- 17:21If they all match fine but if that don't it might make sence to warn the user that the handle changed control between the time the link was created and now
- Aaron Goldman
- @wclayferguson:matrix.org19:59that was too hard for meh. :)
- 19:59I just want to "click" reply and it does whatever. haha.
- 20:00not that I know what "whatever" means.
- 20:00lolz.
- Aaron Goldman20:33The person clicking reply should not need to know any of this
- 20:37Bruno Hahn Gonçalves joined the room
- 21:43jinglescode joined the room
Tue, Nov 8, 2022
- 01:58Ömer Aykol joined the room
- 02:25boscovn joined the room
- Anders C. Sørby04:30I want to create a rust server node which is both compatible with ActivityPub and ATproto. It will combine Web3 stuff with the existing fediverse. It is not a bridge but a node. Using nixos to make it easy to deploy. IPFS and Filecoin support. GraphQL API. Support for wallet logins. Calling it rustyverse now. https://github.com/Anderssorby/rustyverse
- Anders C. Sørby04:36Potentially compilable to WASM
- 04:36Intends to usesurrealdb.com
- 05:14@bradstewart:matrix.org left the room
- 05:16Eduards Puķins-Slava joined the room
- 06:39@rimuru:gentoo.chat changed their profile picture
- 07:37Hacklet joined the room
- 07:41Steve joined the room
- 08:01@neilalexander:matrix.org joined the room
- mikestaub09:46pfrazee: what tool will be used to manage the schema lifecycles? I think something like https://github.com/odpf/stencil will be needed
- 10:02hacklet changed their display name to Hacklet
- 10:02Hacklet set a profile picture
- 10:23Børlaag changed their profile picture
- 11:05William Bernardi joined the room
- pfrazee11:15stencil is interesting. I'd like it to be a self hosted protocol (meaning you could ask
https://sync.atproto.com
for thecom.atproto.sync.getRepo
definition) - 11:15then that opens the door to somebody running a registry service
- 11:20Jackson White joined the room
- whyrusleeping11:27Yeah, some sort of “well known” convention for storing the schemas
- 11:27Versioning will be interesting too
- pfrazee11:27I was thinking we'd bake some methods into the xrpc spec for it
- whyrusleeping11:28So meta
- Jackson White11:28beta still open?
- pfrazee11:28versioning is effing hard. After spending a lot of time on it, I specced a
"revision"
field on lexicon which is a simple number. It's meant to only be used for changes that are forwards AND backwards compatible - George Antoniadis11:28schema versioning on decentralised protocols is the stuff of nightmares
- pfrazee11:28
In reply to this message
signups at https://bsky.app, we havent started the closed beta yet but we're v close - Jackson White11:29ok gotcha
- pfrazee11:29
In reply to this message
it is. Continuing from before, the revisions don't allow significant changes to the schema. My thinking is, if you need to make forwards OR backwards breaking changes, you create a new schema and stick a v2 on it - 11:30in essence, once a schema is out there, the only thing you can do is add optional new constraints
- 11:30I wrote up some stuff about this and will talk more about it in the future, but yeah in a decentralized setting you are very constrained on what changes you make to a schema after you publish it
- bnewbold11:31I feel like inevitably there will be a bunch of schemas like
app.bsky.v2.feed.post
(or some variation of inserting the "v2"). the old golang namespace situation comes to mind - pfrazee11:31almost certainly yeah
- 11:32cruft is guaranteed in this line of work
- whyrusleeping11:33Welcome to the jungle
- Jackson White11:33interested to see how schemas r handled
- pfrazee
- Jackson White11:35versioning*
- pfrazee11:35ah yeah. Well I pretty much described the game plan above
- pfrazee11:55
In reply to this message
No. And -- this is confusing but -- we don't use IPFS technically, we use IPLD. So by default, nothing in ATP is accessible via IPFS, and we have no plans to make that part of the spec - 11:56we looked at using CIDs to identify schemas but felt like that was more rigid than we need, and it suffers from the downside of having to reference the types by huge opaque numbers
- 12:01@julian:one.ems.host joined the room
- George Antoniadis12:04Pretty sure this has been answered before but since it's not in the FAQ (and can't find it using search here) I'll ask again :D How come you didn't go for a full URI instead of NSIDs? Since you're sticking to HTTP for the foreseeable future (I assume?) you might as well.
- pfrazee12:06you can think of ATP as an addressable space that exists atop HTTP. It's an abstraction over it, in the same way TCP is abstracted by HTTP1 & 2
- 12:07HTTP could be swapped out, much like how HTTP swapped out TCP in recent versions
- 12:07critically, the
at://
address space identifies data independently of the host machine - @planetoryd:matrix.org12:09immutable storage ? or just like ipns
- pfrazee12:10the best analogy for data repos is git repos
- @planetoryd:matrix.org12:11whats the difference compared to locutus ?
- pfrazee12:11I dont know locutus
- @planetoryd:matrix.org12:11locutus is dynamic but the state is bytes
- whyrusleeping12:18its all built on freenet?
- @planetoryd:matrix.org
- whyrusleeping12:19sure but its basically still freenet, which is a not-great protocol base for a number of reasons
- @planetoryd:matrix.org12:19not the same thing at all
- 12:19it doesn't have the anonymity routing, or deniability kind of thing
- whyrusleeping12:20but it still has the same basic trust assumptions that puts a ton of work on people running nodes in the network
- @planetoryd:matrix.org12:21just DHT with arbitrary function of verifying the values (for ipfs the blocks are supposed to match the CID
- whyrusleeping12:21very vulnerable to churn causing huge disruptions in the QoS
- 12:22freenets big problem was that it relied on a bunch of altruistic people hosting content for others, with no clear incentive mechanism, and no way of guaranteeing a particular level of service
- George Antoniadis12:22p@planetoryd:matrix.org: is there a specification for Locutus? Docs are not really doing a good job at explaining how it works, and there are a lot of references of "contracts" which is not promising. ps. Should we move this to #general-bsky:matrix.org ?
- whyrusleeping12:22locutus' docs are extremely light, but it seems to say "use a DHT-like thing to store key-value pairs, and have some web-assembly code to do... something"
- 12:23and yeah, best to move this to general
- 13:14manish Wadhwani joined the room
- jmcasey14:22I'm working on a rust crate for ATP: https://gitlab.com/jmcasey/rust-atp/ Any feedback/advice appreciated
- pfrazee14:24
In reply to this message
nice. I'd suggest starting with implementing the repo data structure, which is a combination of IPLD and a Merkle Search Tree implementation. That's relatively stable and is going to be core to any implementation - jmcasey14:25Yup, that's the next big step. Right now it's just making XRPC calls so I can get a sense of what types we're passing around feel like
- 14:26IPLD/CBOR/MST needed before it can be used in a PDS
- pfrazee14:26cool. whyrusleeping has been working on a golang library and generating APIs from the lexicons, much like we do for typescript. Consequently, he's been giving us good feedback on how we can shape the schemas to make that easier in a more strictly typed language
- jmcasey14:27Awesome. Will take a look, I'm new to rust so I'm sure I have some naive approaches
- pfrazee14:28this will be a fast way to learn! I've never coded in it either. I'm turning into a typescript-only dinosaur lately
- jmcasey14:29I know it's a meme, but so far I've really enjoyed it.
- 14:29The compiler is a character in the show
- pfrazee14:30we'll see if I ever get a chance to build in it. My 5 year is probably going to take me away from code so I may never get the chance
- 15:21simonearle joined the room
- 15:41Nad changed their display name to Nad // away
- Dean17:01It looks like bsky's
getProfile()
method requires authentication currently. Wouldn't it make more sense to have that as a public method so profiles are publicly viewable? Or will all bsky related methods always require authentication? - @wclayferguson:matrix.org17:05
In reply to this message
I'll go a step further and say all public data should be able to be consumed anonymously. - bnewbold17:25
In reply to this message
I have a partially working Rust PDS and CLI client implementation at https://gitlab.com/bnewbold/adenosine - 17:28the MST and generic XRPC stuff is mostly working, and i'm part of the way through app.bsky feature stuff. I'm not doing real validation against the Lexicons, or code-gen. and the whole thing could use refactoring/iteration. but close to working as a proof of concept. I expect the specs to continue to change at more than the Lexicon level, so I would be hesitant to invest much time in to a polished implementation. eg, how inter-PDS communication will work feels unspecified so far, and could have a large impact on how to architect things
- bnewbold
- 17:37jetpack_joe joined the room
- bnewbold
- 17:43@wclayferguson:matrix.org left the room
- MightySpaceman (OLD -> m_spaceman:matrix.org)18:03:'(
- jmcasey
- pfrazee19:08
In reply to this message
we've got an open issue to clean up the renaming user and name items in there. I think the current approach for when either is allowed is to use "actor" - Dean19:50Note regarding atproto's api package, it currently does not work natively with vite SSR apps e.g. built with sveltekit when loaded server-side. You'll get the error
exports is not defined
atajv-formats/dist/index.js:4:23
. Guessing this has to do with ajv-formats being CommonJS. The api can only be imported in the client-side when doing it likeconst api = await import("@atproto/api");
in the browser side (i.e. inside anonMount()
), which adds a big initial page load time since it's loaded into the browser session separately rather than being a part of the vite-generated js chunks. - 20:35Børlaag changed their profile picture
- wolix20:40Hello there iam have question bluesky be like Twitter only differences be similar looks
- 20:40Be interface be different
Wed, Nov 9, 2022
- 00:51i99988 joined the room
- i9998801:03Hope better than all social media apps
- Steven Franssen
- 01:33@xentra:foss.wtf joined the room
- 02:24Freezlex joined the room
- 03:15-THMS- the Founder joined the room
- 03:54@tomtau.:matrix.org joined the room
- @tomtau.:matrix.org04:00Hi everyone! I skimmed through the AT protocol spec, and one question I had was "How is this different from the Solid protocol?" (which, to my understanding, is a lot about account/user data portability at its core)... perhaps something good to add to FAQ: https://atproto.com/guides/faq
- 07:27Prashant Mittal joined the room
- Aaron Goldman07:52
In reply to this message
bnewbold: I had been using
publisher
ashandle
orDID
Mostly just trying to get away from
host
.
Theorigin
for https isprotocol
, hostname, and port. The origin for ATP is protocol and publisher. We don't care what host gave it to you just if the signature is valid.Actor is pretty generic and has some overlap with the Erlang model but I don't know as long as we pick something and use it constantly probably fine. As long as we are clear what the origin is for the same origin policy
¯_(ツ)_/¯
url = new URL('http://example:8080/path') url.hostname // "example" url.host // "example:8080"
- Prashant Mittal09:07Hi everyone, really new here. I have been doing a deep dive into ATProtocol and have been reading the codebase for past week or so. and I have a few questions. I would be really greatful if you can help me out. Observations: 1. It looks like PDS is instantiated with a keypair which is used to sign all the data being stored 2. a jwt token is assigned to every account created on a PDS Question: 1. Does this mean there can be only one user per PDS 2. Can the account change the signing key for data that they are generating and not use the PDS's key Thanks in advance. Also, if you can guide me to a proper channel(if there is any) to ask general queries arounf the protocol and it's working
- Dean09:17
In reply to this message
I think this answers question 2: https://atproto.com/specs/did-plc#account-recovery - pfrazee
- pfrazee09:53
In reply to this message
Hey Prashant, I'll answer here instead of twitter DMS.
- No, you can host as many as you want. The signing key gets reused across repos. This is done so that PDSes can use the secure key managers that cloud services provide, which cost too much money per keypair to support 1 key per account.
- That is possible but you'd need to customize the PDS software to do that. Our implementation expects writes to be handled by the "home" PDS. You could make a version that is declared as the home PDS but actually just rehosts the current repo, allowing the signed updates to come from somewhere else.
- 09:55the account recovery flow that dean mentions allows you to change the signing key and the current PDS without your current PDS's involvement. The current expectation is that you're moving to another PDS that will manage your signing key for you, but like I described you could create custom configurations if you wanted
- Dean09:57Somewhat related question I just posted in#general-bsky:matrix.org: Would it be according to standard to have 2 different handles on 2 different DID providers (i.e. a secondary domain) resolve to the same DID?(edited)
- pfrazee09:58so first to clarify some language there, you'd have 2 different handles on 2 different handle providers that point to the same DID
- 09:59you know, obviously - the DID provider gives out DIDs, the handle provider gives out handles
- 10:00as for 2 handles to 1 DID, yeah that's possible but a bit underspecced. The DID document declares its current handle to help establish which one is the primary one. I think secondary handles could logically be handled like a redirect
- Dean10:10
In reply to this message
Right sorry, I still need to adjust to the protocol's terminology :D thanks for the info! - 10:10
In reply to this message
Is this something you believe the spec might need to include? i.e. returning an array instead of a string in the did resolver'salsoKnownAs
field? - pfrazee10:10no prob
- Dean10:19
In reply to this message
Currently when you resolve a handle, according to the Identity docs, you get analsoKnownAs
field which returns a single domain handle. Perhaps to support alternative names for the same did, it could be an array instead of a string, and the array's first value is the primary name while any following values are things like shorteners/alternative handles on other handle providers. - 10:20
Instead of:
123456
await didPlc.resolve('did:plc:12345') /* => { id: 'did:plc:12345', alsoKnownAs: `https://alice.pds.com`, verificationMethod: [...], service: [{serviceEndpoint: 'https://pds.com', ...}] }*/
It could be:
123456
await didPlc.resolve('did:plc:12345') /* => { id: 'did:plc:12345', alsoKnownAs: [`https://alice.pds.com`, `https://alicia.pds.xyz`], verificationMethod: [...], service: [{serviceEndpoint: 'https://pds.com', ...}] }*/
(edited) - pfrazee10:21yeah that makes sense but what are you aiming to solve?
- Dean10:22For example showing the primary handle in a user profile feed when a secondary one is visited
- 10:24So visiting
https://frontend.com/alicia.pds.xyz
, the frontend looks upalicia.pds.xyz
, finds the primary handle (alice.pds.com
) as the first value in thealsoKnownAs
array, and displays that as the handle in the profile (perhaps with a list of "alternative handles" somewhere in the profile as well)(edited) - pfrazee10:26That makes sense. I'm inclined right now to keep it simple and just handle secondary handles as redirects
- Dean10:26That way the displayed profile information will remain consistent regardless of handle
- pfrazee10:27I can see some interesting benefits from it, like being able to alias with your work identities
- 10:27
@pfrazee.com (aka @paul.bsky.team)
- Dean10:27Exactly
- pfrazee10:28I'll bring it up with the team. For the moment I'm inclined not to add to our complexity, and there's no time pressure on this. We can add your suggestion in the future by a variety of means
- Dean10:28Perhaps the
alsoKnownAs
field could be an array with a single value just to future-proof it? - 10:28So front-ends don't break if it's added in the future
- pfrazee10:28let me check the DID spec real quick, but we can solve this no matter what
- 10:29(like, worst case if we're backed against the wall we can add a field to the did doc)
- 10:29I just dont want to add complexity to this yet
- 10:29complexity aka sophistication
- 10:30oh you know what,
alsoKnownAs
is supposed to be an array according to the DID spec - 10:30I'll open an issue to make sure our code and docs are on spec
- 10:31guess that means we are tackling this now
- pfrazee10:36okay issue filed
- Dean10:42
In reply to this message
I filed an issue for this other problem just now: https://github.com/bluesky-social/atproto/issues/330 - 10:47Why are profile descriptions supported in the lexicon but not avatars? I can't find any lexicons or protocol specs on handling avatars and other common profile media (e.g. a profile banner)
- 10:48I guess it's a waste to store image blobs in atproto
- pfrazee10:48we haven't implemented "blobs" yet, which is the generic system for non-record data and which will handle media
- Dean10:48Oh ok
- pfrazee10:48actually no, mercifully we can do blobs in atproto, unlike say git
- Dean10:48So there will eventually be an "avatar" attribute in
bsky.getProfile()
Response object?(edited) - pfrazee10:49something like that yeah
- 10:52we've got some plans for blobs, including schemas for establishing size limits and expected variations of an image (small thumb, large thumb)
- 10:53you want a pretty well-coordinated system for blobs because they're a potentially huge performance problem. You want predictable sizes on things like avatars so that you don't end up downloading a 2mb png to display a profile
- Dean10:53Yeah exactly
- 10:54I suppose IPFS is out of the question for the spec? It would allow you to check headers for filesize before downloading the image data.(edited)
- 10:56It's possible to create an isolated IPFS network between indexers and PDSs to serve data for avatars
- pfrazee10:57we can talk about it. It's cool to leverage the open horizontal scaling, but the cost is that you lose a lot of concrete windows when you use p2p, plus you introduce the overhead of the swarm stack
- 10:58the overhead is non-trivial -- you have to regularly announce all the objects you're hosting to the discovery network. Imagine sending 1 packet for every blob once every minute
- 10:59RE the windows, you lose the ability to contact the host and say "it timed out" after 5 seconds or so. Instead you have to watch the peer discovery system to collect all the IPs, which has a variable window of time, and then you have to contact each of the peers, each with their own timeouts. It makes the system much harder to reason about
- 11:00consequently, it's only really valuable for situations akin to how bittorrent is traditionally used: very large datasets with no strong expectation for time of delivery
- Aaron Goldman11:30That said a Merkel tree is a reasonable way to reference a large blob as a collection of chunks. One could imagine a few record types: Small blob < 1MiB (literal bytes) L1 blob < 1MiB (list of Small blob CIDs) L2 blob < 1MiB (list of L1 blob CIDs) L0 1MiB L1 32 GiB L3 1 PiB Assuming about 32 bytes per CID and a 1 MiB max_record_size You would also probably want to support a check out of the repository where you ignore the nodes under a block record until they are requested so you never spend more than 1 MB per blob
- @penaiple:midov.pl
- 11:41what blobs are you talking about
- 11:42proprietary firmware blobs?
- Dean11:42image data blobs
- @penaiple:midov.pl11:42oh nevermind then
- 11:42the word "blob" rings alarm bells in my brain
- 12:04dutterbutter joined the room
- MightySpaceman (OLD -> m_spaceman:matrix.org)15:52blob
- 16:12jdenoy joined the room
- 18:19airwaves changed their profile picture
- 19:29manuelbonnelly joined the room
- 19:56@stephane_huonder:matrix.org joined the room
- 20:54robesdornagon joined the room
Thu, Nov 10, 2022
- @tomtau.:matrix.org04:02Since ActivityPub has gained a lot of momentum, will there be a way for AT to interoperate with it in some way?
- 10:19@michael_zks:matrix.org joined the room
- Aaron Goldman11:10
In reply to this message
This is not currently a priority and not on the road map.
There are three possible ways forward for this.
* One could create an activity pub event for wrapping an ATP record.
* Lots of complexity for Activity pub clients
* Two wrap activity pub events in a ATP record
* Lots of complexity for ATP clients
* There a bridge that translates between Activity pub events and ATP records
* Impedance mismatch between schemas and identity conceptsAll three have real drawbacks and would likely slow progress. Probably better not to put limited time of the dev team in expensive interoperability with activity pub at this time.
- 12:25nc163 joined the room
- 12:48nc163 set a profile picture
- 12:49nc163 changed their profile picture
- 12:50nc163 changed their profile picture
- bnewbold13:32I feel like there is a middle path on using IPFS-like system for blobs (media artifacts)? have norms around block sizes and specific derived media formats/sizes. reference these by CID directly from ATP records. don't use bitswap for transfer, instead do transfer in CAR format over HTTP via an XRPC request (supplying a list of CIDs). store those blocks in whatever same blockstore being used for the MST. PDS doesn't need to pull CIDs for all content unless it wants a "deep" checkout. the current repo host of a handle is expected to have all the blobs references by any records in that repo, other PDS instances can cache and redistribute as they see fit. maybe optionally also have the "root" IPLD node point to a UnixFS DAG with all the blobs for the repo stored with filenames as well? that would make export-to-disk and exploring existing files easier. but not really needed and extra complexity
- pfrazee13:33I'm reading quickly because I'm hackin but that sounds like what we're thinking. We put blobs in the repo's IPLD. We transfer them via ATP sync protocols. We bake specific behaviors for them
- bnewbold13:34i'm guessing video is going to be the tricky thing, going to be more than a "block" even for compressed versions, probably going to need a bunch of sizes, and might want to be able to stream by fetching early blocks first (like webtorrent). and at the same time can really bloat up caches/blockstores
- 13:37it would probably be nice to somehow stay compatible with whatever cloudflare is offering for IPFS CDN (I haven't looked in to that in a while). in the expectation that other providers (or FLOSS equivalents) will have similar semantics
- pfrazee13:45we're paying close attention to integration with cloud systems since we're deploying a service, so we'll export decisions like that in the design
- 13:45aka if the IPFS CDN or similar are useful
- 15:31Karl Abbott changed their display name to Karl Abbott (Away until 14-November-2022)
- 19:14ne.dmtro joined the room
- 23:03quentez joined the room
Fri, Nov 11, 2022
- 03:00Nad // away changed their display name to Nad
- 04:02@michael_zks:matrix.org left the room
- George Antoniadis06:09Message deleted by pfrazee
- 06:29Maiquel De Brito joined the room
- 07:01Agustín Cantero joined the room
- 10:45pere joined the room
- 10:46@fr33domlover:matrix.org left the room
- 13:26Nad changed their display name to Nad // away
- 14:44slap19 joined the room
- 17:07@elsif:matrix.org left the room
- 17:39misaakidis changed their display name to marios
- bnewbold18:04I made some progress on adenosine, a rust hobby implementation of atproto: https://gitlab.com/bnewbold/adenosine there is an example single-user instance at https://pierre-menard.robocracy.org/, and an example multi-user instance at, eg, https://hobbes.demo.adenosine.social/ you can explore the repository a bit: https://demo.adenosine.social/at/did:plc:3vp347vrxvzcs2dulat5sqwj the web interface is read-only, but the CLI works with these domains. DM me for an invite code to use with the CLI ondemo.adenosine.social. everything will definitely get wiped, is totally naive about abuse etc. and there is no server-to-server stuff, which would be the interesting part
- Aaron Goldman18:17Cool
- pfrazee18:18wow. Does that mean you have a working MST written in rust?
- bnewbold18:20sort of? it imported and validated against a CAR file exported from the atproto imlementation. but it is really crude, reads or writes the entire tree every time, instead of trying to update in place. and doesn't support partial trees https://gitlab.com/bnewbold/adenosine/-/blob/main/adenosine-pds/src/mst.rs
- pfrazee18:20nice
- bnewbold18:21to some degree, this was really the only part of the projet that made sense to write in Rust (strong typing, memory management, etc). I was pretty happy that it just worked the first time it finally compiled
- 18:22it is built on existing IPLD serialization libraries and an existing IPFS blockstore (using sqlite), so I didn't need to implement any of that
- pfrazee18:22yeah, makes sense
- bnewbold18:30if I have time i'll push debian packages and maybe binaries in homebrew. I expect that the upstream schema is going to continue changing rapidly, so maybe best not to do that until things stabilize (aka, a public release). there are a couple divergences and issues already (like the TID format)
- pfrazee18:30yeah I'd recommend that just so we don't confuse folks
- bnewbold18:34if there are places I can add louder disclaimers about being unofficial/incompatible/whatever please send recommendations or PRs and i'll push those right in
- pfrazee18:34oh I appreciate it, I think you've done fine there
Sat, Nov 12, 2022
- 04:29Evren joined the room
- 10:14Evren left the room
- 11:29skyllaadalove joined the room
- 20:40Andre Simao joined the room
Sun, Nov 13, 2022
- 05:33jaygreasley joined the room
- 08:30sethetter joined the room
- 09:14@wings:perthchat.org left the room
- 14:46Tavin Turner joined the room
- 14:57@blahverse:matrix.org joined the room
- Dean17:41Message deleted
- Dean17:52I posted in #general-bsky:matrix.org since it seems to belong there, but the dev-related note is: it looks like "likes" have been replaced with upvotes and downvotes, I made a PR to represent these changes in the lexicon docs: https://github.com/bluesky-social/atproto-website/pull/6
- 17:56Jean-François Cotenord Tatoo joined the room
Mon, Nov 14, 2022
- 01:03kolby joined the room
- kolby01:06Hello
- 01:08kolby set a profile picture
- MightySpaceman (OLD -> m_spaceman:matrix.org)05:25hi
- Prashant Mittal05:48Is a handle globally unique for a particular subdomain or is it locally unique in the context of a PDS?
- 07:15Karl Abbott (Away until 14-November-2022) changed their display name to Karl Abbott
- 07:54@geoma:matrix.org joined the room
- Dean09:54
In reply to this message
I put up a Github issue with my thoughts on the recent update that replaced likes with upvotes/downvotes: https://github.com/bluesky-social/atproto/issues/347 - sylphrenetic09:56
In reply to this message
pfrazee: am I correct in thinking that this change doesn't pigeon-hole implementors of ATP into ONLY using likes/dislikes, it just allows likes/dislikes as an option? so one implementation of the protocol could choose to not use dislikes at all, and only use likes as "favorites", and that wouldn't allow random people to start disliking posts on that implementation because that implementation isn't taking dislikes into account, right? - Dean09:57
In reply to this message
The protocol shows both counters separately i.e.upvotesCount
anddownvotesCount
, so it's up to the client if they want to account for one or both - 09:57But that will cause clients to divert from a standard way of displaying "scores"
- sylphrenetic09:58I think the solution then is to add a "favorite" to the schema as well, so that if someone wants to use favorites they can use them without overlapping with the likes/dislikes paradigm
- Dean09:59I thought of this as well but that would still cause discrepancies
- 09:59Imagine you make a post and just see Likes on it, and you want to invoke a friendly discussion
- 09:59And people from another client show up with "idk why you're getting downvotes but..." and "lmao jump on the downvote train!!"
- 10:00Imagine every tweet you write also shows up on Reddit, and every reply also shows up in a reddit comment section
- sylphrenetic10:00make them mutually exclusive
- Dean10:00Then you should have 2 protocols
- sylphrenetic10:01wut, why?
- Dean10:01Because the respective purposes of the 2 types of posts are incredibly different, no?(edited)
- pfrazee10:02This is an interesting question
- 10:03My default assumption is that Dean’s view is the “safer” assumption: that on core social interactions it’s pretty important to stay consistent
- 10:03For the reasons that Dean described in the issue: it drives the perception and thus the conversation
- 10:04That said, it’s really hard to predict how big that impact is or how important it is
- 10:05Obviously you can have some divergence. Which ones are okay and which aren’t? It’ll be messy but stabilize over time.
- Dean10:06Another problem is it lets people negatively impact other people's accounts e.g. with botted downvotes
- 10:07Without downvotes the only possible interactions generally are a net positive for the author: like/repost/reply
- sylphrenetic10:07Right, but I think the protocol is hamstrung by not allowing certain basic functionalities on posts. Why not just make it so that once the post is created, you have to choose whether it's of type "likes only" or "voted"?
- Dean10:07But once you feel like you're being constantly unfairly downvoted by trolls or bots, you no longer want to post on the protocol
- pfrazee
- sylphrenetic10:08That's what would make intuitive sense to me at least
- 10:08as a dev
- pfrazee10:09There’s a lot of interesting configuration options for posts like that, also including how replies are done: who can reply, who can moderate
- Dean
- 10:09I posted the same idea in #general-bsky:matrix.org but also argued why this might not be the best solution ^
- 10:10
Screenshots broken, here's a quote:
I think the solution in this case would be to have a downvotesEnabled attribute in the Post object where posts created on certain platforms cannot be downvoted (clients that prefer downvotes could apply a different algorithm to posts that have downvotes disabled)
(if downvotes are something definite for atproto)
Either way, the respective purposes of posts with either a Likes or Upvotes/Downvotes scheme are inherently so different that they might as well belong to two different protocols - pfrazee10:10Fair. If it’s configured on the post then a different schema for like would certainly reduce confusion
- Dean10:11Indexing servers would still be indexing all posts (wasted bandwidth) and then filter out the types that they don't want to include
- 10:11Yeah if it's a different schema entirely and servers don't have to sync that specific schema type then I agree it could work
- 10:12I do think that the current integration, which is an outright replacement of likes, is not the right solution
- pfrazee10:13I get the logic. As for the debate around likes vs votes, that one is spicy. I told the team when we did it, “we’re going to all be debating this a lot”
- Dean10:16As long as indexers have the option to sync both or either types so they don't waste sync time + bandwidth would be the most ideal if you want to support both.
- 10:16That still might cause a lot of "why does my post on ATP client A not show up on B?"
- pfrazee10:19Yeah. I think your understanding of the technical implications is right. For the moment we’re going to focus on testing the experience and seeing if this is even a worthwhile direction
- Dean10:22The main issue with voting on a decentralized protocol like this would be botting, where botting with Likes generally leaves no lasting negative effect on any third party (if someone bots likes on their own posts the user can lose their followers' trust/goodwill), basically only affected upon oneself, but botting with downvotes could be a simple but effective way to negatively impact others(edited)
- pfrazee10:24yeah first of all, that's very accurate. For one, Im concerned about any way that people can be given a negative experience
- 10:25for two, botting is a big problem for us to solve across the board
- 10:27the entire notion of the "userbase" gets much more complex when you no longer lean on a central authority. There's no initial curation pass like what twitter has of establishing who can be on the platform
- 10:27my hope is that we can find ways to organize users and create reputational signals that replace that kind of function without leaning on server admins
- 10:28which could then act as a counter-botting mechanism
- whyrusleeping10:30I also think it will pay to be extremely judicious with how downvotes are used. I dont think they should have their counts displayed in UIs, and should only carefully affect feed generation
- 10:30Who a downvote is from is way more important than how many there are
- pfrazee10:31whyrusleeping: if we dont show downvote counts, I can see the logic to that but I'm concerned about hiding them flat out, because people might wrongly think they cant be revealed. But again I'm open to thoughts
- whyrusleeping10:32Ah, yeah. Maybe they are hidden off the main post view, but you can drill down and find them
- 10:32And then downvotes by people you follow are listed at the top of that view
- Dean10:33
In reply to this message
Maybe clients can show downvotes as "reports" and only show a count of reports from accounts that you follow - 10:34Something like a
downvoteFromFollowedCount
but shorter 😂(edited) - pfrazee10:34they're not quite reports but I see the thinking
- George Antoniadis10:35I assume servers can chose/code how to display/deal-with votes, and how they affect the ranking algorithm? ie I'm a fan of how HN deals with votes, but on a more federated system I'd like to also have a sort of "web of trust" thingie that friends' and friend of friends' votes have way orders of magnitude more significance. (issue here is that following someone does not make them a "friend" or trust-worthy person which is another issue)
- 10:36^ also s/server/indexer probably?
- pfrazee
- Dean10:36If keeping the timeline-feed networking theme I would prefer the idea that a "downvote" is only expected to be made if the post is objectively something the voter believes breaks a client's tos (e.g. nsfw content on a family-friendly client)
- 10:37Hence in my mind it makes more sense as a "report"
- @numero6:codelutin.com10:37(not so off-topic) I sometimes view upvote and downvote has reducing emoji reaction to only allowing two emojis: 👍️ and 👎️. Any system who allow more emojis may have more algorithmic possibilities. I know that FB scored post differently if those had ❤️ or 😠 reactions. I understand the simplicity of +1, -1 and integers are easy to understand but counting emotions might be a more humanly way to "count" a post quality.
- pfrazee
- 10:38
In reply to this message
yeah that is the trick though, aggregation of reactions loses some meaning. I think it's great in chat but in public performance, I think you need to reduce it down to fast button presses - 10:38(I generally see Twitter as a public performance space, for better or worse)
- Dean10:39
In reply to this message
Then what do you ideally expect downvotes to be used for? Disagreeing? - @numero6:codelutin.com10:40
In reply to this message
why does a public performance space wouldn't have one push button emoji reaction? (I know some ActivityPub app has those, including some Mastodon forks)(edited) - pfrazee10:42
In reply to this message
1, it's at least two presses, sometimes with a scroll to find the right emoji
2, the purpose of reactions in a performance space is to affect its spread
3, it reduces the clarity of the metrics, which is your scorecard in the performance game - Dean10:43
In reply to this message
If downvotes aren't expected to be delivered for content that breaks certain guidelines, then I suppose it would be for when someone disagrees with said post, which would not be a good basis for reducing a post's exposure(edited) - 10:46A good example being FTX happenings on Twitter in the past week. If exposure based on downvotes were a thing, many people may not have seen tweets with warnings of an FTX insolvency due to the large number of downvotes they might have received from FTX fans and token holders, which would have caused many people to not have withdrawn their funds in time.(edited)
- whyrusleeping10:47You’re assuming a particular algorithm for feed generation
- Dean10:48Well I highly doubt any client would integrate downvotes as a positive factor
- pfrazee
- 10:49what why is getting at is that we expect there to be multiple different algorithms operating on the votes
- 10:49including things like who the algorithm listens to
- 10:49in which case the FTX fanboi algorithm is going to suppress it, but the Crypto Critics algorithm wouldnt
- Dean10:51
In reply to this message
This begs another question: should reposts have separate voting stats? - 10:52If something from a third party is being massively downvoted and I still want my followers to be exposed to it
- pfrazee10:52my general sensibility about RTs is that they are a kind of non-algorithmic override
- 10:52if you RT something then your followers should see it
- Dean10:53Btw currently the override is that reposts don't show up on anyone's timeline 😅 https://github.com/bluesky-social/atproto/issues/346(edited)
- pfrazee10:53haha that's a bug yeah
- 10:54so we separate the concept space here into "Timelines" and "Feeds." Timelines are what you see on your home page. Feeds are what some entity emits as a sequence of posts
- Dean10:54Yeah I got that
- pfrazee10:54and youre going to be able to create/use different variations on both
- 10:54a "reverse chronological" Timeline would basically cause all RTs to show up
- 10:54but a chron TL would also include content from algorithmic feeds
- 10:55which is a nice option for people who like to make sure they get everything even if it's noisy
- 10:55an algorithmic or customized TL would apply rules and depending on those rules, this "RTs always show up" question would then depend on the rules you are using
- 11:41@blahverse:matrix.org removed their display name (blah verse)
- 14:16duc [they] changed their profile picture
- 16:26martnx joined the room
- 16:27martnx set a profile picture
Tue, Nov 15, 2022
- 02:49alexbourlier joined the room
- MightySpaceman (OLD -> m_spaceman:matrix.org)
- alexbourlier04:13Funny
- 09:10kursato joined the room
- 09:14kolby changed their profile picture
- 09:39Giovanni Fersan joined the room
- 10:58franx_ear joined the room
- 11:53@secu:mtrx.nz left the room
- 19:36@toneji:nibbana.jp joined the room
Wed, Nov 16, 2022
- mikestaub00:33No pressure, but we may need this protocol sooner than we thought. https://twitter.com/IanColdwater/status/1592188128944705537
- Giovanni Fersan00:47Is the waitlist currently only for users or for developers as well?
- 00:48Giovanni Fersan set a profile picture
- 00:48giovannifersan changed their display name to Giovanni Fersan
- 02:54@amine_mdx:matrix.org joined the room
- @blahverse:matrix.org09:30I think an ability to allow server admins choose some of the names/labels would be really cool. e.g. some admins can go repost and upvote route, while others can choose retweet and like - based on the audience they are targeting.
- Aaron Goldman09:46
In reply to this message
I think this is counter to the separation of server from application goal.
If you want to make a new application with different record types you can complete withbsky.appand if useful reuse some of each other's record types.
However the server should just be ATP and agnostic to your app so that migration remains trivial.To allow per server customization is to introduce vendor lock in to the protocol.
If you want to sell a custom experience make an app with the record schema that you want and let the users pick their servers.
- pfrazee
- Aaron Goldman09:58I think it's a mental adjustment from the ActivityPub model that their Server Admin role has been split into the App and the Server. A PDS should be low administration. Mostly choosing the Blocklists or Allowlists to subscribe to for repo's/blobs so you don't host what you don't want to host.
- @blahverse:matrix.org10:03Interesting. Thanks Aaron Goldman for the detailed feedback. I will look out for the beta launch to play different things out.
- 10:36@amine_mdx:matrix.org left the room
- Hacklet13:52Is there a separate discussion for the app ethos over the technology?
- b0gg3r
- mikestaub
- 15:34g5646545646587 joined the room
- 16:45aaron-- joined the room
- mikuhl20:20I luv umbrel
- Aaron Goldman23:10I have been thinking about the analogy to talk about the relationship between PDSs, Apps, and Lexicon. I think browser features may be a good one. For the most part browsers from the beginning had mostly overlapping features. Some added unique features that the others copied. That has the risk of name collisions. After that the browser makers added vendor prefixes to features. `-moz-window-inactive` was made by Mozilla but was supported by chrome. In the same way Lexicon made bybsky.appcould be rendered by any app when it reads the Repo. Chrome is not Firefox but can implement a -moz-* feature if it wants to. The http server does not need to understand all HTML, CSS, and JS to serve the content. There are many browsers (apps) that can render content. But unlike the browser features that leads to the https://caniuse.com/ nightmare the Lexicon structure should give more structured fallbacks to common functionality. So what do y'all think is this a useful analogy or just too confusing to be helpful?
- mikestaub23:15
In reply to this message
Ii think this is a perfect analogy. What is a shame with web browsers is that they don't know which APIs the website is using so if the site is depending on an unsupported API it will just break. Ideally apps could declare all the APIs/schemas they depend on in a manifest so clients can gracefully degrade. - Aaron Goldman23:22What, you don't find writing browser feature detector code in Turing complete JS with custom hand rolled fallbacks a fun time. 😡
Thu, Nov 17, 2022
- 01:29polyeven joined the room
- airwaves
- MightySpaceman (OLD -> m_spaceman:matrix.org)
- 06:41@graham_o:matrix.org joined the room
- 06:58@graham_o:matrix.org set a profile picture
- 06:59@graham_o:matrix.org changed their profile picture
- 08:58michaelvigor joined the room
- pfrazee09:49lol woah
- pfrazee11:38I agree with this browser framing. The difference is that rather than HTML tags, CSS features, and JS APIs, the "ATP browser" is speaking a semantic data language (lexicon)
- 11:40posts and follows instead of divs and styles
- mikestaub12:37
In reply to this message
conceptually there are the same though. CSS features and schemas both affect an apps ability to render properly. Would what be amazing is if the client could detect the schema and prompt the user to install a plugin to view it, like VSCode does when you open a Dockerfile - pfrazee12:39right yeah -- that's what I mean, that it's an equivalence, but the primitives of the "language" are different
- 12:40as for plugins or similar, there's basically two questions we'll have to navigate as this evolves
- 12:41@graham_o:matrix.org changed their profile picture
- pfrazee12:41the first is choosing the dividing line between publisher control and client control. The Web went through this exact thing; the notions of separating style from content and of "user agent stylesheets" were born out of a desire for client control. This is a matter of debate now but I'd argue "publisher control" won heavily on the Web, and that's largely because the Web browser is a thin client which transfers UIs
- 12:43as ATP operates at a higher level of semantic, it is feasible that we can favor client control, but this will be a constant debate
- 12:44the second is how we handle client handling of new lexicons. A plugin architecture is certainly possible, and it's very much a "client control" style solution
- 12:45plugin systems are hard but I'd wager the community will do some interesting experimentation there. Theyre hard in general -- hard to design well, hard to manage resource usage, hard to secure, hard to create a clean UX -- but then you have stuff like Apple making it hard to do that sort of thing on iOS with their policies(edited)
- @graham_o:matrix.org12:49Message deleted by pfrazee
- 12:50whyrusleeping banned @graham_o:matrix.org: spam
- Aaron Goldman12:52Plugin vs many apps Is a non-obvious trade off. You could imagine many apps that do one thing and do it well. With the browser that "open with"s the unixy apps or a super app that handles all kinds of things.
- 17:42kt654vh57j joined the room
- mikuhl
- 19:49what did elon do LMAO
- pfrazee19:49Whew buddy
- mikestaub19:50
In reply to this message
I think Chrome really nailed it with their extension model. This is the best example I am aware of. https://classycraig.com The user has full control of their own experience, if AT protocol can make the data layer as interoperable as the client layer, things will get super interesting fast. - mikuhl20:07So.. how you guys doin lol.
- sylphrenetic20:08I'm in Hank Green's Twitter space j chillin tweeting about #butts
- pfrazee20:10Haha we’re good. Can’t share deets juuuust yet but it’s coming together fast
- 23:29guicaldaso joined the room
- 23:30gui.caldas.o changed their display name to guicaldaso
- 23:30guicaldaso set a profile picture
Fri, Nov 18, 2022
- 00:02Yuki joined the room
- 00:49Aditya Sharma joined the room
- 03:22James Fan joined the room
- James Fan03:23Hi! Dear ! I am experienced in Full Stack Web & Mobile Application using ➤React, Vue, Angular, Django, Flask, Solidity, Golang ➤Python with AI technology, SciPy, NumPy, Matplotlib, Web Scrapping, AI, ML, DL, NN, Prediction, Decision-Making, Image/Audio & Video Processing, ➤Effective Algorithm Design & Analysis ➤Smart Contract, Blockchain, dApp, ➤SQL/NoSQL, Real-Time Processing, Multithreading, Automation Processing, Bot, Electronic Automation with Micropython. if you need my help, please send me DM! Thanks!
- 03:28kobagen joined the room
- 03:45@vyach:matrix.org joined the room
- 05:28hikae joined the room
- Dean
- 08:13@neilalexander:matrix.org changed their profile picture
- magicofazi08:21Between AT Protocol and Matrix Protocol, which is superior and why
- 08:21Personally think matrix is the furthest ahead in terms of federated and p2p messaging and has client diversity already
- bsjdjjdj
- sylphrenetic08:51"Two weeks" -Elon Musk 😘 /j
- bsjdjjdj08:54I'm thinking about building an android client and wanted to get a sense of the roadmap
- 09:53thenameless joined the room
- 10:13Robert Johnson joined the room
- pfrazee10:25no eta unfortunately. We're really close but we've got a few key pieces to stabilize still
- 10:27@enapiuz:envs.net joined the room
- sylphrenetic11:08I noticed a problem today with ActivityPub and Mastodon: if I'm on server A and want to migrate my followers (and posts, but I don't think that's supported on Mastodon) to server B but server A goes completely offline, there's no way for me to migrate ANYTHING over to server B, I'd have to create an entirely new account and start from scratch. that's a horrible experience, analogous to if Twitter was to lose its entire database and require users to create all new accounts on some new server. will ATproto solve this in any way? I would hope I could take my data (posts and all) and up and move home servers to anywhere I wanted if my current PDS is online, but what if its offline? I guess I would have to have some data redundancy in there somehow, like my own offline copy of my data that gets updated every time my home server does (but with lower priority on speed, so updates could be batched to it over time, like when you get to be on wifi from a smartphone). are there any plans to natively solve this?
- @toneji:nibbana.jp11:14sylphrenetic: Nice to meet you.
Isn't Huzzilla the one with the mechanism to guarantee DB redundancy?
I think even matrix has to be set up from scratch on another server if your main server goes offline.(edited) - Dean13:20Coming back to likes vs upvotes/downvotes: is the general consensus for the protocol to include both options (poster/client decides which to post, indexer decides which to index) before launch?
- 13:25
In reply to this message
Auto offline backups should be an easy feat. Just set up a local indexer server that indexes only your own activity. - @toneji:nibbana.jp
- sylphrenetic13:42
In reply to this message
I don't think it would be easy, per se, especially for the layman. I think there should be something built into the protocol to allow duplication of your data among multiple PDSs, with only one being the authoritative one and the others being backups (most likely hosted in the cloud by another service provider). - pfrazee13:53
In reply to this message
We've got this as an actively watched UX decision and make a call based on what yall and test users are saying. Book is not closed, but no idea where it'll land - 14:13Karl Abbott changed their display name to Karl Abbott (Away Until 28-November-2022)
- Daniel Holmgren15:11
In reply to this message
the data is very portable. multiple copies can be stored in different places & if your PDS goes down, you can "restore" your account to another service. This could be another PDS in the network, a backup node, or even just local back up on your device. You may not want to include large blobs in a local backup, but posts & social graph can be backed up without much overhead - 15:16Matheus Mósso joined the room
- 15:48rdubs20 joined the room
- 16:06@rimuru:gentoo.chat changed their profile picture
- 16:36joeymueller joined the room
- 19:10dotneb joined the room
- 19:11dotneb set a profile picture
- 19:50bodega joined the room
- 20:10Deans Charbal joined the room
- 21:20n0nmanifest joined the room
- 23:17NikhilM42 joined the room
Sat, Nov 19, 2022
- 04:31erlend_sh joined the room
- 06:10harshveer14 joined the room
- harshveer1406:20Announcing my arrival 😁✌️ New to the chat .. traditional java backend developer based out of the bay area
- Aaron Goldman09:49Don't think anyone has started a community implementation of the server in Java 🤔
- 10:12Brandon Esau joined the room
- 10:24Seve(rino) Zeni joined the room
- 10:45@mrkizoo:matrix.org joined the room
- 11:55@mrkizoo:matrix.org left the room
- 11:56Josh joined the room
- sylphrenetic11:57my original background is in Java, so I might be able to help out once the time comes 🥰
- 11:58Seve(rino) Zeni left the room
- @neilalexander:matrix.org
- Josh12:38Hi
- Aaron Goldman13:04
In reply to this message
Bluesky PBLLC is doing one for Node JS in Type Script. Not sure what others exist. But again it's early and subject to change which limits the motivation to make an implementation - George Antoniadis13:06n@neilalexander:matrix.org: why has some parts implemented in go as well, https://github.com/whyrusleeping/gosky/
- @neilalexander:matrix.org13:16Thanks both
- 13:16* @neilalexander:matrix.org was secretly hoping for something in Go
- George Antoniadis13:17n@neilalexander:matrix.org: yeah figured as much, love your work on pinecone btw. amazing stuff. :D
- whyrusleeping14:49Oh hey
- 14:49Happy to collab on any of the go stuff
- 14:49Its a bit of a toolkit right now as opposed to an “implementation”
- magicofazi15:14n@neilalexander:matrix.org: what’s your opinion of blue sky vs matrix
- @neilalexander:matrix.org
- 15:21
In reply to this message
In my mind they are not easily comparable. Matrix is more about multiplayer data structures and has a lot of complex machinery involved in making sure servers arrive at the same conclusion whilst operating independently (conflict resolution, auth rules etc), whereas ATProto seems to be much more about single entities maintaining and publishing their own conflict-free data structures that other people can just happen to peek into or interact with - 15:22Different solutions for what are arguably different problems, even if there are some overlaps in how they work or what they could be used for
- magicofazi15:23
In reply to this message
Do you think matrix can adopt the portable account features at protocol has? - @neilalexander:matrix.org15:24
In reply to this message
Portable identities have been on the Matrix future roadmap for some time now and it’s bound to happen eventually, but don’t know how much it would resemble ATP’s implementation at a technical level - 15:24Not least because it means you can take some power away from servers, which is quite important to do
- magicofazi15:26Fingers crossed we solve portable identities by next year 🙏
- 15:26Also hope dendrite gets closer to production ready implementation
- George Antoniadis15:28we've very very very off topic here, but just wanted to say dendrite is working like a charm :D using it as we speak
- @neilalexander:matrix.org15:31Apologies for topic drift ;-)
- 15:31A Go-based ATP implementation would be quite exciting though
- 15:32I would happily contribute some personal FOSS hours to that
- George Antoniadis15:32Count me in as well :D Happy to pitch in.
- Aaron Goldman15:33
In reply to this message
What do you think about Theseus Identities?
Is there part of the portable identity problem you think this works poorly for?
- George Antoniadis17:19
In reply to this message
Thank you for sharing this, this direction is really great.
Is there a specification for this somewhere public?Have you happened to have stumbled on KERI by any chance?
I think it fits nicely your notion of Theseus IDs (up to a point) and it might offer some interesting ideas.It's pretty much a series of "events" that change the state of the identity. (the main ones are "key rotation" and "delegated key rotation")
It uses a notion of a "next key" that the user can use to "rotate" to that key if their current one gets leaked/lost (similar to the "recovery key" mentioned in the video
Has a concept of a "witnesses" that can be used to pull the events for an identity, and to recover a lost "next key".
The idea of ticks doesn't really exist for ordering, but instead the moment two conflicting events are seen, both are considered invalid and the key needs to be rotated using the "next key".
Has some ideas in general def worth taking a look at (ie delegation). - Aaron Goldman17:31Yup I like KERI. Duplicity is a question of having a consensus group for updates to prevent the double rotate problem (basically the double spend problem). I don't know how this is done with the witnesses. I should look more at their consistency story.
- Aaron Goldman18:17From the video it seems like more of a local consistency guarantee. With the controller responsible for consistency not a consensus group. That said the witnesses server he called Doug is similar to the PLC server. I think there is a question here for what is the best way to sell an identifier.
- whyrusleeping19:12n@neilalexander:matrix.org: ive got all the serialization working and most of a code generator for the rpc schema files done. I have pieces that together into a proper PDS yet though, but i can do all the client stuff thats needed
- 19:12Ive been using it so far to do some search engine indexing
- magicofazi19:47So i'm the founder ofzo.me, we're forking the matrix protocol, creating global identifiers on a app chain PKI with economic security (ETH,BTC,etc) Servers or Zo Nodes hold local identities but users also hold global identities with social recovery allowing them to remain resistant from deletion by their home server/node
- 21:59@vsco:matrix.org joined the room
- 22:12UnownPlain joined the room
- 22:14UnownPlain left the room
Sun, Nov 20, 2022
- @jan:cloudcheck.io
- whyrusleeping02:59Probably a better discussion for the general channel
- 07:06ghobs joined the room
- 07:56@amirabbas_ir:matrix.org joined the room
- 10:45i3xCx joined the room
- magicofazi12:06Matrix is pretty focused on building out synapse and has shown a disinterest in any crypto integration at the protocol level. My goal is to fork Dendrite, make it production ready, attach a global identity using a appchain based PKI on ETH/BTC, and have a token that is used for spam prevention and economic guarantee of identity data. The goal is also to use Rooms as a CRDT based Database and allow developers to build Local First JS Apps as mini apps
- @penaiple:midov.pl12:09JS is useless slow insecure trash
- 12:09you should build programs in C if possible
- 12:10otherwise C++
- magicofazi12:10we'll be using typescript actually
- @penaiple:midov.pl12:10for web stuff you can use TS, HTML, CSS
- magicofazi12:10but main homeserver will be written in go
- @penaiple:midov.pl12:10JS has been designed in 10 DAYS
- pfrazee12:10let's not get into language wars here
- @penaiple:midov.pl12:10and pretty much all security holes in web browsers are caused by JS
- 12:10half of them by JIT
- 12:11not as nice as C, but far better than python or js
- 12:11synapse constantly needs babysitting because it keeps breaking
- 12:11same thing with pantalaimon
- 12:13my matrix client constantly has problems with pantalaimon because it doesnt support encryption (yet)
- George Antoniadis
- @penaiple:midov.pl12:15honestly, i cant say a single good thing about python or js
- magicofazi12:16I prefer rust over c
- @penaiple:midov.pl12:16i like functional software personally
- 12:16(stable software)
- 12:26@jwnz:matrix.org left the room
- Aaron Goldman
- @penaiple:midov.pl
- @neilalexander:matrix.org12:31Let’s not go completely off topic here
- i3xCx13:55So when blue sky going be open to public?
- pfrazee13:56we're not sure, but soon. We've started internal testing. We've got the app on testflight. We're fixing bugs and stabilizing protocol. Soon soon soon
- i3xCx13:56Nice looking forward to it
- 13:57Is blue sky going be like mastodon?
- pfrazee13:57in a few ways, but it'll feel different, especially over time
- i3xCx13:59Bluesky able to follow anyone on mastodons ,peertube etc?
- pfrazee13:59nah, different protocol
- i3xCx14:00Ok
- i3xCx14:06Is there a screenshot what bluesky going look like?
- pfrazee14:07I suppose I could do another small teaser screenshot
- whyrusleeping
- 14:32I can do a slightly better teaser
- pfrazee14:32woah woah woah that's proprietary
- 14:32"Chilé con Kitty" is copyrighted
- whyrusleeping14:32Fair use brah
- George Antoniadis14:35Awh; it's the oracle poem all over again. http://dacut.blogspot.de/2008/03/oracle-poetry.html (seriously though, can we make the "Chilé con Kitty" this part of the spec somehow? :P)
- pfrazee14:36haha no way about the oracle poetry
- 14:36not sure how we work chilé con kitty into it -- it's what you get when the cat comes in from the outside and their fur is cold
- George Antoniadis14:37I think the the bsky app/client needs a maskot :P One that maybe changes with the seasons? 😂
- pfrazee14:37haha jay was just saying that, I'd be strongly in favor of a kitty mascot
- 14:40there all set
- George Antoniadis14:51hell, I'm trying to "improve" this ^ with midjourey or stable diffusion and there's really no saving it 😂
- pfrazee14:51hahaha
- George Antoniadis
- pfrazee14:52hah sweet
- 14:53"Why is your mascot a cat in winter gear?" "Uhhhh"
- b0gg3r15:38Run sentiment analysis on the last 24 hours of posts and update the kitty's style accordingly
- George Antoniadis15:57^ yeah, basically ai generated logos, like google's doodles they add to their logo for special events
- Aaron Goldman23:19Might be worth trying to answer this question in the bsky model. https://twitter.com/NuttySwiss/status/1594358607835201537
- 23:31Ricardo Poh joined the room
Mon, Nov 21, 2022
- 02:46Nad // away changed their display name to Nad
- 06:01@yuenso:matrix.org left the room
- mikuhl10:10Me not making a Mastodon, Hive, Truth, whatever else there is out there, because I know a bigger storm is coming. 🤩
- @numero6:codelutin.com
- pfrazee10:33the only thing the team is building rn is a mobile app
- mikuhl12:48I might make something in Flutter, which can be compiled to Linux, Windows, Mac, iOS, Android, Web, Nintendo Switch, Toaster, Wall, etc. 😂
- 12:51thenameless removed their profile picture
- mikuhl
- 12:52For real Flutter runs on a wall. 😂
- 12:53evck joined the room
- mikuhl12:58that new Hive app everyones talking about is Flutter.
- 14:27kryptoshrimp joined the room
- boscovn
- 14:33Has that changed?
- mikuhl14:45
In reply to this message
I don't use linux so I am not sure exactly lol https://docs.flutter.dev/development/platform-integration/linux/building(edited) - mikuhl14:50oops accidentally put windows
- 16:55@sterence:sleeby.cafe joined the room
- 17:11arunwadhwa joined the room
- 17:41parhamb joined the room
- 18:55@sonotap:matrix.org joined the room
- jmcasey22:47
Been playing more with the MST in rust. It'll deserialize from DAGCBOR and can use rocksdb to verify CIDs.
I've seen other implementations (such as adenosine bnewbold ) which use a SQL-like db almost as a "presentation layer".
Because I'm naive (or optimistic?), I liked the idea of avoiding a db other than the blockstore. If CIDs can be checked locally, and retrieved from IPFS, then why can't we traverse the tree instead of querying for posts in a db?
First problem is retrieval time from IPFS, but that's a bit out of scope here.
The second problem is that given the CID of a repo, it takes between 3 and infinite calls to the blockstore to resolve a post.
The third problem is I'm having trouble deserialising into ATP native objects instead of Ipld objects using serde, so I'm currently making an additional blockstore call on each recursion.
Now that I have it somewhat functional, it seems too slow to be viable. Not sure if anyone has thoughts on this - it might be the case that an additional db is the right solution here.
I'm not super happy with it but you can see the code here:
(edited) - @planetoryd:matrix.org22:52a database can definitely be built on ipfs directly
- 22:54you can store indexed data in it and avoid copying data
- 22:56it's an open problem of how do we turn ipfs (or any protocol like this. eg. locutus) into a database (eg kv store)(edited)
- jmcasey22:57So something like CBOR > Tables of users, posts,etc > Db on IPFS?
- 22:58And I guess you would write to the tree, broadcast the changes, which are indexed into the db, which is queried for reads?
- Aaron Goldman23:02If you want an example of a SQL built on top of a keyvalue store I recommend looking at myrocks. http://myrocks.io/
- 23:07The collection is a collection of pages rather then records. The tables and the indexes are all b-trees on top of the pages. If you trust the publisher then you can use the indexes with the same confidence as the primary tables.
- committed23:07just got back from IIW conference, you might be interested in Authentic Chained Data Containers https://wiki.trustoverip.org/display/HOME/ACDC+%28Authentic+Chained+Data+Container%29+Task+Force
- 23:11As part of the Trust over IP stack there was a presentation on CESR - Composable Event Streaming Representation which solves a lot of the problems encountered when dealing with CBOR and codec interop and message signing in the context of streaming signed data.
- jmcasey23:12
In reply to this message
Sorry, who is the publisher in this scenario? The entity maintaining the db? - bnewbold23:21
In reply to this message
rocksdb is a database! just a simple key/value database. sqlite is also quite simple and makes a totally reasonable key/value store.
adenosine mostly implements the atproto stuff using just the blockstore (which is also in sqlite, but that is abstracted away, any key/value store would work fine). meaning of the com.atproto Lexicon.
it does use sqlite tables for DID documents. those are mutable, so if you want to put them in IPLD you would need to figure out something like IPNS on top. for that matter the DID-to-root-commit mapping is mutable and you need to figure out some way to store that mapping - 23:23parts of the bsky stuff are stored in sqlite directly, in the form of "secondary views" or indices into the repositories. sure seems easier to have things like a table of who-follows-who and being able to do queries against that to generate author feeds and timelines. and SQL databases give you things like types and relations (foreign keys) to help you get this right out of the box
- jmcasey23:25Yes! But I couldn't see how a key-val db could store records which could then be queried by key/uri - I thought it would need a separate db
- 23:26Do secondary views have implications for cache invalidation?
- 23:27As in, do they link directly through to the "blocks" or would they need to be updated each time a block is added to the db?
- bnewbold23:29the secondary views (indices) potentially need to be updated every time records are updated in a repo
- 23:30this is the same as how a SQL database internally maintains things like multiple indices on a single table
- jmcasey23:31Gotcha
- 23:31I see the appeal of ipfs_sqlite_block_store
- 23:32* bnewbold dinner time
Tue, Nov 22, 2022
- 00:58@sonotap:matrix.org left the room
- bnewbold
- jmcasey03:26iroh?
- bnewbold03:27oops didn't paste link: https://github.com/n0-computer/iroh/tree/main/iroh-store(edited)