Bluesky Dev
Community discussion of the AT Protocol and Bluesky. (This room is not officially affiliated with the Bluesky team.)
Previous group of messages
  1. whyrusleeping
    Deciding what that means is worth some thought
  2. Probably makes sense to write a few like “RFC” type issues describing things the community could be helpful on
  3. mikestaub

    In reply to this message

    That sounds good. There are 600+ people in this room now, perhaps we should do a poll and ask "What level of commitment to contribution are you willing to make per week? ( 40 hours, 20, 10, 3 )" If there are less than 10 devs at this time perhaps this can wait. I just wanted to do a pulse check and see where things are at.
  4. blackfreethinkers joined the room
  5. bnewbold
    re: the earlier pfrazee comments earlier about DMs and private messages, as a thought experiment I would propose that atproto just not support any form of non-public messaging. keep the protocol small and focused. leave private messaging (DM or group) to existing applications/protocols like Signal and Matrix, don't try to compete with them. an application (like a mobile app) could potentially support these other protocols with shared contacts. some glue to bridge DIDs as persistent contacts/identifiers for those other mediums would be helpful
  6. I guess i'm not a particularly active user of any modern social media tools, so maybe i'm missing things, but it always feels like the private messaging feature is bolted on and undesirable in many cases. what people do want is to have reasonably strong/safe linking between identities, so you know if you are contacting XYZ on twiter or bsky via email or matrix, you would actually be getting the same person. but that is trivial with a simple link or profile entry.
  7. the important privacy properties for private messaging, like minimizing side-channel metadata leakage and deniability (refutability, perfect future secrecy) are kind of hard to get right, I feel like
  8. bnewbold
    a docs request (which I get is sort of low priority at this stage as things are still in flux) would be extending the existing identity docs to explicitly talk through how rewriting history would (or would not) work. aka, if a user or PDS starts publishing a new history chain, with a valid signature (verified by current DID doc verification key), are other PDS servers immediately drop all prior histories? how are other servers supposed to verify if a new "HEAD" is the now-canonical HEAD? eg, if there are multiple branching HEADs. the kind of thing i'm thinking about is replicating from multiple third parties (maybe aggregators, or caches, or "high-availablilty persistence services" like P's old hashbase for dat) and getting conflicting responses. or receiving a random repo PUT request which is valid but maybe out of date. should receiving PDS servers always do a quick sync root request to the DID's canonical PDS?
  9. ^ this is ecosystem-impactful
  10. somebody above was discussing trying to block indexers/aggregators. my current feeling is that a distinction between atproto and a large subset of the mastadon community, is that atproto is welcoming of "big world" aggregation (in addition to "small world" hosting stuff), while in contrast some think that "big world" stuff is potentially undesirable on principle, and are skeptical of random aggregation. eg, I think of atproto as being closer to a service like tumblr, which as I remember/understand was totally open to being crawled and indexed by things like Google, as compared to Twitter which was originally open but ended up having very restricted ($$$ or grandfathered) access to the firehose, lots of expectations about not storing user content that may have been deleted, etc. I think there are legitimate concerns and arguments in both directions; the way I have been thinking of atproto is in the "sure anybody can aggregate, user content is public and users need to keep that in mind" direction
  11. some tangential references to the indexing/crawling issue are:

  12. auraelion joined the room
  13. @cmoog:matrix.org joined the room
  14. Charlie Moog changed their display name to cmoog
  15. @wclayferguson:matrix.org
    bnewbold: I agree, the "Minimum Viable Protocol" should definitely not include support for private and secure encrypted/secret messages, in version 1.0, because that adds an order of magnitude to complexity, and what people around the world were kinda hoping for is a simple protocol for sending messages, that's somehow or other superior to ActivityPub.
    (edited)
  16. It needs to be simple enough that a developer can understand it after 30min of reading. Not 30 hours.
    (edited)
  17. @wclayferguson:matrix.org
    IMO, all that was really needed, from my perspective, was an ActivityPub-variant that used a PublicKey as identity rather than a DNS-based name.
  18. @planetoryd:matrix.org

    In reply to this message

    It doesn't support ring signature i'm disappointed
  19. bnewbold

    In reply to this message

    The way I think about it is that atproto talks about the canonical representation of user content, and how to replicate it, like git repositories (any checkout is as good as any other; there is some awkwardness about "completeness" with large blobs/media and sub-repos). While ActivityPub is about notifications and feeds, like RSS (original content is often simplified or truncated in RSS feeds, though sometimes it does not matter).
    DID and account migration are another big difference, but feels more up in the air with the DID "placeholder" mechanism not being a robust proven tech. did:web is a good backup, any extremely trivial (I guess a good thing?), but does not deliver on the interesting new account migration stuff (just falls back to DNS)
    (edited)
  20. whyrusleeping
    Adding new did methods will definitely happen at some point, PLC is just the only supported method for now to make all the development smoother
  21. @wclayferguson:matrix.org
    bnewbold: With ActivityPub you can lookup a user, find their outbox, and just simply read it. The problem with it is that the DNS owner owns your username, and is therefore your overlord. People are tired of overlords and want to own their own data.
  22. whyrusleeping
    Im not really interested in having more activitypub discussions tbh
  23. Weve had those
  24. weve moved on
  25. @wclayferguson:matrix.org
    Also having names built into URLs it makes it hard for someone who doesn't know how to setup a DNS name (SSH) and run their own complete server is a problem. So using a PublicKey like ATP is doing is good.
  26. whyrusleeping
    Yeah, usernames/handles need to be extremely simple for people to use and setup
  27. bnewbold

    In reply to this message

    huh, I haven't seen much interesting atproto/ActivityPub discussion here or elsewhere, just a lot of noise. but if it isn't wanted here will take it elsewhere!
  28. @wclayferguson:matrix.org
    I'd like to see something where a non technical person (my mom even) can just run a piece of software and start participating in social media conversations. Personally that's my ideal scenario.
  29. George Antoniadis
    I can see a bot in this channel's future that auto replies to anything with "activity pub" with a link to the faq xD https://atproto.com/guides/faq#why-not-use-activitypub
  30. whyrusleeping
    bnewbold: youre fine, dont worry. Im just wanting to steer things towards discussion of ATP and away from AP
  31. If a make a broad statement like that i don’t really mean to call anyone out
  32. @wclayferguson:matrix.org
    By pointing out the flaws in AP, that people don't like about it, that will help make ATP at least be aware of what users need and want.
  33. whyrusleeping
    Yeah, i want to find the right balance
  34. Too much AP comparison feels weird, none also feels weird
  35. On the private messages discussion, i tend to agree that it should be a separate protocol
  36. The way to do identity should allow people to build messaging applications that integrate with ATP identities
  37. But we shouldnt focus on it up front
  38. We’re about the public conversation afterall
  39. Aaron Goldman

    In reply to this message

    #general-bsky:matrix.org
    for dSocialCommons is meant as lobby for many approaches to compare and hopefully copy each other as distinct from the Dev channel that is focused on building this project.
  40. bnewbold
    here is a nitty-gritty question to try and keep the conversation back at the "dev" layer:
    why are the publicKeyMultibase keys in did:plc docs so much longer than the did:key-encoded keys? I suspect this is a DID thing not specifically an atproto or did:plc thing, but nothing obvious turned up, I think they are both supposed to be base58btc encoded
    eg, looking at: https://atproto.com/specs/did-plc#example
  41. mikestaub

    In reply to this message

    for PMs I don't think it makes sense to invent something new but rather use signal or peergos
  42. @wclayferguson:matrix.org
    I say there's huge benifit to also focusing on "Public Append-Only" content for the first MVP. Merkle trees make the syncing perfect and simple. No one knows that around here better than whyrusleeping because of his IPFS expertise! (oh, and me too). Once you start trying to do editable data a vast array of difficult problems come up instantly and multiply complexity 100x.
    (edited)
  43. And if it's done with PublicKey identities you can tell other "unnamed protocols" they're just obsolete now. hahah. :)
  44. one final point, it can be the simplicity that makes ATP "better". Be simpler than Matrix for example. Simple is indeed a feature in itself.
  45. `did:key:zDnaeSezF2TgCD71b5DiiFyhHQwKAfsBVqTTHRMvP597Z5Ztn` Is an entire key and is better compared to the object rather than the key bytes. ``` { id: 'did:plc:7iza6de2dwap2sbkpav7c6c6#signingKey', type: 'EcdsaSecp256r1VerificationKey2019', controller: 'did:plc:7iza6de2dwap2sbkpav7c6c6', publicKeyMultibase: 'zSSa7w8s5aApu6td45gWTAAFkqCnaWY6ZsJ8DpyzDdYmVy4fARKqbn5F1UYBUMeVvYTBsoSoLvZnPdjd3pVHbmAHP' }, ```
  46. Definitely get the points for a technical question for that one 😁
  47. whyrusleeping

    In reply to this message

    Yeah, we need something thats easy to integrate, works with our identity system, and that is scalable to the user load we are expecting to see
  48. In reply to this message

    Yeah, we've realllly been trying to keep things as simple as we can make them. A lot of time was spent trying to toe the line between 'as simple as possible' and 'not being obsolete in the next year'
  49. trying to maximize the pair of (simplicity, flexibility)
  50. onlytangerine joined the room
  51. whyrusleeping
    onlytangerine: why not grapefruit or kumquats?
  52. Dean

    In reply to this message

    Well it's working so far. I've been able to use the provided APIs fine with the dev-env server!
  53. It's actually super fun to work with the protocol and not have to worry about the back-end.
  54. jmcasey joined the room
  55. jmcasey
    Did the createRecord parameters change? Getting "{"message":"Unknown parameter: did"}" and can't see where in the code it's doing that check
  56. Steven Franssen

    In reply to this message

    but there is those that say append only can't scale
  57. Aaron Goldman
    "Append only" is just one of the persistent data structures.
  58. @wclayferguson:matrix.org

    In reply to this message

    It could just wrap each "month's" posts into a separate ZIP/CAR file and each one can point back to the merkel root of the previous. Just like how Blockchain works and is trustworthy even if you're not running a full node.
  59. Everything in the world that's got large data usually has some kinda paging. Letting months or weeks be the paging is super clean and simple.
  60. especially since this is definitely a time-series of data if it's append only.
  61. Aaron Goldman

    In reply to this message

    The problem with an append only block chain is that purge requests become difficult to honor.
  62. The tree is better if you may want to remove something much later you just recalculate the path from root to leaf instead of all subsequent pages.
  63. At least in my mind this is the main win from separating the DID Document deltas from the repo. You need the full history of the DID Document to validate the current state. You don't need the history of the repo just the latest commit and tick. Then it's a Merkel tree of the current state.
  64. mikuhl
    I am going to spontaneously combust because I am so excited to use this protocol!!
  65. Aaron Goldman

    In reply to this message

    Um ... Thank you? 😕
  66. mikuhl
  67. @wclayferguson:matrix.org

    In reply to this message

    Two things:

    1. you jumped to "mutable data" when I was talkin' about immutable. :P
    2. but if we look at the mutable case. The master "record" be a single list of all the root merkels for every month of a person's life. What that means is each "rolled up" monthly chunk of data can be updated and it simply results in a new root merkle hash for that one month ZIP/CAR file.

    The benefit if the "by month" approach is that when I want to follow someone, I don't get ALL their data unless I ask for it...but I do always get that "master record" which is essentially a list of monthly hashes.

  68. One merkel tree per month.
  69. the only thing that makes this stuff "big" really is the images/attachments. You could probably store someone's full social media history of typed text in a single ZIP and not even break it up.
  70. IF these decisions have already been finalized let me know and I'll shut up. :)
  71. @wclayferguson:matrix.org
    BTW. mutable data doesn't work well with Social Media anyway. Somebody can say "Do you like puppies" and 100 people reply yes, then they go back and edit it to ask "Do you kick puppies". It just doesn't work for pure Social Media use cases.
    (edited)
  72. You really just need a permanent record of who said what and when to have conversations where people aren't driven insane by "revisions of history"
  73. Even if no one has replied to a post yet people can share links to it and say "Look at this thing I agree with"....and then somebody changes said thing to some form of evil. It just cannot work.
  74. Sorry to flood the room with text. I'm done I hope. :)
  75. whyrusleeping
    Bruh
  76. mikuhl
    now that was a TED talk :)
  77. @wclayferguson:matrix.org
    I didn't wanna get too persnickety about how things need to be done....but I failed.
  78. whyrusleeping
    Okay, ill read the text wall, hang on
  79. @wclayferguson:matrix.org
    It's about puppies. :) TL;DR....the issue about the "time window" in which editing should be allowed.
    (edited)
  80. whyrusleeping
    So, in terms of “things that have been decided”, posts are all in a big merkletree keyed by their “TID” which is basically just a timestamp. This makes it really easy to filter the tree by a specific time range
  81. How blobs get stored and linked in isn’t necessarily figured out yet, its pretty tricky tbh
  82. @wclayferguson:matrix.org
    gotcha. thanks.
  83. whyrusleeping
    Posts are inevitably mutable, but replies (IIUC) reference both TID and CID, which means you can detect if a post has changed, and application logic can figure out what to do
  84. @wclayferguson:matrix.org
    so anyway if the content changes the CID changes too. so...there's that. In a way it means everything is immutable only in the sense that the CID chagnes if an edit happens.
  85. whyrusleeping
    Yeah, just confirmed, replies reference both cid and a “mutable reference “
  86. @wclayferguson:matrix.org
    ah, ok, that extra level of 'indirection' is allowing the edit then. got it.
  87. reference==indirection
  88. whyrusleeping
    Yeah, allows it but makes it detectable
  89. @wclayferguson:matrix.org
    that's how my own platform works too. If you edit then the digital signature changes. But my platform is more of a CMS than Social Media. In social media really the simplest challenge to solve in MVP is the "Immutable Public Data" case.
  90. What i meant was I have a "permanent" record ID that isn't a hash.
  91. but that is almost like a wrench in the gears of Social Media if people can "revise history" in the middle of a discussion for example.
  92. (in other words the "puppy problem"....of revising history)
  93. whyrusleeping
    Only if the application doesnt do anything about it
  94. @wclayferguson:matrix.org
    I guess the issue is once something is posted it has a URL, that can be shared, and people can say "look I agree with ABC", and then ABC changes.
  95. whyrusleeping
    Same problem as replies really
  96. @wclayferguson:matrix.org
    I haven't "Solved it" in my own app either. I'm just saying it's a justification for immutability.
  97. But yes, it's same exact problem as replies.
  98. whyrusleeping
    Urls could contain both references too, and the application can display a warning
  99. @wclayferguson:matrix.org
    yep.
  100. or just require a delete an repost new message.
  101. but that's same as 'immutable' so....i retract that. lol.
    (edited)
  102. anyway thanks for listening. just wanted to raise that issue.
  103. whyrusleeping
    Yeah, its really important, and one of the first things i was thinking about when proposing the repo architecture
  104. gonzo_marx joined the room
  105. Steven Franssen

    In reply to this message

    so append only edits are not considered mutations?
  106. @wclayferguson:matrix.org
    In my mind I'd say an append-only edit is not a mutation. correct. You're not editing your original words (which could the change meaning even if you only change one letter of the text) It's impossible for software to know if you're just correcting a typo or possibly intentionally 'inverting' the meaning in some clever way.
    (edited)
  107. valere joined the room
  108. plywooder joined the room
  109. @kf.walkow:converser.eu left the room
  110. @jaklt:tchncs.de joined the room
  111. mimonelu joined the room
  112. Jorge Eduardo da Silva Figueiredo joined the room
  113. Jorge Eduardo da Silva Figueiredo
    Olá
  114. Aaron Goldman
    The question of wether the repo is mutable or immutable comes down to the reference URI not the repo itself. Let's take git as an example: The git hub url `https://github.com/bluesky-social/atproto/blob/d44cee71139af2267996f8b4c9412cde099ea37b/README.md` contains the hash of the commit. If I want to validate the blob at that url I can pull the Commit, root tree object, and the blob. The sha1s prove the contents of the blob are the blob referenced in the URL. Even if GitHub disappeared tomorrow If I have the URL and the repo I can still prove this. On the other hand the URL `https://github.com/bluesky-social/atproto/blob/main/README.md` references `main` this is mutatable as it is referencing whatever
    github.com
    believes is the head of main. If GitHub disappeared tomorrow this URL is meaningless or at best means anything that was ever served from it. So is a git repo mutable or immutable? If we were designing it today which type of reference should we use? To get from mutable to immutable We need to know the commit hash. To get from immutable to mutable We need to know the branch name. Both references feel somehow incomplete. The immutable reference doesn't tell me a authority to get the latest value and the mutable reference doesn't give me a cryptographic hash to get the bytes as they existed at the time of the reference. `at://repo_did/collection/record_tid` doesn't give you a hash for the commit. `at://commit_hash/collection/record_tid` doesn't give you the DID to find the latest version of the repo. (Note: Not the actual ATP URL just making a point. Still well defined URL parsing since https://username@host/path is a valid url) `at://repo_did@commit_hash/collection/record_tid` would give you both. It is an immutable reference because you can use the commit hash to know exactly what the bytes were but it is also a mutable reference since you have the DID to find the latest version of that record.
    (edited)
  115. Dean
    Something I'd like to see is native support for horizontal scaling on ATP indexer servers considering these are most likely to draw the majority of bandwidth. It would help when there's a sudden influx of users to a specific indexer/name provider. I'm sure this is already in consideration, I'm just writing down thoughts as I'm coding with the current 0.0.1 protocol.
  116. Aaron Goldman
    Not pick: slightly more accurate to refer to hash identified content as semi-immutable having only two states the content and don't know. The repo model lets you redact data from the repo by deleting the blob without invalidating the rest of the content proofs. It's just that that one record @ that commit is now "don't know"
  117. marios joined the room
  118. @wclayferguson:matrix.org
    Aaron Goldman: I think you're right. My statement of immutable v.s. mutable was highly specific. The terminology fails us in this case I think, too. The problem to be solved is the "puppy problem" where somebody says, do you "like" puppies, gets a bunch of replies, and then edits it to say "kick" instead of "like". I guess the replies need to point to the original post and not be carried forward to the "edited" post?
    (edited)
  119. I think whyrusleeping is aware and has is solved probably. Thanks for clarifying so thoroughly..
    (edited)
  120. Aaron Goldman
    at://repo_did@commit_hash/collection/record_tid at://handle@commit_hash/collection/record_tid Is slightly different from the current implementation which has the URL and CID are different fields but yes including the CID is good for preventing the puppy attacks you describe. Not yet clear how we should get both the repo ID and commit CID into the same URI.
    (edited)
  121. @lisko:matrix.org left the room
  122. @wclayferguson:matrix.org
    perhaps leaving out the 'commit_hash' can make it pull up the 'latest commit'?
  123. (not saying it solves 'puppy problem' tho)
  124. BUT including the commit_hash would basically "solve it", by being commit specific.
  125. another approach possibility:


    Every node has a "type"

    1. "Type=Comment" nodes are be immutable. You cannot change a comment. Required to delete and repost new one (resulting in new record ID).
    2. "Type=Paragraph" (like some chunk of "long form" content/document), should be mutable, because it's like a blog, or document.
  126. bnewbold

    In reply to this message

    yeah, it would be nice to have a way to encode the commit CID in at:// URIs. or, just add as a field in references, along with the URI and the content CID.
    otherwise the CID reference in mentions/links are a bit "deeplink" and require a big index of record-CID to commit for verification. AKA, it is the commit that has a signature and can be validated.
    not a huge issue, there is a way to validate with the way things are today, just takes a bit of extra work
  127. in implementing a PDS, i'm leaning towards retaining all the commits and MST nodes Forever, immutably. record contents are what could get purged/redacted/hidden in response to abuse or legal issues. that way things like references can be confirmed to have actually linked to an actual thing, even if the thing isn't returned
  128. Daniel Holmgren
    hey just saw this one: the reason is that did:key is encoded with the compressed 33byte public key and publicKeyMultibase is meant to be the uncompressed 65 byte public key
  129. ope aaron actually got to it already 😛
  130. bnewbold
    I'm curious what the "scene" vs. "user" repo types are. "scene" is like... a group?
  131. pfrazee
    we're leaking alpha!
  132. scenes are like a group, yeah
  133. George Antoniadis

    In reply to this message

    "leaking alpha" as in bsky app alpha? :P
  134. Daniel Holmgren
    but better 😜
  135. pfrazee

    In reply to this message

    heh yep
  136. George Antoniadis

    In reply to this message

    Is "dibs" still a thing in 2022? :D
  137. pfrazee
    hah oh to be clear, "leaking alpha" means revealing details of the unfinished app
  138. George Antoniadis

    In reply to this message

    Awh got too excited for a sec there hehe, I'll go back to my coding 😂
  139. pfrazee
  140. I'm working on the onboarding copy now...
  141. the details have been blurred to protect the ~innocent~ alpha
  142. not exactly easy to surprise people with features when our backend is foss
  143. but a couple technical details of those recent updates, since I'm here!
  144. an "account" -- as in, some entity in the network with a repo and an identity -- is now generically referred to as an actor
  145. actors are typed. "User" and "Scene" are the two kinds of actors we have atm, but you can bet that'll grow
  146. this means that scenes behave much in the same way as regular users. They have their own handle, their own profile page, their own feed, etc
  147. the actor type gets declared in the app.bsky.system.declaration record
  148. you'll notice declarationCid variously in the APIs, eg on follows. This is the CID of that declaration record. It's so software can detect if the actor type changes -- so if you follow a scene, it doesnt become a user in a surprising way
  149. the declaration is meant to be immutable but there's no way to enforce that other than this kind of referential integrity check (the declarationCid)
  150. you may also notice two new record types, app.bsky.graph.assertion and app.bsky.graph.confirmation. This is a generic utility for asserting social-graph relationships bidirectionally
  151. we'll use this in a number of things, but scenes are membership-based and require both sides to agree with the membership. (And either can revoke it)
  152. so in essence the scene publishes an assertion of membership, and the user publishes a confirmation of that assertion
  153. so the "follow" record type represents a uni-directional graph relationship, while the "assertion/confirmation" types represent bi-directional graph relationships
  154. if you like this, it was all my idea. If it seems too complex, this was all Daniel Holmgren's idea
  155. (jk obviously, I'm pretty confident this will work well and have a lot of utility)
  156. this all represents some of the interesting, more sophisticated data modeling you have to do with independent actors publishing
  157. the CID usage for referential integrity == "how do we handle information reliability in a distributed setting"
  158. the bidirectional record linkages == "how do we coordinate action between actors in a distributed setting"
  159. it's a different mindset than centralized services to be sure
  160. iafarouk joined the room
  161. George Antoniadis
    Are users and scenes different types? I can see app.bsky.system.actorScene but not app.bsky.system.actorUser.
    If different, do the schemas support some way of either inheritance for relationships (ie you can "like" all "actor" types) or maybe just allow for unions (ie "actorUser" or "actorScene")
  162. pfrazee
    they're different "actor types." We dont have formal inheritance in lexicon, but that's the implied relationship
  163. the way to think about ATP's records is more akin to a database, and lexicon is providing the schemas for the database. We don't formalize OO-style inheritance in lexicon for the same reason, say, SQL doesnt
  164. that said, lexicon is also used to generate code/apis, and you can produce APIs which represent the implied concepts in the way you're describing
  165. @wclayferguson:matrix.org
    Makes sense that a User and a Scene are kind of both at the top level (root) of a sort of hierarchy, that can contain all of the details inside it. Sounds good. Thanks for the update!
    (edited)
  166. justthisguyatx joined the room
  167. Alejandro_
    I'm curious about how a closed beta can work for a decentralized, open-source thing? Will there be only one instance for testing? will a limited amount of people be able to create an instance and make a federation?
  168. pfrazee
    yeah it's a good question. It's specifically: access to our mobile app, and access to our instance
  169. and while a lot of the foundations for federation are done (see above for all the ways we model the data in preparation for it) the actual server-to-server sync protocols haven't been implemented yet
  170. we know exactly how the sync protocols will work, but saving it for the last task before the open beta allows us to understand all of our requirements before implementing it
  171. I really enjoy building fully in the open and normally would -- I live streamed my last project's development and I'm kind of dying to do that here -- but bluesky is in a kind of weird position where there's a lot of press attention, and we really didn't want "bluesky unveils a hot box of garbage" headlines
  172. George Antoniadis
    pfrazee: missed the led glasses xD
  173. pfrazee
    plus it's really hard to communicate with a community while primary development is happening (notice some of the enormous change in recent PRs) and this is meant to avoid miscommunicating how stable things are
  174. Alejandro_

    In reply to this message

    i see, thanks for answering
    (edited)
  175. pfrazee

    In reply to this message

    hehe meee too. Were you a part of the ctzn crew??
  176. In reply to this message

    sure thing
  177. George Antoniadis

    In reply to this message

    lurked mostly, loved having them on while working, they were really good fun.
  178. pfrazee
    nice. Those were good times
  179. I'm practically begging jay to let me livestream again so ... hopefully that can happen
  180. George Antoniadis
    Once the cat's out of the bag I guess it might be easier to find some time to work on secondary projects around bsky. Or is the bsky app going to be open source as well?
  181. pfrazee
    we're definitely going to FOSS a reference client and all the tooling we use. Not sure yet whether every piece of the client will be though
  182. Dean

    In reply to this message

    Is the current spec (as described in the docs) stable enough to work on front-ends and not expect a complete rewrite from the existing ATP protocol spec? I noticed some things already differ (e.g. session helpers and did/cid arguments for lookups) but the method calls are the same.
  183. pfrazee
    the core concepts are pretty darn stable, but I imagine building a frontend right now would suck
  184. for instance, last week we changed the NSIDs of every schema, which would break everything
  185. Dean

    In reply to this message

    I started playing around with it this weekend and the experience has been great so far!
  186. Aaron Goldman

    In reply to this message

    for
    at://repo_did@commit_hash/collection/record_tid
    or
    at://handle@commit_hash/collection/record_tid
    the "big index of record-CID to commit" is not like searching IPFS the RepoID (DID or handle) is also there so you can look up the home server and then pass the url with the commit_hash to it.
    The home server is the one that keeps the old commits, path nodes, and records to return for a uri request. Or it can respond that this record has been deleted. Probably we would want it to return not only the exact record but the current record for the same path in the latest repo.

    This also serves as a guard agents the handle being reassigned by the DNS system. If the commit hash matches then it is the content that was the handle at the time the link was generated. Irrespective of wether the publisher later decided to stop paying their DNS name bill.

  187. Dean changed their profile picture
  188. atnorl changed their display name to xonampo
  189. xonampo changed their display name to atnorl
  190. whyrusleeping

    Fun things you can do already, once you export your repo to a car file (using com.atproto.sync.getRepo) you can import that into ipfs and explore the data:

    123456789why@sirius ~/c/gosky (master)> ipfs dag import repo.car
    Pinned root	bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe	success
    why@sirius ~/c/gosky (master)> ipfs dag get bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe
    {"root":{"/":"bafyreifslfzyxiaqeaeeoydnb7fwmxq7cwjqrqtbvjw2rbq7ibarnl4fui"},"sig":{"/":{"bytes":"aaRPVJa/QRbhCOuUTJS8M7x/KEgpBW25HHfLVZvB9bTqrfDNMuESuVdxdT1SbV0p5FfVFpYLsL5KyXejjTP2dA"}}}
    why@sirius ~/c/gosky (master)> ipfs dag get bafyreifslfzyxiaqeaeeoydnb7fwmxq7cwjqrqtbvjw2rbq7ibarnl4fui
    {"auth_token":null,"data":{"/":"bafyreife3hhlk6ukkpjjvp3fmn4eocqfczlf472kgjgzfygz3bjsw43l3u"},"meta":{"/":"bafyreidevl4oqvw4gfoxnyctsr373zi7dnnj4pjkbh4nclubkqaf37vtvq"},"prev":{"/":"bafyreigc5zp7rzfuuxoylmbsdj4meog6esxuqgpehty6g466f2flxnj4ra"}}⏎                                                                                                       why@sirius ~/c/gosky (master)> ipfs dag get bafyreife3hhlk6ukkpjjvp3fmn4eocqfczlf472kgjgzfygz3bjsw43l3u
    {"e":[{"k":"app.bsky.feed.post/3jgwsf5q64c2z","p":0,"t":null,"v":{"/":"bafyreig4s2bn7yjo2i2r2gut7uvxvvcgsyf4jcyndcbnjdm5jkjx35o6yu"}},{"k":"tjxhkc2z","p":24,"t":null,"v":{"/":"bafyreiduauw76xk4gz3srdhdnkwzmwuisviqp72nyia3crklgxwhddv7ga"}},{"k":"system.declaration/self","p":9,"t":null,"v":{"/":"bafyreid27zk7lbis4zw5fz4podbvbs4fc5ivwji3dmrwa6zggnj4bnd57u"}}],"l":null}
    why@sirius ~/c/gosky (master)> ipfs dag get bafyreiejwkm2e5j2cf7ft5gt673bkjb56qkhm5r7ylddzjvarcf6pm3awe/root/data/e/0/v
    {"$type":"app.bsky.feed.post","createdAt":"2022-11-07T11:04:03-08:00","text":"i like cats and dogs too"}
    
  191. Scott Edwards joined the room
  192. sables joined the room
  193. Aaron Goldman
    Message deleted
  194. Aaron Goldman
    it is a UX chalenge that there are three ways to follow
  195. do you want what was linked to
  196. latest from that publisher
  197. or the latest with that name
  198. If they all match fine but if that don't it might make sence to warn the user that the handle changed control between the time the link was created and now
  199. Aaron Goldman
  200. @wclayferguson:matrix.org
    that was too hard for meh. :)
  201. I just want to "click" reply and it does whatever. haha.
  202. not that I know what "whatever" means.
  203. lolz.
  204. Aaron Goldman
    The person clicking reply should not need to know any of this
  205. Bruno Hahn Gonçalves joined the room
  206. jinglescode joined the room
  207. Ömer Aykol joined the room
  208. boscovn joined the room
  209. Anders C. Sørby
    I want to create a rust server node which is both compatible with ActivityPub and ATproto. It will combine Web3 stuff with the existing fediverse. It is not a bridge but a node. Using nixos to make it easy to deploy. IPFS and Filecoin support. GraphQL API. Support for wallet logins. Calling it rustyverse now. https://github.com/Anderssorby/rustyverse
  210. Anders C. Sørby
    Potentially compilable to WASM
  211. Intends to use
    surrealdb.com
  212. @bradstewart:matrix.org left the room
  213. Eduards Puķins-Slava joined the room
  214. @rimuru:gentoo.chat changed their profile picture
  215. Hacklet joined the room
  216. Steve joined the room
  217. @neilalexander:matrix.org joined the room
  218. mikestaub
    pfrazee: what tool will be used to manage the schema lifecycles? I think something like https://github.com/odpf/stencil will be needed
  219. hacklet changed their display name to Hacklet
  220. Hacklet set a profile picture
  221. Børlaag changed their profile picture
  222. William Bernardi joined the room
  223. pfrazee
    stencil is interesting. I'd like it to be a self hosted protocol (meaning you could ask https://sync.atproto.com for the com.atproto.sync.getRepo definition)
  224. then that opens the door to somebody running a registry service
  225. Jackson White joined the room
  226. whyrusleeping
    Yeah, some sort of “well known” convention for storing the schemas
  227. Versioning will be interesting too
  228. pfrazee
    I was thinking we'd bake some methods into the xrpc spec for it
  229. whyrusleeping
    So meta
  230. Jackson White
    beta still open?
  231. pfrazee
    versioning is effing hard. After spending a lot of time on it, I specced a "revision" field on lexicon which is a simple number. It's meant to only be used for changes that are forwards AND backwards compatible
  232. George Antoniadis
    schema versioning on decentralised protocols is the stuff of nightmares
  233. pfrazee

    In reply to this message

    signups at https://bsky.app, we havent started the closed beta yet but we're v close
  234. Jackson White
    ok gotcha
  235. pfrazee

    In reply to this message

    it is. Continuing from before, the revisions don't allow significant changes to the schema. My thinking is, if you need to make forwards OR backwards breaking changes, you create a new schema and stick a v2 on it
  236. in essence, once a schema is out there, the only thing you can do is add optional new constraints
  237. I wrote up some stuff about this and will talk more about it in the future, but yeah in a decentralized setting you are very constrained on what changes you make to a schema after you publish it
  238. bnewbold
    I feel like inevitably there will be a bunch of schemas like app.bsky.v2.feed.post (or some variation of inserting the "v2"). the old golang namespace situation comes to mind
  239. pfrazee
    almost certainly yeah
  240. cruft is guaranteed in this line of work
  241. whyrusleeping
    Welcome to the jungle
  242. Jackson White
    interested to see how schemas r handled
  243. pfrazee
  244. In reply to this message

    we have a fair amount published on https://atproto.com
  245. Jackson White
    versioning*
  246. pfrazee
    ah yeah. Well I pretty much described the game plan above
  247. committed
    pfrazee: are schemas stored in IPFS? do they have CIDs?
  248. pfrazee

    In reply to this message

    No. And -- this is confusing but -- we don't use IPFS technically, we use IPLD. So by default, nothing in ATP is accessible via IPFS, and we have no plans to make that part of the spec
  249. we looked at using CIDs to identify schemas but felt like that was more rigid than we need, and it suffers from the downside of having to reference the types by huge opaque numbers
  250. @julian:one.ems.host joined the room
  251. George Antoniadis
    Pretty sure this has been answered before but since it's not in the FAQ (and can't find it using search here) I'll ask again :D How come you didn't go for a full URI instead of NSIDs? Since you're sticking to HTTP for the foreseeable future (I assume?) you might as well.
  252. pfrazee
    you can think of ATP as an addressable space that exists atop HTTP. It's an abstraction over it, in the same way TCP is abstracted by HTTP1 & 2
  253. HTTP could be swapped out, much like how HTTP swapped out TCP in recent versions
  254. critically, the at:// address space identifies data independently of the host machine
  255. @planetoryd:matrix.org
    immutable storage ? or just like ipns
  256. pfrazee
    the best analogy for data repos is git repos
  257. @planetoryd:matrix.org
    whats the difference compared to locutus ?
  258. pfrazee
    I dont know locutus
  259. @planetoryd:matrix.org
    locutus is dynamic but the state is bytes
  260. In reply to this message

    plz read about it https://github.com/freenet/locutus
    (edited)
  261. whyrusleeping
    its all built on freenet?
  262. @planetoryd:matrix.org

    In reply to this message

    its a new protocol and network
  263. whyrusleeping
    sure but its basically still freenet, which is a not-great protocol base for a number of reasons
  264. @planetoryd:matrix.org
    not the same thing at all
  265. it doesn't have the anonymity routing, or deniability kind of thing
  266. whyrusleeping
    but it still has the same basic trust assumptions that puts a ton of work on people running nodes in the network
  267. @planetoryd:matrix.org
    just DHT with arbitrary function of verifying the values (for ipfs the blocks are supposed to match the CID
  268. whyrusleeping
    very vulnerable to churn causing huge disruptions in the QoS
  269. freenets big problem was that it relied on a bunch of altruistic people hosting content for others, with no clear incentive mechanism, and no way of guaranteeing a particular level of service
  270. George Antoniadis
    @planetoryd:matrix.org: is there a specification for Locutus? Docs are not really doing a good job at explaining how it works, and there are a lot of references of "contracts" which is not promising. ps. Should we move this to #general-bsky:matrix.org ?
  271. whyrusleeping
    locutus' docs are extremely light, but it seems to say "use a DHT-like thing to store key-value pairs, and have some web-assembly code to do... something"
  272. and yeah, best to move this to general
  273. manish Wadhwani joined the room
  274. jmcasey
    I'm working on a rust crate for ATP: https://gitlab.com/jmcasey/rust-atp/ Any feedback/advice appreciated
  275. pfrazee

    In reply to this message

    nice. I'd suggest starting with implementing the repo data structure, which is a combination of IPLD and a Merkle Search Tree implementation. That's relatively stable and is going to be core to any implementation
  276. jmcasey
    Yup, that's the next big step. Right now it's just making XRPC calls so I can get a sense of what types we're passing around feel like
  277. IPLD/CBOR/MST needed before it can be used in a PDS
  278. pfrazee
    cool. whyrusleeping has been working on a golang library and generating APIs from the lexicons, much like we do for typescript. Consequently, he's been giving us good feedback on how we can shape the schemas to make that easier in a more strictly typed language
  279. jmcasey
    Awesome. Will take a look, I'm new to rust so I'm sure I have some naive approaches
  280. pfrazee
    this will be a fast way to learn! I've never coded in it either. I'm turning into a typescript-only dinosaur lately
  281. jmcasey
    I know it's a meme, but so far I've really enjoyed it.
  282. The compiler is a character in the show
  283. pfrazee
    we'll see if I ever get a chance to build in it. My 5 year is probably going to take me away from code so I may never get the chance
  284. simonearle joined the room
  285. Nad changed their display name to Nad // away
  286. Dean
    It looks like bsky's getProfile() method requires authentication currently. Wouldn't it make more sense to have that as a public method so profiles are publicly viewable? Or will all bsky related methods always require authentication?
  287. @wclayferguson:matrix.org

    In reply to this message

    I'll go a step further and say all public data should be able to be consumed anonymously.
  288. bnewbold

    In reply to this message

    I have a partially working Rust PDS and CLI client implementation at https://gitlab.com/bnewbold/adenosine
  289. the MST and generic XRPC stuff is mostly working, and i'm part of the way through app.bsky feature stuff. I'm not doing real validation against the Lexicons, or code-gen. and the whole thing could use refactoring/iteration. but close to working as a proof of concept. I expect the specs to continue to change at more than the Lexicon level, so I would be hesitant to invest much time in to a polished implementation. eg, how inter-PDS communication will work feels unspecified so far, and could have a large impact on how to architect things
  290. pfrazee: as a request for the lexicons, the "handle" cleanup is good, but I think there are still a bunch of "user" and "name" fields. It would be nice if there was a consistent name for "DID or handle (domain)". maybe... "handle_or_did"? or whatever, as long as it is consistent
  291. bnewbold

    In reply to this message

    (hadn't pushed to that public repo in a couple days, just pushed now)
  292. jetpack_joe joined the room
  293. bnewbold
    jmcasey: I have some notes/thoughts/pointers on doing ATP stuff in Rust, can send you a DM or post a doc somewhere if you are interested
  294. @wclayferguson:matrix.org left the room
  295. MightySpaceman (OLD -> m_spaceman:matrix.org)
  296. jmcasey

    In reply to this message

    You're further into it than I, but agree. Digging into it now
  297. In reply to this message

    Absolutely, please DM
  298. pfrazee

    In reply to this message

    we've got an open issue to clean up the renaming user and name items in there. I think the current approach for when either is allowed is to use "actor"
  299. Dean
    Note regarding atproto's api package, it currently does not work natively with vite SSR apps e.g. built with sveltekit when loaded server-side. You'll get the error exports is not defined at ajv-formats/dist/index.js:4:23. Guessing this has to do with ajv-formats being CommonJS. The api can only be imported in the client-side when doing it like const api = await import("@atproto/api"); in the browser side (i.e. inside an onMount()), which adds a big initial page load time since it's loaded into the browser session separately rather than being a part of the vite-generated js chunks.
  300. Børlaag changed their profile picture
  301. wolix
    Hello there iam have question bluesky be like Twitter only differences be similar looks
  302. Be interface be different
  303. i99988 joined the room
  304. i99988
    Hope better than all social media apps
  305. Steven Franssen

    In reply to this message

    i expect there will be many uis
  306. @xentra:foss.wtf joined the room
  307. Freezlex joined the room
  308. -THMS- the Founder joined the room
  309. @tomtau.:matrix.org joined the room
  310. @tomtau.:matrix.org
    Hi everyone! I skimmed through the AT protocol spec, and one question I had was "How is this different from the Solid protocol?" (which, to my understanding, is a lot about account/user data portability at its core)... perhaps something good to add to FAQ: https://atproto.com/guides/faq
  311. Prashant Mittal joined the room
  312. Aaron Goldman

    In reply to this message

    bnewbold: I had been using publisher as handle or DID

    Mostly just trying to get away from host.
    The origin for https is protocol, hostname, and port. The origin for ATP is protocol and publisher. We don't care what host gave it to you just if the signature is valid.

    Actor is pretty generic and has some overlap with the Erlang model but I don't know as long as we pick something and use it constantly probably fine. As long as we are clear what the origin is for the same origin policy

    ¯_(ツ)_/¯

    url = new URL('http://example:8080/path') url.hostname // "example" url.host // "example:8080"

  313. Prashant Mittal
    Hi everyone, really new here. I have been doing a deep dive into ATProtocol and have been reading the codebase for past week or so. and I have a few questions. I would be really greatful if you can help me out. Observations: 1. It looks like PDS is instantiated with a keypair which is used to sign all the data being stored 2. a jwt token is assigned to every account created on a PDS Question: 1. Does this mean there can be only one user per PDS 2. Can the account change the signing key for data that they are generating and not use the PDS's key Thanks in advance. Also, if you can guide me to a proper channel(if there is any) to ask general queries arounf the protocol and it's working
  314. Dean
  315. pfrazee

    In reply to this message

    Would you mind filing an issue for that?
  316. pfrazee

    In reply to this message

    Hey Prashant, I'll answer here instead of twitter DMS.

    1. No, you can host as many as you want. The signing key gets reused across repos. This is done so that PDSes can use the secure key managers that cloud services provide, which cost too much money per keypair to support 1 key per account.
    2. That is possible but you'd need to customize the PDS software to do that. Our implementation expects writes to be handled by the "home" PDS. You could make a version that is declared as the home PDS but actually just rehosts the current repo, allowing the signed updates to come from somewhere else.
  317. the account recovery flow that dean mentions allows you to change the signing key and the current PDS without your current PDS's involvement. The current expectation is that you're moving to another PDS that will manage your signing key for you, but like I described you could create custom configurations if you wanted
  318. Dean
    Somewhat related question I just posted in
    #general-bsky:matrix.org
    : Would it be according to standard to have 2 different handles on 2 different DID providers (i.e. a secondary domain) resolve to the same DID?
    (edited)
  319. pfrazee
    so first to clarify some language there, you'd have 2 different handles on 2 different handle providers that point to the same DID
  320. you know, obviously - the DID provider gives out DIDs, the handle provider gives out handles
  321. as for 2 handles to 1 DID, yeah that's possible but a bit underspecced. The DID document declares its current handle to help establish which one is the primary one. I think secondary handles could logically be handled like a redirect
  322. Dean

    In reply to this message

    Right sorry, I still need to adjust to the protocol's terminology :D thanks for the info!
  323. In reply to this message

    Is this something you believe the spec might need to include? i.e. returning an array instead of a string in the did resolver's alsoKnownAs field?
  324. pfrazee
    no prob
  325. In reply to this message

    what do you think needs to be accounted for?
  326. Dean

    In reply to this message

    Currently when you resolve a handle, according to the Identity docs, you get an alsoKnownAs field which returns a single domain handle. Perhaps to support alternative names for the same did, it could be an array instead of a string, and the array's first value is the primary name while any following values are things like shorteners/alternative handles on other handle providers.
  327. Instead of:

    123456await didPlc.resolve('did:plc:12345') /* => {
      id: 'did:plc:12345',
      alsoKnownAs: `https://alice.pds.com`,
      verificationMethod: [...],
      service: [{serviceEndpoint: 'https://pds.com', ...}]
    }*/
    

    It could be:

    123456await didPlc.resolve('did:plc:12345') /* => {
      id: 'did:plc:12345',
      alsoKnownAs: [`https://alice.pds.com`, `https://alicia.pds.xyz`],
      verificationMethod: [...],
      service: [{serviceEndpoint: 'https://pds.com', ...}]
    }*/
    
    (edited)
  328. pfrazee
    yeah that makes sense but what are you aiming to solve?
  329. Dean
    For example showing the primary handle in a user profile feed when a secondary one is visited
  330. So visiting https://frontend.com/alicia.pds.xyz, the frontend looks up alicia.pds.xyz, finds the primary handle (alice.pds.com) as the first value in the alsoKnownAs array, and displays that as the handle in the profile (perhaps with a list of "alternative handles" somewhere in the profile as well)
    (edited)
  331. pfrazee
    That makes sense. I'm inclined right now to keep it simple and just handle secondary handles as redirects
  332. Dean
    That way the displayed profile information will remain consistent regardless of handle
  333. pfrazee
    I can see some interesting benefits from it, like being able to alias with your work identities
  334. @pfrazee.com (aka @paul.bsky.team)
  335. Dean
    Exactly
  336. pfrazee
    I'll bring it up with the team. For the moment I'm inclined not to add to our complexity, and there's no time pressure on this. We can add your suggestion in the future by a variety of means
  337. Dean
    Perhaps the alsoKnownAs field could be an array with a single value just to future-proof it?
  338. So front-ends don't break if it's added in the future
  339. pfrazee
    let me check the DID spec real quick, but we can solve this no matter what
  340. (like, worst case if we're backed against the wall we can add a field to the did doc)
  341. I just dont want to add complexity to this yet
  342. complexity aka sophistication
  343. oh you know what, alsoKnownAs is supposed to be an array according to the DID spec
  344. I'll open an issue to make sure our code and docs are on spec
  345. guess that means we are tackling this now
  346. pfrazee
    okay issue filed
  347. Dean

    In reply to this message

    I filed an issue for this other problem just now: https://github.com/bluesky-social/atproto/issues/330
  348. Why are profile descriptions supported in the lexicon but not avatars? I can't find any lexicons or protocol specs on handling avatars and other common profile media (e.g. a profile banner)
  349. I guess it's a waste to store image blobs in atproto
  350. pfrazee
    we haven't implemented "blobs" yet, which is the generic system for non-record data and which will handle media
  351. Dean
    Oh ok
  352. pfrazee
    actually no, mercifully we can do blobs in atproto, unlike say git
  353. Dean
    So there will eventually be an "avatar" attribute in bsky.getProfile() Response object?
    (edited)
  354. pfrazee
    something like that yeah
  355. we've got some plans for blobs, including schemas for establishing size limits and expected variations of an image (small thumb, large thumb)
  356. you want a pretty well-coordinated system for blobs because they're a potentially huge performance problem. You want predictable sizes on things like avatars so that you don't end up downloading a 2mb png to display a profile
  357. Dean
    Yeah exactly
  358. I suppose IPFS is out of the question for the spec? It would allow you to check headers for filesize before downloading the image data.
    (edited)
  359. It's possible to create an isolated IPFS network between indexers and PDSs to serve data for avatars
  360. pfrazee
    we can talk about it. It's cool to leverage the open horizontal scaling, but the cost is that you lose a lot of concrete windows when you use p2p, plus you introduce the overhead of the swarm stack
  361. the overhead is non-trivial -- you have to regularly announce all the objects you're hosting to the discovery network. Imagine sending 1 packet for every blob once every minute
  362. RE the windows, you lose the ability to contact the host and say "it timed out" after 5 seconds or so. Instead you have to watch the peer discovery system to collect all the IPs, which has a variable window of time, and then you have to contact each of the peers, each with their own timeouts. It makes the system much harder to reason about
  363. consequently, it's only really valuable for situations akin to how bittorrent is traditionally used: very large datasets with no strong expectation for time of delivery
  364. Aaron Goldman
    That said a Merkel tree is a reasonable way to reference a large blob as a collection of chunks. One could imagine a few record types: Small blob < 1MiB (literal bytes) L1 blob < 1MiB (list of Small blob CIDs) L2 blob < 1MiB (list of L1 blob CIDs) L0 1MiB L1 32 GiB L3 1 PiB Assuming about 32 bytes per CID and a 1 MiB max_record_size You would also probably want to support a check out of the repository where you ignore the nodes under a block record until they are requested so you never spend more than 1 MB per blob
  365. @penaiple:midov.pl

    In reply to this message

    did i just hear blobs?
  366. what blobs are you talking about
  367. proprietary firmware blobs?
  368. Dean
    image data blobs
  369. @penaiple:midov.pl
    oh nevermind then
  370. the word "blob" rings alarm bells in my brain
  371. dutterbutter joined the room
  372. MightySpaceman (OLD -> m_spaceman:matrix.org)
    blob
  373. jdenoy joined the room
  374. airwaves changed their profile picture
  375. manuelbonnelly joined the room
  376. @stephane_huonder:matrix.org joined the room
  377. robesdornagon joined the room
  378. @tomtau.:matrix.org
    Since ActivityPub has gained a lot of momentum, will there be a way for AT to interoperate with it in some way?
  379. @michael_zks:matrix.org joined the room
  380. Aaron Goldman

    In reply to this message

    This is not currently a priority and not on the road map.

    There are three possible ways forward for this.

    * One could create an activity pub event for wrapping an ATP record.
    * Lots of complexity for Activity pub clients
    * Two wrap activity pub events in a ATP record
    * Lots of complexity for ATP clients
    * There a bridge that translates between Activity pub events and ATP records
    * Impedance mismatch between schemas and identity concepts

    All three have real drawbacks and would likely slow progress. Probably better not to put limited time of the dev team in expensive interoperability with activity pub at this time.

  381. nc163 joined the room
  382. nc163 set a profile picture
  383. nc163 changed their profile picture
  384. nc163 changed their profile picture
  385. bnewbold
    I feel like there is a middle path on using IPFS-like system for blobs (media artifacts)? have norms around block sizes and specific derived media formats/sizes. reference these by CID directly from ATP records. don't use bitswap for transfer, instead do transfer in CAR format over HTTP via an XRPC request (supplying a list of CIDs). store those blocks in whatever same blockstore being used for the MST. PDS doesn't need to pull CIDs for all content unless it wants a "deep" checkout. the current repo host of a handle is expected to have all the blobs references by any records in that repo, other PDS instances can cache and redistribute as they see fit. maybe optionally also have the "root" IPLD node point to a UnixFS DAG with all the blobs for the repo stored with filenames as well? that would make export-to-disk and exploring existing files easier. but not really needed and extra complexity
  386. pfrazee
    I'm reading quickly because I'm hackin but that sounds like what we're thinking. We put blobs in the repo's IPLD. We transfer them via ATP sync protocols. We bake specific behaviors for them
  387. bnewbold
    i'm guessing video is going to be the tricky thing, going to be more than a "block" even for compressed versions, probably going to need a bunch of sizes, and might want to be able to stream by fetching early blocks first (like webtorrent). and at the same time can really bloat up caches/blockstores
  388. it would probably be nice to somehow stay compatible with whatever cloudflare is offering for IPFS CDN (I haven't looked in to that in a while). in the expectation that other providers (or FLOSS equivalents) will have similar semantics
  389. pfrazee
    we're paying close attention to integration with cloud systems since we're deploying a service, so we'll export decisions like that in the design
  390. aka if the IPFS CDN or similar are useful
  391. Karl Abbott changed their display name to Karl Abbott (Away until 14-November-2022)
  392. ne.dmtro joined the room
  393. quentez joined the room
  394. Nad // away changed their display name to Nad
  395. @michael_zks:matrix.org left the room
  396. George Antoniadis
    Message deleted by pfrazee
  397. Maiquel De Brito joined the room
  398. Agustín Cantero joined the room
  399. pere joined the room
  400. @fr33domlover:matrix.org left the room
  401. Nad changed their display name to Nad // away
  402. slap19 joined the room
  403. @elsif:matrix.org left the room
  404. misaakidis changed their display name to marios
  405. bnewbold
    I made some progress on adenosine, a rust hobby implementation of atproto: https://gitlab.com/bnewbold/adenosine there is an example single-user instance at https://pierre-menard.robocracy.org/, and an example multi-user instance at, eg, https://hobbes.demo.adenosine.social/ you can explore the repository a bit: https://demo.adenosine.social/at/did:plc:3vp347vrxvzcs2dulat5sqwj the web interface is read-only, but the CLI works with these domains. DM me for an invite code to use with the CLI on
    demo.adenosine.social
    . everything will definitely get wiped, is totally naive about abuse etc. and there is no server-to-server stuff, which would be the interesting part
  406. Aaron Goldman
    Cool
  407. pfrazee
    wow. Does that mean you have a working MST written in rust?
  408. bnewbold
    sort of? it imported and validated against a CAR file exported from the atproto imlementation. but it is really crude, reads or writes the entire tree every time, instead of trying to update in place. and doesn't support partial trees https://gitlab.com/bnewbold/adenosine/-/blob/main/adenosine-pds/src/mst.rs
  409. pfrazee
    nice
  410. bnewbold
    to some degree, this was really the only part of the projet that made sense to write in Rust (strong typing, memory management, etc). I was pretty happy that it just worked the first time it finally compiled
  411. it is built on existing IPLD serialization libraries and an existing IPFS blockstore (using sqlite), so I didn't need to implement any of that
  412. pfrazee
    yeah, makes sense
  413. bnewbold
    if I have time i'll push debian packages and maybe binaries in homebrew. I expect that the upstream schema is going to continue changing rapidly, so maybe best not to do that until things stabilize (aka, a public release). there are a couple divergences and issues already (like the TID format)
  414. pfrazee
    yeah I'd recommend that just so we don't confuse folks
  415. bnewbold
    if there are places I can add louder disclaimers about being unofficial/incompatible/whatever please send recommendations or PRs and i'll push those right in
  416. pfrazee
    oh I appreciate it, I think you've done fine there
  417. Evren joined the room
  418. Evren left the room
  419. skyllaadalove joined the room
  420. Andre Simao joined the room
  421. jaygreasley joined the room
  422. sethetter joined the room
  423. @wings:perthchat.org left the room
  424. Tavin Turner joined the room
  425. @blahverse:matrix.org joined the room
  426. Dean
    Message deleted
  427. Dean
    I posted in #general-bsky:matrix.org since it seems to belong there, but the dev-related note is: it looks like "likes" have been replaced with upvotes and downvotes, I made a PR to represent these changes in the lexicon docs: https://github.com/bluesky-social/atproto-website/pull/6
  428. Jean-François Cotenord Tatoo joined the room
  429. kolby joined the room
  430. kolby
    Hello
  431. kolby set a profile picture
  432. MightySpaceman (OLD -> m_spaceman:matrix.org)
  433. Prashant Mittal
    Is a handle globally unique for a particular subdomain or is it locally unique in the context of a PDS?
  434. Karl Abbott (Away until 14-November-2022) changed their display name to Karl Abbott
  435. @geoma:matrix.org joined the room
  436. Dean

    In reply to this message

    I put up a Github issue with my thoughts on the recent update that replaced likes with upvotes/downvotes: https://github.com/bluesky-social/atproto/issues/347
  437. sylphrenetic

    In reply to this message

    pfrazee: am I correct in thinking that this change doesn't pigeon-hole implementors of ATP into ONLY using likes/dislikes, it just allows likes/dislikes as an option? so one implementation of the protocol could choose to not use dislikes at all, and only use likes as "favorites", and that wouldn't allow random people to start disliking posts on that implementation because that implementation isn't taking dislikes into account, right?
  438. Dean

    In reply to this message

    The protocol shows both counters separately i.e. upvotesCount and downvotesCount, so it's up to the client if they want to account for one or both
  439. But that will cause clients to divert from a standard way of displaying "scores"
  440. sylphrenetic
    I think the solution then is to add a "favorite" to the schema as well, so that if someone wants to use favorites they can use them without overlapping with the likes/dislikes paradigm
  441. Dean
    I thought of this as well but that would still cause discrepancies
  442. Imagine you make a post and just see Likes on it, and you want to invoke a friendly discussion
  443. And people from another client show up with "idk why you're getting downvotes but..." and "lmao jump on the downvote train!!"
  444. Imagine every tweet you write also shows up on Reddit, and every reply also shows up in a reddit comment section
  445. sylphrenetic
    make them mutually exclusive
  446. Dean
    Then you should have 2 protocols
  447. sylphrenetic
    wut, why?
  448. Dean
    Because the respective purposes of the 2 types of posts are incredibly different, no?
    (edited)
  449. pfrazee
    This is an interesting question
  450. My default assumption is that Dean’s view is the “safer” assumption: that on core social interactions it’s pretty important to stay consistent
  451. For the reasons that Dean described in the issue: it drives the perception and thus the conversation
  452. That said, it’s really hard to predict how big that impact is or how important it is
  453. Obviously you can have some divergence. Which ones are okay and which aren’t? It’ll be messy but stabilize over time.
  454. Dean
    Another problem is it lets people negatively impact other people's accounts e.g. with botted downvotes
  455. Without downvotes the only possible interactions generally are a net positive for the author: like/repost/reply
  456. sylphrenetic
    Right, but I think the protocol is hamstrung by not allowing certain basic functionalities on posts. Why not just make it so that once the post is created, you have to choose whether it's of type "likes only" or "voted"?
  457. Dean
    But once you feel like you're being constantly unfairly downvoted by trolls or bots, you no longer want to post on the protocol
  458. pfrazee

    In reply to this message

    That sort of thing is possible
  459. sylphrenetic
    That's what would make intuitive sense to me at least
  460. as a dev
  461. pfrazee
    There’s a lot of interesting configuration options for posts like that, also including how replies are done: who can reply, who can moderate
  462. Dean
  463. I posted the same idea in #general-bsky:matrix.org but also argued why this might not be the best solution ^
  464. Screenshots broken, here's a quote:

    I think the solution in this case would be to have a downvotesEnabled attribute in the Post object where posts created on certain platforms cannot be downvoted (clients that prefer downvotes could apply a different algorithm to posts that have downvotes disabled)
    (if downvotes are something definite for atproto)
    Either way, the respective purposes of posts with either a Likes or Upvotes/Downvotes scheme are inherently so different that they might as well belong to two different protocols

  465. pfrazee
    Fair. If it’s configured on the post then a different schema for like would certainly reduce confusion
  466. Dean
    Indexing servers would still be indexing all posts (wasted bandwidth) and then filter out the types that they don't want to include
  467. Yeah if it's a different schema entirely and servers don't have to sync that specific schema type then I agree it could work
  468. I do think that the current integration, which is an outright replacement of likes, is not the right solution
  469. pfrazee
    I get the logic. As for the debate around likes vs votes, that one is spicy. I told the team when we did it, “we’re going to all be debating this a lot”
  470. Dean
    As long as indexers have the option to sync both or either types so they don't waste sync time + bandwidth would be the most ideal if you want to support both.
  471. That still might cause a lot of "why does my post on ATP client A not show up on B?"
  472. pfrazee
    Yeah. I think your understanding of the technical implications is right. For the moment we’re going to focus on testing the experience and seeing if this is even a worthwhile direction
  473. Dean
    The main issue with voting on a decentralized protocol like this would be botting, where botting with Likes generally leaves no lasting negative effect on any third party (if someone bots likes on their own posts the user can lose their followers' trust/goodwill), basically only affected upon oneself, but botting with downvotes could be a simple but effective way to negatively impact others
    (edited)
  474. pfrazee
    yeah first of all, that's very accurate. For one, Im concerned about any way that people can be given a negative experience
  475. for two, botting is a big problem for us to solve across the board
  476. the entire notion of the "userbase" gets much more complex when you no longer lean on a central authority. There's no initial curation pass like what twitter has of establishing who can be on the platform
  477. my hope is that we can find ways to organize users and create reputational signals that replace that kind of function without leaning on server admins
  478. which could then act as a counter-botting mechanism
  479. whyrusleeping
    I also think it will pay to be extremely judicious with how downvotes are used. I dont think they should have their counts displayed in UIs, and should only carefully affect feed generation
  480. Who a downvote is from is way more important than how many there are
  481. pfrazee
    whyrusleeping: if we dont show downvote counts, I can see the logic to that but I'm concerned about hiding them flat out, because people might wrongly think they cant be revealed. But again I'm open to thoughts
  482. whyrusleeping
    Ah, yeah. Maybe they are hidden off the main post view, but you can drill down and find them
  483. And then downvotes by people you follow are listed at the top of that view
  484. Dean

    In reply to this message

    Maybe clients can show downvotes as "reports" and only show a count of reports from accounts that you follow
  485. Something like a downvoteFromFollowedCount but shorter 😂
    (edited)
  486. pfrazee
    they're not quite reports but I see the thinking
  487. George Antoniadis
    I assume servers can chose/code how to display/deal-with votes, and how they affect the ranking algorithm? ie I'm a fan of how HN deals with votes, but on a more federated system I'd like to also have a sort of "web of trust" thingie that friends' and friend of friends' votes have way orders of magnitude more significance. (issue here is that following someone does not make them a "friend" or trust-worthy person which is another issue)
  488. ^ also s/server/indexer probably?
  489. pfrazee

    In reply to this message

    right
  490. In reply to this message

    right
  491. Dean
    If keeping the timeline-feed networking theme I would prefer the idea that a "downvote" is only expected to be made if the post is objectively something the voter believes breaks a client's tos (e.g. nsfw content on a family-friendly client)
  492. Hence in my mind it makes more sense as a "report"
  493. @numero6:codelutin.com
    (not so off-topic) I sometimes view upvote and downvote has reducing emoji reaction to only allowing two emojis: 👍️ and 👎️. Any system who allow more emojis may have more algorithmic possibilities. I know that FB scored post differently if those had ❤️ or 😠 reactions. I understand the simplicity of +1, -1 and integers are easy to understand but counting emotions might be a more humanly way to "count" a post quality.
  494. pfrazee

    In reply to this message

    we might just want a report record for that
  495. In reply to this message

    yeah that is the trick though, aggregation of reactions loses some meaning. I think it's great in chat but in public performance, I think you need to reduce it down to fast button presses
  496. (I generally see Twitter as a public performance space, for better or worse)
  497. Dean

    In reply to this message

    Then what do you ideally expect downvotes to be used for? Disagreeing?
  498. @numero6:codelutin.com

    In reply to this message

    why does a public performance space wouldn't have one push button emoji reaction? (I know some ActivityPub app has those, including some Mastodon forks)
    (edited)
  499. pfrazee

    In reply to this message

    1, it's at least two presses, sometimes with a scroll to find the right emoji
    2, the purpose of reactions in a performance space is to affect its spread
    3, it reduces the clarity of the metrics, which is your scorecard in the performance game
  500. Dean

    In reply to this message

    If downvotes aren't expected to be delivered for content that breaks certain guidelines, then I suppose it would be for when someone disagrees with said post, which would not be a good basis for reducing a post's exposure
    (edited)
  501. A good example being FTX happenings on Twitter in the past week. If exposure based on downvotes were a thing, many people may not have seen tweets with warnings of an FTX insolvency due to the large number of downvotes they might have received from FTX fans and token holders, which would have caused many people to not have withdrawn their funds in time.
    (edited)
  502. whyrusleeping
    You’re assuming a particular algorithm for feed generation
  503. Dean
    Well I highly doubt any client would integrate downvotes as a positive factor
  504. pfrazee
    Dean: youre making good points and if that's how it turns out, then I agree it's a problem
  505. what why is getting at is that we expect there to be multiple different algorithms operating on the votes
  506. including things like who the algorithm listens to
  507. in which case the FTX fanboi algorithm is going to suppress it, but the Crypto Critics algorithm wouldnt
  508. Dean

    In reply to this message

    This begs another question: should reposts have separate voting stats?
  509. If something from a third party is being massively downvoted and I still want my followers to be exposed to it
  510. pfrazee
    my general sensibility about RTs is that they are a kind of non-algorithmic override
  511. if you RT something then your followers should see it
  512. Dean
    Btw currently the override is that reposts don't show up on anyone's timeline 😅 https://github.com/bluesky-social/atproto/issues/346
    (edited)
  513. pfrazee
    haha that's a bug yeah
  514. so we separate the concept space here into "Timelines" and "Feeds." Timelines are what you see on your home page. Feeds are what some entity emits as a sequence of posts
  515. Dean
    Yeah I got that
  516. pfrazee
    and youre going to be able to create/use different variations on both
  517. a "reverse chronological" Timeline would basically cause all RTs to show up
  518. but a chron TL would also include content from algorithmic feeds
  519. which is a nice option for people who like to make sure they get everything even if it's noisy
  520. an algorithmic or customized TL would apply rules and depending on those rules, this "RTs always show up" question would then depend on the rules you are using
  521. @blahverse:matrix.org removed their display name (blah verse)
  522. duc [they] changed their profile picture
  523. martnx joined the room
  524. martnx set a profile picture
  525. alexbourlier joined the room
  526. MightySpaceman (OLD -> m_spaceman:matrix.org)
  527. alexbourlier
    Funny
  528. kursato joined the room
  529. kolby changed their profile picture
  530. Giovanni Fersan joined the room
  531. franx_ear joined the room
  532. @secu:mtrx.nz left the room
  533. @toneji:nibbana.jp joined the room
  534. mikestaub
    No pressure, but we may need this protocol sooner than we thought. https://twitter.com/IanColdwater/status/1592188128944705537
  535. Giovanni Fersan
    Is the waitlist currently only for users or for developers as well?
  536. Giovanni Fersan set a profile picture
  537. giovannifersan changed their display name to Giovanni Fersan
  538. @amine_mdx:matrix.org joined the room
  539. @blahverse:matrix.org
    I think an ability to allow server admins choose some of the names/labels would be really cool. e.g. some admins can go repost and upvote route, while others can choose retweet and like - based on the audience they are targeting.
  540. Aaron Goldman

    In reply to this message

    I think this is counter to the separation of server from application goal.
    If you want to make a new application with different record types you can complete with

    and if useful reuse some of each other's record types.
    However the server should just be ATP and agnostic to your app so that migration remains trivial.

    To allow per server customization is to introduce vendor lock in to the protocol.

    If you want to sell a custom experience make an app with the record schema that you want and let the users pick their servers.

  541. pfrazee

    In reply to this message

    devs too
  542. In reply to this message

    yeah I agree
  543. Aaron Goldman
    I think it's a mental adjustment from the ActivityPub model that their Server Admin role has been split into the App and the Server. A PDS should be low administration. Mostly choosing the Blocklists or Allowlists to subscribe to for repo's/blobs so you don't host what you don't want to host.
  544. @blahverse:matrix.org
    Interesting. Thanks Aaron Goldman for the detailed feedback. I will look out for the beta launch to play different things out.
  545. @amine_mdx:matrix.org left the room
  546. Hacklet
    Is there a separate discussion for the app ethos over the technology?
  547. mikestaub

    In reply to this message

    Would be great to build an
    umbrel.com
    app
  548. g5646545646587 joined the room
  549. aaron-- joined the room
  550. mikuhl
    I luv umbrel
  551. Aaron Goldman
    I have been thinking about the analogy to talk about the relationship between PDSs, Apps, and Lexicon. I think browser features may be a good one. For the most part browsers from the beginning had mostly overlapping features. Some added unique features that the others copied. That has the risk of name collisions. After that the browser makers added vendor prefixes to features. `-moz-window-inactive` was made by Mozilla but was supported by chrome. In the same way Lexicon made by
    bsky.app
    could be rendered by any app when it reads the Repo. Chrome is not Firefox but can implement a -moz-* feature if it wants to. The http server does not need to understand all HTML, CSS, and JS to serve the content. There are many browsers (apps) that can render content. But unlike the browser features that leads to the https://caniuse.com/ nightmare the Lexicon structure should give more structured fallbacks to common functionality. So what do y'all think is this a useful analogy or just too confusing to be helpful?
  552. mikestaub

    In reply to this message

    Ii think this is a perfect analogy. What is a shame with web browsers is that they don't know which APIs the website is using so if the site is depending on an unsupported API it will just break. Ideally apps could declare all the APIs/schemas they depend on in a manifest so clients can gracefully degrade.
  553. Aaron Goldman
    What, you don't find writing browser feature detector code in Turing complete JS with custom hand rolled fallbacks a fun time. 😡
  554. polyeven joined the room
  555. airwaves

    In reply to this message

    Well, there's always Modernizer :-)
  556. MightySpaceman (OLD -> m_spaceman:matrix.org)
    👾
  557. @graham_o:matrix.org joined the room
  558. @graham_o:matrix.org set a profile picture
  559. @graham_o:matrix.org changed their profile picture
  560. michaelvigor joined the room
  561. pfrazee
    lol woah
  562. pfrazee
    I agree with this browser framing. The difference is that rather than HTML tags, CSS features, and JS APIs, the "ATP browser" is speaking a semantic data language (lexicon)
  563. posts and follows instead of divs and styles
  564. mikestaub

    In reply to this message

    conceptually there are the same though. CSS features and schemas both affect an apps ability to render properly. Would what be amazing is if the client could detect the schema and prompt the user to install a plugin to view it, like VSCode does when you open a Dockerfile
  565. pfrazee
    right yeah -- that's what I mean, that it's an equivalence, but the primitives of the "language" are different
  566. as for plugins or similar, there's basically two questions we'll have to navigate as this evolves
  567. @graham_o:matrix.org changed their profile picture
  568. pfrazee
    the first is choosing the dividing line between publisher control and client control. The Web went through this exact thing; the notions of separating style from content and of "user agent stylesheets" were born out of a desire for client control. This is a matter of debate now but I'd argue "publisher control" won heavily on the Web, and that's largely because the Web browser is a thin client which transfers UIs
  569. as ATP operates at a higher level of semantic, it is feasible that we can favor client control, but this will be a constant debate
  570. the second is how we handle client handling of new lexicons. A plugin architecture is certainly possible, and it's very much a "client control" style solution
  571. plugin systems are hard but I'd wager the community will do some interesting experimentation there. Theyre hard in general -- hard to design well, hard to manage resource usage, hard to secure, hard to create a clean UX -- but then you have stuff like Apple making it hard to do that sort of thing on iOS with their policies
    (edited)
  572. @graham_o:matrix.org
    Message deleted by pfrazee
  573. whyrusleeping banned @graham_o:matrix.org: spam
  574. Aaron Goldman
    Plugin vs many apps Is a non-obvious trade off. You could imagine many apps that do one thing and do it well. With the browser that "open with"s the unixy apps or a super app that handles all kinds of things.
  575. kt654vh57j joined the room
  576. mikuhl
  577. what did elon do LMAO
  578. pfrazee
    Whew buddy
  579. mikestaub

    In reply to this message

    I think Chrome really nailed it with their extension model. This is the best example I am aware of. https://classycraig.com The user has full control of their own experience, if AT protocol can make the data layer as interoperable as the client layer, things will get super interesting fast.
  580. mikuhl
    So.. how you guys doin lol.
  581. sylphrenetic
    I'm in Hank Green's Twitter space j chillin tweeting about #butts
  582. pfrazee
    Haha we’re good. Can’t share deets juuuust yet but it’s coming together fast
  583. guicaldaso joined the room
  584. gui.caldas.o changed their display name to guicaldaso
  585. guicaldaso set a profile picture
  586. Yuki joined the room
  587. Aditya Sharma joined the room
  588. James Fan joined the room
  589. James Fan
    Hi! Dear ! I am experienced in Full Stack Web & Mobile Application using ➤React, Vue, Angular, Django, Flask, Solidity, Golang ➤Python with AI technology, SciPy, NumPy, Matplotlib, Web Scrapping, AI, ML, DL, NN, Prediction, Decision-Making, Image/Audio & Video Processing, ➤Effective Algorithm Design & Analysis ➤Smart Contract, Blockchain, dApp, ➤SQL/NoSQL, Real-Time Processing, Multithreading, Automation Processing, Bot, Electronic Automation with Micropython. if you need my help, please send me DM! Thanks!
  590. kobagen joined the room
  591. @vyach:matrix.org joined the room
  592. hikae joined the room
  593. Dean

    In reply to this message

    Good timing! 👏
  594. @neilalexander:matrix.org changed their profile picture
  595. magicofazi
    Between AT Protocol and Matrix Protocol, which is superior and why
  596. Personally think matrix is the furthest ahead in terms of federated and p2p messaging and has client diversity already
  597. bsjdjjdj

    In reply to this message

    Eta on FOSS reference client ?
  598. sylphrenetic
    "Two weeks" -Elon Musk 😘 /j
  599. bsjdjjdj
    I'm thinking about building an android client and wanted to get a sense of the roadmap
  600. thenameless joined the room
  601. Robert Johnson joined the room
  602. pfrazee
    no eta unfortunately. We're really close but we've got a few key pieces to stabilize still
  603. @enapiuz:envs.net joined the room
  604. sylphrenetic
    I noticed a problem today with ActivityPub and Mastodon: if I'm on server A and want to migrate my followers (and posts, but I don't think that's supported on Mastodon) to server B but server A goes completely offline, there's no way for me to migrate ANYTHING over to server B, I'd have to create an entirely new account and start from scratch. that's a horrible experience, analogous to if Twitter was to lose its entire database and require users to create all new accounts on some new server. will ATproto solve this in any way? I would hope I could take my data (posts and all) and up and move home servers to anywhere I wanted if my current PDS is online, but what if its offline? I guess I would have to have some data redundancy in there somehow, like my own offline copy of my data that gets updated every time my home server does (but with lower priority on speed, so updates could be batched to it over time, like when you get to be on wifi from a smartphone). are there any plans to natively solve this?
  605. @toneji:nibbana.jp
    sylphrenetic: Nice to meet you.
    Isn't Huzzilla the one with the mechanism to guarantee DB redundancy?
    I think even matrix has to be set up from scratch on another server if your main server goes offline.
    (edited)
  606. Dean
    Coming back to likes vs upvotes/downvotes: is the general consensus for the protocol to include both options (poster/client decides which to post, indexer decides which to index) before launch?
  607. In reply to this message

    Auto offline backups should be an easy feat. Just set up a local indexer server that indexes only your own activity.
  608. @toneji:nibbana.jp

    In reply to this message

    Oh, I understand.
  609. sylphrenetic

    In reply to this message

    I don't think it would be easy, per se, especially for the layman. I think there should be something built into the protocol to allow duplication of your data among multiple PDSs, with only one being the authoritative one and the others being backups (most likely hosted in the cloud by another service provider).
  610. pfrazee

    In reply to this message

    We've got this as an actively watched UX decision and make a call based on what yall and test users are saying. Book is not closed, but no idea where it'll land
  611. Karl Abbott changed their display name to Karl Abbott (Away Until 28-November-2022)
  612. Daniel Holmgren

    In reply to this message

    the data is very portable. multiple copies can be stored in different places & if your PDS goes down, you can "restore" your account to another service. This could be another PDS in the network, a backup node, or even just local back up on your device. You may not want to include large blobs in a local backup, but posts & social graph can be backed up without much overhead
  613. Matheus Mósso joined the room
  614. rdubs20 joined the room
  615. @rimuru:gentoo.chat changed their profile picture
  616. joeymueller joined the room
  617. dotneb joined the room
  618. dotneb set a profile picture
  619. bodega joined the room
  620. Deans Charbal joined the room
  621. n0nmanifest joined the room
  622. NikhilM42 joined the room
  623. erlend_sh joined the room
  624. harshveer14 joined the room
  625. harshveer14
    Announcing my arrival 😁✌️ New to the chat .. traditional java backend developer based out of the bay area
  626. Aaron Goldman
    Don't think anyone has started a community implementation of the server in Java 🤔
  627. Brandon Esau joined the room
  628. Seve(rino) Zeni joined the room
  629. @mrkizoo:matrix.org joined the room
  630. @mrkizoo:matrix.org left the room
  631. Josh joined the room
  632. sylphrenetic
    my original background is in Java, so I might be able to help out once the time comes 🥰
  633. Seve(rino) Zeni left the room
  634. @neilalexander:matrix.org

    In reply to this message

    Which implementations are out there (incomplete or otherwise)?
  635. Josh
  636. Aaron Goldman

    In reply to this message

    Bluesky PBLLC is doing one for Node JS in Type Script. Not sure what others exist. But again it's early and subject to change which limits the motivation to make an implementation
  637. George Antoniadis
    @neilalexander:matrix.org: why has some parts implemented in go as well, https://github.com/whyrusleeping/gosky/
  638. @neilalexander:matrix.org
    Thanks both
  639. @neilalexander:matrix.org was secretly hoping for something in Go
  640. George Antoniadis
    @neilalexander:matrix.org: yeah figured as much, love your work on pinecone btw. amazing stuff. :D
  641. whyrusleeping
    Oh hey
  642. Happy to collab on any of the go stuff
  643. Its a bit of a toolkit right now as opposed to an “implementation”
  644. magicofazi
    @neilalexander:matrix.org: what’s your opinion of blue sky vs matrix
  645. @neilalexander:matrix.org

    In reply to this message

    That would be great!
  646. In reply to this message

    In my mind they are not easily comparable. Matrix is more about multiplayer data structures and has a lot of complex machinery involved in making sure servers arrive at the same conclusion whilst operating independently (conflict resolution, auth rules etc), whereas ATProto seems to be much more about single entities maintaining and publishing their own conflict-free data structures that other people can just happen to peek into or interact with
  647. Different solutions for what are arguably different problems, even if there are some overlaps in how they work or what they could be used for
  648. magicofazi

    In reply to this message

    Do you think matrix can adopt the portable account features at protocol has?
  649. @neilalexander:matrix.org

    In reply to this message

    Portable identities have been on the Matrix future roadmap for some time now and it’s bound to happen eventually, but don’t know how much it would resemble ATP’s implementation at a technical level
  650. Not least because it means you can take some power away from servers, which is quite important to do
  651. magicofazi
    Fingers crossed we solve portable identities by next year 🙏
  652. Also hope dendrite gets closer to production ready implementation
  653. George Antoniadis
    we've very very very off topic here, but just wanted to say dendrite is working like a charm :D using it as we speak
  654. @neilalexander:matrix.org
    Apologies for topic drift ;-)
  655. A Go-based ATP implementation would be quite exciting though
  656. I would happily contribute some personal FOSS hours to that
  657. George Antoniadis
    Count me in as well :D Happy to pitch in.
  658. Aaron Goldman

    In reply to this message

    What do you think about Theseus Identities?

    Is there part of the portable identity problem you think this works poorly for?

    https://youtu.be/Z04RGWgHzvU

  659. George Antoniadis

    In reply to this message

    Thank you for sharing this, this direction is really great.
    Is there a specification for this somewhere public?

    Have you happened to have stumbled on KERI by any chance?
    I think it fits nicely your notion of Theseus IDs (up to a point) and it might offer some interesting ideas.

    It's pretty much a series of "events" that change the state of the identity. (the main ones are "key rotation" and "delegated key rotation")
    It uses a notion of a "next key" that the user can use to "rotate" to that key if their current one gets leaked/lost (similar to the "recovery key" mentioned in the video
    Has a concept of a "witnesses" that can be used to pull the events for an identity, and to recover a lost "next key".
    The idea of ticks doesn't really exist for ordering, but instead the moment two conflicting events are seen, both are considered invalid and the key needs to be rotated using the "next key".
    Has some ideas in general def worth taking a look at (ie delegation).

  660. Aaron Goldman
    Yup I like KERI. Duplicity is a question of having a consensus group for updates to prevent the double rotate problem (basically the double spend problem). I don't know how this is done with the witnesses. I should look more at their consistency story.
  661. Aaron Goldman
    From the video it seems like more of a local consistency guarantee. With the controller responsible for consistency not a consensus group. That said the witnesses server he called Doug is similar to the PLC server. I think there is a question here for what is the best way to sell an identifier.
  662. whyrusleeping
    @neilalexander:matrix.org: ive got all the serialization working and most of a code generator for the rpc schema files done. I have pieces that together into a proper PDS yet though, but i can do all the client stuff thats needed
  663. Ive been using it so far to do some search engine indexing
  664. magicofazi
    So i'm the founder of
    zo.me
    , we're forking the matrix protocol, creating global identifiers on a app chain PKI with economic security (ETH,BTC,etc) Servers or Zo Nodes hold local identities but users also hold global identities with social recovery allowing them to remain resistant from deletion by their home server/node
  665. @vsco:matrix.org joined the room
  666. UnownPlain joined the room
  667. UnownPlain left the room
  668. @jan:cloudcheck.io

    In reply to this message

    Could you elaborate a bit, why to fork and not extend?
  669. whyrusleeping
    Probably a better discussion for the general channel
  670. ghobs joined the room
  671. @amirabbas_ir:matrix.org joined the room
  672. i3xCx joined the room
  673. magicofazi
    Matrix is pretty focused on building out synapse and has shown a disinterest in any crypto integration at the protocol level. My goal is to fork Dendrite, make it production ready, attach a global identity using a appchain based PKI on ETH/BTC, and have a token that is used for spam prevention and economic guarantee of identity data. The goal is also to use Rooms as a CRDT based Database and allow developers to build Local First JS Apps as mini apps
  674. @penaiple:midov.pl
    JS is useless slow insecure trash
  675. you should build programs in C if possible
  676. otherwise C++
  677. magicofazi
    we'll be using typescript actually
  678. @penaiple:midov.pl
    for web stuff you can use TS, HTML, CSS
  679. magicofazi
    but main homeserver will be written in go
  680. @penaiple:midov.pl
    JS has been designed in 10 DAYS
  681. pfrazee
    let's not get into language wars here
  682. @penaiple:midov.pl
    and pretty much all security holes in web browsers are caused by JS
  683. half of them by JIT
  684. In reply to this message

    golang is pretty good
  685. not as nice as C, but far better than python or js
  686. synapse constantly needs babysitting because it keeps breaking
  687. same thing with pantalaimon
  688. my matrix client constantly has problems with pantalaimon because it doesnt support encryption (yet)
  689. George Antoniadis

    In reply to this message

    Well this escalated quickly xD
  690. @penaiple:midov.pl
    honestly, i cant say a single good thing about python or js
  691. magicofazi
    I prefer rust over c
  692. @penaiple:midov.pl
    i like functional software personally
  693. (stable software)
  694. @jwnz:matrix.org left the room
  695. Aaron Goldman

    In reply to this message

    Probably a discussion better suited for
    https://discord.gg/N7vjnhWN
  696. @penaiple:midov.pl

    In reply to this message

    i dont use proprietary malware, sorry
  697. @neilalexander:matrix.org
    Let’s not go completely off topic here
  698. i3xCx
    So when blue sky going be open to public?
  699. pfrazee
    we're not sure, but soon. We've started internal testing. We've got the app on testflight. We're fixing bugs and stabilizing protocol. Soon soon soon
  700. i3xCx
    Nice looking forward to it
  701. Is blue sky going be like mastodon?
  702. pfrazee
    in a few ways, but it'll feel different, especially over time
  703. i3xCx
    Bluesky able to follow anyone on mastodons ,peertube etc?
  704. pfrazee
    nah, different protocol
  705. i3xCx
  706. i3xCx
    Is there a screenshot what bluesky going look like?
  707. pfrazee
    I suppose I could do another small teaser screenshot
  708. whyrusleeping
  709. I can do a slightly better teaser
  710. pfrazee
    woah woah woah that's proprietary
  711. "Chilé con Kitty" is copyrighted
  712. whyrusleeping
    Fair use brah
  713. George Antoniadis
    Awh; it's the oracle poem all over again. http://dacut.blogspot.de/2008/03/oracle-poetry.html (seriously though, can we make the "Chilé con Kitty" this part of the spec somehow? :P)
  714. pfrazee
    haha no way about the oracle poetry
  715. not sure how we work chilé con kitty into it -- it's what you get when the cat comes in from the outside and their fur is cold
  716. George Antoniadis
    I think the the bsky app/client needs a maskot :P One that maybe changes with the seasons? 😂
  717. pfrazee
    haha jay was just saying that, I'd be strongly in favor of a kitty mascot
  718. there all set
  719. George Antoniadis
    hell, I'm trying to "improve" this ^ with midjourey or stable diffusion and there's really no saving it 😂
  720. pfrazee
    hahaha
  721. pfrazee
    hah sweet
  722. "Why is your mascot a cat in winter gear?" "Uhhhh"
  723. b0gg3r
    Run sentiment analysis on the last 24 hours of posts and update the kitty's style accordingly
  724. George Antoniadis
    ^ yeah, basically ai generated logos, like google's doodles they add to their logo for special events
  725. Aaron Goldman
    Might be worth trying to answer this question in the bsky model. https://twitter.com/NuttySwiss/status/1594358607835201537
  726. Ricardo Poh joined the room
  727. Nad // away changed their display name to Nad
  728. @yuenso:matrix.org left the room
  729. mikuhl
    Me not making a Mastodon, Hive, Truth, whatever else there is out there, because I know a bigger storm is coming. 🤩
  730. @numero6:codelutin.com

    In reply to this message

    will there be a linux desktop client available?
  731. pfrazee
    the only thing the team is building rn is a mobile app
  732. mikuhl
    I might make something in Flutter, which can be compiled to Linux, Windows, Mac, iOS, Android, Web, Nintendo Switch, Toaster, Wall, etc. 😂
  733. thenameless removed their profile picture
  734. mikuhl
  735. For real Flutter runs on a wall. 😂
  736. evck joined the room
  737. mikuhl
    that new Hive app everyones talking about is Flutter.
  738. kryptoshrimp joined the room
  739. boscovn

    In reply to this message

    Doesn't flutter pakcage linux stuff only as snaps?
  740. Has that changed?
  741. mikuhl

    In reply to this message

    I don't use linux so I am not sure exactly lol https://docs.flutter.dev/development/platform-integration/linux/building
    (edited)
  742. mikuhl
    oops accidentally put windows
  743. @sterence:sleeby.cafe joined the room
  744. arunwadhwa joined the room
  745. parhamb joined the room
  746. @sonotap:matrix.org joined the room
  747. jmcasey

    Been playing more with the MST in rust. It'll deserialize from DAGCBOR and can use rocksdb to verify CIDs.

    I've seen other implementations (such as adenosine bnewbold ) which use a SQL-like db almost as a "presentation layer".

    Because I'm naive (or optimistic?), I liked the idea of avoiding a db other than the blockstore. If CIDs can be checked locally, and retrieved from IPFS, then why can't we traverse the tree instead of querying for posts in a db?

    First problem is retrieval time from IPFS, but that's a bit out of scope here.

    The second problem is that given the CID of a repo, it takes between 3 and infinite calls to the blockstore to resolve a post.

    The third problem is I'm having trouble deserialising into ATP native objects instead of Ipld objects using serde, so I'm currently making an additional blockstore call on each recursion.

    Now that I have it somewhat functional, it seems too slow to be viable. Not sure if anyone has thoughts on this - it might be the case that an additional db is the right solution here.

    I'm not super happy with it but you can see the code here:

    https://gitlab.com/jmcasey/rust-atp/-/blob/main/src/mst.rs

    (edited)
  748. @planetoryd:matrix.org
    a database can definitely be built on ipfs directly
  749. you can store indexed data in it and avoid copying data
  750. it's an open problem of how do we turn ipfs (or any protocol like this. eg. locutus) into a database (eg kv store)
    (edited)
  751. jmcasey
    So something like CBOR > Tables of users, posts,etc > Db on IPFS?
  752. And I guess you would write to the tree, broadcast the changes, which are indexed into the db, which is queried for reads?
  753. Aaron Goldman
    If you want an example of a SQL built on top of a keyvalue store I recommend looking at myrocks. http://myrocks.io/
  754. The collection is a collection of pages rather then records. The tables and the indexes are all b-trees on top of the pages. If you trust the publisher then you can use the indexes with the same confidence as the primary tables.
  755. committed
    just got back from IIW conference, you might be interested in Authentic Chained Data Containers https://wiki.trustoverip.org/display/HOME/ACDC+%28Authentic+Chained+Data+Container%29+Task+Force
  756. As part of the Trust over IP stack there was a presentation on CESR - Composable Event Streaming Representation which solves a lot of the problems encountered when dealing with CBOR and codec interop and message signing in the context of streaming signed data.
  757. jmcasey

    In reply to this message

    Sorry, who is the publisher in this scenario? The entity maintaining the db?
  758. bnewbold

    In reply to this message

    rocksdb is a database! just a simple key/value database. sqlite is also quite simple and makes a totally reasonable key/value store.
    adenosine mostly implements the atproto stuff using just the blockstore (which is also in sqlite, but that is abstracted away, any key/value store would work fine). meaning of the com.atproto Lexicon.
    it does use sqlite tables for DID documents. those are mutable, so if you want to put them in IPLD you would need to figure out something like IPNS on top. for that matter the DID-to-root-commit mapping is mutable and you need to figure out some way to store that mapping
  759. parts of the bsky stuff are stored in sqlite directly, in the form of "secondary views" or indices into the repositories. sure seems easier to have things like a table of who-follows-who and being able to do queries against that to generate author feeds and timelines. and SQL databases give you things like types and relations (foreign keys) to help you get this right out of the box
  760. jmcasey
    Yes! But I couldn't see how a key-val db could store records which could then be queried by key/uri - I thought it would need a separate db
  761. Do secondary views have implications for cache invalidation?
  762. As in, do they link directly through to the "blocks" or would they need to be updated each time a block is added to the db?
  763. bnewbold
    the secondary views (indices) potentially need to be updated every time records are updated in a repo
  764. this is the same as how a SQL database internally maintains things like multiple indices on a single table
  765. jmcasey
    Gotcha
  766. I see the appeal of ipfs_sqlite_block_store
  767. bnewbold dinner time
  768. @sonotap:matrix.org left the room
  769. bnewbold
    jmcasey: there is also this Rust implementation of a blockstore using RocksDB. it hasn't quite been released as a crate yet though
  770. jmcasey
    iroh?
Next group of messages