Bluesky Dev
Community discussion of the AT Protocol and Bluesky. (This room is not officially affiliated with the Bluesky team.)
Previous group of messages
  1. pfrazee
    eh XRPC is currently the plan. You might be able to use IPFS wire protocol stuff but that's not how we're currently thinking about it
  2. bnewbold
    the XRPC 'sync' stuff (in the
    atproto.com
    lexicon) seems pretty minimal. like, by default syncGetRepo would return the entire history as a single CAR file, right? if you could get the entire root object (instead of just CID) from 'syncGetRoot', then you could request "everything since previous CID" which would be like a shallow clone of the current state of repo, as a CAR.
  3. maybe i'm overthinking this, i'm sure there will be iteration and it will all work out
  4. pfrazee
    oh yeah the xrpc sync methods are a hold-over from a much earlier iteration. Essentially it's unimplemented
  5. bnewbold
    i'm mostly curious from the standpoint of implementing an aggregation service and how efficient that could be. and whether it will be easy/common to gossip repo content around from multiple services (aka, instead of needing to download only from the current PDS for the user/repository)
  6. w/r/t XRPC: it is a little too bad that there isn't a more REST-like interface to individual records in repositories. eg, "get" an individual record from a collection and TID with the obvious HTTP path. I get that having consistency is important (aka, that all methods are XRPC methods)
  7. pfrazee
    not sure yet how we'll approach gossiping. There's no technical barriers -- just a question of the economics of rehosting and when we can expect it to be done
  8. In reply to this message

    fwiw the at:// URL scheme is certainly closer to that, though it only supports GET atm
  9. eg at://pfrazee.com/app.bsky.post/1234 is the canonical URL for a post with key 1234
  10. bnewbold
    right, but I can't just curl --get https://pfrazee.com/xrpc/app.bsky.post/1234
  11. but maybe we will be able to atp-cli get at://pfrazee.com/app.bsky.post/1234 or whatever, and that is good enough
  12. pfrazee
    yeah that would be https://pfrazee.com/xrpc/com.atproto.repoGetRecord?uri=at://pfrazee.com/app.bsky.post/1234
  13. yeah, we can also talk about whether there's upside to a kind of REST passthrough for records
  14. bnewbold
    oh yeah, another question I had:
  15. how would you implement a 140 character limit :)
  16. the website's rendering of the lexicons doesnt currently capture all of the constraints
  17. bnewbold
    oh, it is actually JSON schema or whatever which allows those extra constraints. great
  18. pfrazee
    yeah
  19. bnewbold
    cool cool that makes codegen much easier
  20. pfrazee
    yeah
  21. bnewbold
    I wasn't sure what the schema DSL in the docs was. ok great. that was probably mentioned in docs and I just skimmed that part
  22. another question was to confirm was whether there was actually any need to have branching and merging in the repo history format. I think it mentions that the "previous" commit CID reference could be plural (which would be an N-way merge)
  23. but as I understand it there would only be one canonical PDS for a repository at a time, and XRPC requests would presumably be fielded in transactions or something. so even with multiple user devices trying to write in parallel, would end up as a simple linear history
  24. pfrazee
    no current plans to use it but not impossible that it can be relevant in the future
  25. bnewbold
    as a DX thing, you might want to clarify that even if all the records in repositories are serialized as CBOR, clients can always use 100% JSON to interact with the system over XRPC. except the 'sync' methods I guess. and I sort of presume that HTTP headers can be used so clients can communicate in either JSON or CBOR. by my reading there is a huge developer win that the client application stuff can all be agnostic to the binary dweb/p2p format things going on in the PDS backend, and just do regular ol' web tech APIs
  26. pfrazee
    yeah ^ that is the situation
  27. bnewbold
    I haven't followed the GraphQL space much, but I wonder if it will be possible to ~deterministically derive a GraphQL schema from Lexicons. eg, as a parallel/alternate API as a layer on top of the XRPC API
  28. pfrazee
    might be possible for us to use graphql at some point. The broader question is whether the graphql queries are efficient and dont over-tax the PDS
  29. lk251
    Is there anything I can do to increase my chances of being approved to join the private beta ?
  30. Jeremy Kauffman joined the room
  31. brh28
    Recloned the repo, ran yarn (v1.22.19) and got a build issue, which I created an issue for: https://github.com/bluesky-social/atproto/issues/274
  32. djspacebunny
    i am willing to host a server if i can. things are weird in life right now.
  33. brh28
    Playing around with the repl, how do I execute schema methods (e.g createAccounts) on a PDS server running locally?
  34. Daniel Holmgren

    In reply to this message

    we haven't added the ability to make xrpc requests from the repl yet. you'd have to do it with curl or programmatically (which you can see examples of in pds/tests)
  35. bnewbold
    here is a quickstart I just wrote about getting the prototype server running, and interacting with http (aka HTTPie): https://pad.robocracy.org/s/f2_HTquiu
  36. remember that this is all ephemeral, in-RAM, and all content and accounts will be lost after shutting down. work in progress, everything will get hacked, not all implemented, yadda yadda
  37. Daniel Holmgren

    In reply to this message

    is it private? won't let me open
  38. bnewbold

    In reply to this message

    whoops, I mucked up the permissions for a minute, I think it should work again now
  39. Harlan Wood

    In reply to this message

    Super useful, thanks very much for sharing!
  40. jplegard joined the room
  41. @jwnz:matrix.org joined the room
  42. @cel:pussy.accountants changed their profile picture
  43. @toranosora:matrix.org joined the room
  44. karetski joined the room
  45. mikuhl
    Twitter honestly might implode lmao. Hopefully there will be a perfectly timed escape boat. 🙂
  46. @wclayferguson:matrix.org
    I hope Elon embraces BlueSky and realizes this is the path to create the kind of social media everyone wants, regardless of any politics.
  47. mikuhl
    99% sure he will make it a dogecoin platform
  48. He's too deep into that shit
  49. karetski
    I hope Elon has nothing to do with Bluesky to be honest
  50. @wclayferguson:matrix.org
    I hope he provides more funding to BlueSky, but yes, he should basically let the tech guys take the lead.
  51. (but we're probably "off topic" since this room is really about 'development'... my apologies if that's true)
  52. karetski
    Just funding — maybe. But looking at his track record, it's very unlikely.
    (edited)
  53. @wclayferguson:matrix.org
    We'd probably need a room called "BlueSky Opinions & Politics" where random discussions can happen. There's probably a room like that I forgot about.
  54. pfrazee
    @wclayferguson:matrix.org: appreciate you keeping us on topic! Yall are fine but yeah, just because we could spend all day talking about it
  55. @penaiple:midov.pl

    In reply to this message

    that would (finally) be over then
  56. good thing that elon would never use XMR for any of his crap
  57. otherwise it would be more successful
  58. hampson joined the room
  59. bf6dmbmdsv joined the room
  60. @nurpu:matrix.org left the room
  61. arcalinea
    Bottom line is we're going to keep building and (try to) stay focused. Very curious to see what will happen, but Elon's approach of changing Twitter from the top down is philosophically the opposite of our approach: changing the nature of social platforms from the bottom up by making the core pieces neutral, protocol-based infrastructure. It's our job right now to prove that this approach is usable, scalable, and compelling.
    (edited)
  62. Father Maurer joined the room
  63. gabisous joined the room
  64. Anton Kent - Anytype
  65. Greetings :-)
  66. Waiting for Jay Graber’s talk
  67. 45kwv8q6s9 joined the room
  68. @wclayferguson:matrix.org

    In reply to this message

    I think Elon is saying he wants people to be in control of their own feeds, rather than power to censor being algorithms and centralized in the hands of a small unelected group of individuals. So in that regard all of us and Elon too, are thinking the same thing. There are already successful protocols like Matrix and ActivityPub/Fediverse so I'm excited to see what BlueSky can bring to the table, and if you believe what Elon has said publicly about open source and freedom he should be very much behind all of the above as well.
  69. whyrusleeping
    I think what Jay is saying is not that the goals are necessarily different, but that the approach is
  70. Brad Brown joined the room
  71. lk251
    Those of us building an iOS client for bluesky, should we wait for (test?) indexing services to be available, or is there anything we can do right now?
  72. pfrazee

    In reply to this message

    it's 🔥 that you are but you're going to be in for pain right now. We don't even have a fully working client yet. I'm able to make a dev build work because I know where all the TODOs are
  73. we're not trying to slow you up! It's just not done yet. This is also why we dont communicate a lot of the dev tooling to yall yet, because we'd just be handing you a sandwich full of glass
  74. part of why we want to do this private beta is so that we can stabilize this stuff. Having been on both sides of the process, when you're building something and upstream keeps changing in dramatic ways, it's super frustrating
  75. (and to fully give my credentials on this) prior to this project I've done ~8 years of foss decentralized projects where we did the opposite -- we would share and explain everything with devs as we worked
  76. my last project, in fact, I live streamed all development (https://ctzn.network/dev-vlog)
  77. even then it was frustrating for folks, and I just didnt want to repeat that. So that's why we're doing it this way
  78. dogod M joined the room
  79. rangak joined the room
  80. arcalinea

    In reply to this message

    To use a LOTR metaphor, it's possible to seize the "one ring to rule them all" in order to destroy it, but very difficult, and most end up keeping that ring. So we'll see what character Elon ends up playing in this next chapter of the saga.
  81. Michael Calvey joined the room
  82. @penaiple:midov.pl changed their profile picture
  83. alextoma joined the room
  84. alextoma
    Hello
  85. I'm kind of a newbie developer
  86. was wondering where you guys think the best place to start would be - I want to eventually build on top of ATProtocol
  87. I have a good idea of what frontend development is like but am less clear about backend
  88. would greatly appreciate any help or advice
  89. drstrangeknight joined the room
  90. alextoma set a profile picture
  91. pfrazee
    alextoma: welcome! if you scroll back just a smidge you can see me explain where the code is at. Short answer: things just aren't quite stable enough for yall to build yet
  92. drstrangeknight
    Hi guys, I am a Data Scientist/Tech Entrepreneur (with PhD in AI), currently a Data Architect/Tech Lead, and I am all for a safe, connected and progressive world. I would love to know what bluesky is about and see if I can be of help.
    (edited)
  93. alextoma

    In reply to this message

    I'm new to development and have mostly just been doing frontend Angular/React stuff - but no real backend development. Do you have nay suggestions or can you maybe point me to some resources that would position me nicely once we're stable enough to build stuff here
  94. for fullstack development ^
    (edited)
  95. pfrazee
    at this stage, https://atproto.com really has everything
  96. alextoma
    👍🏼
  97. alextoma
    Message deleted
  98. testgonzo joined the room
  99. testgonzo
    New here. How are Bluesky and Matrix the same and different?
  100. alextoma
    I believe Matrix is built on top an open source decentralized protocol built specifically for decentralized communication
  101. AT protocol is being built for social applications
  102. social networking specifically
  103. pfrazee
    yep that pretty much nails it. Different purpose, so different design choices
  104. David Cueter joined the room
  105. tehhp joined the room
  106. sanequest joined the room
  107. Daniel Holmgren
    yup! as well as the various views tests
  108. anath4ema joined the room
  109. @qeboos:matrix.org joined the room
  110. @qeboos:matrix.org left the room
  111. bnewbold
    I started working on a CLI too to poke at PDS implementations: https://gitlab.com/bnewbold/adenosine
  112. was able to poke at the CAR repo output (from atproto implementation) using the 'ipfs' command line tool, which was kind of cool: https://gitlab.com/bnewbold/adenosine/-/blob/main/notes/ipld_car_explore.md
  113. Aaron Goldman
    https://github.com/web3-storage/ipfs-car Is a good model for working with car files
  114. bnewbold
    Aaron Goldman: that looks like it is mostly oriented around IPFS files (aka, files and directories encoded as UnixFS, including all the tree metadata and raw content blocks) in CAR files.
    In the case of atproto, the content is in IPLD dag-cbor blocks (including both the MST tree and user records), which can then get serialized out at CAR files
  115. so, "unpacking" the CAR file to disk probably wouldn't work. though it might make sense to have a tool that would "unpack" a user's repository in to record files on disk, and then "repack" that in to a CAR? hmm...
  116. Aaron Goldman
    Yeah this one is for UnixFS not ATP
  117. bnewbold
    I actually couldn't find a full CAR implementation in rustlang. 'fvm_ipld_car' seems unmaintained, and I didn't see an implementation in libipld. this code hasn't been pushed as a crate, but I noticed other folks have vendored it out: https://github.com/n0-computer/iroh/tree/main/iroh-car
  118. Aaron Goldman
    I like CAR files but am still not sure if it would be better to have SQLite as the default on disk format so we could have the block store, the DID Document store, and a table with the DID to latest comment mapping all in a single file
  119. If clients are doing a lot of syncing are they better off just appending more blocks to the end of a CAR or inserting into a database.
  120. bnewbold
    there are several IPLD-in-sqlite implementations out there, it looks like. I think CAR is a good interchange format (just about the simplest thing). the CARv2 stuff would work better for working from disk. but yeah for anything "online" (heavy reads/writes) a different blockstore makes sense
  121. the "heavy" client use case isn't very clear to me reading the current docs. there are hints of UCANs and delegated authority? but i'm not sure that most clients ever need to look at the actual repo format (binary, MST, IPLD, CBOR) as opposed to just records in JSON that they can cache locally
  122. Max (@max:maxstuff.net) joined the room
  123. lk251

    In reply to this message

    Thanks for the information, I understand why you may want to build this way. My client is for twitter right now but will make it bluesky compatible as soon as you guys open up. Forgive me for this terrible terrible question but is there an ETA? 😄
  124. In reply to this message

    Here in the EU we now take natsec extremely seriously and we urgently need an open version of Twitter one way or the other. The thinking is , list of “tweets” is Creative Commons and then private sector builds clients with their own content curation capabilities
  125. Børlaag joined the room
  126. bnewbold
    i'm pretty curious if the ATP model would work well for things like reddit or forums. it's certainly technically possible implement, but I would guess that expectations around tight coupling and experience could be hard to achieve. eg, comment threads updating immediately. in that case you would expect moderation to be local to the group or specific content, but it probably won't work that way
  127. authors could manage comments on a blog post by curating comments from the full atp-space (?) and marking them as interesting via link as a specific record type. eg, similar to how
    arstechnica.com
    has author-curate interesting comments (plus a full feed of comments). I think some newspapers work this way now also?
  128. bookmarking and link tagging (like delicious), or webannotations, also interesting to think about. fit well with the local + aggregation model
  129. Tauqueer Khan joined the room
  130. Gerben

    In reply to this message

    FYI I just made a
    proposal
    /
    PoC
    for locally aggregated web annotations (based on the w3c web annotation protocol, so http, but that's easily swapped for any other).
  131. duc [they] joined the room
  132. lethargicpanda joined the room
  133. Neil M joined the room
  134. jrevi joined the room
  135. arcalinea

    In reply to this message

    The protocol development is taking place on Github, but we're not publicly committing to timelines as to when we think it'll be "done" right now because there are still unknown unknowns in getting everything to work right. We'll definitely let people know when we think it's mature enough to handle serious usage.
  136. izotos joined the room
  137. yukaii joined the room
  138. @tehaess:mozilla.org joined the room
  139. @likecoin:matrix.org joined the room
  140. alkabir joined the room
  141. lk251
  142. Forgive me if off-topic, I’m not sure if it is.
  143. In reply to this message

    Oh fantastic!! Yes that is totally understandable. Thank you for answering my questions. I believe your work is extremely important. Hope I can somehow contribute in the future (I’m a machine learning guy with some experience in iOS dev)
  144. PS. I’ll totally be interested in hearing when you guys are ready for un-serious usage too.
  145. hazycrow joined the room
  146. flo24 joined the room
  147. @tehaess:mozilla.org left the room
  148. realgizmoo joined the room
  149. maia joined the room
  150. whyrusleeping
    I dont know if anyone outside of elons head knows what elon is going to do
  151. We would welcome the challenge of migrating twitter to bluesky
  152. @penaiple:midov.pl
    if its free speech now that means you can tell elon to go fuck himself for making proprietary software
  153. if he censors it, its not free speech
  154. realgizmoo
    I have already migrated from Twitter. We can turn Twitter to yahoo
  155. Aaron Goldman

    In reply to this message

    more a discussion for #general-bsky:matrix.org then the dev channel but censoring speech that is designed to disrupt the ongoing conversation can enable more free speech.
    Like how having rules for how you use roads can enable more freedom of movement. The fact that we restrict to driving in the right side of the road enable cars to move further faster despite removing some movement options.
  156. @penaiple:midov.pl
    speech doesnt get in the way, especially on text
  157. unlike cars or other physical things
  158. ignore it if you feel annoyed by it
  159. Aaron Goldman

    In reply to this message

    It would be a shame it there where only delegated clients this would leave the ability to post as you and move your home sever in the hands of the PDS. One of my goals is that the PDS can go out of business with no warning and you from your client can authorize a new PDS and upload a full copy of your repo with no assistance from the old/non-existent server.

    This means that you need a client with a always up to date repo of your content. For something like the "my twitter data" archive or ,i think, the mastodon migrate your data archive it is expensive to export an archive everyday and infeasible to do it on every action.

    For a Merkel Search Tree it is a persistent data structure with good structural sharing so the diffs are small. You can compose your change(posts) commit locally and then upload to the PDS. You have an up to date repo and the PDS lags only by what is committed locally but not pushed to the server.

    Users could use only the delegate client but that like using GitHub but only ever the web editor would lose much of the value of repos.

    (edited)
  160. In reply to this message

    happy to disuse this in a different channel so as not to disrupt this one
  161. naturaprodet joined the room
  162. Boofcario joined the room
  163. naturaprodet set a profile picture
  164. Boofcario set a profile picture
  165. thursdays joined the room
  166. neo joined the room
  167. Melinda Black joined the room
  168. doodlewhale joined the room
  169. Phillip Ressler joined the room
  170. @swelch:matrix.org joined the room
  171. sq76gz6xgr joined the room
  172. yancyscot joined the room
  173. @kitdiesel:matrix.org joined the room
  174. bnewbold

    In reply to this message

    For sure I think the ability to keep a local mirror/copy important. the existing client/structure allows that easily; it is the same mechanism by which aggregators would scrape, etc.
    The open question to me is whether the "append records locally and push to PDS" makes much as much sense. my read between the lines is that it is probably possible/supported for folks that really want to do that, but the majority of users would not do things that way
  175. bala thebug joined the room
  176. hajx joined the room
  177. hajx
    Hi guys, I can't seem to access bluesky. Can someone help me out?
  178. In reply to this message

    So are you using Mastodon now while waiting for bluesky to release or are you not using any socials now?
  179. MN joined the room
  180. probegalaxy joined the room
  181. probegalaxy
    gr8 to be in this new world..
  182. Anders C. Sørby joined the room
  183. @joejenett:matrix.org left the room
  184. MN

    Find here from IPFS Camp 👋

    https://youtu.be/OxmndRBGuUM?t=3704

  185. astrohsu changed their display name to MN
  186. gooseism joined the room
  187. whintr joined the room
  188. jeff_barutt joined the room
  189. sevensixseven joined the room
  190. Pilot_51 joined the room
  191. rexamazing joined the room
  192. teksquisite joined the room
  193. awestin joined the room
  194. c8lin joined the room
  195. Ilan Shamir joined the room
  196. @jerrylerman:matrix.org joined the room
  197. salted joined the room
  198. korvv joined the room
  199. @null_radix:matrix.org changed their profile picture
  200. Magnolia 🇺🇦 joined the room
  201. incloud joined the room
  202. @daniel793:matrix.org joined the room
  203. @oscarrdg:matrix.org joined the room
  204. Mark Diaz joined the room
  205. skopintsev.denis joined the room
  206. frh2pqtgzx joined the room
  207. @rimuru:gentoo.chat changed their profile picture
  208. Rimuru changed their display name to spooky spook 🎃
  209. Human Mirjalali joined the room
  210. otsaloma joined the room
  211. @hms_brexit:matrix.org joined the room
  212. alrg joined the room
  213. Dwayne joined the room
  214. Alex (@ajs1998:matrix.org) joined the room
  215. Andrew joined the room
  216. Zufrizal Yordan joined the room
  217. u64 joined the room
  218. Ennis joined the room
  219. @washran:matrix.org joined the room
  220. tapatio joined the room
  221. hslim joined the room
  222. Tony Catapano joined the room
  223. Søren Gravesen-Hvass joined the room
  224. alexozy joined the room
  225. @dk:effektio.org joined the room
  226. 97dv77c46q joined the room
  227. willscott
    arcalinea's talk on "Bluesky and IPLD" from IPFS Camp this weekend is also on youtube: https://www.youtube.com/watch?v=jGbBZbl-V8Y
  228. Michael Trainor joined the room
  229. jacobrussell joined the room
  230. jzs joined the room
  231. puemos joined the room
  232. zekarlos joined the room
  233. @vinay-keshava:mozilla.org joined the room
  234. @alexandro:foilhat.tk joined the room
  235. George Antoniadis

    I'm trying to wrap my head around what options indexers/crawlers have and was wondering about a couple of things I couldn't find information on.

    a. Can everyone methods that sync repos from a PDS? ie com.atproto.syncGetRepo?
    b. Is a client allowed to rewrite their commits to the MST/history after the fact? If so will the PDS complain/warn the user?
    c. Is there a plan for the PDSes to provide aggregates about their repos, or will the indexer/crawler have do deal with retrieving updates for individual repos from the same PDS? I'm wondering mainly about how actions such as follows/likes can be easily retrieved without having to pull all repos from all "relevant" PDSes.

    ps. Although I tried searching in this channel as well as the github repo, I still might have missed something so I do apologise. In which case a link or pointer to where I should be looking at would be appreciated :D

  236. @alexandro:foilhat.tk left the room
  237. vicente depdel joined the room
  238. mustafa_123 joined the room
  239. @lisko:matrix.org joined the room
  240. Mark Foster SSI: @mfoster.io
    Good Morning today’s forecast calls for Blue Skies
  241. pfrazee
    haha you have no idea how often that song gets stuck in my head
  242. Mark Foster SSI: @mfoster.io
  243. sylphrenetic
    for me it's Goodbye Blue Sky by Pink Floyd
  244. @lisko:matrix.org
    Does the project have an official soft drink? Because I have an idea
  245. pfrazee

    In reply to this message

    that's the other one
  246. Mark Foster SSI: @mfoster.io
    pfrazee: It’s one of my favorites, gives me hope for a brighter future. The internet is upside down, time to turn it right side up again!
  247. pfrazee

    In reply to this message

    I know! It's a very lively and hopeful song. We need to get the rights or something if we ever do media
  248. Mark Foster SSI: @mfoster.io
    I saw Pink Floyd in concert with all my high school buddies, it was a memory that I will never forget!
  249. In reply to this message

    💯
  250. pfrazee

    In reply to this message

    no way. What year??
  251. (this is so totally breaking the on-topic rules but I'm abusing my power as moderator to allow it)
  252. @char.mitro:matrix.org joined the room
  253. bnewbold

    In reply to this message

    (not an authority, just my thoughts)
    a: I assume anybody can pull repos at any time... and also mirror/redistribute them, though not canonically. unclear what the norms will be about bandwidth and rate-limiting, but HTTP/XRPC has a rate-limit status code (429). checking if things have changed is a lightweight read. I think of https://www.softwareheritage.org/ bulk scraping git repos, or RSS feed readers, as prior art
    b: I'm not sure how clients could, via the existing protocol. pushing new commits via sync import which are not children of the current HEAD commit? the related issue I would see, which would impact the whole ecosystem (aggregators, etc) is if a user updates their DID doc to point to a new PDS location, which has a new origin commit. the analogy there would be swapping out a git repo with an entirely new commit history with no shared origin commit
    c: it feels like one would need to do at least partial repo pulls to have accurate follow/like counts. eg, could just pull the follow/like 'collection' key prefix parts of MST trees. I sort of assume an ecosystem of big providers/aggregators that have relatively "complete" picture of the graph, and then some fraction of smaller PDS instances. the small instances can just aggregate "small world" stuff (eg, direct follows/likes then one or two hops), and then users of those could optionally use the large aggregators to discover new folks to follow or links back, if they are curious. most users would probably go with a large/complete provider if care about seeing all backlinks
  254. Ghanshyam Ramawat joined the room
  255. Ghanshyam Ramawat set a profile picture
  256. Mark Foster SSI: @mfoster.io

    In reply to this message

    Oh man, you are going to force me to show my age. It was 1994 right when I graduated highs school, yikes!
  257. protonwav joined the room
  258. codebarber joined the room
  259. Patrick van Kleef joined the room
  260. committed joined the room
  261. pfrazee
    haha okay 10 years before I did, it's cool. In tech years we're the olds around here
  262. but that's awesome, I would've loved to see that. I have a live pink floyd album called Pulse that I think was recorded around then. It's my favorite version of many of their songs
  263. anyway we now return to our previously-schedule channel topic
  264. Mark Foster SSI: @mfoster.io

    In reply to this message

    Ha yes it is was the Pulse tour
    (edited)
  265. pfrazee
    ah man, super jealous
  266. zdeexploit joined the room
  267. Aaron Goldman

    In reply to this message

    B) The latest head of the repo is the one that the home server returns and is signed by the key from the DID Document.
    If the server updated to a commit that had no previous pointer or one that did not point to the last commit that is its wright. The old stale head is still valid just stale but there is no obligation that it be reachable from the latest head. Useful if you need to purge data from the repo.
    C) The problem with aggregates is that you are trusting whoever made the aggregates that they are not adding or removing data. The servers will likely public some materialized views like the list and number of likes but to validate any particular like you would need to pull the repo with that like in it. Fortunately the repos are trees so a server that wanted to validate a like could do a narrow checkout of the repo with only the one record and the path to it. log(n) in the number of records in the repo.

    The value of an indexer is that it is doing the work of pulling from many repos and building the aggregates so that a PDS that wants to show big would data like the set of replies to a post.

  268. snarfed joined the room
  269. snarfed set a profile picture
  270. bnewbold
    have problematic CIDs been a problem for IPFS? aka, brute-forcing block content to generate a CID (hash) with an offensive word in it. not much of a problem if the CID is never included in a URL or user-visible I guess
  271. pfrazee
    yeah we dont use CIDs in URLs, usually just as a strong version identifier (and then, ofc, internally within the repo data structure)
  272. what attack vector do you have in mind?
  273. bnewbold
    just people being trolls
  274. pfrazee
    sure but how are you thinking?
  275. whyrusleeping
    With my brain, i think
  276. pfrazee
    therefore you are
  277. bnewbold
    I guess it would be something like generating a post or piece of content, getting other folks to link or reply to it without seeing the CID/URI, and then some alternative user interface might show the full CID/URI/whatever by default. then it looks like the person has reposted/etc something with content in it
  278. whyrusleeping
    Im just your friendly neighborhood ipfs developer here to tell you that brute forcing cids isn’t real
  279. Breaking a 256 bit hash is hard
  280. pfrazee
    heh yeah shouldnt be a concern
  281. bnewbold
    ^ I don't think this is a real/serious concern
  282. pfrazee
    good to check in on it though
  283. bnewbold
    separately, I was thinking about users being able to brute-force arbitrary MST structures (eg, a top-level node with like 100k entries). I guess that just makes things slow though, you would presumably already have some other limit on receiving IPLD blocks with weird/large sizes. so I think this also isn't a real/serious concern
  284. pfrazee
    separately but related, we use CIDs in various places to minimize this exact kind of concern
  285. for instance, likes and reposts include the CIDs of the content so that future edits cant be used to troll people
  286. bnewbold
    oh yeah, I saw that and was wondering if it was the CID of the content, or the commit CID of the repo tree containing the content at that point in time
  287. pfrazee
    I believe the CID of the record, Daniel Holmgren would have to confirm
  288. they have similar meaning but record CID is more convenient IMO
  289. and would continue to be valid if the repo ever had to rebase itself, which would be a strong positive
  290. bnewbold

    In reply to this message

    I mean, brute-forcing .onion URL prefixes is a thing, right? I guess even that is sort of expensive/hard
  291. pfrazee
    it is but only to create vanity prefixes
  292. @vecn:halogen.city joined the room
  293. pfrazee
    meaning they brute-force a small subset of the output hash
  294. I'm not a cryptologist, but I believe it's supposed to be computationally infeasible to bruteforce a full sha256
  295. bnewbold
    right, but that is all I meant by CID trolling earlier. creating a CID where the first 5 characters or whatever are a slur
  296. pfrazee
    oh yeah. Well. Yeah I suppose you could do that.
  297. mikuhl
    People create vanity Bitcoin addresses like that, it's possible yeah
  298. pfrazee
    I dont think CIDs will ever show up in the UI. DIDs will but in very rare cases
  299. I suppose if you were going to mine for naughty words, that's where you'd do it
  300. bnewbold
    I can already foresee a market for did:plc identifiers with as many leading "lucky 8" digits as possible
  301. mikuhl
    When do you guys think you'll have some UI to show off or use?
  302. I am excited to see what the client looks like 😌
  303. pfrazee
    got the dev build right in front of me
  304. mikuhl
  305. pfrazee
  306. here's a small preview screenshot:
  307. mikuhl
  308. I think I see that!
  309. pfrazee
    haha oh no I've shared too much
  310. mikuhl
    That must be a home icon or something though 😂
  311. pfrazee
    heh yep
  312. Aaron Goldman

    In reply to this message

    And fortunately because did:plc allows the rotation of the account and recovery keys a user can buy a did:plc and still have full control independent from the original creator.
  313. `did:plc:[2-7a-z]{24}` but maybe not '8's
  314. I believe it was b32
  315. -=DIESEL=- joined the room
  316. @hms_brexit:matrix.org left the room
  317. -=DIESEL=- set a profile picture
  318. random() joined the room
  319. Mark Foster SSI: @mfoster.io
    whyrusleeping: pfrazee I had an idea to see if I could create a CID Hash for some the common W3 rdf vocabularies and use them in SPARQL queries I did run into a problem and had to build a cloud flare worker to create content-type:turtle I didn't see a way to do this in dns link
  320. pfrazee
    yeah you were showing me this earlier right?
  321. Mark Foster SSI: @mfoster.io
    Yeah not really a BlueSky question more of a dns link and IPFS question
  322. pfrazee
    right yeah
  323. Mark Foster SSI: @mfoster.io
    It did get me to thinking about the context of blue sky and IPLD
  324. multiformats
  325. @wclayferguson:matrix.org
    I'm just glad ATP is using Merkle Trees! I've had "marketing geniuses" telling me for decades to stop using the word 'tree'...and I always politely decline that advise!
  326. There's a good reason that since about 1967 "trees" have been how humans organize our info. :)
  327. Mark Foster SSI: @mfoster.io
    pfrazee: I am finally getting around to read through the docs, what I am picking up on is there are two Lexicon examples
    atproto.com
    and
    bsky.app
    . I am interpreting it as if I want to create
    cats.com
    build ontology and shapes around cats I would do that in JSON Schema and only allow that Lexicon in my app at
    cats.com
    . Is this correct thinking?
    (edited)
  328. Medhat Yassin changed their display name to -=DIESEL=-
  329. 민주 joined the room
  330. Aaron Goldman

    In reply to this message

    Mark Foster: did you just invent cats again?
  331. Daniel Holmgren

    In reply to this message

    yup this is right. CID of the record & for the reasons you said: that the CID stays the same through rebases & doesn't require having the repo history
  332. CIDs also have a multiformats prefix on them (usually "bafy") so you can't mine for the first letters in it. basic you could do is bafy${word}... which is less obvious & CIDs will basically never make it into userland
  333. Mark Foster SSI: @mfoster.io

    In reply to this message

    Absolutely I am ready to give Small Kizzle an account! @
    SmallKizzle.com
  334. @wclayferguson:matrix.org
    ...but the young kidz (i.e. under 30) are all gunna be mining for 80085 in the first chars.
  335. we simply cannot allow them to "abuse" hexadecimal in this way.
  336. pfrazee

    In reply to this message

    it's an epidemic
  337. In reply to this message

    time to ban <30 y/os
  338. Cas Hoefman joined the room
  339. @wclayferguson:matrix.org
    I used to type naughty numbers into my radio shack calculator as a kid, and my teachers would bust me by hitting the memory recall buttons on the dang thang.
  340. @lisko:matrix.org
    That sounds extreme
  341. Mark Foster SSI: @mfoster.io
    What people call Web3 I call Webπ because the SemanticWeb is Web3.0. Is the IPFS IPLD web considered Webπ? We have a group called SemGems 🧠💎 for us Web3.O guys by the way. I’m working on Web^(Ω1+Ω2+Ω3+Ω𝝅+Ω5+∞) = Future Internet or FutureWeb, all the webs!
  342. @planetoryd:matrix.org

    In reply to this message

    IPFS won't realise web3 on its own. Use Locutus protocol
    (edited)
  343. @prof.seagull:matrix.org joined the room
  344. prof.seagull changed their display name to zack
  345. zack changed their display name to zack (he/him)
  346. zack (he/him) changed their display name to Zack (he/him)
  347. @prof.seagull:matrix.org removed their display name (Zack (he/him))
  348. DFPintol joined the room
  349. Ashish Tandi joined the room
  350. mib1289

    In reply to this message

    I’m adding nothing to the conversation here, but that is literally the greatest metaphor ever…..
  351. day27182 joined the room
  352. day27182 set a profile picture
  353. @vecn:halogen.city left the room
  354. day27182 changed their profile picture
  355. whyrusleeping
    Alright y’all stay on topic
  356. Mark and planetoryd those posts both sound like something youd see under a tweet that mentions metamask, lets keep the discussion focused on bluesky development
  357. @lisko:matrix.org
  358. @daniel793:matrix.org left the room
  359. Matteo Totaro joined the room
  360. donflaminggo changed their display name to DFPintol
  361. ozguner joined the room
  362. doesitsoeasley joined the room
  363. @georgishere:matrix.org joined the room
  364. @secu:mtrx.nz joined the room
  365. @toranosora:matrix.org removed their display name (Eren)
  366. Mark Foster SSI: @mfoster.io
    whyrusleeping: Definitely, I had some questions about IPLD and how it will allow tree repositories to person data stores. Is there some documentation or previous project references on some of the approaches in the Context of BlueSky?
  367. pfrazee
    do we use ADLs?
  368. Aaron Goldman
    I think most of the HAMTs where replaced by MSTs
  369. pfrazee
    yeah I think that's right. I think we use IPLD for multiformats/codecs/etc, the canonicalized cbor encoding, the CAR file format, and the general tooling
  370. Aaron Goldman
    But it is an example of IPLD trees
  371. pfrazee
    I dont think we ever got into using ADLs (advanced data layouts) to abstract it
  372. anyway Daniel Holmgren is the canonical source of knowledge here, but why might know too
  373. Mark Foster SSI: @mfoster.io

    In reply to this message

    What are your thoughts on Schema mapping to ADL? https://ipld.io/specs/advanced-data-layouts/hamt/spec/#schema
  374. pfrazee
    it can be a useful abstraction but I'm not in a position to evaluate if ATP should use it
  375. which really should indicate how much the IPLD elements bubble up to the application developers' concerns: almost not at all
  376. @yamcube:halogen.city joined the room
  377. Kaisa joined the room
  378. Mark Foster SSI: @mfoster.io

    In reply to this message

    I agree, I need to dig into IPLD more and haven’t had time., where are the communities congregating around this topic?
  379. sindri_la_mancha joined the room
  380. @yamcube:halogen.city removed their display name (yamcube)
  381. @yamcube:halogen.city left the room
  382. Savi🌹🥰 joined the room
  383. Obi Sopulu changed their display name to Savi
  384. Savi changed their display name to Savi🌹🥰
  385. jasondavies joined the room
  386. SwyZe joined the room
  387. Daniel Holmgren
    We haven't formally specified an ADL, but it's in essence what we're doing with the MST that we use. We use an MST instead of the already-specified HAMT because we want sort-ordered keys while retaining log(n) proof size
  388. SwyZe set a profile picture
  389. swyze changed their display name to SwyZe
  390. Daniel Holmgren

    In reply to this message

    Github, the
    IPFS Forum
    & the
    Fission discord
    are all good places
  391. @cranelab:matrix.org joined the room
  392. Daniel Holmgren
    I used to work at Fission. They heavily use IPLD, and their Discord is a great place to hang out. Not spammy & lots of interesting discussions :)
  393. @cranelab:matrix.org left the room
  394. @mks:matrix.org joined the room
  395. @rimuru:gentoo.chat removed their display name (spooky spook 🎃)
  396. @rimuru:gentoo.chat set their display name to rimuru
  397. @rimuru:gentoo.chat changed their profile picture
  398. Wilian Valle joined the room
  399. Cyanic Cipher joined the room
  400. b0gg3r joined the room
  401. DFPintol
    so...
  402. does this mean we can use IPLD for indexing web3?
  403. i.e. ipfs sites
  404. @wclayferguson:matrix.org

    Have you guys thought much about how ATP might be able to introduce users to the Semantic Web? I mean like how to perhaps let users enter Duck-Typed data into the system without even having to have a special type-specific GUI. One interesting concept is to just have a "standard" (really a convention) where people can just embed YAML directly into their markdown like this:

    123name:
      first: Clay
      last: Ferguson
    

    That would be a way to get the Semantic Web booted up. Apps could choose to strip out the YAMO for most displays, or have a "Display Properties" kind of feature to render it as text, and yes, in some cases even present the typed text as a GUI Editor. Regardless, the indexing engine would be a billion times more powerful if people could just arbitrarily put in typed info that can become searchable, if the DB is something like MongoDB and can hold arbitrary key/values on records, etc.

  405. Sales Manager joined the room
  406. @wclayferguson:matrix.org

    In 10 years from now, people can just be told, "Cast your vote by putting something like the following in your Gov't Record"

    12potus:
         MrWhoever
    

    And that can literally be how voting is done.

  407. That's a "pie in the sky" example, but proves the power of the concept.
  408. Remember when HTML was first invented people would've said Markdown was completely silly and outdated. :) Simpler always wins. Another analogy: XML v.s. JSON.
  409. @wclayferguson:matrix.org
    ...but really even a variation on the #hashtag that has an extra delimiter allowing key/value might even be better since the entire planet already knows what a hashtag is and does.
  410. (not to mention "microformats"...but I'll leave it at that)
  411. pfrazee
    I'm not totally sure whether at this stage whether RDF support like that will be big gainz, but let's ignore that for the moment --
  412. lexicon and ATP have at least one key property that's important for RDF support and that's global term names
  413. lexicon has a URI scheme, lex:, and ATP has a scheme at:
  414. @wclayferguson:matrix.org
    cool, like a namespace.
  415. pfrazee
    so the URI for a post record schema is lex:app.bsky.post
  416. and the URI for a post record is eg at://alice.com/app.bsky.post/1234
  417. both of these URI schemes also have a defined behavior for the hash segment: they're interpreted as JSON pointers
  418. @wclayferguson:matrix.org
    I started thinking about this last week because I have an index of about a million Fediverse Usernames (collected by crawling) and I was going to build a searchable index using each person's Bio Text but there's really no structure there. Would be nice if data had some structure to it, for searching purposes.
  419. pfrazee
    so within the app.bsky.post you'll find {..., "record": "properties": {"text": {......}. Aka at postSchema.record.properties.text you'll find the definition of the text field for posts
  420. which means the text field in a post should be globally termed as lex:app.bsky.post#/record/properties/text
  421. @wclayferguson:matrix.org
    BTW, I'm aware of your Beaker work and how much you've had experience with typing, etc.
  422. pfrazee
    haha cheers yeah, lots of years
  423. so anyway, my point is that I think the RDF graph triple for a post's text record would be
  424. <at://alice.com/app.bsky.post/1234> <lex:app.bsky.post#/record/properties/text> "Hello, world!"@en
  425. @wclayferguson:matrix.org
    I'm glad you mentioned RDF I had temporarily forgotten about that.
  426. But yeah that RDF is mostly "computer consumable" not "human editable" tho, which is good for what it's doing, of course
  427. pfrazee
    yeah I mean, RDF has a design constraint that we don't: it has to generalize to systems that share almost zero assumptions
  428. @wclayferguson:matrix.org
    Even if the world knew what "#hobbies=cars,motorcycles,coffe,linux" meant it would be cool, but maybe just plain hashtags gets us 75% of the way towards that.
  429. Anyway I just wanted to bring up the topic, that's all.
    (edited)
  430. pfrazee
    yeah np, it's something I've been idling over recently
  431. I understand RDF decently well -- more than the average dev, but I've never spent real time programming against RDF databases or anything
  432. but even then I'm only... 80% sure I formed the graph triple correctly above
  433. @wclayferguson:matrix.org
    I need to dig into RDF much more myself, for sure.
  434. pfrazee
    RDF is extremely elegant for what it's trying to do, but it's like the assembly-language of semantics. It's really hard to get to basic comprehension and it's super verbose
  435. Troubletortuga joined the room
  436. Abhishek Gedela joined the room
  437. Aaron Goldman

    In reply to this message

    I think the query parameters syntax dose this with already existing parsers

    1?arg1=value+1&arg2=value2&arg3=123abcABC-_&arg4=true&arg5=5
    

    one could image a syntax like

    1lex:app.bsky.rel_me?did=did:plc:abc&handle=alice.example.com
    

    that rendered oddly

    (edited)
  438. @wclayferguson:matrix.org
    Aaron Goldman: What I had been thinking about was if there were some sort of way to embed structured information in the actual content text, like how hashtags are used, and I guess it would be some sort of extension of the Markdown syntax or even a slight variation on the hashtag "syntax". Of course "syntax" is a big word I'm using just for saying it's something that starts with "#" hash symbol.
  439. Of course, embedding YAML in a fenced code block in Markdown can do it beautifully, actually without a new syntax, and all existing parsers can basically handle it.
    (edited)
  440. By 'fenced code block' I mean between three backticks above and below the YAML.
  441. so...for example people could include that in their Bio Text (like an ActivityPub User Bio) and specify things like name, email, gender pronouns, hobbies, interests, etc.
  442. but really it just opens up a whole new world of "Semantic Web" if apps start finding and parsing that.
  443. vinay-keshava changed their display name to vinay
  444. uguboz joined the room
  445. Kev Cas joined the room
  446. @jeromu:matrix.org left the room
  447. digharatta joined the room
  448. @obsmatr:matrix.org joined the room
  449. digharatta

    In reply to this message

    Please check the work done in this regard at the Open Recognition Framework https://www.openrecognition.org . This would really open up a whole new world of "Semantic Web".
  450. Video conference started by arcalinea
  451. zeandrn joined the room
  452. André LFV joined the room
  453. Video conference ended by arcalinea
  454. arcalinea
    Oops -- I don't even know what I clicked to start a video conference in here
  455. pfrazee
  456. maybe one of those
  457. @numero6:codelutin.com

    In reply to this message

    I think it's still jitsi but in the future, it will be element call (still in beta but with some P2P in it) https://call.element.io/ give it a shot 😉
    (edited)
  458. arcalinea
    i don't recall even opening element. might've just clicked it when reshuffling screens. anyways, didn't mean to start a video call, sorry for the interruption.
  459. tmw (he/they) changed their profile picture
  460. Henrique S. Oliveira joined the room
  461. b0gg3r
    Been digging into the docs on
    atproto.com
    I realise it's early days, but is there a "shape" for what server-to-server syncing might look like? Trying to get a sense for how public data like posts, etc are stored compared to DMs, etc.
    (edited)
  462. pho joined the room
  463. pfrazee

    In reply to this message

    the core of server-to-server sync is akin to git repo sync. The servers are replicating each others' data repos.
  464. there are two "parameters" to a sync: narrow (yes/no) and shallow (yes/no)
  465. narrow is where you're syncing a subset of a repository, aka just the record types you're interested in
  466. shallow is where you're not interested in the full history, just the latest state
  467. narrow & shallow will be the most common form of sync we expect, since it gives just the data needed to run a given application
  468. default sync is pull based; my PDS actively syncs the repos I follow
  469. events which require a push, like mentions & replies & etc, will do a very-narrow & shallow push of the relevant record
  470. selectively-shared data like DMs are not fully specced yet but have been actively discussed by the team and we have solutions in mind. Just dont want to talk about them before we're sure, to avoid confusing folks
  471. @wclayferguson:matrix.org
    Merkle Tree syncing algorighm will be super simple (and efficient) too because once you hit a branch with matching hash (or even the root of someone's repo) you'll know the entire subgraph is a match and not even have to drill down into that node of the tree. Don't tell my ActivityPub Frenz I said so, but it'll be light-years ahead of how ActivityPub outboxes work. :)
    (edited)
  472. b0gg3r
    Yeah, kinda blown away at how viable crazy architecture is made by merkle trees
  473. Would be interesting to integrate SIEM style/little snitch style security alerts on the PDS Would catch scraping, or if an application attempts to access data it shouldn't. Could be federated as well if applications are known to be malicious
  474. I'm sure ye've considered this. Just spitballing
  475. @wclayferguson:matrix.org
    Probably secret data will be encrypted, so it won't matter if the entire repo is publicly accessible.
  476. pfrazee
    I'm personally not a huge fan of handling selectively-shared data by published encrypted records
  477. b0gg3r
    Would my photos be encrypted, and then I furnish a key to an application? Or would they be considered public (in a instagram style scenario)
  478. In reply to this message

    Oh?
  479. pfrazee
    yeah. It's a supportable choice but you're allowing your ciphertext to be distributed widely. If that key leaks somehow, you're going to be pwned in a very public fashion
  480. I'd prefer we use at-rest encryption plus an alternative system for sending the data
  481. b0gg3r
    So decrypt within the PDS, then re-encrypt and send when requested over the network?
  482. pfrazee
    honestly I'd rather just use a separate protocol. Every user has a public server. You can just send POST requests to them
  483. @wclayferguson:matrix.org
    I was thinking along the lines of if my repo root has one hash, that has to contain all the data.
  484. I guess there could be a public repo, and a separate one for each secret group of friends or something. But I'll quit speculating until I read the latest docs, which I haven't done yet. :)
  485. b0gg3r

    In reply to this message

    I think what pfrazee: was proposing is that the entire repo is shared publicly, but that the access control is not done via the decryption key. Instead the PDS manages access.

    Lmk if I got that wrong

  486. @wclayferguson:matrix.org

    In reply to this message

    I see. thanks.
  487. I was thinking in terms of big indexing services eating up all the data and making it searchable, but it sounds like you mean something more "peer-to-peer" than I had thought...where a server sends data only to, and directly to, the users who are allowed to see it.
    (edited)
  488. pfrazee
    that's actually not 100% my position right now but yeah, this is what I meant with not talking about it publicly -- because we haven't nailed down how we think this can work and I dont want to give half baked answers
  489. b0gg3r
    You gave a pretty good "this ain't done" disclaimer
  490. It's an interesting exercise
  491. pfrazee
    yeah this one is spicy, it has big implications and it's a difficult problem
  492. to roughly encapsulate my position, I believe it's possible to provide fine-grained selective sharing with at rest encryption, but I'm pursuing a solution that does that "off repo" so to speak
  493. having worked on very similar systems through the years (SSB and Dat/Hyper both had repo-like structures at their core) I've just never been satisfied with the "on repo" solutions to selective sharing
  494. @wclayferguson:matrix.org
    One thing I see is that for merkle-optimizations of syncing all public data will need to be under a single root/branch, and the stuff that's fine-grain shared would need to be in it's own separate "branch".
  495. pfrazee
    if you put private data in a branch then you end up producing new commit heads for private writes
  496. so you reveal when you do a private write, plus it adds a lot of useless data sync
  497. @wclayferguson:matrix.org
    well if you touch anything on a tree the merkle-root changes. All nodes that are an ancestor of the one written to will change their hash.
  498. For a merkle write you have to walk up the tree at every level updating the hash and eventually the root hash has to chagne.
  499. IPFS cannot do this, which is why they require that you can only add children, and unless you're using something like MFS you can't do that "walk up to the root" kinda thing. if I' making sense.
  500. That's the benefit of a merkle-tree for syncing. If the root hash matches then the whole tree is a match.
  501. pfrazee
    right, that all supports my point though
  502. @wclayferguson:matrix.org

    In reply to this message

    Cool.
  503. pfrazee
    I think the system remains much simpler if you don't try to put access control within a repo
  504. repo-level access controls, that might be doable (can I access anything or nothing)
  505. at-rest encryption in the repo doesnt complicate the system too much, but it's hard to produce the exact security properties you want
  506. @wclayferguson:matrix.org
    Right, I was thinking more about efficiencies of syncing rather than the "who can even see data" part of it.
  507. pfrazee
    yeah
  508. @wclayferguson:matrix.org
    but both are equally important things of course.
  509. b0gg3r
    Seems like branches wouldn't solve it either since you'd need a branch for every DM conversation, group chat, private forum, etc
  510. pfrazee
    right
  511. b0gg3r
    And if it's left to the application layer, portability isn't as guaranteed?
  512. pfrazee
    generally that's true, but we intend to solve this
  513. b0gg3r
    So an alternate protocol might be some way of storing private data such that the access issues and portability is enforced, and that protocol would be part of the ATP stack? Really appreciate the explanations
    (edited)
  514. pfrazee
    that's the route I'm advocating/pursuing within the team. Nothing here is settled
  515. b0gg3r
    I look forward to the next spec revision :)
  516. pfrazee
  517. @micbz:matrix.org left the room
  518. quiltmeow joined the room
  519. @bjf:matrix.org joined the room
  520. bsjdjjdj joined the room
  521. Mert Topkar joined the room
  522. molonelaveh joined the room
  523. Mark Rachapoom joined the room
  524. Mark Rachapoom
    hello, I am new to @protocol and I have read everything on the dev section on
    atproto.com
    and, am, not quite sure where to start building. Any suggestion would be really appreciated!
  525. Are you comfy sending http requests?
  526. Mark Rachapoom
    yuppp
  527. I am
  528. I am familiar with axios
  529. In reply to this message

    oh wow, thank you so much!!
  530. 😄
  531. b0gg3r
    That was enough to get me going, get a feel for things. At this time the code is a lot of the documentation
  532. Mark Rachapoom

    In reply to this message

    sounds good! thanks!!
  533. pfrazee
    It’s worth noting that we don’t intend to leave things this difficult to use
  534. It’s just very unfinished atm
  535. b0gg3r
  536. pfrazee
    Also, if you look in the pds package’s tests, you can learn a lot — including how to use the API package, which completely wraps everything with well defined types
  537. b0gg3r
    Yeah, like it says V0.0.1 when it starts up
  538. EARLY days
  539. pfrazee
  540. Asha Ramegowda changed their display name to alrg
  541. instantchow joined the room
  542. Pavel Kofman 🇺🇦 joined the room
  543. steffen joined the room
  544. ztr joined the room
  545. Martin Shaw joined the room
  546. tangmo1994 joined the room
  547. @maxi:kayla.horse joined the room
  548. wolix joined the room
  549. wolix
    Hello there
  550. @vx_omega:matrix.org joined the room
  551. mayydayyy joined the room
  552. Ismail Esuola joined the room
  553. u joined the room
  554. Usman Maqsood changed their display name to u
  555. heli0s joined the room
  556. @bjf:matrix.org
    Hi All, I have an idea for a web app that I was planning on using bubble for as I have no real coding skills. This protocol seems to have attributes, like decentralization, portability, etc that I think would be really important to my users and the product. Can some help explain what languages are used for the UI layers for all this? I'm having trouble picturing data and task flow beginning from the user interface. Is it possible to use JavaScript? Thanks for any help!
  557. b0gg3r
    Anything that can send HTTP requests
  558. nuhae joined the room
  559. kedas joined the room
  560. @halfor:matrix.org joined the room
  561. Alessandro Festa joined the room
  562. j joined the room
  563. victorguedes joined the room
  564. @wasp:wasp.dev joined the room
  565. @bjf:matrix.org
    Thank you
  566. Dean joined the room
  567. SURGAT joined the room
  568. Dean

    Hey guys, I'm trying to use the api to create a sessionClient. Currently running into this error when connecting sessionClient

    123		const sessionClient: SessionClient = (await import('../atproto/packages/api/src'))
    			.sessionClient;
    		const client: SessionServiceClient= sessionClient.service('http://localhost:2583');
    

    Results in:

    1234567session.ts:37 
            
           Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sessionManager')
        at new SessionServiceClient (session.ts:37:37)
        at SessionClient.service (session.ts:25:12)
        at +page.svelte:17:47
    
    
  569. It's related to xrpc being undefined when initializing a new SessionClient. From api/session.ts:

    12345678export class SessionServiceClient extends ServiceClient {
      xrpc: SessionXrpcServiceClient
      sessionManager: SessionManager
      constructor(baseClient: Client, xrpcService: SessionXrpcServiceClient) {
        super(baseClient, xrpcService)
        this.sessionManager = this.xrpc.sessionManager
      }
    }
    
  570. pfrazee
    Dean: we're not ready to support devs yet unfortunately, but the API and PDS packages' tests folders are the best ways to see usage
  571. Dean
    Thanks pfrazee. I copied the logic from the test directory which is why I'm confused. I guess I'll dig deeper.
  572. atnorl joined the room
  573. Steven Franssen

    In reply to this message

    what about hash collisions?
  574. @planetoryd:matrix.org

    In reply to this message

    impossible its cryptographic hash function
  575. Aaron Goldman

    In reply to this message

    Odds of Sha256 collision:

    1/(2^256) + p(algorithm broken) ≈ p(algorithm broken)

  576. b0gg3r
    Plus I presume the hashing function could be swapped out if that day comes
  577. @wclayferguson:matrix.org
    That's why IPFS built a type of CID where the first byte (or two?) indicates the type of hashing algorithm used, but I think even Bitcoin relies on SHA256 so who knows when it will become obsolete.
  578. b0gg3r
    Infinite timeline I guess
  579. @wclayferguson:matrix.org
    People keep worrying about Quantum Computing, and i'm not up to speed on all that shenanigans. :)
    (edited)
  580. whyrusleeping
    Bluesky uses CIDs (which contain multihashes) switching hash algo down the road will be not too hard if we have to
  581. But realtalk, sha256 is fine, we really dont need to worry about hash collisions. Theres like ten billion other more likely to happen bad things
  582. mrtsunami4 joined the room
  583. orcam joined the room
  584. cz88 joined the room
  585. @bewildee:matrix.org joined the room
  586. @bewildee:matrix.org

    Hey everyone 👋

    Was checking out the site and saw the matrix link. Super excited for blue sky.

  587. Aaron Goldman

    In reply to this message

    Quantum is not a problem for sha256 but bluesky does use elliptical curve signatures that are not quantum safe
  588. nickrick joined the room
  589. Rhino Unknown joined the room
  590. hrjet joined the room
  591. @mrhacking:matrix.org joined the room
  592. @mrhacking:matrix.org
    Also interested in Bluesky. Thanks for having this chat. Hacker here, if anyone needs me to poke around and try to help iron out some bugs.
  593. mathieu joined the room
  594. b0gg3r
    Some folks have sha256 collision concerns if you can report back @mrhacking
  595. @mrhacking:matrix.org
    I can report that I don't have quantum computing capacity lol. The security of sha256 will undoubtedly be tested when quantum computing becomes the norm.
  596. However, that won't happen for awhile and if it does the hashing algorithm can be switched.
  597. Aaron Goldman

    In reply to this message

    At the moment I would bet there are work application attacks that would slow down a home server but those are fixable. I would worry more about ways to spoof someone else's repo.

    The only one that comes to mind is games with the Handels. DNS unicode lookalikes, http interception, PDS login

    If you can spoof a binding between a handle and a DID they will know you by the DID there after. Not clear how to recover from this

  598. Abhishek Rajan joined the room
  599. @mrhacking:matrix.org

    In reply to this message

    Let me think on it and get back to you.
  600. desilin joined the room
  601. MightySpaceman (OLD -> m_spaceman:matrix.org) joined the room
  602. xbolo joined the room
  603. Winning Together joined the room
  604. bnxs joined the room
  605. Shibby Shabba changed their display name to Shibby Shabba (Old)
  606. Shibby Shabba (Old) changed their display name to bnxs
  607. bnxs set a profile picture
  608. kolijoe joined the room
  609. Stefan joined the room
  610. @jdmark:matrix.org joined the room
  611. dome joined the room
  612. Aaron Goldman

    In reply to this message

    Pen-testing will more interesting once there is a beta instance to get into the details.
  613. @vx_omega:matrix.org left the room
  614. josam joined the room
  615. Joan Samper changed their display name to josam
  616. daviding joined the room
  617. @mrhacking:matrix.org

    In reply to this message

    Without a doubt.
  618. rogama25 joined the room
  619. rogama25 set a profile picture
  620. Aaron Goldman

    In reply to this message

    I suppose it's a good idea to go ahead and set up
    security@blueskyweb.xyz
    as a google group so that engineers can be emailed about vulnerabilities rather than public issues in GitHub 🤔
  621. Some things are nice to have setup before you need them 😜
  622. MainTobias joined the room
  623. @jebba:matrix.org joined the room
  624. @jebba:matrix.org
    First thing I read is about setting up a google group.... yikes.
  625. @snzeh:matrix.org joined the room
  626. @snzeh:matrix.org
    Hey
  627. @snzeh:matrix.org
    Is it possible to change you user id?
  628. @asdflance3030:matrix.org joined the room
  629. @asdflance3030:matrix.org left the room
  630. pfrazee

    In reply to this message

    like your handle? eg @bob?
  631. @snzeh:matrix.org
    Yes please
  632. pfrazee
    yeah, that's doable
  633. everybody has a DID and (at least one) handle
  634. the DID is like a UUID that you use internally. It's persistent
  635. the handle is a domain name, and it can change
  636. @snzeh:matrix.org
    the @snzeh bit before the :
    matrix.org
  637. pfrazee
    ...you mean, in matrix?
  638. if you meant matrix, dunno, I imagine
  639. @snzeh:matrix.org
    The "snzeh" part.
  640. I just get the option to upgrade to my domain
  641. pfrazee
    yeah no idea. It's definitely possible but I havent poked around
  642. @snzeh:matrix.org
  643. Alejandro_ joined the room
  644. Alejandro_
    Hello
  645. alejandro_ changed their display name to Alejandro_
  646. Alejandro_ set a profile picture
  647. snarfed
    hi all! I'm trying to follow the federation part of ATP. https://atproto.com/guides/overview#federation implies that it happens via sync, eg com.atproto.sync.updateRepo. is that right?
  648. so for example, when a bsky user creates a post, would their client send something like com.atproto.repo.createRecord with an
    app.bsky.feed.post
    to their PDS, and then their PDS would make com.atproto.sync.upateRepo calls to each of their followers' PDSes?
  649. @dennethan:matrix.org joined the room
  650. Alberto Robles joined the room
  651. Pavel Kofman 🇺🇦

    In reply to this message

    That's interesting. Can you please give a reference 🙏?
  652. mark roth joined the room
  653. @secu:mtrx.nz set a profile picture
  654. whyrusleeping
    Im sure theres more interesting things to talk about than quantum cryptography
  655. Id love it if the channel could move on from being worried about hash functions breaking
  656. In reply to this message

    Thats roughly the idea, we havent worked out the algorithm precisely but since we have everything in merkle trees the incremental updates are cheap
  657. George Antoniadis

    If a repo contains records from a single user, that means that replies are stored in the repo of the person replying?

    If so, how does replying to a thread/user on a different pds work?

    ie thread’s repo R1 is on pds S1, user with repo R2 from pds S2 replies. Does R2 now needs to be synced to S1 for let’s say app.bsky.getPostThread to work on S1?

    Does that means S2 needs to announce to S1 that there is update to a repo it cares about? And does that happen with updateRepo or is it different method?

  658. whyrusleeping
    That is correct, replies are stored in the replyers repo. The interesting thing about this is that it forces you to rethink what replies mean. Every random person reacting to a post doesn’t necessarily deserve to be included in the same thread, and you dont even necessarily want to notify the poster about that all the time
  659. There are some tricky questions about notifications here, but for the most part it will look like the replier (or the repliers pds) notifying the OPs pds of the post
  660. Then its up the the OPs policy whether or not to do anything about thay
  661. You could say only include replies from people i follow, or only include replies from a trusted set of PDSs, or only from people with a particular badge
  662. The default will be set by your pds, and likely will be “all replies from users of our selected set of PDSs”
  663. @snzeh:matrix.org
    Does anyone know how long the waitlist takes?
  664. whyrusleeping
  665. snarfed
    thanks whyrusleeping! I'm starting to implement parts of ATP based on the docs. thinking of building a bridge to/from indieweb. I know you all are early and probably not fully ready for that yet, I'll be patient and keep expectations low 😀
  666. whyrusleeping
    Oh sweet
  667. Its definitely in the phase where people can tinker with it
  668. Not production ready by any means, and things are evolving pretty regularly, but ive been having fun hacking with it
  669. snarfed
    nice!
  670. @jebba:matrix.org left the room
  671. s3k joined the room
  672. @alxjsn:matrix.org joined the room
  673. totalolage joined the room
  674. George Antoniadis
    Is an official go server or any sort of client sdk being worked on or planned?
  675. 64BitAsura joined the room
  676. SAMBATH KUMAR LOGAKRISHNAN changed their display name to 64BitAsura
  677. David Phoenyx joined the room
  678. Not official, but ive been writing things as i need them
  679. Just finishing up some code gen for lexicon
  680. David Phoenyx set a profile picture
  681. George Antoniadis
    Thanks, this looks like a solid start. I'd like to find some time playing around with how indexers/scrapers would work.
  682. whyrusleeping
    Thats my goal too
  683. Probably gonna wire things up to elasticsearch or something and see how far that gets me
  684. @wclayferguson:matrix.org
    I couldn't really understand the elasticsearch license. https://github.com/elastic/elasticsearch/blob/main/LICENSE.txt
  685. I tend to only go for MIT License, or GPL.
  686. Definitely the Lucene technology itself is awesome. It's what powers search in MongoDB too. Then there's SOLR, which is cool.
  687. George Antoniadis
    I think the TL;DR is "It's Apache (fixed) unless you want to offer elastic search service for money" which AWS was trying to do.
    (edited)
  688. @wclayferguson:matrix.org
    I'd personally never touch AWS with a ten foot pole. Proprietary.
    (edited)
  689. George Antoniadis
    You can probably find more about the server side license thingie from mongo's' faqs who I think started this whole thing. https://www.mongodb.com/licensing/server-side-public-license/faq
  690. @wclayferguson:matrix.org
    I'm using MongoDB and their license basically says you can't compete directly with them by providing any "MongoDB as a service" offerings. Other than that you're free.
  691. Free to use it in a product that makes money.
  692. George Antoniadis
    That's pretty much the same with Elastic afaik
  693. @wclayferguson:matrix.org
    They say it's "Elastic License 2.0" but don't define what that means. Definitely hiding some details in there, imo.
  694. George Antoniadis
    Yeah true, that's annoying
  695. @wclayferguson:matrix.org
    I wonder if the text even exists in github. Should be in the LICENSE file if you ask me. In full text.
  696. If it was good news they wouldn't be hiding it. lol.
  697. And Amazon has shut down customer websites (built on AWS) for political reasons. They are to be avoided like the plague.
    (edited)
  698. Felipe Garcia joined the room
  699. Felipe Garcia
    Hey guys! Got interested in bluesky work and saw the atproto project. Are you guys encouraging open-source contributions atm? If so, is there any guide for running a local environment + any part of the project that are needing extra hands?
  700. Willow Rubenstein joined the room
  701. Willow Rubenstein
    curious if anyone has details on what we can actually expect from bluesky? It obv isn't going to be 1:1 to twitter, but looking at what is public, it seems relatively similar. If any of you have access, can you elaborate on whether or not there is direct messaging functionality (iirc I saw this when I looked through, not sure)
  702. @elsif:matrix.org
    are you planning to use mongodb? 🙁
  703. George Antoniadis
    @elsif:matrix.org: the above discussion was an off-topic rant about the licensing of elastic search and mongodo, nothing to do with what the Bluesky devs are working on. Sorry for any confusion it might have caused.
    (edited)
  704. @elsif:matrix.org
    okay, cool 🙂
  705. @10allday:matrix.org
    Why isn't usenet used instead of creating a new protocol?
  706. MightySpaceman (OLD -> m_spaceman:matrix.org)
  707. Bunnings moment
  708. Im in Bunnings
  709. 😎
  710. Aaron Goldman

    In reply to this message

    Usenet doesn't sign messages for authentication. The Identity concept didn't migrate from server to server. There is no negative caching for an indexer to tell you that a publisher hasn't published a post with that name. It tends to assume you will download all the post titles and then discard rather than a narrow query for a subset of the collections.
  711. That said it is certainly an inspiration and I still recommend anyone developing a distribution protocol to read about it's history as they think through their design.
  712. whyrusleeping
    Lol, i feel like the usenet comment must be a troll
  713. Aaron Goldman

    In reply to this message

    Perhaps I have spent slightly too much time researching all the previous content distribution protocols.

    Usenet is much better than the BBS(Bulletin Bord System)s that preceded the internet.

  714. Fernando Rey joined the room
  715. Fernando Rey
    I dont understand what does they mean by "Portable accounts"???
  716. kickboxer changed their display name to Pavel Kofman 🇺🇦
  717. Pavel Kofman 🇺🇦 set a profile picture
  718. whyrusleeping
    You can take your data out at any time, switch to a different service, and everything still works
  719. Your followers can still see your posts
  720. You can still engage with the world without starting over
  721. @kf.walkow:converser.eu joined the room
  722. Steven Franssen

    In reply to this message

    thats like saying the internet is much better than my lan
  723. In reply to this message

    you can change servers with the same id, in a system like nostr you can use a combination of servers at anytime, their pubkey based ids have no real home
  724. MightySpaceman (OLD -> m_spaceman:matrix.org)
  725. MightySpaceman (OLD -> m_spaceman:matrix.org) changed their profile picture
  726. MightySpaceman (OLD -> m_spaceman:matrix.org)
  727. Steven Franssen
  728. MightySpaceman (OLD -> m_spaceman:matrix.org)
  729. me right now
  730. @lisko:matrix.org
    will change this to xmpp, fedora, and pleroma or something
  731. @penaiple:midov.pl
    ubuntu is nonfree software
  732. @lisko:matrix.org
    Close enough
  733. kalkant joined the room
  734. @massimo:kanik0.wtf joined the room
  735. wstier072 joined the room
  736. unholy0 joined the room
  737. unholy0

    In reply to this message

    lol
  738. @lisko:matrix.org changed their profile picture
  739. Miloš Havlíček joined the room
  740. irazorx0 joined the room
  741. Dean
    Do I understand correctly that any front-end application can authenticate users by reading the root domain in their handle? So bluesky (if they allow it) could log in a user by authenticating alice.some-other-atp-provider.com?
    (edited)
  742. Sorry not the root domain, but the full handle contains a response that has the ATP DID data, correct?
    (edited)
  743. Dean
    So with the current dev-env prototype server you can enter any URL to create a handle with (using app.blsky.createAccount()), but I suppose a public production server would only be allowing handles to be created using subdomains from its root domain?
    (edited)
  744. Aaron Goldman

    In reply to this message

    LAN may not be the best analogy as the BBS(Bulletin Bord System) used phone calls you could sync up with any BBS you wanted just a long distance charge may apply.
    Certainly no push.
    The gossip strategy is interesting.

    Because the repo's are self authenticating it makes sense when you establish a connection with a peer PDS to let it send you repos it has but is not the PDS for.

    If there ends up being a clique in the PDS graph it would take O(log(n)) time and O(log(n)) messages to gossip the repo around rather than O(n) time and O(n²) messages.

  745. Aaron Goldman
    Granted my expectations based on no evidence is that the indexers will cache most of the popular repos and the narrow checkouts of the responses to popular repos so a PDS could get most of its updates from an indexer and only for unpopular content need to contact PDSs directly. If this is true most of the scaling pain is pushed to the indexes and the PDSs will only need to serve a small number of syncing requests
  746. schultzter joined the room
  747. @bradstewart:matrix.org joined the room
  748. txusblack.eth joined the room
  749. txusblack.eth set a profile picture
  750. txusblack.eth
    Hi everyone. I'm Diego, also interested in Bluesky. Thank you for creating this community. I'm a devops leader in a startup and developer, I would like to contribute to the project if possible
  751. Diego Moreno changed their display name to txusblack.eth
  752. Ozan Duman joined the room
  753. mikestaub
    arcalinea: It may soon be time to create a
    CONTRIBUTING.md
    file in the main repo. There are many devs now that want to volunteer, leveraging them in a smart way would accelerate the roadmap. The core team should likely focus on "vertical" slices through the stack, "horizontal" should be left to the community. What do you think?
  754. Wes Souza joined the room
  755. wessouza changed their display name to Wes Souza
  756. Wes Souza set a profile picture
  757. Laurent Bouchard joined the room
  758. EMMANUEL ADUSU joined the room
  759. whyrusleeping
    Speaking from my experience with ipfs and Filecoin, this is much much easier said than done. We absolutely want the communities involvement, but doing that in a way that serves as an accelerant takes thought. Most random PRs ive gotten in the past decade have cost me more time than they have saved me. The real value add of open source tends to be bug reports and “third party” libraries— people hacking integrations and publishing them to their own repos.
    That said, we have gotten a lot of very good contributors who have turned full time over the years, but its definitely not the bulk of it
  760. I really want to enable the open source community here, but also want to temper expectations, and the vertical vs horizontal framing i think is a pretty good starting point
Next group of messages