If you have a user that trys to follow a handle.

1. user puts in a handle e.g. aarondgoldman.bsky.social
2. PDS looks up the handle to get the did. https://aarondgoldman.bsky.social/.well-known/atproto-did did:plc:toxy3kpelhv5gwubytayrsbw
3. read the plc directory to find the PDS for that DID. https://plc.directory/did:plc:toxy3kpelhv5gwubytayrsbw
4. get the repo https://bsky.social/xrpc/com.atproto.sync.getRepo?did=did:plc:toxy3kpelhv5gwubytayrsbw
5. check for update periodically https://bsky.social/xrpc/com.atproto.sync.getHead?did=did:plc:toxy3kpelhv5gwubytayrsbw
   Note: don't put the handle in the folow list only the DID otherwise you will break handle update. Users should be able to update their handle without losing thier followers.
   https://github.com/bluesky-social/atproto/blob/85e3cdaf7b06e40627b198ca1288c1a1028c65ae/lexicons/com/atproto/sync/getRepo.json#L17

With cooperation of the PDS, it's theoretically possible for a client to sign their own posts, with the (slightly weak) assumption that you trust the signature of the previous commit to your repo.

It could be done like so:

You submit a new post to the PDS, the PDS updates the MST, and returns back to you:

1. The record you just submitted
2. the chain of MST blocks leading back to the root of the new commit
3. For every block referenced by the blocks in 2), chain of MST blocks leading back to the root of the previous commit

Your client can then be sure that it has visibility of any new data being added to the MST (and for the referenced blocks it can't see directly, it is sure that they were in the previous commit and therefore trusted), and assuming it all checks out, then it signs the new commit and returns that signature to the PDS.

Honestly at this point the value of authenticated data structures is very vague and hand wavy.

Either:

• The PDS is cooperative and then it will send ranges without censoring any values.
• The PDS is censoring some values, and there is nothing you can do about it, other than just detect it.

Seems like signing individual blobs is more versatile, especially if only servers are going to verify signatures.

The only issue left is caching, and I believe detecting that nothing changed so far is not that expensive, using Xor fingerprints like here https://github.com/hoytech/negentropy

constantly committing to the past

I understand this sentiment, but:
1- This is definitely a blockchain mindset (adversarial thinking, authenticity, and audibility), where it really isn't needed.
2- You can build append-only logs or DAGs with signed blobs if you want, just don't make it a requirement without a clear reason that justifies the cost.

A web of repos is a light cone of commitment.

Who asked for that though? What is the purpose of this? because we all know the massive cost.

Finally, the web is loose liquid, and chaotic, trying to force structure on it will fail, just like trying to force semantics on it failed, entropy wins, because the alternative is too expensive.

I don't actually mind that PDSs are doing repos under the hood, as long as you stick to the server doing the signing, and only one canonical server at a time, because the cost, in this case, is low, and versioning/verifying becomes cheap.

Except it:
1- Makes PDSs more complex and harder to independently implement by others (centralization like in Matrix).
2- Makes running a PDS more involved, as you need to worry about rebasing as we see happening right now on Bluesky.

Hey Aaron, thanks for the reply. Mostly that I want to disconnect my account handle + did completely from a specific server. I want to manage my own keys. I have already setup .well-known/atproto-did for my domain, but I can not login to BlueSky with it. Changing the handle also fails because it expects the did to be set to did:plc. I don't want to host my own PDS as well, unless I really have to(I can if the answer is "you have to").

My assumption was that the UI would not work with a did:web, since BlueSky has no chance to access my keys but I could at least send some posts via CLI/HTTP requests by signing on the client-side myself.

Checking the PDS code a little bit, it seems like PDS is doing the signing in the backend (BlueSky). So unless I add their signing key to my did:web, they won't be able to add posts to the PDS?

In practice today all mutations to the MST are signed on the PDS. Using the verificationMethod id #atproto key. Updates to did:plc are signed by rotationKeys. If you were using the bsky.social PDS then you can upload your own recovery key as the highest priority rotation key. You could then rotate the PDS' keys and leave only your key. This means there is a path from the PDS controlling your DID to controlling it yourself.

In design but not anywhere close to implemented yet. The same logic applies to verificationMethod id #atproto You could store all the atproto verificationMethod keys on your clients and sign all the repos there. Only uploading the signed roots to the PDS that have been signed on the client. The PDS would then enforce the prev field points to the head that the PDS has. If it does not the PDS will reject your update and tell you to rebase your client and try again with the prev that does point to the current head. No forking.

Short term I would say you need to run your own PDS to control your keys but the protocols core data structure MST and did:plc were both designed to empower users to gradually take more and more responsibility over their own keys. You should be able to start with a DID and repo that are controlled by the PDS.

When you are ready, add your rotation key so you can recover.
When you are ready, remove the PDS rotation key so the PDS can't steal your DID.
When you are ready, move signing from the PDS to your user agents so the PDS can't publish as you.

It is important to have low friction adoption by having high trust in the PDS but that this is redeemable as user come to value their repos more, they can take more responsibility.

Warning: If you remove your PDS' keys from your rotationKeys then the PDS has no ability to help you recover your account.

The did:web vs did:plc there are non-trivial tradeoffs. On the did:web site we are trusting DNS and the CA system to certify that the DID Document returned is authentic. Just like any other https connection. This means that is DNS and a CA move your domain to someone else's control they move your repo to their control. Granted we rely on this system for a lot of very valuable https connections and it does mostly seem to work. On the did:plc side we are relying on the hash from the DID string to auth the initial state of the DID Document and the rotation keys in each version to auth the next delta. If you keep your rotation keys secure no one but you can update your DID Document. If you are careful not to fork your DID Document then this is good. If two rotation key holders each try to update the DID Document we have a data race so one fork must win and become the curent DID Document and the other fork must be pruned. This is where the directory comes in. The directory is the timestamp server and the first to file a delta with the directory wins. The second filer gets rejected as having an invalid prev field. Any PDS that is presented with conflicting document logs must go to the directory to find out who is the curent and who is pruned.

The did:plc is the more secure architecture as the directory is given far less trust than the DNS, and CA system is given for did:web but the directory has also done far less to earn our trust then the DNS, and CA system that have years of technical and legal work put into earning our trust.

Aaron Goldman Firstly, huge thanks for taking your time to give detailed answers. I really appreciate the time and energy you put into these messages.

I definitely understand how did:plc can be safer than did:web given that I may forgot to renew my domain and someone else can buy the same domain. Or that someone may hack my hosting to change the .well-known files. However, I still think that did:web is a very strong contender in the DID methods space. As you have said, both domain registration and CA certificate issuance is what the internet is running on for the last few decades. Even though they may have their shortcomings, I think it is a very valid way of managing a digital identity, especially for things like sending BSKY posts.

For these reasons, I want to use did:web, not did:plc. I am not choosing did:web because ATProto supports it alongside with did:plc. I am choosing to use ATProto, because it supports did:web. Hope that makes my standpoint more clear 😃 More context: I have been working with SSI and DIDs for the last few years. So the did:web setup I have is not just for using ATProto.

The curent implementation only does server-side signing. 😭

I think the PDS if username.example.com/.well-known/atproto-did resolves to your did web DID String,
your DID Document has alsoKnownAs your handle, verificationMethod atproto, and service atproto_pds
then you should be able to use your did:web.

123456789101112131415161718"alsoKnownAs": [
    "at://username.example.com"
  ],
  "verificationMethod": [
    {
      "id": "#atproto",
      "type": "EcdsaSecp256k1VerificationKey2019",
      "controller": "$Your_did_web",
      "publicKeyMultibase": "$Your_Key"
    }
  ],
  "service": [
    {
      "id": "#atproto_pds",
      "type": "AtprotoPersonalDataServer",
      "serviceEndpoint": "https://bsky.social"
    }
  ]

e.g.
https://aarondgoldman.bsky.social/.well-known/atproto-did
https://plc.directory/did:plc:toxy3kpelhv5gwubytayrsbw/

https://github.com/bluesky-social/pds

You can federate with most other servers, but not the main bluesky one

Keep in mind it's currently just a sandbox to test federation, and the PLC will be wiped from time to time

In reply to

DDust
Bluesky staff I guess

In reply to

I can't speak for Bluesky, but there are many, many valid reasons why a startup might be better served by sticking to a strategy of gradual and deliberate growth. Technical reasons, business reasons, strategic reasons, financial reasons, legal reasons, etc.

quick question about https://atproto.com/guides/identity#did-methods

A variety of existing methods have been published so we must establish criteria for inclusion in this proposal

Strong consistency [...] High availability

how exactly is consistency and availability in a decentralized system at the same time going to be implemented

I'm trying to find what all the configuration options are for a PDS. The

I did manage to find

Is this the right place to look? I can't seem to find any code that translates the values from the guide into the values in this config class.

Ahh I found it. The pds guide mentions a different branch.

https://github.com/bluesky-social/atproto/blob/simplify-pds/packages/pds/src/config/env.ts

It looks like the config has changed pretty dramatically. I wonder why this stuff hasn't been merged

I'm not sure if this is a quick question but I will give it a shot.

"Strong consistency" refers to the consistency vs availability trade off. The protocol chose consistency for DID PLC.

"High availability" refers to the uptime. Typically measured in 9s of availability ratio. uptime/total time.

For PLC

For did:web it's your domain name. Your DNS. Your web server. It's whatever consistency and availability you set up with your domain certificate.

did:plc is the do it AtProto way
option.
did:web is the have it your way option.

Hi everyone, I just managed to set up a PDS on my server, and I seem to have successfully made an account through the Android app, but when I try to load my profile, I get "Failed to load profile", and I see this in the logs:

12345678910111213141516171819202122232425Jul 03 11:53:31 dead10ck.dev bsky-pds[63781]: {"level":30,"time":1688
385211667,"pid":63781,"hostname":"dead10ck.dev","name":"pds","req":{"
id":924,"method":"GET","url":"/xrpc/app.bsky.feed.getAuthorFeed?actor
=did%3Aplc%3Aiss4k5djxrakxv6bvmkv2ic5&limit=30","query":{"actor":"did
:plc:iss4k5djxrakxv6bvmkv2ic5","limit":"30"},"params":{},"headers":{"
host":"bsky.dead10ck.dev","connection":"close","authorization":"Beare
r did:plc:iss4k5djxrakxv6bvmkv2ic5","accept-encoding":"gzip","user-ag
ent":"okhttp/4.9.2","if-none-match":"W/\"b-SSk8i5UPzi6JB6Bv3a+V47dp7V
Q\""}},"res":{"statusCode":304,"headers":{"x-powered-by":"Express","a
ccess-control-allow-origin":"*","etag":"W/\"b-SSk8i5UPzi6JB6Bv3a+V47d
p7VQ\""}},"responseTime":40,"msg":"request completed"}
Jul 03 11:53:34 dead10ck.dev bsky-pds[63781]: {"level":50,"time":1688
385214043,"pid":63781,"hostname":"dead10ck.dev","name":"xrpc-server",
"err":{"type":"InvalidRequestError","message":"Error: Params must hav
e the property \"actor\"","stack":"Error: Error: Params must have the
 property \"actor\"\n    at <anonymous> (/opt/bsky/pds/node_modules/@
atproto/xrpc-server/src/server.ts:195:17)\n    at newFn2 (/opt/bsky/p
ds/node_modules/node_modules/express-async-errors/index.js:16:20)\n
  at Layer2.handle2 (/opt/bsky/pds/node_modules/node_modules/express/
lib/router/layer.js:95:5)\n    at next (/opt/bsky/pds/node_modules/no
de_modules/express/lib/router/route.js:144:13)\n    at <anonymous> (/
opt/bsky/pds/node_modules/@atproto/xrpc-server/src/server.ts:377:7)\n
    at process.processTicksAndRejections (node:internal/process/task_
queues:95:5)","errorMessage":"Error: Params must have the property \"
actor\""},"msg":"error in xrpc method app.bsky.graph.getLists"}

So, my logic for consistency in the DID has a few steps and may well be reversed now that I am no longer on the team but here it is.

1. Adoption is key to success of AtProto
2. Key management is a speedbump. Use just want to sign up and not think about keys.
3. By letting the server control the keys we get minimal friction and account recovery works.
4. Some users will come to value their Identities and want to take controll of their repos after their initial creation.
5. This requires the ability to permanently transfer an identity from one party to a different party
6. If there is not an update mechanism with finality the PDS that created an Identity could withhold a update and publish it late to take back an Identity controlled by a user

At first you just have a website that you use like https://bsky.social/ then at some point you decided you want to control your ID so you can move to any PDS and https://bsky.social/ can't steal your ID. (note: we have already seen the first bug where a pds was tricked in to giving away someones DID) When you want even more control you mode to client side repo signing where the PDS host your repo but can't publish without the cooperation of one of your user agents.

There are a number of levels of trust in your PDS
rotationKeys: PDS only | PDS and User Agent | User Agent only
atproto verificationMethods: PDS only | PDS and User Agent | User Agent only

I stood up a PDS yesterday, generated an invite code, and created an account, but it seems to be in some half-working state. I can log in via the web sandbox app, but I can't do anything meaningful because everything results in a Profile not found XRPCError.

Rough sequence of actions from the logs:

12342023-07-07 00:55:33.479 /xrpc/com.atproto.server.createInviteCode
2023-07-07 00:55:54.681 GET  /xrpc/com.atproto.server.describeServer from https://app.bsky-sandbox.dev/
2023-07-07 00:57:13.516 POST /xrpc/com.atproto.server.createAccount from https://app.bsky-sandbox.dev
2023-07-07 00:57:13.952 "Profile not found" in app.bsky.actor.getProfile

Given that it can take some time for the on-the-fly certificates to become available, I waited a while and created a couple posts. However, even now after letting it simmer overnight, anything like attempting to post or follow results in Actor not found errors in the logs.

Would anyone have any suggestions for how to go about troubleshooting this?

I used the bluesky-social/pds repository. Instance seems healthy (according to the healthcheck), and caddy and pds aren't throwing any errors other than the "Profile not found" and "Actor not found" messages when I try to do anything.

12$ curl https://pds.holmosapien.com/xrpc/_health
{"version":"0.2.0-beta.5"}

That should be supported, since I see other people in the network with their servers hosted on subdomains. Now, you did mention nginx. Configuring the reverse proxy isn't part of the official documentation -- it assumes the PDS will be handling the traffic directly -- so I built a config similar to what I use for the other services that I host on this server; it handles pds.holmosapien.com (which is my $HOSTNAME) and *.pds.holmosapien.com and proxies to the Docker container using HTTP 1/1.

Since we're both having this problem and we're both using nginx, as well as the guy in

Yeah, these are the important parts for websockets:

1234        proxy_http_version 1.1;

        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

I'm not using a wildcard cert fwiw, but my main config looks like this

1234567891011	location / {
		proxy_pass http://localhost:31337/;
		proxy_http_version 1.1;
        proxy_set_header Connection "upgrade";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 1d;
        access_log off;
	}

My nginx is similar to retr0id's, but here's how I have it configured to handle both port 80 (which is necessary for the auto-SSL generation) and 443:

1234567891011121314151617181920212223242526272829303132333435363738server {
    listen 80;
    listen [::]:80;

    server_name ~^(.+\.)?pds.holmosapien.com$;

    location / {
        proxy_pass http://localhost:16780;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server {
    listen [::]:443 ssl;
    listen 443 ssl;

    server_name ~^(.+\.)?pds.holmosapien.com$;

    location / {
        proxy_pass https://localhost:16743;

        proxy_http_version 1.1;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_ssl_server_name on;
        proxy_ssl_name $host;
    }

    <certbot stuff goes here>
}

First no need to apologize "newbies" are welcome and more importantly anyone trying to learn or share information about the protocol. A good question can move a community just as much as a good code contribution.

As for whether federation implies free API use it is a little complicated so I will split the question.
A repo is a Hash Tree with a signed root. This implies that if you have the data you can validate the signature and trust the repo as if you got it directly from the user or their PDS. So you could get the repo from anyone who has it driving down the cost of retrieving the repo to the lowest bidder.

On the flip side a PDS has no obligation to talk to you. They could put a charge for serving the content.

So you might end up in a situation where a PDS charged $10 per GiB for API but there is a BGS that is changing $10 per TiB. Your PDS would try to get the data from the cheap BGS before bothering the PDS. This kind of structure could allow PDSs that don't have much bandwidth to push traffic to catches that can handle it better.

On the flip side you would not expect popular content to be expensive because resellers would buy the expensive content and sell it for much less. Only rare content that only the original PDS cares to store would likely be expensive.

My expectation is that API costs for most major PDSs and BGSs would be about the AWS egress costs and small PDSs will be free but when overloaded will redirect to a BGS that indexed them.

Does that make sense?

It makes sense in principle, and is helpful, but it doesn't fully resolve my question. The reason I am still unsure is that I expect that the vast majority of atproto users won't have their own personal data servers, and will instead be relying on a PDS supplied by the Bluesky corporation. And while that PDS will be federated with independent PDSs, to get the equivalent to the developer experience of accessing the Enterprise-level Twitter API, one will have to get the data from that particular PDS.
You say: “On the flip side a PDS has no obligation to talk to you. They could put a charge for serving the content.”

So, it seems like the Bluesky corporation could choose to charge $42K per month for access to user “tweets” beyond a certain number per month.

 Am I wrong?