-
@ Juraj
2024-08-31 20:37:00
OK, so we can do rotating bloom filters.
Architecture:
We are syncing relays (which can be on a slow network and sync while we sleep). Let's say we have clever filters, so we don't sync everything (relay knows about follows and replies), but this is not that important, maybe for small networks, we can sync everything.
Algorithm:
- client relay creates a bloom filter with nonce (just a random number that gets applied to all hash functions in the filter construction). It sends filter ("give me notes for the past 24 hours"), nonce and the bloom filter of note ids matching this description it already has.
- server relay (or relays, we'll see if we can somehow broadcast this without overloading the network) collects the events. It fills a preefined number of packets with content that matches the filter and does not match the bloom filter. If there's nothing, it sends back "you've seen all" message (this can be trusted, bloom filter does not have false negatives).
- after receiving or after some time, the process is repeated with a different nonce.
The bloom filter for sure filters out note IDs that the client already has. It might filter out some other note IDs, but this is mitigated at the next request, because the bloom filter will have different false positives, because of different nonce.
Now we need the broadcast protocol for Reticulum and let's get coding. Since all reticulum nodes right now are basically Linux machines (RNode needs control), everyone can run their own relay, listen to wire traffic, update notes from the air and sync from time to time on everything it missed.
We will use unchanged relay software, unchanged Nostr clients (Voyage is perfect for this), only the syncer is required.
nostr:npub1f49twdlzlw667r74jz6t06xxlemd8gp2j7g77l76easpl8jsltvqvlzpez any inputs?