-
@ laoc42
2023-07-15 22:12:01I want to propose a way on how to standardize metadata sent along with the tags attribute. For this I suggest the use of Linked Data technology to make it easy for developers and applications to use data from different sources.
Nostr is great. It is very easy to work with as a developer, has tons of use cases, is permissionless and censorship resistant. You can build lots of applications with it without having to be afraid to lock your users in (or to get locked in if you are the user). I mean in theory, because with applications getting more complex and more data is sent along, different applications will have to handle that same data sent by other clients. If there is no easy way to know what attributes actually mean or it is hidden in some remote documentation we will have implicit lock-in effects, because it will be hard to know what another developer really means with that attribute. But what do I even mean? Let's look at a typical Nostr Event I used when working on the first round of the Hack-Nostr-On hackathon:json ["EVENT", "RAND", {"id":"5b456114558503ebe99c588c915251006d35e8ad23dc3b0d4407f828eb596a82", "kind":1, "pubkey":"8af30833be407219d6a2d2a6a84d2cea7d3c212693090fcc2b110ca944c3a617", "created_at":1687806335, "content":"Hello! A new resource with title \"Bitcoin werden nicht geschürft, sondern im Laufe der Zeit verteilt - Einundzwanzig Portal\" got added to the library!\n https://portal.einundzwanzig.space/storage/1164/conversions/3100ea413cc63910d5238b31f817d244-seo.jpg\nVon allen Missverständnissen im Zusammenhang mit Bitcoin ist das Missverständnis, dass Miner Bitcoin erzeugen, wahrscheinlich das dümmste.\n\nNein, Miner erzeugen keine Bitcoin. Miner versuchen, gültige Blöcke zu erzeugen und werden vom Bitcoin-Netzwerk mit neu ausgegebenen Sats belohnt, wenn sie erfolgreich sind. Diese Belohnung wird nur während der Initialisierung-Phase des Netzwerks gezahlt..\nSent from nostr:npub1j8wdpuqqct23l4gyyxa5dkjvgkxzkjh3n3mzzjj79uhcvh424pcqurytqu (Bitcoin Library).\n ", "tags":[ [ "p","8af30833be407219d6a2d2a6a84d2cea7d3c212693090fcc2b110ca944c3a617" ], [ "metadata", "{ \"id\":\"https://portal.einundzwanzig.space/library-item/bitcoin-werden-nicht-geschuerft-sondern-im-laufe-der-zeit-verteilt\", \"name\":\"Bitcoin werden nicht geschürft, sondern im Laufe der Zeit verteilt - Einundzwanzig Portal\", \"description\":\"Von allen Missverständnissen im Zusammenhang mit Bitcoin ist das Missverständnis, dass Miner Bitcoin erzeugen, wahrscheinlich das dümmste.\\n\\nNein, Miner erzeugen keine Bitcoin. Miner versuchen, gültige Blöcke zu erzeugen und werden vom Bitcoin-Netzwerk mit neu ausgegebenen Sats belohnt, wenn sie erfolgreich sind. Diese Belohnung wird nur während der Initialisierung-Phase des Netzwerks gezahlt.\",\"image\":\"https://portal.einundzwanzig.space/storage/1164/conversions/3100ea413cc63910d5238b31f817d244-seo.jpg\", \"resourceType\":[{\"id\":0,\"checked\":true,\"title\":\"Article\",\"uri\":\"https://w3id.org/bitcoin-library/bots/f631971e-ee83-4c95-9fcf-b1c1639bb4f6\"}],\"keywords\":[{\"id\":1,\"title\":\"General Bitcoin\",\"uri\":\"https://w3id.org/bitcoin-library/bots/4e42c22d-0b45-4d8a-bb49-6d95d56e2558\",\"checked\":true},{\"id\":18,\"title\":\"Mining\",\"uri\":\"https://w3id.org/bitcoin-library/bots/3b8d08d5-f72b-4a3f-9e02-5b4c5dbdc5f1\",\"checked\":true},{\"id\":6,\"title\":\"Philosophy\",\"uri\":\"https://w3id.org/bitcoin-library/bots/d41061ee-8e61-4c75-8a69-f2a2616920c5\",\"checked\":true}],\"authors\":[{\"name\":\"Gigi\",\"npub\":\"npub1dergggklka99wwrs92yz8wdjs952h2ux2ha2ed598ngwu9w7a6fsh9xzpc\",\"id\":\"Q197kqbe\"}],\"metadataContributor\":{\"name\":\"laoc42\",\"npub\":\"npub1r30l8j4vmppvq8w23umcyvd3vct4zmfpfkn4c7h2h057rmlfcrmq9xt9ma\"}}"]],"sig":"22aa98e58432c0251094df7754f720dd8e48e6b70efd5d7e336fe7e054694ebabf275e66dca50a1c5db8056a3bfd0e2765823b2e7a4e5fc959ce465cac0e7d78"}]To publish the metadata of a resource I added a custom
metadataattribute and JSON-stringified the content. The content itself is JSON Linked Data (https://www.w3.org/TR/json-ld/) which is a format to serialize Linked Data like RDF. But what is it and why is it important?What is Linked Data?
You have probably heard about the Semantic Web (maybe not, but it was the original Web 3.0, not that Web3 crap. Vitalik stole that term and I bet he knew what he did). The idea is to make data on the web not only readable for humans, but for machines as well to facilitate data exchange between different machines. To bring semantics to the data technologies like RDF (Resource Description Framework) are used to annotate resources with metadata. It's basically hypertext, but for data. Think of it as a standardized API for data in a web page. Tim Bernes-Lee formulated these four principles I will explain in a minute:
- Use URIs as names for things
- Use HTTP URIs so that people can look up those names.
- When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)
- Include links to other URIs. so that they can discover more things.
So how does it actually work? Let me explain using something everyone likes. Indian Curry.
Linking Indian Curry
Let's say I have a website with recipes. I provide all the useful information in the HTML, like the name, image, ingredients, instructions, cooking time and so on. Let's also assume I want my website to be crawled, be it Google or some other search engine actually not doing any evil.
How can I provide useful information on my recipe page, so a search engine or some other data aggregator can actually use my info to display meaningful info like above? Of course an aggregator could look a the web page, identify the wanted information in the HTML, then write some custom parser using XPath or similiar and extract that data. And then it all breaks when I decide to redesign my recipe page a little bit. So how to overcome that issue? How about embedding the data as data in the web page itself? So no custom crawler any more. Just loading the recipe page and looking for the data and being happy! There are lots of ways to do that with RDF, but I want to introduce the most simple one for web devs. Say hello to JSON Linked Data: JSON-LD.
JSON-LD
JSON is great, everybody loves it. However, it is also pretty meaningless. Consider the following data:
json { "title": "Indian Curry", "cookingTime": "20min", "ingredients": [ ... ] }I mean you can possibly guess from the name what I mean, but the next recipe website has this:
json { "name": "Indian Curry", "time": "PT1H", "recipeIngredients": [ ... ] }You can guess again what is meant, but harmonizing all that data is again a cumbersome task. So how about adding some context to our data?
json { "@context": { "@vocab": "https://schema.org/" }, "@type": "Recipe", "name": "Indian Curry", "time": "PT1H", "recipeIngredient": [ ... ] }What happened? We added:
- a
@contexttag to give some context to our data - inside we have an
@vocabtag, which defines what "vocabulary" we are using - a
@typetag to say about what kind of thing we are talking.
So what's now better than before? If we want to know what
namemeans, we can now visit https://schema.org/name and actually get a meaningful definition of what I mean, when I talk aboutname. Same goes for all the other attributes. Visiting https://schema.org/Recipe will give me information about what I mean when talking about a recipe and there are plenty of other attributes I can use to describe a recipe. Schema.org is just one site providing such vocabularies, there are others as well (though schema.org is really popular, useful and you can easily add your ones, opening a PR on GitHub) or you could also just set up your own RDF vocabularies. So now we have a common ground when we speak about attributes, making them more easily interchangeable between different machines and platforms. (This is of course just a tow in the water, you can do amazing stuff with this. You can create links between these vocabs, like schema.org/name === mydefinition.org/name, implement things like reasoning and there is also query language for this kind of data -> SPARQL)Adding RDF to Nostr
So to bring this approach to Nostr I propose something as simple as adding a
metadatatag to the tags attribute. This can then be stringified and later read out by clients. By doing so we will simplify the way developers and machines can read out data from different applications.Open Questions
- What event kind should we use?
- Do we need a custom NIP for this approach?
- I asked Melvin Carvalho about this topic as well.Beside the above mentioned approach he suggested "sending an HTTP reference that dereferences to LD". This sounds also interesting. Do you have any opinions on one over the other?
Links
- https://www.w3.org/DesignIssues/LinkedData
- https://www.w3.org/TR/json-ld/