-
@ Arjen
2025-03-18 20:11:54Time isn't constant, or at least my perception of it isn't. The past two weeks have been like 2 months for me! In a good way... I'm playing catch-up with my updates so this one will only focus on week 10 because so much happened!
I'm lucky enough to be joining #SovEng for the second time, where a bunch of Nostr geeks get together and all work on our part to fix the internet, join in deliberation, hike, demo, and repeat...
GitHub actions in GitWorkshop
I hit off the first week by working with nostr:npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr to see what the Git Actions workflow looks like in a decentralized world. Because now anyone can be an operator, the flow will be different than on legacy git collaboration tools. We hashed out an early version of a user interface. You can check it out at vnext.gitworkshop.dev. Go to profile icon > settings > experimental mode (on).
The issues we face
We're having some debates wether or not the workflow runners fit within the DVM spec. The main difference between this and all DVM implementations that i'm aware of. Is that most DVM are quite cookie-cutter input, processing, then answer within a few seconds.
Costs not known up front
The workflow runners are long-running jobs, they can run for hours potentially, AND it's impossible to know how long a job is going to take beforehand, meaning there can be no set price per invocation. So we send a pre-payment for the maximum runtime (
price/sec*timeoutSec) and expect the runner to return the change when the job is done, so we need to account for that.Handling cancellations
Also when a job is cancelled, which in the DVM spec is done by a delete request, which forms a problem because we want to keep a history of our runs, even if they're cancelled.
DVM partial results
We currently use partial results to 'stream' back logs from the job. However the log output of jobs can be massive and we might not want to keep them around for long, Ephemeral would might be better suited for that and the final log output could be saved on blossom to be included in the final status.
Queueing runs
The DVM spec does not specify any queuing messaging/statuses. Workflows can rely heavily on the type of machine they run on. Architecture, OS, memory or vCPU's. That means you might want to schedule jobs even though all potential runners are currently busy. Therefore a queueing system makes a lot of sense. We might be able to squeeze such a system into our implementation of the DVM spec. But it's just one of the things that makes us wonder if we're not drifting too far from the spec to still call it a DVM.
Some screenshots of our implementation: https://vnext.gitworkshop.dev/arjen@swissdash.site/dvm-cicd-runner
|
| |
| ----------------------------------------------------------------------------------------------------- | --- |
| 
| |
| | |
Infernal Insights
Aaah... Logging and metrics, our favourite hard-to-manage infrastructure thing we don't really want to do but kind of need to make a good product.Coming from the DevOps world, these systems usually can be incredibly valuable, but can be a pain to set up and/or maintain.
I ended up on a fun proof-of-concept together with nostr:npub1ye5ptcxfyyxl5vjvdjar2ua3f0hynkjzpx552mu5snj3qmx5pzjscpknpr and nostr:npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr we made this dead simple nostr message:
json { "kind": 1314, "expires": "tags": [ ["expiration", nostrNow() + oneMonth], ["n" "tollgate-site"] // Application Namespace ["p, "1096f6b...a74c7ecc"] // pubkey of person behind current deployment ], "content": "The button on the top-right disappears when i hover on it." }It basically tags the developer of the application/site and a namespace (aka: the name of the software). @hzrd created this simple button to add to a website header that creates the ability to send these events.The
ntag is there to give the name of the software, that way different people (npubs) can gather feedback on seperate deployments, and the developer of said software could also utilize this data for its own purposes by filtering on thentag. Whereas a developer is probably only interested in their own deploymentThe events are ONLY meant as raw data, like human written feedback, maybe crash logs. It can be anything and they're not meant to be kept around for long, it should be ingested by a pipeline, which can then run one or more rounds of filtering, tagging, categorizing. Each resulting in a new refined event, ingestable by the next.
However for this experiment we stuck with ingesting it, and do just one step of refinement and send it off to a logging-aggregation system, in this case Grafana Loki. We basically sent the logs straight over there, but we also added a fun extra feature, sentiment analysis!
We sent the contents of the logs over to ppq.ai and let it give us a sentiment score from 1 to 10. So we can get a sense of how positive/negative the feedback is. This same principle can be applied in so many ways. Think of categorizing by LLM. You can group feedback based on if it's about the UI for example. It'd be easy to group them and create a Nostr git issue out of those.
Find my software for the dat ingestion here.