@ ChipTuner

original post

In a recent job interview, we discussed some of my work on Nostr and specifically my Noscrypt library. I was asked basic questions, such as what it is and what purpose it serves. Most of my response centered on the idea that while the secp256k1 library is excellent, it is easy to misuse.

So what footguns?

It had been a while since I heard that term, and for whatever reason, I struggled to give a satisfactory response. The interviewer was pleased, but I was not, so I want to take some time to reflect on better ways of explaining the challenges I've encountered while working with libsecp256k1 when building on Nostr.

First, libsecp256k1 is functionally perfect as it is, and I don't see any need for changes. The real challenge is implementing all of the Nostr-specific cryptographic tasks on top of secp256k1. This includes complex but necessary operations like note encryption and signing. The most difficult part is converting between all of the different required encodings: Hex, Base64, Bech32, JSON, UTF-8, and so on.

Really I'm just talking about implementation details, what I got wrong, and what was missing between nostr operations and libsecp256k1.

Some backstory

When I first fell into Nostr, I had some cryptography experience from working on my .NET libraries called VNLib. I quickly realized that Nostr identities rely on your ability to keep a 32-byte number safe, but also accessible. With my HTTP framework in hand, everything seemed like a nail. I wanted my key to be securely stored and signed remotely across a network, where the data is sent between the client/server, and my key never has the opportunity to leave the server. So that's what I started building and continue to use, even though it's not complete. That project is called NVault, and I quietly work on it in the background as things break.

Originally, I built a C# wrapper for libsecp256k1, manually worked through all the C/C# barriers, and implemented the note encryption and signing in higher-level C#. However, when NIP-44 came out, everything changed. I realized how much more complex the wrapper would become if I kept trying to stack onto it. Another issue was performance. I still believe my skills in C allow me to write more efficient and performant code than I can in C# for the same tasks, especially when it comes to cryptography and functional programming.

NIP-44 significantly increased the complexity of Nostr cryptography, and after seven months of regular work on NIP-44, I know I'm not the only one still grappling with this complexity, along with NIP-59 and other related DM and crypto specs.

Okay, so a library needed to address the complexities and inefficiencies of Nostr cryptography while also being portable. To me, no other language meets those goals better than C. We can target virtually every operating system, as most high-level languages can bind to a C ABI at load time or runtime. Additionally, we can target hardware/RTOS systems. C can do it all and usually do it better if you are "smart" and careful, in my opinion. Defensive C programming, if you will.

Why noscrypt

If you are building a client or relay in C, or a derivative that relies on libsecp256k1, or a similar library, because it's the most tested and stable library for the secp256k1 elliptic curve, you will need to implement every Nostr-related spec by hand as helper code in your project. Libraries handle this. However, each time note encryption changes (which it has frequently), you will need to revise your cryptographic helpers to accommodate those changes and hope you don't make a fatal mistake that compromises your users' keys or notes. You aren't a cryptographer; you're a client/relay developer. I'm not saying you can't do it; I'm saying you don't want to.

Working with libsecp256k1

First, there is no concept of a key pair associated with Nostr in libsecp256k1. Nostr is "special" in that we treat the secret key and public key as equally distinct parts of your identity on the network and often store them separately.

Public Keys are X-Only

Public keys in Nostr only use the x-coordinate of the xy point on the curve. Libsecp256k1 handles this fine, but not in a foolproof way, in my opinion.

Here is some code taken from Noscrypt to illustrate the steps needed to implement public key conversion:

```c int _convertToPubKey( const NCContext ctx, const NCPublicKey compressedPubKey, secp256k1_pubkey* pubKey ) { int result; uint8_t compressed[sizeof(NCPublicKey) + 1];

/* Set the first byte to 0x02 to indicate a compressed public key */
compressed[0] = BIP340_PUBKEY_HEADER_BYTE;

/* Copy the compressed public key data into a new buffer (offset by 1 to store the header byte) */
MEMMOV((compressed + 1), compressedPubKey, sizeof(NCPublicKey));

result = secp256k1_ec_pubkey_parse(
    ctx->secpCtx, 
    pubKey, 
    compressed, 
    sizeof(compressed)
);

return result;

} ```

The NCPublicKey type is a 32-byte structure (a byte array) so it's far easier to work with and store with higher-level Nostr applications. We have encountered these issues time and again in Aedile due to the way we use public keys in Nostr.

I regularly failed to find a structured way to store Nostr-specific public keys that libsecp256k1 could work with. I ended up making all the above function calls (and about two more) into the library to do this conversion. So, it's just implementation details.

Keypairs

Again, libsecp256k1 requires some complexity in keypair structures which Nostr has almost no 1:1 mapping to, and really doesn't need. A secret key is all that is required to sign (encrypt) a message using ECDSA, so there is no reason to store keypairs in this case.

Again here is an example from noscrypt

```c NCResult NCSignDigest( const NCContext ctx, const NCSecretKey sk, const uint8_t random32[32], const uint8_t digest32[32], uint8_t sig64[64] ) { int result; secp256k1_keypair keyPair; secp256k1_xonly_pubkey xonly;

/* Fill keypair structure from the callers secret key */
if (secp256k1_keypair_create(ctx->secpCtx, &keyPair, sk->key) != 1)
{
    return E_INVALID_ARG;
}

/* Sign the digest */
result = secp256k1_schnorrsig_sign32(ctx->secpCtx, sig64, digest32, &keyPair, random32);

/* x-only public key from keypair so the signature can be verified */
result = secp256k1_keypair_xonly_pub(ctx->secpCtx, &xonly, NULL, &keyPair);

/* Verify the signature is valid */
result = secp256k1_schnorrsig_verify(ctx->secpCtx, sig64, digest32, 32, &xonly);

return result == 1 ? NC_SUCCESS : E_INVALID_ARG;

} ```

Some validation code was removed for brevity, assertions are used for describing expected results

You will notice that a key pair structure is required (and must be assigned) from our secret key. This is where I think the strict typing of the NCPrivateKey structure ensures safer signing and a more straightforward API. Again, Nostr implementation details, but this function helps mitigate complexities in an obvious and difficult-to-misuse way.

In the case of polyglot linking, I think it also makes it reasonably simple to sign a message digest with a single function call on some byte arrays. Direct, which I think is important.

EC Diffie Hellman

ECDH or Elliptic Curve Diffie Hellman gives us the ability to take a set of points on an elliptic curve and create a shared key that only the key holders can generate. More specifically, each party only needs to know each other's public information in order to generate said shared point. We exploit that feature in multiple ways by using symmetric encryption in both NIP-04 and NIP-44 encryption specifications.

Ignoring all of the encryption specifications, we need a function to safely generate our shared point from our Nostr-specific secret and public keys. So some strict typing and our _convertToPubKey() function from above re-appear. You will also notice a callback function called _edhHashFuncInternal(). This function does the work of "computing the hash" which in Nostr we just need to return the shared x-coordinate as per NIP-04

```c NCResult _computeSharedSecret( const NCContext ctx, const NCSecretKey sk, const NCPublicKey otherPk, struct shared_secret sharedPoint ) { int result; secp256k1_pubkey pubKey;

/* Recover pubkey from compressed public key data */
if (_convertToPubKey(ctx, otherPk, &pubKey) != 1)
{
    return E_INVALID_ARG;
}

/*
* Compute the shared point using the ecdh function.
*
* The above callback is invoked to "compute" the hash (it
* copies the x coord) and it does not use the data pointer
* so it is set to NULL.
*/
result = secp256k1_ecdh(
    ctx->secpCtx,
    (uint8_t*)sharedPoint,
    &pubKey,
    sk->key,
    &_edhHashFuncInternal,
    NULL
);

ZERO_FILL(&pubKey, sizeof(pubKey));

/* Result should be 1 on success */
return result == 1 ? NC_SUCCESS : E_OPERATION_FAILED;

} ```

A lot of things to remember right? Passing buffers and keys around to callback functions. Yeah, in C# I sure had fun doing this in nvault. That was sarcasm if you couldn't tell.

Read The Docs?

The libsecp256k1 project has almost no public documentation. Like many C projects, documentation is just the header files and readme cover. Which is fine if you are me and take the time to really learn how to use their public API. One of the larger projects I build is my website (and CMNext blog tool) to publish readable and up-to-date dedicated documentation. It is assumed you should take header files as authoritative, but dedicated documentation and examples can make or break a project's adoption IMO.

Wrapping up footguns

Honestly, my goal was to provide insight into my experience writing over 4k lines of C code to avoid some of the complexities I initially encountered when scaling NVault to compete with the latest NIPs. I believe Nostr will only become more complex, especially as we delve deeper into post-quantum and forward-secret algorithms.

Backward compatibility

A lesson learned outside of libsecp256k1 is the need for backward and forward API compatibility. When I first started building NVault, only NIP-04 existed, and I assumed that any future spec would deprecate the old spec. As it turns out, that is not the case, and I needed to rethink compatibility for future upgrades.

Many parts of Nostr are either slow to upgrade (due to complexity) or other contributors don't see the need to upgrade. Nostr is inherently anarchistic, and no one developer has the final say on what specs are used in the wild, which is an appeal to me. Every system has tradeoffs; in this case, we will have app compatibility issues. A perfect example is NIP-17 DMs collected by cloud fodder, where some clients support spec interoperability and others do not. I will disregard my personal opinions on their reasoning. I, and GitCitadel, will have our own reasons for our support or lack thereof, and I aim to be as transparent as possible when the time comes.

Making the case for noscrypt

By this point, you might be thinking: "Okay, yeah, you just made a case for another library, great." That would be correct, but I showed some simple code snippets that I believe highlight the requirement for high-level cryptography libraries incorporating the Nostr spec. While Nostr was always meant to be simple enough for a caveman to implement, it relies on the legacy existence of high-level libraries and users' ability to implement them correctly. This ecosystem predominantly exists in the JavaScript/browser world.

When trying to implement compact, portable, and powerful applications, you need the horsepower and efficiency of a dedicated library built for this purpose. We can already see this with existing Nostr SDKs (NDKs) in multiple languages. I don't aim to lock you into an NDK; for very high-level spec functionality, I think simple libraries are suitable for most smart and passionate developers. You don't need to rely on an NDK developer for a specific language.

Tag me on nostr

Use noscrypt for your C project

Use Aedile in your nostr C++ project

Check out all of my open source software