Cloudflare Worker Compatibility/Integration

[nickchomey]

Proposed change

It currently isnt possible to use nats.js in Cloudflare workers due to some missing nodejs modules. Specifically fs and dns (at least from what I found - there could be more)

It appears that fs is only used if you specify TLS certs in the connection. But it doesn't appear to be possible to bypass dns.

In a conversation in Slack, a team member said that an option could be added to bypass dns resolution.

Additionally, he said that changes to/related to nuid.ts might be needed.

Use case

It would be great to be able to use NATS with Cloudflare Workers, so that connections could be made to NATS servers from the largest edge network.

One use case (among surely infinite) would be to use your NATS Jetstream KV instead of Cloudflare Workers KV, which has up to a 60 second propagation delay for KV writes while NATS KV is "immediately consistent".

I could also see use cases such as wanting to create a NATS connection from CF Workers, rather than directly from a browser client.

In fact, it would be SUPER cool if we could connect from, say, a browser to a CF Worker, which then opens and maintains a connection to a NATS server and subscribes to subjects/streams or watches KV changes. Then any changes could be returned to the browser via, say, an SSE keepalive connection. This would avoid the browser having to do anything with NATS, and could also make use of the CF worker to do any heavy processing prior and just returning, say, HTML to the browser.

It is perhaps possible that nats.ws could already be used with CF workers, but surely it makes most sense to use the native protocol.

More generally, Cloudflare's mission (I've seen it stated somewhere) seems to be to be a sort of operating system for the global internet. An integration with NATS, which makes it so simple to integrate different backend services, seems like a natural fit. Cloudflare could be the front-end/gateway to connect to a NATS-based backend infrastructure.

Contribution

I doubt I could provide much code, but would be happy to help test it if needed.

[nickchomey]

It might be relevant that CF Workers has a tcp sockets connect() api. They use it for creating database connectors and much more. Here's a relevant blog article as well https://blog.cloudflare.com/workers-tcp-socket-api-connect-databases/ which seems to address all of these issues (dns, tls etc..).

Perhaps that API could be used to make this a simpler process for you? It might not need any "development" beyond just providing a snippet (or small library, e.g. nats-cf.js) for connecting to NATS via CF Workers connect().

They also seem to be open to collaborating with seemingly competing products, such as Upstash, Turso and more, so would surely help you as well.

Finally, they're also proposing this api as a general standard https://github.com/wintercg/proposal-sockets-api

[nickchomey]

Their even newer Hyperdrive service seems like it might be great fit for all of this as well. It does two main things:

1. Maintains a set of regional database connection pools across Cloudflare’s network, so a Cloudflare Worker avoids making a fresh connection to a database on every request. Instead, the Worker can establish a connection to Hyperdrive (fast!), with Hyperdrive maintaining a pool of ready-to-go connections back to the database. This connection pooling will be forever-free.
2. Second, it understands the difference between read (non-mutating) and write (mutating) queries and transactions, and can automatically cache your most popular read queries: which represent over 80% of most queries made to databases in typical web applications. This will be charged when it is out of open beta.

I'm sure that this wouldn't be useful for every NATS implementation, but surely could be useful for things like KV operations?

Blog announcement: https://blog.cloudflare.com/hyperdrive-making-regional-databases-feel-distributed/
Docs: https://developers.cloudflare.com/hyperdrive/

[nickchomey]

I fiddled around for a while and finally got this to work after learning about telnet. Importing connect from nats throws all sorts of errors, but using connect from cloudflare:sockets seems to work just fine.

I ran a NATS docker server on a remote server, and connected to it in one local terminal with nats --server=remote.server.ip.address sub hello.nick

Then using cloudflare wrangler on my local machine I ran and triggered the worker via the browser. The worker connects to the remote nats server and sends the PUB message.

The subscriber on the local machine using nats-cli then receives the message successfully.

Obviously telnet is a disaster, so it would be great if cloudflare:sockets connect() could be integrated into nats.js, perhaps with some sort of option/flag to choose it vs the standard mechanisms. Surely not a big job!

Some docs that I found helpful:
Setting up a new CF Worker Project and Wrangler
Using CF Worker TCP Sockets

Also, once this has been integrated, it seems to me that adding a quick doc (and perhaps even blog post or video) on this would be useful as well! Likewise letting CF know about it so that they can it to their integration docs, which would bring more attention to NATS. I created a thread in their Discord server about this - hopefully someone will reply.

import { connect } from 'cloudflare:sockets';
//import { connect as natsconnect } from "nats";

export default {
    async fetch(request, env, ctx) {
        const nats = { hostname: "5.161.125.2", port: "4222" };
        let writer;

        try {
            const socket = connect(nats);
            writer = socket.writable.getWriter();
            
            const encoder = new TextEncoder();
            const message = "hello world";
            
            const encodedMessage = encoder.encode(message);
            const payload_size = encodedMessage.length;
            
            const encoded = encoder.encode(`PUB hello.nick ${payload_size}\r\n${message}\r\n`);
            await writer.write(encoded);

            return new Response(socket.readable, { headers: { "Content-Type": "text/json" } });
        } catch (error) {
            return new Response(error.message, { status: 500 });
        } finally {
            if (writer) {
                await writer.close();
            }
        }
    }
};

[aricart]

Just for capturing:

 wrangler dev
 ⛅️ wrangler 3.19.0
-------------------
✔ Would you like to help improve Wrangler by sending usage metrics to Cloudflare? … no
Your choice has been saved in the following file: ../../../../Users/aricart/Library/Preferences/.wrangler/metrics.json.

  You can override the user level setting for a project in `wrangler.toml`:

   - to disable sending metrics for a project: `send_metrics = false`
   - to enable sending metrics for a project: `send_metrics = true`
✘ [ERROR] Could not resolve "util"

    node_modules/nats/lib/src/mod.js:33:49:
      33 │     const { TextEncoder, TextDecoder } = require("util");
         ╵                                                  ~~~~~~

  The package "util" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "crypto"

    node_modules/nats/lib/src/mod.js:38:22:
      38 │     const c = require("crypto");
         ╵                       ~~~~~~~~

  The package "crypto" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "stream/web"

    node_modules/nats/lib/src/mod.js:49:32:
      49 │         const streams = require("stream/web");
         ╵                                 ~~~~~~~~~~~~

  The package "stream/web" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "net"

    node_modules/nats/lib/src/node_transport.js:40:22:
      40 │ const net_1 = require("net");
         ╵                       ~~~~~

  The package "net" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "tls"

    node_modules/nats/lib/src/node_transport.js:42:22:
      42 │ const tls_1 = require("tls");
         ╵                       ~~~~~

  The package "tls" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "path"

    node_modules/nats/lib/src/node_transport.js:43:28:
      43 │ const { resolve } = require("path");
         ╵                             ~~~~~~

  The package "path" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "fs"

    node_modules/nats/lib/src/node_transport.js:44:41:
      44 │ const { readFile, existsSync } = require("fs");
         ╵                                          ~~~~

  The package "fs" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "dns"

    node_modules/nats/lib/src/node_transport.js:45:20:
      45 │ const dns = require("dns");
         ╵                     ~~~~~

  The package "dns" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


✘ [ERROR] Could not resolve "util"

    node_modules/nkeys.js/lib/index.js:35:25:
      35 │     const util = require("util");
         │                          ~~~~~~
         ╵                          "./util"

  The package "util" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.


╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [b] open a browser, [d] open Devtools, [l] turn off local mode, [c] clear console, [x] to exit                                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✘ [ERROR] Failed to build

[aricart]

It's a severely limited node environment.

[nickchomey]

CF workers has compatibility for most of those modules, as described in this link. https://developers.cloudflare.com/workers/runtime-apis/nodejs/

If you add node_compat = true to your wrangler.toml file as they suggest, most of those errors go away. I'm away from my computer, but I believe it says the only one remaining is stream/web (which seems odd, given that they have support for the Streams API, as can be seen in that link as well as the successful usage of streams in the snippet I shared)

And, as described above, using the connect() function from cloudflare workers TCP socket mechanism rather than nats.js' connect() allowed me to successfully connect to NATS and send a telnet PUB message that was received by a subscriber via nats-cli.

So, I'm hopeful that you could add an option that allows us to bypass nats.js' connect() in favour of Cloudflare's

Though I guess we would still have issues deploying the script due to missing (perhaps unused) node dependencies in cloudflare - perhaps a lightweight nats-cf.js fork could work? Whatever the case, I don't think that much needs to be done to allow for cloudflare compatibility! I'd be happy to help test etc

[aricart]

That would require a whole new client transport - effectively requiring yet another javascript client.

With that said using the nats.ws - which is an ES compliant library, I was able to get it running with a single change in the library using denoflare. How to do it with wrangler is another exercise.

import { NatsConnection, connect } from '/Users/aricart/Dropbox/code/src/github.com/nats-io/nats.ws/src/mod.ts';
let nc : NatsConnection;

export default {
	async fetch(request: Request, env: any) {
		if(!nc || nc.isClosed()) {
			nc = await connect({ servers: ['wss://demo.nats.io:8443'] });
		}
		try {
			return new Response('connected to ' + nc.getServer());
		} catch (e) {
			return new Response(e.message)
		}
	},
}

[nickchomey]

Thank you!

I just confirmed that using compatibility_flags = [ "nodejs_compat" ] in wrangler.toml only throws an error for stream/web (which you can just comment-out since streams are already in globalThis.ReadableStream. Also, as it turns out, instead of using node_compat = true (which apparently is a legacy mechanism), you can set compatibility_flags = [ "nodejs_compat" ] in wrangler.toml Then if you add node: in front of all of the required modules (e.g. require("node:stream/web") you no longer get any of the errors listed above.

Either way, the following error is then shown

✘ [ERROR] service core:user:nats: Uncaught Error: Some functionality, such as asynchronous I/O, timeouts, and generating random values, can only be performed while handling a request.

    at index.js:154:27 in fillRandom
    at index.js:193:9 in setPre
    at index.js:173:14 in init
    at index.js:163:14 in Nuid
    at index.js:233:20 in node_modules/nats/lib/nats-base-client/nuid.js
    at index.js:18:50 in __require2
    at index.js:272:18 in node_modules/nats/lib/nats-base-client/core.js
    at index.js:18:50 in __require2
    at index.js:525:18 in node_modules/nats/lib/nats-base-client/util.js
    at index.js:18:50 in __require2


✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

If you modify fillRandom to the following, this error no longer happens - the issue being that crypto.getRandomValues() seems to not be available in node (I found many reports of this)

function fillRandom(a) {
    var _a;
    if ((_a = globalThis === null || globalThis === void 0 ? void 0 : globalThis.crypto) === null || _a === void 0 ? void 0 : _a.getRandomValues) {
        try {
            globalThis.crypto.getRandomValues(a);            
        } catch {
            _getRandomValues(a);    
        }
    }
    else {
        _getRandomValues(a);
    }
}

But then another error pops up. I assume that'll keep happening and have no idea how deep it'll go.

So, yeah, it seems like unless you folks really want to roll up your sleeves (or get collaboration from Cloudflare's team), this might not be worth your effort (unless you find it to be a strategically important priority!)

I will give the nats.ws library a try and report back - I'm averse to using websockets in the browser for various reasons, but I don't have any particular objection to using them in Cloudflare Workers.

Thanks again for your attention and help!

[nickchomey]

I get this error with the nats.ws code you provided.

✘ [ERROR] No matching export in "node_modules/nats.ws/esm/nats.js" for import "NatsConnection"

    src/index.js:18:9:
      18 │ import { NatsConnection, connect } from 'nats.ws';
         ╵          ~~~~~~~~~~~~~~

If I add export { NatsConnectionImpl as NatsConnection }; to the bottoms of nats.js, then that error goes away. But it then throws this error, presumably because the functions in that class aren't sufficiently exported?

[wrangler:err] TypeError: nc.isClosed is not a function
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/index.js:23:16)
    at __facade_modules_fetch__ (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:46:16)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at Object.next (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:50:11)
    at jsonError (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:30)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at __facade_invoke__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:63:9)
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:114:11)
[wrangler:inf] GET / 500 Internal Server Error (191ms)
[wrangler:err] TypeError: nc.isClosed is not a function
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/index.js:23:16)
    at __facade_modules_fetch__ (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:46:16)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at Object.next (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:50:11)
    at jsonError (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:30)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at __facade_invoke__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:63:9)
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:114:11)

If I ignore NatsConnection altogether and use this code,

import { connect  } from 'nats.ws';

export default {
	async fetch(request, env) {
		
        nc = await connect({ servers: ['wss://demo.nats.io:8443'] });
	
		try {
			return new Response('connected to ' + nc.getServer());
		} catch (e) {
			return new Response(e.message)
		}
	},
} 

then i get this error sporadically:

✘ [ERROR] workerd/jsg/util.c++:281: error: e = kj/compat/http.c++:3300: failed: expected result == Z_OK || result == Z_BUF_ERROR || result == Z_STREAM_END; Decompression failed; result = -3;  with reason; ctx.msg = invalid stored block lengths

  stack:
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2875f0a
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2878dbd
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@287a8a4
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2896da0
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2896fda
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f9b1f0
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28b5a30
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@21aad48;
  sentryErrorContext = jsgInternalError


✘ [ERROR] Uncaught (in promise) Error: internal error


✘ [ERROR] Uncaught (async) Error: internal error

other times I get this in the browser

If I use step debugging, the issue seems to come from these lines

const cp = this.transport.connect(srv, this.options);
await Promise.race([

Ultimately, I just don't think I can get this to work. I'm a php developer so this is all beyond my capabilities/knowledge. But if someone more competent is willing to roll up their sleeves (or guide me, but then they might as well just do it themselves...), then I really do think it should be possible to get it working for Cloudflare Workers!

[Hebilicious]

Thank you for your efforts @nickchomey , the JS world can be an endless rabbithole of stuff like that...

I believe this would be a really good thing to get working and the reason why it hasn't happened yet might be because the JavaScript community tends to prefer cloud services instead of self-hosting things, and NATS unfortunately doesn't have something like upstash redis (afaik) ; an HTTP client for Redis which makes it much easier to use Redis with non node.js environments. Unfortunately the HTTP client doesn't work with redis pub/sub, hence how I found-out about this thread.

I would love to have an official way to use NATS with cloudflare workers and will subscribe to this issue.