ReferenceError: TransformStream is not defined - javascript

I'm attempting to test the implementation of Fast Google Fonts with Cloudflare Workers, from Cloudflare's blog for inlining the Google Fonts stylesheet directly into the HTML. The implementation itself seems to be working fine when running via the Cloudflare worker. But I wrote some tests, and when running the tests, I get this error that says TransformStream is not defined.
Error when running my Jest test via npm run test:
ReferenceError: TransformStream is not defined
This is the code that the test is hitting:
async function processHtmlResponse(request, response) {
...
// This is where the reference error comes up
const { readable, writable } = new TransformStream();
const newResponse = new Response(readable, response);
modifyHtmlStream(response.body, writable, request, embedStylesheet);
return newResponse;
}
This test looks something like this, where we basically expect that the stylesheet link will be replaced by the stylesheet itself, containing all the #font-face styles.
describe('inlineGoogleFontsHtml', () => {
it('inlines the response of a Google Fonts stylesheet link within an HTML response', async () => {
const request = new Request('https://example.com')
const response = new Response(
'<html><head><link rel="stylesheet" media="screen" href="https://fonts.googleapis.com/css?family=Lato:300"></head><body></body></html>',
{
headers: {
'Content-Type': 'text/html'
}
}
)
const rewrittenResponse = processHtmlResponse(request, response)
const rewrittenResponseText = await rewrittenResponse.text()
expect(rewrittenResponseText).toContain('#font-face')
})
I'm not really sure what the issue is here. Does TransformStream work in Node? Is there some polyfill that's needed?
Related:
Cloudflare streams

TransformStream is part of the Streams API, a browser-side standard. It is not implemented by Node (because they had their own streams long before this spec existed), so you will need a polyfill when testing your code in Node.
Incidentally, the example you're following is fairly old. These days, it would be better to use HTMLRewriter to implement this kind of transformation -- it is much more efficient for rewriting HTML specifically. (However, it is a Cloudflare-specific feature, so you wouldn't be able to test it under Node at all.)

Related

I am creating electron app and getting Uncaught Exception: TypeError: $.ajax is not a function

I am creating electron app that hits an endpoint from API and shows the response on the page. Since electron separated the main process from the renderer process I have to use preload script to communicate between the two processes so I can update the page on load. I use jQuery AJAX to make async call to the API, but got error like this
Uncaught Exception:
TypeError: $.ajax is not a function
at IpcMainImpl.<anonymous> (/Users/***/***/***/***/main.js:41:7)
...
I put the AJAX call in the main.js. Here are some snippets from my code
main.js
const { app, BrowserWindow, ipcMain } = require('electron')
const path = require('path')
let $ = require('jquery')
let text
let win
const createWindow = () => {
win = new BrowserWindow({
// some code...
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
nodeIntegration: true
}
})
win.loadFile('index.html');
win.webContents.send("showText", text);
// code...
ipcMain.on("receiveData", (event, formData) => {
$.ajax({
// code...
}
});
});
preload.js
const { contextBridge, ipcRenderer } = require('electron')
let postData = (formData) => {
ipcRenderer.send("receiveData", formData)
}
let showText = (callback) => {
ipcRenderer.on("showText", (callback))
}
let bridge = {postData: postData, showText: showText}
contextBridge.exposeInMainWorld("bridge", bridge)
renderer.js
$("#submit").on("click", function () {
// code...
window.bridge.postData(formData);
});
window.bridge.showText((event, text) => {
$("#result").append(text)
});
Also I got this error in console, thought it somewhat related.
Error: Cannot bind an API on top of an existing property on the window object
at Object.exposeInMainWorld (VM386 renderer_init:45:364)
I have tried importing jQuery and placing the AJAX in renderer according to this answer but it didn't work because the renderer doesn't have access to node environment, I guess. I already have the regular build of jQuery instead of the slim one as said in many answers. Been searching many solutions about similar error but all of them didn't seem to work for me. I've only learnt electron days ago so perhaps there's something terribly obvious I have missed. I would appreciate any help.
JQuery does not work in a NodeJS environment without a (mocked) DOM, as also explicitly stated on its NPM documentation. You will either have to rely on NodeJS' own web request tools (such as the https module) or use the solution provided in the NPM ReadMe of JQuery (using a third-party package to mock a window object).
Generally, it may be worthwile to read into Electron's process model.

Importing "#daily-co/daily-js" into SvelteKit app throws "global is not defined" error

What I tried:
I tried work around it via if (browser), more specifically{
if (!browser) { let DailyIframe = await import('daily-co/daily-js) } in the load function inside <script context="module"> ) so the code is always executed on the server). Then pass it as a prop to a component. However, although it worked on the server, the local dev environment re-runs the load function (which has to return an empty prop as it never imported anything) and overrides DailyIframe's value (might be a bug with Vite/SvelteKit).
I tried to import the library in an end-point e.g. api.json.js instead, which is always executed on the server. However, it has to return a json, and I can't pass an entire library variable onto it.
After research
It seems like a combination of problems from Vite, SvelteKit and certain libraries where global is undefined: SvelteKit With MongoDB ReferenceError: global is not defined)
But I cannot use his solution of putting it in an endpoint, because I need the DailyIframe and the mic audio stream from the client to create a video conference room
Also, why would certain libraries Daily (and looking at other related Stackoverflow posts, MongoDB) throw this error in the first place, while other libraries are safe to use?
Anyway suggestion is appreciated!
Why ?
Vite doesn't include shims for Node builtins variables.
Read suggestion to understand:
https://github.com/vitejs/vite/issues/728
https://github.com/angular/angular-cli/issues/9827#issuecomment-369578814
Anyway suggestion is appreciated!
In index.html add :
<script>
var global = global || window;
</script>
then for example in App.svelte :
<script>
import { onMount } from 'svelte'
import DailyIframe from '#daily-co/daily-js'
onMount(async () => {
let callObject = DailyIframe.createFrame()
const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
let recorder = new MediaRecorder(stream)
recorder.start()
})
</script>
👉 Demo
https://stackblitz.com/edit/sveltekit-1yn6pz?devtoolsheight=33
logs preview

Possible to unload, delete, or unrequire a Nodejs module

We are building an Electron app that allows users to supply their own 'modules' to run. We are looking for a way to require the modules but then delete or kill the modules if need be.
We have looked a few tutorials that seem to discuss this topic but we can't seem to get the modules to fully terminate. We explored this by using timers inside the modules and can observe the timers still running even after the module reference is deleted.
https://repl.it/repls/QuerulousSorrowfulQuery
index.js
// Load module
let Mod = require('./mod.js');
// Call the module function (which starts a setInterval)
Mod();
// Delete the module after 3 seconds
setTimeout(function () {
Mod = null;
delete Mod;
console.log('Deleted!')
}, 3000);
./mod.js
function Mod() {
setInterval(function () {
console.log('Mod log');
}, 1000);
}
module.exports = Mod;
Expected output
Mod log
Mod log
Deleted!
Actual output
Mod log
Mod log
Deleted!
Mod log
...
(continues to log 'Mod log' indefinitely)
Maybe we are overthinking it and maybe the modules won't be memory hogs, but the modules we load will have very intensive workloads and having the ability to stop them at will seems important.
Edit with real use-case
This is how we are currently using this technique. The two issues are loading the module in the proper fashion and unloading the module after it is done.
renderer.js (runs in a browser context with access to document, etc)
const webview = document.getElementById('webview'); // A webview object essentially gives us control over a webpage similar to how one can control an iframe in a regular browser.
const url = 'https://ourserver.com/module.js';
let mod;
request({
method: 'get',
url: url,
}, function (err, httpResponse, body) {
if (!err) {
mod = requireFromString(body, url); // Module is loaded
mod(webview); // Module is run
// ...
// Some time later, the module needs to be 'unloaded'.
// We are currently 'unloading' it by dereferencing the 'mod' variable, but as mentioned above, this doesn't really work. So we would like to have a way to wipe the module and timers and etc and free up any memory or resources it was using!
mod = null;
delete mod;
}
})
function requireFromString(src, filename) {
var Module = module.constructor;
var m = new Module();
m._compile(src, filename);
return m.exports;
}
https://ourserver.com/module.js
// This code module will only have access to node modules that are packaged with our app but that is OK for now!
let _ = require('lodash');
let obj = {
key: 'value'
}
async function main(webview) {
console.log(_.get(obj, 'key')) // prints 'value'
webview.loadURL('https://google.com') // loads Google in the web browser
}
module.exports = main;
Just in case anyone reading is not familiar with Electron, the renderer.js has access to 'webview' elements which are almost identical to iframes. This is why passing it to the 'module.js' will allow the module to access manipulate the webpage such as change URL, click buttons on that webpage, etc.
There is no way to kill a module and stop or close any resources that it is using. That's just not a feature of node.js. Such a module could have timers, open files, open sockets, running servers, etc... In addition node.js does not provide a means of "unloading" code that was once loaded.
You can remove a module from the module cache, but that doesn't affect the existing, already loaded code or its resources.
The only foolproof way I know of would be to load the user's module in a separate node.js app loaded as a child process and then you can exit that process or kill that process and then the OS will reclaim any resources it was using and unload everything from memory. This child process scheme also has the advantage that the user's code is more isolated from your main server code. You could even further isolate it by running this other process in a VM if you wanted to.

using WebAssembly in chrome extension

I have a chrome extension that includes a complicated function comp_func(data) which takes a lot of CPU by performing many bitwise operations. Because of that, I'm trying to use WebAssembly.
I've tried to follow several tutorials, for example this one and this one.
The first link says:
fetch('simple.wasm').then(response =>
response.arrayBuffer()
).then(bytes =>
WebAssembly.instantiate(bytes, importObject)
).then(results => {
results.instance.exports.exported_func();
});
but I get an error:
Uncaught (in promise) TypeError: WebAssembly Instantiation: Import #0 module="env" error: module is not an object or function
I've tried a lot to use this approach, but it didn't work. I can't understand how to use WebAssembly that is loaded from the .wasm file.
So I've tried an easier approach:
The second link says to put this line in the html file:
<script src="index.js"></script>
and then just use the exported function:
var result = _roll_dice();
BUT, I'm in an extension so I only have a background.html file.
So I'm looking for a way to access the Module which was loaded in the background file.
And things get complicated, because the function comp_func(data) is called from a Worker.
This is what I've tried so far:
If I call chrome.extension.getBackgroundPage() I can access the Module
but I can't send it to the Worker:
Failed to execute 'postMessage' on 'Worker': # could not be cloned.
And if I try to stringify it first:
Uncaught TypeError: Converting circular structure to JSON
(I tried to un-circular it, didn't work...)
And I can't call chrome.extension.getBackgroundPage() from the Worker because I can't access chrome API from there.
So my questions are:
Did someone try to load .wasm file in chrome extension and succeed?
The second approach (loading the js file) sounds simpler but if you have a working example for this approach it would be great.
or 2. How to access the Module that has been loaded in background.html (from the second example)?
or 3. How to pass the functions that I needed from the js file to the Worker (via postMessage)?
To summarize, did someone try to use WebAssembly in a chrome extension and survive to tell?
EDIT:
I eventually left the approach of WebAssembly.
I also posted this question at bugs-chromium,
and after few month got an answer. Not sure if this is really working, but maybe this, along with the marked answer, will help someone.
I've been fiddling with WebAssembly recently, and found a way to make it work. Here are the script files:
main.js
chrome.browserAction.onClicked.addListener(function(tab) {
chrome.tabs.executeScript(null, {file: "content_script.js"});
});
content_script.js
var importObject = { imports: { imported_func: arg => console.log(arg) } };
url = 'data:application/wasm;base64,' + "AGFzbQEAAAABCAJgAX8AYAAAAhkBB2ltcG9ydHMNaW1wb3J0ZWRfZnVuYwAAAwIBAQcRAQ1leHBvcnRlZF9mdW5jAAEKCAEGAEEqEAAL";
WebAssembly.instantiateStreaming(fetch(url), importObject)
.then(obj => obj.instance.exports.exported_func());
The data URL belongs to the common tutorial wasm sample (simple.wasm), which writes 42 on the console.
PS. If it seems like cheating or bad practice to you, this content_script.js also works:
var importObject = {
imports: {
imported_func: function(arg) {
console.log(arg);
}
}
};
var response = null;
var bytes = null;
var results = null;
var wasmPath = chrome.runtime.getURL("simple.wasm");
fetch(wasmPath).then(response =>
response.arrayBuffer()
).then(bytes =>
WebAssembly.instantiate(bytes, importObject)
).then(results => {
results.instance.exports.exported_func();
});
Only if you include the code files in the web_accessible_resources section in manifest.json, though:
...
"web_accessible_resources": [
"content_script.js",
"main.js",
"simple.wasm"
],
...
Github: https://github.com/inflatablegrade/Extension-with-WASM
It can also be made compatible with Manifest V3.

Node.js listen for module load

With RequireJS on the front-end, we can listen to see when modules get loaded into the runtime module cache using:
requirejs.onResourceLoad = function (context, map, depArray) {
console.log('onResourceLoad>>>', 'map.id:', map.id, 'context:', context);
};
Can we do this with Node.js somehow? Will be useful for debugging. Especially when servers are loading different files (or in different order) based on configuration.
I assume this might be documented in
https://nodejs.org/api/modules.html
but I am not seeing anything
If you look at the source code for require(), you will find this:
Module._load = function(request, parent, isMain) {
if (parent) {
debug('Module._load REQUEST %s parent: %s', request, parent.id);
}
This shows that you can leverage the debug() call to get the information you need. In order to do this, you will notice that module is setup using util.debuglog('module'). This means that you need to run your application with with the NODE_DEBUG variable set to module. You can do it the following way from the console:
NODE_DEBUG=module node main.js
This will log what you are looking for.
I'm not aware of a documented callback API for the purpose of module load callbacks (although a logging mechanism for module loading appears to exist).
Here's a quick workaround to the apparent lack of such a callback, by monkeypatching Module._load:
const Module = require('module');
const originalModuleLoad = Module._load;
Module._load = function() {
originalModuleLoad.apply(this, arguments);
console.log("Loaded with arguments " + JSON.stringify(arguments));
}
I executed the above code in a REPL and then did require('assert'). Lo and behold:
> require('assert')
Loading with arguments {"0":"assert","1":{"id":"<repl>","exports":{},"filename":null,"loaded":false,"children":[],"paths":["/Users/mz2/Projects/manuscripts-endnote-promo/repl/node_modules","/Users/mz2/Projects/manuscripts-endnote-promo/node_modules","/Users/mz2/Projects/node_modules","/Users/mz2/node_modules","/Users/node_modules","/Users/mz2/.nvm-fish/v6.1.0/lib/node_modules","/Users/mz2/.node_modules","/Users/mz2/.node_libraries","/Users/mz2/.nvm-fish/v6.1.0/lib/node"]},"2":false}
Please don't think about using code like above for anything but debug only purposes.
Because node.js modules are imported (required) synchronously, simply having the require statement means the module is imported.
While RequireJS can import modules asynchronously, the even listening is an important feature, but native require in Node.js leaves this necessity out. This way, as you probably know:
const module = require('module')
// You can use the module here, async or sync.
To add to that, not only require is sync, but also in order to use a module it has to be explicitly required in the same file where it's used. This can be bypassed in several ways, but best practice is to require in every module where you use a module.
For specific modules which require async initialization, either the module should provide an event, or you can wrap the init function using a promise or a callback. For example, using a promise:
const module = require('module')
// Create a promise to initialize the module inside it:
const initialized = new Promise((resolve, reject) => {
// Init module inside the promise:
module.init((error) => {
if(error){
return reject(error)
}
// Resolve will indicate successful init:
resolve()
})
})
// Now with wrapped init, proceed when done:
initialized
.then(() => {
// Module is initialized, do what you need.
})
.catch(error => {
// Handle init error.
})

Categories

Resources