Develop Reference

How to eval a function that use external modules in a web worker

I am trying to implement a simple library to offload given tasks to a pool of web-workers. To do so, I am using the eval operator inside the worker to evaluate stringified functions that come from the main thread. To do so I am leveraging the webpack's plugin worker-plugin to automatically bundle workers inside the application.
First of all, I format the task to normalize arrow functions, class members, and standard functions
function format_task(task: string): string
{
if(task.indexOf('=>') !== -1)
return task;
if(task.indexOf('function') !== -1)
return `(function ${task.slice('function'.length).trim()})`;
return `(function ${task})`;
}
Then, I simply eval the resulting function calling it:
self.onmessage = (event: MessageEvent) =>
{
const {task, data} = event.data;
const formatted_task = format_task(task);
const runnable_task = eval(formatted_task);
const result = await runnable_task(...data);
self.postMessage(result);
}
The main usage would be something like:
const pool = new WorkerPool(new Worker('./worker.ts'), 10);
const task = (a: number, b: number) => a + b;
const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12);
So far, so good. The WorkerPool instance is handling a pool of 10 workers, the function gets executed in one of them, and the result gets returned to the main thread.
The problem comes in when I try to use an external dependency inside the task as
import {toUpper} from 'lodash';
const pool = new WorkerPool(new Worker('./worker.ts'), 10);
const task = (a: number, b: number) =>
{
return toUpper('Result is ') + (a + b));
}
const worker_task = await pool.create_task(task);
const result = await worker_task.run(42, 12);
In this case, I get that lodash_1.toUpper is not defined inside the worker. The only solution would be to manually import the same function also inside the worker environment (it works). However, I would like to keep the web-worker implementation as clean as possible for the final user. Something like:
const context: Worker = self as any;
(async () =>
{
const pool_worker = new PoolWorker(context.postMessage.bind(context));
context.onmessage = pool_worker.on_message.bind(pool_worker);
await pool_worker.run();
})().catch(console.error);
I tried importing the needed libraries inside the main worker file but it does not work.
import {toUpper} from 'lodash';
const context: Worker = self as any;
(async () =>
{
const pool_worker = new PoolWorker(context.postMessage.bind(context));
context.onmessage = pool_worker.on_message.bind(pool_worker);
await pool_worker.run();
})().catch(console.error);
It only works if I import it inside the PoolWorker class.
Can you think of a way to "pass" dependencies to the PoolWorker instance letting it importing it through webpack without writing a custom webpack loader/plugin?
Thank you in advance

At an object's instantiation time how does one handle best the asynchronous initialization of one of its properties?

I've never created a Javascript module/library before so this is a bit new to me so apologizes for my lack of knowing what to google.
I'm creating a library that will hold information from a URL that is provided by a user. I want to parse the URL's path (the part that comes after the domain) as well as retain a header value that's provided by the URL's response.
It's basic but here's what I have so far:
function Link(someURL) {
this.url = someURL;
this.urlPath = "";
this.uuid = "";
this.getPath = function (someURL) {
// do regexp parsing and return everything after the domain
};
this.getUUID = function (someURL) {
// fetch the URL and return what is in the response's "uuid" header
}
}
Ideally, I'd the module to automatically get all the information upon construction:
var foo = new Link("http://httpbin.org/response-headers?uuid=36d09ff2-4b27-411a-9155-e82210a100c3")
console.log(foo.urlPath); // should return "uuid"
console.log(foo.uuid); // should return the contents in the "uuid" header in the response
How do I ensure the this.urlPath and this.uuid properties get initialized along with this.url? Ideally, I'd only fetch the URL once (to prevent rate limiting by the target server).

After a lot of trial and error, I ended up doing something more like this:
class Link {
constructor (url_in) {
const re = RegExp("^https://somedomain.com\/(.*)$");
this.url = re[0];
this.linkPath = re[1];
}
async getUUID() {
const res = await fetch("https://fakedomain.com/getUUID?secret=" + this.linkPath);
this.uuid = res.uuid;
}
async getJSON() {
const res = await fetch("https://fakedomain.com/getJSON?uuid=" + this.uuid);
this.json = await res.json();
}
async initialize() {
await this.getUUID();
await this.getJSON();
}
}
const someLinkData = new Link("https://reallydumbdomain.com/2020/10/4/blog");
someLinkData.initialize()
.then(function() {
console.log(this.json); // this now works
});
I think a future iteration of this will require me to send a promise with the initialize function but for now, this works.

Results from query are undefined

I have been adding custom functionality in Strapi code through the controller.
The below result variable comes up as undefined. If I return the result variable directly in the function the result is correctly returned in the http response. Is there any issues in the syntax below ? I appreciate any help i can get.
'use strict';
/**
* Read the documentation (https://strapi.io/documentation/3.0.0-beta.x/concepts/controllers.html#core-controllers)
* to customize this controller
*/
module.exports = {
async getItemAggregate (context) {
var itemid = context.params.itemid
strapi.log.info(itemid)
strapi.log.info("test aggregate");
const result = await strapi.query('aggregate').findOne({"id": itemid});
strapi.log.info(result)
// const fields = result.toObject();
// strapi.log.info(result)
// entities = await strapi.services.aggregate.search({"id":1});
// entities2 = await strapi.services.item.search({"id":1});
// strapi.log.info(entities)
strapi.log.info(result)
// strapi.log.info(entities2)
//get latest aggregate
//get latest tranacitons
}
};

Had a similar issue. The thing I found out is that strapi.log.<function> doesn't stringify the query. Try logging the query wrapped with JSON.stringify(<query>).
Like this:
const result = await strapi.query('aggregate').findOne({"id": itemid});
strapi.log.info(JSON.stringify(result));

Create global variable in utils.js file in node.js

Hi I'm currently trying to figure it out how to properly define global variable in node.js. I know it is not a good practice to do this, but in this particular screnario it's the only way to do this without using connection to database.
I'm getting data from github API to display some information, I'm trying to store them in the global variable. It should allow me to e.g pass specific object from first global list to new global list that display only chosen items.
I have file called utils.js that have this two empty arrays that should be global:
let repoItems = [];
let bookmarkedItems = [];
exports.repoItems = repoItems;
exports.bookmarkedItems = bookmarkedItems;
Then I have another file that fetch and should assign items to the first global variable, but it looks like it doesn't. Because in the moment I'm trying to chose one item & push it into second empty array it's impossible, I'm getting empty array. I'm not sure if the mistake is taken because of bad implementation of global variable from other file, or something else :/
Below I include the fetching part of code, with bold part that I'm confused with:
let utils = require('../utils');
let {
repoItems,
} = utils;
router.get('/', async (req, res) => {
try {
const result = await fetchGithubAPI(`searching word`);
const urls = result.map(url => url);
console.log(urls);
res.render('repositories.ejs', {
'data': urls
});
} catch (e) {
console.log(e);
}
});
async function fetchGithubAPI(search) {
const response = await fetch(`https://api.github.com/?q=${search}`, {
method: 'GET',
headers: {
'Accept': 'application/vnd.github.v3+json',
'Content-Type': 'application/json',
},
});
const data = await response.json();
**repoItems = data.items.map(item => item);**
return repoItems;
}

Try:
let repoItems = require('./utils').repoItems
instead of
let {
repoItems,
} = utils;

if you remove
let {
repoItems,
} = utils;
you can try using
utils.repoItems = data.items.map(item => item)
I tried an example setup for it
--utils.js
module.exports.someValue = 3;
module.exports.setSomeValue = (value) => {someValue = value}
--index.js
const utils = require('./utils');
console.log(utils.someValue);// 3
utils.someValue = 5
console.log(utils.someValue);// 5
Update after knowing about getter and setter methods in Js
You can refer to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/set
this is another way to change value of any private properties in JS

Problems when using Chosen Selectors in external helper functions

I use Testcafe to test a website which is using the jquery plugin Chosen and
I want to make an assertion in my test code depending on a value returned by an external helper function (getSelectedOption).
This function gets a Chosen Selector as a parameter and should return the selected value to the assertion, but the function always returns the first element of the list instead of the chosen one.
When I use the function code in my test, everything works fine.
It seems that the function doesn't have the actual state about the HTML data and can't see that an element is already selected.
This is a snippet from the test code:
await t
.click(await getOptionByText('salutation', 'Frau'))
.expect(await getSelectedOption('gender')).eql('weiblich')
This is a snippet from the external functions:
export const getChosenSelectorFromName = selectName => `#${selectName}_chosen`;
export const getSelectedOption = async selectName => {
const selectedOptionText = await
Selector(getChosenSelectorFromName(selectName))
.find('.chosen-single')
.innerText;
return selectedOptionText.toLowerCase().trim()
};
export const getOptionByText = async (selectName, optionText) => {
const chosenSelectorString = getChosenSelectorFromName(selectName);
await t.click(Selector(chosenSelectorString));
return await Selector(chosenSelectorString)
.find('.chosen-drop')
.find('li')
.withText(optionText);
};
When I use similar code like the getSelectedOption function inside my test, everything works fine:
const genderSelect = Selector('#gender_chosen);
.click(await getOptionByText('salutation', 'Frau'))
.expect(genderSelect.innerText).eql('WEIBLICH')

If you call await Selector(<some value>) then TestCafe immediately retries the data from the web page at the current moment.
You can tell TestCafe to retry data from web page until it becomes equal to the expected value.
To do it, you need to move the DOM manipulation function into ClientFunction:
import { Selector, ClientFunction } from "testcafe";
fixture `Fixture`
.page('https://harvesthq.github.io/chosen/');
const getChosenSelectorFromName = selectName => `#${selectName}_chosen`;
const getSelectedOption = ClientFunction(selector => {
var choosenDiv = document.querySelector(selector);
var singleValueEl = choosenDiv.querySelector('.chosen-single');
return singleValueEl.innerText;
});
test('test', async t => {
await t.expect(getSelectedOption('.chosen-container')).eql('Choose a Country...');
});