Develop Reference

How to run a pickle file in node js

Working in an AI/ML project, I need a way to run the pickle file inside nodejs so I can use it to run the algorithm on the data they submitted.

Try to use the node-pickle library to convert the pickle file to the JSON object. Here documentation of node-pickle
const nodePickle = require('node-pickle');
// Convert pickled object to JSON object
nodePickle.load(pickledData)
.then(data => ({
// data is a JSON object here
})
Then you can use tensorflow.js to run that JSON object as a model.
Tenrsorflow.js Documentation

The only one I could find from a quick google search is node-jpickle. According to it's docs, it seems to be able to handle most of the situations pickle can, even some of the more advanced ones such as classes:
function MyClass() {}
var jpickle = require('jpickle');
jpickle.emulated['__main__.MyClass'] = MyClass;
var unpickled = jpickle.loads(pickled);
If you're just trying to pickle/unpickle, you can just do it like you would in Python:
var unpickled = jpickle.loads(pickled);
The docs don't state anything about a normal load function however.

node.js dynamic module(file) export

how can I exports this dynamic module?
// ./data/example5.js
module.exports = {title : example5, recentCheck : "2018-08-22"}
The contents change in real time. And I execute the followed function once a minute and connect the module.
var files = fs.readdirSync(`./data/`);
var i = 0;
var link = [];
while(i<files.length){ // read all file
link[i] = require(`../data/${files[i]}`);
　
//main.js
setInterval(start,10000);
I try to create and connect a new module file once a minute, but the first stored file(module) is extracted. The modulefile is being saved in real time correctly.
Shutting down and running the node will extract the changed modules correctly.
How do I handle a dynamically changing module?

I would recommend saving the data in a JSON file and then reading the data from the file rather than trying to use it as a module.

Just make the objects you're updating variables in the module that you're including.
Make a function called getVariable, and simply return the variable.
Include getVariable in your main module.

NodeJS & Gulp Streams & Vinyl File Objects- Gulp Wrapper for NPM package producing incorrect output

Goal
I am currently trying to write a Gulp wrapper for NPM Flat that can be easily used in Gulp tasks. I feel this would be useful to the Node community and also accomplish my goal. The repository is here for everyone to view , contribute to, play with and pull request. I am attempting to make flattened (using dot notation) copies of multiple JSON files. I then want to copy them to the same folder and just modify the file extension to go from *.json to *.flat.json.
My problem
The results I am getting back in my JSON files look like vinyl-files or byte code. For example, I expect output like
"views.login.usernamepassword.login.text": "Login", but I am getting something like {"0":123,"1":13,"2":10,"3":9,"4":34,"5":100,"6":105 ...etc
My approach
I am brand new to developing Gulp tasks and node modules, so definitely keep your eyes out for fundamentally wrong things.
The repository will be the most up to date code, but I'll also try to keep the question up to date with it too.
Gulp-Task File
var gulp = require('gulp'),
plugins = require('gulp-load-plugins')({camelize: true});
var gulpFlat = require('gulp-flat');
var gulpRename = require('gulp-rename');
var flatten = require('flat');
gulp.task('language:file:flatten', function () {
return gulp.src(gulp.files.lang_file_src)
.pipe(gulpFlat())
.pipe(gulpRename( function (path){
path.extname = '.flat.json'
}))
.pipe(gulp.dest("App/Languages"));
});
Node module's index.js (A.k.a what I hope becomes gulp-flat)
var through = require('through2');
var gutil = require('gulp-util');
var flatten = require('flat');
var PluginError = gutil.PluginError;
// consts
const PLUGIN_NAME = 'gulp-flat';
// plugin level function (dealing with files)
function flattenGulp() {
// creating a stream through which each file will pass
var stream = through.obj(function(file, enc, cb) {
if (file.isBuffer()) {
//FIXME: I believe this is the problem line!!
var flatJSON = new Buffer(JSON.stringify(
flatten(file.contents)));
file.contents = flatJSON;
}
if (file.isStream()) {
this.emit('error', new PluginError(PLUGIN_NAME, 'Streams not supported! NYI'));
return cb();
}
// make sure the file goes through the next gulp plugin
this.push(file);
// tell the stream engine that we are done with this file
cb();
});
// returning the file stream
return stream;
}
// exporting the plugin main function
module.exports = flattenGulp;
Resources
https://github.com/gulpjs/gulp/blob/master/docs/writing-a-plugin/README.md
https://github.com/gulpjs/gulp/blob/master/docs/writing-a-plugin/using-buffers.md
https://github.com/substack/stream-handbook

You are right about where the error is. The fix is simple. You just need to parse file.contents, since the flatten function operates on an object, not on a Buffer.
...
var flatJSON = new Buffer(JSON.stringify(
flatten(JSON.parse(file.contents))));
file.contents = flatJSON;
...
That should fix your problem.
And since you are new to the Gulp plugin thing, I hope you don't mind if I make a suggestion. You might want to consider giving your users the option to prettify the JSON output. To do so, just have your main function accept an options object, and then you can do something like this:
...
var flatJson = flatten(JSON.parse(file.contents));
var jsonString = JSON.stringify(flatJson, null, options.pretty ? 2 : null);
file.contents = new Buffer(jsonString);
...
You might find that the options object comes in useful for other things, if you plan to expand on your plugin in future.
Feel free to have a look at the repository for a plugin I wrote called gulp-transform. I am happy to answer any questions about it. (For example, I could give you some guidance on implementing the streaming-mode version of your plugin if you would like).
Update
I decided to take you up on your invitation for contributions. You can view my fork here and the issue I opened up here. You're welcome to use as much or as little as you like, and in case you really like it, I can always submit a pull request. Hopefully it gives you some ideas at least.
Thank you for getting this project going.

access module of other context in requirejs

I am trying to access a togetherJS (https://togetherjs.com/docs/contributing.html) module from an external requireJS app. It seems impossible.
TogetherJSConfig_noAutoStart = true;
var CJS = CJS || {};
CJS.require = require.config({
paths: {
togetherjs: 'https://togetherjs.com/togetherjs-min',
cjs: 'scripts/c'
}
});
CJS.require(['togetherjs'], function() {
// not working
peers = require({context: "togetherjs"})("peers");
// not working
// Module name "peers" has not been loaded yet for context: togetherjs. Use require([])
TogetherJS.require = require.config(TogetherJS.requireConfig);
TogetherJS.require("peers");
});
If I do not use requireJS in my app, I can access the module I want:
// works if I do not use requireJS in my app.
var peers = TogetherJS.require('peers').getAllPeers();
Is it possible, and if so, how? I could not find any information anywhere.
Thanks

I'm not entirely sure of what you are trying to accomplish here. It looks like you are mixing up require.js asynchronous api with node.js synchronous require api.
I think that when you do require.config without specifying a context name, you are configuring the global require context. So calling CJS.require(..) should be equivalent to calling window.require(..).
The line
peers = require({context: "togetherjs"})("peers");
looks strange to me. Usually you would specify the context name in the require.config call and the first parameter to a require(..) call would be the dependency list. Also, you would normally pass a callback function to the require(..) call as well.
I hope those pointers can help you (or others) get better acquainted with require.js

Building a javascript app shared across node.js and client side

I am used to cross platform dev. On the C/C++ side it's simple, but I'm stuck with a problem when using javascript. I want to be able to reuse code from my web-service on the client-side. Currently node.js requires me to write my program in a most unpleasant way but I can handle it:
//MyClass.js
function MyClass()
{
this.test = function(){}
}
module.exports.MyClass = MyClass;
//server.js
var m = new require('MyClass.js').MyClass();
m.test();
//client.js
$.getScript("MyClass.js", function(){
var m = new MyClass();
m.test();
});
To that point it is all fine, but I have an issue when I need to load classes from MyClass.js. How can I make it work across all the platforms? Is there a way to achieve reusability without processing the files?

Node requires you to do nothing. If you aren't a fan of the way the Node module system works, simply don't use it.
//MyClass.js
MyClass = function ()
{
this.test = function(){}
}
//server.js
require('./MyClass.js');
var m = new MyClass();
m.test();
Now your application is compatible with what you have going on client-side. Just bear in mind that you are now creating classes in the global namespace, which is one reason for using Node's module layout.
I suggest also looking into some of the ways to use a Node-style require on the client. There are many scripts available, such as RequireJS or Browserify.