Develop Reference

Parent process exits when child process tries to load keras model from python script

I'm working on an application using a keras model as a classifier. This works fine by itself, however it is when I try to pipe this data to a js script for use in the frontend/GUI part I'm running into problems. The code snippet failing is the following:
JS
const sensor = spawn('python', ['../../scripts/realtime_test.py']);
let count = 0;
sensor.stdout.on('data', function(data) {
// Parse int if it is a classification, if it isn't, it is a debugging message
let val = parseInt(data);
if(!isNaN(val)){
temperatures.push(val)
} else{
val = String.fromCharCode.apply(null, data)
temperatures.push(val)
}
console.log(count + " " + temperatures[count++]);
});
python
if __name__ == "__main__":
openzen.set_log_level(openzen.ZenLogLevel.Off)
# Make client
error, client = openzen.make_client()
if not error == openzen.ZenError.NoError:
print("Error while initializing OpenZen library", flush=True, end='')
sys.exit(1)
# Scan, connect and syncronize sensors
sensors_found = scan_for_sensors(client)
#name, s, imu, b = connect_to_sensor(client, sensors_found[0])
sensor_bank = Sensor_Bank()
for sensor in sensors_found:
name, s, imu = connect_to_sensor(client, sensor)
sensor_bank.add_sensor(name, s, imu)
#print(len(sensor_bank.sensor_arr))
data_queue = Data_Queue(len(sensor_bank.sensor_arr))
sync_sensors(client, sensor_bank)
model = keras.models.load_model(f'model/models/ANN_model_{NUM_SENSORS}.h5') # FAILING HERE
classify_thread = threading.Thread(target=classify, args=[model, data_queue], daemon=True)
classify_thread.start()
collect_data(client, data_queue, sensor_bank)
What seems to be the problem is the loading of the model. The process just exits with no error message, working perfectly fine up until the line I've marked as FAILING in the python code snippet. AFAIK the keras.load function doesn't spawn a separate process of it's own, so I don't suspect multiple processes clashing could be the culprit... Any ideas as to why this is happening?
Thanks in advance!

XML parsing syntax for OS X Yosemite JavaScript for Automation?

Has anyone deduced syntax which successfully loads XML from file/string and gives access to the data in OS X Yosemite (10.10) Javascript for Automation ?
Documentation and code samples are still fairly thin (as of Nov 2014), and my inductive skills are running dry on three separate approaches to reading an XML (OPML) file at the moment:
Most promising: $.NSXMLDocument
Getting hold of the string data in various ways goes well,
function readTextFromFile(strPath) {
return $.NSString.stringWithContentsOfFile(strPath);
}
function readDataFromFile(strPath) {
return $.NSData.alloc.initWithContentsOfFile(strPath);
}
function filePath(strPath) { return Path(strPath); }
But no permutations on this theme have borne fruit:
var strPath='/Users/houthakker/Desktop/notes-2014-11-04.opml',
dataXML = readData(strPath),
strXML = readTextFile(strPath),
oXMLDoc1, oXMLDoc2;
oXMLDoc1 = $.NSXMLDocument.alloc.initWithXMLString(strXML,0,null);
oXMLDoc2 = $.NSXMLDocument.alloc.initWithData(dataXML,0,null);
(the 'function undefined' error messages suggest that those two init functions may not be exposed, though initWithRootElement() does seem to be)
Most progress: $.NSXMLParser
var oParser = $.NSXMLParser.alloc.initWithData(dataXML);
return oParser.parse; //true
But event-driven parsing seems to require some further complexities which remain opaque to me, and which may not match my simple needs (reading and converting modestly sized local OPML files).
Most familiar: Application("System Events")
In Applescript this can be done with System Events code:
set fOPML to (POSIX file "/Users/houthakker/Desktop/notes-2014-11-04.opml" as alias) as string
tell application "System Events"
tell XML file fOPML
-- now access XML Elements by name or index
end tell
but I haven't found a successful javascript idiom for initializing the XMLFile object with any permutation of a unix Path(), string, or colon-delimited mac path string.
Any thoughts or more successful experience here ?

This turns out to work for the (very slow executing) Application("System Events") route:
var app = Application("System Events"),
strPath = '~/Desktop/summarise.opml';
var oFile = app.xmlFiles[strPath],
oRoot = oFile.xmlElements[0],
oHead = oRoot.xmlElements.head,
oBody = oRoot.xmlElements.body,
lstOutlineXML = oBody.xmlElements.whose({
name: 'outline'
});
And the function for initialising an NSXMLDocument from an XML string is, according to the JSExport convention (in which the letter following each ":" is capitalized, and then the ":"s are removed) .initWithXMLStringOptionsError()
Thus, to choose a local OPML file and parse it to a simple JSON outline:
function run() {
var app = Application.currentApplication();
app.includeStandardAdditions = true;
function readTextFromFile(strPath) {
return $.NSString.stringWithContentsOfFile(strPath);
}
var strPath = (
app.chooseFile({withPrompt: "Choose OPML file:"})
).toString(), // Path → String
strXML = strPath ? readTextFromFile(strPath) : '';
if (!strXML) return;
var oXMLDoc1 = $.NSXMLDocument.alloc.initWithXMLStringOptionsError(strXML, 0, null),
oRoot = oXMLDoc1.rootElement,
oBody = ObjC.unwrap(oRoot.children)[1],
lstOutline = ObjC.unwrap(oBody.children),
lstParse, strJSON;
function parseOPML(lst) {
var lstParse=[], oNode, dctNode, lstChiln;
for (var i = 0, lng=lst.length; i<lng; i++) {
oNode = lst[i];
dctNode = {};
dctNode.txt = ObjC.unwrap(oNode.attributeForName('text').stringValue);
lstChiln = ObjC.unwrap(oNode.children);
if (lstChiln && lstChiln.length)
dctNode.chiln = parseOPML(lstChiln);
lstParse.push(dctNode);
}
return lstParse;
}
lstParse = parseOPML(lstOutline);
strJSON = JSON.stringify(lstParse,null, 2);
app.setTheClipboardTo(strJSON);
return strJSON;
}

Access to files from extension return sometimes NS_ERROR_FILE_IS_LOCKED

Our extension (Addon SDK) looking for new files in folder C:\scan and send it to server. Every second extension look for latest file creation time and defined it as latest.(compare new file creation time and file creation time 1 sec ago.)
Files put to C:\scan from scanner Brother 7050 on Windows 7.
But sometimes into console.error we see:
Exception
message: "Component returned failure code: 0x8052000e (NS_ERROR_FILE_IS_LOCKED)
[nsIFileInputStream.init]",
result: 2152857614,
name: "NS_ERROR_FILE_IS_LOCKED"
I think Brother 7050 application have no time to unlock file before our extension can start to read it.
Q: How we can read latest file in folder true way without read file lock error?
/*
adr- folder path
array2 - array for search
mode - search or not search in array2 (0-1)
*/
function getfilelist(adr,array2, mode)
{
filelist2=[];
filelist2[0]="";
filelist2[1]=0;
var file = new FileUtils.File(adr);
var enumerator = file.directoryEntries;
while (enumerator.hasMoreElements())
{
inner = enumerator.getNext().QueryInterface(Ci.nsIFile);
if (inner.isFile())
{
namearray=inner.leafName.split(".");
r=namearray[namearray.length-1];
if (r=="jpg" || r=="jpeg")
{
if (mode==0)
{
if (inner.lastModifiedTime>filelist2[1])
{
filelist2[0]=inner.leafName;
filelist2[1]=inner.lastModifiedTime;
}
}
else if (mode==1)
{
if (inner.lastModifiedTime>array2[1] && inner.isReadable()==true)
return inner.leafName;
}
}
}
}
if (mode==0)
{
return filelist2;
}
return false;
}

The reason why you see NS_ERROR_FILE_IS_LOCKED is most likely that the file is still being written and you are trying to access it too early. However, it is also possible that some other software immediately locks the file to check it, e.g. your anti-virus.
Either way, there is no way to ignore the lock. Even if you could, you might get an incomplete file as a result. What you should do is noting that exception and remembering that you should try to read that file on next run. Something along these lines:
var {Cr} = require("chrome");
var unaccessible = null;
setInterval(checknewfiles, 1000);
function checknewfiles()
{
var files = getfilelist(...);
if (unaccessible)
{
// Add any files that we failed to read before to the end of the list
files.push.apply(files, unaccessible);
unaccessible = null;
}
for (var file of files)
{
try
{
readfile(file);
}
except(e if e.result == Cr.NS_ERROR_FILE_IS_LOCKED)
{
if (!unaccessible)
unaccessible = [];
unaccessible.push(file);
}
}
}
For reference:
Components.results
Chrome authority
Conditional catch clauses
for..of loop

How do I pass a filename to an emscripten-compiled node script on the command line?

I'm having problems opening a user-specified file in a C program that has been compiled with emscripten (and is being run through node). Take this simple cat-like program:
#include <stdio.h>
#define BUFSIZE 100
int main( int argc, char *argv[] )
{
char *filename;
FILE *file;
char buf[BUFSIZE];
if (argc != 2) {
fprintf (stderr, "Usage: %s <filename>\n", argv[0]);
return 1;
}
file = fopen (argv[1], "r");
while (fgets (buf, BUFSIZE, file))
fputs (buf, stdout);
fclose (file);
return 0;
}
I compile this successfully with emscripten:
% /usr/local/src/emscripten/emcc tsrc/mycat.c
clang: warning: argument unused during compilation: '-nostdinc++'
It runs, and expects an argument:
% node a.out.js
Usage: /bin/this.program <filename>
But when I give it that argument, it barfs:
% node a.out.js somefile
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
TypeError: Function.prototype.apply: Arguments list has wrong type
at Function.APPLY_PREPARE (native)
at Pointer_stringify (/path/to/mycat/a.out.js:624:34)
at _fopen (/path/to/mycat/a.out.js:1917:14)
at Object._main (/path/to/mycat/a.out.js:2531:15)
at Object.callMain (/path/to/mycat/a.out.js:2585:25)
at doRun (/path/to/mycat/a.out.js:2624:20)
at run (/path/to/mycat/a.out.js:2647:12)
at Object.<anonymous> (/path/to/mycat/a.out.js:2663:1)
at Module._compile (module.js:441:26)
at Object..js (module.js:459:10)
Here's the compiled JavaScript that's barfing:
function Pointer_stringify(ptr, /* optional */ length) {
// Find the length, and check for UTF while doing so
var hasUtf = false;
var t;
var i = 0;
while (1) {
t = HEAPU8[(((ptr)+(i))|0)];
if (t >= 128) hasUtf = true;
else if (t == 0 && !length) break;
i++;
if (length && i == length) break;
}
if (!length) length = i;
var ret = '';
if (!hasUtf) {
var MAX_CHUNK = 1024; // split up into chunks, because .apply on a huge string can overflow the stack
var curr;
while (length > 0) {
curr = String.fromCharCode.apply(String, HEAPU8.subarray(ptr, ptr + Math.min(length, MAX_CHUNK))); // ERROR OCCURS ON THIS LINE
ret = ret ? ret + curr : curr;
ptr += MAX_CHUNK;
length -= MAX_CHUNK;
}
return ret;
}
var utf8 = new Runtime.UTF8Processor();
for (i = 0; i < length; i++) {
assert(ptr + i < TOTAL_MEMORY);
t = HEAPU8[(((ptr)+(i))|0)];
ret += utf8.processCChar(t);
}
return ret;
}
The problem line from the error message (line 624) is the one with String.fromCharCode.apply.
Versions of relevant software:
% clang --version
clang version 3.2 (tags/RELEASE_32/final)
...
% node --version
v0.6.15
% python --version
Python 2.7.1
% /usr/local/src/emscripten/emcc --version
emcc (Emscripten GCC-like replacement) 1.3.6 (commit 17da251d334ce62d633d51f874b92e19ad9dbf45)
...
Eventually I want to load files in the browser environment too... I know that is a whole different ballgame, with file preloading etc.... for now I just want to get the command line working though!
Updated to add: after looking at https://github.com/kripken/emscripten/wiki/Filesystem-Guide I tried preloading the file & get a different error:
% /usr/local/src/emscripten/emcc mycat.c --preload-file somefile
...
% node a.out.js somefile
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
ReferenceError: XMLHttpRequest is not defined
at...
Does this mean I need to run it in the browser?
Updated again to add:
% /usr/local/src/emscripten/emcc mycat.c -o mycat.html --preload-file README.md
clang: warning: argument unused during compilation: '-nostdinc++'
% open mycat.html
Browser window shows "Preparing...", console log reveals error:
XMLHttpRequest cannot load file://localhost/path/to/mycat.data. Cross origin requests are only supported for HTTP. mycat.html:1
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101 mycat.html:2816
still waiting on run dependencies: mycat.html:61
dependency: fp somefile mycat.html:61
dependency: datafile_mycat.data mycat.html:61
(end of list)
...and then the last four lines -- from 'still waiting on run dependencies: mycat.htm' onwards -- repeat every few seconds.
Do I need to be prepping these filesystem calls with FS on the javascript side, somehow?

You are almost there. The error Cross origin requests are only supported for HTTP simply means that your browser does not allow loading web pages from your hard disk directly.
Some browsers supports a command-line parameter to enable this, but I strongly suggest you simply run your web page on a simple HTTP server. This is easy to achieve using Python:
Open a command prompt in your app folder
Execute python -m SimpleHTTPServer
Point your browser to http://127.0.0.1:8000

MSScriptControl Issue on Windows Server 2008

So I'm using the MSScriptControl to run some javascript in my app and I want to be able to get some information about any errors the script may cause.
MSScriptControl.ScriptControlClass script = new MSScriptControl.ScriptControlClass();
try
{
script.Language = "JScript";
script.Timeout = 15000;
script.Eval(Code);
}
catch (Exception ex)
{
MSScriptControl.Error err = script.Error;
ret = new Exception("Error on line: " + err.Line + ", Description: " + err.Description);
}
The code works fine on my development machine, a Windows 7 box, and gives me a line number with an error. So I happily publish and push it to the production machine which always tells me the error occurred at line 0 and no description is provided.
I tried going to http://www.microsoft.com/download/en/details.aspx?id=1949 to download the latest version but installing it had no effect. I also set the property Embed Interop Types to false as well as copying my own msscript.ocx file into the Windows 2008 server's system32 directory but neither of these attempts resolved anything.
Anyone have any recommendations?

If you want to do it in all native c# without any 3rd party or "component" external dependencies use a CodeDomProvider with a tiny JScript bootstrap, like this:
private static readonly MethodInfo eval = CodeDomProvider
.CreateProvider("JScript")
.CompileAssemblyFromSource(new CompilerParameters(), "package e{class v{public static function e(e:String):Object{return eval(e);}}}")
.CompiledAssembly
.GetType("e.v")
.GetMethod("e");
private static object JsEval(string jscript)
{
try
{
return eval.Invoke(null, new[] { jscript });
}
catch (Exception ex)
{
return ex;
}
}
that creates a JsEval(string) method that you can use anywhere in your code to "eval" a string as JavaScript (well JScript)... So calling:
MessageBox.Show("" + JsEval("2 + 2")); // 4
MessageBox.Show("" + JsEval("(function(){ return 3+7; })();")); // 10
MessageBox.Show("" + JsEval("function yay(a) { return a + 1; } yay(2);")); // 3
depending on your use you may not want to instantiate these members statically. if you want to manipulate complex objects you will need create a wrapper to reflectively extract data (or you could cast as the appropriate JScript counterpart, but I've never tried this as you'd have to include the JScript assemblies).
here is an example of a wrapper class that does everything JavaScript will let you do natively, adding anymore high level functionality would probably be cumbersome enough so that you'd be better off either extracting the members into a dictionary / hash table OR alternatively serializing and deserializing on the other end
private class JsObjectWrapper : IEnumerable
{
public readonly object jsObject;
private static PropertyInfo itemAccessor = null;
private static MethodInfo getEnumerator = null;
public JsObjectWrapper(object jsObject)
{
this.jsObject = jsObject;
if (itemAccessor == null)
{
itemAccessor = jsObject.GetType().GetProperty("Item", new Type[] { typeof(string) });
}
if (getEnumerator == null)
{
getEnumerator = jsObject.GetType().GetInterface("IEnumerable").GetMethod("GetEnumerator");
}
}
public object this[string key]
{
get { return itemAccessor.GetValue(jsObject, new object[] { key }); }
set { itemAccessor.SetValue(jsObject, value, new object[] { key }); }
}
IEnumerator IEnumerable.GetEnumerator()
{
return (IEnumerator)getEnumerator.Invoke(jsObject, null);
}
}
you can see this in action by doing this:
var jsObj = JsEval("var x = { a:7, b:9 };");
var csObj = new JsObjectWrapper(jsObj);
MessageBox.Show("a: " + csObj["a"]); // a: 7
MessageBox.Show("b: " + csObj["b"]); // b: 9
csObj["yay!"] = 69;
foreach (string key in csObj)
{
MessageBox.Show("" + key + ": " + csObj[key]); // "key": "value"
}
i personally have used code similar to this to great effect at one point or another and can vouch for it's availability and runnability inside a server environment.. I hope this helps -ck

Regarding the problem you face just some thoughts:
according to the link you provided this control neither supports Windows 7 nor Windows 2008
it might be a security issue with regards to COM/UAC etc.
it might be a problem because of bitness if you compiled for AnyCPU, try using x86
Regarding possible alternatives:
Using JScript you can build an evaluator rather easily which is supported anywhere .NET 4 runs (including Windows Server 2008).
Using JInt as a JavaScript interpreter