Python and the Spidermonkey Javascript engine on Linux - javascript

I have successfully installed Spidermonkey JS engine on my Linux machine ( Ubuntu ).
Basically my goal is to make it execute Ajax (js) scripts and return the result back to my Python script. I'm basically trying to build a good O.O. web scraper. But it's pretty hard for me to get all of this working.
I'm now at the point where when I type JS in my terminal I can start executing Javascript.
I've been Googling and found this little snipet on Stackoverflow :
import urllib2
import spidermonkey
js = spidermonkey.Runtime()
js_ctx = js.new_context()
script = urllib2.urlopen('http://etherhack.co.uk/hashing/whirlpool/js/whirlpool.js').read()
js_ctx.eval_script(script)
js_ctx.eval_script('var s="abc"')
js_ctx.eval_script('print(HexWhirpool(s))')
but it failed to run with the error that module Spidermonkey can not be found.
I'm a bit lost now. Anyone able to help?

I also tried easy_install python-spidermonkey with no luck, for libnspr-dev package is absent.
So, I've built package from source. Instructions from project page (Debian Stretch):
Building
Check out the Python-Spidermonkey module from the SVN repository ( I downloaded it as source archive, direct link )
Unpack, and cd to ./python-spidermonkey/trunk
CPPFLAGS="-Wno-format-security" python setup.py build (these flags for Debian)
Error jsemit.h:508:32: error: expected ‘(’ before ‘)’ token uintN decltype); means that decltype cannot be used as variable (maybe it's a macro or something else), fix it this way:
sed -e 's/decltype/dectyp/' -i.ORIG ./js/src/jsemit.h
sed -e 's/decltype/dectyp/' -i.ORIG ./js/src/jsemit.cpp
Error jsemit.cpp:6490:1: error: narrowing conversion of ‘-1’ from ‘int’ to ‘uint8 {aka unsigned char}’ inside { } [-Wnarrowing] means illegal variable conversion, recompile it manually:
cd js/src
g++ -o Linux_All_DBG.OBJ/jsemit.o -c -Wall -Wno-narrowing -Wno-format -MMD -g3 -DXP_UNIX -DSVR4 -DSYSV -D_BSD_SOURCE -DPOSIX_SOURCE -DHAVE_LOCALTIME_R -DHAVE_VA_COPY -DVA_COPY=va_copy -DPIC -fPIC -DDEBUG -DDEBUG_user -DEDITLINE -ILinux_All_DBG.OBJ jsemit.cpp
Error spidermonkey.c:1:2: error: #error Do not use this file, it is the result of a failed Pyrex compilation. - some trouble with pyrex. There is a patch. Do it this way:
wget -O - https://storage.googleapis.com/google-code-attachments/python-spidermonkey/issue-14/comment-4/cinit.patch | patch -p1 ./spidermonkey.pyx
Installation
su, and python setup.py install as root.
Running
By default, setup script installs libjs.so to /usr/local/lib/, so I did ln -s /usr/local/lib/libjs.so /usr/lib/libjs.so (but you'd better use solution from Seagal82)
Without this step, python keeps complaining about import ImportError: libjs.so: cannot open shared object file: No such file or directory
I also had an error ImportError: cannot import name Runtime after from spidermonkey import Runtime. The reason possibly was in old easy_install data in ~/.local/lib/python2.7/site-packages/spidermonkey/. After removing it, all runs smooth

Recently i got a task need to do something like Web scraping,
and for the javascript part, currently want to try using python-spidermonkey to resolve it and see if this might work for me ...
and i seem to meet situation might alike, after i think i finished install python-spidermonkey, i execute the script above, i got this error:
Traceback (most recent call last):
File "spidermonkeytest.py", line 2, in <module>
import spidermonkey
ImportError: libjs.so: cannot open shared object file: No such file or directory
then after some searching by google...i found the solution probably in the end of here:
http://hi.baidu.com/peizhongyou/item/ec1575c3f0e00e31e80f2e48
i setup these things:
$sudo vi /etc/ld.so.conf.d/libjs.so.conf
fill in this line:
/usr/local/lib/
save & exit, execute ldconfig:
$sudo ldconfig
then i can run the script provided above by #Synbitz Prowduczions
don't know if this is the answer you need, or this still helps?

You need to try libnspr4. If that doesn't work, you can always download it from Mozilla and build the code yourself.
It is not difficult to type ./config && make && make install to build the library yourself after untarring the source. If you build yourself, files will likely be in
/usr/local/{include,lib}
Also just try Googling for "YOUR_OS_NAME install nspr4".
I believe someone wrote a C/C++ header file translator for Python ctypes. Although I can't say much else because I don't use Python.
SpiderMonkey also has its own implementation of ctypes modeled after Python. So technically if you know javascript you could forego using Python altogether since you want to do some ajax with it. You will need to brush up on the NSPR or C runtime sockets to meet the requirements for your projects using only Spidermonkey.
OR a web search for Python +AJAX might turn up exactly what you need.

Related

Convert the ssd_mobilenet_v1_coco model with tensorflow_js

I want to build an app with javascript which integrates object-detection. For this, I wanna use the ssd_mobilenet_v1_coco model and use it in tensorflow.
However this line of code:
C:\Users\Jonas\AppData\Roaming\Python\Python36\Scripts\tensorflowjs_converter --input_format=tf_saved_model --output_node_names='image_tensor, detection_boxes, detection_scores, detection_classes, num_detections' \saved_model\saved_model \saved_model\web_model
does not work. It gives me file not found error, but the file is actually there unless I'm very dump and turned back into computer beginner.
OSError: SavedModel file does not exist at: \saved_model\saved_model
Also, I'm not quite sure about the output node names but this is secondary.
Thanks for support, hopefully I'm not totally dump :)
This might be because you are using an absolute path instead of a relative path.
On mac or linux, if you are in the directory that contains the downloaded unzipped model, you would run a command of that type :
tensorflowjs_converter --input_format=tf_saved_model --output_node_names='detection_boxes,detection_classes,detection_scores,num_detections' --saved_model_tags=serve ./ssd_mobilenet_v1_coco/saved_model ./ssd_mobilenet_v1_coco/web_model
From what i can see you are on Windows.
If you are running your command from the directory that contains the saved_model folder, you should run the following command :
C:\Users\Jonas\AppData\Roaming\Python\Python36\Scripts\tensorflowjs_converter --input_format=tf_saved_model --output_node_names='image_tensor, detection_boxes, detection_scores, detection_classes, num_detections' .\saved_model\saved_model .\saved_model\web_model

What is the proper way to set up a jsbindings class with cocos2d-js?

The documentation explains how to generates jsbindings, but it does not tell the proper way to integrate it into a project. What steps do I must follow? Where should I store my manually written C++ files? Where should I store the generated js and c++ files? What CMakeList.txt files should I edit?
I believe I found some way to do this.
Please confirm me I don't do anything wrong (I copied this message on the official forum).
Let's integrate the js-bindings test samples in a cocos2d-js project.
First generate the tests : cd tools/bindings-generator/test && ./test.sh && cd ../../.. (that may need some configuration). At the moment it fails on Linux because of missing headers but I submited a merge request.
Copy files into the project
cp -R tools/bindings-generator/test/simple_test/ frameworks/runtime-src/Classes
cp -R tools/bindings-generator/test/simple_test_bindings/ frameworks/runtime-src/Classes
Update CMakeLists.txt and frameworks/runtime-src/proj.android/jni/Android.mk and add the added files autogentestbindings.cpp and simple_class.cpp to the target lists.
Register the jsb functions in the runtime sources in frameworks/runtime-src/Classes/AppDelegate.cpp by adding sc->addRegisterCallback(register_all_autogentestbindings); in AppDelegate::applicationDidFinishLaunching
Then classes defined in simple_class.h are available in Javascript. The following JS code should display 1337.
console.log((new SimpleNativeClass(1337)).getSomeField())

How to get Hello Word compiling from Swift to JavaScript using Emscripten

Given the simplest of swift files:
println("lol")
It's trivial to run this on the command line with xcrun swift -i lol.swift or compile to an executable with xcrun swift lol.swift -o lol, but how about a simple proof of concept for emscripten?
I haven't used emscripten before, but got a hello world example using C++ working from http://kripken.github.io/emscripten-site/docs/getting_started/Tutorial.html, and wanted to compile my Swift code too.
I tried
xcrun swift lol.swift -emit-bc -o lol.bc
emcc lol.bc
But get
Value: %1 = call { i8*, i64, i64 } #_TFSS37_convertFromBuiltinUTF16StringLiteralfMSSFTBp17numberOfCodeUnitsBw_SS(i8* bitcast ([4 x i16]* #0 to i8*), i64 3)
LLVM ERROR: Unrecognized struct value
Traceback (most recent call last):
File "/Users/glen/Downloads/emsdk_portable/emscripten/1.16.0/emcc", line 1540, in <module>
shared.Building.llvm_opt(final, link_opts)
File "/Users/glen/Downloads/emsdk_portable/emscripten/1.16.0/tools/shared.py", line 1267, in llvm_opt
assert os.path.exists(target), 'Failed to run llvm optimizations: ' + output
AssertionError: Failed to run llvm optimizations:
Thoughts?
The problem is LLVM can't find a types/symbols, used in that call, during the linking process. These symbols are most likely specific to the swift framework. If you run emcc with the -v option you can get more debug information. You might consider also providing --llvm-opts hinting where that information can be found.
I ran xcrun swift -v test.swift to see what command is actually executed.
Swift version 1.0 (swift-600.0.34.4.5)
Target: x86_64-apple-darwin13.2.0
/Applications/Xcode6-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c -primary-file test.swift -enable-objc-attr-requires-objc-module -target x86_64-apple-darwin13.2.0 -module-name test -color-diagnostics -o /var/folders/69/l9w0zkqn38s1td4_gm5c__km0000gn/T/test-d800d3.o
/usr/bin/ld /var/folders/69/l9w0zkqn38s1td4_gm5c__km0000gn/T/test-d800d3.o -force_load /Applications/Xcode6-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a -lSystem -arch x86_64 -L /Applications/Xcode6-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx -rpath /Applications/Xcode6-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx -macosx_version_min 10.9.0 -no_objc_category_merging -o test
You might consider exploring how to apply these linking options to emscripten to get what you want. There will not be any documentation on this, because I don't think they intended swift to be used like this.
According to this GitHub issue you filed on the Emscripten repo, what you are trying to do isn't possible:
Unfortunately it is not possible to link .dylib files to Emscripten. The reason for that is that the .dylibs already contain native machine code for x86/x64, and Emscripten cannot "go back" and get that to LLVM IR form again.
-- juj (GitHub user), on 2014-06-14

WIKI: How to use Lime (how to use closure-compiler with 3rd party (closure) libraries)

The following post inspired me to have a look at limeJS, as a side project I'm working on and off an a Yatzee game (mostly off) and thought that might be a nice library to use.
As a beginner in google-closure I had some difficulties running uncompiled code and getting code compiled, mostly due to not knowing what the options and commands are and because of dependencies.
For other beginners with google-closuse I have written this tutorial.
Note that jQuery can be used by your closure compiled code but you need the jQuery externs file. You can't compile jQuery into your code, closure library has a dressed down dojo that can be found in third_party. When using that you can compile your code to one file.
What you need:
Python
Git client
Closure compiler for compiling code (minifying and merging all files into one)
Closure library like jQuery and jQuery ui but can be compiled along with your code
Python (I use 2.7)
LimeJS As a 3rd party library (use git client to get it, the url:https://github.com/digitalfruit/limejs.git)
JAVA JRE
Directory structure
I'm using Windows and have my projects in D:\projects, if you have your projects somewhere else you have to replace D:\projects references to your own.
In D:\projects I make a directory called libraries in that directory I copy the goog and third_party directories from closure library (goog is under the closure directory) since you'll use them for many projects I keep it at projects root, you can include a copy of it in every project you create but in this case I'll share the library among all projects.
Now I copy the contents of the src directory from limeJS ([lime clone dir]\lime\src) to D:\projects\libraries\lime (the one containing the sub directories called animation, audio ...).
Next I'll use an existing project from the limeJS library, copy the [lime clone dir]\lime\demos\roundball to D:\projects\roundball
At this time your directory structure should look like this:
D:
projects
libraries
goog
lime
animation
audio
css
...
third_party
closure
...
roundball
assets
... other roundball stuff
cacldeps.py
When you open D:\projects\roundball\rb.html and check out the console (press F12 in Chrome or in FireFox preferably having firebug plugin installed) you'll see an error: "ReferenceError: goog is not defined"
Open up D:\projects\roundball\rb.html and change:
<script type="text/javascript" src="../../../closure/closure/goog/base.js"></script>
to
<script type="text/javascript" src="../libraries/goog/base.js"></script>
Now when you open up rb.html again you get a different error: "goog.require could not find: lime.Director"
This is because closure uses deps.js to find dependencies and since lime is not in there it can't find it. Lucky for us there is a tool called calcdeps.py in the bin directory of the closure library that can create this file for us. It'll look in your code and and use goog.require to figure out what other files are needed. If your file structure is good than this tool will find it.
It will expect that Director is in a file called director.js in a directory called lime (and it is). That js file should have a goog.provide statement providing lime.Director.
You can add directories for calcdeps.py to look in with the --path parameter.
In D:\projects\roundball I'll create a makedeps.bat with the following content:
set calc="D:\software\closure compiler\library\closure\bin\calcdeps.py"
c:\Python27\python.exe %calc% ^
--path D:\projects\roundball\ ^
--path D:\projects\libraries\ ^
--input D:\projects\roundball\rb.js ^
--output_mode deps ^
--output_file D:\projects\libraries\goog\deps.js
pause
Uncompiled code uses /goog/deps.js to load dependencies. caclcdeps.py will check your code starting with rb.js (as this is the starting point of the code) and add entries to deps.js according to what your project uses.
Once again, cacldeps.py can be found in the bin directory of closure library.
Note that when you have another project; let's say D:\projects\project2 then you have to create a makedeps.bat in that project directory that re creates the deps.js for you before you can run the uncompiled code. This because multiple projects share one google closure library so when you switch projects make sure you'll build the deps.js first before running your uncompiled code.
Resolving missing dependency
Opening the rt.html we still get an error but a different one: Error: "Undefined nameToPath for lime.css"
Looking in goog/deps.js we can find that lime.css is needed by lime but doesn't have an entry in deps.js. Why is this? Looking in D:\projects\libraries\lime there is no css directory or css.js file. But in the directory where you cloned the git lime project there is a directory called css.
Copy the css directory to D:\projects\libraries\lime and run makedeps.bat again, now when you open rt.html it will run.
The whole thing is uncompiled and you can set breakpoints to step into 3rd party code.
Compile your code
In production you would want to compile the code to one file. To compile the code I created a compile.bat in D:\projects\roundball with the following content:
set calc="D:\software\closure compiler\library\closure\bin\calcdeps.py"
c:\Python27\python.exe %calc% ^
--path D:\projects\roundball\ ^
--path D:\projects\libraries\ ^
--input D:\projects\roundball\rb.js ^
--compiler_jar "D:\software\closure compiler\compiler.jar" ^
--output_mode compiled ^
--compiler_flags="--compilation_level=ADVANCED_OPTIMIZATIONS" ^
--compiler_flags="--formatting=PRETTY_PRINT" ^
--output_file D:\projects\roundball\compiled\roundball.js
pause
;Need this flag for production compile:
;--compiler_flags="--define goog.DEBUG=false" ^
;Remove the following flag from production compile:
;--compiler_flags="--formatting=PRETTY_PRINT" ^
;lime libraries is giving me errors with stricter compilation
; so had to remove this flag (have to fix the errors or no type checking for your code)
;--compiler_flags="--warning_level=VERBOSE" ^
The file compiler.jar can be found where you downloaded (and extracted) closure compiler
Now when you run the compile.bat and open D:\projects\roundball\compiled\roundball.html you'll see in the developer tools of your browser that only roundball.js is loaded.
The answer is in the question because it's a howto article that could help someone (like me in the future when I forgot how to do it).

IO Error in d3py example code -- no such file or directory: 'static/d3.js'

I am trying to run through the d3py example code available in the d3py readme file (see link below), and am receiving the following error:
https://github.com/mikedewar/d3py/blob/master/README.md
(python 2.7.3 on 32-bit Windows)
>>> # instantiate the figure object
>>> fig = d3py.Figure(df, name="basic_example", width=300, height=300)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build\bdist.win32\egg\d3py\d3py.py", line 97, in __init__
IOError: [Errno 2] No such file or directory: 'static/d3.js'
>>>
I installed the d3py package by using the following command in PowerShell:
easy_install https://github.com/mikedewar/d3py/tarball/master
and, importing the package within a python session (i.e. import d3py) does not result in any errors. What can I do to make the example code work?
So d3py is a really raw project, and to be honest our setup.py sucks quite hard. This means you either need to make a pull request with some fixes to setup.py or you need to make a folder called static in your project folder and make sure that d3.js is inside the folder.
On the assumption that the second option is the one you'll use, just to make it clear:
In a terminal, change to the directory that holds your project, like
cd ~/Desktop/my/awesome/project
Make a folder called static:
mkdir static
then move into that directory and get the d3.js file from the repo:
cd static
curl -O https://raw.github.com/mikedewar/d3py/static/d3.js
Apologies for this terrible hack! d3 based plotting from inside python is definitely going to happen, but maybe not with this project!
Having said that, d3py will work as advertised once you get it to run! A really popular use case of d3py, which we hadn't quite realised would be popular, is that people use it to quickly mock up some javascript, using the output of d3py as a starting point for fancier visualisations.

Categories

Resources