Making my own Module vs Using em++ generated module - javascript

Is making my own module.js (fetch parse and create my own Webassembly.Module) a stupid idea? Compared to just using the em++ generated one?
Compiling a program using embind in emscripten yields both my module.wasm file and a module.js. However the module.js file is 1.5MB, and I dont seem to have control over Memory management and such.
I'm currently used a custom compiled c++ program that uses OpenCV, and I keep running out of memeory, I cant allocate the memory properly if I dont create my own Webassembly.Module.
My Solution:
I read through settings.js (which lies in the same folder as your em++ executable).
There I learnt that the only flag i needed to set is -s ALLOW_MEMORY_GROWTH=1 while compiling.
This allowed me to to decide the TOTAL_MEMORY limit of the module while creating in in javascript.

Maybe you want to use emcc/em++ compiler options.
As you mentioned in the comment, you need -s ALLOW_MEMORY_GROWTH=1 but it does not do what you may think, and you are not supposed to set TOTAL_MEMORY in runtime neither.
You should set TOTAL_MEMORY as a compiler flag. For example, -s TOTAL_MEMORY=100MB. It is a good idea to keep -s ALLOW_MEMORY_GROWTH=1 as well.
Documentations doesn't help you at all, I highly recommend reading setting.js from the source code directly to learn about compiler flags.

I just tested size difference between c++ module that uses opencv and one that does not, difference was 177KB to 1.5MB.
This may suggest that yes, it is dumb to attempt to write your own Module object.
But I am not 100% sure.

Related

Find dead code in large Javascript Application [duplicate]

In order to refactor a client-side project, i'm looking for a safe way to find (and delete) unused code.
What tools do you use to find unused/dead code in large react projects? Our product has been in development for some years, and it is getting very hard to manually detect code that is no longer in use. We do however try to delete as much unused code as possible.
Suggestions for general strategies/techniques (other than specific tools) are also appreciated.
Thank you
Solution:
For node projects, run the following command in your project root:
npx unimported
If you're using flow type annotations, you need to add the --flow flag:
npx unimported --flow
Source & docs: https://github.com/smeijer/unimported
Outcome:
Background
Just like the other answers, I've tried a lot of different libraries but never had real success.
I needed to find entire files that aren't being used. Not just functions or variables. For that, I already have my linter.
I've tried deadfile, unrequired, trucker, but all without success.
After searching for over a year, there was one thing left to do. Write something myself.
unimported starts at your entry point, and follows all your import/require statements. All code files that exist in your source folder, that aren't imported, are being reported.
Note, at this moment it only scans for source files. Not for images or other assets. As those are often "imported" in other ways (through tags or via css).
Also, it will have false positives. For example; sometimes we write scripts that are meant to simplify our development process, such as build steps. Those aren't directly imported.
Also, sometimes we install peer dependencies and our code doesn't directly import those. Those will be reported.
But for me, unimported is already very useful. I've removed a dozen of files from my projects. So it's definitely worth a shot.
If you have any troubles with it, please let me know. Trough github issues, or contact me on twitter: https://twitter.com/meijer_s
Solution for Webpack: UnusedWebpackPlugin
I work on a big front-end React project (1100+ js files) and stumbled upon the same problem: how to find out which files are unused anymore?
I've tested the next tools so far:
findead
deadfile
unrequired
None of them really worked. One of the reason is that we use "not standard" imports. In additional to the regular relative paths in our imports we also use paths resolved by the webpack resolve feature which basically allows us to use neat import 'pages/something' rather than cumbersome import '../../../pages/something'.
UnusedWebpackPlugin
So here is the solution I've finally come across thanks to Liam O'Boyle (elyobo) #GitHub:
https://github.com/MatthieuLemoine/unused-webpack-plugin
It's a webpack plugin so it's gonna work only if your bundler is webpack.
I personaly find it good that you don't need to run it separately but instead it's built into your building process throwing warnings when something is not ok.
Our research topic: https://github.com/spencermountain/unrequired/issues/6
Libraries such as unrequired and deadcode only support legacy code.
In order to find the unused assets, to remove manually, you can use deadfile
library:https://m-izadmehr.github.io/deadfile/
Out of box support for ES5, ES6, React, Vue, ESM, CommonJs.
It supports import/require and even dynamic import.
It can simply find unused files, in any JS project.
Without any config, it supports ES6, React, JSX, and Vue files:
First of all, very good question, in large project coders usually try many lines of code test and at the end of result, hard to find the unused code.
There is two possible that must be work for you - i usually do whenever i need to remove and reduce the unused code into my project.
1st way WebStorm IDE:
If you're using the web-storm IDE for JS development or React JS / React Native or Vue js etc it's tell us and indicate us alote of mention with different color or red warning as unused code inside the editor
but it's not works in your particular scenario there is another way to remove the unused code .
2nd Way unrequired Library: (JSX is not supported)
The second way to remove the unused code inside the project is unrequired library you can visit here : unrequired github
another library called depcheck under NPM & github here
Just follow their appropriate method - how to use them you will fix this unused issue easily
Hopefully that helps you
I think the easiest solution for a create-react-app bootstrapped application is to use ESLint. Tried using various webpack plugins, but ran into out of memory issues with each plugin.
Use the no-unused-modules which is now a part of eslint-plugin-import.
After setting up eslint, installing eslint-plugin-import, add the following to the rules:
"rules: {
...otherRules,
"import/no-unused-modules": [1, {"unusedExports": true}]
}
My approach is an intensive use of ESlint and make it run both on IDE ad before every push.
It points out unused variables, methods, imports and so on.
Webpack (which has too nice plugins for dead code detection) take care about avoiding to bundle unimported code.
findead
With findead you can find all unused components in your project. Just install and run:
Install
npm i -g findead
Usage
findead /path/to/search
This question recalls me that react by default removes the deadcode from the src when you run the build command.
Notes:
you need to run build command only when you want to ship your app to production.

Compile Dlib for webassembly

I am trying to build dlib for webassembly using emscripten but I am not sure how to do so.
Currently, dlib generates executables and not bytecode which is needed for emscripten. Is there some way to get around this issue?
Currently this is what i am doing (from within the dlib-19.4 folder.
cd examples
mkdir build
cd build
cmake ..
emmake make
for the next step, I need to input a bytecode file however, dlib seems to just generate executables which the emcc command will not accept.
From what I can read from Dlib's documentation is that it has a lot of features that emscripten does not provide including threading and so on. So you will need to exclude functionality thats not supported.
Also referring to How to Compile section, it mentions how to use Dlib without Cmake.
It has a simple gcc example for SVM.
You should probably use this approach ie include the appropriate dlib headers in your project, write your functions and expose them for emscipten.
Then after building the js file you should be able to call the "exposed" functions from JS.

How can I convert a multi-file node.js app to a single file?

If I have a node.js application that is filled with many require statements, how can I compile this into a single .js file? I'd have to manually resolve the require statements and ensure that the classes are loaded in the correct order. Is there some tool that does this?
Let me clarify.
The code that is being run on node.js is not node specific. The only thing I'm doing that doesn't have a direct browser equivalent is using require, which is why I'm asking. It is not using any of the node libraries.
You can use webpack with target: 'node', it will inline all required modules and export everything as a single, standalone, one file, nodejs module
https://webpack.js.org/configuration/target/#root
2021 edit: There are now other solutions you could investigate, examples.
Namely:
https://esbuild.github.io
https://github.com/huozhi/bunchee
Try below:
npm i -g #vercel/ncc
ncc build app.ts -o dist
see detail here https://stackoverflow.com/a/65317389/1979406
If you want to send common code to the browser I would personally recommend something like brequire or requireJS which can "compile" your nodeJS source into asynchronously loading code whilst maintaining the order.
For an actual compiler into a single file you might get away with one for requireJS but I would not trust it with large projects with high complexity and edge-cases.
It shouldn't be too hard to write a file like package.json that npm uses to state in which order the files should occur in your packaging. This way it's your responsibility to make sure everything is compacted in the correct order, you can then write a simplistic node application to reads your package.json file and uses file IO to create your compiled script.
Automatically generating the order in which files should be packaged requires building up a dependency tree and doing lots of file parsing. It should be possible but it will probably crash on circular dependencies. I don't know of any libraries out there to do this for you.
Do NOT use requireJS if you value your sanity. I've seen it used in a largish project and it was an absolute disaster ... maybe the worst technical choice made at that company. RequireJS is designed to run in-browser and to asynchronously and recursively load JS dependencies. That is a TERRIBLE idea. Browsers suck at loading lots and lots of little files over the network; every single doc on web performance will tell you this. So you'll very very quickly end up needing a solution to smash your JS files together ... at which point, what's the point of having an in-browser dependency resolution mechanism? And even though your production site will be smashed into a single JS file, with requireJS, your code must constantly assume that any dependency might or might not be loaded yet; in a complex project, this leads to thousands of async load barriers wrapping every interaction point between modules. At my last company, we had some places where the closure stack was 12+ levels deep. All that "if loaded yet" logic makes your code more complex and harder to work with. It also bloats the code increasing the number of bytes sent to the client. Plus, the client has to load the requireJS library itself, which burns another 14.4k. The size alone should tell you something about the level of feature creep in the requireJS project. For comparison, the entire underscore.js toolkit is only 4k.
What you want is a compile-time step for smashing JS together, not a heavyweight framework that will run in the browser....
You should check out https://github.com/substack/node-browserify
Browserify does exactly what you are asking for .... combines multiple NPM modules into a single JS file for distribution to the browser. The consolidated code is functionally identical to the original code, and the overhead is low (approx 4k + 140 bytes per additional file, including the "require('file')" line). If you are picky, you can cut out most of that 4k, which provides wrappers to emulate common node.js globals in the browser (eg "process.nextTick()").

Compile JavaScript to Native Code with V8

Is it really possible, with Google's V8 Engine, to compile JavaScript into Native Code, save it as a binary file, and execute it whenever I want through my software envorinment, on any machine?
You can use the V8 snapshot functionality to precompile the code. This still means that you have to have a full version of V8 running to load the snapshot (i.e., you don't get stand-alone native code, it needs to run inside the V8 VM), so all you save is the compilation time.
Also, the quality of snapshot code isn't necessarily as good as JIT'ed code because JIT code can use, e.g., SSE2/SSE3 if it's available, which snapshots can't assume.
As far as I know, V8 is purely a just-in-time compiler, and does not have an ahead-of-time option.
As discussed at the articles I linked, JITs allow better, more flexible optimizations.
Instead, it might be possible to use a .NET JavaScript/JScript compiler to create a .NET exe, then convert the .NET exe to a native .exe using the Mono ahead-of-time compiler.
The closest you might get to acheiving your goal is to create a self-executing Javascript bytecode wrapper.
A project that does this is pkg
It somehow creates a self-contained binary executable from Javascript, including module dependencies and asset files and produces a self-contained executable.
Installation and use is easy:
$ npm install -g pkg
$ pkg index.js -o my-program
$ ./my-program
My understanding is that this binary contains nodejs bytecode. It also appears that you can cross-compile.
Note: I've tried ncc and nexe also, but I haven't found them to be as useful. ncc just creates a self-contained Javascript file and nexe encountered a Python error when I tried to use it.

How can I compile CoffeeScript from .NET?

I want to write an HttpHandler that compiles CoffeeScript code on-the-fly and sends the resulting JavaScript code. I have tried MS [JScript][1] and IronJS without success. I don't want to use [Rhino][2] because the Java dependency would make it too difficult to distribute.
How can CoffeeScript be compiled from .NET?
CoffeeScript-dotnet
Command line tool for compiling CoffeeScript. Includes a file system watcher to automatically recompile CoffeeScripts when they change. Roughly equivalent to the coffee-script node package for linux / mac.
CoffeeSharp
Includes a command line tool similar to CoffeeScript-dotnet as well as a http handler that compiles CoffeeScripts when requested from an asp.net site.
SassAndCoffeeScript
Library for Asp.net mvc that compiles sass and coffeescript files on request. Also supports minification and combination.
Manually Compile With IronJS
IronJS is a .NET javascript interpreter that can successfully load the CoffeeScript compiler and compile CoffeeScript.
Manually Compile With Node.js
Get the node binaries and add the bin directory to your path. Write a node.js script to load the CoffeeScript compiler and your CoffeeScript files and save the compiled javascript.
CoffeeScript is now fully supported by Chirpy:
http://chirpy.codeplex.com/
You specifically said that you wanted to write a runtime compiler, so this may not be exactly what you are looking for, but if the main point is to have a way to generate the javascript result, the Mindscape Web Workbench is interesting. It is a free extension for Visual Studio.NET 2010 and available in the Extension Manager. It gives Intellisense, syntax highlighting and compiles to JS as you write. I am just getting started using it but looks promising. Scott Hanselman talks about it here. It also supports LESS and Sass.
I've managed to compile CoffeeScript from .NET using IKVM, jcoffeescript and Rhino. It was straightforward, except that the JCoffeeScriptCompiler constructor overload without parameters didn't work. It ran OK with a java.util.Collections.EMPTY_LIST as parameter.
This is how I did it:
Download IKVM, jcoffeescript and Rhino.
Run ikvmc against js.jar, creating js.dll.
Run ikvmc against the jcoffeescript jar.
Add a reference to the jcoffeescript dll in Visual Studio. More references may be needed, but you will be warned about those.
Run new org.jcoffeescript.JCoffeeScriptCompiler(java.util.Collections.EMPTY_LIST).compile() in your code.
The next step would be to create a build task and/or an HTTP handler.
Check out the new coffeescript-dotnet project, which uses the Jurassic JavaScript implementation.
Since the CoffeeScript compiler now runs on Internet Explorer, after a couple of recent tweaks, it should be good to go within other MS-flavors of JavaScript as well. Try including extras/coffee-script.js from the latest version, and you should be good to go with CoffeeScript.compile(code).
I tried running the bundled extras/coffee-script.js through Windows Based Script Host (or just wscript) and it didn't report any issues. I then added this line:
WScript.Echo(CoffeeScript.compile('a: 1'));
at the end of the file and run it through wscript again and it printed the resulting JavaScript correctly.
Are you using COM objects? Can you share some more of the code responsible for initialising the MScript object reference?
CoffeeScript in Visual Studio 2010
It's Chirpy's fork (chirpy is a tool for mashing, minifing, and validating javascript, stylesheet, and dotless files)
"OK, I think I got it working on my fork, based mostly on other peoples' work. Check it out:
http://chirpy.codeplex.com/SourceControl/network/Forks/Domenic/CoffeeScriptFixes"
from http://chirpy.codeplex.com/workitem/48
I don't have a direct answer, (I hope you find one), but maybe take a look at the following to see how it might be done.
Jint - JavaScript interpreter for .NET
Using IKVM to compile Rhino would get rid of the Java runtime requirement.
jcoffeescript. I haven't looked at jcoffeescript, but I think it depends on JRuby and Rhino. You could possibly IKVM.NET this as well.
IronJS now supports CoffeeScript and is generally faster than the other .NET JS engines:
I have a blog post about wiring the two together here:
http://otac0n.com/blog/2011/06/29/CoffeeDemo-A-Simple-Demo-Of-IronJS-Using-CoffeeScript.aspx
My main editor is VS 2010 and I love the WorkBench extension. it's nice it auto compiles to js everytime you hit save on your .coffee file, also introduces you to SASS which I had read about but never got around.
They offer a pay version to that will autmaically shrink/minify your js and css files as well, since your.coffee and .scss are your source files anyway.
I'd encourage all VS users to go ahead and install this especially if you run VS 2010.
The only knock, and someone please correct me or enlighten me, is that with .coffee syntax it's not highlighted the way say html, js, c# code is. it might be because I am using a color scheme from http://studiostyl.es/ and for the record http://studiostyl.es/schemes/coffee- just shares the name coffee no special syntax highlight support for coffeescript that I am aware of. but no reason not to start using the workbench addin today!
Okay workbench website claims: syntax highlighting so again maybe it's the studiostyle.es i chose.
I know this is old but I came here to answer a very similar question: How do I get my CoffeeScript to compile using Visual Studio 2012 Express? Note that the free Express version does not allow any extensions so I could not continue to use the Mindscape Workbench extension that had served me well for quite some time.
It turns out to be very easy. Just use NuGet to install the Jurassic-Coffee package and off you go.
One advantage of using this package over mindscape workbench is that you can reference your coffee directly from the script tags in the html. It minifies and caches the compiled JS so you only do work if the requested coffee file has changed.
<head>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script src="home.coffee"></script>
</head>
The mindscape workbench allows you to bundle together different coffescript files which is very handy for modularising your coffeescript. You can also do this using Jurassic Coffee by utilising the #= require statement to include other coffee module files, for example:
#= require Classes\GridWrapper.coffee
class UsersGrid
constructor:->
#grid = new GridWrapper()
I think having the #= require staement in the coffee file is actually cleaner and clearer than the mindscape workbench approach, which kind of hides all this behind their interface so you forget easily what dependencies you have.
Note
There is one potential gotcha. The Nuget installer will put in an httphandler entry into your web.config that may not be compatible with IIS Express integrated managed pipeline mode.
You might therefore see the following error:
An ASP.NET setting has been detected that does not apply in Integrated
managed pipeline mode.
To fix this just remove the handler shown below.
<system.web>
//other stuff
<httpHandlers>
<add type="JurassicCoffee.Web.JurassicCoffeeHttpHandler,JurassicCoffee" validate="false" path="*.coffee" verb="*" />
</httpHandlers>
</system.web>
You could simply write a port of it to C#. I have ported Jison to C# (which is the underlying project that makes CoffeeScript run). I would think it may be a bit different, but both Jison parsers work the same.
I have not pull requested it back yet to Jison's main architecture, but will be doing so soon.
https://github.com/robertleeplummerjr
Instead of shelling out to CScript you could shell out to Node.js (here are self-contained Windows binaries)
I've tried to compile the extras/coffee-script.js file, unmodified, to jsc, the JScript.NET compiler for .NET, and I got many errors. Here are the noteworthy ones:
'require' is a new reserved word and should not be used as an identifier
'ensure' is a new reserved word and should not be used as an identifier
Objects of type 'Global Object' do not have such a member
Other errors were caused by the above said errors.
You might also want to check out jurassic-coffee, it is also a coffee-script compiler running the original compiler in jurassic.
It features sprocket style "#= require file.coffee" or "#= require file.js" wich can be used to keep .coffee files modular and combined right before compilation as well as embedding .js files.
It sports a HttpHandler with file watchers for .js and .coffee files that keeps track of what .coffee files needs to be re-compiled and pass through to the compiled *.js files for the rest.
jurassic-coffee is also available as a Nuget package
https://github.com/creamdog/JurassicCoffee
I've done an HttpHandler that uses Windows Script Host behind the scenes: https://github.com/duncansmart/LessCoffee and works great (it also compiles *.less files).
It's on NuGet: http://nuget.org/List/Packages/LessCoffee
It's based on this simple wrapper: https://github.com/duncansmart/coffeescript-windows
I wrote an inteructive shell using v8.
https://github.com/mattn/coffee-script-v8
This work as single executable file. (Don't use external files)
It can't use require(). But enough to learn coffeescript.

Categories

Resources