My source code looks like this:
API/index.ts (includes 1 firebase function trigger)
Extra/index.ts which includes
exports.ml_files = require("./ai-func");
exports.authHandler = require("./user-function");
exports.fileUploadListen = require("./fileupload.listener");
// exports.graphql = require("../API/src/index");
Currently my functions source directory directory is Extra. How do I make sure that the functions defined in API also get included in the final build? If I try to use exports.graphql the build breaks.
You must move all source code into the "functions" folder for deployment. The CLI will only deploy content from that folder, and from nowhere else. It's not possible to configure it otherwise - you must have a single folder where everything lives. As such, paths given to require() that look outside the folder (as you are doing now with ..) will simply not work.
I would recommend to try this concept of separation explained here.
We use it every of our Firebase projects and we are very happy with it :)
I am new to Nodejs and I have a Nodejs project with around 80 JS files distributed in different sub-directories and some of the js files are reused in the project. I would like to obfuscate the code as this project is shipped and installed on client systems where I do not have control and anyone who has access to the client can see the code.
Currently I am using gulp node-module to build a tar file of my project artifact. I came across posts which talks about javascript obfuscation using node-modules: javascript-obfuscator and gulp-javascript-obfuscator. The examples I have seen is for a single file obfuscation but not for an entire nodejs project.
I see a snippet on javascript-obfuscator npm page about obfuscating entire directory:
javascript-obfuscator ./dist --output ./dist/obfuscated [options]
// creates a folder structure with obfuscated files under ./dist/obfuscated path
I would like to use the gulp-javascript-obfuscator node-module to obfuscate my entire project. It would be great if one of you can guide me or post a snippet of Gulp task which can do this.
On a different note: If one of my js file is importing another file with require(mycode-api) then after the obfuscation does this change to require(mycode-api-obfuscated).
Ref:
https://www.npmjs.com/package/javascript-obfuscator
https://github.com/javascript-obfuscator/gulp-javascript-obfuscator
To obfuscate entire project, try this:
gulp.src(['src/**/*.js'])
.pipe(javascriptObfuscator())
.pipe(gulp.dest('../dist'));
var gulp = require('gulp');
var pipelines = require('readable-stream').pipeline;
const javascriptObfuscator = require('gulp-javascript-obfuscator');
gulp.task('min-js', function () {
return pipelines(
gulp.src('js/*.js'),
javascriptObfuscator(),
gulp.dest('dist')
);
});
Is it possible to auto-generate the dependencies of a bazel target? It seems like there should be a way to look at the imports of the module and know which bazel dependencies are needed at least in a lot of cases common cases. This could save a lot of boilerplate code.
load("#npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = glob(include = ["**/*.ts"]),
# Is there any easy way to generate this list?
deps = [],
)
I know there are packages for Java that do this. https://github.com/johnynek/bazel-dep. I haven't been able to find anything for any other languages.
If it doesn't exist I think it could be pretty straightforward to write. Create a template file for you to work off of creating the real BUILD file. Then run typescript to pull the AST of the module. Look through the imports. The 3rd part imports will be easier since they should resolve to an npm module.
For other files that may or may not be in this library then there might be a way to query what package they live int. That could probably work. Any pointers would be very much appreciated.
Disclosure: I am one of the authors of this library.
https://github.com/evertz/bzlgen
It can generate BUILD files (or, more precisely it generates buildozer commands) for Angular (ng_module) and SCSS (scss_library and scss_binary) libs.
I've just moved this in to opensource from our internal repo. It works in a similar way to what you suggest, however it doesn't query for labels. It uses a file or directory as a starting point, parse into an AST, query the AST to fetch imports and reexports, convert the paths into labels.
Adding ts_library support is a logical next step.
It doesn't always get you a 100% working BUILD file currently, but it will get you ~80-90% of the way there, and do the boilerplate parts for you.
Another approach would be to interact with the Gazelle API, and manipulate the BUILD files directly.
A previous version of this tool generated the BUILD files from a string and it got difficult to work with when manipulating the files in other ways.
hello and today i want to know if i can have only one node_modules folder for all my node js bots, note they are not in app form just single files that use ..... = require('module'); my point is that i have easily more than 7 bots and i go back and forth improving them and maintaining the node_modules folder can be tedius, is there a way where i can have only one folder with all my modules and just do .... = require('path/to/node_modules/module');
or .... = require('path/to/node_modules/'); could someone please point me in the right direction, because when i update my code on my various bots i have to make sure the module is installed, can i have only one node_modules folder???
I can think of these different options:
You can put all your bot files in the same directory and run them all from that directory so all the modules installed in that directory can be used by all the bot files.
You can install all your modules globally.
You could make one shared directory where you installed all the modules and then explicitly point to that every time you want to load a module by referencing that path. This seems less clean because every single user of a module has to know where to load it from.
For each module you want to use in a project, create symlinks to the place where the modules are actually installed. This doesn't seem like it saves any work over just installing the modules again for each bot though.
In another file say myLibs.js you could do something like
module.exports = {
lib1: require('lib1'),
...
libn: require('libn')
}
then in the original file you would do
var myLibs=require('./myLibs.js');
and reference them by myLibs.lib1...
A little clunky, I know, but it should work.
I keep getting this error in the aws-lambda console when uploading code from a zip file. I have tried uploading other zip files and they work correctly. The .js file is named "CreateThumbnail.js" in the zip file. I believe the handler is also named properly "CreateThumbnail.handler". the node_modules subdirectory is also setup. Anyone have any idea?
{
"errorMessage": "Cannot find module 'CreateThumbnail'",
"errorType": "Error",
"stackTrace": [
"Function.Module._resolveFilename (module.js:338:15)",
"Function.Module._load (module.js:280:25)",
"Module.require (module.js:364:17)",
"require (module.js:380:17)"
]
}
The way I was able to get this to work was:
Name the file exports.js
Name the handler, within the file, exports.handler
Set the handler in the lambda config to exports.handler
Zip up only the contents of the folder, not the folder itself (as mentioned above) and rename the zip file exports.zip
Ok, I did this myself, just make sure that you make the zip such that the .js file doesn't end up inside a folder, because AWS would unzip the file you upload and tries to find a .js file by the name of handler you gave, and if its inside a folder it won't help you.
One possible problem is if you upload the lambda as a zip file created via PowerShell Compress-Archive. Compress-Archive has a bug which causes AWS to extract the files into a flat tree (no subdirectories), with backslashes in filenames:
This exact error can show up if your zipped file(s) do not have world-wide read permission. (chmod -R ugo+r).
Check the file permissions before they are zipped. This is not emphasized enough unfortunately by AWS and it caused a lot of headaches for many.
If you are using AWS Lambda Layers you need to validate if your directory structure is on the needed structure for a layer:
For example for the moment.js node.js module you need the following structure:
aws-lambda-layer.zip
│ nodejs
│ nodejs/node_modules
└ nodejs/node_modules/moment
So to create a layer zip file with the correct structure we can use the following command on the root of our project:
mkdir -p nodejs && cp -r node_modules nodejs/ && zip -r aws-lambda-layer.zip nodejs
Some library files might not have global Read so lambda will not be able to read to content and build the content.
Make sure all files in node_modules are readable before packaging:
chmod -R +r node_modules
Then zip and upload.
This is the instruction from https://docs.aws.amazon.com/lambda/latest/dg/nodejs-package.html that I have followed and it works.
To update a Node.js function with dependencies
Open a command line terminal or shell. Ensure that the Node.js version in your local environment matches the Node.js version of your function.
Create a folder for the deployment package. The following steps assume that the folder is named my-function.
Install libraries in the node_modules directory using the npm install command.
npm install the_package_that_is_missing
Create a .zip file that contains the contents of your project folder. Use the r (recursive) option to ensure that zip compresses the subfolders.
zip -r function.zip .
Upload the package using the update-function-code command.
aws lambda update-function-code --function-name my-function --zip-file fileb://function.zip
Now your function is ready to run!
I had this problem on a custom module I had built that was in the node_modules dir. Everything ran fine in testing on my Win10 machine, but when uploaded I kept getting that same "Cannot find module 'modulename'" error.
It turns out that I had a mismatch; here's the package.json line from the module that couldn't be found:
"main": "./build/modulename.js",
and here's the actual filename:
Modulename.js
Case-sensitive; Windows isn't, linux (and thus AWS) is.
This is unrelated but google brought me here, so:
AWS will give you an error:
Unable to import module '<myfile>': Error
What was really happening for me, was that was requiring an unexisting JS file. The error is a bit misleading.
I ran into this same scenario, solved it by using these specific steps to create a Layer, then hook that up to the Lambda function.
make a new empty directory:
mkdir newdir && cd newdir
install whatever npm things:
npm install --save xyz
make a directory skeleton that matches the expected Lambda structure for Node14 (there's a different structure for Node12, or various other languages; see https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html?icmpid=docs_lambda_help):
mkdir -p nodejs/node14
copy the "node_modules" directory into that newly made directory skeleton:
cp -R node_modules nodejs/node14
zip the whole thing up (name it whatever you want):
zip -r custom-drivers-node14.zip nodejs
from there, go to AWS console, Lambda, then "Layers" and create a new layer. In the dialog, upload your .zip file ("custom-drivers-node14.zip").
finally, edit your Lambda function in AWS console, and add a new Layer – the interface might change, but as of now, this is under the main screen for a single function, then scroll way down to the bottom. Follow the "Add a layer" flow, choose the Layer you made, and then try your code.
One final note, this code structure worked:
const xyz = require('xyz');
exports.handler = async (event) => {
xyz.doSomething();
}
AWS Lambda uses the name of the file and the name of the handler function, so if you defined your handler like this: exports.myHandler = function(event, context) in a file named index.js, your handler is index.myHandler.
This turned out to be a simple one for me.
I was getting, cannot create index. in my case, my main lambda file with the exports.handler in had to be called index.js
Try calling your main file CreateThumbnail.js
The tutorial tells you to include the following items in your zip file:
CreateThumbnail.js
/node_modules/gm
/node_modules/async
What it fails to consider is that there are dependencies of the two packages (gm, async) that also need to be part of the package.
So here's what you have to do:
Change directory to node_modules folder in your project folder, and run the command 'npm install gm async'. This will install gm, async and all their dependencies in this folder.
Now package the 'CreateThumbnail.js' file and the complete 'node_modules' folder into a zip file and upload it. It should work now.
So your complete package should look something like this:
CreateThumbnail.js
/node_modules/.bin
/node_modules/array-parallel
/node_modules/array-series
/node_modules/async
/node_modules/cross-spawn
/node_modules/debug
/node_modules/gm
/node_modules/isexe
/node_modules/lodash
/node_modules/lru-cache
/node_modules/ms
/node_modules/pseudomap
/node_modules/which
/node_modules/yallist
File Name:
app.js
Lambda Function in "app.js":
exports.handler = function(event, context)...
Lambda Handler on Amazon Console:
app.handler ({app}.js + exports.{handler} = app.handler)
When you unzip the folder, you should see:
app.js
node_modules