I think, because I'm so new to Node, I'm not quite understanding how to use an NPM package with my React project running on Node.
Just a quick explanation:
I have a React component that uploads a zip file through Node server scripts. This part is working great, but now the next step I'm working on is getting node to unzip it after upload into a temp folder..
I'm using "axios" to post, and "multer" to save to file system and "adm-zip" for the unzip portion.
Here's a quick look at my on submit post function within my React Component:
onSubmit = (e) => {
e.preventDefault();
const { description, selectedFile } = this.state;
let formData = new FormData();
formData.append('description', description);
formData.append('selectedFile', selectedFile);
console.log('form data ',formData)
let that = this;
axios.post('/', formData)
.then((result, req) => {
this.setState({imagePath: result.data.path})
this.setState({fileName: result.data.filename})
this.setState({fileUploadStatus:"Upload Successful.."})
Unzip.extract(result.data.filename);
})
.catch(function (error) {
console.log(error);
//alert('File upload failed. Please ensure you are uploading a .zip file only')
that.setState({fileUploadStatus:"Upload failed.. Please ensure you are uploading a .zip file only"})
})
}
This is what I've put on the top of the React Component :
import React, { Component } from 'react';
import axios from 'axios';
import Unzip from './unzip';
This is my "unzip.js" file:
var AdmZip = require('adm-zip');
// reading archives
module.exports = (function() {
var extract = function extract(fileName, cb){
zip = new AdmZip("../uploads/"+fileName);
var zipEntries = zip.getEntries(); // an array of ZipEntry records
zipEntries.forEach(function(zipEntry) {
console.log(zipEntry.toString()); // outputs zip entries information
if (zipEntry.entryName == "my_file.txt") {
console.log(zipEntry.getData().toString('utf8'));
}
});
// extracts everything
zip.extractAllTo(/*target path*/"/home/me/zipcontent/", /*overwrite*/true);
};
});
So the part I'm struggling with is understanding how to write a node script javascript file) with functions/properties (getter, setter, etc) that can be accessed from my React component..
I tried something like this:
var AdmZip = require('adm-zip');
// reading archives
var extract = function(thePath) {
zip = new AdmZip("../uploads/"+thePath);
var zipEntries = zip.getEntries(); // an array of ZipEntry records
zipEntries.forEach(function(zipEntry) {
console.log(zipEntry.toString()); // outputs zip entries information
if (zipEntry.entryName == "my_file.txt") {
console.log(zipEntry.getData().toString('utf8'));
}
});
zip.extractAllTo(/*target path*/"/home/me/zipcontent/", /*overwrite*/true);
// extracts everything
};
But I got errors saying the function did not exist.. so I realized that I might have to set it up as a Node Module with an export function, but I couldn't figure out how to get that to work either..
Can someone explain to me, the proper way to approach this type of external script (script that runs outside of React and returns or executes some kind of action)?
As I mentioned in some of my other recent questions, I'm a beginner with Node so please go easy on me :)
Thanks!
React is a frontend package. You can use tools like unzip on the backend side. for example with ExpressJS.
Related
I am trying to set up a simple serverless function on Netlify just to test out usage of environment variables. I have defined the following two environment variables in Netlify for my site:
Variable Name
Value
ALPHABET_SEPARATION
2
CHARS_BETWEEN
3
I have also updated my functions directory as follows:
Functions directory: myfunctions
I am using continuous deployment from github. As I do not know the use of npm at present and finding it convenient to directly test the production deploy, I have defined a subdirectory called myfunctions inside my root directory and have placed my javascript file containing the "serverless" function inside it on my local machine. I have built in logic so that the "serverless" function gets called only when a "netlify" flag is set, otherwise, an alternate function gets executed client-side. Basically it works as follows:
const deploy = "netlify" //Possible valid values are "local" and "netlify"
async function postRandomString() {
const stringToUpdate = "THISISATESTSTRING"
var stringToPost = "DUMMYINITIALVALUE";
if (deploy === "local") {
stringToPost = updateString(stringToUpdate); //updateString is a function defined elsewhere and executes client-side;
}
else if (deploy === "netlify") {
const config = {
method: 'GET',
headers: {
'Accept': 'application/json',
}
};
const res = await fetch(`myfunctions/serverUpdateString?input=${stringToUpdate}`, config);
const data = await res.json();
stringToPost = data.retVal;
console.log(data.retVal);
}
else {
stringToPost = "##ERROR##";
}
postString(stringToPost); //postString is a function defined elsewhere and executes client-side;
}
The serverless function file serverUpdateString.js is coded as follows (it basically sets a character at a certain position (determined by CHARS_BETWEEN) in the string to an alphabetical character which is a certain number (determined by ALPHABET_SEPARATION) of places in the alphabet after the first character of the string (don't ask why - the point is that it never even receives/handles the request):
exports.handler = async function (event) {
const { CHARS_BETWEEN, ALPHABET_SEPARATION } = process.env;
const charsBetween = CHARS_BETWEEN;
const alphabetSeparation = ALPHABET_SEPARATION;
const initString = event.queryStringParameters.input;
const rootUnicode = initString.charCodeAt(0);
const finalUnicode = "A".charCodeAt(0) + (rootUnicode - "A".charCodeAt(0) + alphabetSeparation) % 26;
const finalChar = String.fromCharCode(finalUnicode);
const stringArray = initString.split("");
stringArray[charsBetween + 1] = finalChar;
const stringToReturn = stringArray.join("");
const response = {
statusCode: 200,
retVal: stringToReturn,
}
return JSON.stringify(response);
}
When I run it, I get a 404 error for the GET request:
In the above image, script.js:43 is the line const res = await fetch(myfunctions/serverUpdateString?input=ATESTSTRIN, config); in the calling file, as shown in the first code block above.
What am I doing incorrectly? Surely Netlify should be able to pick up the serverless function file given that I have specified the folder alright and have placed it at the right place in the directory structure? I have given the whole code for completeness but the problem seems quite elementary. Look forward to your help, thanks.
I got assistance from Netlify forums. Basically the following changes needed to be made:
The fetch request -- line 43 in the calling code (script.js) -- needed to be changed to
const res = await fetch(`https://netlifytestserverless.netlify.app/.netlify/functions/serverUpdateString?input=${stringToUpdate}`, config);
The return statement in the lambda function needed to be changed to:
const response = {
statusCode: 200,
body: JSON.stringify(stringToReturn),
}
Other minor changes such as using parseInt with the environment variables.
The code works now.
I wanted to delete multiple files which are ending with .pdf under the current directory. Suppose I have 3 different pdf files, 1 image file, and one text file, so in these, I want to delete those 3 different pdf files only.
What I have tried.
1st method
fs.unlinkSync('./'+*+pdfname); -> I know this does not make sense
2nd method
try {
var files = (here the list of files should come. However i am failing to get those);
var path="./"
files.forEach(path => fs.existsSync(path) && fs.unlinkSync(path))
} catch (err) {
console.error("not exist")
}
Any different approaches would be appreciated.
Update for the solution:
I have got the solution for my requirement, I just wanted my function to delete all the pdf files and function to be synchronous. However 99% of the solution given by the below author -> https://stackoverflow.com/a/66558251/11781464
fs.readdir is asynchronous and just needs to make it synchronous fs.readdirSync.
below is the updated code and all the credit should go to the author https://stackoverflow.com/a/66558251/11781464.
Updated code
try {
const path = './'
// Read the directory given in `path`
fs.readdirSync(path).forEach((file) => {
// Check if the file is with a PDF extension, remove it
if (file.split('.').pop().toLowerCase() === 'pdf') {
console.log(`Deleting file: ${file}`);
fs.unlinkSync(path + file)
}
});
console.log("Deleted all the pdf files")
return true;
} catch (err) {
console.error("Error in deleting files",err);
}
You can read the directory using fs.readdir and then check for PDF files and delete them. Like this:
fs = require('fs');
try {
path = './'
// Read the directory given in `path`
const files = fs.readdir(path, (err, files) => {
if (err)
throw err;
files.forEach((file) => {
// Check if the file is with a PDF extension, remove it
if (file.split('.').pop().toLowerCase() == 'pdf') {
console.log(`Deleting file: ${file}`);
fs.unlinkSync(path + file)
}
});
});
} catch (err) {
console.error(err);
}
Preliminary Reading
The current working directory of the Node.js process - see update below answer
Path methods
resolve
extName
File System methods
fs.readdirSync
fs.unlinkSync
and classes
dirent
Example
"use strict";
const fs = require('fs');
const path = require('path');
const cwd = process.cwd();
fs.readdirSync( cwd, {withFileTypes: true})
.forEach( dirent => {
if(dirent.isFile()) {
const fileName = dirent.name;
if( path.extname(fileName).toLowerCase() === ".pdf") {
fs.unlinkSync( path.resolve( cwd, fileName));
}
}
});
Notes
untested code
If unlinkSync fails I would assume it returns -1 as per the unlink(2) man page linked in documentation. Personally I would test this using a filename that doesn't exist in cwd.
I believe the {withFileTypes: true} option for readdirSync returns dirent objects with a mime-type value that would allow you to check for files of type application/pdf regardless of extension (not attempted in the example).
Update: path(resolve) adds the current working directory at the beginning of the returned path, when necessary, by default. path.resolve(fileName) will work equally as well as path.resolve(cwd, fileName) in the example.
It seems you know how to delete (unlink) the files - you are asking how to get the file paths?
Try using glob:
const pdfFiles = require("glob").globSync("*.pdf");
Hi here i am attaching tested code for delete all ( only ) .pdf files and not other extension files like .txt , .docs etc from directory.
Note : You can delete any files or directory only from server side.
const fs = require('fs');
const path = require('path')
fs.readdir('../path to directory', (err, files) => {
const pdfFiles = files.filter(el => path.extname(el) === '.pdf')
pdfFiles.forEach(file => {
console.log("Removing File -> ",file);
var filename = "../path to directory/"+file;
fs.unlink(filename,function(err){
if(err) return console.log(err);
console.log('file deleted successfully');
});
});
});
This will gives you a following result in console log.
Removing File -> note.pdf
Removing File -> note2.pdf
file deleted successfully
file deleted successfully
Please feel free to comment any query if have..
I have a requirement to read the zip file contents , This is how the zip folder will look like Manifest.zip , The files in the Manifest.zip are outline.png, publish.png, manifest.json. I need to read the manifest.json(key value pairs) on the fly in a react application.
Basically requirement is when user clicks on "maifest" link a pop up needs to shown which show manifest.json contents in the UI react components.
I tried using JSZip library, and the code is as below
var zip = new JSZip();
zip.loadAsync("Manifest.zip")
.then(function (zip) {
console.log(zip.files);
// Expected outline.png, publish.png, manifest.json
});
I get the error as shown below?
Don't know what the error is about, have you tried another zip file?
Anyway, here is my tsx code with reactjs hooks:
const [files, setFiles] = useState<File[]>(undefined);
const [fileInfo, setFileInfo] = useState<IFileInfo>(undefined);
useEffect(() => {
if (files) {
const f = files[0];
const dateBefore = new Date();
JSZip.loadAsync(f) // 1) read the Blob
.then((zip) => {
const contents = [];
zip.forEach((relativePath, zipEntry) => { // 2) print entries
contents.push(zipEntry.name);
});
const loadTime = moment(new Date()).diff(moment(dateBefore));
setFileInfo({
loadTime,
contents: contents.sort(),
error: null
});
}, (e) => {
const loadTime = moment(new Date()).diff(moment(dateBefore));
setFileInfo({
loadTime,
contents: [],
error: "Error reading " + f.name + ": " + e.message
});
});
}
}, [files]);
// below some code that adds a file to files with drag and drop interface
Take a look at this library. Seems stable and easy to use.
var AdmZip = require('adm-zip');
// reading archives
var zip = new AdmZip("./my_file.zip");
var zipEntries = zip.getEntries(); // an array of ZipEntry records
zipEntries.forEach(function(zipEntry) {
console.log(zipEntry.toString()); // outputs zip entries information
});
I have the following code that does not work currently.
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models.
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: true, force: false }).then(function() {
models.users.create({
name: 'joe',
loggedIn: true
}).then( task => {
console.log("saved user!!!!!");
});
process.exit();//close the nodeJS Script
}).catch(function(error) {
console.log(error);
});
sequelize-lib.js
var Sequelize = require('sequelize');
exports.getSequelizeConnection = function(stage){
var argv = require('minimist')(process.argv.slice(2)); //If this file is being used in a script, this will attempt to get information from the argument stage passed if it exists
//Change connection settings based on stage variable. Assume localhost by default.
var dbname = argv['stage'] ? argv['stage']+"_db" : 'localdb';
var dbuser = argv['stage'] ? process.env.RDS_USERNAME : 'admin';
var dbpass = argv['stage'] ? process.env.RDS_PASSWORD : 'local123';
var dbhost = argv['stage'] ? "database-"+argv['stage']+".whatever.com" : 'localhost';
//If state variable used during require overide any arguments passed.
if(stage){
dbname = stage+"_db";
dbuser = process.env.RDS_USERNAME
dbpass = process.env.RDS_PASSWORD
dbhost = "database-"+stage+".whatever.com"
}
var connection = new Sequelize(dbname,dbuser,dbpass, {
dialect: 'mysql',
operatorsAliases: false, //This gets rid of a sequelize deprecated warning , refer https://github.com/sequelize/sequelize/issues/8417
host: dbhost
});
return connection;
}
exports.setModels = function(connection){
//Import all the known models for the project.
const fs = require('fs');
const dir = __dirname+'/../models';
var models = {}; //empty model object for adding model instances in file loop below.
//#JA - Wait until this function finishes ~ hence readdirSync vs regular readdir which is async
fs.readdirSync(dir).forEach(file => {
console.log(file);
//Split the .js part of the filename
var arr = file.split(".");
var name = arr[0].toLowerCase();
//Create a modle object using the filename as the reference without the .js pointing to a created sequelize instance of the file.
models[name] = connection.import(__dirname + "/../models/"+file);
})
//Showcase the final model.
console.log(models);
return models; //This returns a model with reference to the sequelize models
}
I can't get the create command to work however with this setup. My guess is the variables must not be passing through correctly somehow. I'm not sure what I'm doing wrong?
The create command definitely works because if in the sequelize-lib.js I modify the setModels function to this...
exports.setModels = function(connection){
//Import all the known models for the project.
const fs = require('fs');
const dir = __dirname+'/../models';
var models = {}; //empty model object for adding model instances in file loop below.
//#JA - Wait until this function finishes ~ hence readdirSync vs regular readdir which is async
fs.readdirSync(dir).forEach(file => {
console.log(file);
//Split the .js part of the filename
var arr = file.split(".");
var name = arr[0].toLowerCase();
//Create a modle object using the filename as the reference without the .js pointing to a created sequelize instance of the file.
models[name] = connection.import(__dirname + "/../models/"+file);
models[name].create({
"name":"joe",
"loggedIn":true
});
})
//Showcase the final model.
console.log(models);
return models; //This returns a model with reference to the sequelize models
}
Then it works and I see the item added to the database! (refer to proof image below)
Take note, I am simply running create on the variable at this point. What am I doing wrong where the model object is not passing between files correctly? Weird part is I don't get any errors thrown in the main file?? It's as if everything is defined but empty or something and the command is never run and nothing added to the database.
I tried this in the main file also and no luck.
models["users"].create({
name: 'joe',
loggedIn: true
}).then( task => {
console.log("saved user!!!!!");
});
The purpose of this all is to read models automatically from the model directory and create instances that are ready to go for every model, even if new one's are added in the future.
UPDATE::
So I did another test that was interesting, it seems that the create function won't work in the .then() function of the sync command. It looks like it was passing it correctly though. After changing the front page to this...
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models using connection previosly created.
models["users"].create({
"name":"joe",
"loggedIn":true
});
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: true, force: false }).then(function() {
process.exit();//close the nodeJS Script
}).catch(function(error) {
console.log(error);
});
Doing this seems to get create to work. I'm not sure if this is good form or not though since the database might not be created at this point? I need a way to get it to work in the sync function.
Well I answered my question finally, but I'm not sure I like the answer.
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models using connection previosly created.
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: false, force: false }).then( () => {
models["users"].create({
"name":"joe",
"loggedIn":true
}).then( user => {
console.log("finished, with user.name="+user.name);
process.exit();
}).catch( error => {
console.log("Error Occured");
console.log(error);
});
}).catch(function(error) {
console.log(error);
});
turns out that process.exit was triggering before create would occur because create happens async. This means that all my code will have to constantly be running through callbacks...which seems like a nightmare a bit. I wonder if there is a better way?
I'm running mongodb 3.0.6 (homebrew) on OS X 10.11/15A263e and can't figure my way through the js code for ShardingTest to specify a non-default data file path.
I want to avoid messing with symlinks or actual data files in /data for reasons but more importantly want to learn hot get the code below that class ShardingTest to work:
manoa:dev mike$ mongo --nodb
MongoDB shell version: 3.0.6
> var options = { shardOptions : { dbpath : "/usr/local/var/mongodb" } };
> cluster = new ShardingTest( { shards : 3 }, { other : options } );
Resetting db path '/data/db/test0'
2015-08-31T07:54:03.707-0500 E QUERY Error: boost::filesystem::create_directory: No such file or directory: "/data/db/test0"
at Function.MongoRunner.runMongod (src/mongo/shell/servers.js:589:13)
at new ShardingTest (src/mongo/shell/shardingtest.js:259:36)
at (shell):1:11 at src/mongo/shell/servers.js:589
I looked over this helpful post, but my JS is too weak to generalize from the help there to figure out how dbpath is set and which arguments will override the default path.
Could someone propose a way to debug this script or provide the actual syntax to feed my preferred db path to the ShardingTest function?
MongoRunner.dataPath is used to construct db paths:
this.getDBPaths = function() {
return _alldbpaths.map((path) => {
return MongoRunner.dataPath + path;
});
};
So you might set it to whatever directory you want, e.g:
// don't forget the trailing slash
MongoRunner.dataPath = '/home/your_user/data/db/'