I'm building a web app using Node.JS that at the very least should allow users to to upload excel spreadsheets (.xlsx) and then using an excel parser (currently using node-xlsx - https://www.npmjs.org/package/node-xlsx), I want to be able to find this file, parse it, and print its contents to the console. So far, I have the file uploaded and stored, but am having trouble specifying the file path my app should search down.
I believe my troubles are that I am trying to do this on the server-side, and I am telling my app to search through a users directory for this file when it does not have access.
Here is example code:
var fullfile;
app.post('/upload', function (request, response) {
var fstream;
request.pipe(request.busboy);
request.busboy.on('file', function (fieldname, file, filename) {
console.log('Uploading: ' + filename);
fstream = fs.createWriteStream('./storedFiles/' + filename);
file.pipe(fstream);
fstream.on('close', function () {
response.redirect('success');
console.log('Uploaded to ' + fstream.path);
fullfile=fstream.name;
var obj = xlsx.parse(__dirname + fullfile);
console.log(obj);
});
});
});
This produces the error:
return binding.open(pathmodule._makelong(path) stringtoflags(flags) mode)
error: ENOENT, no such file or directory 'C\Users(file path on my local machine here)
Can anyone point out a way of doing this that I am missing? it has to do with fs methods I feel.
Thank you
First of all, don't use the filename that user provided when saving the file - you will get duplicates and it could be a security risk (in general, never trust user provided data). Just use your own value instead - it is more standard to use your own naming convention to prevent duplicates or to use a tmp file provided by the OS.
To solve your issue, try:
Requiring path at the top of your file:
var path = require('path');
and changing the value of fullfile to:
fullfile = path.join(__dirname,fstream.path)
then pass fullfile to xlsx.parse:
var obj = xlsx.parse(fullfile);
Related
I am very new to Node server/javacsript. So I am sorry if this might be stupid
question/topic.
I intended to create a very simple solution to open JSON file, load to list, and save it back to my local disk (running node.js server).
Could you please help me out, what I am doing wrong? I am running app in browser using react.
index.js containing
var fs = require('fs');
var fileName = './test.json';
var file = require('./test.json');
alert(file.name + " " + file.age);
file.name = "Peter";
alert(file.name + " " + file.age);
fs.writeFile('./test.json', JSON.stringify(file), function (err) {
if (err) return alert(err);
console.log(JSON.stringify(file));
alert('writing to ' + fileName);
});
Before I was not even able to open JSON file. I needed to include this property into the webpack config file.
node: {
fs: 'empty'
}
Now I am able to open JSON file, change it virtually, but unable to save it.
In chrome developer tools, it prints "fs.writeFile is not a function" into console.
Thank you very much.
When you included the property in your webpack config
node: {
fs: 'empty'
}
You told webpack that the module fs should just be an empty object. You can confirm this by simply putting a console.log(fs) in your file to see it is indeed empty.
Beyond that, fs is not going to work in your browser. fs expects the node.js runtime (which includes non-JavaScript things in order to make it work), not your browser's runtime.
If you want a user to save a file, you'll have to use a browser based saving solution. You won't be able to just arbitrarily write files like that outside of something like your browser's local storage.
Is there a possible way to read a local file in JavaScript.
MyFolder:
db.csv
Parse.js
Trying to fetch the contents of file db.csv in Parse.js, But in vain.
Can you share some links where I can get enough knowledge how to read a file.
Running Instruments in Xcode5, with test scripts in .js file where I have to feed in some values from a .csv file.
iOS UIAutomation, apple provides an api for running a task on the target's host.
performTaskWithPathArgumentsTimeout
Using this, we can have a bash script to printout the contents of a file that we wanted to fetch in the first case.
Bash script can be as simple as this for this requirement.
#! /bin/bash
FILE_NAME="$1"
cat $FILE_NAME
Save it as for example FileReader.sh file.
And in your automation script,
var target = UIATarget.localTarget();
var host = target.host();
var result = host.performTaskWithPathArgumentsTimeout(executablePath,[filePath,fileName], 15);
UIALogger.logDebug("exitCode: " + result.exitCode);
UIALogger.logDebug("stdout: " + result.stdout);
UIALogger.logDebug("stderr: " + result.stderr);
where in,
executablePath is where the command need to be executed.
var executablePath = "/bin/sh";
filePath is the location of the created FileReader.sh file. When executed, outputs the content to standard output (in our requirement).
[give full absolute path of the file]
fileName is the actual file to fetch contents from.
[give full absolute path of the file] In my case I had a Contents.csv file, which I had to read.
and the last parameter is the timeout in seconds.
Hope this helps others, trying to fetch contents (reading files) for performing iOS UIAutomation.
References:
https://stackoverflow.com/a/19016573/344798
https://developer.apple.com/library/iOS/documentation/UIAutomation/Reference/UIAHostClassReference/UIAHost/UIAHost.html
If the file is on the same domain as the site you're in, you'd load it with Ajax. If you're using Ajax, it's be something like
$.get('db.csv', function(csvContent){
//process here
});
Just note that the path to the csv file will be relative to the web page you're in, not the JavaScript file.
If you're not using jQuery, you'd have to manually work with an XmlHttpRequest object to do your Ajax call.
And though your question doesn't (seem to) deal with it, if the file is located on a different domain, then you'd have to use either jsonP or CORS.
And, just in case this is your goal, no, you can't, in client side JavaScript open up some sort of Stream and read in a file. That would be a monstrous security vulnerability.
This is a fairly simple function in Illuminator's host functions library:
function readFromFile(path) {
var result = target.host().performTaskWithPathArgumentsTimeout("/bin/cat", [path], 10);
// be verbose if something didn't go well
if (0 != result.exitCode) {
throw new Error("readFromFile failed: " + result.stderr);
}
return result.stdout;
}
If you are using Illuminator, this is host().readFromFile(path).
I am currently building a project with node.js in Windows. I am using a batch file to assemble resources and build jade templates via the command line. With Jade, I am using the switch -o to defines a JS object that fills localized content in the template
For awhile, everything worked nicely. However, changes to my JSON lookup have resulted in an error:
"The input line is too long"
Researching the error, I found that windows shell has a limit on how long your lines can be. Unfortunately, I need the whole lookup object for my project. However, I started wondering if jade can accept a path to my lookup file instead of a string with the contents of the file. Currently, I'm building the contents into a variable and calling jade with that ala:
SetLocal EnableDelayedExpansion
set content=
for /F "delims=" %%i in (%sourcedir%\assets\english.json) do set content=!content! %%i
::use the json file as a key for assembling the jade templates
call jade %sourcedir% --out %destdir% -o"%content%"
EndLocal
If I could use a path to the lookup file, it would be much easier. However, I am usure how to do that (if it's even possible). and Jade's documentation is a bit lacking.
So, in short, is it possible for Jade to accept a filepath to a JS object rather than a string containing the object? Is there a better way to contruct the jade call that wont push it past the limit?
Write a node.js script that will read your "assets" and will call a jade. Something like:
var fs = require('fs'),
_ = require('underscore'),
async = require('async');
var sourceDir = 'path to the directory with your jade templates',
destinationDir = 'path to the directory where you want the result html files to be contained in';
async.waterfall([
async.parallel.bind(null, {
serializedData: fs.readFile.bind(null, 'assets/english.json'),
files: fs.readDir.bind(null, sourceDir),
}),
function (result, callback) {
var data = JSON.parse(result.serializedData),
files = result.files;
async.parallel(_.map(files, function (file) {
return async.waterfall.bind(null, [
fs.readFile.bind(null, sourceDir + file),
function (jadeSource, callback) {
process.nextTick(callback.bind(null, jade.compile(jadeSource)(data)));
},
fs.writeFile.bind(null, destinationDir + file)
]);
}), callback);
}
], function (err) {
if (err) {
console.log("An error occured: " + err);
} else {
console.log("Done!");
}
});
Then in your batch file call this script directly, instead of enumerating the directory and calling the jade manually.
It will not only solve your problem, but also work much faster because:
I/O operations are done in parallel;
Node.js is only started once during the build process, as opposed to starting it for every single file as you do now.
I need to get the file type of a file with the help of node.js to set the content type. I know I can easily check the file extension but I've also got files without extension which should have the content type image/png, text/html aso.
This is my code (I know it doesn't make much sense but that's the base I need):
var http = require("http"),
fs = require("fs");
http.createServer(function(req, res) {
var data = "";
try {
/*
* Do not use this code!
* It's not async and it has a security issue.
* The code style is also bad.
*/
data = fs.readFileSync("/home/path/to/folder" + req.url);
var type = "???"; // how to get the file type??
res.writeHead(200, {"Content-Type": type});
} catch(e) {
data = "404 Not Found";
res.writeHead(404, {"Content-Type": "text/plain"});
}
res.write(data);
res.end();
}).listen(7000);
I haven't found a function for that in the API so I would be happy if anyone can tell me how to do it.
There is a helper library for looking up mime types https://github.com/broofa/node-mime
var mime = require('mime');
mime.getType('/path/to/file.txt'); // => 'text/plain'
But it still uses the extension for lookup though
Have a look at the mmmagic module. It is a libmagic binding and seems to do exactly what you want.
You should have a look at the command line tool file (Linux). It attempts to guess the filetype based on the first couple of bytes of the file. You can use child_process.spawn to run it from within node.
You want to be looking up the mime type, and thankfully node has a handy library just for that:
https://github.com/bentomas/node-mime#readme
edit:
You should probably look into a static asset server and not be setting any of this stuff yourself. You can use express to do this really easily, or there's a whole host of static file modules, e.g. ecstatic. On the other hand you should probably use nginx to serve static files anyway.
2018 solution
The accepted answer appears to have a Python dependency and the other answers are either out-of-date or presume the file name has some sort of extension.
Please find my more up-to-date answer here
I used this:
npm install mime-types
And, inside the code:
var mime = require('mime-types');
tmpImg.contentType = mime.lookup(fileImageTmp);
Where fileImageTmp is the copy of image stored on the file system (in this case tmp).
The result I can see is: image/jpeg
The best approach I believe is using the file command of the system, that way you have three advantages:
No dependencies,
You will make sure through magic numbers the type of content the file has,
You will be able to create type of contents through magic files.
Example:
let pathToFile = '/path/to/file';
const child_process = require('child_process');
child_process.exec(`"file" ${path}`, (err, res) => {
let results = res.replace('\n', '').split(':');
let stringPath = results[0].trim();
let typeOfFile = results[1].trim();
console.log(stringPath, typeOfFile);
});
Docs:
https://www.man7.org/linux/man-pages/man1/file.1.html
https://nodejs.org/docs/latest-v13.x/api/child_process.html#child_process_child_process_exec_command_options_callback
I know Express has the res.contentType() method, but how to set automatically content type based on path/file (including static content)?
Also, if you want to extend the mime-types that express(connect) knows about, you can do
express.static.mime.define({'text/plain': ['md']});
or
connect.static.mime.define({'text/plain': ['md']});
PS: the mime module is now located at https://github.com/broofa/node-mime
The Express documentation shows that it can do this if you pass in the file name.
var filePath = 'path/to/image.png';
res.contentType(path.basename(filePath));
// Content-Type is now "image/png"
[Edit]
Here's an example which serves files from a relative directory called static and automatically sets the content type based on the file served:
var express = require('express');
var fs = require('fs');
var app = express.createServer();
app.get('/files/:file', function(req, res) {
// Note: should use a stream here, instead of fs.readFile
fs.readFile('./static/' + req.params.file, function(err, data) {
if(err) {
res.send("Oops! Couldn't find that file.");
} else {
// set the content type based on the file
res.contentType(req.params.file);
res.send(data);
}
res.end();
});
});
app.listen(3000);
Connect will automatically set the content type, unless you explicitly set it yourself. Here's the snippet that does it. It uses mime.lookup and mime.charsets.lookup
// mime type
type = mime.lookup(path);
//<SNIP>....
// header fields
if (!res.getHeader('content-type')) {
var charset = mime.charsets.lookup(type);
res.setHeader('Content-Type', type + (charset ? '; charset=' + charset : ''));
}
If this isn't working for you, post your code as your custom code is likely interfering with the default behavior somehow.
Express uses Connect, Connect uses Mime, and Mime includes the files mime.types (with default mime types from Apache) and node.types (with some further types contributed by node community). You could just customize one of these files within your copy of mime in node_modules to add your required content type, or Mime also has an API that lets you specify additional content-types or .types files to load from your code.
https://github.com/broofa/node-mime
Download this database (or Another link ) : mime.types: , then
var db_mimes=[],mime_ext=''
$.get('mime.types',{},function(d){
var lines=d.split('\n').filter(function(e){ /* filter which starts with #*/})
lines.forEach(function(line){
mime_ext=line.split(' ')
mime_ext[1].split(' ').forEach(function(ext){
db_mimes.push({e:ext,m:mime_ext[0]})
});
//create object for each line . i.e: {mime:'',extension}
});
});
Then if you have fo example var fname="myfile.png"
var extension=fname.substr((~-this.lastIndexOf(".") >>> 0) + 2) // get extension from name
var minme=db_mimes.filter(function(el){return el.e === extension})[0]
Run the following cmd :
npm install xmimetype ;
Then , in your code :
var xm=require("xmimetype");
xm.mimetypeOf("java");
xm.mimetypeOf("./lib/Person.java");
// -> text/x-java-source
xm.mimetypeOf("docx");
xm.mimetypeOf("./lib/overview.docx");
// -> application/vnd.openxmlformats-officedocument.wordprocessingml.document
For more info , check GIT repository .
The opposite is available :
xm.extensionsOf("image/jpeg");
// -> [ 'jpeg', 'jpg', 'jpe' ]