When trying to send form-data that includes an image to the node multer library in the backend, the iOS simulators are giving me the error below in newer ios versions (version 13.7 didn't have any issue sending the image):
Exception '*** -[__NSCFConstantString substringToIndex:]: Index 3 out of bounds; string length 0' was thrown while invoking sendRequest on target Networking with params (
{
data = {
formData = (
{
fieldName = name;
headers = {
"content-disposition" = "form-data; name=\"name\"";
};
string = "User";
},
{
fieldName = email;
headers = {
"content-disposition" = "form-data; name=\"email\"";
};
string = "user#gmail.com";
},
{
fieldName = topic;
headers = {
"content-disposition" = "form-data; name=\"topic\"";
};
string = "New Calc Request";
},
{
fieldName = comment;
headers = {
"content-disposition" = "form-data; name=\"comment\"";
};
string = Test;
},
{
fieldName = attachment;
headers = {
"content-disposition" = "form-data; name=\"attachment\"; filename=\"\"";
"content-type" = "";
};
name = "";
type = "";
uri = "";
}
);
trackingName = unknown;
};
The issue seems to stem from not using JSON.stringify on the attachment. However, when I do use it, I can't seem to parse the image attachment on the server for use by the multer library. (Note: the attachment info is empty in the error because the simulator doesn't work well with the M1 chip in ios verisons later than 13).
Form-data being created as such:
userFeedBackInputs.append('name', this.currentUser.givenName + ' ' + this.currentUser.surname);
userFeedBackInputs.append('email', this.currentUser.email);
userFeedBackInputs.append('topic', this.state.topic.value);
userFeedBackInputs.append('comment', (this.state.comment.value : ''));
userFeedBackInputs.append('attachment', {
uri: this.state.imageSrc.value,
name: this.state.imageSrc.fileName,
type: this.state.imageSrc.type
}
)
How can I either a) send the data without using JSON.stringify or b) parse the JSON of the attachment within the formdata BEFORE it gets to the multer library. (It seems like the usual node body-parsers don't work with nested JSON, since this is a multi-form request)
I have exactly the same issue when test my app with ios devices that run ios 13, and I found the solution.
The reason is because the attachment has no data, just log the 'this.state.imageSrc.value' 'this.state.imageSrc.fileName' 'this.state.imageSrc.type', you'll see imageSrc is empty, may be there is a bug related to the file picker library,
if you're using 'react-native-image-crop-picker', update it to 0.33.3 then delete Node_modules, Pods, and re run, problem'll be solved
more detail : https://github.com/ivpusic/react-native-image-crop-picker/issues/1130
I'm trying to build a custom NodeJS logger, that logs data to a file.
To append data, I use fs.createWriteStream with the a flag.
var fs = require("fs");
var stream = fs.createWriteStream("./test.log", {
encoding: "utf-8",
flags: "a",
autoClose: true
});
// Data to log to file
var data = {
timestamp: new Date().toJSON(),
message: "OK"
};
// Write data
stream.write(JSON.stringify(data) + "\n");
Logging a few times results in a file looking like this:
{"timestamp":"2020-08-25T17:45:27.733Z","message":"OK"}
{"timestamp":"2020-08-25T17:45:34.820Z","message":"OK"}
{"timestamp":"2020-08-25T17:45:41.142Z","message":"OK"}
(Heres a newline, StackOverflow removes them)
My problem is, I don't know how to remove the trailing newline.
I thought about adding the newline before each log entry, but this requires me to detect the beginning of the file (I didn't find a way to do this).
What's the best way to remove the trailing newline, if that is even possible?
Any help would be greatly appreciated!
You need to check if file is empty:
var fs = require("fs");
var stream = fs.createWriteStream("./test.log", {
encoding: "utf-8",
flags: "a",
autoClose: true
});
// Data to log to file
var data = {
timestamp: new Date().toJSON(),
message: "OK"
};
// Check if file is empty
var stats = fs.statSync("./test.log");
var isEmpty = stats["size"] == 0;
// Write data
stream.write((isEmpty ? "" : "\n") + JSON.stringify(data) );
Solution
I wanted to post the actual modified code and solution. I had to get the Content-Type individually.
const url = 'api/test';
const result = ApiRequestFactory.build(ApiRequestFactory.MusiQuestApiType, url);
const contentType = result.headers.get('Content-Type');
expect(result).not.toBeNull('The result is null.');
expect(result.url).toBe(urlJoin(config.MusiQuestApi.url, 'api/test'), 'The url is incorrect.');
expect(result.options).toBeFalsy('There is an options object and should not be.');
expect(result.credentials).toBe('omit', 'The credentials is not set to omit.');
expect(contentType).toBeTruthy('The Content-Type header is missing.');
expect(contentType).toBe('application/json', 'The Content-Type header is not set to application/json.');
expect(result.method).toBe('GET', 'The method is not set to GET.');
expect(result.mode).toBe('cors', 'The mode is not set to cors.');
Environment
Test Runner: Karma
Test Framework: Jasmine
Test Browser: Chrome
Fetch Polyfill: isomorphic-fetch
ES6 Promise Library: es6-promise
Problem
When testing the object that comes back from defaultHeaders.values() I get an empty object even though I append a header.
Code
I have a class that builds a Request object. You can see below it adds a default header for Content-Type when building the object.
export default class MusiQuestApiRequest {
static build(url, body, options) {
const defaultCredentials = 'omit';
let defaultHeaders = new Headers();
defaultHeaders.append('Content-Type', 'application/json');
const defaultMethod = 'GET';
const defaultMode = 'cors';
let init = {
credentials: options.credentials || defaultCredentials,
headers: options.headers || defaultHeaders,
method: options.method || defaultMethod,
mode: options.mode || defaultMode
};
if (body) {
init.method = options.method || 'POST';
init.body = JSON.stringify(body);
}
return new Request(url, init);
}
}
But when I check for the Content-Type property on the values object I get undefined because the values object returned is empty.
const url = 'api/test';
const result = ApiRequestFactory.build(ApiRequestFactory.MusiQuestApiType, url);
const headersResult = result.headers.values();
console.log('headersResult:', JSON.stringify(headersResult));
expect(result).not.toBeNull();
expect(result.url).toBe(urlJoin(config.MusiQuestApi.url, 'api/test'));
expect(result.options).toBeFalsy();
expect(result.credentials).toBe('omit');
expect(headersResult).toBeTruthy('headers missing');
expect(headersResult['Content-Type']).toBeTruthy('Content-Type missing');
expect(headersResult['Content-Type']).toBe('application/json');
expect(result.method).toBe('GET');
expect(result.mode).toBe('cors');
What did I do wrong here? Seems pretty straight forward.
result.headers is a Headers object which is a bit different from a plain JavaScript object. To retrieve a value from the headers object you can use the get function.
For example:
myHeaders.get('foo')
Alternatively you can use values to return an Iterator which will enable you to use a for...of loop to extract the values.
You can find more information here.
Hope this helps.
My current method is this:
var request = require('request');
var mime = require('mime');
var fs = require('fs');
var uri = 'http://www.sweetslyrics.com/images/img_gal/25646_christina-perri-213968.jpg';
request({
'method':'GET',
'uri': uri
},function(err, response,body){
var tmp_path = '/tmp/123456';
fs.writeFile(tmp_path, body, function(err) {
console.log(mime.lookup(tmp_path)); //application/octet-stream ?????
});
});
The image is obviously a picture, but node-mime says it's application/octet-stream. Why?
Note:
- I do not want to rely on the Response Headers content-type, because based on my experience, sometimes those response headers are set incorrectly...and they do not determine the true file type. (that's why I save it to a file, and then have node-mime determine it for me!)
I want to know the best way to determine if a file is an image, with 0 margin of error.
Edit: I just realized that node-mime isn't "magic". It just checks for the extension :( ...
Edit2: I found this: https://github.com/SaltwaterC/mime-magic
Just read the first bytes of the stream, and check it for the so called "magic number".
Magic numbers are the first bits of a file which uniquely identify the
type of file.
For example:
-Every JPEG file begins with ff d8 (hex).
-Every png file begins with a 89 50 4e 47.
-There is a comprehensive table of magic numbers here
This way even if you have a file without extension you can still detect its type.
Hope this helps.
This code shows a working solution for the magic numbers approach (summary of the existing answers and information on https://github.com/request/request).
var request = require('request');
var url = "http://www.somedomain.com/somepicture.jpg";
var magic = {
jpg: 'ffd8ffe0',
png: '89504e47',
gif: '47494638'
};
var options = {
method: 'GET',
url: url,
encoding: null // keeps the body as buffer
};
request(options, function (err, response, body) {
if(!err && response.statusCode == 200){
var magigNumberInBody = body.toString('hex',0,4);
if (magigNumberInBody == magic.jpg ||
magigNumberInBody == magic.png ||
magigNumberInBody == magic.gif) {
// do something
}
}
});
There are two modules that can help you achieve this:
https://github.com/SaltwaterC/mime-magic
https://github.com/bentomas/node-mime
In the intervening time since this question was first asked, mime-magic has become unsupported and its author recommends the use of mmmagic. I don't know what happened to node-mime, the link above is a 404. I found the following article which discusses the topic as well: https://nodejsmodules.org/tags/mime
i developped this code and i test it and it work for me you can use it
var express = require('express')
var app = express()
var http = require('http').Server(app).listen(80)
var upload = require('express-fileupload')
app.use(upload())
app.get("/",(req,res)=>{
res.sendFile(__dirname+"/file.html")
})
app.post('/',(req,res)=>{
var options = {
method: 'GET',
url: req.files.filename,
encoding: null
}
if (req.files) {
if (req.files.filename.data.toString('hex',0,4) == '89504e47' || req.files.filename.data.toString('hex',0,4) == 'ffd8ffe0' || req.files.filename.data.toString('hex',0,4) == '47494638' ) {
var file = req.files.filename
filename = file.name
file.mv('./upload/'+filename,(err)=>{
if (err) {
console.log('small err')
} else {
res.send('DONE')
}
})
} else {
console.log('it not an image')
}
}
})
I want to be able to set a single cookie, and read that single cookie with each request made to the nodejs server instance. Can it be done in a few lines of code, without the need to pull in a third party lib?
var http = require('http');
http.createServer(function (request, response) {
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello World\n');
}).listen(8124);
console.log('Server running at http://127.0.0.1:8124/');
Just trying to take the above code directly from nodejs.org, and work a cookie into it.
There is no quick function access to getting/setting cookies, so I came up with the following hack:
const http = require('http');
function parseCookies (request) {
const list = {};
const cookieHeader = request.headers?.cookie;
if (!cookieHeader) return list;
cookieHeader.split(`;`).forEach(function(cookie) {
let [ name, ...rest] = cookie.split(`=`);
name = name?.trim();
if (!name) return;
const value = rest.join(`=`).trim();
if (!value) return;
list[name] = decodeURIComponent(value);
});
return list;
}
const server = http.createServer(function (request, response) {
// To Read a Cookie
const cookies = parseCookies(request);
// To Write a Cookie
response.writeHead(200, {
"Set-Cookie": `mycookie=test`,
"Content-Type": `text/plain`
});
response.end(`Hello World\n`);
}).listen(8124);
const {address, port} = server.address();
console.log(`Server running at http://${address}:${port}`);
This will store all cookies into the cookies object, and you need to set cookies when you write the head.
If you're using the express library, as many node.js developers do, there is an easier way. Check the Express.js documentation page for more information.
The parsing example above works but express gives you a nice function to take care of that:
app.use(express.cookieParser());
To set a cookie:
res.cookie('cookiename', 'cookievalue', { maxAge: 900000, httpOnly: true });
To clear the cookie:
res.clearCookie('cookiename');
RevNoah had the best answer with the suggestion of using Express's cookie parser. But, that answer is now 3 years old and is out of date.
Using Express, you can read a cookie as follows
var express = require('express');
var cookieParser = require('cookie-parser');
var app = express();
app.use(cookieParser());
app.get('/myapi', function(req, resp) {
console.log(req.cookies['Your-Cookie-Name-Here']);
})
And update your package.json with the following, substituting the appropriate relatively latest versions.
"dependencies": {
"express": "4.12.3",
"cookie-parser": "1.4.0"
},
More operations like setting and parsing cookies are described here
and here
As an enhancement to #Corey Hart's answer, I've rewritten the parseCookies() using:
RegExp.prototype.exec - use regex to parse "name=value" strings
Here's the working example:
let http = require('http');
function parseCookies(str) {
let rx = /([^;=\s]*)=([^;]*)/g;
let obj = { };
for ( let m ; m = rx.exec(str) ; )
obj[ m[1] ] = decodeURIComponent( m[2] );
return obj;
}
function stringifyCookies(cookies) {
return Object.entries( cookies )
.map( ([k,v]) => k + '=' + encodeURIComponent(v) )
.join( '; ');
}
http.createServer(function ( request, response ) {
let cookies = parseCookies( request.headers.cookie );
console.log( 'Input cookies: ', cookies );
cookies.search = 'google';
if ( cookies.counter )
cookies.counter++;
else
cookies.counter = 1;
console.log( 'Output cookies: ', cookies );
response.writeHead( 200, {
'Set-Cookie': stringifyCookies(cookies),
'Content-Type': 'text/plain'
} );
response.end('Hello World\n');
} ).listen(1234);
I also note that the OP uses the http module.
If the OP was using restify, he can make use of restify-cookies:
var CookieParser = require('restify-cookies');
var Restify = require('restify');
var server = Restify.createServer();
server.use(CookieParser.parse);
server.get('/', function(req, res, next){
var cookies = req.cookies; // Gets read-only cookies from the request
res.setCookie('my-new-cookie', 'Hi There'); // Adds a new cookie to the response
res.send(JSON.stringify(cookies));
});
server.listen(8080);
Let me repeat this part of question that answers here are ignoring:
Can it be done in a few lines of code, without the need to pull in a third party lib?
Reading Cookies
Cookies are read from requests with the Cookie header. They only include a name and value. Because of the way paths work, multiple cookies of the same name can be sent. In NodeJS, all Cookies in as one string as they are sent in the Cookie header. You split them with ;. Once you have a cookie, everything to the left of the equals (if present) is the name, and everything after is the value. Some browsers will accept a cookie with no equal sign and presume the name blank. Whitespaces do not count as part of the cookie. Values can also be wrapped in double quotes ("). Values can also contain =. For example, formula=5+3=8 is a valid cookie.
/**
* #param {string} [cookieString='']
* #return {[string,string][]} String Tuple
*/
function getEntriesFromCookie(cookieString = '') {
return cookieString.split(';').map((pair) => {
const indexOfEquals = pair.indexOf('=');
let name;
let value;
if (indexOfEquals === -1) {
name = '';
value = pair.trim();
} else {
name = pair.substr(0, indexOfEquals).trim();
value = pair.substr(indexOfEquals + 1).trim();
}
const firstQuote = value.indexOf('"');
const lastQuote = value.lastIndexOf('"');
if (firstQuote !== -1 && lastQuote !== -1) {
value = value.substring(firstQuote + 1, lastQuote);
}
return [name, value];
});
}
const cookieEntries = getEntriesFromCookie(request.headers.Cookie);
const object = Object.fromEntries(cookieEntries.slice().reverse());
If you're not expecting duplicated names, then you can convert to an object which makes things easier. Then you can access like object.myCookieName to get the value. If you are expecting duplicates, then you want to do iterate through cookieEntries. Browsers feed cookies in descending priority, so reversing ensures the highest priority cookie appears in the object. (The .slice() is to avoid mutation of the array.)
Settings Cookies
"Writing" cookies is done by using the Set-Cookie header in your response. The response.headers['Set-Cookie'] object is actually an array, so you'll be pushing to it. It accepts a string but has more values than just name and value. The hardest part is writing the string, but this can be done in one line.
/**
* #param {Object} options
* #param {string} [options.name='']
* #param {string} [options.value='']
* #param {Date} [options.expires]
* #param {number} [options.maxAge]
* #param {string} [options.domain]
* #param {string} [options.path]
* #param {boolean} [options.secure]
* #param {boolean} [options.httpOnly]
* #param {'Strict'|'Lax'|'None'} [options.sameSite]
* #return {string}
*/
function createSetCookie(options) {
return (`${options.name || ''}=${options.value || ''}`)
+ (options.expires != null ? `; Expires=${options.expires.toUTCString()}` : '')
+ (options.maxAge != null ? `; Max-Age=${options.maxAge}` : '')
+ (options.domain != null ? `; Domain=${options.domain}` : '')
+ (options.path != null ? `; Path=${options.path}` : '')
+ (options.secure ? '; Secure' : '')
+ (options.httpOnly ? '; HttpOnly' : '')
+ (options.sameSite != null ? `; SameSite=${options.sameSite}` : '');
}
const newCookie = createSetCookie({
name: 'cookieName',
value: 'cookieValue',
path:'/',
});
response.headers['Set-Cookie'].push(newCookie);
Remember you can set multiple cookies, because you can actually set multiple Set-Cookie headers in your request. That's why it's an array.
Note on external libraries:
If you decide to use the express, cookie-parser, or cookie, note they have defaults that are non-standard. Cookies parsed are always URI Decoded (percent-decoded). That means if you use a name or value that has any of the following characters: !#$%&'()*+/:<=>?#[]^`{|} they will be handled differently with those libraries. If you're setting cookies, they are encoded with %{HEX}. And if you're reading a cookie you have to decode them.
For example, while email=name#domain.com is a valid cookie, these libraries will encode it as email=name%40domain.com. Decoding can exhibit issues if you are using the % in your cookie. It'll get mangled. For example, your cookie that was: secretagentlevel=50%007and50%006 becomes secretagentlevel=507and506. That's an edge case, but something to note if switching libraries.
Also, on these libraries, cookies are set with a default path=/ which means they are sent on every url request to the host.
If you want to encode or decode these values yourself, you can use encodeURIComponent or decodeURIComponent, respectively.
References:
Cookie Syntax
Set-Cookie Syntax
Additional information:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cookie
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
You can use the "cookies" npm module, which has a comprehensive set of features.
Documentation and examples at:
https://github.com/jed/cookies
To get a cookie splitter to work with cookies that have '=' in the cookie values:
var get_cookies = function(request) {
var cookies = {};
request.headers && request.headers.cookie.split(';').forEach(function(cookie) {
var parts = cookie.match(/(.*?)=(.*)$/)
cookies[ parts[1].trim() ] = (parts[2] || '').trim();
});
return cookies;
};
then to get an individual cookie:
get_cookies(request)['my_cookie']
Cookies are transfered through HTTP-Headers
You'll only have to parse the request-headers and put response-headers.
Here's a neat copy-n-paste patch for managing cookies in node. I'll do this in CoffeeScript, for the beauty.
http = require 'http'
http.IncomingMessage::getCookie = (name) ->
cookies = {}
this.headers.cookie && this.headers.cookie.split(';').forEach (cookie) ->
parts = cookie.split '='
cookies[parts[0].trim()] = (parts[1] || '').trim()
return
return cookies[name] || null
http.IncomingMessage::getCookies = ->
cookies = {}
this.headers.cookie && this.headers.cookie.split(';').forEach (cookie) ->
parts = cookie.split '='
cookies[parts[0].trim()] = (parts[1] || '').trim()
return
return cookies
http.OutgoingMessage::setCookie = (name, value, exdays, domain, path) ->
cookies = this.getHeader 'Set-Cookie'
if typeof cookies isnt 'object'
cookies = []
exdate = new Date()
exdate.setDate(exdate.getDate() + exdays);
cookieText = name+'='+value+';expires='+exdate.toUTCString()+';'
if domain
cookieText += 'domain='+domain+';'
if path
cookieText += 'path='+path+';'
cookies.push cookieText
this.setHeader 'Set-Cookie', cookies
return
Now you'll be able to handle cookies just as you'd expect:
server = http.createServer (request, response) ->
#get individually
cookieValue = request.getCookie 'testCookie'
console.log 'testCookie\'s value is '+cookieValue
#get altogether
allCookies = request.getCookies()
console.log allCookies
#set
response.setCookie 'newCookie', 'cookieValue', 30
response.end 'I luvs da cookies';
return
server.listen 8080
Using Some ES5/6 Sorcery & RegEx Magic
Here is an option to read the cookies and turn them into an object of Key, Value pairs for client side, could also use it server side.
Note: If there is a = in the value, no worries. If there is an = in the key, trouble in paradise.
More Notes: Some may argue readability so break it down as you like.
I Like Notes: Adding an error handler (try catch) wouldn't hurt.
const iLikeCookies = () => {
return Object.fromEntries(document.cookie.split('; ').map(v => v.split(/=(.+)/)));
}
const main = () => {
// Add Test Cookies
document.cookie = `name=Cookie Monster;expires=false;domain=localhost`
document.cookie = `likesCookies=yes=withARandomEquals;expires=false;domain=localhost`;
// Show the Objects
console.log(document.cookie)
console.log('The Object:', iLikeCookies())
// Get a value from key
console.log(`Username: ${iLikeCookies().name}`)
console.log(`Enjoys Cookies: ${iLikeCookies().likesCookies}`)
}
What is going on?
iLikeCookies() will split the cookies by ; (space after ;):
["name=Cookie Monster", "likesCookies=yes=withARandomEquals"]
Then we map that array and split by first occurrence of = using regex capturing parens:
[["name", "Cookie Monster"], ["likesCookies", "yes=withARandomEquals"]]
Then use our friend `Object.fromEntries to make this an object of key, val pairs.
Nooice.
If you don't care what's in the cookie and you just want to use it, try this clean approach using request (a popular node module):
var request = require('request');
var j = request.jar();
var request = request.defaults({jar:j});
request('http://www.google.com', function () {
request('http://images.google.com', function (error, response, body){
// this request will will have the cookie which first request received
// do stuff
});
});
var cookie = 'your_cookie';
var cookie_value;
var i = request.headers.indexOf(cookie+'=');
if (i != -1) {
var eq = i+cookie.length+1;
var end = request.headers.indexOf(';', eq);
cookie_value = request.headers.substring(eq, end == -1 ? undefined : end);
}
I wrote this simple function just pass
req.headers.cookie and cookie name
const getCookieByName =(cookies,name)=>{
const arrOfCookies = cookies.split(' ')
let yourCookie = null
arrOfCookies.forEach(element => {
if(element.includes(name)){
yourCookie = element.replace(name+'=','')
}
});
return yourCookie
}
I know that there are many answer to this question already, but here's a function made in native JS.
function parseCookies(cookieHeader) {
var cookies = {};
cookieHeader
.split(";")
.map(str => str.replace("=", "\u0000")
.split("\u0000"))
.forEach(x => cookies[x[0]] = x[1]);
return cookies;
}
It starts by taking in the document.cookie string. Every key-value pair is separated by a semicolon (;). Therefore the first step is to divide the string up each key-value pair.
After that, the function replaces the first instance of "=" with a random character that isn't in the rest of the string, for this function I decided to use the NULL character (\u0000). The key-value pair can now be split into just two pieces. The two pieces can now be combined into JSON.
You can use cookie lib to parse incoming multiple cookies, so that you won't have to worry about exceptions cases:
var cookies = cookie.parse('foo=bar; equation=E%3Dmc%5E2');
// { foo: 'bar', equation: 'E=mc^2' }
To write a cookie you can do like this:
response.writeHead(200, {
"Set-Cookie": `mycookie=cookie`,
"Content-Type": `text/plain`
});
First one needs to create cookie (I have wrapped token inside cookie as an example) and then set it in response.To use the cookie in following way install cookieParser
app.use(cookieParser());
The browser will have it saved in its 'Resource' tab and will be used for every request thereafter taking the initial URL as base
var token = student.generateToken('authentication');
res.cookie('token', token, {
expires: new Date(Date.now() + 9999999),
httpOnly: false
}).status(200).send();
To get cookie from a request on the server side is easy too.You have to extract the cookie from request by calling 'cookie' property of the request object.
var token = req.cookies.token; // Retrieving Token stored in cookies