I cannot get the simplest example of NeDB to run properly. My code only works in-memory, persistence to file keeps failing without any error messages.
The error callbacks for the loaddatabase and insert events always pass a null reference as error, so no information there. Oddly it seems no one else has this issue, so I guess I'm missing something here. All help is much appreciated.
Here is the code:
var Datastore = require('nedb'), db = new Datastore({ filename: 'test.db' });
db.loadDatabase(function (err) {
alert(err); // err is null, with the autoload flag no error is thrown either
});
var doc = { hello: 'world'};
db.insert(doc, function (err, newDoc) {
alert(err); // err is null here as well. Doc will be in the memory storage but no persisted to file
});
Although this question is pretty old, I'd like to share my experience for anyone facing a similar issue.
NeDB API does not allow JSON input. You have to put in a javascript object. When you use JSON input, no error is returned and nothing will be persisted.
'null' is returned as error in callback to signal that no problem occurred. When saving the first JSON document it is indexed with 'undefined' key, because NeDB calls 'key = obj[fieldname[0]]' which returns 'undefined', when the obj is just a (JSON) string. No error is returned unfortunately. Inserting a second document will cause a unique constraint violation error in the callback as the key 'undefined' has already been taken. Anyhow, nothing will be persisted.
Try
var Datastore = require('nedb'), db = new Datastore({ filename: 'test.db' });
db.loadDatabase(function (error) {
if (error) {
console.log('FATAL: local database could not be loaded. Caused by: ' + error);
throw error;
}
console.log('INFO: local database loaded successfully.');
});
// creating the object with new, just to make it clear.
// var doc = {hello: 'world'}; should work too.
function myDoc(greeting)
{
this.hello=greeting;
}
var doc = new myDoc('world');
db.insert(doc, function (error, newDoc) {
if (error) {
console.log('ERROR: saving document: ' + JSON.stringify(doc) + '. Caused by: ' + error);
throw error;
}
console.log('INFO: successfully saved document: ' + JSON.stringify(newDoc));
});
Maybe it helps someone. :)
This question is quite old but since I had very similar problem I thought that I'll write my resolution for anyone facing similar issues.
In my case I was writing Electron app using electron-webpack as an application builder. It turns out that NeDB loaded by Webpack was running in browser mode without access to file system.
To get it working I had to change import statement from:
import DataStore from 'nedb';
to this:
const DataStore = require('nedb');
Also I had to add NeDB to Webpack configuration as external module (in package.json):
"electronWebpack": {
"externals": {
"nedb": "commonjs nedb"
}
}
I have found this resolution on NeDB github page: https://github.com/louischatriot/nedb/issues/329
All I had to do to fix this was delete the .db file and let the program make one for me by running it one more time.
The other thing I did that could have fixed it was making sure my package.json had all the required information. this can be easily done with a quick "npm init" in the terminal.
Related
I'm working on setting up a Lambda function in JavaScript. I want this function to take some data when a dynamodb record is deleted, and use that to find and remove the S3 object it corresponds to (In a versioned bucket). Here's what I have so far:
import { Context, APIGatewayProxyResult, APIGatewayEvent } from 'aws-lambda';
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
region: 'eu-west-2'
});
export const handler = async (event: APIGatewayEvent, context: Context): Promise<APIGatewayProxyResult> => {
event.Records.forEach(async record => {
if (record.eventName == "REMOVE") {
_processRecord(record.dynamodb.OldImage).promise();
}
});
};
async function _processRecord(oldImage) {
const parameters = {
Bucket: process.env.BUCKETNAME,
Key: oldImage.propertyId.S
};
try {
s3.headObject(parameters).promise();
console.log('File located in S3');
try {
s3.deleteObject(parameters).promise();
console.log('File deleted from S3');
}
catch (error) {
console.log("ERROR in file Deleting : " + JSON.stringify(error));
}
} catch (error) {
console.log("File not Found ERROR : " + error.code)
}
}
Everything seems fine until I get to the S3 section. When I invoke the function I get a 202 response which all looks fine, but the files are not being deleted when I check in S3. I've tried adding in a version to the parameters but that doesn't seem to work either. Any help would be greatly appreciated.
When you delete and object from an S3 bucket which has versioning enabled, the object is not permanently deleted. Instead the latest version is specially marked as deleted. Any get object issued to that object will return as if the object has been deleted.
To permanently delete an object in a versioned bucket you must delete all the versions of that object.
You can find more detail on the Deleting object versions docs.
I've done some more digging and was able to figure out where things were going wrong. I think my main issue was actually using foreach in the async function. Replacing that with for (const record of Records) got things running smoothly.
So I am making a local App using Javascript , React and Electron and I want it to be able to work just fine without internet.
I can't use 'localStorage' because the data might get deleted if the user deletes the cache.
I tried reading/writing using differant Modules, none of them worked mostly because of CROS. Using XMLHTTPrequests and Ajax doesn't work either and am running out of time.
When I use them on the test server, they return the index.html for the mainpage (They can at least access that ... and still they can't read the data) but when I try it on the build I get CORS the error.
My Idea for now is to enable CORS on my webpage since I have no worries about security : The App will run ONLY offline so there is no danger.
But After many hours...I didn't find a solution to do it on the client side.
If anyone has an idea or suggestion I would be grateful.
I tried : fs,FileReader,FileSaver, $.ajax,XMLHTTPrequests
//using $ajax
var test = $.ajax({
crossDomain:true,
type: 'GET',
url:'../data/DefaultCategorie.txt',
contentType: 'text/plain',
success: function(data){
console.log(data);
},
error: function(){
alert('failed');
},
})
//using fs
fs.readFile('../data/DefaultCategorie.txt', 'utf8', (err, data) => {
if (err) {
console.log("Failed");
throw err
}
console.log(data);
fs.close(data, (err) => {
if (err) throw err;
});
});
This article covers the 3 most common ways to store user data: How to store user data in Electron
The Electron API for appDatadoes what you want. It is very easy to use.
From the above article:
const userDataPath = (electron.app || electron.remote.app).getPath('userData');
this.path = path.join(userDataPath, opts.configName + '.json')
this.data = parseDataFile(this.path, opts.defaults);
function parseDataFile(filePath, defaults) {
try {
return JSON.parse(fs.readFileSync(filePath));
} catch(error) {
// if there was some kind of error, return the passed in defaults instead.
return defaults;
}
}
Docs
app.getPath(name)
name String
Returns String - A path to a special directory or file associated with
name. On failure, an Error is thrown.
You can request the following paths by the name:
appData - Per-user application data directory, which by default points to:
%APPDATA% on Windows
$XDG_CONFIG_HOME or ~/.config on Linux
~/Library/Application Support on macOS
userData - The directory for storing your app's configuration files,
which by default it is the appData directory appended with your app's
name.
I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.
I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.
I have followed many solutions provided in the previous questions but mine is not working. The problem is in .json extension. Whenever I use filename.json, the app will crash with ERR_CONNECTION_RESET but successfully created an empty .json file. However, if I change the extension to filename.txt, the fs.writeFile will successfully create the filename.txt with the data inside and the app will work as expected. Did I miss any configuration here to create the JSON file?
Here is the example code I used.
var jsonData = '{"persons":[{"name":"John","city":"New York"},{"name":"Phil","city":"Ohio"}]}';
// parse json
var jsonObj = JSON.parse(jsonData);
console.log(jsonObj);
// stringify JSON Object
var jsonContent = JSON.stringify(jsonObj);
console.log(jsonContent);
fs.writeFile("./public/output.json", jsonContent, 'utf8', function(err) {
if (err) {
console.log("An error occured while writing JSON Object to File.");
return console.log(err);
}
console.log("JSON file has been saved.");
});
So, ERR_CONNECTION_RESET means that the connection was closed midway. My guess, as in the comments, would be that it's a reloading server.
Try using --ignore public/**/*.json and it should work.
I am wondering how to parse a result after a query in azure using Easy API in Azure? For me, it seems that creating an objet inside the function(results) would lead to an endless loop and in the end, return httpcode 500.
Please help. There must be a way to parse the result, store it in a Javascript object and then return that result as a Json.
var azureMobileApps = require('azure-mobile-apps');
var queries = require('azure-mobile-apps/src/query');
var api = {
get: (req, res, next) => {
if(Object.keys(req.query).length === 0) {
res.status(400).type('text/plain').send("Error! Please add event id");
return;
}
if(req.query.eventId === 'undefined' || req.query.eventId.length === 0) {
res.status(400).type('text/plain').send("Error! missing eventId parameter");
console.log("worked!");
return;
}
var query = {
sql: 'Select .... where eventId=#eventId'
,
parameters: [
{ name: 'eventId', value: req.query.eventId }
]
};
req.azureMobile.data.execute(query)
.then(function (results) {
TODO: here! Parse results, add properties to objects and then return that instead!
res.status(200).type('application/json').send({sessions: results});
},function(error){
console.log('Found an error: ', error);
});
}
};
api.get.access = 'authenticated';
module.exports = api;
Your code snippet don't have any problem. In node.js backend Mobile Apps, the req.azureMobile.data.execute(query) returns an array list of queried results. Here is a code sample on GitHub https://github.com/Azure/azure-mobile-apps-node/blob/master/samples/custom-api-sql-stmt/api/completeall.js provided by Azure Dev Team.
And Mobile App in Node.js is based on Expressjs framework. You can simply use res.json(obj) to directly return a json response.
Separately for your issue, usually, the stmt exception will raise your issue. Please double check your query object whether it is correct before it querying data.
You can use Visual Studio Team Services (was Visual Studio Online) for troubleshooting of nodejs backend Mobile Apps.
Any further concenr, please feel free to let me know.