I tried upload file to Heroku using https://www.npmjs.com/package/express-fileupload, on my PC it works great, but on Heroku there are this error:
{"errno":-2,"code":"ENOENT","syscall":"open","path":"./public/owner_photo/f28397baea8fb4d6f6dafed9f5586a9ac0b46843acf1120a0ecad24755cfff57.jpeg"}
How can I fix it?
Heroku has an immutable file system, meaning you can't make changes to, or additions to the file system. You'll need to store your uploads somewhere else, like Amazon S3.
Also, many upload packages by default store the uploaded file in a temp directory. So even if you are sending them to S3, you'll still need to make sure the methods you use don't attempt to do that, or set an option to disable it. I'm not familiar with express-fileupload so I can't say what methods do or do not attempt to store copies on the filesystem.
I have successfully implemented this using multiparty so I could be of more specific help with that package.
Related
I am using the s3 bucket to serve the frontend app. And I am syncing code in that s3 bucket from the Jenkins server. While building the code in every deployment. Some files get updated some files get renamed, some files get deleted as well in the new folder on the Jenkins server.
Now when we use the s3 sync command on that build folder which was just created and has all the new files created in it. It basically pushes all the code to s3 and doesn't touch the code for example:- the test-123.jpg image which was been pushed in an earlier build. But has been deleted in the recent build.
This kind of file is renamed and all are not getting synced so they are just staying on the server.
May I know which is the ideal way to deal with this kind of issue while pushing the code to S3. My main motive is that I should be able to automatically delete the files which are not getting used or served anymore in the angular app.
I'm learning Node and Git and I have a Heroku app that is reading and writing to a local file on the server (a very simple JSON database).
If I add the file to my gitignore locally, it disappears from my Heroku app and causes the app to error. But if I don't add it to my gitignore, it overwrites the latest version (on the server) with an old one I have locally.
Obviously the issue is because the changes on the server file aren't being committed. However, I don't know how to do that remotely, or if it's even possible. I can run heroku git:clone locally, but I can't run heroku:git add.
How do I handle this?
Generally, you should not commit a file that will be modified by the server.
It seems not a good idea because, as you said, this file will be overwrited by next push.
Usually you do not want to commit from your deployment branch, so it is not a good idea either to use git from server (and I doubt you can with Heroku).
Instead you could make your app check if the file exists and if not create that file on server.
That will work in a dedicated server you manage yourself, but Heroku doesn't work the same. Each push you make to your Heroku repository will in fact bundle your application before launching it on a dyno, and this process overwrite all the file, including your database JSON file, which will be no more persistent.
So I think you have no choice than switch to another storage method, for exemple subscribe a free Heroku postgreSQL plan or another database you prefer.
I have a nodejs application using hapi.js and I'm trying to download an image from a url. Heroku is giving me errors with the pathing.
My code:
Request(uri).pipe(fs.createWriteStream(__dirname+'/../public/images/'+filename)).on('close', callback);
My errors:
Error: ENOENT: no such file or directory, open '/app/../public/images/1430540759757341747_4232065786.jpg'
My file structure is simple:
app.js
-public
-images
-sampleimage.jpg
-videos
-samplevideo.mp4
-audio
-sampleaudio.wav
As you can see the __dirname for heroku application is /app. I've tried using __dirname+'all sorts of pathing ../ ./ etc' and I've also tried it without __dirname.
I will be creating a lot of these files using ffmpeg and a speech tool. So could anyone explain to me what kind of problem I am having? Is it something that can be solved by using the correct path name or is it my hapijs server configurations that I need to configure?
You just have the wrong path in your project.
On Heroku, you can't write to the folder BELOW the root of your project.
In your case, your code is running in app.js, which is in the 'root' folder of your project.
So, on Heroku's filesystem, this means your project looks like this:
/app
/app/app.js
/app/public
/app/public/images
...
Heroku puts all your code into a folder called app.
Now, in your code pasted above, you show:
Request(uri).pipe(fs.createWriteStream(__dirname+'/../public/images/'+filename)).on('close', callback);
If this code is running in your app.js, it means that by going BACK a folder (eg: ..), you're trying to write to a non-writable part of Heroku's filesystem.
Instead, you want to write to:
Request(uri).pipe(fs.createWriteStream(__dirname+'/public/images/'+filename)).on('close', callback);
This will correctly write your file into the images folder like you want.
HOWEVER
Here's where things are going to get complicated for a moment.
On Heroku, you can indeed write files to the filesystem, but they will DISAPPEAR after a short period of time.
Heroku's filesystem is EPHEMERAL, this means that you should treat it like it doesn't exist.
The reason Heroku does this is because they try to force you to write scalable software.
If your application writes files to your webserver disk, it won't scale very much. The reason why is that disk space is limited. Each web server has its own disk. This can lead to confusing / odd behavior where each webserver has a copy of the same file(s), etc. It just isn't a good practice.
Instead: what you should do is use a file storage service (usually Amazon S3) to store your files in a central location.
This service lets you store all of your files in a central location. This means:
You can easily access your files from ALL of your web servers.
You can have 'reliable' storage that is managed by a company.
You can scale your web applications better.
The folder you hosted on heroku is considered as "app" which you can see from the error you got. I m commenting this after 5 years just to let future viewers know. If any folder is empty, it is not pushed to github or heroku when you pushed the entire project as the folder is empty.
When we try to access a folder which is empty initially, we get the above error as the folder is not pushed in the first place. So, if you want to get rid of the error, place a temp file of any type ( I used a txt file) and push the code. Now the error won't be there anymore as this time the folder is pushed and it can access it.
I created a project, which is here: https://github.com/dartem/upload_files, and it uploads a file and saves it using FilesCollection. However, it looks like that an actual file is getting saved only temporarily in /cdn/storage and once I restart Meteor or if I open an incognito window an actual file doesn't exist.
I specify the path directory, which is assets/app/uploads/Images, but an image doesn't get saved in that directory. How can I save an actual file in that directory?
I ran your demo up, and it does save files to that directory.
It's not advisable to store files in .meteor/local - basically the files there are temporary - meteor will remove and replace them as it rebuilds the app. This explains why you can't find them again later.
It is also possible to write the files to the public directory, but that will trigger a rebuild of your app every time you save a file, which isn't a good side effect.
I would recommend that you only save the files to gridFS or AWS (or any of the other storage options). You could save them to a folder somewhere else in the file system, as long as you have a way of serving them up (some kind of web server like apache, express or whatever). Your choice based on what sysadmin capability you have.
I want to deploy a meteor app to meteor.com.
Unfortunately I have to write some tmp files to the public folder of meteor.
Example Code:
var fs = Npm.require('fs');
var filepath = "../../../../../public/resizing/tmp~";
fs.open(localpath, 'w', function(err,fd)
{
if(err) throw "error opening file";
fs.close(fd,function(){});
}
( ../../../../../public is the location of the meteor public folder after bundling! )
This works fine on my local machine, because I have write privileges inside the public folder. Is there a way how I can write to a tmp file in an app that is deployed to meteor?
Most cloud providers don't allow access to file system, because this would put too much constraints on the architecture. Meteor.com is not different. For Meteor, there's also a problem of /public dir being precached by the engine, so every change to that folder would result in server being restarted (you can see that on your local machine as well).
Whatever you're trying to achieve, there are different ways. The most common ones are:
Use an external storage system, such as Amazon S3.
Simply, write the files you need as a new collection in the database. It's obvious how to do it for text files, but as easy for binary ones - just convert them to base64.
Meteor allows people to deploy apps to meteor.com as a courtesy, but I agree that writing to the file system dynamically would be a security risk for them. You can package your app and deploy it to your own server somewhere if that's easier, but you might want to rewrite the app logic itself to keep that data somewhere other than the same location of your app (S3, for instance). I'm assuming you're trying to store images or something. If it's not, just use Meteor's data stores and keep it there.