I have an pinterest like application. Images and other related information are stored in MongoDb. Generally size of images is about 1 mb. Images are displaying with infinite scroll. When long script with base64 string is loaded, browser crashes or response time is really high (especially for Internet Explorer)
What is the best way to display images that are stored in MongoDb?
I think the best way to achieve this, is to have you file physically in some public folder on your server. This should be accesible in a way that you will only need to use something like
http://www.myhost.com/images/my/path/to/image.jpg
You can still maintain your Base64 image in mongodb as backup, however, this is not the best way to retrieve you images due performance issues (as you have seen). I recommend you to do the following:
Each time you store the image on mongo, be sure to also store the "image file" as itself on some public place on your server. Have in mind that you should keep the path to that file on the mongo model you are using. So, the next time you call the object, rather than get the base 64 image, you should only get the path to the image.
Lets say, you have this model
myModel = {
name: "some name",
image64: "someextralongstringveryveryveryweird......",
imageUrl: "/images/my/path/to/image/imagename-id.jpg"
}
the next time you query on it, you can just ignore the image64 using mongo projection, and in you client side you just use some html tag that makes use of that url.
<img src="/images/my/path/to/image/imagename-id.jpg">
This will help you lots on performance.
There are some libraries that could help you to manage the image file creation. ImageMagick is one that I have used and is so versatile.
I guess you have some server side part of this application? Why don't you create a tiny API to retrieve images?
So your browser will have information about image and can ask your server for it, something in line of http://your_server/api/image/imageID or http://your_server/images/imagename and then your server would just stream this image, you don't need to store this in the file system.
On the client side (browser) you just need to implement 'lazy loading'.
If you're using MongoDB, you should be storing images in your database using GridFS (http://excellencenodejsblog.com/gridfs-using-mongoose-nodejs/), a feature which exposes something like a virtual filesystem for your application.
Using GridFS, you could write a controller method which streams a requested file from your MongoDB instance and pipes the file content to the response.
I would recommend storing the images on the filesystem and using your web server to handle serving them to clients.
For performance, I would put them on a CDN - that will be able to handle the traffic.
In your application storage (mongo), you can store a URL/location to the image and then use that when retrieving the image in your javascript code.
In addition, in your code I would recommend preloading images via javascript that preloads images before the user has scrolled to see them. There are some great tools out there that you can leverage for that.
In the off chance that you cannot change the storage and you have to use mongo the way it is - I would look at preloading images with javascript.
I was having the same issue. So i used a mongodb to store my images.
This is how I proceeded:
Define a logo schema:
var logoSchema = new mongoose.Schema({
name: String,
url: String
});
Compile the logo schema into a model:
var Logo = mongoose.model("Logo", logoSchema)
Creating a new logo:
var rectblack = new Logo({
name:"rect&black",
url:"public/image.jpg"
});
Saving it :
rectblack.save(function(err, logo){
if(err){
console.log("some went wrong")
} else {
console.log("logo saved")
console.log("logo")
}
});
Now to use this logo or image I just called it with the image tag (just the file name!!!):
<img src="/image.jpg">.
Related
I have found weird bug in my application. I am using VueJS3 and typescript. I am creating a media gallery, where users can manage assets. All endpoints and backend side of application works perfectly. Currently i am working on updating image files(updating image field). My code works in theory, image url updates to a new one, but even the url is newer, image that is rendered is old image(before update). When i try to view my application in incognito window, my newer image is showing. But it does not work otherwise(works if and only user clears his cache) I hope i explained myself clearly, it is a little complicated :)
When i fetch the data from api, they don't have any src, so i have to create a property when using the data. Here is the code below.
const { res, err } = await this.$api.image.getAll(params)
res.data.items.forEach(async (image: Image<Populated>) => {
image.src = this.$fileHandler(image._id)
})
Note: File handler is a special method that gives the image source by id
Apparently your assets get cached, and there are multiple ways to solve that.
Not diving too much into caching strategies and cache control tags, just do
image.src = `${this.$fileHandler(image._id)}?_=${+new Date()}`
This will generate a unique (almost) URL for your image, invalidating every previous cached one.
I know the traditional way is to store image/video file in one place and then just save the reference index into db's table.
Now I am learning about gundb, I can store key-value json type data very easily, but since it is decentralized, if I wanna make a say chatroom app, how should I handle image storing (eg: user's avatar)?
I am also wondering if it is possible to make a movie-sharing app using gundb?
#Retric, great question! I'm not sure why people are downvoting you, they must be haters.
You are right, it is better to store that image/video and reference it via GUN. For videos specifically, WebTorrent/BitTorrent has done P2P video sharing for over a decade now and at one point handled 40% of the world's internet traffic!
However WebTorrent/BitTorrent is not very good with discovering/sharing those URIs (magnet links, etc.) but GUN is. So I'd recommend that as one option.
For images, especially small ones like avatars/icons/profiles, I do often store them in GUN directly by Base64 encoding them (many websites around the world inline images/icons/sprites/avatars into CSS files with base64 data-URLs, except now you could use GUN for this).
If you are interested in this, I wrote a small utility using jQuery that lets you drag&drop images into your website, and it'll auto-resize (pass options to overwrite it) and base64 encode it for you to then save to GUN:
https://github.com/amark/gun/blob/master/lib/upload.js
Here is a small example of how I use it:
$('#profile').upload(function resize(e, up){
if(e.err){ return } // handle error
$('#profile').addClass('pulse'); // css to indicate image processing
if(up){ return up.shrink(e, resize, 64) } // pass it `e` drag&drop/upload event, then I reuse the current function (named resize) as the callback for it, and tell it resize to 64px.
$('#profile').removeClass('pulse'); // css indicate done processing.
$("#profile img").attr('src', e.base64).removeClass('none'); // set photo in HTML!
gun.user().get('who').get('face').get('small').put(e.base64); // save profile thumbnail to GUN
});
Finally, what about storing videos in GUN if you don't want to use BitTorrent?
I would highly recommend using the HLS format to store videos in GUN, this would allow you to do decentralized realtime video streaming. It is a beautifully simple format that allows video streaming to work even from static files, because it stores the video in small chunks that can be streamed - which fits perfectly with GUN.
There already is a JS based video-player for the HLS format:
https://github.com/video-dev/hls.js/
Based off the demo page, you can see an example of how the video is stored, like here on GitHub:
https://github.com/video-dev/streams/tree/master/x36xhzz
(if you click on the m3u8 file, you'll see it has metadata that 720p is stored in url_0 folder, which themselves have sub-files)
Rather than storing the HLS video files on BitTorrent or a centralized server, you could store it in GUN using the same folder structure gun.get('videos').get('x36xhzz').get('url_0').get('url_496').get('193039199_mp4_h264_aac_hd_7.ts').once(function(video_chunk){ passToHLSplayer(video_chunk) }) such that it would be easy for HLS.js to integrate with GUN.
Now you'll have P2P decentralized video streaming!!!
And even cooler, you can combine it with GUN's lib/webrtc adapter and do this fully browser to browser!
I hope this helped.
The thing to understand here is the difference between content addressed space (frozen space) and user space in gun.
Let's say you have some media encoded as base64, and you know its content type (I've used text here to keep example short, but you could use image video etc):
// put avatar in frozen space:
let media = JSON.stringify({ b64 : "U2hlIHdhcyBib3JuIGFuIGFkdmVudHVyZXIuLi4=", type : "text/plain"})
// get hash of stringified media obj using gun's SEA lib:
let mediaID = await SEA.work(media, null, null, {name: "SHA-256"});
// put media in hash-addressed gundb
gun.get('#').get(mediaID).put(media,(r)=>console.log('Media put acknowledged?',r))
For a hypothetical chat app, you could use user space and put the media under the "avatar" name:
// put avatar in user space:
let user = await SEA.pair();
await gun.user().auth(user)
gun.get('~' + user.pub).get('avatar').put('#' + mediaID)
// retrieve a user's avatar
gun.get('~' + usera.pub).get('avatar').once((hashid,k)=>{
gun.get('#').get(hashid).once(media=>{
console.log("Got user's avatar :-)",media)
//do something with media
})
})
I am using redactor plugin with meteorjs, to format the texts and add images, videos and links.Redactor supports uploading images as a web link, but here I wanted to add a button to upload images from local machine. For this I have added the button to the redactor. It is showing the browse image selection and is working.
$('.editor').redactor({
imageUpload:"/upload",
imageUploadCallback: function(image, json)
{
console.log(image);
console.log(json);
},
imageUploadErrorCallback: function(json)
{
console.log(json.error);
}
});
When I print console.log("#redactor_file").val()); it gives the name of the file. But, when I do console.log(console.log(html);) its gives <p><img src="undefined"></p>. Can I get the link or the url of the image here. If yes then how?
HTTP.methods({
'upload': function(data) {
console.log(data)
return JSON.stringify( {'data':"data"})
}
});
When I do console.log(data) using the above code it shows me Image in String format. Now I want to store the Image using GridFS. Or is there any better way to store Image in MongoDB. If yes then tell me or guide me to store image using GridFS ?
While Redactor provides a client-side method for uploading files, it does not provide a server-side method for receipt of uploaded files. Also, stock Meteor doesn't provide such a method either.
So you have a few choices. First is through using a router such as Iron Router, which you could create a server-side /uploads route, and Iron Router gives you access to the request, response objects. Which might work, but because this is no longer over DDP, you don't have access to Meteor.userId, etc. See How to respond server-side to routes using Meteor and Iron-Router?
A more Meteor-like way of doing the file upload is Meteor File or CollectionFS though I'm not sure how simply these would integrate with Redactor. You could definitely get them to work together, just off the top of my head I don't know how easy it would be.
OK, this one is rather complicated to explain, which explains the verbose title:
In my Objective C application I generate a JSON string to hold all of my properties for the objects I need to draw in the ARchitect browser of the wikitude SDK (as far as I know the Wikitude SDK only handles JSON) via:
NSString *javaScript = [self convertPoiModelToJson:self.poiData.pois];
NSString *javaScriptToCall = [NSString stringWithFormat:#"newData('%#')", javaScript];
One particular object I am interested in is stored as a string in that JSON string, it's the URL to an image. However, this image is behind a password protected area on our webserver and the app handles the authentication.
The problems start when I am in the ARchitect Browser, which is basically a .html file with calls to specific wikitude javascript functions to build the augmented reality world and show it in a UIWebView in the app. I want to show that image when a POI in the augmented view is clicked in the footer popup which is a basic html div container. So now I have a URL to an image resource on the webserver, which I cannot directly access with
document.getElementById("thumb").src = jsonObject.thumbUrl;
because of the authentication needed and the only way I was successful to load that image was via the var poiImage = new AR.ImageResource(jsonObject[i].iconURL, {onError: errorLoadingImage}); method but then I can only display it in the augmented view but not in the footer.
I tried it with providing a static string from some other image in the web or to local resources to the img element in the footer section in the view without problems like that: document.getElementById("thumb").src="marker.png"; and it works fine, also the image is correctly loaded in the augmented view.
I have read about encoding the image (which I can access and download in the objective c part of the app) in base64 and storing that string in an additional JSON property to load it into the src property of the html img element with a <img src="data:image/png;base64,BASE&$_ENCODED_DATA"></img> but this seems like a really dirty and overly impractical workaround/hack for what I try to accomplish. Also I don't know if it's a better idea to start reading about how to implement the authentication to access that image in the protected area of the webserver or rather begin implementing the ugly base64 encoding or continue searching for alternatives.
I'm not asking for a solution but rather for suggestions what possibilities I have left to access that image. Since I ran out of ideas, any help is appreciated.
Thank you!
Short summary:
image accessible and downloaded in objective c part
image is accessible with the AR.ImageResource method of the wikitude SDK (but not needed)
image cannot be accessed directly via the url from javascript because authentication is needed
(I hope my question is comprehensible, feel free to ask If something is unclear, especially since English is not my first language and it would be even complicated to explain that in German..)
Have you tried downloading the image in Objective-C / Cocoa, store it locally on your iOS device and pass the local path of the image via JSON into your Architect World?
You can then load the local image with your AR.ImageResouce and your div container.
Ok, let's think a bit about this, as it seems like a good mess.
If I'm understanding well your problem, all is about your javascript code needing to reach a url where an image is located, being unable to do so because that place requires some sort of validation to access.
In this scenario I'd look for moving images to a place out of the restricted site. Maybe trying to save them from the objective-C part into one public place that's reachable by javascript. This may be tricky as the I/O operation could slow your code execution... Of course you'll always have the option to move those images just outside of the restricted area but I supose that's not feasible as you could suposed that solution by yourself.
Other way... if your environment configuration allows it (I fear there isn't enough info about it on the question) is to try to execute your javascript part (I wonder if it's enclosed into a webpage executed from a webserver) with a user with permissions into the restricted area. This, of course, could be totally a no-no depending on what your javascript part does and why your restricted area is protected by validation.
If you don't want to move out the files, copy them temporarily gives performance problems and errors, and user impersonation through the javascript executing user is not an option I fear your unique alternative is to efectively try to pass a binary stream of data with the image through the JSON string... dirty...
Looks like wikitude maintains 2 browser contexts: the host browser that loaded your HTML and an embedded browser-like object (or iframe or proxy server) represented by the UIWebView instance.
Only one of those (not clear which from your discussion) has the user/pass for access to your image. You will need to call something to repeat the authentication step or transfer credentials to the other context.
I am new to HTML/Javascript, as well as coding in general so bear with me :). I am trying to create a "Spot the Difference" game in html5 using javascript. Everything is local (on my machine). I have two pictures, of the same size, one with differences. To generate data about the clickable fields, I have a java program that reads both of the images and outputs all of the positions in which pixels are different into a XML file. My plan was to then use this XML file with my javascript to define where the user could click. However, it appears (correct me if I'm wrong) that javascript cannot read local XML files for security reasons. I do not want to use an ActiveXObject because I plan on putting this onto mobile devices via phone gap or a webkit object. Does anyone have a better approach to this problem, or perhaps a way to read local XML files via javascript? Any help would be greatly appreciated, thanks.
If you are planning to put this into a smart phones (iOS and Android) and read local files, I have done similar things with JSON (yes, please don't use XML).
Convert your output to JSON
Put this as part of your application package. For example, in Android, I put it as part of the .apk in /appFiles/json
Create a custom content provider that would read the local file. I create mine as content:// but you create whatever scheme you want. You could leverage android.content.ContentProvider in order to achieve custom URL Scheme. iOS has its own way to create custom scheme as well. The implementation simply read your local storage and give the content
To read it from Javascript, I simply call ajax with the custom scheme to get the json file. For example content://myfile/theFile.json simply redirect me to particular directory in local storage with /myfile/theFile.json appended to it
Below is the sample to override openFile() in the ContentProvider
public ParcelFileDescriptor openFile (Uri uri, String mode) {
try {
Context c = getContext();
File cacheDir = c.getCacheDir();
String uriString = uri.toString();
String htmlFile = uriString.replaceAll(CUSTOM_CONTENT_URI, "");
// Translate the uri into pointer in the cache
File htmlResource = new File(cacheDir.toString() + File.separator + htmlFile);
File parentDir = htmlResource.getParentFile();
if(!parentDir.exists()) {
parentDir.mkdirs();
}
// get the file from one of the resources within the local storage
InputStream in = WebViewContentProvider.class.getResourceAsStream(htmlFile);
// copy the local storage to a cache file
copy(in, new FileOutputStream(htmlResource));
return ParcelFileDescriptor.open(htmlResource, ParcelFileDescriptor.MODE_READ_WRITE);
} catch(Exception e) {
throw new RuntimeException(e);
}
}
I hope it helps
I would suggest modifying your java program to output a JSON formatted file instead of XML. JSON is native to JavaScript and will be much simpler for you to load.
As for actually loading the data, i'm not sure what the best option is as you say you want to evenutally run this on a mobile device. If you were just making a normal website you could setup a web server using either Apache or IIS depending on your OS and put the files in the document root. Once you've done that you can load the JSON file via Ajax, which can easily be googled for.
Not sure if this helps any.
Since this is a local file, you can do this with jQuery
$.ajax({
type: "GET",
url: "your.xml",
dataType: "xml",
success: function(xml){
///do your thing
}
});
http://api.jquery.com/jQuery.ajax/