I'm coding an app for a device, such device will receive a POST request, and send back a multipart/x-mixed-replace binary data stream. I must display such stream on one section of my app's home page.
I searched through, there's very limited resource on such case. So far, I found that if the Motion JPEG is sent from a specific URL, then maybe it's possible to use iframe/img tag to display it. However, my case is different, it seems I have to parse such binary stream then create an Observable to alter the img element on an image html tag once I get a frame from the Motion JPEG binary stream.
Is there a simpler way to do that ? I found https://gist.github.com/legege/5301477, can I use this ?
I actually solved this by myself, and learned a lot in the research.
The core idea is to use a xmlHTTP request to fetch the motionJPEG data, It's about how to transfer binary data.
Then use a web worker to process the binary data.
And finally use canvas to draw the image on ionic page.
Due to the front end JS delay on image loading, such motionJPEG preview would not be so smooth currently in my implementation. But this is possible now with JS.
Please check github https://github.com/makoto-unity/ThetaWifiStreaming
Related
I'm Developing a WordPress plugin for customers to design custom T-Shirts, with the option of uploading their own images. The plugin takes several screenshots and emails them to a print department.
In Javascript I convert the screenshots to base64 data, which is then sent via Ajax to a PHP file, this creates a folder for the customer's design, creates the images from the data and stores the screenshots in there.
Most screenshots/base64 data send across just fine, for example just adding in text creates no problems. However if the user uploads an image and it's scaled up too much it causes various errors (sometimes 400 error, sometimes 404 and sometimes 500).
Running this through my local setup on Windows with Wamp, it's fine. I can upload images and scale them to 12x with no issues. However when I try this with the live site, I get the above problems if I scale any of the images past 4x, and with most images this happens if I even try to scale them up at all past 1x.
The resolution/file size of the image seems to have an effect, though not in an obvious way. I can send a huge plain red square, or normal image at 1x scale.
At first I thought this was a POST data limit issue, except the live site's POST limit is double that what I had set on my WAMP setup, which doesn't have this problem.
Also, and even stranger. I tested uploading the image but replacing the base64 data with simple characters (so the scaled up image exists in the page but it's base64 data isn't sent via POST), and I still have the same issue. So I don't think it's a simple POST limit issue.
Cannot for the life of me find a solution to this, any help would be hugely appreciated.
Figured a way round it, I'll give my solution in case anyone else has the same issue and comes across this post.
Basically I converted my base64 image data to a blob, and appended that to a newly created formData object. I found that also appending my nonce and action (amended to work with the admin-ajax way of using AJAX) to the formData helped deal with most issues on the javascript side. In the AJAX request I set processData and contentType to false.
As for the PHP side, I set a variable equal to the specific $_FILES array element I just sent. I used file_get_contents() on that variable (i.e the blob data), and wrapped that in file_put_contents() to actually write the image.
That's the quick version. If anyone wants a more detailed explanation let me know.
I know the traditional way is to store image/video file in one place and then just save the reference index into db's table.
Now I am learning about gundb, I can store key-value json type data very easily, but since it is decentralized, if I wanna make a say chatroom app, how should I handle image storing (eg: user's avatar)?
I am also wondering if it is possible to make a movie-sharing app using gundb?
#Retric, great question! I'm not sure why people are downvoting you, they must be haters.
You are right, it is better to store that image/video and reference it via GUN. For videos specifically, WebTorrent/BitTorrent has done P2P video sharing for over a decade now and at one point handled 40% of the world's internet traffic!
However WebTorrent/BitTorrent is not very good with discovering/sharing those URIs (magnet links, etc.) but GUN is. So I'd recommend that as one option.
For images, especially small ones like avatars/icons/profiles, I do often store them in GUN directly by Base64 encoding them (many websites around the world inline images/icons/sprites/avatars into CSS files with base64 data-URLs, except now you could use GUN for this).
If you are interested in this, I wrote a small utility using jQuery that lets you drag&drop images into your website, and it'll auto-resize (pass options to overwrite it) and base64 encode it for you to then save to GUN:
https://github.com/amark/gun/blob/master/lib/upload.js
Here is a small example of how I use it:
$('#profile').upload(function resize(e, up){
if(e.err){ return } // handle error
$('#profile').addClass('pulse'); // css to indicate image processing
if(up){ return up.shrink(e, resize, 64) } // pass it `e` drag&drop/upload event, then I reuse the current function (named resize) as the callback for it, and tell it resize to 64px.
$('#profile').removeClass('pulse'); // css indicate done processing.
$("#profile img").attr('src', e.base64).removeClass('none'); // set photo in HTML!
gun.user().get('who').get('face').get('small').put(e.base64); // save profile thumbnail to GUN
});
Finally, what about storing videos in GUN if you don't want to use BitTorrent?
I would highly recommend using the HLS format to store videos in GUN, this would allow you to do decentralized realtime video streaming. It is a beautifully simple format that allows video streaming to work even from static files, because it stores the video in small chunks that can be streamed - which fits perfectly with GUN.
There already is a JS based video-player for the HLS format:
https://github.com/video-dev/hls.js/
Based off the demo page, you can see an example of how the video is stored, like here on GitHub:
https://github.com/video-dev/streams/tree/master/x36xhzz
(if you click on the m3u8 file, you'll see it has metadata that 720p is stored in url_0 folder, which themselves have sub-files)
Rather than storing the HLS video files on BitTorrent or a centralized server, you could store it in GUN using the same folder structure gun.get('videos').get('x36xhzz').get('url_0').get('url_496').get('193039199_mp4_h264_aac_hd_7.ts').once(function(video_chunk){ passToHLSplayer(video_chunk) }) such that it would be easy for HLS.js to integrate with GUN.
Now you'll have P2P decentralized video streaming!!!
And even cooler, you can combine it with GUN's lib/webrtc adapter and do this fully browser to browser!
I hope this helped.
The thing to understand here is the difference between content addressed space (frozen space) and user space in gun.
Let's say you have some media encoded as base64, and you know its content type (I've used text here to keep example short, but you could use image video etc):
// put avatar in frozen space:
let media = JSON.stringify({ b64 : "U2hlIHdhcyBib3JuIGFuIGFkdmVudHVyZXIuLi4=", type : "text/plain"})
// get hash of stringified media obj using gun's SEA lib:
let mediaID = await SEA.work(media, null, null, {name: "SHA-256"});
// put media in hash-addressed gundb
gun.get('#').get(mediaID).put(media,(r)=>console.log('Media put acknowledged?',r))
For a hypothetical chat app, you could use user space and put the media under the "avatar" name:
// put avatar in user space:
let user = await SEA.pair();
await gun.user().auth(user)
gun.get('~' + user.pub).get('avatar').put('#' + mediaID)
// retrieve a user's avatar
gun.get('~' + usera.pub).get('avatar').once((hashid,k)=>{
gun.get('#').get(hashid).once(media=>{
console.log("Got user's avatar :-)",media)
//do something with media
})
})
I have an pinterest like application. Images and other related information are stored in MongoDb. Generally size of images is about 1 mb. Images are displaying with infinite scroll. When long script with base64 string is loaded, browser crashes or response time is really high (especially for Internet Explorer)
What is the best way to display images that are stored in MongoDb?
I think the best way to achieve this, is to have you file physically in some public folder on your server. This should be accesible in a way that you will only need to use something like
http://www.myhost.com/images/my/path/to/image.jpg
You can still maintain your Base64 image in mongodb as backup, however, this is not the best way to retrieve you images due performance issues (as you have seen). I recommend you to do the following:
Each time you store the image on mongo, be sure to also store the "image file" as itself on some public place on your server. Have in mind that you should keep the path to that file on the mongo model you are using. So, the next time you call the object, rather than get the base 64 image, you should only get the path to the image.
Lets say, you have this model
myModel = {
name: "some name",
image64: "someextralongstringveryveryveryweird......",
imageUrl: "/images/my/path/to/image/imagename-id.jpg"
}
the next time you query on it, you can just ignore the image64 using mongo projection, and in you client side you just use some html tag that makes use of that url.
<img src="/images/my/path/to/image/imagename-id.jpg">
This will help you lots on performance.
There are some libraries that could help you to manage the image file creation. ImageMagick is one that I have used and is so versatile.
I guess you have some server side part of this application? Why don't you create a tiny API to retrieve images?
So your browser will have information about image and can ask your server for it, something in line of http://your_server/api/image/imageID or http://your_server/images/imagename and then your server would just stream this image, you don't need to store this in the file system.
On the client side (browser) you just need to implement 'lazy loading'.
If you're using MongoDB, you should be storing images in your database using GridFS (http://excellencenodejsblog.com/gridfs-using-mongoose-nodejs/), a feature which exposes something like a virtual filesystem for your application.
Using GridFS, you could write a controller method which streams a requested file from your MongoDB instance and pipes the file content to the response.
I would recommend storing the images on the filesystem and using your web server to handle serving them to clients.
For performance, I would put them on a CDN - that will be able to handle the traffic.
In your application storage (mongo), you can store a URL/location to the image and then use that when retrieving the image in your javascript code.
In addition, in your code I would recommend preloading images via javascript that preloads images before the user has scrolled to see them. There are some great tools out there that you can leverage for that.
In the off chance that you cannot change the storage and you have to use mongo the way it is - I would look at preloading images with javascript.
I was having the same issue. So i used a mongodb to store my images.
This is how I proceeded:
Define a logo schema:
var logoSchema = new mongoose.Schema({
name: String,
url: String
});
Compile the logo schema into a model:
var Logo = mongoose.model("Logo", logoSchema)
Creating a new logo:
var rectblack = new Logo({
name:"rect&black",
url:"public/image.jpg"
});
Saving it :
rectblack.save(function(err, logo){
if(err){
console.log("some went wrong")
} else {
console.log("logo saved")
console.log("logo")
}
});
Now to use this logo or image I just called it with the image tag (just the file name!!!):
<img src="/image.jpg">.
Is there a reason PNG and JPG images would be embedded in a JavaScript file like this:
// Template/Image data
var LOGO = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADgAAAA4 etc";
var BACKGROUND = "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEASABIA etc";
If I remove these portions and call files stored on a server instead, will there be a performance penalty or something? The only thing I can think of is Apache serving extra requests for those images, but I'm not even sure it works that way. Is there anything else?
Its mostly convenience and to avoid preloading the images. Since no additional requests have to be sent to the server to display the image, the image will be displayed as soon as you set this value to the src attribute.
In terms of amount of data downloaded, this technique avoids the extra overhead of additional requests but the total sizes downloaded could be a bit larger since the entire image data is encoded in base64. In applications where you might have lots of such images preloading could be a better.
A request to that image file takes longer than showing the image from the binary. So your page has saved some requests :)
First, a little background:
I apologize ahead of time for the long-winded nature of this preface; however, it may assist in providing an alternate solution that is not specific to the nature of the question.
I have an ASP.NET MVC application that uses embedded WinForm UserControls. These control provide "ink-over" support to TabletPCs through the Microsoft.Ink library. They are an unfortunate necessity due to an IE8 corporate standard; otherwise, HTML5 Canvas would be the solution.
Anyway, an image URL is passed to the InkPicture control through a <PARAM>.
<object VIEWASEXT="true" classid="MyInkControl.dll#MyInkControl.MyInkControl"
id="myImage" name="myImage" runat="server">
<PARAM name="ImageUrl" value="http://some-website/Content/images/myImage.png" />
</object>
The respective property in the UserControl takes that URL, calls a method that performs an HttpWebRequest, and the returned image is placed in the InkPicture.
public Image DownloadImage(string url)
{
Image _tmpImage = null;
try
{
// Open a connection
HttpWebRequest _HttpWebRequest = (HttpWebRequest)HttpWebRequest.Create(url);
_HttpWebRequest.AllowWriteStreamBuffering = true;
// use the default credentials
_HttpWebRequest.Credentials = CredentialCache.DefaultCredentials;
// Request response:
System.Net.WebResponse _WebResponse = _HttpWebRequest.GetResponse();
// Open data stream:
System.IO.Stream _WebStream = _WebResponse.GetResponseStream();
// convert webstream to image
_tmpImage = Image.FromStream(_WebStream);
// Cleanup
_WebResponse.Close();
}
catch (Exception ex)
{
// Error
throw ex;
}
return _tmpImage;
}
Problem
This works, but there's a lot of overhead in this process that significantly delays my webpage from loading (15 images taking 15 seconds...not ideal). Doing Image img = new Bitmap(url); in the UserControl does not work in this situation because of FileIO Permission issues (Full trust or not, I have been unsuccessful in eliminating that issue).
Initial Solution
Even though using canvas is not a current option, I decided to test a solution using it. I would load each image in javascript, then use canvas and toDataUrl() to get the base64 data. Then, instead of passing the URL to the UserControl and have it do all the leg work, I pass the base64 data as a <PARAM> instead. Then it quickly converts that data back to the image.
That 15 seconds for 15 images is now less than 3 seconds. Thus began my search for a image->base64 solution that worked in IE7/8.
Here are some additional requirements/restrictions:
The solution cannot have external dependencies (i.e. $.getImageData).
It needs to be 100% encapsulated so it can be portable.
The source and quantity of images are variable, and they must be in URL format (base64 data up front is not an option).
I hope I've provided sufficient information and I appreciate any direction you're able to give.
Thanks.
You can use any of the FlashCanvas, fxCanvas or excanvas libraries, which simulate canvas using Flash or VML in old internet explorer versions. I believe all of these provide the toDataURL method from the Canvas API, allowing you to get an encoded representation of your image.
After extensive digging around (I'm in the same fix) I believe this is the only solution, short of writing a PHP script that you can send the image to. The problem with that of course is that there isn't a way to send images to the PHP script unless any of these three conditions is true:
The browser supports typed arrays (Uint8Array)
The browser supports sendAsBinary
The image is being uploaded by someone via a form (in which case it can be sent to a PHP script that responds with the base 64 encoding)