How to dynamically find main rectangular image within an image? - javascript

I'd like to take a screenshot of an image, and extract the image out of the screenshot.
For example:
I'd like to dynamically extract that image out of the screenshot. However, I want to be able to dynamically detect where the image to be extracted is in the screenshot image. So for example, if I screenshotted an image on Instagram, I'd like to dynamically extract the image from the screenshot. So I feel like I just need to come up with a calculation to find where "the main subject" inside of the screenshot image is.
I've done some research but most of what I've found are people wanting to extract an image from a scanned image in which everything surrounding the subject is mostly a solid color so I don't think that's going to work here.
I'm using Jimp (https://www.npmjs.com/package/jimp) as an image processor as it has no native dependencies and this is going into a React Native app.
Any help would be greatly appreciated. Thanks in advance!

I never did end up finding something that already existed so I built something myself. Using my img-items node module, I could accomplish this by doing the following:
const Jimp = require('jimp')
const imgItems = require('imgItems')
Jimp.read('image.png')
.then(image => {
return imgItems(image)
.then(items => {
const largest = items.reduce((p, c) => ((p.width + p.height) > (c.width + c.height)) ? p : c)
return image
.crop(largest.left, largest.top, largest.width, largest.height)
.writeAsync('largest.png')
})
})

Related

Prepended Image not showing

I've been working on a small Chrome Extension to put together the things I've been learning. It's intended to remove the suggested reel on youtube and replace it with a motivational image to stay focused while studying. So far it works as intended but the image won't show, I've rewriting the 4th line in many different ways and can't seem to figure out what I should do different or if the issue is elsewhere. If anyone needs me to upload the code for may manifest.json folder let me know.
Thanks in Advance!
const contents = $("#contents");
const contentsParent = contents.parent();
contents.remove();
contentsParent.prepend("<img src='./images/motivated.jpg'>");
I managed to find a solution to the issue I was having and thought I'd post it in case someone sees this in the future. It looks like there was an issue with contents not being assigned it's value before the rest of the code ran so I worked around it with a setInterval that checks to see if contents has a value. Also created a URL version of the image I had saved since it was not uploaded before and this worked. Lastly I also added web accessible resources to my manifest.json file as you will see below.
"matches": ["<all_urls>"],
"resources": ["images/motivated.png"]
}]
function main(){
const contents = $("#contents");
if(!contents[0]) return;
window.clearInterval(checkId);
const contentsParent = contents.parent();
contents.remove();
// contentsParent.prepend("<img src='images/motivated.png'>");
const imgURL = chrome.runtime.getURL("images/motivated.png");
const htmlString = `<img class= "image" src="${imgURL}">`;
contentsParent.prepend(htmlString)
contentsParent.prepend("<h1>Get back to studying!</h1>").addClass("words")
} ```

How do I display in HTML the cover of an epub book using epub.js?

I'm using EPUB.js and Vue to render an Epub. I want to display the cover images of several epub books so users can click one to then see the whole book.
There's no documentation on how to do this, but there are several methods that indicate that this should be possible.
First off, there's Book.coverUrl() method.
Note that I'm setting an img src property equal to bookCoverSrc in the Vue template. Setting this.bookCoverSrc will automatically update the src of the img tag and cause an image to display (if the src is valid / resolves).
this.book = new Epub(this.epubUrl, {});
this.book.ready.then(() => {
this.book.coverUrl().then((url) => {
this.bookCoverSrc = url;
});
})
The above doesn't work. url is undefined.
Weirdly, there appears to be a cover property directly on book. So, I try:
this.book = new Epub(this.epubUrl, {});
this.book.ready.then(() => {
this.coverSrc = this.book.cover;
});
this.book.cover resolves to OEBPS/#public#vhost#g#gutenberg#html#files#49010#49010-h#images#cover.jpg, so at least locally when I set it to a src results in a request to http://localhost:8080/OEBPS/#public#vhost#g#gutenberg#html#files#49010#49010-h#images#cover.jpg, which 200s but returns no content. Probably a quirk of webpack-dev-server to 200 on that, but if I page through sources in Chrome dev tools I also don't see any indicate that such a URL should resolve.
So, docs not helping. I googled and found this github question from 2015. Their code is like
$("#cover").attr("src", Book.store.urlCache[Book.cover]);
Interesting, nothing in the docks about Book.store.urlCache. As expected, urlCache is undefined, though book.store exists. I don't see anything on there that can help me display a cover image though.
Using epub.js, how can I display a cover image of an Epub file? Note that simply rendering the first "page" of the Epub file (which is usually the cover image) doesn't solve my problem, as I'd like to list a couple epub files' cover images.
Note also that I believe the epub files I'm using do have cover images. The files are Aesop's Fables and Irish Wonders.
EDIT: It's possible I need to use Book.load on the url provided by book.cover first. I did so and tried to console.log it, but it's a massive blog of weirdly encoded text that looks something like:
����
So I think it's an image straight up, and I need to find a way to get that onto the Document somehow?
EDIT2: that big blobby blob is type: string, and I can't atob() or btoa() it.
EDIT3: Just fetching the url provided by this.book.cover returns my index.html, default behavior for webpack-dev-server when it doesn't know what else to do.
EDIT4: Below is the code for book.coverUrl from epub.js
key: "coverUrl",
value: function coverUrl() {
var _this9 = this;
var retrieved = this.loaded.cover.then(function (url) {
if (_this9.archived) {
// return this.archive.createUrl(this.cover);
return _this9.resources.get(_this9.cover);
} else {
return _this9.cover;
}
});
return retrieved;
}
If I use this.archive.createUrl(this.cover) instead of this.resources.get, I actually get a functional URL, that looks like blob:http://localhost:8080/9a3447b7-5cc8-4cfd-8608-d963910cb5f5. I'll try getting that out into src and see what happens.
The reason this was happening to me was because the functioning line of code in the coverUrl function was commented out in the source library epub.js, and a non-functioning line of code was written instead.
So, I had to copy down the entire library, uncomment the good code and delete the bad. Now the function works as it should.
To do so, clone down the entire epub.js project. Copy over the dependencies in that project's package.json to your own. Then, take the src, lib, and libs folders and copy them somewhere into your project. Find a way to disable eslint for the location you put these folders into because the project uses TAB characters for spacing which caused my terminal to hang due to ESLINT exploding.
npm install so you have your and epub.js dependencies in your node_modules.
Open book.js. Uncomment line 661 which looks like
return this.archive.createUrl(this.cover);
and comment out line 662 which looks like
// return this.resources.get(this.cover);
Now you can display an image by setting an img tag's src attribute to the URL returned by book.coverUrl().
this.book = new Epub(this.epubUrl, {});
this.book.ready.then(() => {
this.book.coverUrl().then((url) => {
this.bookCoverSrc = url;
});
})

How do I get a custom sized thumbnail of a Facebook feed image?

UPDATE: The original question was about how to get a square thumbnail, which apparently is not possible. I have changed the question to be how do I get a custom sized thumbnail. The aspect ratio can be left as-is.
I can piece together a solution from Using facebook graph api how to get news feed with large picture size if the feed type is photo? but I am hoping for something better.
I can get a list of image sizes using
https://graph.facebook.com/v2.0/{object_id}?access_token={token}
and then I could rummage through that list to find one that is large enough but not too large and make a second request for that image.
Or, following the answers in Facebook Graph API : get larger pictures in one request, I could get a full size image returned directly in the post feed request with
https://graph.facebook.com/v2.2/{object_id}/posts?fields=full_picture
But I don't need a full size image. Is there not a way to do something like in the second example but for arbitrary image dimensions, or at least for one of the presets like you get from the first example?
Original question:
I use the Javascript below to dump a custom styled facebook feed onto a website. It shows the date and the posted text and if there is an image it grabs that as well. It looks like Facebook defaults the image to 130px wide and maintains the aspect ratio of the original image. But Facebook has a way of generating square versions of images right?
I see /s130x130/in the image URL but unfortunately changing that results in nothing being returned, because it looks like there are some random IDs in the query string that only FB knows about. I saw an example of different sized square thumbnails somewhere, but those random IDs changed for each size. So is there a way to retrieve the square versions of my images within my Javascript or if not, is there any way at all to do it?
$(function() {
$.getJSON("https://graph.facebook.com/v2.2/1551867661729482/feed?access_token=XXXXXXXX", function(data) {
var maxPosts = 3
$.each(data.data, function(x) {
if (maxPosts > 0) {
if (data.data[x].from.name == "D'Arts") {
var message = data.data[x].message || //normal post
data.data[x].story.replace(/^You /, '').capitalizeFirstLetter() || //other updates without user text
"";
var pic = data.data[x].picture;
pic = pic ? '<img class="fb-thumb" src="' + pic + '"></img>' : '';
var date = new Date( data.data[x].created_time ).toString().replace(/.* (\w\w\w \d\d) .*/,"$1");
var link = data.data[x].link;
$('#Facebook .feed-goes-here').append('<h3>'+date+'</h3>' + pic + '<p>' + message + '</p>');
maxPosts--;
}
} else {
return false
}
})
})
})

Google Docs Add-On - Dealing with Images

I'm trying to create a Google Docs add-on in which someone:
Selects an image
Clicks a menu item
A dialog is displayed, showing the image (on a canvas) with a couple tools
Canvas is modified using tools
Canvas data is saved and replaces the original image
Meta data for the image is saved, so it can be re-edited from the original.
I know how to get the image selection (from the GS code) and trigger the menu item and dialog. I also know how to do all of my custom code things.
I need to know:
How to get the original image URL (or extract it as a base64 string) that I can put in to a canvas
Replace the image and save it in the document.
Save metadata on a per-image basis so it can be re-edited.
Examples would be awesome, though links to documentation would also be great. I've found a lot of things, but nothing concrete on how to extract the data as anything but a blob.
(This answer is a work-in-progress. Starting an answer to put the bits as I figure them out. If someone else helps me figure out the missing bits, I'll accept theirs instead of this one).
How to get the original image URL (or extract it as a base64 string)
As far as I can tell, there isn't a way to get the default URL. I was however able to get the base64 string. It's slightly convoluted, but works.
Code.gs
// Gets an InlineImage in some way. I'm using the currently selected image,
// but that's irrelevant to the code sample.
// #return {InlineImage}
function getImage() {
// gets the InlineImage element somehow
}
// Gets the actual data URI.
// Note: this function uses image/png only. You can change this by changing
// it in the two places, or using a variable. Just be sure the two spots
// match.
// #return {string}
function getDataUri() {
return 'data:image/png;base64,' + Utilities.base64Encode(getImage().getAs('image/png').getBytes());
}
MyDialogJavaScript.html
$(function () {
google.script.run
.withSuccessHandler(function (data) { console.log(data); })
.withFailureHandler(function (err) { console.log('failure: ' + err); })
.getDataUri();
});
An important note: you must SandboxMode.IFRAME when creating the dialog or else you'll get something like:
Rejecting <img>.setAttribute('src', blahblahblah
This is apparently due to a limitation in the Caja compiler normally used. See the answer here for more info: Using base64-encoded images with HtmlService in Apps Script

Tag images in the image itself? HOW-TO

How to tag images in the image itself in a web page?
I know Taggify, but... is there other options?
Orkut also does it to tag people faces... How is it done?
Anyone knows any public framework that is able to do it?
See a sample bellow from Taggify:
I know this isn't javascript but C# 3.0 has an API for doing this. The System.Windows.Media.Imaging namespace has a class called BitmapMetadata which can be used to read and write image metadata (which is stored in the image itself). Here is a method for retrieving the metadata for an image given a file path:
public static BitmapMetadata GetMetaData(string path)
{
using (Stream s = new System.IO.FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
{
var decoder = BitmapDecoder.Create(s, BitmapCreateOptions.None, BitmapCacheOption.OnDemand);
var frame = decoder.Frames.FirstOrDefault();
if (frame != null)
{
return frame.Metadata as BitmapMetadata;
}
return null;
}
}
The BitmapMetadata class has a property for tags as well as other common image metadata. To save metadata back to the image, you can use the InPlaceBitmapMetadataWriter Class.
There's a map tag in HTML that could be used in conjunction with Javascript to 'tag' different parts of an image.
You can see the details here.
I will re-activate this question and help a bit. Currently the only thing i have found about is http://www.sanisoft.com/downloads/imgnotes-0.2/example.html . A jQuery tagging implementation. If anyone knows about another way please tell us.
;)
You can check out Image.InfoCards (IIC) at http://www.imageinfocards.com . With the IIC meta-data utilities you can add meta-data in very user-friendly groups called "cards".
The supplied utilities (including a Java applet) allow you to tag GIF's, JPEG's and PNG's without changing them visually.
IIC is presently proprietary but there are plans to make it an open protocol in Q1 2009.
The difference between IIC and others like IPTC/DIG35/DublinCore/etc is that it is much more consumer-centric and doesn't require a CS degree to understand and use it...

Categories

Resources