I have a folder that contains several images, the directory structure looks like this:
./images/
./images/1.png
./images/2.png
./images/3.png
./images/4.png
./images/{n}.png
These images have been downloaded and saved using the request and fs modules by a script called update.js.
Each file is named after the length of items in the folder (I.E: length + 1).
The update.js script downloads (and saves) each image, regardless of whether or not it exists.
I can get around this by deleting the images folder but this is a waste of resources.
What's the most efficient way to prevent this behaviour?
NOTE: I can't use a simple file name check since, the names are indexes.
Thanks.
You can issue a HTTP head request for each file and get its headers. Then you can see how big the target file is and avoid re-downloading it if the size matches exactly.
This isn’t ideal though as different files may have the same size.
Some servers give you a content md5 which would probably be the best. The md5 is unlikely to match between any two files you have unless your use case is very large.
You would be better served by just working to fix the script though so it store proper metadata, all this is quite hacky :). You can store the real file names and modified timestamps as another file in a sibling directory and be fairly sure it won’t affect anything. Then you can just check those before doing a download.
Related
I would like to load images dynamically from an images folder.
Names in the folder like {company_name}.png. The names are stored in a json file along with other company data like name, type, etc.
e.g.:
name: meta
logo: "./images/meta.png"
Is there any way to dynamically load these images like require(changing variable based on the json logo string).
I found some solution online like:
https://stackoverflow.com/a/52598171/7990652
But all the other things are build around the json structure to load all the other data.
Is there any way to load everything from a folder one by one based on the provided logo link in the json?
I also tried to look around for a solution with babilon require all plugin, but I found nothing about that.
P.S. I know there are sevaral threads about this question, but in almost all the threads someone asking "what about if I want to load 100-1000 images?". That is the case with me too, I would like to load a lot of images, not just up to 10-20 which is complately okay with static .js require list.
Dynamically loading local assets at run-time including images is not supported within React Native.
In order to use static assets (images, fonts, documents,...). All assets must be bundled at compile-time with your app.
Attempts to load assets dynamic at run-time without pre-bundled at compile time will cause the app to crash.
Bundle all images you need at compile-time then use image reference in memory at run-time.
const staticImages = {
image_01: require("./path/to/image_01.png"),
image_02: require("./path/to/image_02.png"),
image_03: require("./path/to/image_03.png"),
// ... more images
};
Define this object containing images reference globally and make sure it is executed as early as possible when the app is initializing.
Later access any image reference as below:
<Image source={staticImages[IMAGE_REFERENCE_NAME]} />
No, unfortunately your question is a duplicate of the others you have found, and it is not possible to use require dynamically, no matter how many times you want to do it.
If you can break up the images into multiple components, you can conditionally load those components. But require must be called with a fixed string.
I've done some looking around on the site and every time I pull up a solution to this problem, one of the requirements is to have a naming convention and a list of every image to pull from the directory (example: image1.jpg, image2.jpg, etc.) All of the file names are different and there are thousands of them to pick from (so listing each one as a random opportunity in an array is not going to work).
I typically use CMS services and I'm writing this webpage from scratch in Notepad in an attempt to better my coding skills... and I'm not sure where to begin. I'm decent with HTML and CSS, but j Query and JavaScript are not my friends haha.
Thank you for any help! (Even if it's just pointing me to a tutorial or a solution I could not find!!!)
Are all file names image1 image2 image3 etc? Then you could try to generate a random number, create a new img element and have it's source pointing to image+randomnumber.jpg and append it to the DOM
One of the main problems your facing here is about your thinking when it comes to how content is delivered, in a standalone static website you do not have access to the file system. This means that if we want to query things outside of the browsers context we are not allowed, obviously without being able to access directories we can not generate a list of file names which can be loaded.
If your wondering why we can't access the file system directly from say the JavaScript it's because of the sandbox that most modern browsers live in, otherwise people could attack your native directories from the front end languages. Your question is interesting as electron removes this sandboxing in a sophisticated esk manner, which is necessary as it's used for building desktop apps with chromium.
These days the most obvious solution would be to use some form of back end language and to create a web server that has direct access to the native directories around it. Node, PhP, GoLang and many other populatr backend languages can parse a directory of files and then interpolate those into the frontend code which is the most common method.
The other popular method at the moment is to create API's which is just a fancy web server with a queryable end point that then executes code against our web server and provides back a list of such items. You could then for instance take the items and then print those out using javascript.
Reference directories method in php:
http://php.net/manual/en/ref.dir.php
List contents of directory in nodejs:
https://code-maven.com/list-content-of-directory-with-nodejs
The best place to really start with the easiest route to understand more would be to start a backend language in either node or php, with php being the simpler of the two.
https://www.w3schools.com/php/
First you need to get your file list from server side. then you can use a code like following:
var imageList = //your image list as an array of urls;
var imageNumber = Math.random() * imageList.length; //gives you a random number in the range of imageList's size
var imageToLoad = new Image();
imageToLoad.addEventListener("load", function(){
console.log( "image is loading" );
$('#my-container').append(this); //in this case this will return image dom
});
imageToLoad.src = imageList[imageNumber];
this will add image to a container with id 'my-container' its just an example you can do anything you want using 'this'
So after much help and guidance from the community, I have figured out the answer! To clarify my process in extreme detail, here is what I did to achieve the desired outcome:
Create the page as a .php file instead of a .html file (in my case, index.php). If you are using notepad to create the file, make sure you change the file extension to .php, the encoding to UTF-8, and save file type as "All Files". As I understand it, PHP can pick the file at random but cannot pass this info to a static HTML page.
Place this block of code into the webpage where the image should show. Currently, it is set up to reference a folder named, "images" out of the root directory (aka mysite.com/images/). This can be changed by modifying the text between the apostrophes after $imagesDir. All other html markup on the page will work correctly if it is outside of the php code block.
Code Block:
<?php
$imagesDir = 'images/';
$images = glob($imagesDir . '*.{jpg,jpeg,png,gif}', GLOB_BRACE);
$randomImage = $images[array_rand($images)];
echo "<img src='$randomImage'>";
?>
Thank you #bardizba for the code! Although there may be less resource intensive ways to write this, my situation was a bit different because the file names in the directory did not follow a naming convention and there was a mix of file types (jpg, gif, etc.)
Thanks again to everyone that helped me out!
Some websites seem to have file names such as 'assets/app-02b4523sev8fsd56e.js'. I have noticed that these numbers do not change though, so I thought it has something to do with security but I am not sure. Is there any reason behind this?
This is normally to break caches stored by the browser so that the latest version of a file is loaded. Every time a file is changed this value will normally be changed also. This can be done manually by changing the filename and/or the paths in other files referencing this file, or this can be done programatically in some way. You may also see this done like the following and it may also contain a timestamp rather than a hash like the above:
assets/scripts.js?v=20150611190618
This is often referred to as a 'cache-buster' amongst other names.
I am making a browser game (client side only). I am trying to make it smaller (meaning file sizes), which is first step for mobile version. I have minified CSS using LESS, JS using uglify and also angular templates using grunt-angular-templates. So at this moment I am loading very small number of files:
index.html
app.js
app.css
images.png (one file with all images)
But the remaining problem are JSON data files. There are (or will be) many levels and each level has its own JSON data file. Also there are some rule definitions etc. The problem is, that these JSON files are loaded dynamically when needed.
I am now trying to find a way, how to somehow get these files (at build time, probably some grunt task) into one file, or even better - directly into app.js. I have no problem in writing PHP script + JS class, that would do this, but I first tried to find some finished solution.
Does anybody know about something like that, or is there any other solution that I am not thinking about? Thanks for any help.
====
EDIT:
1) The point of this is getting rid of X requests and making one request (or zero) for JSON files.
2) The compiled thing does not have to be JSON at all. Part of my idea:
JsonManager.add('path/to/json/file.json', '{"json":"content of file"}');
making all these lines manually is bad idea, I was asking about something, if there is anything, that could do this job for me.
3) Ideally i am looking for some solution similar to what grunt-angular-templates task does with HTML templates (minifies them and adds them to app.js using Angular's $templateCache)
Say you have two JSONs: {'a':1} and {'b':2}.
You cannot simply concatenate them into one chunk as together they will not be a valid JSON, e.g. this {'a':1}{'b':2} is not valid JSON. You can do this with JS and CSS but not JSON.
The only option is to include them into larger structure:
[
{'a':1},
{'b':2}
]
If your code structure allows to do this then you can use any existing JS compressor/uglifier to compress the result.
For anybody who has same problem as me:
I gave up finding already finished solution, and made my own:
The solution
I have written PHP script, that iterates over files in data directory and lists all JSON files. It also minifies their contents and creates one big array, with keys as relative file names and values as JSON content of files. It then creates a .js file, in which this big array is encoded as JSON again and given to a JavaScript variable (module constant in my case - Angular)
I created a wrapper class, which serves this data as files, e.g.:
var data = dataStorage.getData('levels/level01.json'); // returns JSON content of file located at path/to/data/files/levels/level01.json but without any AJAX call or something
I used grunt-shell to automate running this php file
I added the result .js file to list of files, which should be minified by uglify (and connected together).
The result:
I can create any number of JSON files in any structure and link to them from js code using that wrapper class, but no AJAX calls are fired.
I decreased number of files needed to load at startup (but increased app.js size a bit, which is better than second request).
Thanks for your ideas and help. Hope this also helps someone
My script adds some annotations to each page on a site, and it needs a few MBs of static JSON data to know what kind of annotations to put where.
Right now I'm including it with just var data = { ... } as part of the script but that's really awkward to maintain and edit.
Are there any better ways to do it?
I can only think of two choices:
Keep it embedded in your script, but to keep maintainable(few megabytes means your editor might not like it much), you put it in another file. And add a compilation step to your workflow to concatenate it. Since you are adding a compilation you can also uglify your script so it might be slightly faster to download for the first time.
Get it dynamically using jsonp. Put it on your webserver, amazon s3 or even better, a CDN. Make sure it will be server cachable and gzipped so it won't slow down the client network by getting downloaded on every page! This solution will work better if you want to update your data regularly, but not your script(I think tampermonkey doesn't support auto updates).
My bet would would definetly be to use special storage functions provided by tampermonkey: GM_getValue, GM_setValue, GM_deleteValue. You can store your objects there as long as needed.
Just download the data from your server once at the first run. If its just for your own use - you can even simply insert all the data directly to a variable from console or use temporary textarea, and have script save that value by GM_setValue.
This way you can even optimize the speed of your script by having unrelated objects stored in different GM variables.