I am using firebase storage bucket to save files and save their download link to database.
All should should work well, except that the file index are mixed up after I do a foreach loop(_lodash).
getFiles(e){
this.outPutFiles = e;
_.each(this.outPutFiles, ((file) => {
const ref = this._storage.ref(file);
return ref.getDownloadURL().subscribe(url => this.img_array.push(url));
}));
}
Expected behavior should be:
this.outPutFiles = [
0:"o-t-status-files/.....Qd5bGm"
1:"o-t-status-files/.....dz2bd8"
]
And the
this.img_array = [
0:"https://firebase....%2FJ6Qx9........Qd5bGm"
1:"https://firebase....%2FJ6Qx9........dz2bd8"
]
Unfortunately sometimes the file index in the this.img_array doesn't match the file at index on the this.outPutFiles variable.
For instance, the this.img_array may become something like this...
this.img_array = [
0:"https://firebase....%2FJ6Qx9........dz2bd8"
1:"https://firebase....%2FJ6Qx9........Qd5bGm"
]
The file at index 0 moved to index 1 and the file at index 1 moved to index 0.
How can I prevent this, to make sure the files index match in both the this.outPutFiles and this.img_array arrays?
Cause ref.getDownloadURL() is async, so the outPutFiles[1].getDownloadURL() may finish prior outPutFiles[0].getDownloadURL().
Try something like this (or using rxjs operators):
Promise.all(_.map(this.outPutFiles, file => this._storage.ref(file).getDownloadURL()))
.then(img_array => console.log(img_array))
Related
I build on a web-scrapper, that, lets say scrap URLs from google
I get an array of URLs from google results:
const linkSelector = 'div.yuRUbf > a'
let links = await page.$$eval(linkSelector, link => {
return link.map( x => x.href)
})
the output of 'links' is something like that:
[
'https://google.com/.../antyhing'
'https://amazon.com/.../antyhing'
'https://twitter.com/.../antyhing'
]
Now I have a 'blacklist', with something like that:
[
'https://amazon.com'
]
At the moment I stuck at that point where I can compare both arrays, and remove these URLs from 'links' which are listed within my blacklist.
So I came up with the idea, to get the domain of the url within my links array - like so:
const linkList = []
for ( const link of links ) {
const url = new URL(link)
const domain = url.origin
linkList.push(domain)
}
Yes, now i got two arrays which i can compare against each other and remove the blacklisted domain, but i lost the complete url i need to work with...
for( let i = linkList.length - 1; i >= 0; i--){
for( let j=0; j < blacklist.length; j++){
if( linkList[i] === blacklist[j]){
linkList.splice(i, 1);
}
}
}
Code Snippet is part of the give answer, here:
Compare two Javascript Arrays and remove Duplicates
Any ideas how can i do this, with puppeteer and node.js?
I couldn't find an obvious dupe, so converting my comments to an answer:
.includes:
const allowedLinks = links.filter(link => !blacklist.some(e => link.includes(e)))
.startsWith:
const allowedLinks = links.filter(link => !blacklist.some(e => link.startsWith(e)))
The second version is more precise. If you want to use the URL version, this should work:
const links = [
"https://google.com/.../antyhing",
"https://amazon.com/.../antyhing",
"https://twitter.com/.../antyhing",
];
const blacklist = ["https://amazon.com"];
const allowedLinks = links.filter(link =>
!blacklist.some(black =>
black.startsWith(new URL(link).origin) // or use ===
)
);
console.log(allowedLinks);
As for Puppeteer, I doubt it matters whether you do this Node-side or browser-side, unless these arrays are enormous. On that train of thought, technically we have a quadratic algorithm here but I wouldn't worry about it unless you have many hundreds of thousands of elements and are noticing slowness. In that case, you can put the blacklisted origins into a Set data and look up each link's origin in that. The problem with this is it's a precise ===, so you'd have to build a prefix set if you need to preserve .startsWith semantics. This is likely unnecessary and out of scope for this answer, but worth mentioning briefly.
Working with an array of data that we want to be able to sort for display in a component, and it doesn't seem to be sorting or updating the DOM, however I have a working code sample that properly demonstrates the concept, and it should be sorting, but in the angular app, it's simply not getting sorted.
The parent component that houses the original data stores the data on an Input parameter object called Batch, and the array we're sorting is on Batch.Invoices.Results. The event from the child component is fine, and the appropriate data is confirmed to bubble to the parent component.
The function that's supposed to sort the array looks like this:
public OnInvoiceSortChange({orderValue, orderAscending}){
console.log(`Invoice Sorting has been called. Value: ${orderValue} . Ascending? ${orderAscending}`);
console.log(`Before:`);
console.log(this.BatchViewModel.Invoices.Results.map(x => x.VendorName));
const sortingArray = [...this.BatchViewModel.Invoices.Results];
if(orderAscending){
const sorted = sortingArray.sort((a, b) => a[orderValue] > b[orderValue] ? 1 : 0);
this.BatchViewModel.Invoices.Results = sorted;
console.log('Sorted');
console.log(sorted.map(x => x.VendorName));
} else {
const sorted = sortingArray.sort((a, b) => a[orderValue] < b[orderValue] ? 1 : 0);
this.BatchViewModel.Invoices.Results = sorted;
console.log(sorted.map(x => x.VendorName));
}
console.log(`After:`);
console.log(this.BatchViewModel.Invoices.Results.map(x => x.VendorName));
}
All the console logs are for debugger visibility, and the output is this:
Where in my testing file (non-angular) looks like this:(where data is a direct copy of the array from the Angular app.
const ascendingData = [...data];
const descendingData = [...data];
const sortedDescending = descendingData.sort((a, b) => a['VendorName'] < b['VendorName']? 0 : 1)
const sortedAscending = ascendingData.sort((a, b) => a['VendorName'] > b['VendorName']? 0 : 1);
const vendorListAscending = sortedAscending.map(x => x.VendorName);
const vendorListDescending = sortedDescending.map(x => x.VendorName);
console.log(vendorListDescending);
console.log(vendorListAscending);
and the output looks like this:
So I see that the sorting should work, but it's just not happening in Angular.
How can I get the array sorted, and update the DOM as well?
The function you pass to sort is wrong. It is supposed to return a negative value for "less", a positive value for "greater" or zero for "equal". If orderValue is numeric then it's easiest to just return a[orderValue] - b[orderValue], if not then just change your 0 to -1.
(By the way, name orderKey could be a bit clearer maybe?)
I don't think angular has anything to do here, but I cannot tell now why you get different results. Anyway, your sort function is invalid (it states that a equals b, but at the same time b is greater than a), I hope fixing this function helps.
I want to replace a particular line using javascript with new content.
This is the file content,
SERVER=1055#localhost
GROUP_SERVERS=2325#localhost
LINE_IAM_INTERESTED=KTYUIERVW:2800
FILE_PATH="C:\Program Files\somefile\Shared Files\"
In this line, LINE_IAM_INTERESTED=KTYUIERVW:2800 .I want to replace KTYUIERVW with KJHTERDY and 2800 with 78945
I have shown what I tried using fs appendfilesync
fs.appendFileSync('file_name').toString().split('\n').forEach(function(line){
app.console.log("called append");
var sEntry = line.split("=");
if (sEntry.length == 2) {
if (sEntry[0] == "LINE_IAM_INTERESTED") {
app.console.log("found one!!!!");
}
}
});
you can try READ -> REPLACE -> WRITE flow:
fs.writeFileSync('file_name', fs.readFileSync('file_name').replace('KTYUIERVW:2800', 'KJHTERDY:78945'))
appendFile and appendFileSync are used for adding to the end of the file. Although you can do it as a one liner as shown in the other answer, I've kept the structure of your code the same. Here you want to read the data, modify it then re-write it. Using the block of code you have, you can modify it to use readFileSync and writeFileSync.
let newData = fs.readFileSync('file_name', 'utf-8').split('\n').map(line => {
let sEntry = line.split('=')
if (sEntry.length == 2) {
if (sEntry[0] == "LINE_IAM_INTERESTED") {
console.log("found one!!!!");
return sEntry[0] + '=' + 'KJHTERDY:78945'
}
}
return line
}).join('\n')
fs.writeFileSync('file_name', newData)
I've swapped the forEach for map which lets us change the array data by returning the desired value. The new array is then joined back together and written back to the file.
I'm setting up a test to ensure that a faceted Solr query 'contents' are correctly displayed within a page element, using javascript.
The Solr query result, which I've named "ryanlinkstransmissionpage", is;
{ Transmission: [ 'Manual', 12104, 'Automatic', 9858 ] }
What I would like to do is extract the 'Manual' and 'Automatic' only, so I can then test that these values are displayed on a page.
However, it is more the functionality involved in this that I cannot get my head around, as I will be using this method on other Solr query results.
To possibly complicate things, this Solr query result "ryanlinkstransmissionpage" is from a dynamic 'live' Solr, so the values may change each time it's run (so there may be more or less values within this array when it's tested on the following day for example).
I've tried a few javascript commands, but to no avail.
JSON.parse(ryanlinkstransmissionpage)
JSON.stringify(ryanlinkstransmissionpage)
Object.values(ryanlinkstransmissionpage)
Any help would be greatly appreciated. Thanks.
If possible, i highyl recommend changing the transmission field to be an object, rather than an array. That will give you far greater ability to read the data within.
Ignoring that, are you looking to extract the string values and the number values that follow them? ie. "Manual" and "12104"? Or are you simply trying to assert that the string values are present on the page?
Either way, here are two possible approaches.
const ryanlinkstransmissionpage = { Transmission: [ 'Manual', 12104, 'Automatic', 9858 ] };
// Pull out the string values
const strngVals = ryanlinkstransmissionpage.Transmission.filter(val => typeof val === 'string');
// Pull out the string values and the numbers that follow
const strngNumVals = ryanlinkstransmissionpage.Transmission.reduce((keyVals, val, idx, srcArr) => {
if (typeof val === 'string') keyVals[val] = srcArr[idx + 1];
return keyVals;
}, {});
The reduce approach is not stable or robust to changes in data provided from this Solr query result you refer to, nor is it tested. #shrug
Javascript has a built in method called Array.prototype.find(() =>). If you just want to check if this value exists to ensure its on the page, you can simply do:
const ryanlinkstransmissionpage = { Transmission: [ 'Manual', 12104, 'Automatic', 9858 ] };
const manual = ryanlinkstransmissionpage.Transmission.find((ele) => ele === 'Manual'); // returns 'Manual'
const automatic = ryanlinkstransmissionpage.Transmission.find((ele) => ele === 'Automatic'); // returns 'Automatic'
console.log(automatic);
console.log(manual);
// or
const findInArray = (arr, toFind) => {
const result = arr.find((ele) => ele === toFind);
return !!result;
}
console.log(findInArray(ryanlinkstransmissionpage.Transmission, 'Automatic')); // true
console.log(findInArray(ryanlinkstransmissionpage.Transmission, 'HelloWorld')); // false
console.log(findInArray(ryanlinkstransmissionpage.Transmission, 'Manual')); // true
i got some sort of an issue with my custom sorting. So, basically i have this array:
[ 'src/app/account/account.js',
'src/app/account/dashboard/characters/characters.js',
'src/app/account/dashboard/characters/detail/detail.js',
'src/app/account/dashboard/dashboard.ctrl.js',
'src/app/account/dashboard/dashboard.js',
'src/app/account/dashboard/panels/admin.ctrl.js',
'src/app/account/dashboard/panels/users.ctrl.js',
'src/app/account/donate/donate.ctrl.js',
'src/app/account/donate/donate.js',
'src/app/account/settings/settings.ctrl.js',
'src/app/account/settings/settings.js',
'src/app/account/vote/vote.ctrl.js',
'src/app/account/vote/vote.js',
'src/app/membership/dialogs/login.ctrl.js',
'src/app/membership/dialogs/register.ctrl.js',
'src/app/membership/dialogs/termsOfService.ctrl.js',
'src/app/membership/membership.ctrl.js',
'src/app/membership/membership.module.js',
'src/app/news/news.ctrl.js',
'src/app/news/news.js',
'src/app/noctis.ctrl.js',
'src/app/noctis.js',
'src/app/widgets/playersOnline/playersOnline.js',
'src/app/widgets/rankings/rankings.js',
'src/app/widgets/serverDetails/serverDetails.js',
'src/common/directives/feeds/feeds.js',
'src/common/directives/panel/panel.js' ]
And what i would like that after the src/app/ the very first js that comes after the very first folder after src/app/ in our case: account, membership(can be more custom names) to be loaded first, like in the next example:
['src/app/membership/membership.module.js',
'src/app/membership/membership.ctrl.js',
'src/app/membership/dialogs/login.ctrl.js',
'src/app/membership/dialogs/register.ctrl.js',
'src/app/membership/dialogs/termsOfService.ctrl.js',]
Can you guys help me with some code for my needs? src/app will always be a fixed name except of the next directory that comes after src/app/.
Basically what comes after the unknown name of the directory after src/app, the sub directories in our case(dialogs) or can be something else like(detail, detail/character), to be loaded latest no matter what.
Basically this is the whole function:
function sortJSFiles(files) {
var src = [];
var vendor = [];
files.forEach(function(item) {
if (item.startsWith('src')) {
src.push(item);
} else {
vendor.push(item);
}
});
src.sort(function(a, b) {
var replace = ['noctis.js', 'noctis.ctrl.js', '.module.js', '.ctrl.js'];
function replaceCB(previousValue, currentValue, currentIndex, array) {
return previousValue.replace(currentValue, currentIndex);
}
return replace.reduce(replaceCB, a).localeCompare(replace.reduce(replaceCB, b));
});
return vendor.concat(src);
}
What it does, is that in paramater files comes a lot of paths with js files and i'm trying to sort them after my rule. The problem is, for example taking membership example:
['src/app/membership/dialogs/login.ctrl.js',
'src/app/membership/dialogs/register.ctrl.js',
'src/app/membership/dialogs/termsOfService.ctrl.js',
'src/app/membership/membership.module.js',
'src/app/membership/membership.ctrl.js']
It succesffully change the sort like loading *.js files that starts first with .module.js and than with .ctrl.js but there is a problem in my code that i need that any js file that comes after src/app/somefolder to be loaded first and any subfolders that are in that somefolder to be loaded latest no matter what.
I am not sure I understood you correctly (it would have been nice if you would have added the literal expected output for your sample data).
I think you want to have the folders sorted, but within the same folder, you want the files in there to be sorted before any of the subfolders in that same folder. And this should be true at every nested level.
To get the files sorted first in every folder, you should in fact extract the folders only, and sort those, and only when two items have exactly the same folder sequence, sort by the file name.
This you can do as follows:
src = src.map(function (path) {
var i = path.lastIndexOf('/');
return [path.substr(0, i), path.substr(i)];
}).sort(function (a, b) {
var i = +(a[0] == b[0]);
return a[i].localeCompare(b[i]);
}).map(function (pair) {
return pair[0] + pair[1];
});
var src = [ 'src/app/account/account.js',
'src/app/account/dashboard/characters/characters.js',
'src/app/account/dashboard/characters/detail/detail.js',
'src/app/account/dashboard/dashboard.ctrl.js',
'src/app/account/dashboard/dashboard.js',
'src/app/account/dashboard/panels/admin.ctrl.js',
'src/app/account/dashboard/panels/users.ctrl.js',
'src/app/account/donate/donate.ctrl.js',
'src/app/account/donate/donate.js',
'src/app/account/settings/settings.ctrl.js',
'src/app/account/settings/settings.js',
'src/app/account/vote/vote.ctrl.js',
'src/app/account/vote/vote.js',
'src/app/membership/dialogs/login.ctrl.js',
'src/app/membership/dialogs/register.ctrl.js',
'src/app/membership/dialogs/termsOfService.ctrl.js',
'src/app/membership/membership.ctrl.js',
'src/app/membership/membership.module.js',
'src/app/news/news.ctrl.js',
'src/app/news/news.js',
'src/app/noctis.ctrl.js',
'src/app/noctis.js',
'src/app/widgets/playersOnline/playersOnline.js',
'src/app/widgets/rankings/rankings.js',
'src/app/widgets/serverDetails/serverDetails.js',
'src/common/directives/feeds/feeds.js',
'src/common/directives/panel/panel.js' ];
src = src.map(function (path) {
var i = path.lastIndexOf('/');
return [path.substr(0, i), path.substr(i)];
}).sort(function (a, b) {
var i = +(a[0] == b[0]);
return a[i].localeCompare(b[i]);
}).map(function (pair) {
return pair[0] + pair[1];
});
console.log(src);
Explanation of the sort callback function
The sort callback function will receive argument a and b. Each of them is a pair including a path at index 0, and a filename at index 1.
The callback uses a variable i that is intended to get the value 0 or 1. If the paths of a and b are the same, then i will be 1, else it will be 0. It determines whether a comparison is needed on the paths or on the filenames.
The unitary + is used to convert the boolean expression a[0] == b[0] to a number. The conversion is: true=>1, false=>0.