Develop Reference

mongo db JS .map not working in GAE despite it working in localhost? or something else

I have Node app with mongo db that works perfectly in localhost, but as I deploy it to Google App Engine one particular function of the app stops working. This function uses .map to get information from mongo db; fetching a value into javascript object based on the mongo generated _id field.
Questions are:
Is there a reason (perhaps environment) why this would not work in Google App Engine?
Is there a good alternative to .map? .forEach is the best?
Any other pointers where I could start troubleshooting from with something working in localhost and not deployed to GAE?

1 and 3 - Can you explain how did you set your app? Can you check the logs and see what's going wrong? You can check this community tutorial on how to connect Node.js to Mongo db. My guess is that you missed installing some libraries that are on the localhost server but not once you deploy, but without further clarification I cannot answer. I will update if more info is provided.
2- .forEach execute a function for each element of one array but .map creates an array with the result of executing a function for each element, the difference is very subtle . In general, .forEach is better if you are not going to change data in the original array (such as printing the value) and .map is better if you want to change it (converting to float for example).

How to debug timed out waiting for asynchronous Angular tasks? Failure to find elements on angular page occurring

Edit: Note that I found the root of my problem after help from #ernst-zwingli, so if you have this same error one of his noted fixes might help you out. My problem is a known issue with Protractor itself, if you think this may be you, I've expanded on my steps to pinpoint the root of the problem after my original question.
I'm trying to use Protractor in an Angular2 (just Angular) application built using angular-cli.
My problem: Elements on an Angular app page are not being found when browser.waitForAngularEnabledis at it's default setting of true (as in 'I believe I am on an angular page and would like for Protractor to do it's magic'). They are being found just fine if I set browser.waitForAngularEnabledto false (as in 'I am not on an angular page and would like to handle this myself, take a seat Protractor'). How do I track down what's causing this on my definitely Angular pages?
I have a product with a non-Angular Auth0 login page that gates access to the rest of the product that is written in Angular (Angular 4.3.2 to be exact). I have successfully traversed logging in on the non-Angular login page. I flipped the waitForAngularEnabled switched to false to facilitate the non-Angular login. I turned it back to true at the point where I expected my initial landing page (Angular) to be loaded, after clicking the submit button. Code is as follows:
browser.waitForAngularEnabled(false);
browser.driver.get('https://dashboard.net/projects');
browser.driver.sleep(10000);
browser.driver.findElement(By.css("[type='email']"));
browser.driver.findElement(By.css("[type='email']")).sendKeys("email#example.com");
browser.driver.findElement(By.css(".auth0-label-submit")).click();
browser.driver.findElement(By.id("passwordInput")).sendKeys("password");
browser.driver.findElement(By.id("submitButton")).click();
browser.driver.sleep(5000); // needed if not waiting for Angular
//browser.waitForAngularEnabled(true); // Back to Protractor land we go
let elementToFind = element(by.className("header-text"));
elementToFind.isDisplayed().then(function() {grabTheDarnLocalStorage()});
expect(elementToFind.isDisplayed()).toBeTruthy();
If I uncomment the browser.waitForAngularEnabled(true); line to state that I'm back in Angular code I get the error trace as follows:
Failed: Timed out waiting for asynchronous Angular tasks to finish after 30 seconds. This may be because the current page is not an Angular application. Please see the FAQ for more details: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
While waiting for element with locator - Locator: By(css selector, .header-text)
ScriptTimeoutError: asynchronous script timeout: result was not received in 30 seconds
(Session info: chrome=61.0.3163.100)
(Driver info: chromedriver=2.32.498550 (9dec58e66c31bcc53a9ce3c7226f0c1c5810906a),platform=Windows NT 10.0.14393 x86_64)
at WebDriverError (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:27:5)
at ScriptTimeoutError (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:203:5)
at Object.checkLegacyResponse (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\error.js:505:15)
at parseHttpResponse (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\http.js:509:13)
at doSend.then.response (C:\Users\c-shouston\AppData\Roaming\npm\node_modules\protractor\node_modules\selenium-webdriver\lib\http.js:440:13)
at process._tickCallback (internal/process/next_tick.js:109:7)
From: Task: Protractor.waitForAngular() - Locator: By(css selector, .header-text)
I've referenced the FAQ: https://github.com/angular/protractor/blob/master/docs/timeouts.md#waiting-for-angular
I have my devs stating that they don't use $timeout (they use (Edit: NOT $interval) Observable Interval thank you very much) and they're not sure about $http.
I found this solution about the canonical way to debug protractor Angular sync issue issue: Canonical way to debug Protractor-to-Angular sync issues
but I'm not sure the solution works without access to modifying the dev code to run the programmatic tracker. (Edit: I never did figure out how to get this to work)
I also found this about a long timeout you add before each test, but I feel this is unnecessary overhead that makes your overall test execution take longer than it should without understanding the root cause of the problem: https://stackoverflow.com/a/37217167/2718402 (Edit: yeah, this is a bad idea and adds unnecessary time to your tests, please don't do this)
The frustrating bit is that this seems to be a common occurrence and there doesn't seem to be a streamlined documentation on how to deal with it. Logging in with a non-Angular page only to transition to an Angular page. Angular pages not being picked up properly by Protractor. All of the examples I find online are bits of code that I don't have a reference for where they should be at in my overall test framework. I would kill for a full example of someone testing a non-Angular login that transitions to a fully Angular website, with a setup config and real world test cases. (Edit: This is still true, but I can't make one myself as my application is in a bad grey area, note my RCA below for more details.)
I just want the ability to do my login and then successfully transition over to my Angular pages and be able to rely on Protractor to work with my Angular pages. I need to know what to look for that may be a long running asynchronous process (What specifically can I check for in the Chrome dev tools?). I would love to understand what Protractor needs as defaults in order to successfully work with the Angular parts of my app/website (Is there something beyond the presence of <app-root _nghost-c0="" ng-version="4.3.2"> in the HTML?). Before this job I worked in Java, so all of this asynchronicity and Angular is new to me, so I know I'm missing the known things that a seasoned Javascript dev is aware of.
My Solution/Root Cause Analysis
Starting down the list suggested by #ernst-zwingli:
for Angular(2) Check if the object window.getAllAngularRootElements returns at least one value.
It returned at least one value, so I moved on.
useAllAngular2AppRoots: true,
I tried this and still ran into the async timeout.
And if $interval or other long lasting asynchronous tasks are used, there can be issues, because of the zones
Previously #ernst-zwingli also mentioned looking at the testability method, except it was the old way. Through research and testing I found the window object also has a getAllAngularTestabilities method. This led down an interesting rabbit hole. An example output from the Chrome console (put window.getAllAngularTestabilities() in the Chrome console window, look at the resulting list) is as follows:
t:
_callbacks:...,
_didWork:true,
_isZoneStable: true (this looks promising, but why isn't Protractor working then?!?)
_ngZone:
hasPendingMacrotasks: true,
hasPendingMicrotasks: false,
isStable: true
I would think isZoneStable would be enough, but apparently not so for Protractor. Then looking at Macrotasks being true, I had to look up what the heck a Macrotask was: What does hasPendingMacrotasks and hasPendingMicrotasks check for?.
A macrotask can be:
i.e. setTimeout, setInterval, setImmediate
Thus #ernst-zwingli's note about interval's causing problems in the zones was remembered and something finally clicked.
First github issue, about zone instability
Another github issue complaining about the necessity of using browser.driver to get things done along with browser.waitForAngularEnabled. Apparently this is expected behavior, it led me to issue #3349
Issue #3349 - The actual root cause of my issue. My developers do not actively jump in and out of zones around observables. Even though these observables only have one subscriber. Since they live in the angular zone at this time, they are a long running "Macrotask" that Protractor waits infinitely on.
I can't rewrite the code with these wrappers as I am not currently versed enough in Angular to do it safely and we are currently hurtling toward a November deadline. I think I'll have to deal with using browser.driver for the time being and hope I can't get it fixed later. Hopefully my RCA was helpful for you.

In the following I list a set of potential causes and possibilities to fix/resolve them.
How does AngularJS and Angular(2) Work / What can I check in the Browser Dev Mode
I can't explain it as well as Andrey Agibalov in his Blog here, so check it out (also for developers).
Basically, the objects required by Protractor you can check in your Chrome Dev.
for AngularJS
Check if the object window.angular is properly defined, i.e. lookup window.angular.version and also try window.angular.getTestability of your Root element
for Angular(2)
Check if the object window.getAllAngularRootElements returns at least one value.
Root Element (AngularJS)
Potentially your Angular App is somewhere wrapped within the Body as something like <div ng-app="my-app">.
In that case, you must adjust your rootElement: body inside config.ts. Check this answer for details.
Angular(2)
If you're using Angular (aka Angular2), then there are ngZone's introduced. In this case your config.js should additionally contain this:
exports.config = {
framework: 'jasmine',
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['spec.js'],
useAllAngular2AppRoots: true,
// rootElement: 'root-element'
};
check in your browser for window.getAllAngularRootElements as the additional line in conf.js is about this.
If you can, maybe use the advantage of multiple zones possible. Create a 2nd zone, configure rootElement: 'root-element'
to only focus on one zone and then move some asynchronous tasks into the other zone until you found, which task(s) lead to timeout. Keep those tasks (if possible) in the separate zone, so Protractor ignores those tasks.
And if $interval or other long lasting asynchronous tasks are used, there can be issues, because of the zones. Repeatedly or long lasting tasks should be started outside the zone and then be moved into the zone as else Protractor could run into timeouts. There is a workaround for developers to apply, in order to avoid these problems for Protractor.
read all about it here
browser.driver. - side remark
browser.driver.get() works as if ignoreSynchronization = true, since you directly assign the Browser Driver and you kind of bypass the synchronization logic of Protractor.
Read more about it in this answer here.
Hope I could give you some more input and you can solve your issue. Please let me know the results.

Could you please set
browser.ignoreSynchronization = true;
and try

Writing git API for javascript node module

Hi recently I have to implement a git API for a node module, so we can have version control through programmatic interface. I came upon this post to find some good modules that I can use.
However, after playing around with some of them, I realized that none of them provide the feature of querying the commit history of a single file (like calling 'git log --follow '). I would like to extend that feature into my module but since I have no experience before, does anyone know where I should start?

The libraries you mentioned should be a good starting point. Take node-git's lib/git.js file for example. There you can find the following lines:
// Call the native git binary
Git.prototype.call_git = function(prefix, command, postfix, options, args, callback) {
// ...
This is a good starting point to understand how the author maps Node functions on system calls and and allows the callback to work with the output.
I would start a fork of the project. Look for the log function in the same file, which starts like
Git.prototype.log = function(commit, path, options, callback) {
Copy the function, give it a proper name and try to adopt it to meet your needs.
If you succeed and like to help, you could start a pull request on GitHub to help the author and add the functionallity to the official project.
I would do it this way. I hope this helps.

Command-output (stdout, stderr) to the web in Python

Recently I have developed a short Python script using the Flask framework to start a process over HTTP (see here). Now there is the question for an adequate technology to "pipe" the standard streams (I am especially interested in stdout and stderr) of started processes to the Web.
What would you say is the most suitable way to accomplish this? First I thought of websockets to fit perfectly. But then it seemed to me that most implementations veil their connected clients. I think that I need this information in order to be able to send the output to all of them.
Edit: I think that my question was a bit unclear: I want to see the output of the executed command in a web-interface. So I have to "convey" data from stdout/stderr of the executed process via HTTP somehow to the web-interface(s). It might be possible that the started command may run for some (longer) time. The example which invokes the dd command is not representative as it does not output anything in the given setting. And sure, I would have to use the output of the subprocess.Process class and objects (e. g. by using the communicate() method).

Not very clear what your goal is. Your code has this. Perhaps that's a clue?
current_app.process = subprocess.Popen(
['dd', 'if=/dev/zero', 'of=/dev/null'])
Trawl through the docs for subprocess. You'll find you can specify stdout and stderr.
http://docs.python.org/library/subprocess.html#module-subprocess
One strategy might be to capture stdout/err to file then rewrite that file to your http response.

One approach that will work (and is non-blocking, can serve multiple clients) is using Twisted:
Connect a ProcessProtocol
http://twistedmatrix.com/documents/current/core/howto/process.html
to a Twisted WebSocket server
https://github.com/tavendo/AutobahnPython
Disclosure: I am author of Autobahn.

A JavaScript frontend logging system that logs to our backend?

We have an established logging system for our server-side services. Specifically, our Django project makes heavy use of the Python logging module, so call calls to logger.info(), logger.warn() and logger.error() get picked up by our centralized logging system.
I would like an equivalent on our frontend, and I've got a few ideas:
There would be some sort of custom logging object exposed via JavaScript that would send messages to backend via an XmlHttpRequest.
I'd like to have equivalent logging levels on the client-side: debug, info, warning and error.
When we're developing locally (debug mode), I'd like those logging messages to be logged to the browser/Firebug console via console.log().
In production, debug messages should be dropped completely.
I recall seeing a way to capture all uncaught JavaScript exceptions, so these should be logged at the error level.
We're already using Google Analytics event tracking, and it'd be nice for whatever system we create to tie into that somehow.
Is this a good idea? How would you do this? Are there existing solutions?
(FWIW, we're using jQuery on the frontend.)
Update: Simplified question here: https://stackoverflow.com/questions/1423267/are-there-any-logging-frameworks-for-javascript

First, I wrote and maintain log4javascript, so I'm declaring my interest up front. I also use it every day in my work, so I have some experience of it as a user. Here's how I would deal with your questions, specifically relating to log4javascript:
Use log4javascript's AjaxAppender for server logging;
debug, info, warning and error are all supported, as well as trace and fatal;
Use a BrowserConsoleAppender to log to FireBug or the native browser console;
If you don't want to remove all debug logging calls from you production code, you can either adjust your logger's threshold (using log.setLevel(log4javascript.Level.ERROR), for example, which will suppress all log calls with priority less than ERROR). If you want to suppress all logging calls, you can drop in a stub version of log4javascript in your production code.
You'll need to write a bit of code to do this using window.onerror. Something like window.onerror = function(msg, file, line) { log.error("Error in " + file + " on line " + line + ": " + msg); }
I'm not sure how you want to tie in with Google Analytics. log4javascript has no particular support for it.

The idea sounds good here. Just be aware of what exactly it is you are looking to log client-side and have at it.
I would recommend using log4javascript for logging. The log4 api is pretty straight foward.

Here is another question from here on SO about this very issue.
Some recommendations are: log4js, log4javascript, and Blackbird.
FWIW, log4js was the accepted answer there. I don't have experience with any of these platforms, so I can't really recommend one over the other.