Recently I have developed a short Python script using the Flask framework to start a process over HTTP (see here). Now there is the question for an adequate technology to "pipe" the standard streams (I am especially interested in stdout and stderr) of started processes to the Web.
What would you say is the most suitable way to accomplish this? First I thought of websockets to fit perfectly. But then it seemed to me that most implementations veil their connected clients. I think that I need this information in order to be able to send the output to all of them.
Edit: I think that my question was a bit unclear: I want to see the output of the executed command in a web-interface. So I have to "convey" data from stdout/stderr of the executed process via HTTP somehow to the web-interface(s). It might be possible that the started command may run for some (longer) time. The example which invokes the dd command is not representative as it does not output anything in the given setting. And sure, I would have to use the output of the subprocess.Process class and objects (e. g. by using the communicate() method).
Not very clear what your goal is. Your code has this. Perhaps that's a clue?
current_app.process = subprocess.Popen(
['dd', 'if=/dev/zero', 'of=/dev/null'])
Trawl through the docs for subprocess. You'll find you can specify stdout and stderr.
http://docs.python.org/library/subprocess.html#module-subprocess
One strategy might be to capture stdout/err to file then rewrite that file to your http response.
One approach that will work (and is non-blocking, can serve multiple clients) is using Twisted:
Connect a ProcessProtocol
http://twistedmatrix.com/documents/current/core/howto/process.html
to a Twisted WebSocket server
https://github.com/tavendo/AutobahnPython
Disclosure: I am author of Autobahn.
Related
I'm using Azure service bus to send messages across containers in my K8s setup. Implementation of service that receives message is done in Node.js.From the doc page, here is the code I'm using to receive the messages.
serviceBusService.receiveQueueMessage('feedback', function(error, receivedMessage) {
if (!error) {
// Message received and deleted
console.log(receivedMessage);
//..
}
});
Everything works as expected for the first time but the messages are not received for the second time. It works as expected when the code snippet is kept inside the setInterval block. Which seems not the intended way of doing this.
Any ideas what could be wrong?
I have not used the js library for ServiceBus client, but the .NET library that has it seems similar methods. So if the js versjon works the same way, so then it receives only one message and you have to continue yourself.
In .Net library there are generally two ways:
You write your own while loop with whatever you think necessary
You use RegisterMessageHandler that seems to be absent from js library
The option number 2 generally does the same and starts the while loop inside the pump so that you don't have to implement it yourself.
You can take a look here on how the while loop is implemented and do something similar to it in js.
Like Mikhail Shilkov has mentioned previously in the comments, I suspect it only receives one message and I'd also recommend using the AMQP libraries.
I know this is a late reply, but just in case someone is running into issues, refer to the links below.
Version 7.0.0 of #azure/service-bus(based on AMQP) has been recently published.
#azure/service-bus - 7.0.0
Samples for 7.0.0
Guide to migrate from #azure/service-bus v1 to v7
The print statement in Python is not thread-safe. Is it safe to use console.log in Node.js concurrently?
If so, then is it also interleave-safe? That is, if multiple (even hundreds) of callbacks write to the console, can I be sure that the output won't be clobbered or interleaved?
Looking at the source code, it seems that Node.js queues concurrent attempts to write to a stream (here). On the other hand, console.log's substitution flags come from printf(3). If console.log wraps around printf, then that can interleave output on POSIX machines (as shown here).
Please show me where the async ._write(chunk, encoding, cb) is implemented inside Node.js in your response to this question.
EDIT: If it is fine to write to a stream concurrently, then why does this npm package exist?
Everything in node.js is basically "atomic". That's because node.js is single threaded - no code can ever be interrupted.
The events loop of nodejs is single thread, but all the async calls of nodejs are multi-threaded, it use libuv under the hood, libuv is library that use multi threads.
link:
https://medium.com/the-node-js-collection/what-you-should-know-to-really-understand-the-node-js-event-loop-and-its-metrics-c4907b19da4c
Based on what I see on my Node.js console it is NOT "interleave-safe".
I can see my console-output is sometimes "clobbered or interleaved". Not always. When I run my program it is maybe every 5th time that I see interleaved output from multiple log-statements.
This may of course depend on your Node.js version and the OS you are running it on. For the record my Node.js version is v12.13.0 and OS is Windows 10.0.19042.
I'm just starting to use nodejs. I've done some basic learning course but I miss any basic example to start becaue I'm having problems to deploy an scenario for the first time.
The scenario I need to cover is:
I just need to write and read data from an Access.MDB file.
I have a basic example that works if I execute the js with the sentence \node appnodemdb.js from the command line.
But I need to execute it from another js in a web.
So I'd appreciate a lot if anyone could send me an example about execute it from an other js without the command line.
Thanks a lot for your help!
You can't call directly some NodeJS code with the browser. To achieve what you want, the simplest way to do is:
Create an HTTP server with a URL to will launch your write/write action and return some value (with Express for example).
This URL will be called by your JS in the browser (using XHR/Ajax) and process the data returned.
They are tons of alternative to do this, but since you tell us that you are beginner, I suggest this approach.
I have an existing Javascript/HTML Application. I wanted to use power of R Programming's scientific computing and graphics.
My aim is to,
Send some data from Javascript app.
Call predefined R functions with the data input.
Get the output get the output in the form of both text and graphics.
Display it in the HTML page.
How to achieve this,
Should I run R continuously, use something like web sockets and connect to R? If doing How to pass R scripts to execute and get the output pack?
There is Rserve. There are some nodeJS implementation for Rserve. But problem with this is, each line of code should be passed through the evaluate commands. Even though if I do so, how to handle the graph output?
I explored a bit of openCPU. If using openCPU R package, R should be continuously Running with opencpu library and each we start R and openCPU, it starts with different port number. And if i close the R session, opencpu server also terminates.
If I install standalone opencpu server in my machine, how to use R with this? I've installed openCPU standalone server and a kind of stuck after that.
How should I proceed, What should I do to accomplish my task. I'm like a kind of don't know which direction to go. Please throw some light on this. I'm sure most people would need this.
I have worked with shiny, but in this case, I can not make use of it. Need to connect R from external Web Application.
FastRWeb sounds like it would be perfect for your needs. From the documentation:
FastRWeb is an infrastructure that allows any webserver to use R
scripts for generating content on the fly, such as web pages or
graphics. URLs are mapped to scripts and can have optional arguments
that are passed to the R function run from the script. For example
http://my.server/cgi-bin/R/foo.png?n=100 would cause FastRWeb to look
up a script foo.png.R, source it and call run(n="100"). So for example
the script could be as simple as
run <- function(n=10, ...) {
p <- WebPlot(800, 600)
n <- as.integer(n)
plot(rnorm(n), rnorm(n), col=2, pch=19)
p
}
This can potentially then be called using JavaScript to dynamically load images and display them.
You might also like to think about shiny, though that's more of a complete solution.
You can call R from javascript efficiently using Rserve package. There is javascript implementation of Rserve client available rserve-js.
Additionally you may find it interesting the httupv implementation of R as a service described in this Suggestions needed for building R server REST API's that I can call from external app?.
I'm trying to use Python to download a voice file (mp3) from this site:
I've tried to use mechanize and twill but I haven't got the proper result. I've heard about selenium as well. Anyway, I don't even know if it makes sense technically (mechanize + Javascript).
I was looking for an answer and finally I've found this solution.
It seems like a similar problem to mine but I have absoutely no idea what should I put instead of the 4th line.
If you have any proposition (maybe completely different to mine) I'll be glad to see it.
(Moving to answer to get more space)
Thanks for pointing me to selenium, I didn't know that and it looks cool; anyways: I had a quick look and the js seems to fire an ajax request, that yields no apparent result -- I guess the audio transfer is done some way in the background via flash or so; anyways: what about using the service they offer: developer.ivona.com instead of trying hacking your webservice out of the demo interface..?
Anyways, here it is the hacked web-service you asked for:
This is the URL you want to GET:
"http://www.ivona.com/voicetest.php?rtr=1&t2r=%(the_text)s..&v2r=dXNfc2FsbGk.&lang=us" % dict(
the_text=base64.b64encode("Hello, world!"),
the_voice=base64.b64encode("us_salli"),
)
You can test that with, for example, mplayer:
mplayer "http://www.ivona.com/voicetest.php?rtr=1&t2r=SGVsbG8sIHdvcmxkIQ==..&v2r=dXNfc2FsbGk.&lang=us"
And, from Python, use urllib to retrieve and store somewhere.
Beware that, since this is an unauthorized use, you might get blocked / incur in legal issues / etc. -- don't use in a production application!