I have got a c program,which reads integer values from stdin. I wrote a nodejs program to execute the c file, and the nodejs program will read a text file(containg numbers in multiple lines) and pipe this data to stdin of the child process.
The problem is, if the no of inputs in the txt file is less than the expected number then the child process will be supplied with value 0. I want the child process to wait until the data is received.
c program
#include<stdio.h>
int main(){
int a;
printf("hekllowworls");
scanf("%d",&a);
printf("%d",a);
scanf("%d",&a);
printf("%d",a);
scanf("%d",&a);
printf("%d",a);
}
Node JS Program
var fs = require('fs'),
cp = require('child_process');
stream = fs.createReadStream('myfile.txt');
var obj = cp.spawn('/home/arju/Desktop/exec/a.out');
stream.pipe(obj.stdin);
obj.stdout.on('data',function(data){
console.log(data.toString());
});
obj.on('error',function(err){
console.log(err);
});
TextFile - Myfile.txt
10
20
According to man scanf
The value EOF is returned if the end of input is reached before either
the first successful conversion or a matching failure occurs. EOF is
also returned if a read error occurs, in which case the error indicator
for the stream (see ferror(3)) is set, and errno is set indicate the
error.
I feel that you should use something like that:
r = scanf("%d",&a);
if (r != EOF) { ...
Logically, simple wait for data could look like this:
while(r = scanf("%d",&a)) {
if (r == EOF) continue;
printf("%d",a);
}
EDIT
As i understand, you should use unbuffered IO operations, like read/write syscals. Try this:
int main(){
int n;
char buf[255];
while(1) {
while((n = read(0,buf,sizeof(buf))) != 0){
write(1,buf,n);
}
}
}
Get the info from: write() to stdout and printf output not interleaved?, good answer, check it out.
Your C program reads value from buffer. In my first example it awaits data to fill the output buffer, so i had no output.
EDIT 2: How to turn the buffer off
With small modifications of code, you can use fflush(stdout) after each printf call.
If you can't edit the code, take a look at this answer from unix.stackexchange: https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe, there are two good solutions. (i'd prefer the second, because it is from coreutils).
Related
Specifically, how to connect <input type="file"> with this function in Go?
I know there is "syscall/js" package, but I didn't find any examples with file reading.
func parseCSVFile(filePath string) []LabelWithFeatures {
fileContent, _ := ioutil.ReadFile(filePath)
lines := bytes.Split(fileContent, newline)
numRows := len(lines)
labelsWithFeatures := make([]LabelWithFeatures, numRows-2)
for i, line := range lines {
// skip headers
if i == 0 || i == numRows-1 {
continue
}
labelsWithFeatures[i-1] = NewLabelWithFeatures(bytes.Split(line, comma))
}
return labelsWithFeatures
}
I've wanted a satisfactory answer for this for years, finally figured it out the other night.
You can essentially boil the whole thing down to:
fileInput := document.Call("getElementById", "fileInput")
fileInput.Set("oninput", js.FuncOf(func(v js.Value, x []js.Value) any {
fileInput.Get("files").Call("item", 0).Call("arrayBuffer").Call("then", js.FuncOf(func(v js.Value, x []js.Value) any {
data := js.Global().Get("Uint8Array").New(x[0])
dst := make([]byte, data.Get("length").Int())
js.CopyBytesToGo(dst, data)
// the data from the file is in dst - do what you want with it
return nil
}))
return nil
}))
I wrote a little blog post about it here with the working WASM code running at the bottom
https://donatstudios.com/Read-User-Files-With-Go-WASM
You can't really access the filesystem in the browser. wasm_exec.js is used to execute Go webassembly in the browser, it mocks out some filesystem functionality, but I don't think it's very useful to you: https://github.com/golang/go/blob/9d23975d/misc/wasm/wasm_exec.js#L41-L73
The file read method even returns an error by default.
You mentioned <input type="file">. You can get bytes from an uploaded file: Getting byte array through input type = file. You could then pass those bytes to the Golang wasm runtime.
Define a global syscall/js callback in your Go code and call it from the browser to pass the bytes down to the Go runtime.
I would look for blogposts on how to define callbacks from within the Go runtime. Also look out for changes between go 1.11 and 1.12, the api has breaking changes.
I'm trying to run a python script using my javascript file through the child_process.spawn system, but it seems to never run on the aws lambda.
The relevant code is :
getEntities: function (){
var spawn = require('child_process').spawn;
var py = spawn('python', ['mainPythonFile.py']);
var outputString = "starting string";
console.log("BEFORE ANY INPUT");
py.stdout.on('data', function (data) {
console.log("----Getting information from the python script!---");
outputString += data.toString();
console.log(outputString);
});
py.stdout.on('end', function (){
console.log("===hello from the end call in python files===");
console.log("My output : " + outputString);
});
console.log("NO INPUT CHANGED??");
return outputString;
}
These files are in the same level of the folder structure (surface level).
The python file being run is quite simple and only contains a few print statements:
MainPythonFile:
import sys;
print("Hello There");
print("My name is Waffles");
print("Potato Waffles");
sys.stdout.flush()
The output I get from the aws service is this :
BEFORE ANY INPUT
NO INPUT CHANGED??
starting string
I've tried different paths on trying to access the python file, such as
*mainPythonFile.py ./mainPythonFile.py etc.
I think the code seems to be fine as this works on my local machine, but there's a subtlety of trying to make it run on AWS that I cannot understand.
I can provide any other info if need be.
NOTE: the "getEntities" function is being called by another node.js file, but I moved the code to the calling function, I get the same result.
Due to the asynchronous nature of JS, as explained by Chris, the function reaches the "return" statement before the "end" in the spawned thread is actually called.
This means the code never got a chance to actually set the correct output text.
I changed my function calls to take in a callback, that would then respond when the program had replied with the information.
My new function is slightly changed to this (without the prints):
getEntities: function(callbackFunction, that){
var spawn = require('child_process').spawn;
var py = spawn('python', ['mainPythonFile.py']);
var outputString = "starting string";
py.stdout.on('data', function (data) {
outputString += data.toString();
});
// that = "this == alexa" that's passed in as input.
py.stdout.on('end', function (){
callbackFunction(outputString, that);
});
The function that called this function is now as follows :
HelperFunctions.getEntities(function(returnString,that){
that.response.speak(returnString);
that.emit(':responseReady');
}, this);
I'm sure there's a prettier way to do this, but this seems to be working for now. Thanks to ChrisG
I have a Javascript file being executed by Nodejs as a script in a Unix shell pipeline. The script needs to read (a few thousand lines) of text from STDIN, parse it for some bits, then return a json stream on STDOUT with the results of its work.
var readline = require('readline');
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout,
terminal: false
});
Then I have some functions to process the data, then I setup an event listener that gets called on each line of input and does all the processing I need and appends it to an array:
function extract_references (data) {
...
references.push(item);
}
rl.on('line', extract_references);
So far so good, my data array ends up populated with all the components I want. The trouble comes when I try to write it back out to STDOUT. I've tried a couple things being triggered by the close event on the input stream:
function output_references () {
var output = JSON.stringify(references, null, ' ');
process.stdout.write(output);
}
rl.on('close', output_references);
This works fine if the input stream is relatively short (a couple hundred lines) but when I crank it up to my full data set I start getting truncated output and an error such as:
write error: Resource temporarily unavailable
...from the next command in my pipeline.
I've also tried using rl.write(), but that doesn't seem to give me anything at all on STDOUT. What am I doing wrong and what is the correct way to handle a stream like this?
I'm writing a GTK javascript program that downloads a file and writes it to disk. Here's what my code looks like:
const Gio = imports.gi.Gio;
const Soup = imports.gi.Soup;
// start an http session to make http requests
let _httpSession = new Soup.SessionAsync();
Soup.Session.prototype.add_feature.call(_httpSession, new Soup.ProxyResolverDefault());
// open the file
let file = Gio.file_new_for_path(path);
let fstream = file.replace(null, false, Gio.FileCreateFlags.NONE, null);
// start the download
let request = Soup.Message.new('GET', url);
request.connect('got_chunk', Lang.bind(this, function(message, chunk){
// write each chunk to file
fstream.write(chunk, chunk.length, null);
}));
this._httpSession.queue_message(request, function(_httpSession, message) {
// close the file
fstream.close(null);
});
I get an error on the fstream.write() line:
JS ERROR: !!! Exception was: Error: Unhandled GType GCancellable unpacking GArgument from Number
JS ERROR: !!! message = '"Unhandled GType GCancellable unpacking GArgument from Number"'
JS ERROR: !!! fileName = '"./torbrowser-launcher"'
JS ERROR: !!! lineNumber = '402'
JS ERROR: !!! stack = '"([object _private_Soup_Message],[object _private_Soup_Buffer])#./torbrowser-launcher:402
("2.3.25-2")#./torbrowser-launcher:122
wrapper("2.3.25-2")#/usr/share/gjs-1.0/lang.js:204
("2.3.25-2")#/usr/share/gjs-1.0/lang.js:145
("2.3.25-2")#/usr/share/gjs-1.0/lang.js:239
#./torbrowser-launcher:489
"'
The only reference to this error that I can find is in this thread: https://mail.gnome.org/archives/gnome-shell-list/2012-July/msg00126.html
That person ended up giving up and porting his code to python.
I'm also confused by what the 'got_chunk' callback passes. The chunk field is a Soup.Buffer (http://www.roojs.com/seed/gir-1.2-gtk-3.0/gjs/Soup.Buffer.html). I can get its length with chunk.length, but when I try printing chunk.data it's undefined. When I just print chunk it prints: [object _private_Soup_Buffer].
fstream is a Gio.FileOutputStream (http://www.roojs.com/seed/gir-1.2-gtk-3.0/gjs/Gio.FileOutputStream.html). The write method is: write(String buffer, guint32 count, Cancellable cancellable), and cancellable is optional. Weirdly enough, if I replace the write line with this I still get the exact same error:
fstream.write('test ', 5, null);
I was hitting exactly the same problem. After a lot of trial and error, it boiled down to two issues with the write() call:
It seems that the documentation of the write function you are using (http://www.roojs.com/seed/gir-1.2-gtk-3.0/gjs/Gio.FileOutputStream.html) is wrong; the write method signature is (as far as I can tell):
write(String buffer, Cancellable cancellable, guint32 count)
Yet if you just use fstream.write(chunk, null, chunk.length); you will write a file full of zeros. I don't know why (something to do with the way GJS binds to the underlying C library) but you should use chunk.get_data() instead of just chunk. I.e. replace the write call in your code with:
fstream.write(chunk.get_data(), null, chunk.length);
I'm trying to write a function, that would use native openssl to do some RSA heavy-lifting for me, rather than using a js RSA library. The target is to
Read binary data from a file
Do some processing in the node process, using JS, resulting in a Buffer containing binary data
Write the buffer to the stdin stream of the exec command
RSA encrypt/decrypt the data and write it to the stdout stream
Get the input data back to a Buffer in the JS-process for further processing
The child process module in Node has an exec command, but I fail to see how I can pipe the input to the process and pipe it back to my process. Basically I'd like to execute the following type of command, but without having to rely on writing things to files (didn't check the exact syntax of openssl)
cat the_binary_file.data | openssl -encrypt -inkey key_file.pem -certin > the_output_stream
I could do this by writing a temp file, but I'd like to avoid it, if possible. Spawning a child process allows me access to stdin/out but haven't found this functionality for exec.
Is there a clean way to do this in the way I drafted here? Is there some alternative way of using openssl for this, e.g. some native bindings for openssl lib, that would allow me to do this without relying on the command line?
You've mentioned spawn but seem to think you can't use it. Possibly showing my ignorance here, but it seems like it should be just what you're looking for: Launch openssl via spawn, then write to child.stdin and read from child.stdout. Something very roughly like this completely untested code:
var util = require('util'),
spawn = require('child_process').spawn;
function sslencrypt(buffer_to_encrypt, callback) {
var ssl = spawn('openssl', ['-encrypt', '-inkey', ',key_file.pem', '-certin']),
result = new Buffer(SOME_APPROPRIATE_SIZE),
resultSize = 0;
ssl.stdout.on('data', function (data) {
// Save up the result (or perhaps just call the callback repeatedly
// with it as it comes, whatever)
if (data.length + resultSize > result.length) {
// Too much data, our SOME_APPROPRIATE_SIZE above wasn't big enough
}
else {
// Append to our buffer
resultSize += data.length;
data.copy(result);
}
});
ssl.stderr.on('data', function (data) {
// Handle error output
});
ssl.on('exit', function (code) {
// Done, trigger your callback (perhaps check `code` here)
callback(result, resultSize);
});
// Write the buffer
ssl.stdin.write(buffer_to_encrypt);
}
You should be able to set encoding to binary when you make a call to exec, like..
exec("openssl output_something_in_binary", {encoding: 'binary'}, function(err, out, err) {
//do something with out - which is in the binary format
});
If you want to write out the content of "out" in binary, make sure to set the encoding to binary again, like..
fs.writeFile("out.bin", out, {encoding: 'binary'});
I hope this helps!