JavaScript/Node + cURL - javascript

I have a line of cURL:
curl -X POST --form "file=#Calvin Harris - Thinking About You (Tez Cadey Remix)_165299184_soundcloud.mp3" https://api.idolondemand.com/1/api/async/recognizespeech/v1
I'm building a hybrid mobile app with Meteor/Ionic as the framework. Therefore, I have access to any Node library that leverages cURL.
Can anyone:
1) Suggest one of the many node-curl libraries
2) Show me how to properly output the above cURL line in the context of the right library?
My primary issue that is stopping me is the --form flag. I've poured over several libraries/docs and none explicitly reference how to use this form flag. I cannot drop this flag, it's a requirement of the API.

You could just use node's fs and https APIs
var fs = require('fs');
var https = require('https');
var rs = fs.createReadStream(
'Calvin Harris - Thinking About You (Tez Cadey Remix)_165299184_soundcloud.mp3'
);
var req = http.request({
hostname: 'api.idolondemand.com',
path: '/1/api/async/recognizespeech/v1',
method: 'POST'
}, function(res) {
// do something when you get a response
});
rs.pipe(req);
Or use the requests module as #ffk mentioned

Related

zsh vs bash on query string object

I'm running a very basic NodeJS application just to mess around with and learn ZSH but the queryStringObject I've defined from my parsedUrl in bash returns {fizz:'buzz'} and zsh returns [Object: null prototype] {}. What can I do in ZSH or in my app to make this console.log the JSON formatting instead of what ZSH is currently giving me?
var http = require('http');
var url = require('url');
var server = http.createServer(function(req,res){
var parsedUrl = url.parse(req.url, true); //true indicates to include query object
var queryStringObject = parsedUrl.query;
res.end('Hello World\n'); //curls to dom
console.log('Request received with query string:',queryStringObject);
}
I'll happily go back to using Bash to get what I'm used to seeing or simply use postman for this kind of testing, but I'm trying to learn little bits and pieces of ZSH as I go along and this behavior is likely to hit me often so I'd like to know how to best handle it.
It isn't a bash, zsh or any specific shell problem. Try JSON.parse() to print your object in JSON format.
console.log('Request received with query string:',JSON.parse(queryStringObject));

How to connect a node server without browser

I am using node-jet library as message broker using a websocket.
In my code, there is a deamon and there are peers connect to it. In libary documentation its says peers can connect either by websocket or trivial tcp connection (never used second one before)
Here is server code I try to connect (its running when I try to connect)
https://github.com/lipp/node-jet/blob/master/examples/todo/todo-server.js
Everything works fine if I run my peer javascript inside a browser however, I am not able to run it from console.
var peer3 = new jet.Peer({
url: 'wss://217.78.109.178:8090'
//url: (window.location.protocol === 'http:' ? 'ws://' : 'wss://') +
window.location.host
})
documentation says it will use trivial tcp if no url provided so I also tried this. Again works in browser but not standalone. Any idea ? how can I solve this problem ?
var peer3 = new jet.Peer({
})
$ node Peer.js
Peer: connect failed { no remote stack
name: 'jet.ConnectionClosed',
message: '',
url: 'https://github.com/lipp/node-
jet/blob/master/doc/peer.markdown#jetconnectionclosed'
}

How open a password protected image url with python? [duplicate]

Im trying to do a HTTPS GET with basic authentication using python. Im very new to python and the guides seem to use diffrent librarys to do things. (http.client, httplib and urllib). Can anyone show me how its done? How can you tell the standard library to use?
In Python 3 the following will work. I am using the lower level http.client from the standard library. Also check out section 2 of rfc2617 for details of basic authorization. This code won't check the certificate is valid, but will set up a https connection. See the http.client docs on how to do that.
from http.client import HTTPSConnection
from base64 import b64encode
# Authorization token: we need to base 64 encode it
# and then decode it to acsii as python 3 stores it as a byte string
def basic_auth(username, password):
token = b64encode(f"{username}:{password}".encode('utf-8')).decode("ascii")
return f'Basic {token}'
username = "user_name"
password = "password"
#This sets up the https connection
c = HTTPSConnection("www.google.com")
#then connect
headers = { 'Authorization' : basic_auth(username, password) }
c.request('GET', '/', headers=headers)
#get the response back
res = c.getresponse()
# at this point you could check the status etc
# this gets the page text
data = res.read()
Use the power of Python and lean on one of the best libraries around: requests
import requests
r = requests.get('https://my.website.com/rest/path', auth=('myusername', 'mybasicpass'))
print(r.text)
Variable r (requests response) has a lot more parameters that you can use. Best thing is to pop into the interactive interpreter and play around with it, and/or read requests docs.
ubuntu#hostname:/home/ubuntu$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> r = requests.get('https://my.website.com/rest/path', auth=('myusername', 'mybasicpass'))
>>> dir(r)
['__attrs__', '__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_content', '_content_consumed', 'apparent_encoding', 'close', 'connection', 'content', 'cookies', 'elapsed', 'encoding', 'headers', 'history', 'iter_content', 'iter_lines', 'json', 'links', 'ok', 'raise_for_status', 'raw', 'reason', 'request', 'status_code', 'text', 'url']
>>> r.content
b'{"battery_status":0,"margin_status":0,"timestamp_status":null,"req_status":0}'
>>> r.text
'{"battery_status":0,"margin_status":0,"timestamp_status":null,"req_status":0}'
>>> r.status_code
200
>>> r.headers
CaseInsensitiveDict({'x-powered-by': 'Express', 'content-length': '77', 'date': 'Fri, 20 May 2016 02:06:18 GMT', 'server': 'nginx/1.6.3', 'connection': 'keep-alive', 'content-type': 'application/json; charset=utf-8'})
Update: OP uses Python 3. So adding an example using httplib2
import httplib2
h = httplib2.Http(".cache")
h.add_credentials('name', 'password') # Basic authentication
resp, content = h.request("https://host/path/to/resource", "POST", body="foobar")
The below works for python 2.6:
I use pycurl a lot in production for a process which does upwards of 10 million requests per day.
You'll need to import the following first.
import pycurl
import cStringIO
import base64
Part of the basic authentication header consists of the username and password encoded as Base64.
headers = { 'Authorization' : 'Basic %s' % base64.b64encode("username:password") }
In the HTTP header you will see this line Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=. The encoded string changes depending on your username and password.
We now need a place to write our HTTP response to and a curl connection handle.
response = cStringIO.StringIO()
conn = pycurl.Curl()
We can set various curl options. For a complete list of options, see this. The linked documentation is for the libcurl API, but the options does not change for other language bindings.
conn.setopt(pycurl.VERBOSE, 1)
conn.setopt(pycurlHTTPHEADER, ["%s: %s" % t for t in headers.items()])
conn.setopt(pycurl.URL, "https://host/path/to/resource")
conn.setopt(pycurl.POST, 1)
If you do not need to verify certificate. Warning: This is insecure. Similar to running curl -k or curl --insecure.
conn.setopt(pycurl.SSL_VERIFYPEER, False)
conn.setopt(pycurl.SSL_VERIFYHOST, False)
Call cStringIO.write for storing the HTTP response.
conn.setopt(pycurl.WRITEFUNCTION, response.write)
When you're making a POST request.
post_body = "foobar"
conn.setopt(pycurl.POSTFIELDS, post_body)
Make the actual request now.
conn.perform()
Do something based on the HTTP response code.
http_code = conn.getinfo(pycurl.HTTP_CODE)
if http_code is 200:
print response.getvalue()
A correct way to do basic auth in Python3 urllib.request with certificate validation follows.
Note that certifi is not mandatory. You can use your OS bundle (likely *nix only) or distribute Mozilla's CA Bundle yourself. Or if the hosts you communicate with are just a few, concatenate CA file yourself from the hosts' CAs, which can reduce the risk of MitM attack caused by another corrupt CA.
#!/usr/bin/env python3
import urllib.request
import ssl
import certifi
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(certifi.where())
httpsHandler = urllib.request.HTTPSHandler(context = context)
manager = urllib.request.HTTPPasswordMgrWithDefaultRealm()
manager.add_password(None, 'https://domain.com/', 'username', 'password')
authHandler = urllib.request.HTTPBasicAuthHandler(manager)
opener = urllib.request.build_opener(httpsHandler, authHandler)
# Used globally for all urllib.request requests.
# If it doesn't fit your design, use opener directly.
urllib.request.install_opener(opener)
response = urllib.request.urlopen('https://domain.com/some/path')
print(response.read())
Based on the #AndrewCox 's answer with some minor improvements:
from http.client import HTTPSConnection
from base64 import b64encode
client = HTTPSConnection("www.google.com")
user = "user_name"
password = "password"
headers = {
"Authorization": "Basic {}".format(
b64encode(bytes(f"{user}:{password}", "utf-8")).decode("ascii")
)
}
client.request('GET', '/', headers=headers)
res = client.getresponse()
data = res.read()
Note, you should set encoding if you use bytes function instead of b"".
requests.get(url, auth=requests.auth.HTTPBasicAuth(username=token, password=''))
If with token, password should be ''.
It works for me.
using only standard modules and no manual header encoding
...which seems to be the intended and most portable way
the concept of python urllib is to group the numerous attributes of the request into various managers/directors/contexts... which then process their parts:
import urllib.request, ssl
# to avoid verifying ssl certificates
httpsHa = urllib.request.HTTPSHandler(context= ssl._create_unverified_context())
# setting up realm+urls+user-password auth
# (top_level_url may be sequence, also the complete url, realm None is default)
top_level_url = 'https://ip:port_or_domain'
# of the std managers, this can send user+passwd in one go,
# not after HTTP req->401 sequence
password_mgr = urllib.request.HTTPPasswordMgrWithPriorAuth()
password_mgr.add_password(None, top_level_url, "user", "password", is_authenticated=True)
handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
# create OpenerDirector
opener = urllib.request.build_opener(handler, httpsHa)
url = top_level_url + '/some_url?some_query...'
response = opener.open(url)
print(response.read())
GET & POST request is usually used to submit forms. Here is a brief example of its usage
Views.py
def index(request)
col1 = float(request.GET.get('col1'))
index.html
<div class="form-group col-md-2">
<label for="Col 1">Price</label>
<input type="number" class="form-control" id="col1" name="col1">
</div>

Indesign javascript Socket always performs requests using host: localhost

I'm trying to use a Socket connection to read a file on a remote website. So far, my code:
conn = new Socket;
if( conn.open( 'example.com:80' ) ) {
conn.write( 'GET /indesign-page/ HTTP/1.0' + "\n\n" );
reply = conn.read(999999);
conn.close();
} else {
alert( 'Problem connecting to server' );
}
The socket connects to example.com fine, but the request comes across as this:
GET http://localhost/indesign-page/ HTTP/1.0
when it should be this:
GET http://example.com/indesign-page/ HTTP/1.0
I've tried changing the conn.write parameters to 'GET http://example.com/indesign-page/ ...', but then it comes across as:
GET http://localhosthttp://example.com/indesign-page/ HTTP/1.0
The webserver requires that the host be set correctly to serve correctly.
You need to set the "Host" header.
conn.write( 'GET /indesign-page/ HTTP/1.0' + "Host: example.com\r\n" + "\n\n" );
Because conn.open( 'example.com:80' ) means find example.com's server ip then connect that ip address at 80 port, so the web server does not know that you had resolved example.com before connected to it.
Do you need to use a manual socket object? On Adobe's Community Site there's a mention to this already created FTP Script where you could call a GET or PUT to a file on a FTP server.
Otherwise which OS are you using? If you'll always be on a Mac, you could shell out to an AppleScript command and place the file anywhere you'd like:
var command = 'do shell script "curl http://localhost/indesign-page"';
var response = app.doScript(command, ScriptLanguage.APPLESCRIPT_LANGUAGE);
The nice thing about the AppleScript is that you can execute the command manually using the AppleScript Editor (or Script Editor if you're earlier than 10.6).

Create HTTPS client in NodeJS

I've been having a heck of a time figuring out how to use Node.js (v0.3.8) to securely connect to an HTTP server. I have the following code:
var http = require("http");
var client = http.createClient(443, host, /* secure= */ true);
var request = client.request("GET", relativeUrl, { host: host });
When I run it, I get:
node.js:116
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Parse Error
at Client.onData [as ondata] (http.js:1287:27)
at Client._onReadable (net.js:648:27)
at IOWatcher.onReadable [as callback] (net.js:156:10)
I've been Googling for answers for the past half hour, and have read the documentation at http://nodejs.org/ . What am I missing?
It turns out I was using an old version of the Node documentation, which didn't include a reference to the https module. Referring to the current docs, I found http://nodejs.org/docs/latest/api/https.html#https_https_get_options_callback, which provides an example:
https.get({ host: 'encrypted.google.com', path: '/' }, function (res) { … });
If you are using node.js as a client you should be able to simply substitute http for https.
That is according to the following website
https://github.com/danwrong/Restler/
"Transparently handle SSL (just specify https in the URL)"

Categories

Resources