Develop Reference

How to bring a gRPC defined API to the web browser

We want to build a Javascript/HTML gui for our gRPC-microservices. Since gRPC is not supported on the browser side, we thought of using web-sockets to connect to a node.js server, which calls the target service via grpc.
We struggle to find an elegant solution to do this. Especially, since we use gRPC streams to push events between our micro-services.
It seems that we need a second RPC system, just to communicate between the front end and the node.js server. This seems to be a lot of overhead and additional code that must be maintained.
Does anyone have experience doing something like this or has an idea how this could be solved?

Edit: Since Oct 23,2018 the gRPC-Web project is GA, which might be the most official/standardized way to solve your problem. (Even if it's already 2018 now... ;) )
From the GA-Blog: "gRPC-Web, just like gRPC, lets you define the service “contract” between client (web) and backend gRPC services using Protocol Buffers. The client can then be auto generated. [...]"
We recently built gRPC-Web (https://github.com/improbable-eng/grpc-web) - a browser client and server wrapper that follows the proposed gRPC-Web protocol. The example in that repo should provide a good starting point.
It requires either a standalone proxy or a wrapper for your gRPC server if you're using Golang. The proxy/wrapper modifies the response to package the trailers in the response body so that they can be read by the browser.
Disclosure: I'm a maintainer of the project.

Unfortunately, there isn't any good answer for you yet.
Supporting streaming RPCs from the browser fully requires HTTP2 trailers to be supported by the browsers, and at the time of the writing of this answer, they aren't.
See this issue for the discussion on the topic.
Otherwise, yes, you'd require a full translation system between WebSockets and gRPC. Maybe getting inspiration from grpc-gateway could be the start of such a project, but that's still a very long shot.

An official grpc-web (beta) implementation was released on 3/23/2018. You can find it at
https://github.com/grpc/grpc-web
The following instructions are taken from the README:
Define your gRPC service:
service EchoService {
rpc Echo(EchoRequest) returns (EchoResponse);
rpc ServerStreamingEcho(ServerStreamingEchoRequest)
returns (stream ServerStreamingEchoResponse);
}
Build the server in whatever language you want.
Create your JS client to make calls from the browser:
var echoService = new proto.grpc.gateway.testing.EchoServiceClient(
'http://localhost:8080');
Make a unary RPC call
var unaryRequest = new proto.grpc.gateway.testing.EchoRequest();
unaryRequest.setMessage(msg);
echoService.echo(unaryRequest, {},
function(err, response) {
console.log(response.getMessage());
});
Streams from the server to the browser are supported:
var stream = echoService.serverStreamingEcho(streamRequest, {});
stream.on('data', function(response) {
console.log(response.getMessage());
});
Bidirectional streams are NOT supported:
This is a work in progress and on the grpc-web roadmap. While there is an example protobuf showing bidi streaming, this comment make it clear that this example doesn't actually work yet.
Hopefully this will change soon. :)

https://github.com/tmc/grpc-websocket-proxy sounds like it may meet your needs. This translates json over web sockets to grpc (layer on top of grpc-gateway).

The grpc people at https://github.com/grpc/ are currently building a js implementation.
The repro is at https://github.com/grpc/grpc-web (gives 404 ->) which is currently (2016-12-20) in early access so you need to request access.

GRPC Bus WebSocket Proxy does exactly this by proxying all GRPC calls over a WebSocket connection to give you something that looks very similar to the Node GRPC API in the browser. Unlike GRPC-Gateway, it works with both streaming requests and streaming responses, as well as non-streaming calls.
There is both a server and client component.
The GRPC Bus WebSocket Proxy server can be run with Docker by doing docker run gabrielgrant/grpc-bus-websocket-proxy
On the browser side, you'll need to install the GRPC Bus WebSocket Proxy client with npm install grpc-bus-websocket-client
and then create a new GBC object with: new GBC(<grpc-bus-websocket-proxy address>, <protofile-url>, <service map>)
For example:
var GBC = require("grpc-bus-websocket-client");
new GBC("ws://localhost:8080/", 'helloworld.proto', {helloworld: {Greeter: 'localhost:50051'}})
.connect()
.then(function(gbc) {
gbc.services.helloworld.Greeter.sayHello({name: 'Gabriel'}, function(err, res){
console.log(res);
}); // --> Hello Gabriel
});
The client library expects to be able to download the .proto file with an AJAX request. The service-map provides the URLs of the different services defined in your proto file as seen by the proxy server.
For more details, see the GRPC Bus WebSocket Proxy client README

I see a lot of answers didn't point to a bidirectional solution over WebSocket, as the OP asked for browser support.
You may use JSON-RPC instead of gRPC, to get a bidirectional RPC over WebSocket, which supports a lot more, including WebRTC (browser to browser).
I guess it could be modified to support gRPC if you really need this type of serialization.
However, for browser tab to browser tab, request objects are not serializsed and are transfered natively, and the same with NodeJS cluster or thread workers, which offers a lot more performance.
Also, you can transfer "pointers" to SharedArrayBuffer, instead of serializing through the gRPC format.
JSON serialization and deserialization in V8 is also unbeatable.
https://github.com/bigstepinc/jsonrpc-bidirectional

Looking at the current solutions with gRPC over web, here is what's available out there at the time of writing this (and what I found):
gRPC-web: requires TypeScript for client
gRPC-web-proxy: requires Go
gRPC-gateway: requires .proto modification and decorations
gRPC-bus-websocket-proxy-server: as of writing this document it lacks tests and seems abandoned (edit: look at the comments by the original author!)
gRPC-dynamic-gateway: a bit of an overkill for simple gRPC services and authentication is awkward
gRPC-bus: requires something for the transport
I also want to shamelessly plug my own solution which I wrote for my company and it's being used in production to proxy requests to a gRPC service that only includes unary and server streaming calls:
gRPC-express
Every inch of the code is covered by tests. It's an Express middleware so it needs no additional modifications to your gRPC setup. You can also delegate HTTP authentication to Express (e.g with Passport).

LDAP Bind Error using node.js and ldapjs

I am trying to implement a basic ldap bind with the following node.js file. Unfortunately, I keep getting a bind error with code 128. I looked online and found no references of code 128. The LDAP server I am trying to search is an eDirectory. Does anyone have any experience with this or have you had similar problems? My node version is v0.10.22 and my ldapjs version is v0.7.1
var ldap = require('ldapjs');
var creds = {
url: "ldaps://ldap.url.com:636",
bindDN: "cn=ldap,o=com"
};
var opts = {
filter: "(cn=username)",
scope: "sub"
};
function authDN(client, dn, password, cb) {
client.bind(dn, password, function (err) {
client.unbind();
cb(err === null, err);
});
}
function output(res, err) {
if (res) {
console.log('success');
} else {
console.log(['Error',err.code, err.dn, err.message ]);
}
}
var client = ldap.createClient(creds);
authDN(client, '(cn=username)', 'password', output);

This authenticates when i added the following to the top of my file:
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
I haven't researched enough to know why this works but I found this answer here: https://github.com/mikeal/request/issues/418

In general when debugging an eDirectory issue, ask for access to iMonitor, so you can look at DStrace with the +LDAP option. That would show you what the LDAP server is sending back, making troubleshooting easier.

To augment Kaiser's answer, an explanation on why adding process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"; to the code may work is found at the top of this link: https://github.com/visionmedia/superagent/issues/205.
Potential fixes:
Add process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0; to the top of your script for node v0.10.x (and above)
Setup a trusted CA certificate on the server instead of a self-signed certificate (must have server admin rights and pay for a valid cert)
Use the LDAP server IP or load balancer IP instead of dns for the url parameter.
Because you are using the secure protocol (ldaps:// instead of ldap://), and I'm assuming you are trying to connect to a server with a self-signed certificate, you will get a failure if using node v0.10.x (and probably all later versions as well) and the code/module you are using doesn't specifically set the process.env.NODE_TLS_REJECT_UNAUTHORIZED to false.
NODE_TLS_REJECT_UNAUTHORIZED was changed to true by default for a reason. If you choose to set NODE_TLS_REJECT_UNAUTHORIZED to false, you are opening up more security risks, and I would advise only doing this on private networks at best, and never in production environments. Without going down a security discussion rabbit hole, it's always best to use a cert signed by a CA. More info on the differences on certs can be found here. This can also cause problems if your application is robust enough to make multiple connections to various secured servers where only some use self signed certs, again mentioned in this link.
If the cert wasn't self-signed, you most likely shouldn't be getting this error, so another potential fix is to setup and use a trusted CA Certificate on the LDAP server instead.
On the other hand, if you are using a normal, non-secure ldap connection (not through TLS), and/or you get this error only occasionally while other times it goes through, you should try setting the ldap url to the LDAP server IP or load balancer IP (and use port 3268 to allow searching in all domains). In larger network setups this will avoid potential round robin dns queries that sometimes point you to a slow server or one you can't route to.

javascript and oracle database

Is there anyway so that i can connect to oracle database through javascript and access the database values

It depends on where the JavaScript is running.
In most cases, it runs in a web browser, and that host environment doesn't provide any way to connect to a remote database. Some browsers have built in SQL databases (which were in the HTML 5 specification, I'm not sure if they still are), I think they use SQLite (certainly not Oracle).
If you want to connect to Oracle from a web browser, then you will need some sort of intermediary. This usually means a web server that provides an HTTP based API and uses Perl, PHP, ASP.NET, etc, etc to talk to the database.
Other host environments may provide an Oracle API, but you would have to be more specific about which one you are using.

You could try to create an AJAX application building upon Oracle PL/SQL Gateway. See also http://www.stanford.edu/dept/itss/docs/oracle/10g/appdev.101/b10795/adfns_we.htm .
As it happens, I have long ago written about this: http://www.adp-gmbh.ch/blog/2006/11/30.php .

you can use server-side JScript, although this, i'm pretty sure, is only supported by iis. this is how you accomplish it anyways;
<%#Language="JScript"%>
<%
var sqlbeg = "SELECT * FROM sbc WHERE disp = 'F00' AND rec_uid IS NULL";
var aconn = Server.CreateObject("ADODB.Connection");
var recset = Server.CreateObject("ADODB.Recordset");
aconn.Open("DSN=ACUSQL_SBC;UID=sbc;PWD=sbc;");
recset.Open(sqlbeg, aconn, 3, 3);
%>

Server-side Javascript in production fails to open connection to a named instance of SQL2008

I've got a production site that has been working for years with a SQL Server 2000 default instance on server named MDWDATA. TCP port 1433 and Named Pipes are enabled there. My goal is to get this web app working with a copy of the database upgraded to SQL Server 2008. I've installed SQL2008 with SP1 on a server called DEVMOJITO and tested the new database using various VB6 desktop programs that exercise various stored procs in a client-server fashion and parts of the website itself work fine against the upgraded database residing on this named instance of SQL2008. So, while I am happy that the database upgrade seems fine there is a part of this website that fails with this Named Pipes Provider: Could not open a connection to SQL Server [1231]. I think this error is misleading. I disabled Named Pipes on the SQL2000 instance used by the production site, restarted SQL and all the ASP code still continued to work fine (plus we have a firewall between both database servers and these web virtual directories on a public facing webserver.
URL to my production virtual directory which demos the working page:
URL to my development v-directory which demos the failing page:
All the code is the same on both prod and dev sites except that on dev I'm trying to connect to the upgraded database.
I know there are dozens of things to check which I've been searching for but here are a few things I can offer to help you help me:
The code that is failing is server-side Javascript adapted from Brent Ashley's "Javascript Remote Scripting (JSRS)" code package years ago. It operates in an AJAX-like manner by posting requests back to different ASP pages and then handling a callback. I think the key thing to point out here is how I changed the connection to the database: (I cannot get Javascript to format right here!)
function setDBConnect(datasource)
{
var strConnect; //ADO connection string
//strConnect = "DRIVER=SQL Server;SERVER=MDWDATA;UID=uname;PASSWORD=x; DATABASE=StagingMDS;";
strConnect = "Provider=SQLNCLI10;Server=DEVMOJITO\MSSQLSERVER2008;Uid=uname;Pwd=x;DATABASE=StagingMDS;";
return strConnect;
}
function serializeSql( sql , datasource)
{
var conn = new ActiveXObject("ADODB.Connection");
var ConnectString = setDBConnect(datasource);
conn.Open( ConnectString );
var rs = conn.Execute( sql );
Please note how the connection string differs. I think that could be the problem but I don't know what to do. I am surprised the error returned says "named pipes" was involved because I really wanted to use TCP. The connection string syntax here is the same as used successfully on a different part of the site which uses VBScript which I'll paste here to show:
if DataBaseConnectionsAreNeeded(strScriptName) then
dim strWebDB
Set objConn = Server.CreateObject("ADODB.Connection")
if IsProductionWeb() Then
strWebDB = "DATABASE=MDS;SERVER=MDWDATA;DRIVER=SQL Server;UID=uname;PASSWORD=x;"
end if
if IsDevelopmentWeb() Then
strWebDB = "Provider=SQLNCLI10;Server=DEVMOJITO\MSSQLSERVER2008;Database=StagingMDS;UID=uname;PASSWORD=x;"
end if
objConn.ConnectionString = strWebDB
objConn.ConnectionTimeout = 30
objConn.Open
set oCmd = Server.CreateObject("ADODB.Command")
oCmd.ActiveConnection = objConn
This code works in both prod and dev virtual directories and other code in other parts of the web which use ASP.NET work against both databases correctly. Named pipes and TCP are both enabled on each server. I don't understand the string used by the Pipes but I am using the defaults always.
I wonder why the Javascript call above results in use of named pipes instead of TCP. Any ideas would be greatly appreciated.

Summary of what I did to get this working:
Add an extra slash to the connection string since this is server-side Javascript:
Server=tcp:DEVMOJITO\MSSQLSERVER2008,1219;
Explicitly code tcp: as a protocol prefix and port 1219. I learned that by default a named instance of SQL uses dynamic porting. I ended up turning that off and chose, somewhat arbitrarily, the port 1219, which dynamic had chosen before I turned it off. There are probably other ways to get this part working.
Finally, I discovered that SET NOCOUNT ON needed to be added to the stored procedure being called. Otherwise, the symptom is the message: "Operation is not allowed when the object is closed".

How to connect to SQL Server database from JavaScript in the browser?

Can anybody give me some sample source code showing how to connect to a SQL Server 2005 database from JavaScript locally? I am learning web programming on my desktop.
Or do I need to use any other scripting language? Suggest some alternatives if you have them, but I am now trying to do it with JavaScript. My SQL Server is locally installed on my desktop — SQL Server Management Studio 2005 and IE7 browser.

You shouldn´t use client javascript to access databases for several reasons (bad practice, security issues, etc) but if you really want to do this, here is an example:
var connection = new ActiveXObject("ADODB.Connection") ;
var connectionstring="Data Source=<server>;Initial Catalog=<catalog>;User ID=<user>;Password=<password>;Provider=SQLOLEDB";
connection.Open(connectionstring);
var rs = new ActiveXObject("ADODB.Recordset");
rs.Open("SELECT * FROM table", connection);
rs.MoveFirst
while(!rs.eof)
{
document.write(rs.fields(1));
rs.movenext;
}
rs.close;
connection.close;
A better way to connect to a sql server would be to use some server side language like PHP, Java, .NET, among others. Client javascript should be used only for the interfaces.
And there are rumors of an ancient legend about the existence of server javascript, but this is another story. ;)

This would be really bad to do because sharing your connection string opens up your website to so many vulnerabilities that you can't simply patch up, you have to use a different method if you want it to be secure. Otherwise you are opening up to a huge audience to take advantage of your site.

A perfect working code..
<script>
var objConnection = new ActiveXObject("adodb.connection");
var strConn = "driver={sql server};server=QITBLRQIPL030;database=adventureworks;uid=sa;password=12345";
objConnection.Open(strConn);
var rs = new ActiveXObject("ADODB.Recordset");
var strQuery = "SELECT * FROM Person.Address";
rs.Open(strQuery, objConnection);
rs.MoveFirst();
while (!rs.EOF) {
document.write(rs.fields(0) + " ");
document.write(rs.fields(1) + " ");
document.write(rs.fields(2) + " ");
document.write(rs.fields(3) + " ");
document.write(rs.fields(4) + "<br/>");
rs.movenext();
}
</script>

Web services
SQL 2005+ supports native WebServices that you could almost use although I wouldn't suggest it, because of security risks you may face. Why did I say almost. Well Javascript is not SOAP native, so it would be a bit more complicated to actually make it. You'd have to send and receive SOAP via XmlHttpRequest. Check google for Javascript SOAP clients.
http://msdn.microsoft.com/en-us/library/ms345123.aspx - SQL native WebServices
http://www.google.com/search?q=javascript+soap - Google results for Javascript SOAP clients

Playing with JavaScript in an HTA I had no luck with a driver={SQL Server};... connection string, but a named DSN was OK :
I set up TestDSN and it tested OK, and then var strConn= "DSN=TestDSN"; worked, so I carried on experimenting for my in-house testing and learning purposes.
Our server has several instances running, e.g. server1\dev and server1\Test which made things slightly more tricky as I managed to waste some time forgetting to escape the \ as \\ :)
After some dead-ends with server=server1;instanceName=dev in the connection strings, I eventually got this one to work :
var strConn= "Provider=SQLOLEDB;Data Source=server1\\dev;Trusted_Connection=Yes;Initial Catalog=MyDatabase;"
Using Windows credentials rather than supplying a user/pwd, I found an interesting diversion was discovering the subtleties of Integrated Security = true v Integrated Security = SSPI v Trusted_Connection=Yes - see Difference between Integrated Security = True and Integrated Security = SSPI
Beware that RecordCount will come back as -1 if using the default adOpenForwardOnly type. If you're working with small result sets and/or don't mind the whole lot in memory at once, use rs.Open(strQuery, objConnection, 3); (3=adOpenStatic) and this gives a valid rs.RecordCount

As stated before it shouldn't be done using client side Javascript but there's a framework for implementing what you want more securely.
Nodejs is a framework that allows you to code server connections in javascript so have a look into Nodejs and you'll probably learn a bit more about communicating with databases and grabbing data you need.

I dont think you can connect to SQL server from client side javascripts. You need to pick up some server side language to build web applications which can interact with your database and use javascript only to make your user interface better to interact with.
you can pick up any server side scripting language based on your language preference :
PHP
ASP.Net
Ruby On Rails

(sorry, this was a more generic answer about SQL backends--I hadn't read the answer about SQL Server 2005's WebServices feature. Although, this feature is still run over HTTP rather than more directly via sockets, so essentially they've built a mini web server into the database server, so this answer is still another route you could take.)
You can also connect directly using sockets (google "javascript sockets") and by directly at this point I mean using a Flash file for this purpose, although HTML5 has Web Sockets as part of the spec which I believe let you do the same thing.
Some people cite security issues, but if you designed your database permissions correctly you should theoretically be able to access the database from any front end, including OSQL, and not have a security breach. The security issue, then, would be if you weren't connecting via SSL.
Finally, though, I'm pretty sure this is all theoretical because I don't believe any JavaScript libraries exist for handling the communications protocols for SSL or SQL Server, so unless you're willing to figure these things out yourself it'd be better to go the route of having a web server and server-side scripting language in between the browser and the database.