Develop Reference

How to bring a gRPC defined API to the web browser

We want to build a Javascript/HTML gui for our gRPC-microservices. Since gRPC is not supported on the browser side, we thought of using web-sockets to connect to a node.js server, which calls the target service via grpc.
We struggle to find an elegant solution to do this. Especially, since we use gRPC streams to push events between our micro-services.
It seems that we need a second RPC system, just to communicate between the front end and the node.js server. This seems to be a lot of overhead and additional code that must be maintained.
Does anyone have experience doing something like this or has an idea how this could be solved?

Edit: Since Oct 23,2018 the gRPC-Web project is GA, which might be the most official/standardized way to solve your problem. (Even if it's already 2018 now... ;) )
From the GA-Blog: "gRPC-Web, just like gRPC, lets you define the service “contract” between client (web) and backend gRPC services using Protocol Buffers. The client can then be auto generated. [...]"
We recently built gRPC-Web (https://github.com/improbable-eng/grpc-web) - a browser client and server wrapper that follows the proposed gRPC-Web protocol. The example in that repo should provide a good starting point.
It requires either a standalone proxy or a wrapper for your gRPC server if you're using Golang. The proxy/wrapper modifies the response to package the trailers in the response body so that they can be read by the browser.
Disclosure: I'm a maintainer of the project.

Unfortunately, there isn't any good answer for you yet.
Supporting streaming RPCs from the browser fully requires HTTP2 trailers to be supported by the browsers, and at the time of the writing of this answer, they aren't.
See this issue for the discussion on the topic.
Otherwise, yes, you'd require a full translation system between WebSockets and gRPC. Maybe getting inspiration from grpc-gateway could be the start of such a project, but that's still a very long shot.

An official grpc-web (beta) implementation was released on 3/23/2018. You can find it at
https://github.com/grpc/grpc-web
The following instructions are taken from the README:
Define your gRPC service:
service EchoService {
rpc Echo(EchoRequest) returns (EchoResponse);
rpc ServerStreamingEcho(ServerStreamingEchoRequest)
returns (stream ServerStreamingEchoResponse);
}
Build the server in whatever language you want.
Create your JS client to make calls from the browser:
var echoService = new proto.grpc.gateway.testing.EchoServiceClient(
'http://localhost:8080');
Make a unary RPC call
var unaryRequest = new proto.grpc.gateway.testing.EchoRequest();
unaryRequest.setMessage(msg);
echoService.echo(unaryRequest, {},
function(err, response) {
console.log(response.getMessage());
});
Streams from the server to the browser are supported:
var stream = echoService.serverStreamingEcho(streamRequest, {});
stream.on('data', function(response) {
console.log(response.getMessage());
});
Bidirectional streams are NOT supported:
This is a work in progress and on the grpc-web roadmap. While there is an example protobuf showing bidi streaming, this comment make it clear that this example doesn't actually work yet.
Hopefully this will change soon. :)

https://github.com/tmc/grpc-websocket-proxy sounds like it may meet your needs. This translates json over web sockets to grpc (layer on top of grpc-gateway).

The grpc people at https://github.com/grpc/ are currently building a js implementation.
The repro is at https://github.com/grpc/grpc-web (gives 404 ->) which is currently (2016-12-20) in early access so you need to request access.

GRPC Bus WebSocket Proxy does exactly this by proxying all GRPC calls over a WebSocket connection to give you something that looks very similar to the Node GRPC API in the browser. Unlike GRPC-Gateway, it works with both streaming requests and streaming responses, as well as non-streaming calls.
There is both a server and client component.
The GRPC Bus WebSocket Proxy server can be run with Docker by doing docker run gabrielgrant/grpc-bus-websocket-proxy
On the browser side, you'll need to install the GRPC Bus WebSocket Proxy client with npm install grpc-bus-websocket-client
and then create a new GBC object with: new GBC(<grpc-bus-websocket-proxy address>, <protofile-url>, <service map>)
For example:
var GBC = require("grpc-bus-websocket-client");
new GBC("ws://localhost:8080/", 'helloworld.proto', {helloworld: {Greeter: 'localhost:50051'}})
.connect()
.then(function(gbc) {
gbc.services.helloworld.Greeter.sayHello({name: 'Gabriel'}, function(err, res){
console.log(res);
}); // --> Hello Gabriel
});
The client library expects to be able to download the .proto file with an AJAX request. The service-map provides the URLs of the different services defined in your proto file as seen by the proxy server.
For more details, see the GRPC Bus WebSocket Proxy client README

I see a lot of answers didn't point to a bidirectional solution over WebSocket, as the OP asked for browser support.
You may use JSON-RPC instead of gRPC, to get a bidirectional RPC over WebSocket, which supports a lot more, including WebRTC (browser to browser).
I guess it could be modified to support gRPC if you really need this type of serialization.
However, for browser tab to browser tab, request objects are not serializsed and are transfered natively, and the same with NodeJS cluster or thread workers, which offers a lot more performance.
Also, you can transfer "pointers" to SharedArrayBuffer, instead of serializing through the gRPC format.
JSON serialization and deserialization in V8 is also unbeatable.
https://github.com/bigstepinc/jsonrpc-bidirectional

Looking at the current solutions with gRPC over web, here is what's available out there at the time of writing this (and what I found):
gRPC-web: requires TypeScript for client
gRPC-web-proxy: requires Go
gRPC-gateway: requires .proto modification and decorations
gRPC-bus-websocket-proxy-server: as of writing this document it lacks tests and seems abandoned (edit: look at the comments by the original author!)
gRPC-dynamic-gateway: a bit of an overkill for simple gRPC services and authentication is awkward
gRPC-bus: requires something for the transport
I also want to shamelessly plug my own solution which I wrote for my company and it's being used in production to proxy requests to a gRPC service that only includes unary and server streaming calls:
gRPC-express
Every inch of the code is covered by tests. It's an Express middleware so it needs no additional modifications to your gRPC setup. You can also delegate HTTP authentication to Express (e.g with Passport).

how to invoke java adapter procedure from client side?

I have created java adapter in MobileFirst 7.0, My problem is how to invoke Java adapter from client side (js). I found Java adapter doesn't have procedures to call from client.
Thanks in Advance :)

How does your Java adapter look like? which environment are you testing this in?
Did you read the tutorial that explains how to call Java adapters in Hybrid applications?
See here: https://developer.ibm.com/mobilefirstplatform/documentation/getting-started-7-0/server-side-development/invoking-adapter-procedures-hybrid-client-applications/
In Java adapters, instead of "procedure name" you need to supply the #path you have set in your Java code.
WLResourceRequest
var resourceRequest = new WLResourceRequest(
"/adapters/RSSReader/getFeedsFiltered",
WLResourceRequest.GET
);
The WLResourceRequest class handles resource requests to MobileFirst
adapters or external resources.
The parameters for the constructor are:
request URL: To access an adapter within the same project, the URL
should be /adapters/AdapterName/procedureName.
To access resources outside of the project, use the full URL.
HTTP method: Most commonly WLResourceRequest.GET or
WLResourceRequest.POST timeout: optional, request timeout in
milliseconds

If you still have any more questions, check out this blogpost on getting started with java adapters.
https://developer.ibm.com/mobilefirstplatform/2015/03/24/getting-familiar-ibm-mobilefirst-platform-foundation-java-adapters/

Wireing Breeze (with Angular) to an existing WCF Data Service

The Short
I have an existing WCF Data Service that I would like to wire up to use in an AngularJS SPA using Breeze.
Can anyone show a noobie level example of how to do that with out access to the actual database (just the OData Service)?
The Long
I have an existing WCF Data Service that is already in use by a WPF app.
I experimenting with web development and would like to wire up to that service using Breeze. In case it matters, I am using Angular (and am setting up via the HotTowel.Angular nuget package).
I have done a fair amount of Googling and I am stuck.
I can see two ways outlined from my searching:
The First
Make is to make a Breeze controller on the server side of my web app.
The problem I see with that is the metadata. From my limited understanding I need to tell breeze all the meta data of my WCF Data Service. I know how to get the meta from my WCF Data Service (the url + $Metadata), but I don't know how to tell this to Breeze.
The Second
This way is more vague in implementation. I got it from the accepted answer on this question: Breeze.js with WCF Data Service.
Basically the answer here does not seem to work. It relies on something called the entityModel that I cannot seem to find (I have an entityManager, but not an entityModel. And the entityManager does not have the properties that the entityModel is shown to have.
In the end I like the idea of the second method best. That way I can directly connect to my odata service with out needed my web app to have a "in-between" server component. But I would happily take anything that does not need entity framework to connect to my database.
I tried several variations of the second option, but I just can't seem to get it to work. I also tried the Breeze samples. It has one for OData, but it still relies on having Entity Framework hook up to the source database.
To to clearly summarize what I am asking: I am looking for a Breeze example that connects to an existing WCF Data Service.

We regret that you were mislead by that old StackOverflow answer which was way out of date and (therefore) incorrect. There is no longer a type called entityModel.
I updated the answer there and repeat here the same advice.
The recommended way to configure Breeze so that it talks to a standard OData source (such as a WCF OData service) is
breeze.config.initializeAdapterInstance('dataService', 'OData', true);
Here's how you might proceed with defining an EntityManager and querying the service:
// specify the absolute URL to the WCF service address
var serviceName = "http://localhost:9009/ODataService.svc";
var em = new breeze.EntityManager(serviceName);
var query = breeze.EntityQuery.from("Customers")
.where("CompanyName", "startsWith", "B")
.orderBy("City");
em.executeQuery(query).then(function(data) {
// process the data.results here.
});
There is some documentation on this subject here.
A Web API OData service differs from a WCF OData service in several respects. But you may still find value in the Angular Web API OData sample.

What is the best/proper configuration? (javascript SOAP)

I need to retrieve data from a web service (via SOAP) during a nightly maintenance process on a LAMP server. This data then gets applied to a database. My research has returned many options and I think I have lost sight of the forest for the trees; partially because of the mix of client and server terms and perspectives of the articles I have read.
Initially I installed node.js and node-soap. I wrote a simple script to test functionality:
var soap = require('/usr/local/lib/node_modules/npm/node_modules/soap');
var url = "https://api.authorize.net/soap/v1/Service.asmx?WSDL";
soap.createClient(url, function(err, client)
{
if(typeof client == 'undefined')
{
console.log(err);
return;
}
console.log('created');
});
This uses a demo SOAP source and it works just fine. But when I use the actual URL I get a 5023 error:
[Error: Invalid WSDL URL: https://*****.*****.com:999/SeniorSystemsWS/DataExportService.asmx?WSDL
Code: 503
Response Body: <html><body><b>Http/1.1 Service Unavailable</b></body> </html>]
Accessing this URL from a browser returns a proper WSDL definition. I am told by the provider that the 503 is due to a same-origin policy violation. Next, I researched adding CORS to node.js. This triggered my stepping back and asking the question: Am I in the right forest? I'm not sure. So, I am looking for a command-line, SOAP capable, CORS app (or equivalent) configuration. I am a web developer primarily using PHP and Javascript, so Javascript is where I turned first, but that is not a requirement. Ideas? Or, is there a solution to the current script error (the best I think I have found is using jQuery in node.js which includes CORS)

Most likely, this error belongs to your website server.
Please go through this link, it might be helpful.
http://pcsupport.about.com/od/findbyerrormessage/a/503error.htm
Also you can open your wsdl in web browser, search for soap:address location tag under services. And figure out correct url, you are trying to invoke from your script. Directly access this url in browser and see what are you getting.

I think I have a better approach to the task. I found over the weekend that PHP has a full SOAP client. I wrote the same basic login script in PHP and it runs just fine. I get a valid authentication code in the response to loginExt (which is required in further requests), so it looks like things are working. I will comment here after verifying that I can actually use the web service.

Publish data from browser app without writing my own server

I need users to be able to post data from a single page browser application (SPA) to me, but I can't put server-side code on the host.
Is there a web service that I can use for this? I looked at Amazon SQS (simple queue service) but I can't call their REST APIs from within the browser due to cross origin policy.
I favour ease of development over robustness right now, so even just receiving an email would be fine. I'm not sure that the site is even going to catch on. If it does, then I'll develop a server-side component and move hosts.

Not only there are Web Services, but nowadays there are robust systems that provide a way to server-side some logic on your applications. They are called BaaS or Backend as a Service providers, usually to provide some backbone to your front end applications.
Although they have multiple uses, I'm going to list the most common in my opinion:
For mobile applications - Instead of having to learn an API for each device you code to, you can use an standard platform to store logic and data for your application.
For prototyping - If you want to create a slick application, but you don't want to code all the backend logic for the data -less dealing with all the operations and system administration that represents-, through a BaaS provider you only need good Front End skills to code the simplest CRUD applications you can imagine. Some BaaS even allow you to bind some Reduce algorithms to calls your perform to their API.
For web applications - When PaaS (Platform as a Service) came to town to ease the job for Backend End developers in order to avoid the hassle of System Administration and Operations, it was just logic that the same was going to happen to the Backend. There are many clones that showcase the real power of this strategy.
All of this is amazing, but I have yet to mention any of them. I'm going to list the ones that I know the most and have actually used in projects. There are probably many, but as far as I know, this one have satisfied most of my news, whether it's any of the previously ones mentioned.
Parse.com
Parse's most outstanding features target mobile devices; however, nowadays Parse contains an incredible amount of API's that allows you to use it as full feature backend service for Javascript, Android and even Windows 8 applications (Windows 8 SDK was introduced a few months ago this year).
How does a Parse code looks in Javascript?
Parse works through classes and objects (ain't that beautiful?), so you first create a specific class (can be done through Javascript, REST or even the Data Browser manager) and then you add objects to specific classes.
First, add up Parse as a script tag in javascript:
<script type="text/javascript" src="http://www.parsecdn.com/js/parse-1.1.15.min.js"></script>
Then, through a given Application ID and a Javascript Key, initialize Parse.
Parse.initialize("APPLICATION_ID", "JAVASCRIPT_KEY");
From there, it's all object manipulation
var Person = Parse.Object.extend("Person"); //Person is a class *cof* uppercase *cof*
var personObject = new Person();
personObject.save({name: "John"}, {
success: function(object) {
console.log("The object with the data "+ JSON.stringify(object) + " was saved successfully.");
},
error: function(model, error) {
console.log("There was an error! The following model and error object were provided by the Server");
console.log(model);
console.log(error);
}
});
What about authentication and security?
Parse has a User based authentication system, which pretty much allows you to store a base of users that can manipulate the data. If map the data with User information, you can ensure that only a given user can manipulate specific data. Plus, in the settings of your Parse application, you can specify that no clients are allowed to create classes, to ensure innecesary calls are performed.
Did you REALLY used in a web application?
Yes, it was my tool of choice for a medium fidelity prototype.
Firebase.com
Firebase's main feature is the ability to provide Real Time to your application without all the hassle. You don't need a MeteorJS server in order to bring Push Notifications to your software. If you know Javascript, you are half way through to bring Real Time magic to your users.
How does a Firebase looks in Javascript?
Firebase works in a REST fashion, and I think they do an amazing job structuring the Glory of REST. As a good example, look at the following Resource structure in Firebase:
https://SampleChat.firebaseIO-demo.com/users/fred/name/first
You don't need to be a rocket scientist to know that you are retrieve the first name of the user "Fred", giving there's at least one -usually there should be a UUID instead of a name, but hey, it's an example, give me a break-.
In order to start using Firebase, as with Parse, add up their CDN Javascript
<script type='text/javascript' src='https://cdn.firebase.com/v0/firebase.js'></script>
Now, create a reference object that will allow you to consume the Firebase API
var myRootRef = new Firebase('https://myprojectname.firebaseIO-demo.com/');
From there, you can create a bunch of neat applications.
var USERS_LOCATION = 'https://SampleChat.firebaseIO-demo.com/users';
var userId = "Fred"; // Username
var usersRef = new Firebase(USERS_LOCATION);
usersRef.child(userId).once('value', function(snapshot) {
var exists = (snapshot.val() !== null);
if (exists) {
console.log("Username "+userId+" is part of our database");
} else {
console.log("We have no register of the username "+userId);
}
});
What about authentication and security?
You are in luck! Firebase released their Security API about two weeks ago! I have yet to explore it, but I'm sure it fills most of the gaps that allowed random people to use your reference to their own purpose.
Did you REALLY used in a web application?
Eeehm... ok, no. I used it in a Chrome Extension! It's still in process but it's going to be a Real Time chat inside a Chrome Extension. Ain't that cool? Fine. I find it cool. Anyway, you can browse more awesome examples for Firebase in their examples page.
What's the magic of these services? If you read your Dependency Injection and Mock Object Testing, at some point you can completely replace all of those services for your own through a REST Web Service provider.
Since these services were created to be used inside any application, they are CORS ready. As stated before, I have successfully used both of them from multiple domains without any issue (I'm even trying to use Firebase in a Chrome Extension, and I'm sure I will succeed soon).
Both Parse and Firebase have Data Browser managers, which means that you can see the data you are manipulating through a simple web browser. As a final disclaimer, I have no relationship with any of those services other than the face that James Taplin (Firebase Co-founder) was amazing enough to lend me some Beta access to Firebase.

You actually CAN use SQS from the browser, even without CORS, as long as you only need the browser to send messages, not receive them. Warning: this is a kludge that would make my CS professors cry.
When you perform a GET request via javascript, the browser will always perform the request, however, you'll only get access to the response if it was from the same origin (protocol, host, port). This is your ticket to ride, since messages can be posted to an SQS queue with just a GET, and who really cares about the response anyways?
Assuming you're using jquery, your queue is https://sqs.us-east-1.amazonaws.com/71717171/myqueue, and allows anyone to post a message, the following will post a message with the body "HITHERE" to the queue:
$.ajax({
url: 'https://sqs.us-east-1.amazonaws.com/71717171/myqueue' +
'?Action=SendMessage' +
'&Version=2012-11-05' +
'&MessageBody=HITHERE'
})
The'll be an error in the console saying that the request failed, but the message will show up in the queue anyways.

Have you considered JSONP? That is one way of calling cross-domain scripts from javascript without running into the same origin policy. You're going to have to set up some script somewhere to send you the data, though. Javascript just isn't up to the task.

Depending in what kind of data you want to send, and what you're going to do with it, one way of solving it would be to post the data to a Google Spreadsheet using Ajax. It's a bit tricky to accomplish though.Here is another stackoverflow question about it.
If presentation isn't that important you can just have an embedded Google Spreadsheet Form.

What about mailto:youremail#goeshere.com ? ihihi
Meantime, you can turn on some free hostings like Altervista or Heroku or somenthing else like them .. so you can connect to their server , if i remember these free services allows servers p2p, so you can create a sort of personal web services and push ajax requests as well, obviously their servers are slow for free accounts, but i think it's enought if you do not have so much users traffic, else you should turn on some better VPS or Hosting or Cloud solution.

Maybe CouchDB can provide what you're after. IrisCouch provides free CouchDB instances. Lock it down so that users can't view documents and have a sensible validation function and you've got yourself an easy RESTful place to stick your data in.