I would like to make some calls to a url from a mapreduce in mongo, but I don't find anyway to do that in javascript except jQuery or lower layers that seem to be accessible only from web browsers
If it may help drive the discussion, I want to call localhost, a c++ server can do complicated things that I can't do in javascript (big libraries)
While MapReduce is written in the JavaScript programming language in MongoDB currently, there is not a general set of APIs that allow access to a richer set of platform capabilities.
By design, you cannot access the network (or localhost), files, process, etc. stacks of the host operating system. It's intentionally limited to just the features of MongoDB, and even that is restricted so that the MapReduce will perform optimally.
You will need to write your queries in a different platform and merge the results, or consider other database options.
No. It's a database, not a programming language.
What you can do, however, is to dynamically create your query (from c++ or another language) which would call the server or put the required data dynamically inside.
Related
Scenario:
I am currently working on a desktop-based application involving multiple independent production sites, each having its own server, database and local clients.
The software undergoes continuous automated updates, as part of these updates each production site's database (postgresql) will need to undergo schema migrations (while maintaining persistence/transformation on the existing data during the migration).
Accessing the client machines manually (e.g. SSH / Remote Desktop) to run scripts/perform the update is not an option. The client would need to be self-sufficient in checking, downloading and installing the updates.
Problem:
I have been looking around for a few months and have not yet suitable option apart from writing SQL scripts and using a separate service to run it one by one or using the updated app to run it.
Question:
What tools would you suggest would be suitable for this scenario to ease the migration process?
It is a generally an accepted practice to run SQL scripts when doing database migrations. Tools like ActiveRecord (in the ruby world) provide an intermediate layer (a DSL), but are essentially wrappers around SQL.
There is a good reason for that - a migration is what is being done on the SQL server and the only strictly defined interface for manipulating tables in SQL servers is the PL/SQL language language flavor that the particular RDBMS server is using.
This means, that if SQL scripts are an acceptable option for migrations (acceptable in the sense that the team knows it, nobody objects to this, etc.) you should use them.
To make the task easier (the way ruby does it) you could take a look at: http://migrate4j.sourceforge.net/
They describe the tool as The initial intent of migrate4j was to make a Java version of Ruby's db:migrate. But this is not a fundamental requirement.
I have questions regarding stored JavaScript Procedures. After reading the Blog Entry from PointBeing, I have some questions.
Is there an advantage to storing my code in the DB? I mean functions like lookups for documents, not adding numbers like the example from PointBeing.
Is MongoDB stored javascript faster than node.js javascript?
Is MongoDB stored javascript queries cached and are they any faster?
I'm interested in MongoDB stored javascript performance compared to Node.js Javascript.
Evaluating functions stored in db.system.js ("Stored procedures", when you would like to call them that) is deprecated. The articles on the db.eval shell function and the eval database command have a "Deprecated since version 3.0" warning and the article on server-sided javascript doesn't mention it anymore. So you should avoid using it. One reason is that you can not run a javascript function when you use sharding. So when you build an application which requires eval, you prevent it from scaling in the future. Another is that javascript functions undermine the permission concept. They always need to be run as admin, which makes it impossible to establish a sane permission system. This is especially problematic from a security standpoint considering that server-sided scripts which use user-provided data can potentially be vulnerable to arbitrary script injections.
The advantage of server-sided javascript is that it runs on the database server. This reduces latency between application server and database server when you need to perform a large number of queries. But you can get the same advantage by opening a mongo shell on the database server and executing it there.
The latency advantage is only relevant when you perform multiple queries from your script. When you have only one query, you will still have the latency when invoking the script. So you gain nothing except unnecessary complexity.
There is no additional caching or other optimization for server-sided javascript. Even worse: It will get reparsed and reinterpreted everytime you run it. So it might even be slower than javascript in your application server.
Further, many complex queries which would require script support to implement only with find() can often be expressed using aggregation which will in most cases be far faster than doing the same with find() and javascript because the aggregation framework is implemented in C++ and has access to the raw BSON documents.
The hilarious thing is that blog post ( http://pointbeing.net/weblog/2010/08/getting-started-with-stored-procedures-in-mongodb.html ) was written when JS only took single threaded global lock.
That means there was no con-currency features or more granular lock associated with it (the lock still being a problem and con-currency is only achieved through multiple isolates still). Just because you see it in some random blog post does not mean it should be used.
To answer your questions directly:
Nope. In fact the disadvantage is that the calling user needs full admin rights. This means you give every single privilege to your web user since the inbuilt JS enigne has hooks for everything, including administration functions as such it requires admin rights in order to run.
Calling JS from JS to JS to C++ in JS? No
No, MongoDB caching does not work like that. I recommend you read the fundamentals documentation: http://docs.mongodb.org/manual/faq/fundamentals/
I'm trying to understand API's.
Question1:
I get it that it helps two applications to talk with each other. So if I were to use a third party service in my application, I can do that by using their API. Understood. But why would we need API within a same company ? If suppose XYZ company has a java/ js application and they get the data using the companies internal API. What's the point of having an API when the resource is within the same company.
Also, should this be a REST or SOAP?
Question2:
Suppose I have users using my API "x1" which has underlying database "d1" which has a schema "s1". Now, I have completely changed the underlying database & schema to "d2" and "s2" and have a new API "x2" . How can the users still continue to use my API without knowing this transition? How can the transition be done transparently without the users knowing this.
Question3:
Also what does it exactly mean by "rest can be cached? " . Any example ?
There are just two types of API's ? REST & SOAP right?
I was asked these questions by someone & I'm totally got messed up. Can someone explain and clear my confusion?
At its simplest, an API is a set of reusable functions for accomplishing a particular task. So if you write a C program using the standard fopen function, you are using the POSIX API, which defines how that function should behave. Crucially, your program doesn't need to know how the function is implemented, because the API is an abstraction for doing the same task on different systems. The particular implementation will be provided by a library on each target system.
This kind of direct API only works if you are using the same computer, and your program and library are written in languages that can be easily bound together. In some cases, you need to bind more varied resources together, so need a different kind of API. For instance, you might write a script-friendly command-line interface to your library (a text-based API), or use signals and pipes to communicate between two running processes (called IPC, for Inter-Process Communication).
This brings us to networked APIs, and in particular Web APIs, which were the main subject of your question. These are just the same as the above, but they use HTTP and related technologies to let processes talk to each other even if they're not on the same host. As you say, this allows an organisation to have a public API, but it also allows a system to be scaled onto multiple servers, using multiple programming languages and technologies. The notion of abstraction means that a section of the system provided as an API service can theoretically be competely replaced by a new implementation without any changes to the programs consuming it. If a redesign is needed, it may be possible to provide a wrapper which looks like the old API, and calls into the new one, translating data as required.
There are many ways to build a Web API, but they broadly fall into two categories:
RPC (Remote Procedure Call) APIs are based around actions, or verbs, with defined parameters, effects, and return values; it as though you are calling a function or procedure on a remote host. SOAP is a particular standard for RPC-style APIs, which defines an "envelope" using XML; it doesn't define much within the envelope, though.
Resource-oriented APIs focus on nouns rather than verbs, and don't aim to resemble a classic function call. You could consider an RSS feed a simple API of this kind - accessing the URL for the feed causes the server to generate the resource, and present it in a well-defined format for some other program to consume. REST is a formalisation of this idea, using URLs to represent objects, and HTTP requests to represent actions on those objects - GET to retrieve, POST to create, PUT to update, etc. By using HTTP, the idea is to make use of existing techniques used for human-readable resources, such as cached responses, without the user needing to know how the API is written.
Since these APIs are supposed to be language neutral, it is necessary to define how the data is represented; the most common formats are XML and JSON, but there can be others, such as Google's Protocol Buffers.
I'm developing a jQuery-based pivot table.
The goal is not a web app but a desktop-based application (C++). In my idea, data is retrieved by the application from the database, then passed to a html page and then showed through the pivot-table plugin.
There is no web server and the web page containing the plugin cannot access to the database.
So, how to pass data in an efficient way? I've seen other questions here on SO around this matter, but I think we're in a different scenario. Of course I can write the data in a txt/xml/js file, but I've experienced that for huge amounts of data, writing down files is costing a lot.
If you don't want to add support for the http protocol to your application, and you don't want to write to additional files, then IMO your best bet is to create a wrapping http server for your application. Then the javascript page can access the running wrapper which can talk to your "real" application.
You could create such a server relatively easily in python using the twisted framework, ruby using rails and the bundled webrick server, or the v8 Javascript engine node.js. (I'm sure there's dozens of other options out there too)
Which of there would be best for you will depend a lot on which languages you have experience with and what your deployment requirements are (supported OSes, existing installed applications, installation size, license terms on your software etc.)
Do you have a specific framework for your desktop app ? If you use Qt for the GUI, you can also integrate javascript quite easily.
http://efforts.embedded.ufcg.edu.br/qt/?p=84
You might be able to use named pipes to pass data to a static page.
It might also be better to just make your c++ program into a simple web server which opens a port, and have it generate the web page when the user goes to http://localhost:8080.
I looking at using Javascript server side and took a look at persevere/pintura but for a novice like me, I need more documentation and tutorials and CouchDB seems to have a lot of that but is it used as a server side js tool?
Kind Regards
Yes (seconding what #Pointy said above). The JavaScript in CouchDB does run server-side. :)
Beyond that, though, JavaScript in CouchDB can do far more than just map/reduce queries. It's also used for document validation, update handling (for processing XML POSTed to CouchDB for example), as well as HTML or other markup output from either a single JSON document (using _show) or from the results of a map/reduce query (using _list).
CouchDB does not, however, offer filesystem access or TCP/IP listeners as Node.js does. Right now, the JavaScript is focused on the "shipping and handling" for the database contents (things like validation, formatting, and querying).
If you're interested in looking into options for handling things that CouchDB doesn't do "internally" (image processing, sending e-mails, etc), you might checkout my reply about CouchApp architecture options.
CouchApp's are definitely a viable option for web apps. Coupling those with replication, it's hard to beat.
http://nodejs.org/ is a great example for server-side javascript. If you don't want to write your complete webapp on NodeJS you can trigger it via the node command.
CouchDB is a database with an interresting feature, it allows you to interact via a RestFull api making it ideally suited to access over the web, either in javascript or other client environments. If you are looking for a server-side javascript development environment, try Servoy. It allows full featured server-side application development in javascript including easy database access to all major databases.
Short Answer: Yes.
Longer Answer:
There is a Javascript Engine shipped with CouchDB, which is SpiderMonkey
(the very first engine, open source, written by Brendan Eich, creator of javascript)
CouchDB's View function is JavaScript function running server side that acts as the Map half of a map/reduce operation to construct view.
Then there is client side javascript API, which can be used to communicate with the REST layer to fetch/update documents, which is also where jQuery is used by CouchDB.
More on CouchDB's both server-side & client side javascript capability:
The default engine is capable of doing many features, while package availability is limited compared to Node.js. Kan.so can be considered "NPM for CouchDB, where some packages are server side.
For example, there are frameworks like duality that try to make use of both server side & client side javascript capability to reuse code for rendering with couchDB.
More on the engine:
(The selection of spider Monkey over v8 is because for couchDB's use case the map reduce is not v8 designed for. Also include packaging, multi-threading and also historical reason v8 did not exist that time when spiderMoney is chosen) see [the mailing list] and tweet2
There is even project to port the whole CouchDB to Node.js (i.e. instead of Erlang), Chesterfield
I am glad if someone can add how Erlang talk with the javscript engine.
Thanks for everyone's answer, this is an important thing to make clear for people like me messing up couchDB with some Node.js packages
No, CouchDB is a document-oriented database that can be queried and indexed in a MapReduce fashion using JavaScript. CouchDB also offers incremental replication with bi-directional conflict detection and resolution.
http://couchdb.apache.org/
CommonJS can be used for server side JavaScript, see here: http://www.commonjs.org/
NodeJS is another: http://nodejs.org/