Develop Reference

Any good javascript library to allow crawling of website [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I need a javascript library to crawl a web application. I found this https://github.com/riccardo-forina/status-jquery-crawler but as the author claims , this is in early stage of development. I could not find anything after a lot of googling
Thanks for any inputs

Javascript has many utilities you can use.
The biggest question when choosing your tool is, "does my site use Javascript to load the content I want?". For example, Google's search page is almost all contained in the HTML they send in response to an HTTP GET request.
Other sites may load comments, notifications, or pictures that aren't contained in the HTML initially using Javascript. This means that if you just said, give me the HTML for Site A, the page you'd get back wouldn't be missing much of the content you wanted.
Static Sites
For most sites where what you want is in the HTML, there are some excellent node.js scraping libraries at your disposal:
x-ray - a neat package that bundles up cheerio inside a declarative scrape object. Provides some simple structure with which to build robust scrapes.
cheerio + request - this is a popular combination, using cheerio to parse the HTML and request to get it for you. You'll find lots of resources explaining the basics of requesting web-pages, extracting the HTML, and even adding authentication and maintaining sessions where required using these tools.
artoo.js - in browser scraping utility. Extremely useful for prototyping, and one-off scrapes. You can add it as a bookmarklet and run it in your browser developer's console. It allows jQuery like selectors and has some basic following logic.
Dynamic Sites
If you need a browser like environment to get content from your site, you'll want to check out headless web browsing and drivers in node.js. PhantomJS is the most popular, but there are many others. Be warned - to use PhantomJS with other Javascript libraries you'll need to find a node.js driver:
Nightmare - a node library that talks to PhantomJS and simplifies basic web-page workflow and scraping.
SpookyJS - a node library for CasperJS, a tool built on top of PhantomJS that is also a separate package.
PhantomJS-Node - the most popular PhantomJS driver for node.
(Sorry for the lack of links - I don't have enough reputation to post more than 2 right now)

PhantomJs is one of the Javascript based headless webkit, so you could use it for crawling. There is something new wrapper came up on top PhantomJS called Nightmare Js http://www.nightmarejs.org/.

Where do I start for writing my own calculator program [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I have been learning programing, mainly JavaScript and C++, but all the programs I have been writing are simple and in note pad like applications.
What would I need to make a simple calculator that takes user input and dose calculations with it.
What would I use to get the GUI on the screen, how would I compile it, and what IDE should I use.

It depends how complicated you're going, it'd be easier to recommend if you had a specific project in mind. You could make a simple 'GUI' that just opens in a CMD/Terminal window with C++ and easily allow input (cin), changing what's displayed as you desire. You could easily do your example this way, without the need of anything overly complex.
In terms of using Notepad you probably want something with at least some sort of formatting to make it a bit easier on yourself (such as Notepad++) although you could just open Notepad and start coding. There are IDEs such as Eclipse with the CDT plugin which are good for starting C++, or Visual Studio. I'm not big on Javascript myself, but I've heard Komodo is good for web coding.
In terms of complex GUIs, as has been said in the comments there are many and it basically comes down to your preference as to which you use.

If it is basic to advance experience you seek then try Windows Forms application. Make a calculator or something. Sky (and memory) is the limit.

The first thing to write an application in my eyes is to need that application in the first place. So try to think of some tool, you or someone else might need and then re-think your process.
With a clear aim you can start off and think of everything you need and - even more important - we can help you in this community, because you can then ask more specific questions regarding the actual development and all the problems that occur alongside it.

Develop deep understanding of c++ concepts. Study more and good books including it's standard. Start with normal applications & go towards advanced programming. for GUI, try QT - it's good of go for MFC Application. Write Code yourself and work on different projects, alogirithms. For book suggestion, i suggest you getting Scott Mayers Effective c++
For Compiler, i suggest using Visual Studio on widnows - it's awesome IDE.

JavaScript library development [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am a java-programmer and a few months ago I discover frontend programming with javascript and html, and it is absolutely cool. So here is a js newbie's question.
Is there any IDE, default workflow or maybe best practices for javascript libs development? I didn't mean coverage and unit testing or something like this. I mean is there any tools or techniques which should I use while developing html control with some js-logic, for example? Should I write small test page in text editor and open it in dozen of browsers pressing F5 each minute and watching at browser's console or maybe there is some magic IDE which will on button press reload all browser instances and collect reports from browsers?

JavaScript Fundamentals
Be sure that you know the language. JavaScript is a bit tricky in this regard: you can easily think that you understand the language, but there are weird things that can sneak up on you. I highly recommend JavaScript: The Good Parts by Douglas Crockford. He also has some talks on Youtube with the same name that are definitively worth watching.
JSLint
Integrate JSLint or a similar tool in your workflow. It is a style checker and static analysis tool for Javascript and helps catching subtle bugs.
Avoiding F5
Check out live.js:
Just include Live.js and it will monitor the current page including
local CSS and Javascript by sending consecutive HEAD requests to the
server. Changes to CSS will be applied dynamically and HTML or
Javascript changes will reload the page. Try it!
Browser Developer Tools
Get familiar with them. I personally strongly prefer Chrome's developer tools over Firefox/Firebug, but regardless which one you choose, learn how to use the debugger.
Node.js
You should also be aware that you don't have to test Javascript logic in your browser: you can use it as any other scripting language using node.js.

As a seasoned programmer here are the steps you should follow for JS lib production.
Separate the UI from the application logic. In this case create one component for the application logic that is completely separated from all API access. API access, which means DOM and WSH and Node.js, should be separated out. I go so far as to use different files to force and ensure separation.
Create UI environments to access your logic. Have a production UI control for access by your audience and also a separate internal UI control for your sandboxed development. For example I have written my application at http://prettydiff.com/ to work from commandline, and from a browser. I have also written access methods to the application access similar to the published HTML but different for my own development for more rapid unit testing. In a sandboxed UI can use a recursive setTimeout to refresh the page in intervals for automated test plan verification as you are writing and saving code.
Focus on availability and distribution. People can get my libraries directly from my website, Github, and NPM for Node. I have a regimented process where by I upload code to the site during a production release and then perform a quick verification in the browser. If the release did not break the application then push to Github and then push this exact same location to NPM.
Distributed access to code operation is even better. Its nice having rapid and even automated access to libraries online, but being able to access unit testing via request is even better. I can remotely unit test my application's code by accessing code samples online directly by telling my application request a code sample via URI. This means that not only is my static code available for distribution and testing, but so is its operation.
Good documentation is everything. I will never even bother to examine a lib if the documentation is weak. I am not talking about code comments, though these are important. I am talking about end user documentation. I want to be able to read documentation and know everything about the subject. Node.js became popular because even before the codebase was stable it's documentation kicked ass. Get other people to QA your documentation before they QA your code. Without stunning documentation your lib is less than worthless to me.
Know your mission. Each and every lib should have a clear, simple, and well stated purpose. If this is not established then the lib is not ready for release. If an enhancement confuses the lib's purpose then it may be time to divide one library into multiple smaller libraries. Focus on precision, clarity, directness, and only the sole stated mission of the lib.
Independence is vital to adoption. I don't like libs dependent upon other libraries or frameworks. Its great that your jQuery library may be the best thing since slice bread, but I won't be looking at it. Independence means greater portability and freedom it mix and match it with other libraries that you are not aware of.
Style is important. This is touchy subject, but style is important. Keep the logic in your lib as simple and declarative as possible. If your code is absolutely declarative in nature then your algorithmic patterns are awesome. Avoid use of the new keyword unless you are an experienced JavaScript badass and severely limit use of this keyword as it will fail you in future maintenance operations. Do not algorithmically build out large strings with concatenation and continually watch the execution speeds of your code. Because even trivial changes to style or seemingly minor enhancements to the logic can destroy execution efficiency I put a timer on all my UI controls. In your development use profilers, such as Chrome's web tools to track down long operations in JS execution.
Be open about your failures. Software is never perfect and other developers will always respect this. If you encounter a bug before anybody else then be open about the existence of the bug. If it will take more than a week to get the solution out then don't delay notification. Notify your users immediately so that they are aware. I recently rolled back a major enhancement to the logic of my diff algorithm because additional unit testing showed heavy slippage. I rolled back the same day I made a decision the prior release or two was flawed and was open about the rollback. If you want people to contribute to your code base or provide bug reports then openness is critical.

I know you may have explicitly said that you didn't mean unit testing, but that is precisely the way I recommend you write javascript libraries.
If you're a Java developer, you may be familiar with jUnit. If so, qUnit may be more natural to you. Otherwise, I recommend you take a look at Jasmine or Mocha. While I prefer Mocha, my experience with Jasmine has generally been better for in-browser development thanks to the awesome Jasmine-Jquery plugin.
If you're writing qUnit tests, take a look at IntelliJ IDE which allows you to execute tests in addition to providing code coverage.
If you're developing on the browser, take a look at LiveReload. It'll watch files for you and auto-refresh your browser - great for instant feedback.
For browser compatibility, I would recommend you just get it to work on one first reasonably before worrying about the others. Check in from time to time to see if you find issues. See if jQuery can abstract some of that mess for you. Otherwise, take a look at Adobes BrowserLab.

Displaying code in blog posts [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
What libraries and/or packages have you used to create blog posts with code blocks? Having a JavaScript library that would support line numbers and indentation is ideal.

The GeSHi text highlighter is pretty awesome. If you're using WordPress, there's a plugin for you already

A simple Google query reveals http://code.google.com/p/syntaxhighlighter/
From initial looks it seems pretty good. Entirly JS based so can be implemented independent of the server side language used.

Syntax Highlighter is used by wordpress and produces nice results.

Copy Visual studio code as HTML http://www.jtleigh.com/people/colin/software/CopySourceAsHtml/

I use Live Writer and I use VS addin that copies source code as html to copy the code and then I change into HTML view in Writer and paste the result
you can download the addin at:http://blogs.microsoft.co.il/blogs/bursteg/archive/2007/11/21/copy-source-as-html-copysourceashtml-for-visual-studio-2008-rtm.aspx

Some time ago I've done some research on this topic and came to the conclusion that using GeSHi is the way to go. However recently I've been looking to some more alternatives:
Using Windows Live Writer with a syntax highlighter plugin (there are several available)
Using the syntaxhighlighter library or the google code prettify library. Both are written in JS and I think the second one is used on stackoverflow
Use some intermediate process, where I write the posts in Markdown for example and let a program generate the final HTML

Personally, I use this website to do it for me: http://puzzleware.net/codehtmler/default.aspx

If that's my own code, I would just use SciTE's export to HTML and paste it.
Otherwise (highlighting code like it is done here), I would prefer to do it on server side: JS highlighting (as seen, for example, on JavaLobby) happens after the page have been displayed in default mode (so there is a sudden change of look, not very nice), and is often slow, plus JS can be disabled.
Actually, such task can be done once, after user input, it doesn't need to be done over and over on each page served to visitor.

I usually use this free online tool that formats C# code. Along with C#, it also formats code for VB, HTML, XML, T-SQL and MSH (code name Monad).

JavaScript editor within Eclipse [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I'm looking for the best JavaScript editor available as an Eclipse plugin. I've been using Spket which is good. But, is there more better one?

Eclipse HTML Editor Plugin
I too have struggled with this totally obvious question. It seemed crazy that this wasn't an extremely easy-to-find feature with all the web development happening in Eclipse these days.
I was very turned off by Aptana because of how bloated it is, and the fact that it starts up a local web server (by default on port 8000) everytime you start Eclipse and you can't disable this functionality. Adobe's port of JSEclipse is now a 400Mb plugin, which is equally insane.
However, I just found a super-lightweight JavaScript editor called Eclipse HTML Editor Plugin, made by Amateras, which was exactly what I was looking for.

Disclaimer, I work at Aptana. I would point out there are some nice features for JS that you might not get so easily elsewhere. One is plugin-level integration of JS libraries that provide CodeAssist, samples, snippets and easy inclusion of the libraries files into your project; we provide the plugins for many of the more commonly used libraries, including YUI, jQuery, Prototype, dojo and EXT JS.
Second, we have a server-side JavaScript engine called Jaxer that not only lets you run any of your JS code on the server but adds file, database and networking functionality so that you don't have to use a scripting language but can write the entire app in JS.

Try the Vjet Javascript IDE from ebay (installation)

Ganymede's version of WTP includes a revamped Javascript editor that's worth a try. The key version numbers are Eclipse 3.4 and WTP 3.0. See http://live.eclipse.org/node/569

There once existed a plugin called JSEclipse that Adobe has subsequently sucked up and killed by making it available only by purchasing and installing FlexBuilder 3 (please someone prove me wrong). I found it to worked excellent but have since lost it since "upgrading" from Eclipse 3.4 to 3.4.1.
The feature I liked most was Content Outline.
In the Outline window of your Eclipse
Screen, JSEclipse lists all classes in
the currently opened file. It provides
an overview of the class hierarchy and
also method and property names. The
outline makes heavy use of the code
completion engine to find out more
about how the code is structured. By
clicking on the function entry in the
list the cursor will be taken to the
function declaration helping you
navigate faster in long files with
lots of class and method definitions

The new release of Eclipse (Helios) has an especific package for javascript web development. I haven't tried it yet, but it certainly worth a look.

Didn't use eclipse for a while, but there are ATF and Aptana.

Oracle Workshop for WebLogic (formally BEA Workshop) has excellent support for JavaScript and for visually editing HTMLs. It support many servers, not only WebLogic, including Tomcat, JBoss, Resin, Jetty, and WebSphere.
It recently became free, check out my post about it. Given that it was an expensive product not long ago, I guess it's worth checking out.