I'm attempting to create my first ever full-blown website. The site will host a form that takes a user's link and scrapes the inputted website. Besides scraping, I'm also looking for click and input functionality (for future projects).
I managed to create a PhantomJS function that scrapes a website depending on the parameters. However, I have absolutely no idea how to implement this into my Xampp server which locally hosts my website...PhantomJS only runs off of a command line as far as I'm aware.
Is it even possible to host a PhantomJS program on a website? Should I be using something else, such as NodeJS or another language? Ultimately, I'll go learn anything in order to make this happen.
Any ideas/suggestions are much appreciated. I apologize if this question seems dumb.
Related
I want to make a website for my band to practice and learn new things. My band already has a site through wix. Is it possible to write a script on the new page to tell it look at the first pages html and load that then use css and js to restyle and add features? that way when the content on the first page changes it will be loaded into the second page.
I do web work and I have got this question a lot from people who use site builders.
All in all, Wix's does not have a way to export or download your source code, as for your website to work, it needs to be on Wix's servers. Wix is advertised as Software-As-A-Service. For Wix, they are a web hosting/building service.
If you were to make a program to see the source code, you would spend more time making the program then getting use out of it. I do not mean to be rude saying that, but there are easier ways of building the site.
For instance, If you want to build the site from scratch and want to have a web builder, you could look at using WordPress. It's free and open source. My website in my bio runs on WordPress and the User interface/editor is very user friendly!
If you did want to play around with it and get it locally set up on your computer, you can download XAMPP to act as a "host" for your "site" on the computer and let you edit your WordPress page.
How can I run WordPress locally on a computer?
Of course, if you do decide to publish your site, I would recommend Namecheap as I use their hosting services too! They have WordPress that comes with their host and is a 1-click install.
I apologize if I got wordy in my answer. I honestly hope this information helps! If you ever need help with your site, my site has my contact info!
P.S: Welcome to Stack overflow! Glad to see another user on here :)
I'm trying to write a test application for hardware and it needs to be webbased however the actual program for controlling the hardware is already made.
I made a webpage with html and javascript but i need to send data between these two applications, the applications both run on the same machine. I tried looking on the web but I cant figure it out. If anyone has an example or a direction for me to look at/in I would appreciate it.
I'm intending to create an app using electron to extract data from a website and save it in a DB. This website is similar to google(an input and a search button, but no API) instead in addition requires for the first search to introduce a captcha code. My concern is to find out what's the simple way to automatize the process of sending requests and collecting the results. Can be Selenium a way or there is no need of other extensions to achieve it? Please offer me advice?(I don't have Electron experience)
I'm sorry, I don't completely understand the problem.
You want to scrap data from a website. But why you want to do so with Electron?
You can do it with selenium in a lot of different ways (and languages), using the drivers (like chromedriver).
The question is about Electron and Selenium or about how to avoid captcha and automatize the process?
In the first case, you can follow:
https://github.com/electron/electron/blob/master/docs/tutorial/using-selenium-and-webdriver.md
In the other case, it depends from the website and from a lot of other things.
PS: If I set up a captcha, it's because I don't want that someone scrap my data :)
My workplace keeps client records in a database which is accessible via an aspx application hosted on our local network. The page allows logging in, simple searching, and retrieving of information, which is displayed on this webpage. A task I am performing requires I access each record, copy a block of text into a new database which we are creating, and move to the next record (thousands of times).
My question is, is there a way to automate this? I've looked at browser macros, but the task of searching for each record does not allow for a simple macro interface. I was hoping to write a program which directly interacts with the aspx application without the need for the browser. In the code I'd be able to tell the application which records i'm looking for (by date, or unique ID, for example), and have the aspx application return the appropriate fields of text I need. I don't have any back-end access to the aspx application (it is a third party, proprietary application and their developers are not interested in helping me with this task)
Can this be done?
If yes, in broad terms, what sort of solution should I be looking at? Is there a particular language that is more suitable to this task?
Any help is appreciated. I realize the question is a bit vague and broad, but the third-party application is only hosted locally so I can't give an example. I know a little ruby, and came across the gem Mechanize, which looks like it might work... but I can't find much about interacting with asp.net
You're on the right track with ruby mechanize. Mechanize will keep track of the cookies for you and make it easy to submit forms (logging in, asp doPostBack actions). Give it a try, and if you get stuck, post specifics in a new question.
It's quite hard for me to figure out if this sort of thing has ever been implemented. I want to look for any libraries that may exist so I don't go about reinventing the wheel.
I have this idea of having a web app that connects the people who are on the site. Every user that is connected to the site may communicate to another user also on the site via the server. So the protocols will be implemented in JavaScript, and the server simply helps to identify users, and just echoes data to enable the communication. For instance I can use this to implement my game networking ideas in javascript, and easily test them without having my testers download any executables, they can just log onto the site.
Now obviously this isn't going to be an effective architecture for any kind of serious application. But I think if I can get it working I could build really cool networking apps without having any sort of download.
What I'm thinking about is using ajax for client->webserver and webserver->client (Comet?) and I can code up the webserver echo bit with PHP or a cgi script. And then I can implement an entirely separate protocol in JS that the webserver does not care or know about.
The reason for having the webserver echo everything is because I don't want to use java or anything else that I can open up sockets in. Why make it harder for me? Because I can and because I happen to be really enamored with javascript at the moment. It's the only web technology I trust. Screw java applets.
Does this make any sense to anyone? Am I crazy?
Don't know about the crazy part (there's a proposal at area51, go check that) but it's definitely doable.
You could use a plain old XMPP server and a javascript XMPP client (there are libraries - for example strophe)
You could do it with AJAX and a PHP backend: Making an AJAX Web Chat
You could use the fancy Websockets from HTML5: Start Using HTML5 WebSockets
You could use some existing component if you can find any (I couldn't find any I would use)
Cheers :)