I need to provide text information in a file, read it in java-script and then work with the output. The output is not supposed to go to a web page, but is a plugin for another program (RPG Maker MV).
I'm trying to load a text-file (*.txt) and output it with document.write() I'm currently checking the code in Firefox. I get why this is prevented from working, but ultimately I do not want to run it in a Browser anyway. However I need to try the code before implementing it.
My text file looks like this:
What is the Capital of France?
-Berlin
-Paris [X]
-Koppenhagen
Who came up with the Theory of Evolution?
-Charles Darwin [X]
-Thomas Eddison
-Nicolas Flamel
The java-script should do something like:
var mytext=fs.readFileSync('quiz.txt');
document.write(mytext);
I could also use other file types than *.txt of course. Important is, that the quiz-file can be adapted without messing with the original java-script file.
Unfortunately, you can't use the same code to read in file in NodeJS and from code running in a browser as they use very different APIs.
NodeJS relies on the fs module while the browser environment relies on the File APIs specified by W3C.
Related
This is a topic where I can't seem to find the answer on the Node.js docs (I know it's possible because of libraries like exif), nor can I find an answer on the internet without everyone saying to just use a library.
I don't want to use a library, so I want to do this natively and learn more about reading file metadata, and maybe eventually updating the metadata too while building my own mini-tool.
If I run something like fs.statSync() I can get generic metadata that returns in the Stats object; but, in my case, I'm looking for all the other metadata, NOT just the basic file info like size, birthtime, etc.
I want the other metadata like dimensions, date taken, and especially things you'd see in image, video, or audio files.
Maybe there's something like:
const deepMetaData = fs.readFileSync().getMetaDataAsString();
console.info(/Date Taken/.test(deepMetaData)); // true
or
const deepMetaData = fs.createReadStream().buffer().toString();
const dateTaken = deepMetaData.match(/Date Taken: (\d{4}-\d{2}-\d{2})/)[1];
console.info(dateTaken);
If I need to work with buffers, streams, whatever, instead of a string output, that's cool too. Ideally something synchronous. So if there's a simple example someone could provide of how to read that kind of meta data without a library, I'll at least be able to look up the methods used from that to understand more later and leverage the docs associated with whatever approach. Thank you!
Nodejs fs functions like fs.statSync() provide OS level metadata on the file only (such as createDate, modificationDate, file size, etc...). These are properties of the file in the file system. These do NOT have anything at all to do with the actual data of the file itself.
When you talk about EXIF (for a photo), this is parsed from the file data itself. To know about that type of data, you must read and parse at least the beginning of the file and you must be able to recognize and understand all the different file formats that you might encounter. For photos, this would include JPEG, PNG, HEIC, GIF, etc... Each of those have different file formats and will require unique code for understanding the metadata embedded in the file.
Nodejs does not have support for any of that built-in.
So, it will take custom code for each file type. If you further want to include other types of files like videos, you need to extend your list of different file types you can read, parse and understand. For the depth of files you're talking about this is a big job, particular when it comes to testing against all the different variants of files and metadata that exist out in the wild.
I personally would be fine with implementing my own code for one particular file type like JPEG, but if I was tasked with supporting dozens of types of files and particularly if tasked with supporting the wide range of video file formats, I'd immediately seek out help from existing libraries that have already done all the time consuming work to research, write and test how to properly read and understand all the variants.
I know it's possible because of libraries like exif
This is an example of a library that reads the beginning of the image file, parses it according to the expected format and knows how to interpret all the possible tags that can be in the EXIF header and what they all mean.
So if there's a simple example someone could provide of how to read that kind of meta data without a library
Go study the code for the EXIF library and see how it works. If you're going to implement it yourself, that's how you have to do it. I'm still not sure why you'd avoid using working libraries if they already exist. That is one of the biggest advantages of the nodejs ecosystem - you can build on all the open source code that already exists without reimplementing it all from scratch yourself and spend your coding time on parts of your problem that someone else has not already implemented.
how would one read that metadata using node?
You literally have to read the data from the file (usually at the start of the file). You can use any of the mechanisms that the fs module provides. For example, you can use fs.createReadStream() and then stream in the file, parsing and interpreting it as data arrives and then stop the stream when you get past the end of the metadata. Of, you can open a file handle using fs.open() and use fs.read() to read chunks of the file until you have read enough to have all the metadata.
You HAVE an example sitting right in front of you of code that does this in the EXIF library on NPM that you already seem to know about. Just go examine its code. The code is ALL there.
I'm just looking for a simple answer on getting that info, even if it's a blob of strings.
This is perhaps your main problem. There is no simple answer to get that info and it doesn't just exist as a blob of strings. These files are sometimes binary files (for space efficiency reasons). You have to learn how to read and parse binary data. Go study the code in the EXIF library and see what it is already doing and you can learn from that. There is no better example to start with.
But, for a simple example using the heic filetype, this will grab the first 5000 characters of the file's metadata, which can then be searched:
const fileDescriptor = fs.openSync(absPathToHeicPhoto);
const charCount = 5000;
const buffer = Buffer.alloc(charCount);
const headerBytes = fs.readSync(fileDescriptor, buffer, 0, charCount);
const bufferAsStr = buffer.toString('utf8', 0, charCount);
console.info(/\d{4}:\d{2}:\d{2}/.test(bufferAsStr));
FYI, I looked at the code for this EXIF library on NPM and it's poorly implemented. It uses fs.readFile() to load the ENTIRE image into RAM (even though it only needs a fraction of the data at the start of the file). This is a poor implementation for this reason (memory and disk inefficient).
But, it does have a method called processImage and one called extractExifData that process the binary data of the file to parse out the EXIF info. These are links to its actual code. You can start learning there.
FYI, as a photographer, I use a command line program called exiftool that will dump exif info to stdout or to a file for many images. As a different approach, you could just run that tool from your nodejs program (using the child_process module and capture its output and use that output, letting it do the hard work you just operate on the generated output.
Sorry for this beginner question, I never used js server-side before.
here is my problem:
I have some javascript downloaded from a remote page (it's encrypted, I can't convert it to php), I need to execute it and read its output.
How can I do it? I'm thinking about something like this:
shell_exec('nodejs code...')
but how to pass the code? It's quite long, about 10 lines of javascript.
Another way would be to store the js to a file and run nodejs script.js, but that would be a useless and slow disk IO...
Important caveat about using exec/shell_exec
I feel the need to prepend a caveat about security to this answer. Always be careful when using exec or shell_exec. You almost always should not be taking data over the network to inject into a shell command for security reasons. Writing the script to a file would be much safer because there is no risk of command injection. If you are confident that this approach is required then I strongly advise you to.
Use the PHP function escapeshellcmd which will try to escape shell control characters.
Really ask yourself how much you trust the source? And how much you trust their security?
Having said that. Here's my original answer to the question as asked:
It sounds like the missing piece for your puzzle is the -e parameter for node. This will allow you to pass a script as part of the command invocation.
E.g.
C:\Users\Cmonahan>node -e "console.log('hello world');"
hello world
You can then use PHP exec or shell exec to get the output.
More information:
PHP shell_exec() vs exec()
Node CLI documentation
Edit: Regarding passing multiline arguments to the command line. This can be a bit of a minefield. For example: It depends on whether it is a Unix-like or Windows environment and then, if Unix-like, what shell is parsing the command.
See for example:
Windows: How to specify multiline command on command prompt?
End of line (new line) escapes in bash
I would recommend just making sure the argument is a single line. In the case of JS you can try minification first, which typically strips out all newlines, and see if that works for you.
Here's a popular PHP based minifier https://github.com/mrclay/minify I believe you should be able to install via composer.
dont know am not that at home with php, with isnt it true that php serves html files to the user at home, so why would it be any different from using a javascript file in a normal html page?
cant you just use:<script src="myscripts.js"></script>"
or <script>"with here your script"</script>
in the file that need to load this javascript?
ive tried everything i cud to figure this out, but i cannot track a piece of javascript in a webpage
so, just to give you some context even though my problem is not related to just this scenario. it depends on a much bigger spectrum.
Anyway, im developing on sugarCRM and im trying to edit the default onclick behavior of a slot in calendar module (you dont need to understand this to help me, so please keep reading). when i click on a slot, a modal dialog window opens that lets me log a meeting or a call.
So i tracked down the javascript behind this. ive used firebug and chrome, and they both give a list of all the JS files that are being used on a given webpage
for example i search for "SUGAR.collection" and firebug tells me its located in a file named "sugar_field_grp.js?v=FVh1Z-v5nA6bYov7-aFFqQ" i can see this piece of code resides in sugar_field_grp.js,
but the code im trying to change resides in "index.php?module=Calendar&action=index&parentTab=Activities", firebug actually tells me this is the file that has the javascript i want to change.
I can also right click view page source and i can see that piece of code inside the script tag. so considering this piece of code doesnt reside in a JS file, i cannot change it, its generated at runtime (i think) but there must be some source, there must be a file thats telling sugarCRM to generate this code
tl;dr how to track down a piece of javascript code that resides on pagesource and theres no JS file specified by firebug or chrome save for index.php (this file doesnt have that javascript either)
i know its been a long post
thanks for reading
Learn how to search for strings in files on disk on your machine.
On Linux, MacOS and most unixen the go-to tool for this is grep. This applies to any programming language you work with. For your case simply cd into the directory of your source code and do:
grep -r SUGAR.collection .
If you're using git as your source control tool then git grep is much faster.
On Windows there are various GUI tools you can use to search for text in files. Just google: grep for windows.
If you're using an IDE then just your IDE's find-in-files functionality.
To track down specific code using Chrome / Webkit go through the following two steps:
Client:
1. Search all static text sources
Open the Dev Panel using CTRL + SHIFT + I
Hit CTRL + SHIFT + F for a global search dialog to pop up
Right next to it you can set pretty printing of the JS code to on: button { }
Enter your search term or terms using regular expressions
Optional: Decide if you need a case insensitive search which has a greater searchspace and takes longer
Example:
2. Search the dynamic user-DOM contents
Go to the Tab 'Elements' hit CTRL + F.
Enter your search term (This will also search iframes, svg's etc... within the parent DOM)
3. Recommended:
Cross-reference the results of step 1. and step 2.
If a given string is present in both the DOM and the static sources, then you can assume that the content is not programmatically created on the client-side.
Server:
Many projects perform a media bundling step prior to content-delivery. They pack web-resources into the main file (e.g. index.php) to save HTTP roundtrips.
Use sourcemaps / and or search the entire codebase for a salient static string or a salient keyword near the static string to locate the original source files.
Searching files:
Locally, I generally use the rapid index, and heuristic search of JetBrain's IDE's (IDEA, PHPStorm,...) and Sublime. The grep-command tool can definitely not compete here in terms of performance. On Windows I additionally use Totalcommander and its archive/regex finding abilities.
When quickly looking up code on the server you may use something like:
grep -r -C10 --color=always 'keyword1|keyword2' htdocs/ | less -R
which will also provide you with line-context. two caveats: you may want to filter out binaries first and symlinks outside the scope will be ignored.
I would like to read a pptx file in javascript, so I would like to unzip it and read the content in memory. I don't want to store the file first on a server. I want to choose a file with a input type file and just use the file of the input-element and read it binary or something like that.
I found a lot of libraries to unzip zip-files from url, I tried to look at the code but I couldn't figure it out to use it for a blob or byte array.
I can read some stuff like the things described here: http://en.wikipedia.org/wiki/ZIP_%28file_format%29#File_headers
But I don't know how deflating works on byte- or bit-level.
(You've said you want to use an input element, so I'm guessing this is browser-based JavaScript.)
Your first step will be to use the File API to read the file as a binary string. See my answer to this other question for an example of that. Then you'll need to find a library. A quick search discovered this one that implements both inflate and deflate. (I don't have personal experience using it, just found it in an answer to this other question.)
Naturally this will only work on quite modern browsers that support the File API. Otherwise, you have no option but to push the file to a server and do the work there, since you can't access the content of the file in the browser without the File API.
I try to get to a page straight from Bash at http://www.ocwconsortium.org/. The page appears when you write mathematics to the field at the top right corner. I tested
open http://www.ocwconsortium.org/#mathematics
but it leads to the main page. It is clearly some javascript thing. How can I get the results straight from Bash on the first page?
[Clarification]
Let's take an example. I have the following lines for a Math search engine in .bashrc:
alias mathundergradsearch='/Users/user/bin/mathundergraduate'
Things in a separate file:
#!/bin/sh
q=$1
w=$2
e=$3
r=$4
t=$5
open "http://www.google.com/cse?cx=007883453237583604479%3A1qd7hky6khe&ie=UTF-8&q=$q+$w+$e+$r+$t&hl=en"
Now, I want something similar to the example. The difference is that the other site contains javascript or something that does not allow me to see the parameters. How could I know where to put the search parameters as I cannot see the details?
open "http://www.ocwconsortium.org/index.php?q=mathematics&option=com_coursefinder&uss=1&l=&s=&Itemid=166&b.x=0&b.y=0&b=search"
You need quotes because the URL contains characters the shell considers to be special.
The Links web browser more or less runs from the commandline (like lynx) and supports basic javascript.
Even though the title of the post sounds general, your question is very specific. It's unclear to me what you're trying to achieve in the end. Clearly you can access sites that rely heavily on javascript (else you wouldn't be able to post your question here), so I'm sure that you can open the mentioned site in a normal browser.
If you just want to execute javascript from the commandline (as the title suggests), it's easy if you're running bash via cygwin. You just call cscript.exe and provide a .js scriptname of what you wish to execute.
I didn't get anything handled by JavaScript - it just took me to
http://www.ocwconsortium.org/index.php?q=mathematics&option=com_coursefinder&uss=1&l=&s=&Itemid=166&b.x=0&b.y=0&b=search
Replacing mathematics (right after q=) should work. You may be able to strip out some of that query string, but I tried a couple of things and and it didn't play nice.
Don't forget to encode your query for URLs.
You will need to parse the response, find the URL that is being opened via JavaScript and then open that URL.
Check this out: http://www.phantomjs.org/.
PhantomJS it's a CLI tool that runs a real, fully-fledged Browser without the Chrome.