Develop Reference

Google Apps Script JSON.parse replaces all ":" (colon) in a URLFetchAPP with =" (equals sign)

JSON.parse of a URLFetchApp on Google App Scripts changes all colons in a string to an equals sign. It's doing the exact opposite of what I thought JSON.parse is supposed to do. I tried parsing the same string in different places and even made a node.js fetch and output all works. Seems to be a problem with Google AppScript
Example:
CODE
var res = UrlFetchApp.fetch(url, options)
var text = res.getContentText()
var json = JSON.parse(text)
Logger.log(text)
Logger.log(json)
LOG:
{"status":{"timestamp":"2021-03-24T16:01:13.657Z","error_code":0,"error_message":null,"elapsed":20,"credit_count":1,"notice":null},"data":[{"id":1,"symbol":"BTC","name":"Bitcoin","amount":1,"last_updated":"2021-03-24T16:00:03.000Z","quote":{"GBP":{"price":41091.43869808855,"last_updated":"2021-03-24T16:00:21.000Z"}}}]}
{status={elapsed=20.0, timestamp=2021-03-24T16:01:13.657Z, error_message=null, notice=null, credit_count=1.0, error_code=0.0}, data=[{amount=1.0, symbol=BTC, last_updated=2021-03-24T16:00:03.000Z, id=1.0, name=Bitcoin, quote={GBP={price=41091.43869808855, last_updated=2021-03-24T16:00:21.000Z}}}]}

It's not a problem, it's a feature.
Instead of Logger.log(json) use Logger.log(JSON.stryngify(json)).
Related
Log Javascript object as string Google App Scripts
Objects with and without quotation marks

Counting the number of lines in Google Document

Problem:
I'd like to be able to count the number of lines in a Google Document. For example, the script must return 6 for the following text.
There doesn't seem to be any reliable method of extracting '\n' or '\r' characters from the text though.
text.findText(/\r/g) //OR
text.findText(/\n/g)
The 2nd line of code is not supposed to work anyway, because according to GAS documentation, 'new line characters are automatically converted to /r'

If you are still looking for the solution, how about this answer? Unfortunately, I couldn't find the prepared methods for retrieving the number of lines in the Google Document. In order to do this, how about this workaround?
If the end of each line can be detected, the number of lines can be retrieved. So I tried to add the end markers of each line using OCR. I think that there might be several workarounds to solve your issue. So please think of this as one of them.
At Google Documents, when a sentence is over the page width, the sentence automatically has the line break. But the line break has no \r\n or \n. When users give the line break by the enter key, the line break has \r\n or \n. By this, the text data retrieved from the document has only the line breaks which were given by users. In your case, it seems that your document has the line breaks for after incididunt and consequat.. So the number of lines doesn't become 6.
I thought that OCR may be able to be used for this situation. The flow is as follows.
Convert Google Document to PDF.
Convert PDF to text data using OCR.
I selected "ocr.space" for OCR.
If you have already known APIs of OCR, you can try to do this.
When I used OCR of Drive API, the line breaks of \r\n or \n were not added to the converted text data. So I used ocr.space. ocr.space can add the line breaks.
Count \n in the converted text data.
This number means the number of lines.
The sample script for above flow is as follows. When you use this, please retrieve your apikey at "ocr.space". When you input your information and email to the form, you will receive an email including API key. Please use it to this sample script. And please read the quota of API. I tested this using Free plan.
Sample script :
var apikey = "### Your API key for using ocr.space ###";
var id = DocumentApp.getActiveDocument().getId();
var url = "https://docs.google.com/feeds/download/documents/export/Export?id=" + id + "&format=pdf&access_token=" + ScriptApp.getOAuthToken();
var blob = UrlFetchApp.fetch(url).getBlob();
var payload = {method: "POST", headers: {apikey: apikey}, payload: {file: blob}};
var ocrRes = JSON.parse(UrlFetchApp.fetch("https://api.ocr.space/Parse/Image", payload));
var result = ocrRes.ParsedResults.map(function(e){return e.ParsedText.match(/\n/g).length})[0];
Logger.log(result)
Result :
When your sentences are used, 6 is obtained as the result of script.
Note :
Even if the last line of the document has no \r\n or \n, the converted text data has \r\n at the end of all lines.
In this case, the precision of OCR is not important. The important point is to retrieve the line breaks.
I tested this script for several documents. In my environment, the correct number of line can be retrieved. But I'm not sure whether this script works for your environment. If this script cannot be used for your environment, I'm sorry.

As you noted in the comments there is no API to do retrieve the number of lines in Google Docs. This happens because the document is rendered dynamically in the client side, so the server doesn't know this number.
One possible solution is scraping the HTML of the Google Doc, because each line is redered with it's own divwith the "kix-lineview" class, however you will need to actually open the page in an iframe or headless browser and then scroll page by page to make them render and then be able to count the divs

After publishing your Google Doc with «Publish to the web» in «File» menu, use the URL in the following script:
var url = "https://docs.google.com/document/d/e/2PACX-1vSElK...iwUhaFo/pub";
var text = UrlFetchApp.fetch(url).getContentText();
var count = (text.match(/<\/br>/g) || []).length;
Logger.log(count.toString());
This is only handy if all of your document lines are ended in </br>, although there is the possibility to add any other variants:
var url = "https://docs.google.com/document/d/e/2PACX-1vSElK...iwUhaFo/pub";
var text = UrlFetchApp.fetch(url).getContentText();
var count1 = (text.match(/<\/br>/g) || []).length;
var count2 = (text.match(/<\/p>/g) || []).length;
var count3 = (text.match(/<hr>/g) || []).length;
var count = coun1 + coung2 + count3;
Logger.log(count);

Google Script HMAC SHA512 output differs when adding '\n'

I'm trying to create a google script that generates a HMAC signature using SHA512 for authentication to an API. Here is the code I'm using:
var SystemTimeMilli = Date.now()
var InputString = ('Some Text' + '\n' + SystemTimeMilli);
var PrivateKey = 'bEDtDJnW0y/Ll4YZitxb+D5sTNnEpQKH67EJRCmQCqN9cvGiB8+IHzB7HjsOs3mSlxLmu4aiPDRpe9anuWzylw==' //sample private key
var signature = Utilities.base64Encode(Utilities.computeHmacSignature(Utilities.MacAlgorithm.HMAC_SHA_512, InputString, PrivateKey, Utilities.Charset.UTF_8));
Logger.log(signature);
This generates a string to the log but when compared to results using web based hash generators the results differ.
However if I use
var InputString = ('Some Text' + SystemTimeMilli);
without the \n the results generated in my Google Script and the web based hash generators match.
So I'm assuming the way Google Script is handling the new line is different to the web base generators but I can't figure out how to get them match or which one is the correct result.
EDIT
To add some further information:
Lets say SystemTimeMilli = 1234567890123
When I enter ('Some Text' + SystemTimeMilli) into my script using
bEDtDJnW0y/Ll4YZitxb+D5sTNnEpQKH67EJRCmQCqN9cvGiB8+IHzB7HjsOs3mSlxLmu4aiPDRpe9anuWzylw== as the PrivateKey
I get
oDVJk38b634G9Ykdqm+hVmSb5C8h6/re5/XG+MjyddTKygWcoZjWu0DfNGY/JWu5Be41zLDKfEZnuFcACrvs7w== returned.
Which is the same if I enter /account/balance1234567890123 into https://quickhash.com using SHA-512 (SHA2) as the Algorithm and Base64 Encoding as the Output Format.
However if I enter ('Some Text' + '\n' + SystemTimeMilli) in my script it returns 3sj1jbx1tQ4nc8XOkk3nW8TvG5zvCam2LN51fQWDXOKU5O/1KBAdzdp+YV+aAdYHWCY2x3jqCfbXKoOmfoD0KA==
But using
/account/balance
1234567890123
in quickhash.com returns something different. I've tried a whole bunch of different combinations but I never get the same result as my script.
I having trouble with getting my script to authenticate to the API so I really want to determine if I'm getting the correct HMAC to begin with.

How to encode a list name for REST in sharepoint 2013

I created a list in the en site called
hello's~`!##$%^&(){}[]:;"'<>?/\|-+.
What I am trying to do is figure out what exactly is the encoded version of the name if I am trying to target it by name in a REST url. I have this
http://mysite/en/_api/web/lists/getbytitle('hello's~`!##$%^&(){}[]:;"'<>?/\|-+.')
but this doesn't work of course. I tried url encoding it to
http://mysite/en/_api/web/lists/getbytitle('hello%27s~%60!%40%23%24%25%5E%26*()%7B%7D%5B%5D%3A%3B%22%27%3C%3E%3F%2F%5C%7C*-%2B.')
but even then, doesn't work.
Does anyone know how to do it?
Thanks

The expression:
hello's~`!##$%^&(){}[]:;"'<>?/\|-+.
contains at least the following invalid symbols <,>,*,%,&,:,\\,? (follow HttpRuntimeSection.RequestPathInvalidCharacters Property for a more details)
So, if list title contains any of the specified invalid symbols then A potentially dangerous Request.Path value was detected from the client error will be returned in REST request.
Regarding ' (single quote) symbol, it should be properly escaped by replacing it with '' (double quote) symbol.

POST data issues

I have an issue with submitting post data. I have a form which have a couple of text fields in, and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string like
title=test&content=some content
which is then submitted to the server. The problem I had is when I have '&' (eg &nbsp) entered into one of the inputs which then breaks up the query string. Eg:
title=test&content=some content &nbsp
How do I get around this?
Thanks in advance,
Harry.

Run encodeURIComponent over each key and value.
var title = "test";
var content = "some content &nbsp ";
var data = encodeURIComponent('title') + /* You don't actually need to encode this as it is a string that only contains safe characters, but you would if you weren't sure about the data */
'=' + encodeURIComponent(title) +
'&' + encodeURIComponent('content') +
'=' + encodeURIComponent(content);

Encode the string..when you want to encode a query string with special characters you need to use encoding. ampersand is encoded like this
title=test&content=some content %26
basically any character in a query string can be replaced by its ASCII Hex equivalent with a % as the prefix
Space = %20
A = %41
B = %42
C = %43
...

You need to encode your query to make it URL-safe. You can refer to the following links on how to do that in JS:
http://xkr.us/articles/javascript/encode-compare/
http://www.webtoolkit.info/javascript-url-decode-encode.html

You said:
...and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string...
In the section where you are building the query string you should also run the value of each input through encodeURIComponent() as David Dorward suggested.
As you do - be careful that you only assign the new value to your processed query string and NOT the form element value, otherwise your users will think their input was somehow corrupted and potentially freak out.
[EDIT]
I just re-read your question and realized something important: you're encoding an &nbsp ;character. This is probably a more complicated issue than other posters here have read into. If you want that character, and other &code; type characters to transfer over you'll need to realize that they are codes. Those characters &, n, b, s, p and ; are not themselves the same as " " which is a space character that does not break.
You'll have to add another step of encoding/decoding. You can place this step either before of after the data is sent (or "POSTed").
Before:
(Using this question's answers)
var data = formElement.value;
data = rhtmlspecialchars(data, 0);
Which is intended to replace your "special" characters like with " " so that they are then properly encoded by encodeURIComponent(data)
Or after:
(using standard PHP functions)
<?PHP
$your_field_name = htmlspecialchars_decode(urldecode($_POST['your_field_name']));
?>
This assumes that you escaped the & in your POST with %26
If you replaced it with some function other than encodeURIComponent() you'll have to find a different way to decode it in PHP.

This should solve your problem:
encodeURIComponent(name)+'='+encodeURIComponent(value)+'&'+encodeURIComponent(name2)+'='+encodeURIComponent(value2)
You need to escape each value (and name if you want to be on the safe side) before concatenating them when you're building your query.
The JavaScript global function encodeURIComponent() does the escaping.
The global function escape() (DOM) does this for you in a browser. Although people are saying it is not doing the escaping well for unicode chars. Anyway if you're only concerned about '&' then this would solve your problem.