Develop Reference

trying to decompress xref stream from pdf - getting "ERROR incorrect header check"

I am trying to parse the xref stream from PDF in JavaScript. I managed to succesfully isolate the stream itself (I checked that it's ok by comparing it in debugging mode with the value between steram. and endstream tags in PDF.
However, when I try to inflate it using pako lib, I get an error saying: ERROR incorrect header check.
The compression method is FlateDecode, which can be seen from the dictionary.
Here is the code in question:
const dict = pdfStr.slice(pdf.startXRef);
const xrefStreamStart = this.getSubstringIndex(dict, 'stream', 1) + 'stream'.length + 2;
const xrefStreamEnd = this.getSubstringIndex(dict, 'endstream', 1) + 1;
const xrefStream = dict.slice(xrefStreamStart, xrefStreamEnd);
const inflatedXrefStream = pako.inflate(this.str2ab(xrefStream), { to: 'string' });
pdfStr is the whole PDF read as a string, while *pdf.startXRef* holds the value of the position of the xref stream object.
Here's the whole PDF if someone wants to have a look: https://easyupload.io/lzf9he
EDIT: As mcernak has suggested I had a problem that I included /r and /n in the stream. However, now that I corrected the code I got a different error: invalid distance too far back

The stream content is located between stream\r\n and \r\nendstream.
You need to take into account those two additional characters (\r\n) at the beginning and at the end to read the correct data:
const dict = pdfStr.slice(pdf.startXRef);
const xrefStreamStart = this.getSubstringIndex(dict, 'stream', 1) + 'stream'.length + 2;
const xrefStreamEnd = this.getSubstringIndex(dict, 'endstream', 1) - 2;
const xrefStream = dict.slice(xrefStreamStart, xrefStreamEnd);
const inflatedXrefStream = pako.inflate(this.str2ab(xrefStream), { to: 'string' });

Python body.encode( ) Javascript alternative

I'm trying to verify a webhook coming from Plaid in NodeJS by calculating the Sha256 of the webhook body and I'm following a Python code here where the code is showing :
# Compute the has of the body.
m = hashlib.sha256()
m.update(body.encode())
body_hash = m.hexdigest()
What's the alternative of body.encode() in Javascript before passing it to the Sha256 function please ? Note that the body I'm getting is an object containing the following data :
{ error: null, item_id: '4zAGyokJ1XiWP63QNl1RuLZV76o55nudVXzNG',
new_transactions: 0, webhook_code: 'DEFAULT_UPDATE', webhook_type:
'TRANSACTIONS' }
However I'm trying to get this hash :
b05ef560b59e8d8e427433c5e0f6a11579b5dfe6534257558b896f858007385a

So, if the body is JSON (NOT JSON STRING) then you need to stringify it and put it in the .update function As the m.body takes a string. If you have your body as STRING then just put it in as is.
This is from the Crypto Example here:
const crypto = require('crypto');
const hash = crypto.createHash('sha256');
const stringBody = JSON.stringify(body);
hash.update(stringBody);
console.log(hash.digest('hex'));
Edit:
If the hash is not same then maybe you need to correct the newlines or whitespaces. You need to make both bodies exactly the same. Here In the below example I am using same exact string and encoding using Python AND NodeJS.
import hashlib
body = '{"error":null,"item_id":"4zAGyokJ1XiWP63QNl1RuLZV76o55nudVXzNG","new_transactions":0,"webhook_code":"DEFAULT_UPDATE","webhook_type":"TRANSACTIONS"}'
m = hashlib.sha256()
m.update(body.encode())
body_hash = m.hexdigest()
print(body_hash)
Output:
python3 file.py
26f1120ccaf99a383b7462b233e18994d0c06d4585e3fe9a91a449e97a1c03ba
And Using NodeJS:
const crypto = require('crypto');
const hash = crypto.createHash('sha256');
const body = {
error: null,
item_id: '4zAGyokJ1XiWP63QNl1RuLZV76o55nudVXzNG',
new_transactions: 0,
webhook_code: 'DEFAULT_UPDATE',
webhook_type: 'TRANSACTIONS'
}
const stringBody = JSON.stringify(body);
hash.update(stringBody);
console.log(hash.digest('hex'));
Output:
node file.js
26f1120ccaf99a383b7462b233e18994d0c06d4585e3fe9a91a449e97a1c03ba

Looking for the easiest way to extract an unknown substring from within a string. (terms separated by slashes)

The initial string:
initString = '/digital/collection/music/bunch/of/other/stuff'
What I want: music
Specifically, I want any term (will never include slashes) that would come between collection/ and /bunch
How I'm going about it:
if(initString.includes('/digital/collection/')){
let slicedString = initString.slice(19); //results in 'music/bunch/of/other/stuff'
let indexOfSlash = slicedString.indexOf('/'); //results, in this case, to 5
let desiredString = slicedString.slice(0, indexOfSlash); //results in 'music'
}
Question:
How the heck do I accomplish this in javascript in a more elegant way?
I looked for something like an endIndexOf() that would replace my hardcoded .slice(19)
lastIndexOf() isn't what I'm looking for, because I want the index at the end of the first instance of my substring /digital/collection/
I'm looking to keep the number of lines down, and I couldn't find anything like a .getStringBetween('beginCutoff, endCutoff')
Thank you in advance!

your title says "index" but your example shows you wanting to return a string. If, in fact, you are wanting to return the string, try this:
if(initString.includes('/digital/collection/')) {
var components = initString.split('/');
return components[3];
}

If the path is always the same, and the field you want is the after the third /, then you can use split.
var initString = '/digital/collection/music/bunch/of/other/stuff';
var collection = initString.split("/")[2]; // third index
In the real world, you will want to check if the index exists first before using it.
var collections = initString.split("/");
var collection = "";
if (collections.length > 2) {
collection = collections[2];
}

You can use const desiredString = initString.slice(19, 24); if its always music you are looking for.

If you need to find the next path param that comes after '/digital/collection/' regardless where '/digital/collection/' lies in the path
first use split to get an path array
then use find to return the element whose 2 prior elements are digital and collection respectively
const initString = '/digital/collection/music/bunch/of/other/stuff'
const pathArray = initString.split('/')
const path = pathArray.length >= 3
? pathArray.find((elm, index)=> pathArray[index-2] === 'digital' && pathArray[index-1] === 'collection')
: 'path is too short'
console.log(path)

Think about this logically: the "end index" is just the "start index" plus the length of the substring, right? So... do that :)
const sub = '/digital/collection/';
const startIndex = initString.indexOf(sub);
if (startIndex >= 0) {
let desiredString = initString.substring(startIndex + sub.length);
}
That'll give you from the end of the substring to the end of the full string; you can always split at / and take index 0 to get just the first directory name form what remains.

You can also use regular expression for the purpose.
const initString = '/digital/collection/music/bunch/of/other/stuff';
const result = initString.match(/\/digital\/collection\/([a-zA-Z]+)\//)[1];
console.log(result);
The console output is:
music

If you know the initial string, and you have the part before the string you seek, then the following snippet returns you the string you seek. You need not calculate indices, or anything like that.
// getting the last index of searchString
// we should get: music
const initString = '/digital/collection/music/bunch/of/other/stuff'
const firstPart = '/digital/collection/'
const lastIndexOf = (s1, s2) => {
return s1.replace(s2, '').split('/')[0]
}
console.log(lastIndexOf(initString, firstPart))

javascript - substitute into a string template, but catch missing variables

I want to carry out variable substitutions on a string (I've already ruled out template literals because the string has to be stored and evaluated later).
Mustache or something like it would seem like a contender, but I want to know if the substitution was incomplete. In this case, it's to produce urls, so missing parts mean invalid urls:
Testing this out in node:
var Mustache = require('mustache');
var result = Mustache.render("/someurl/{{path1}}/{{path2}}/", {path1:"my-path-to-1"})
console.log(`result:${result}:`)
This happens without a problem, but the resulting url is useless because Mustache silently replaced the missing path2 with an empty string. What I would like to see is an exception getting thrown (best) or failing that an easy way to recognize that not everything was substituted.
Note: the template string is arbitrary and so are the substitution object's contents.
output:
result:/someurl/my-path-to-1//:
This is the Python equivalent that I am looking for:
res = "/someurl/%(path1)s/%(path2)s/" % {"path1":"my-path-to-1"}
output:
KeyError: 'path2'

I ended up using sprintf, which has the benefit of having a different format than Mustache (or Django) so that you can embed it in data-url or the like:
const sprintf = require("sprintf-js").sprintf;
var o_substit = {
path1 : "mypath1"
};
var T_URL = "/someurl/%(path1)s/%(path2)s/";
console.log(`\nT_URL:${T_URL}, substitutions:`);
try {
console.dir(o_substit);
console.log("expecting a failure...")
var url = sprintf(T_URL, o_substit);
console.log(` url:${url}:`);
}
catch (e){
console.log(` exception:${e}`);
};
var o_substit = {
path1 : "mypath1"
,path2 : "mypath2"
};
console.log(`\nT_URL:${T_URL}, substitutions:`);
try{
console.dir(o_substit);
console.log("\nexpecting a success:")
var url = sprintf(T_URL, o_substit);
console.log(` url:${url}:`);
}
catch (e){
console.log(` exception:${e}`);
};
output:
T_URL:/someurl/%(path1)s/%(path2)s/, substitutions:
{ path1: 'mypath1' }
expecting a failure...
exception:Error: [sprintf] property 'path2' does not exist
T_URL:/someurl/%(path1)s/%(path2)s/, substitutions:
{ path1: 'mypath1', path2: 'mypath2' }
expecting a success:
url:/someurl/mypath1/mypath2/:

Node.js, parse filename from URL

How can I parse a url?
site.com:8080/someFile.txt?attr=100
or
site.com:8080/someFile.txt/?attr=100
I need to get someFile.txt, where is a file name I set by myself as the format (txt or some other).
UPDATE
I tried
var path = url.parse(req.url).path;
But I still cannot get the path (someFile.txt).

Something like this..
var url = require("url");
var path = require("path");
var parsed = url.parse("http://example.com:8080/test/someFile.txt/?attr=100");
console.log(path.basename(parsed.pathname));

here's my working code, hope it helps
import path from 'path'
const getBasenameFormUrl = (urlStr) => {
const url = new URL(urlStr)
return path.basename(url.pathname)
}

decodeURIComponent(path.basename('http://localhost/node-v18.12.1-x64.msi'))
node-v18.12.1-x64.msi

Your example can easily be dealt with using Node.js’s url module:
var URL = require('url').parse('site.com:8080/someFile.txt?attr=100');
console.log(URL.pathname.replace(/(^\/|\/$)/g,'')); // "someFile.txt"
However, this doesn’t work with Node.js’s exemplary URL (’cause it’s got more path).
By truncanting the complete path starting at its rightmost slash, it’ll yield the file name:
var URL = require('url').parse('http://user:pass#host.com:8080/p/a/t/h?query=string#hash');
console.log(URL.pathname.substring(URL.pathname.lastIndexOf('/')+1)); // "h"
And if that idea is considered safe enough for the appliance, we can do it plain:
var file = url.substring(url.lastIndexOf('/')+1).replace(/((\?|#).*)?$/,'');
/* hashes and query strings ----^ */