Develop Reference

26 digits of number in javascript/typescript

I have an interesting problem in my application. I create a banking application and store a 26 digit account number in the database.
In the MySQL database, I use the decimal(26,0) data type but I need this number to be generated in my NodeJS application from typescript.
I've come up with this solution, but it does not generate a 26 digit account number:
/**
* Generate unique account bill
*/
async getAccountBill(): Promise<Bill | number> {
const accountBill = (
Math.floor(Math.random() * 90000000000000000000) +
10000000000000000000 * 100000
);
console.log(accountBill); // 1.0000387421390274e+24
console.log(typeof accountBill); // number
/**
* Checking if the generated account number already exists in the database
*/
const isAccountBill = await this.getByAccountBill(accountBill);
return isAccountBill ? await this.getAccountBill() : accountBill;
}
I can also change the type to string, but it does not match my database model, so I'd rather return a number type.

let random261 = Math.floor(Math.random() * 10000000000000);
let random262 = Math.floor(Math.random() * 10000000000000);
let rand26 = `${random261}${random262}`;
//Then in mysql you can do this
CAST(rand26 as DECIMAL(26,0));
Not tested though.

In JavaScript Google Flatbuffers, how does one write a ulong?

I have no problem with doing this in a C++ program, but I am stuck on writing a ulong in JS.
FB has no issue if I used the 32bit process.hrtime() value.
But how does do a createLong() for a 64bit ?
see: [ https://nodejs.org/api/process.html#process_process_hrtime_bigint ]
# commented line does not work
# let timeStamp = process.hrtime.bigint()
let timeStamp = process.hrtime()
let ts = builder.createLong(0, timeStamp)
PNT.Telemetry.startTelemetry(builder)
PNT.Telemetry.addSystemTime(builder, ts)
FB template file
// Simple Telemetry data from/to Sim and Sensor
namespace PNT;
enum DeviceType:byte { IMU, VAN, GPS, MAGNAV, SOOP }
struct PosVector {
lat:double;
lon:double;
alt:double;
}
table Telemetry {
source: string;
systemTime:ulong = 0;
systemTimeString: string;
description: string;
position: PosVector;
}
root_type Telemetry;

You can use this function, it works well for timestamp
var flatBufferTimeStamp = function(value) {
var bin = (value).toString(2);
var pad = new Array(64 - bin.length + 1 ).join('0');
bin = pad + bin;
return {
low: parseInt(bin.substring(32), 2),
high: parseInt(bin.substring(0, 32), 2)
};
}
var timeStamp = flatBufferTimeStamp(process.hrtime())
let ts = builder.createLong(timeStamp.low, timeStamp.high);
PNT.Telemetry.startTelemetry(builder)
PNT.Telemetry.addSystemTime(builder, ts)
Source: https://groups.google.com/forum/#!topic/flatbuffers/ieXNEsB_2wc

Use flatbuffers.Long, here is an example: https://github.com/google/flatbuffers/blob/master/tests/JavaScriptTest.js#L160
Sadly there is no code in FlatBuffers to convert a bigint to the 2 32-bit values, which would be good to add. For now you may need to do your own conversion.

Query Instagram posts by hashtag and time range

I'm trying to query posts from Instagram by providing the hashtag and the time range (since and until dates).
I use the recent tags endpoint.
https://api.instagram.com/v1/tags/{tag-name}/media/recent?access_token=ACCESS-TOKEN
My code is written in Node.js using the instagram-node library (see the inline comments):
// Require the config file
var config = require('../config.js');
// Require and intialize the instagram instance
var ig = require('instagram-node').instagram();
// Set the access token
ig.use({ access_token: config.instagram.access_token });
// We export this function for public use
// hashtag: the hashtag to search for
// minDate: the since date
// maxDate: the until date
// callback: the callback function (err, posts)
module.exports = function (hashtag, minDate, maxDate, callback) {
// Create the posts array (will be concated with new posts from pagination responses)
var posts = [];
// Convert the date objects into timestamps (seconds)
var sinceTime = Math.floor(minDate.getTime() / 1000);
var untilTime = Math.floor(maxDate.getTime() / 1000);
// Fetch the IG posts page by page
ig.tag_media_recent(hashtag, { count: 50 }, function fetchPosts(err, medias, pagination, remaining, limit) {
// Handle error
if (err) {
return callback(err);
}
// Manually filter by time
var filteredByTime = medias.filter(function (currentPost) {
// Convert the created_time string into number (seconds timestamp)
var createdTime = +currentPost.created_time;
// Check if it's after since date and before until date
return createdTime >= sinceTime && createdTime <= untilTime;
});
// Get the last post on this page
var lastPost = medias[medias.length - 1] || {};
// ...and its timestamp
var lastPostTimeStamp = +(lastPost.created_time || -1);
// ...and its timestamp date object
var lastPostDate = new Date(lastPostTimeStamp * 1000);
// Concat the new [filtered] posts to the big array
posts = posts.concat(filteredByTime);
// Show some output
console.log('found ' + filteredByTime.length + ' new items total: ' + posts.length, lastPostDate);
// Check if the last post is BEFORE until date and there are no new posts in the provided range
if (filteredByTime.length === 0 && lastPostTimeStamp <= untilTime) {
// ...if so, we can callback!
return callback(null, posts);
}
// Navigate to the next page
pagination.next(fetchPosts);
});
};
This will start fetching the posts with the most recent to least recent ones, and manually filter the created_time.
This works, but it's very very inefficient because if we want, for example, to get the posts from one year ago, we have to iterate the pages until that time, and this will use a lot of requests (probably more than 5k / hour which is the rate limit).
Is there a better way to make this query? How to get the Instagram posts by providing the hashtag and the time range?

I think this is the basic idea you're looking for. I'm not overly familiar with Node.js, so this is all in plain javascript. You'll have to modify it to suit your needs and probably make a function out of it.
The idea is to convert an instagram id (1116307519311125603 in this example) to a date and visa versa to enable you to quickly grab a specific point in time rather then backtrack through all results until finding your desired timestamp. The portion of the id after the underscore '_' should be trimmed off as that refers, in some way, to the user IIRC. There are 4 functions in the example that I hope will help you out.
Happy hacking!
//static
var epoch_hour = 3600,
epoch_day = 86400,
epoch_month = 2592000,
epoch_year = 31557600;
//you'll need to set this part up/integrate it with your code
var dataId = 1116307519311125603,
range = 2 * epoch_hour,
count = 1,
tagName = 'cars',
access = prompt('Enter access token:'),
baseUrl = 'https://api.instagram.com/v1/tags/' +
tagName + '/media/recent?access_token=' + access;
//date && id utilities
function idToEpoch(n){
return Math.round((n / 1000000000000 + 11024476.5839159095) / 0.008388608);
}
function epochToId(n){
return Math.round((n * 0.008388608 - 11024476.5839159095) * 1000000000000);
}
function newDateFromEpoch(n){
var d = new Date(0);
d.setUTCSeconds(n);
return d;
}
function dateToEpoch(d){
return (d.getTime()-d.getMilliseconds())/1000;
}
//start with your id and range; do the figuring
var epoch_time = idToEpoch(dataId),
minumumId = epochToId(epoch_time),
maximumId = epochToId(epoch_time + range),
minDate = newDateFromEpoch(epoch_time),
maxDate = newDateFromEpoch(epoch_time + range);
var newUrl = baseUrl +
'&count=' + count +
'&min_tag_id=' + minumumId +
'&max_tag_id=' + maximumId;
//used for testing
/*alert('Start: ' + minDate + ' (' + epoch_time +
')\nEnd: ' + maxDate + ' (' + (epoch_time +
range) + ')');
window.location = newUrl;*/

To support this excellent answer, an instagram ID is generated via the plpgSQL function:
CREATE OR REPLACE FUNCTION insta5.next_id(OUT result bigint) AS $$
DECLARE
our_epoch bigint := 1314220021721;
seq_id bigint;
now_millis bigint;
shard_id int := 5;
BEGIN
SELECT nextval('insta5.table_id_seq') %% 1024 INTO seq_id;
SELECT FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 1000) INTO now_millis;
result := (now_millis - our_epoch) << 23;
result := result | (shard_id << 10);
result := result | (seq_id);
END;
$$ LANGUAGE PLPGSQL;
from Instagram's blog

Despite a similar getting posts process, Data365.co Instagram API, I currently working at, seems to be more suitable and efficient. It does not have a limit of 5,000 posts per hour, and you can specify the period of time for which your need posts in the request itself. Also, the billing will be taken into account only posts from the indicated period. You won't have to pay for data you don't need.
You can see below a task example to download posts by the hashtag bitcoins for the period from January 1, 2021, to January 10, 2021.
POST request: https://api.data365.co/v1.1/instagram/tag/bitcoins/update?max_posts_count=1000&from_date=2021-01-01&to_date=2021-01-10&access_token=TOKEN
A GET request example to get the corresponding list of posts:
https://api.data365.co/v1.1/instagram/tag/bitcoins/posts?from_date=2021-01-01&to_date=2021-01-10&max_page_size=100&order_by=date_desc&access_token=TOKEN
More detailed info view in API documentation at https://api.data365.co/v1.1/instagram/docs#tag/Instagram-hashtag-search

Convert ObjectID (Mongodb) to String in JavaScript

I want to convert ObjectID (Mongodb) to String in JavaScript.
When I get a Object form MongoDB. it like as a object has: timestamp, second, inc, machine.
I can't convert to string.

Try this:
objectId.str
See the doc.
ObjectId() has the following attribute and methods:
[...]
str - Returns the hexadecimal string representation of the object.

in the shell
ObjectId("507f191e810c19729de860ea").str
in js using the native driver for node
objectId.toHexString()

Here is a working example of converting the ObjectId in to a string
> a=db.dfgfdgdfg.findOne()
{ "_id" : ObjectId("518cbb1389da79d3a25453f9"), "d" : 1 }
> a['_id']
ObjectId("518cbb1389da79d3a25453f9")
> a['_id'].toString // This line shows you what the prototype does
function () {
return "ObjectId(" + tojson(this.str) + ")";
}
> a['_id'].str // Access the property directly
518cbb1389da79d3a25453f9
> a['_id'].toString()
ObjectId("518cbb1389da79d3a25453f9") // Shows the object syntax in string form
> ""+a['_id']
518cbb1389da79d3a25453f9 // Gives the hex string
Did try various other functions like toHexString() with no success.

You can use $toString aggregation introduced in mongodb version 4.0 which converts the ObjectId to string
db.collection.aggregate([
{ "$project": {
"_id": { "$toString": "$your_objectId_field" }
}}
])

Use toString:
var stringId = objectId.toString()
Works with the latest Node MongoDB Native driver (v3.0+):
http://mongodb.github.io/node-mongodb-native/3.0/

Acturally, you can try this:
> a['_id']
ObjectId("518cbb1389da79d3a25453f9")
> a['_id'] + ''
"518cbb1389da79d3a25453f9"
ObjectId object + String will convert to String object.

If someone use in Meteorjs, can try:
In server: ObjectId(507f191e810c19729de860ea)._str.
In template: {{ collectionItem._id._str }}.

Assuming the OP wants to get the hexadecimal string value of the ObjectId, using Mongo 2.2 or above, the valueOf() method returns the representation of the object as a hexadecimal string. This is also achieved with the str property.
The link on anubiskong's post gives all the details, the danger here is to use a technique which has changed from older versions e.g. toString().

In Javascript, String() make it easy
const id = String(ObjectID)

this works, You have mongodb object: ObjectId(507f191e810c19729de860ea),
to get string value of _id, you just say
ObjectId(507f191e810c19729de860ea).valueOf();

In Js do simply: _id.toString()
For example:
const myMongoDbObjId = ObjectID('someId');
const strId = myMongoDbObjId.toString();
console.log(typeof strId); // string

You can use string formatting.
const stringId = `${objectId}`;

toString() method gives you hex String which is kind of ascii code but in base 16 number system.
Converts the id into a 24 character hex string for printing
For example in this system:
"a" -> 61
"b" -> 62
"c" -> 63
So if you pass "abc..." to get objectId you will get "616263...".
As a result if you want to get readable string(char string) from objectId you have to convert it(hexCode to char).
To do this I wrote an utility function hexStringToCharString()
function hexStringToCharString(hexString) {
const hexCodeArray = [];
for (let i = 0; i < hexString.length - 1; i += 2) {
hexCodeArray.push(hexString.slice(i, i + 2));
}
const decimalCodeArray = hexCodeArray.map((hex) => parseInt(hex, 16));
return String.fromCharCode(...decimalCodeArray);
}
and there is usage of the function
import { ObjectId } from "mongodb";
const myId = "user-0000001"; // must contains 12 character for "mongodb": 4.3.0
const myObjectId = new ObjectId(myId); // create ObjectId from string
console.log(myObjectId.toString()); // hex string >> 757365722d30303030303031
console.log(myObjectId.toHexString()); // hex string >> 757365722d30303030303031
const convertedFromToHexString = hexStringToCharString(
myObjectId.toHexString(),
);
const convertedFromToString = hexStringToCharString(myObjectId.toString());
console.log(`convertedFromToHexString:`, convertedFromToHexString);
//convertedFromToHexString: user-0000001
console.log(`convertedFromToString:`, convertedFromToString);
//convertedFromToHexString: user-0000001
And there is also TypeScript version of hexStringToCharString() function
function hexStringToCharString(hexString: string): string {
const hexCodeArray: string[] = [];
for (let i = 0; i < hexString.length - 1; i += 2) {
hexCodeArray.push(hexString.slice(i, i + 2));
}
const decimalCodeArray: number[] = hexCodeArray.map((hex) =>
parseInt(hex, 16),
);
return String.fromCharCode(...decimalCodeArray);
}

Just use this : _id.$oid
And you get the ObjectId string. This come with the object.

Found this really funny but it worked for me:
db.my_collection.find({}).forEach((elm)=>{
let value = new String(elm.USERid);//gets the string version of the ObjectId which in turn changes the datatype to a string.
let result = value.split("(")[1].split(")")[0].replace(/^"(.*)"$/, '$1');//this removes the objectid completely and the quote
delete elm["USERid"]
elm.USERid = result
db.my_collection.save(elm)
})

On aggregation use $addFields
$addFields: {
convertedZipCode: { $toString: "$zipcode" }
}

Documentation of v4 (right now it's latest version) MongoDB NodeJS Driver says: Method toHexString() of ObjectId returns the ObjectId id as a 24 character hex string representation.

In Mongoose, you can use toString() method on ObjectId to get a 24-character hexadecimal string.
Mongoose documentation

Below three methods can be used to get the string version of id.
(Here newUser is an object containing the data to be stored in the mongodb document)
newUser.save((err, result) => {
if (err) console.log(err)
else {
console.log(result._id.toString()) //Output - 23f89k46546546453bf91
console.log(String(result._id)) //Output - 23f89k46546546453bf91
console.log(result._id+"") //Output - 23f89k46546546453bf91
}
});

Use this simple trick, your-object.$id
I am getting an array of mongo Ids, here is what I did.
jquery:
...
success: function (res) {
console.log('without json res',res);
//without json res {"success":true,"message":" Record updated.","content":[{"$id":"58f47254b06b24004338ffba"},{"$id":"58f47254b06b24004338ffbb"}],"dbResponse":"ok"}
var obj = $.parseJSON(res);
if(obj.content !==null){
$.each(obj.content, function(i,v){
console.log('Id==>', v.$id);
});
}
...

You could use String
String(a['_id'])

If you're using Mongoose along with MongoDB, it has a built-in method for getting the string value of the ObjectID. I used it successfully to do an if statement that used === to compare strings.
From the documentation:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.

Is there any way to create mongodb like _id strings without mongodb?

I really like the format of the _ids generated by mongodb. Mostly because I can pull data like the date out of them client side. I'm planning to use another database but still want that type of _id for my document. How can I create these ids without using mongodb?
Thanks!

A very easy pseudo ObjectId generator in javascript:
const ObjectId = (m = Math, d = Date, h = 16, s = s => m.floor(s).toString(h)) =>
s(d.now() / 1000) + ' '.repeat(h).replace(/./g, () => s(m.random() * h))

Use the official MongoDB BSON lib in the client
I have a browser client that generates ObjectIds. I wanted to make sure that I employ the same ObjectId algorithm in the client as the one used in the server. MongoDB has js-bson which can be used to accomplish that.
If you are using javascript with node.
npm install --save bson
Using require statement
var ObjectID = require('bson').ObjectID;
var id = new ObjectID();
console.log(id.toString());
Using ES6 import statement
import { ObjectID } from 'bson';
const id = new ObjectID();
console.log(id.toString());
The library also lets you import using good old script tags but I have not tried this.

Object IDs are usually generated by the client, so any MongoDB driver would have code to generate them.
If you're looking for JavaScript, here's some code from the MongoDB Node.js driver:
https://github.com/mongodb/js-bson/blob/1.0-branch/lib/bson/objectid.js
And another, simpler solution:
https://github.com/justaprogrammer/ObjectId.js

Extending Rubin Stolk's and ChrisV's answer in a more readable syntax (KISS).
function objectId () {
return hex(Date.now() / 1000) +
' '.repeat(16).replace(/./g, () => hex(Math.random() * 16))
}
function hex (value) {
return Math.floor(value).toString(16)
}
export default objectId

ruben-stolk's answer is great, but deliberately opaque? Very slightly easier to pick apart is:
const ObjectId = (rnd = r16 => Math.floor(r16).toString(16)) =>
rnd(Date.now()/1000) + ' '.repeat(16).replace(/./g, () => rnd(Math.random()*16));
(actually in slightly fewer characters). Kudos though!

This is a simple function to generate a new objectId
newObjectId() {
const timestamp = Math.floor(new Date().getTime() / 1000).toString(16);
const objectId = timestamp + 'xxxxxxxxxxxxxxxx'.replace(/[x]/g, () => {
return Math.floor(Math.random() * 16).toString(16);
}).toLowerCase();
return objectId;
}

Here's a link! to a library to do that.
https://www.npmjs.com/package/mongo-object-reader
You can read and write hexadecimal strings.
const { createObjectID, readObjectID,isValidObjectID } = require('mongo-object-reader');
//Creates a new immutable `ObjectID` instance based on the current system time.
const ObjectID = createObjectID() //a valid 24 character `ObjectID` hex string.
//returns boolean
// input - a valid 24 character `ObjectID` hex string.
const isValid = isValidObjectID(ObjectID)
//returns an object with data
// input - a valid 24 character `ObjectID` hex string.
const objectData = readObjectID(ObjectID)
console.log(ObjectID) //ObjectID
console.log(isValid) // true
console.log(objectData) /*
{ ObjectID: '5e92d4be2ced3f58d92187f5',
timeStamp:
{ hex: '5e92d4be',
value: 1586681022,
createDate: 1970-01-19T08:44:41.022Z },
random: { hex: '2ced3f58d9', value: 192958912729 },
incrementValue: { hex: '2187f5', value: 2197493 } }
*/

There is a detailed specification here
http://www.mongodb.org/display/DOCS/Object+IDs
Which you can use to roll your own id strings