I'm getting the error code "403: Quota Error: User Rate Limit Exceeded" with batch requests to the Google Analytics Management API (v3) pointing to management views (profiles): patch.
I'm aware of quota limits from the docs, which suggest that I hit the write limit of 50 queries/day.
However, this only happens with batch requests. Individual calls like this:
gapi.client.analytics.management.profiles.patch({
"accountId": "someAccountId",
"webPropertyId": "some propertyID",
"profileId": "someProfileId",
"resource": {
"excludeQueryParameters" : "someTestValue"
}
})
.then(function(response) {
// Handle the results here (response.result has the parsed body).
console.log("Response", response);
},
function(err) { console.error("Execute error", err); });
});
still come through with a 200er code.
For the batch request, the first request added to the batch always succeeds, whereas all following ones throw the 403er.
The code for batch requests looks something like this:
function runQuery(someArray) {
var batch = gapi.client.newBatch();
var request = function (query) {
return gapi.client.request({
//For demonstration purposes only. Imagin "path" gets adapted to the individual API calls
"path" : "https://www.googleapis.com/analytics/v3/management/accounts/accountId/webproperties/webPropertyId/profiles/profileId",
"method" : "PATCH",
"body" : {
"excludeQueryParameters" : "someTestValue1"
}
});
}
//Add to Batch
someArray.forEach(function(el) {
batch.add(request(el))
});
//Execute Batch Request
batch
.then(function(response) {
console.log("Response", response);
},
function(err) { console.error("Execute error", err);
}
);
};
The full error message is this:
body: "{"error":{"errors":[{"domain":"global","reason":"userRateLimitExceeded","message":"Quota Error: User Rate Limit Exceeded."}],"code":403,"message":"Quota Error: User Rate Limit Exceeded."}}"
I'm guessing you are hitting the 1.5 qps write limit. Since you are sending more than 2 writes at a time in a batch. So the first write succeed then all other writes fails.
Related
I'm surely dumb, but I'm not able to figure out how to handle openai api rejected requests
( for the context, dall-e 2 is an image generator )
when user tries to generate forbidden images, my nodejs app just exits
async function start(arg) {
try{
// generate image
const response = openai.createImage({
prompt: arg,
n: 1,
size: "1024x1024",
});
// on success response
response.then(res =>{
console.log("ok");
})
response.catch(err =>{
console.log(err);
});
} catch(e){
console.log(e);
}
}
it gives me something like that on the exit :
data: {
error: {
code: null,
message: 'Your request was rejected as a result of our safety system. Your prompt may contain text that is not allowed by our safety system.',
param: null,
type: 'invalid_request_error'
}
}
tried using response.catch and try catch without success, the app just exits everytime
I at least want to ignore this error in the first place
in a second hand, I would like to console.log the given message (data.error.message)
I don't know what to do to by honest, don't even understand why try catch isn't working
With the details given, my guess would be that the Promise returned by getImages is being rejected. You could debug this a bit by adding some additional logs into your .catch callback and catch statement.
How to do this really depends on what you're trying to do with this api, the code as it's currently written would log something and exit no matter what happens.
There's a couple ways to handle this
Use your .catch to handle the error. Utilizing promise chainability you can get something like this
openai.createImage({
prompt: arg,
n: 1,
size: "1024x1024",
user: msg.author.id,
})
.catch((e) => {
if (e.data.error.message.includes('safety system')) {
return 'something'
}
console.error(e)
})
If you need the response object, the asnwer might be different. Looks like the openai package is built on axios and you can pass axios options into it. See https://axios-http.com/docs/handling_errors and the Request Options section of https://npmjs.com/package/openai
EDIT
I found my solution thanks to #JacksonChristoffersen
Basically I was getting http status 400
I just added request options from axios to validate http status smaller than 500
Here's the solution:
async function start(arg) {
try{
// generate image
const response = openai.createImage({
prompt: arg,
n: 1,
size: "1024x1024",
},{
validateStatus: function (status) {
return status < 500; // Resolve only if the status code is less than 500
}
});
// on success response
response.then(res =>{
console.log("ok");
})
response.catch(err =>{
console.log(err);
});
} catch(e){
console.log(e);
}
}
I'm using Cloudflare service workers and I want on every request to:
request only the HTML (therefore count as only 1 request)
search the response for a string
Purge that page's cache if the message exists
I've solved points #2 and #3. Can't figure out if #1 is feasible or possible at all.
I need it as only one request because there is a limit per day on the number of free requests. Otherwise I have about 50-60 requests per page.
My current attempt for #1, which doesn't work right:
async function handleRequest(request) {
const init = {
headers: {
'content-type': 'text/html;charset=UTF-8',
},
};
const response = await fetch(request);
await fetch(request.url, init).then(function(response) {
response.text().then(function(text) {
console.log(text);
})
}).catch(function(err) {
// There was an error
console.warn('Something went wrong.', err);
});
return response;
}
addEventListener('fetch', event => {
return event.respondWith(handleRequest(event.request))
});
You can't request "only the html", the worker will act on any request that matches the route that it is deployed at. If you only care about the html, you will need to set up your worker path to filter to only the endpoints that you want to run the worker on.
Alternatively you can use the worker on every request and only do your logic if the response Content-Type is one that you care about. This would be something along these lines:
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request));
})
async function handleRequest(request) {
let response = await fetch(request);
let type = response.headers.get("Content-Type") || "";
if (type.startsWith("text/")) {
//this is where your custom logic goes
}
return response;
}
I am building an express server to receive request (a dict with 10 items) from my react front end and then save the data to database. Below is my code. I found that my code is work and the query does save the record back to Db. But in each for loop, this error is returned in server. What cause this error and the MaxListenersExceededWarning?
The request data:
{{.....}, {.....}, {.....}, {.....}, {.....}} #10 item
Code:
connection.js:
const p = mysql.createPool({
"connectionLimit" : 100,
"host": "example.org",
"user": "test",
"password": "test",
"database": "test",
"multipleStatements": true
});
const getConnection = function(callback) {
p.getConnection(function(err, connection) {
callback(err, connection)
})
};
module.exports = getConnection
routers.js
router.post('/test', (req, res) => {
getConnection(function(err, conn){
if (err) {
return res.json({ success: false, error: err })
} else {
const dict = req.body;
Object.keys(dict).forEach(function(r){
#putting dict's value to query
query = "UPDATE ......;"
conn.query(query, function (err, result, fields) {
conn.release()
console.log(query)
if (err) {
console.log("err")
return res.json({ success: false, error: err });
}
});
});
}
});
return res.json({ success: true });
});
Error:
error: uncaughtException: Connection already released
Error: Connection already released
at Pool.releaseConnection (/home/node_modules/mysql/lib/Pool.js:138:13)
at PoolConnection.release (/home/node_modules/mysql/lib/PoolConnection.js:35:15)
at Query.<anonymous> (/home/routes/test.js:276:22)
at Query.<anonymous> (/home/node_modules/mysql/lib/Connection.js:526:10)
at Query._callback (/home/node_modules/mysql/lib/Connection.js:488:16)
at Query.Sequence.end (/home/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
at Query._handleFinalResultPacket (/home//node_modules/mysql/lib/protocol/sequences/Query.js:149:8)
at Query.OkPacket (/home//node_modules/mysql/lib/protocol/sequences/Query.js:74:10)
at Protocol._parsePacket (/home//node_modules/mysql/lib/protocol/Protocol.js:291:23)
at Parser._parsePacket (/home//node_modules/mysql/lib/protocol/Parser.js:433:10)
(node:15881) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit
One connection (conn) is being retrieved from the pool, and is used to launch 10 queries in the forEach loop.
When the first query finishes to run, the first step of its callback is: conn.release(). The connection is released.
When the second query finishes to run, its callback also tries to release the connection,causing the error.
This problem might be solved in multiple ways:
Solve using a counter
In the callback of the database query, before calling call.release, check the number of queries already processed, and only close the connection when the last product is being processed.
const dict = req.body;
// initialize counter
let itemCount = 0
, errors = []
Object.keys(dict).forEach(function(r){
#putting dict's value to query
query = "UPDATE ......;"
conn.query(query, function (err, result, fields) {
// check whether this is the last callback
if (itemCount === dict.length-1) {
conn.release()
let result = errors.length ? { success: false, error: errors } : { success: true }
res.json(result)
}
// increment counter
itemCount++
console.log(query)
if (err) {
console.log("err")
errors.push(err)
}
});
});
Edit: There is also an issue with the res.json calls: inside the code in the question, res.json({ success: true }) is always executed, without waiting for the queries' execution results. The modified code sample above calls res.json only once after the execution of all queries, this is the only place where res.json should be called. This implies modifying the client-side code so that it can handle an array of errors, rather than only one error.
Solve by using a recursive function instead of for loop.
It is not a good practice to use for loops for the execution of asynchronous code. You might run into Maximum call stack size exceeded errors whenever the data volume gets too large.
Instead, create a recursive function (e.g. updateDictItem) to process one update query at a time. Read more about the asynchronous patterns in node.js in this article.
Other possible enhancements
Rather than firing ten database queries, it is worth considering grouping all the updates in one MERGE update statement, otherwise doing all the updates in a TRANSACTION.
My call to Express's 'res.send(string)' fails to redirect and display the provided text upon completion of a call to MongoDB's 'deleteMany()'. The database is cleared and no error is thrown.
I've tried adjusting my call to 'res' with 'res.json()' and such, but to no avail. I've also adjusted the ordering my other calls within the 'delete' request, with no success. My inclination is that my issue is related to Promises.
.delete(function(req, res){
//if successful response will be 'complete delete successful'
console.log('deleting all documents');
MongoClient.connect(MONGODB_CONNECTION_STRING, { useNewUrlParser: true }, (connectErr, client) => {
if(connectErr) res.json({ "error": "Error connecting to database!", "error": connectErr });
const db = client.db('test-db');
try {
db.collection('testCollection2').deleteMany({}, (err) => {
if(err) throw err;
res.send('complete delete successful');
console.log('complete delete successful');
});
} catch(err) {
console.log('Complete delete failed!');
res.send('Complete delete failed!');
}
});
});
Despite not redirecting or receiving an error message, I still receive the console.log output confirming the successful call to 'deleteMany'. I'm not sure how to test this in more depth, since I'm using Glitch for the project. Thanks in advance for any help!
I am trying to post a video using Twitter modules in NodeJS but I am getting an error - Error 202 “Your credentials do not allow access to this resource”.
Here a snippet of code, it is missing the APPEND and FINISH calls etc but would have expected not to receive an error 202?
Has anyone managed to upload a video using JavaScript than can give me some guidance? I have tried to research and tried several different options.
T.post('media/upload', {
media: data,
encoding: 'base64',
command: 'INIT',
total_bytes: getFilesizeInBytes(image)
}, function(error, media, response) {
if (!error) {
var status = {
status: message
};
if(media) status['media_ids'] = media.media_id_string;
T.post('statuses/update', status, function(error, tweet, response) {
if(!error) console.log(' -- message sent');
else console.log(error);
});
} else {
console.log(error);
}
});