Develop Reference

Run a persistent web worker in the background without getting throttled

I would like to run a web worker in the background (on mobile, it works on desktop) which does an action every minute by sending an HTTP request. Unfortunately, after having the website in the background for about 5 minutes, no HTTP request will be sent anymore... All network requests will resume once you go back to the website.
It seems like the web worker is not allowed to be run in the background, at least not on mobile, as it works fine on desktop. I could not find any source stating why it is throttled, and what can be done about it. I found a similar issue dating from 2017, which is 4 years ago and has not been answered quite the way I want it and might also be outdated.
I thought of the following solutions:
Normally in native Android you can show a notification which will ensure that your app keeps running in the background, I was wondering if this would also work on web in combination with web workers.
If you open a web socket in your web worker, will it keep your web worker alive in the background?
Progressive web app. Does not seem to keep it active unfortunately..
Is there anything I could do about this?

All modern browsers restrict background usage. It has a really simple reason: Background tasks require resources, and users dont want a million websites in the background to eat all your RAM. And malicious websites could just use the CPU from users for bitcoin mining etc.
But there is a way to do stuff in the background. You already mentioned it in the question: You need to send push notifications. You can just include a fetch() in your push notification handler.
But here's the catch: You have to send a notification every time you want your site to fetch something, or your requests will always/sometimes be blocked depending on your browser. From MDN:
Activating a service worker to deliver a push message can result in increased resource usage, particularly of the battery. Different browsers have different schemes for handling this, there is currently no standard mechanism. Firefox allows a limited number (quota) of push messages to be sent to an application, although Push messages that generate notifications are exempt from this limit. The limit is refreshed each time the site is visited. In comparison, Chrome applies no limit, but requires that every push message causes a notification to be displayed.
You just need to set up Push Notifications. There's a great guide from Google that you can follow if you don't know how to set up push notifications.
An implementation in a service worker would look like this:
self.addEventListener('push', function(event) {
if (!(self.Notification && self.Notification.permission === 'granted')) {
return;
}
// The HTTP request
fetch("...");
var data = {};
if (event.data) {
data = event.data.json();
}
var title = data.title || "Background activity";
var message = data.message || "You can ignore this";
var icon = "images/new-notification.png";
var notification = new Notification(title, {
body: message,
tag: 'simple-push-demo-notification',
icon: icon
});
notification.addEventListener('click', function() {
if (clients.openWindow) {
clients.openWindow('https://example.blog.com/2015/03/04/something-new.html');
}
});
});
(Code copied from MDN)

Jumping over to another intent by using the 'agent.setFollowupEvent(targetIntentEventName)' method in the DialogFlow Messenger fulfillment

I'm trying to find a solution to launch another DialogFlow Intent (via its event) from a webhook (node.js) server after collecting users email address and checking their registration status by making a POST API call to a remote organization server.
Here's a basic schema:
User: Hello!
Chatbot: Hello, I'm an assistance bot. Can I have your email address to verify your registration status, please?
User: funnyname#email.com
Chatbot: Thank you, let me check your status ...
---> Fulfilment code makes a POST API call and gets users status back from a server if there's a match.
Here based on the users status I want to launch a new intent: either Registered_Users_Intent or New_Users_Intent. I tried to use the agent.setFollowupEvent(({ "name": "targetIntentEventName" }) method, but it doesn't trigger a desired event[intent] all the time. I'm not sure how to set it up properly in the fulfilment code to make it work all the time.
Would anyone have any suggestions or reference materials to make this set-up work, please?

When you make a webhook fulfillment request, your API should respond with a JSON response that Dialogflow understands. Instead of calling dialogflow APIs to trigger an event from within your webhook API, the JSON response of the API can contain the name of the event to be triggered next and Dialogflow will do the rest.
Here's the reference for the fulfillment response to trigger a follow-up event: https://cloud.google.com/dialogflow/es/docs/fulfillment-webhook#event
Also, the most likely reason that your current approach works sometimes and doesn't the others, is because triggering an event using dialogflowv2 APIs AND sending back a JSON response (which probably doesn't have the name of the follow up event mentioned) creates a race condition.

Since you are using the Dialogflow Fulfillment Library, make sure to use the latest version in your package.json file. Also, when using the agent.setFollowupEvent() method, make sure to include an agent.add() method for the response. This response will not be returned but is a requirement to be able to use the agent.setFollowupEvent() method.
Here’s an example:
function nameOfFunction(agent) {
agent.add(`sample response`);
agent.setFollowupEvent('event_name');
}
For more information on Dialogflow Fulfillment Library, see here. Note that this library is not maintained.
Otherwise, you may create your own code and host it on your chosen web server. From there, you can invoke events by setting the followupEventInput field of the WebhookResponse. Check here for more information.

How to notify without websocket

I want to integrate a simple notification system in my react application. I want to notify for example:
- new post (when the user post the system need time to transcode the media attached and the publication)
- missing settings (the user need to compile some information)
- interesting posts etc..
There is a simple way to add a websocket, like socket.io, to a reactjs app with an aws lambda backend?
All the notification not need to be read in real time, maybe an ajax call every 2 minutes can solve my problem, but, in this case, someone can help me avoid ajax call if the app isn't used(like if the app remain opened in a foreground tab...)
componentDidMount() {
this.liveUpdate()
setInterval(this.liveUpdate, 120000);
}
liveUpdate() {
axios.get(endpoint.posts+'/live/', cfg)
.then(res => {
// ...
});
}
This code is in the footer component, the call happen every 120 seconds, but the call will still happen also if a user leave the application opened in the browser and not use it, this on a lambda backend mean a waste of money.

There are 3 main ways of notifying that I can think of at the moment...
Long polling (using ajax etc)
Websocket
Push Notification
Push (though) requires permission from the user

Publish data from browser app without writing my own server

I need users to be able to post data from a single page browser application (SPA) to me, but I can't put server-side code on the host.
Is there a web service that I can use for this? I looked at Amazon SQS (simple queue service) but I can't call their REST APIs from within the browser due to cross origin policy.
I favour ease of development over robustness right now, so even just receiving an email would be fine. I'm not sure that the site is even going to catch on. If it does, then I'll develop a server-side component and move hosts.

Not only there are Web Services, but nowadays there are robust systems that provide a way to server-side some logic on your applications. They are called BaaS or Backend as a Service providers, usually to provide some backbone to your front end applications.
Although they have multiple uses, I'm going to list the most common in my opinion:
For mobile applications - Instead of having to learn an API for each device you code to, you can use an standard platform to store logic and data for your application.
For prototyping - If you want to create a slick application, but you don't want to code all the backend logic for the data -less dealing with all the operations and system administration that represents-, through a BaaS provider you only need good Front End skills to code the simplest CRUD applications you can imagine. Some BaaS even allow you to bind some Reduce algorithms to calls your perform to their API.
For web applications - When PaaS (Platform as a Service) came to town to ease the job for Backend End developers in order to avoid the hassle of System Administration and Operations, it was just logic that the same was going to happen to the Backend. There are many clones that showcase the real power of this strategy.
All of this is amazing, but I have yet to mention any of them. I'm going to list the ones that I know the most and have actually used in projects. There are probably many, but as far as I know, this one have satisfied most of my news, whether it's any of the previously ones mentioned.
Parse.com
Parse's most outstanding features target mobile devices; however, nowadays Parse contains an incredible amount of API's that allows you to use it as full feature backend service for Javascript, Android and even Windows 8 applications (Windows 8 SDK was introduced a few months ago this year).
How does a Parse code looks in Javascript?
Parse works through classes and objects (ain't that beautiful?), so you first create a specific class (can be done through Javascript, REST or even the Data Browser manager) and then you add objects to specific classes.
First, add up Parse as a script tag in javascript:
<script type="text/javascript" src="http://www.parsecdn.com/js/parse-1.1.15.min.js"></script>
Then, through a given Application ID and a Javascript Key, initialize Parse.
Parse.initialize("APPLICATION_ID", "JAVASCRIPT_KEY");
From there, it's all object manipulation
var Person = Parse.Object.extend("Person"); //Person is a class *cof* uppercase *cof*
var personObject = new Person();
personObject.save({name: "John"}, {
success: function(object) {
console.log("The object with the data "+ JSON.stringify(object) + " was saved successfully.");
},
error: function(model, error) {
console.log("There was an error! The following model and error object were provided by the Server");
console.log(model);
console.log(error);
}
});
What about authentication and security?
Parse has a User based authentication system, which pretty much allows you to store a base of users that can manipulate the data. If map the data with User information, you can ensure that only a given user can manipulate specific data. Plus, in the settings of your Parse application, you can specify that no clients are allowed to create classes, to ensure innecesary calls are performed.
Did you REALLY used in a web application?
Yes, it was my tool of choice for a medium fidelity prototype.
Firebase.com
Firebase's main feature is the ability to provide Real Time to your application without all the hassle. You don't need a MeteorJS server in order to bring Push Notifications to your software. If you know Javascript, you are half way through to bring Real Time magic to your users.
How does a Firebase looks in Javascript?
Firebase works in a REST fashion, and I think they do an amazing job structuring the Glory of REST. As a good example, look at the following Resource structure in Firebase:
https://SampleChat.firebaseIO-demo.com/users/fred/name/first
You don't need to be a rocket scientist to know that you are retrieve the first name of the user "Fred", giving there's at least one -usually there should be a UUID instead of a name, but hey, it's an example, give me a break-.
In order to start using Firebase, as with Parse, add up their CDN Javascript
<script type='text/javascript' src='https://cdn.firebase.com/v0/firebase.js'></script>
Now, create a reference object that will allow you to consume the Firebase API
var myRootRef = new Firebase('https://myprojectname.firebaseIO-demo.com/');
From there, you can create a bunch of neat applications.
var USERS_LOCATION = 'https://SampleChat.firebaseIO-demo.com/users';
var userId = "Fred"; // Username
var usersRef = new Firebase(USERS_LOCATION);
usersRef.child(userId).once('value', function(snapshot) {
var exists = (snapshot.val() !== null);
if (exists) {
console.log("Username "+userId+" is part of our database");
} else {
console.log("We have no register of the username "+userId);
}
});
What about authentication and security?
You are in luck! Firebase released their Security API about two weeks ago! I have yet to explore it, but I'm sure it fills most of the gaps that allowed random people to use your reference to their own purpose.
Did you REALLY used in a web application?
Eeehm... ok, no. I used it in a Chrome Extension! It's still in process but it's going to be a Real Time chat inside a Chrome Extension. Ain't that cool? Fine. I find it cool. Anyway, you can browse more awesome examples for Firebase in their examples page.
What's the magic of these services? If you read your Dependency Injection and Mock Object Testing, at some point you can completely replace all of those services for your own through a REST Web Service provider.
Since these services were created to be used inside any application, they are CORS ready. As stated before, I have successfully used both of them from multiple domains without any issue (I'm even trying to use Firebase in a Chrome Extension, and I'm sure I will succeed soon).
Both Parse and Firebase have Data Browser managers, which means that you can see the data you are manipulating through a simple web browser. As a final disclaimer, I have no relationship with any of those services other than the face that James Taplin (Firebase Co-founder) was amazing enough to lend me some Beta access to Firebase.

You actually CAN use SQS from the browser, even without CORS, as long as you only need the browser to send messages, not receive them. Warning: this is a kludge that would make my CS professors cry.
When you perform a GET request via javascript, the browser will always perform the request, however, you'll only get access to the response if it was from the same origin (protocol, host, port). This is your ticket to ride, since messages can be posted to an SQS queue with just a GET, and who really cares about the response anyways?
Assuming you're using jquery, your queue is https://sqs.us-east-1.amazonaws.com/71717171/myqueue, and allows anyone to post a message, the following will post a message with the body "HITHERE" to the queue:
$.ajax({
url: 'https://sqs.us-east-1.amazonaws.com/71717171/myqueue' +
'?Action=SendMessage' +
'&Version=2012-11-05' +
'&MessageBody=HITHERE'
})
The'll be an error in the console saying that the request failed, but the message will show up in the queue anyways.

Have you considered JSONP? That is one way of calling cross-domain scripts from javascript without running into the same origin policy. You're going to have to set up some script somewhere to send you the data, though. Javascript just isn't up to the task.

Depending in what kind of data you want to send, and what you're going to do with it, one way of solving it would be to post the data to a Google Spreadsheet using Ajax. It's a bit tricky to accomplish though.Here is another stackoverflow question about it.
If presentation isn't that important you can just have an embedded Google Spreadsheet Form.

What about mailto:youremail#goeshere.com ? ihihi
Meantime, you can turn on some free hostings like Altervista or Heroku or somenthing else like them .. so you can connect to their server , if i remember these free services allows servers p2p, so you can create a sort of personal web services and push ajax requests as well, obviously their servers are slow for free accounts, but i think it's enought if you do not have so much users traffic, else you should turn on some better VPS or Hosting or Cloud solution.

Maybe CouchDB can provide what you're after. IrisCouch provides free CouchDB instances. Lock it down so that users can't view documents and have a sensible validation function and you've got yourself an easy RESTful place to stick your data in.

Accessing IndexedDB from multiple javascript threads

Overview:
I am trying to avoid a race condition with accessing an IndexedDB from both a webpage and a web-worker.
Setup:
Webpage that is saving items to the local IndexedDB as the user works with the site. Whenever a user saves data to the local DB the record is marked as "Unsent".
Web-worker background thread that is pulling data from the IndexedDB, sending it to the server and once the server receives it, marking the data in the IndexedDB as "Sent".
Problem:
Since access to the IndexedDB is asynchronous, I can not be guaranteed that the user won't update a record at the same time the web-worker is sending it to the server. The timeline is shown below:
Web-worker gets data from DB and sends it to the server
While the transfer is happening, the user updates the data saving it to the DB.
The web-worker gets the response from the server and then updates the DB to "Sent"
There is now data in DB that hasn't been sent to the server but marked as "Sent"
Failed Solution:
After getting the response from the server, I can recheck to row to see if anything has been changed. However I am still left with a small window where data can be written to the DB and it will never be sent to the server.
Example:
After server says data is saved, then:
IndexedDB.HasDataChanged(
function(changed) {
// Since this is async, this changed boolean could be lying.
// The data might have been updated after I checked and before I was called.
if (!changed){
IndexedDB.UpdateToSent() }
});
Other notes:
There is a sync api according to the W3 spec, but no one has implemented it yet so it can not be used (http://www.w3.org/TR/IndexedDB/#sync-database). The sync api was designed to be used by web-workers, to avoid this exact situation I would assume.
Any thoughts on this would be greatly appreciated. Have been working on it for about a week and haven't been able to come up with anything that will work.

I think I found a work around for this for now. Not really as clean as I would like, but it seems to be thread safe.
I start by storing the datetime into a LastEdit field, whenever I update the data.
From the web-worker, I am posting a message to the browser.
self.postMessage('UpdateDataSent#' + data.ID + '#' + data.LastEdit);
Then in the browser I am updating my sent flag, as long as the last edit date hasn't changed.
// Get the data from the DB in a transaction
if (data.LastEdit == lastEdit)
{
data.Sent = true;
var saveStore = trans.objectStore("Data");
var saveRequest = saveStore.put(data);
console.log('Data updated to Sent');
}
Since this is all done in a transaction in the browser side, it seems to work fine. Once the browsers support the Sync API I can throw it all away anyway.

Can you use a transaction?
https://developer.mozilla.org/en/IndexedDB/IDBTransaction

Old thread but the use of a transaction would solve the Failed Solution approach. I.e. the transaction only needs to span the check that the data in the IndexedDB hasn't change after the send and marking it as sent if there was no change. If there was a change, the transaction ends without writing.