Using NodeJs I'm trying to do something quite similar to Meteor: I want to send only the parts of a page that actually changed. My dilemma is that I know how to create such a framework to respond to link clicks and send partial updates but such a framework won't cater to direct browsing to a page other than the index (which is what is required for search engines and people without javascript to use your site).
I can also figure out how to make a framework to support entire page reloads, handlebars and a simple node server instance would take care of that. Hoeever, I can't figure out how to create a way that would allow me to write one method to tell the framework the partial updates for a page and let the framework figure out what else needs to be loaded.
A way I can think of would be to create the index page every time (for entire page loads) and apply partial updates to that but that can quickly become expensive if a subpage differs a lot from a very crowded index.
An example method would look something like this:
function images(id) {
if (id == null) {
// load all images from database
template.images = db.result();
template.content = template.loadblock('gallery');
}
else {
// retrieve single image
template.content = template.loadblock('single_image');
template.image = db.result();
}
}
On a partisl updste calling this method for domain.com/images would work just fine because it's clear what had changed.
For an entire page load this function would miss things like a header, footer ,navigation, etc.
In an answer I would look for an example where this has been done or some tips that Can point me in the right direction. I'm sorry for any typoes I wrote this post on an ipad. If you have any questions about my question just ask and I'll update as needed.
Update:
A possible example of a solution might be the following code. It's to give an idea, it probably won't actually run
// As a convention, don't pass around raw values if they aren't static but pass around functions such as
data.images = function () {
// run db query
// return an object with the images
}
// This constraint might be limited to the index() method
var routes = {
// This now allows us to do things like this:
index: function() {
var data;
// Initialise everything needed for the index
data.title = 'Index';
data.nav = { Index: '/', Images: '/images' };
data.content = 'Hello World';
},
categories: function() {
var data;
data.content = render('gallery', function () { /* load and return images as object */ }); // Not sure about this dynamic subtemplating but oh well
}
// This now allows us to do the following:
function request(page, type) {
if (type == 'update') {
if (routes[page] != undefined && typeof routes[page] == 'function') {
respond(routes[page]());
}
}
else {
if (routes[page] != undefined && typeof routes[page] == 'function') {
var data = mergeArrays(routes['index'](), routes[page]());
// index.html which is just a Handlebars template
respond(data);
}
}
}
Here is a pattern I often use (in Express apps):
function respond(req, res, name, resource) {
if(req.accepts('json')) {
// Send JSON to clients who want it
res.send(resource);
} else {
// Render with layout only for non-XHR requests
resource.layout = !req.xhr;
res.render('resources/' + name, resource);
}
}
Example usage:
app.get('/images', function(req, res, next) {
getImages(function(err, images) {
if(err) return next(err);
respond(req, res, 'images', images);
});
});
app.get('/images/:id', function(req, res, next) {
getImage(req.params.id, function(err, image) {
if(err) return next(err);
respond(req, res, 'image', image);
});
});
image.jade:
img(src=uri, alt=title)
images.jade:
#gallery
for image in images
include image
Clients who ask for JSON get that, otherwise they get the full page only if it's a non-XHR request. XHR requests get just the HTML snippet for the requested resource. This works well for quite simple apps, where resources mostly correspond to pages.
Related
I have a service worker in sw.js, it uses a template engine to get the commit numbre as a version number. I set the cache name like this:
var version = {{ commit_hash }};
self.cacheName = `cache-` + version;
I have some scripts being added to the cache on the worker's install, but there are scripts that are dynamically loaded on the page. I would like to load all the scripts/css on the first load without forcing the user to wait for the app to install first.
I can get all the content on the page with the following code in the bottom of index.html:
var toCache = ['/'];
var css = document.getElementsByTagName("link");
for(el of css) {
var href = el.getAttribute("href");
if(href) {
toCache.push(href);
}
}
var js = document.getElementsByTagName("script");
for(el of js) {
var src = el.getAttribute("src");
if(src) {
toCache.push(src);
}
}
That works fine, now I would just need to open the correct cache, fetch files that aren't already present, and store them. Something like:
toCache.forEach(function(url) {
caches.match(url).then(function(result) {
if(!result) {
fetch(url).then(function(response) {
caches.open(cacheName).then(cache => {
cache.put(url, response)
});
});
}
});
});
Is there a way to get the cacheName from the service worker inside a script tag in a different file?
And yes, I know that I could simplify this greatly by doing the check in the for/of loops. I broke it apart so it would be easier to describe.
No.
JavaScript executing in the window context cannot access SW's context and vice versa. You have to implement a workaround of some sort.
Remember that you can use postMessage to communicate between the two.
Using this blog I was able to pass messages from the service worker and back. First, I added the following function at the top of sw.js:
function clientPostMessage(client, message){
return new Promise(function(resolve, reject){
var channel = new MessageChannel();
channel.port1.onmessage = function(event){
if(event.data.error){
reject(event.data.error);
}
else {
resolve(event.data);
}
};
client.postMessage(message, [channel.port2]);
});
}
This allows my service worker to post a message to the window, and then do a callback with a promise.
Then, in my index.html file I added the following to a script tag:
navigator.serviceWorker.addEventListener('message', event => {
switch(event.data) {
case "addAll":
var toCache = [];
var css = document.getElementsByTagName("link");
for(el of css) {
var href = el.getAttribute("href");
if(href) {
toCache.push(href);
}
}
var js = document.getElementsByTagName("script");
for(el of js) {
var src = el.getAttribute("src");
if(src) {
toCache.push(src);
}
}
event.ports[0].postMessage(toCache);
break;
default:
console.log(event.data);
}
});
This listens to any service workers asking for messages, and if it is a "addAll" message, it will get all the scripts and linked content on the page and return an array of the scripts.
Finally, I added the following to my activate event listener function in sw.js:
// Get all the clients, and for each post a message
clients.matchAll().then(clients => {
clients.forEach(client => {
// Post "addAll" to get a list of files to cache
clientPostMessage(client, "addAll").then(message => {
// For each file, check if it already exists in the cache
message.forEach(url => {
caches.match(url).then(result => {
// If there's nothing in the cache, fetch the file and cache it
if(!result) {
fetch(url).then(response => {
caches.open(cacheName).then(cache => {
cache.put(url, response);
});
});
}
})
});
});
})
});
For all clients the service worker sends an "addAll" message to the page and gets the result. For each item in the result, it checks if the value is already in the cache and if not, fetches and adds it.
With this method, the install listener of the service worker only needs to contain:
self.addEventListener('install', event => {
if(self.skipWaiting) {
self.skipWaiting();
}
event.waitUntil(
caches.open(cacheName).then(cache => {
return cache.addAll([
'/',
'/index.html',
])
})
);
});
It seems to be working well so far, if anyone has any suggestions or sees any errors I'd be happy to hear! You can also tell me how improper this is, but it makes my life a lot easier for adding service workers for pre-existing projects that rely on scripts that aren't bundled together.
I am trying to get prerender working on both local and prod. I feel like I have tried all implementations. I am still getting no static html in the body when using: ?_escaped_fragment_= at the end of the URL.
Here is my current Meteor implementation:
Meteor.startup(() => {
var prerenderio = Npm.require('prerender-node');
var token;
var serviceUrl;
var protocol;
var settings = Meteor.settings.PrerenderIO;
token = process.env.PRERENDERIO_TOKEN || (settings && settings.token);
protocol = process.env.PRERENDERIO_PROTOCOL || (settings && settings.protocol);
// service url (support `prerenderServiceUrl` (for historical reasons) and `serviceUrl`)
serviceUrl = settings && (settings.prerenderServiceUrl || settings.serviceUrl);
serviceUrl = process.env.PRERENDERIO_SERVICE_URL || serviceUrl;
if (token) {
if (serviceUrl) prerenderio.set('prerenderServiceUrl', serviceUrl);
prerenderio.set('prerenderToken', token);
if (protocol) prerenderio.set('protocol', protocol);
prerenderio.set('afterRender', function afterRender(error) {
if (error) {
console.log('prerenderio error', error); // eslint-disable-line no-console
return;
}
});
WebApp.rawConnectHandlers.use(prerenderio);
}
});
I have my settings file set up as so:
"PrerenderIO": {
"serviceUrl": "http://localhost:3033/",
"token": "mytoken"
},
Same for prod but without the serviceUrl. I did get the prerender server up and the page renders....but its still the default Meteor script rendered page. I also tried: <script> window.prerenderReady = false; </script> and then set it to true after my API content has loaded via our router (using ButterCMS for site content.
I have of course also added: <meta name="fragment" content="!"> to our sites head.
Prerender is still saying its not seen our token get used. I think I could be missing something obvious here....but not certain what it is.
That seems like the prerender middleware is not being run. Does Meteor leave the rawConnectHandlers in the order that they are added? Can you try this:
WebApp.rawConnectHandlers.use(function(req, res, next) {
console.log('before prerender:', req.url)
});
WebApp.rawConnectHandlers.use(prerenderio);
And see if you see any output in your logs for that showing what the incoming URL looks like. If you are accessing the ?_escaped_fragment_= URL, you should see get printed in that console.log statement.
Feel free to email us at support#prerender.io with a URL if you'd like us to help test.
You have an application created in express and angular that allows the user to perform a search. The URL is built based upon the search that was just performed. So if you perform a search on “Will” the url looks like http://localhost.com:9000/search/query?q=Will Everything works fine but you forgot that the app previously performed searches without the /query?= and now all of your old links like http://localhost.com:9000/search/will or http://localhost.com:9000/search/roberto no longer work.
What would be the correct approach to get the old links working again?
Should you use JavaScript on the frontend to look for /query?= missing in the URL and add after the search path but before the queried text?
It'd be easier to do a redirect on the Express back-end.
Say your code for the /search/query path is initially like this :
app.get("/search/query", function (req, res) {
// Do your query validation and fetch your search result.
// Here, I just check if a query value was given or not for the q param.
// I recommend you use better ways to check for empty queries.
// (ex: lodash's `isEmpty()` function)
if (req.query.q) {
// Serve the page !
res.send("What you want to render if the search result finds something.");
}
else {
// Return an error !
res.status(404).send("Nothing was found with the criterias entered.");
}
});
This is probably similar to what you have. Now, here is the answer to your question, based on the initial implementation above :
app.get("/search/query", function (req, res, next) {
// Check if a query value was given AND if the value isn't equal to "query".
// The later condition is to prevent infinite loops.
if (req.query.q && req.query.q !== "query") {
// Redirect using the value assigned to the q query param.
res.redirect("/search/" + req.query.q);
}
else {
// Since there is no query parameter named `q` in the request,
// we can be sure that `query` reffers to a search term.
next();
}
});
app.param("srchterm", function (req, res, next, value) {
// Check, for example, if the value isn't empty.
if (value) {
// Do your query validation and fetch your search result HERE.
// Add those results in an array in the res.locals object.
// Those results can be used later.
res.locals.results = ["all", "your", "search", "results"];
}
next();
});
app.get("/search/:srchterm", function (req, res) {
console.log("another blah");
// We don't need to fetch the data here anymore, since it's handled by the param parser above!
// However, it's still necessary to check if the search gave back some results.
if (res.locals.results) {
// Serve the results !
res.send("A total of " + res.locals.results.length + " results were found for " + req.params['srchterm']);
}
else {
// Return an error !
res.status(404).send("Nothing was found with the criterias entered.");
}
});
So from now on, every query using /search/query?q=123 will redirect towards /search/123. It even lets you use query as the search term!
Just use a regex and redirect
app.use(function(req, res, next) {
var searchRegEx = /\/search/g;
var searchedTerm = req.originalUrl.replace(searchRegEx, '');
var queryPath = req.originalUrl.match(/\/query[?]q=/);
if(!queryPath) {
var regexSlash = /\//g;
res.redirect('query?q=' + searchedTerm.replace(regexSlash, ''));
}
else {
next();
}
});
I'm learning FRP using Bacon.js, and would like to assemble data from a paginated API in a stream.
The module that uses the data has a consumption API like this:
// UI module, displays unicorns as they arrive
beautifulUnicorns.property.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
The module that assembles the data requests sequential pages from an API and pushes onto the stream every time it gets a new data set:
// beautifulUnicorns module
var curPage = 1
var stream = new Bacon.Bus()
var property = stream.toProperty()
var property.onValue(function(){}) # You have to add an empty subscriber, otherwise future onValues will not receive the initial value. https://github.com/baconjs/bacon.js/wiki/FAQ#why-isnt-my-property-updated
var allUnicorns = [] // !!! stateful list of all unicorns ever received. Is this idiomatic for FRP?
var getNextPage = function(){
/* get data for subsequent pages.
Skipping for clarity */
}
var gotNextPage = function (resp) {
Array.prototype.push.apply(allUnicorns, resp) // just adds the responses to the existing array reference
stream.push(allUnicorns)
curPage++
if (curPage <= pageLimit) { getNextPage() }
}
How do I subscribe to the stream in a way that provides me a full list of all unicorns ever received? Is this flatMap or similar? I don't think I need a new stream out of it, but I don't know. I'm sorry, I'm new to the FRP way of thinking. To be clear, assembling the array works, it just feels like I'm not doing the idiomatic thing.
I'm not using jQuery or another ajax library for this, so that's why I'm not using Bacon.fromPromise
You also may wonder why my consuming module wants the whole set instead of just the incremental update. If it were just appending rows that could be ok, but in my case it's an infinite scroll and it should draw data if both: 1. data is available and 2. area is on screen.
This can be done with the .scan() method. And also you will need a stream that emits items of one page, you can create it with .repeat().
Here is a draft code (sorry not tested):
var itemsPerPage = Bacon.repeat(function(index) {
var pageNumber = index + 1;
if (pageNumber < PAGE_LIMIT) {
return Bacon.fromCallback(function(callback) {
// your method that talks to the server
getDataForAPage(pageNumber, callback);
});
} else {
return false;
}
});
var allItems = itemsPerPage.scan([], function(allItems, itemsFromAPage) {
return allItems.concat(itemsFromAPage);
});
// Here you go
allItems.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
As you noticed, you also won't need .onValue(function(){}) hack, and curPage external state.
Here is a solution using flatMap and fold. When dealing with network you have to remember that the data can come back in a different order than you sent the requests - that's why the combination of fold and map.
var pages = Bacon.fromArray([1,2,3,4,5])
var requests = pages.flatMap(function(page) {
return doAjax(page)
.map(function(value) {
return {
page: page,
value: value
}
})
}).log("Data received")
var allData = requests.fold([], function(arr, data) {
return arr.concat([data])
}).map(function(arr) {
// I would normally write this as a oneliner
var sorted = _.sortBy(arr, "page")
var onlyValues = _.pluck(sorted, "value")
var inOneArray = _.flatten(onlyValues)
return inOneArray
})
allData.log("All data")
function doAjax(page) {
// This would actually be Bacon.fromPromise($.ajax...)
// Math random to simulate the fact that requests can return out
// of order
return Bacon.later(Math.random() * 3000, [
"Page"+page+"Item1",
"Page"+page+"Item2"])
}
http://jsbin.com/damevu/4/edit
This is maddening, how do I get a hold of a loopback model so I can programmatically work with it ? I have a Persisted model named "Notification". I can interact with it using the REST explorer. I want to be able to work with it within the server, i.e. Notification.find(...). I execute app.models() and can see it listed. I have done this:
var Notification = app.models.Notification;
and get a big fat "undefined". I have done this:
var Notification = loopback.Notification;
app.model(Notification);
var Notification = app.models.Notification;
and another big fat "undefined".
Please explain all I have to do to get a hold of a model I have defined using:
slc loopback:model
Thanks in advance
You can use ModelCtor.app.models.OtherModelName to access other models from you custom methods.
/** common/models/product.js **/
module.exports = function(Product) {
Product.createRandomName = function(cb) {
var Randomizer = Product.app.models.Randomizer;
Randomizer.createName(cb);
}
// this will not work as `Product.app` is not set yet
var Randomizer = Product.app.models.Randomizer;
}
/** common/models/randomizer.js **/
module.exports = function(Randomizer) {
Randomizer.createName = function(cb) {
process.nextTick(function() {
cb(null, 'random name');
});
};
}
/** server/model-config.js **/
{
"Product": {
"dataSource": "db"
},
"Randomizer": {
"dataSource": null
}
}
I know this post was here a long time ago. But since I got the same question recent days, here's what I figured out with the latest loopback api:
Loopback 2.19.0(the latest for 12th, July)
API, Get the Application object to which the Model is attached.: http://apidocs.strongloop.com/loopback/#model-getapp
You can get the application which your model was attached as following:
ModelX.js
module.exports = function(ModelX) {
//Example of disable the parent 'find' REST api, and creat a remote method called 'findA'
var isStatic = true;
ModelX.disableRemoteMethod('find', isStatic);
ModelX.findA = function (filter, cb) {
//Get the Application object which the model attached to, and we do what ever we want
ModelX.getApp(function(err, app){
if(err) throw err;
//App object returned in the callback
app.models.OtherModel.OtherMethod({}, function(){
if(err) throw err;
//Do whatever you what with the OtherModel.OtherMethod
//This give you the ability to access OtherModel within ModelX.
//...
});
});
}
//Expose the remote method with settings.
ModelX.remoteMethod(
'findA',
{
description: ["Remote method instaed of parent method from the PersistedModel",
"Can help you to impliment your own business logic"],
http:{path: '/finda', verb: 'get'},
accepts: {arg:'filter',
type:'object',
description: 'Filter defining fields, where, include, order, offset, and limit',
http:{source:'query'}},
returns: {type:'array', root:true}
}
);
};
Looks like I'm not doing well with the code block format here...
Also you should be careful about the timing when this 'getApp' get called, it matters because if you call this method very early when initializing the model, something like 'undefined' error will occur.