Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I am working on a web app that will automatically generate Bootstrap sites given a bounded set of parameters. I need help with regards to architectural decisions, as over time, I have made very naive design decisions and I don't want to repeat this mistake with this project. For my PoC, I have taken an MVC approach to design, with the top-level architecture (see above) containing the following:
CanvasComponent - this is the entry point for the application, a UI element in which the user can generate their website and be given to download it;
CanvasService - this service orchestrates the creation of the website, and has dependencies on a set of services (NavbarService, HeroService, FooterService);
DomService - this service is in charge of the creation of the DOM element at a high level sans styles;
ComponentService - these group of services (NavbarService, HeroService, FooterService) are in charge of the creation of the DOM element at a lower level;
FontService - this service is a Google font repository.
The above is something I put together fairly quickly, so I understand that it isn't perfect. I also have a few questions with regards to the direction of the project with regards to the scalability of it as I move towards productionalising it. These are my problems:
I showed it to someone and they said they can't make sense of it... as it sounds like my CanvasService, DomService and component services create raw HTML, all of which are models that describe DOM elements. It seems very overly mis-engineered. the way we are developing our services isn't scaleable or manageable. If nothing else, it is an extreme violation of the open/closed principle. They will ultimately be huge "factories" that don't fully cover all the functionality of DOM elements. This is my main concern. How do I resolve this from an architectural perspective?
Given the nature of the project, I am dealing quite extensively with raw html. I use a mixture of approaches to dynamically generate the required HTML. I wanted to ask if the above is advisable. What would be the cleanest approach architecturally to creating and managing blocks of HTML? Should I use raw string or create something more bespoke?
What would be the best approach to storing an assortment of styles per component (navbar, hero, footer) and then applying them randomly? From what I understand a style can be separated into two categories - a template and a theme. A template can have many themes, and a single theme can be associated with multiple themes. A good example is "light/dark" colour mode on a website. At the moment in NavbarService, I store a range of possible widths and heights in an array.
Are there any issues that my current architectural approach will present down the road besides the violation of the open closed principle (unknown unknowns)? In the past, whenever I have started projects like this, they have quickly become unsustainable. I lose interest, because I am wrestling with the code-base at that point. I want the process to continue to be enjoyable, and keep my code clean and manageable over the course of the project, as well as apply the best principles.
A sample demo is available here: https://stackblitz.com/edit/angular-ivy-2pga8q (each time you reload the page, a new navbar appears).
This is an interesting problem. I also didn't quite understand the diagram but your description makes more sense.
Let's assume for a minute your generated HTML/CSS/JS components are sensibly written and generate reasonably formatted and sensible output. At this point you don't really 'need' to store meta-data about it as anything will be un-required bloat. What you will need is the ability to construct small, modular blocks of code that can be recompiled again after editing or additional input parameters.
To support this I strongly suggest separating the output of the code (HTML) as a version-stamped solution library - and the code that generated that should be a collection of marked-up data from user or data inputs.
To create a simple analogy imagine the formula (8+1+1=10).
In this example the output (10) is the only thing you need to publish/present/store. However you need to keep the starting data (8+1+1) independent - primarily because if one of those values changes (say it's now 8+2+1) you can now re-generate a second version of the output with a new version and get a new answer (11).
As such I'd focus a LOT more on storing the the building blocks/wysiwyg/mark up components than worrying about the string/text format of the output elements. From an architecture perspective if you have any ongoing need to interact/modify this code you need to focus on ease of use and re-usability.
Remember also unless you're creating skynet you're not writing 'code that writes code'. You're writing code that assists humans in writing code. Always assume there will be need for intervention and modification from an outside source - focus on getting the modular form of (algorithm + input = output) and store the process and the inputs. If you get that right you can re-generate the outputs at any time for any reason.
Related
What is the advantage of using shared module over rewriting code in each component/module in Angular?
In my project I've approx 30-40 modules. In all modules in service file same api is written. As per angular standard we should use sharedModule to so that code can be reused. I want to update my Angular project before that wanted to understand what is the advantage of using shared module over re writing code? How will it help to my Angular project?
As per angular standard we should use sharedModule
This isn't per Angular standard. It's per any standard, let alone per development standard.
The phrase exists: "don't reinvent the wheel".
Literally - car needs new tyres? Not going to design whole new ones, you'll grab some more off the shelf and shove them on.
Same applies - 7 places in your app that need to make API requests? Don't design and write 7 whole new ones, use the one you've already made.
Design principal: DRY - Don't Repeat Yourself.
This is especially important with code. You say you have 30-40 modules. Each with their own copy/paste version of some API service.
What happens when authentication is added/removed/modified for that API? Suddenly need to add some token into the header for your requests?
30-40 copy/paste jobs after you've made the change. 30-40... you can't even give us an exact number! How do you know you replaced ALL of them successfully?
Why on Earth would you do that to yourself when you can just keep reusing the one original thing you made?
30-40 modules all use that one API service. One place to make any fixes/changes. One service to test.
Oh lawd the testing - of which I'm nearly 100% certain you have zero tests, and any you do have are likely ineffectual and definitely don't cover nearly as much as you should have covered.
That's 30-40 test classes that you need to update as well (let me guess - copy paste those too?).
And that's just a single mentioned API service. What do you do if you write yourself some kind of helper methods for something in your app?
"Oh, I got fed up of writing these same 5 lines to do x, so I wrote a method to do it for me, it makes it much faster".
Cool - copy paste that another 30-40 times for me into all our other modules so that we can use it too. Thanks.
Put that shizzle into your shared module. One helper class. One class to write tests around. One class to change for additions/fixes. Zero copying and pasting and wasting time and missing things.
Ignoring alllllll of this, how the bejeesus have you managed to go days/weeks/months of repeating yourself over and over and copying/pasting over and over and over and god knows what else over and over and over.... and not once thought "this is a lot of effort, maybe I can save some here by doing something smarter"?!
This isn't even a thought-provoking or discussion-inspiring question. It's a question drawing attention to ones basic common sense and the long-standing human desire to be able to do as much or more with the same or less effort.
Why'd we figure out farming? Because hunting around the whole area for a few berries was more effort.
Why'd we hook animals up to our ploughs? Because it's hard work and we're lazy.
Why'd we replace animals with tractors? Because they can do it better.
Why're we replacing traditional farms with those swanky 'vertical' farm things? Because they're more efficient, can be automated more, etc.
Stop copying and pasting chunks of anything.
The millisecond you do anything for a second time, you refactor that away into a single thing that both can use.
I sincerely hope that you are currently a student and/or just starting out (self taught?). If so, welcome! Keep asking questions, keep hitting Google for your answers (where you'll find better than I can provide), and keep learning. My code was just as bad (worse, likely) back at uni.
If you're not, and are actually a 'seasoned' software developer of some kind, where people are paying you to do this... Please stop, take up farming, and let us all know what you've worked on to date so that we can immediately stop using any of it.
1. The problem
Lately, it seems that many note managers with "infinite" tree structure are choosing a block model (where each paragraph is an entry in the DB), instead of a document or file model.
Blocks
Documents
Notion Workflowy Remnote Dynalist Roam Research
Evernote Obsidian Bear app
If you find any errors in the table, please let me know.
We have been developing an app very similar to Notion for 8 months now, also using the block model, but we are considering making a radical change and switching to the document model. The structure of our blocks in MongoDB currently looks like this:
_id: "61fd3ede7f6d2cc7a53ca669"
children: Array
0: "61fd3ee87f6d2cc7a53ca66b"
1: "61fd3ef37f6d2cc7a53ca671"
2: "61fd3ef77f6d2cc7a53ca673"
backlinks: Array
type: "bullet"
parentPage: Array
_id: "61fd3ede7f6e2ccra53ca664"
userParent: "german-jablo"
permisionParent: "edit, comment, read"
parentParagraph: "61fd3ede7f6d2cc7a53ca668"
content: "<p>This is a paragraph</p>"
isCollapsed: false
createdAt: 2022-02-04T14:57:34.280+00:00
updatedAt: 2022-02-04T14:57:59.585+00:00
Many pages talk about the differences of both approaches (example) although in a very vague way, so we decided to open this thread to find a more scientific answer to the question.
Features of our app
app
Blocks that can be opened as documents
Blocks that can collapse or expand their children
Notion
Page type blocks
Toggle type blocks
Workflowy
All
All
Evernote
Documents
None
Our app
Page type blocks
All others
Our app has two types of "blocks". The page type (which, like in Notion, can be inserted into any note and generate a document "inside" the current document), and the rest of the blocks, which are equivalent to the "toggle" block type in Notion (i.e. they can be collapsed or their nested children can be expanded).
2. What we have tried
In trying to answer our question (which DB model would work best for our application), we've realized that the answer is probably "it depends". Perhaps both models have strengths or weaknesses in different types of operations or situations. That is why we formulated this comparison table describing how we believe the performance of both models would be for each of these operations.
Operation
Blocks
Documents
Apparent Winner
Fetch the contents of a page
Find all paragraphs in the DB.
Search the document in the DB.
Document
Render the content of a page**
Build the tree from the paragraphs recursively. You can omit the children of paragraphs whose isCollapsed property is true
Render the document
Document
Update the content of a paragraph in the DB
Only the modified paragraph is rewritten
The whole document is rewritten
Block
Alternatives for rendering very large documents *
Blocks can be fetched or rendered as you scroll (as Workflowy does), or as you expand child paragraphs that were collapsed.
I thought that Grifds could achieve similar behavior, breaking the document into smaller chunks and bringing them in piecemeal, but it doesn't support updating an individual chunk, or even the entire document. It could also corrupt an HTML by splitting it into binary format.
Block
Import or paste content
In addition to converting the clipboard to HTML and/or sanitizing it, you must set up paragraphs with tree structure recursively. Note: Roam Research e.g. supports importing in JSON format, but generally users do not handle this format beforehand.
Only convert the clipboard to HTML and/or sanitize the clipboard
Document
Copy content**
Clipboard must be sanitized and/or transformed
Correct by default**
Document
Real-Time Collaboration
At the document level, could use some tree-based (Json) library like Automerge, or combine with some CRDT library for paragraph level.
Could use tinymce solution.
Tie? Both seem to have their advantages and disadvantages.
*Render very large documents: Most users probably do not use notes larger than 250 kb (considering that multimedia files are referenced in a separate collection). Still, in the document model, the question arises: how can we load, render or edit large documents in manageable chunks? One idea we came up with is to split HTML documents that reach large dimensions into portions of a certain size in kb, instead of splitting them into paragraphs. (It would be like a kind of Gridfs that allows you to modify the file in parts.) Could this be a good idea?
**Should the DOM be nested? In order to be able to collapse or expand nested child paragraphs, note managers with a block model structure the DOM in a nested way (paragraphs are in divs, inside their parent divs, etc.). However, an alternative in the document model could be that when the user presses tab, only that block (HTML tag such as <p> or <li>) is assigned an attribute with a number less than or equal to 1, representing the nesting levels relative to the previous block. This way when you press tab to nest or shift-tab to un-nest, you only have to modify one attribute of an HTML element instead of many elements; and the DOM stays simple, without having nested blocks.
3. Our conclusions
We believe that for each of the rows in the comparison table, benchmarks could be done measuring the performance of both models.
Other people have done something similar here and here, comparing the performance of note managers using both models. The problem with those tests is that it is difficult to draw an accurate conclusion about the goodness of both models. Obsidian uses documents locally, so you don't have to sync notes. Roam Research is a very new and poorly optimized app. Standard Notes encrypts notes locally. In other words, it's not always apples to apples.
And even if tests could be done, we believe that the answer may even depend on how each user uses the application. Suppose user A usually organizes his notes in long documents using paragraph nesting (to collapse or expand them). On the other hand user B usually organizes his notes by creating new documents within documents. It is likely that a block model based manager would work better for user A while a document based one would work better for B.
So, we tried to push our doubt as far as we could, but we are still not sure of the answer Which of the two models do you think would offer better performance for our app and why?
4. Update
I just found some very interesting information. It seems that both TinyMCE and CKEditor (up to version 4), the view and the model converge with the HTML being based on content-editable. However, CKEditor 5 switched to MVC [source 1], [source 2].
I've done a short test pasting a large clipboard of a few MB in TinyMCE 5, CKEditor 4 and CKEditor 5, and the latter has been a bit slower. I hope soon to be able to do more tests with other things like dragging blocks or rendering large documents.
In a GitHub thread about CKEditor 5's performance when working with large documents, one of the contributors said "It works slower than the native content-editable element obviously, but the typing experience is pretty well".
Looks like you've done your homework. Database modeling is sometimes a bit of an art rather than a science. I think that with both models, you can achieve good performance if you optimize them well. So I would recommend that you go for the one that requires less work. Since you've been working on the block model for 8 months already, that's probably the best option for you.
Love how thorough you thought about the design of your application. I just want to add some suggestions you might look:
Obsidian is using a hybrid approach. It started document-based, but now supports block links and embeds while still being super-fast. From all programs tested in my benchmark you linked above, Obsidian was the fastest.
One of the most important functions of a notetaking tool is effectively searching the probably thousands of notes. I created an amazingly simple test (the "Spaghetti Parmesan"-test) where all block-based approaches currently fail. It is about searching for two ingredients (spaghetti and parmesan) in a recipe. When both ingredients are in different blocks, all common block-based applications ultimately fail to find the recipe. You can read more about this here. I also tried to start a discussion with some of the authors on Twitter but failed to get any serious results. If you want to continue the block-based approach, you might start designing a search algorithm that can handle search terms spread about blocks (if you haven't yet). I tried to outline an algorithm in the thread linked above, but not sure if it will really work with hundreds of thousands of blocks.
You seem to be making your choice based on what is easier to model or develop.
If you intend to compete in this increasingly crowded market, you need to consider what will give you an edge, and what will be sustainable in the long run.
Two of the main issues Evernote (longtime user and former employee) faced were:
a very badly designed back-end that was very expensive to scale and made sharing and collaboration a nightmare (engineering-wise)
a too loose document model that made it very hard to add new features to the editor (real time collaboration) and made collaboration very finicky even for simple features (checkboxes would often generate so many note conflicts as to be unusable)
On the other hand, block-based tools usually have really bad web clippers, because it's very hard to go from HTML to blocks.
Solve some of these hard problems!
Many websites today auto-generate dynamic class names via CSS modules or styled-components. This practice has the unfortunate and perhaps even deliberately malicious effect of disabling many user-defined filters and/or scripts; e.g. Reddit overrides all manner of client-side cosmetic changes; even simple things like karma are being forced upon the user due to the fact that there's apparently no persistent global identifier which the user could invoke to forbid the display of karma:
An example screenshot of Reddit code illustrating the dynamic class names at hand
I think this is authoritarian, inappropriate, and creates a horrible user experience. I don't believe it's reasonable that websites should exercise this kind of control. What exactly can the user do to reign in these seemingly un-manipulable dynamic class names? e.g., is there a way to reverse engineer the build processes used by CSS modules or styled-components in order to then inject the proper identifiers into a user-defined filter and/or script? Or are there better methods?
Note: I've already asked and received no decent response to this question on Webmaster, UX, and SuperUser. Is this the end of the road? Or is there a StackOverflow guru with a workable solution?
To hide karma spans on a comments page on Reddit, today, you could run the following JavaScript:
Array.from(document.querySelectorAll('span')).filter(i => /(\d+)\spoint/.test(i.innerHTML)).forEach(i => i.style.display = 'none');
Tested it here.
However, it only hides karma on the rendered comments, not on the ones that are not yet loaded.
To programatically open all "moreComments" links on a page, you'd need to run:
Array.from(document.querySelectorAll('div')).filter(i => /moreComments-/.test(i.id)).forEach(i => i.querySelector('p').click())
Depending on how many hidden comments are on the page, you'd need to wait a while until all requests resolve and all comments are rendered.
At which point running the first line will hide all karma spans on the page.
And this might not work tomorrow, as Reddit own that code and have the right, the ability and the resources to change its inner structure, layout or design as they see fit, any number of times a day, so that any puny attempts of altering the rendered output of their service, by coder wannabes like me and you, remain futile.
Edit: Ever since I answered your question I wanted to provide some helpful insight on how to approach StackOverflow. I hope you find it helpful or, at least, entertaining.
I've been using KnockoutJS for some time now, mainly on small-to-medium sized projects. However, I'm now working on a very large project with many different views of the same data. For example, a product may be displayed to the customer, allowing them to add products to their shopping cart, and also to an administrator, allowing them to edit the product name, price and available stock. I want to be able to use the same model in both cases, but designing a solution with KnockoutJS is proving difficult. Specifically, I am having two issues:
I want to be able to re-use certain "view" functionality without repeating myself. For example, in both the customer- and administrator- view of products, clicking the product thumbnail displays the product image in fullscreen. Instead of each view containing the same verbose binding code (e.g., <img data-bind="event:{click:function codeToZoom(){}}"/> in both views), I can move the binding information to the model itself. So the above becomes <img data-bind="event:imageEvents()"/>. However, doing the above forces me to include code that responds to user input in my model, which violates the single-responsibility principle - e.g., the model is reponsible for business logic, not responding to user input. If I decided I wanted an administrator clicking the thumbnail to open an "Upload New Image" dialog, then I'd need to implement a imageEventsForAdministrator() function.
I've asked people how they deal with the above, and their answer has been "Write two different models." This sounds good until you realise that a product has a lot of embedded logic, and writing two different models forces you to duplicate that logic. So:
According to KnockoutJS, what is the recommended approach for separating business logic from responding to events/user-input?
Your post is interesting and understandable... yet really rather broad for Stack Overflow.
To answer some of your more specific subquestions though:
I want to be able to re-use certain "view" functionality without repeating myself.
There are two typical constructs for tackling this issue:
Use templates to do so. Knockout will keep the template as a "prototype" view, and instantiate new instances (and clean up) as needed.
Load view html dynamically as "partial views" if you will, and use the appropriate applyBindings overload to render those bits only when needed.
For example, [...] clicking the product thumbnail displays the product image in fullscreen [...] <img data-bind="event:{click:function codeToZoom(){}}"/> in both views
Well, there's several options here. I find that if you want you can actually de-duplicate code as much as theoretically possible. First, you can usually do this:
<img data-bind="event:{click: myHandler}"/>
And you can use some kind of prototypical or classical inheritance on your view models to make sure myHandler is only defined once.
Again, if you find yourself repeating the <img... bit: use KO templates.
Moving on:
This sounds good until you realise that a product has a lot of embedded logic, and writing two different models forces you to duplicate that logic.
In that case you're probably violating the SRP. You probably need to take that logic out of your view models and place it somewhere else (e.g. in controllers, DALs, etc.). But that topic is really broad, our sister site has an entire tag dedicated to it.
Finally, you ask:
According to KnockoutJS, what is the recommended approach for separating business logic from responding to events/user-input?
KnockoutJS recommends no particular approach.
That is, KO is an MVVM library, and as such does nudge you in a certain direction, but it doesn't really force you to take any particular approach with this. It's really up to you to write and structure infrastructure and code for common business logic, think up a good inheritance strategy or something similar, etc.
So it's up to you.
As a footnote, if you haven't already, I'd recommend running through some AngularJS tutorials as well. It has MVVM style bindings, but it is IMO more opinionated about how to solve the problems you're facing. (This is not advice to switch; merely advice to find inspiration in KO-alternatives.)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
My situation is this, I've been tasked with fixing little issues and debugging issues in an app that relies heavily on JavaScript. One of the other tasks will be to make the app easier to understand and more scalable. As new functions and features get added, the current app is more and more difficult to modify and update. Making changes in one function can really change things in another function.
My initial thoughts are to make the JavaScript object oriented, as it stands its just a bunch of functions, if / thens, switches, etc...
If my thoughts are correct and taking it in that direction is the way to go, can anyone suggest a model or format I can use for better breaking down the app. Like what should I turn into an object, what should I turn into methods or properties, etc... I just need some sort of guide.
If my thoughts are not correct and taking it in that direction is not the way to go, can anyone give me some ideas as to another method.
Thanks,
I would consider introducing the jQuery library into the picture.
Convert a few areas to see how much code shrinkage you can expect.
I don't really know what you're code looks like, so I can't really give you anything better than some general advice:
Use a JavaScript library, like jQuery, MooTools, or Prototype. I would recommend jQuery, but I can't say that I've really used the others very much. The point here is that they all are libraries that make common tasks that are rather tiresome to do by yourself much easier.
If you see lines of code that are the same, or even similar, refactor those lines into a new function.
Avoid magic numbers. They makes the code less scalable as well as less readable. For example, if you are referencing a desired length that happens to be 100, assign 100 to a variable and reference the variable instead. You can occasionally scan your code and look for numbers. Other than variable declarations, you shouldn't see very many numbers other than an occasional 1, 0, -1, or 2. The same goes for magic strings.
Keep all this configuration data code in one place so things can easily be changed. You might even have an entirely separate file that just has all the data inside of it.
Put related variables in an object. For example, instead of having xCoords, yCoords, and zCoords, have a coords object with x, y, and z as its properties. You really shouldn't have very many standalone global variables.
Try to have a consistent code style and try to give things sensible names. This alone can make your code much more readable.
For a quick real-life example, look through this small code project and see what sort of things were improved in its reviews. You can hone your code improvement skills by being active on the Code Review site.
Backbone.js
Backbone supplies structure to JavaScript-heavy applications by
providing models with key-value binding and custom events, collections
with a rich API of enumerable functions, views with declarative event
handling, and connects it all to your existing application over a
RESTful JSON interface.
Backbone might be a great solution for you. It's not too invasive. If nothing else the bindings, models, and custom events could really help you get that code in order.