Develop Reference

3D Grid for multiple shapes

A few months ago I made a small terrain generator, like Minecraft, for a school project.
The way I did this was by using multiple chunks. Each chunk contained a 3-dimensional array that stored the blocks.
Every position in this array corresponded with the position of the block it contained.
blocks[x, y, z] = new Block();
Now I would like to add different sizes if blocks. However, I can't do that with the way I am storing the blocks right now, because bigger blocks would have to be spread over multiple positions in the 3-dimensional array.
An example of a game with different sizes of blocks (and different shapes) is LEGO Worlds. How does a game like this store all these little blocks?
I hope someone can help me with this.
The language I am using is Javascript in combination with WebGL.
Thanks in advance!

In my experience there are a few different ways of tackling an issue like this, but the one I'd recommend would depend on the amount of time you have to work on this and the scope (how big) you wanted to make this game.
Your Current Approach
At the moment I think your using what most people would consider the most straightforward approach by storing the voxels in a 3D grid
[Source].
But two problems you seem to be having is that there isn't an obvious way to create blocks that are bigger then 1x1 and that a 3D grid for a world space is fairly inefficient in terms of memory usage (As for an array you have to have memory allocated for every cell, including empty space. JavaScript is no different).
An Alternative Approach
An alternative to using a 3D array would be to instead use a different data structure, the full name being a sparse voxel octree.
This to put it simply is a tree data structure that works by subdividing an area of space until everything has been stored.
The 2D form of this where a square sub divides into four smaller quadrants is called a quad tree and likewise a 3D equivalent divides into eight quadrants, called an octree. This approach is generally preferable when possible as its much more efficient because the trees only occupy more memory when its absolutely essential and they can also be packed into a 1D array (Technically a 3D array can be too).
A common tactic used with quad/octrees in some block based games is to take a region of the same kind of voxel that fit into one larger quadrant of the tree is to simply stop sub division there, as there's no reason to go deeper if all the data is the same.
The other optimization they can make is called sparse where regions of empty space (air) are simply deleted since empty space doesn't do anything special and its location can be inferred.
[SVO Source]
[Z Order Curve Source]
Recommended Approach
Unless you have a few months to complete your game and you're at university I seriously wouldn't recommend an SVO (Though reading up about could impress any teachers you have). Instead I'd recommend taking the same approach that Minecraft appears to visibly has. E.G. A door is 1X2 but blocks can only be 1x1, then just make it two blocks.
In the example of a door you would have four unique blocks in total, two for the upper and lower half, and two variations of each being opened or closed.
E.G.
var cubeProgram; // shader program
var cubeVBO; // vertex buffer (I recommend combining vertex & UV coords)
var gl; // rendering context
// Preset list of block ID's
var BLOCK_TYPES = {
DOOR_LOWER_OPEN: 0,
DOOR_UPPER_OPEN: 1,
DOOR_LOWER_CLOSED: 2,
DOOR_UPPER_CLOSED: 3,
}
var BLOCK_MESHES = {
GENERIC_VBO: null,
DOOR_UPPER_VBO: null
DOOR_LOWER_VBO: null
}
// Declare a Door class using ES6 syntax
class Door {
// Assume X & Y are the lower half of the door
constructor(x,y,map) {
if (y - 1 > -1) {
console.error("Error: Top half of the door goes outside the map");
return;
}
this.x = x;
this.y = y;
map[x][y ] = BLOCK_TYPES.DOOR_LOWER_OPEN;
map[x][y-1] = BLOCK_TYPES.DOOR_UPPER_OPEN;
}
}

JavaScript: How do i use a string as a piece of code

I am making a simple-ish graph maker to visualise equations. I need to be able to have the user input a string in a textbox and then interpret that as a piece of code I can execute to produce the graph. The way I am displaying the graph is by going through x in small increments and using an equation to then calculate the y position and then drawing a line between the points. At the moment I am just manually making a function in the code for example:
function(val) { return (val * val) + 5; }
but I need to be able to create a similar function from a string so the user could just input something like "(x*x)+(2*x)". is there any way to do this?

Canonically, this is done with eval(), although it comes with many caveats and should probably be avoided.
There are several questions on SO that discuss eval alternatives, but in your case, I would suggest a very bare-bones parser -- especially if all you're handling are mathematical equations.

How do I create a leaflet map with thousands of marks that doesn't crash my browser?

I'm using the leaflet package in R to generate a map with a large number of circles on it. The goal is a map I can publish to my website. The problem I'm having is that as I increase the number of circles, the resulting map loads very slowly, I get "unresponsive script" warnings and ultimately it completely freezes up my browser.
I know this sort of thing is possible, because I've found a leaflet map that works the way I want mine to work:
http://cartologic.com/geoapps/map_viewer/5/ny-crimes-2014-dot-density-map
I notice on the above map that the circles don't appear "clickable" like the circles on my map, and that they seem to load in square chunks. I have a hunch that these things are related to my problem. Unfortunately, I'm too much of a novice at leaflet/javascript stuff to figure this out on my own.
Here is a toy example illustrating my problem:
library("leaflet")
library("htmlwidgets")
dots <- data.frame(x=c(runif(10000, -93.701281, -93.533053)),
y=c(runif(10000, 41.515962, 41.644369)))
m <- leaflet(dots) %>%
addTiles('http://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png') %>%
setView(-93.617167, 41.580166, zoom = 12) %>%
addCircles(~x, ~y, weight = 1, radius = 5,
color = "#FFA500", stroke = TRUE, fillOpacity = 0.1)
m
saveWidget(widget = m, file="example.html", selfcontained = TRUE)

mapview can help you here. It builds upon the leaflet library for smaller data sets, but uses special javascript functionality for larger data.
your example with 1 Mio. points:
library(mapview)
library(sp)
dots <- data.frame(x=c(runif(1000000, -93.701281, -93.533053)),
y=c(runif(1000000, 41.515962, 41.644369)))
coordinates(dots) <- ~ x + y
proj4string(dots) <- "+init=epsg:4326"
mapview(dots)
It may still take a while to render, but once rendered it should be quite responsive. Note that mapview is designed to work with spatial* objects, that is why we need the calls to set the coordinate slot and the projection.
For more information have a look here:
http://environmentalinformatics-marburg.github.io/web-presentations/20150723_mapView.html
Hope that helps.

If you want to add a large number of vector objects to a map, it is rare that it can be done easily.
Notice that the raster data is broken into tiles so that all the information does not have to be shown at one time. For your vector data (in this case, circles) you have to do the same thing.
Basically what I like to do is to break the large data set into smaller (vector) tiles, with the same boundaries as the raster tiles you are showing. Duplicate the data if you want it to appear at several zoom level. As you are showing circle, imagine that you partition the circles' center points on the tile boundary.
I have an application similar to this where I basically partition my vector data on tile boundaries and store the information in geojson files. When I get an event that the raster tile has been loaded I can then load the equivalent vector file as a geojson layer (same thing when the raster tile is unloaded). In this way, you can limit the amount of vector data that has to be displayed at any one time.
If you have a lot of points, they are not really going to be visible at low zoom levels anyway, so it might be better just to show them at an appropriate zoom level (perhaps with a different representation at low zooms - like a heat map). This will keep the amount of data being shown at any one time lower.

Since this question has a few upvotes, I'll generally describe both of the solutions I found. Maybe if I have time later I'll get all the files together on GitHub.
First, I found TileMill. Simply load a data file of coordinates into TileMill, style the way you want them to appear, and output tiles (png). Host those tiles on the web somewhere and load them with leaflet. This process was a bit too manual for my liking because TileMill kept crashing when I loaded in csv files that were too large for it to render on my machine.
I found the best solution was use Processing, adapting Robert Manduca's code here: https://github.com/rmanduca/jobmaps. I don't use Python so I rewrote those parts in R and modified the Processing code according to my specifications.

Mapdeck (released on CRAN Aug 2018) uses WebGL (through Deck.gl) and is designed to handle millions of points (depending on your system's hardware of course)
library(mapdeck)
set_token("MAPBOX_TOKEN")
n <- 1e6
dots <- data.frame(x=c(runif(n, -93.701281, -93.533053)),
y=c(runif(n, 41.515962, 41.644369)))
dots$letter <- sample(letters, size = n, replace = T)
mapdeck(
style = mapdeck_style('dark')
) %>%
add_scatterplot(
data = dots
, lon = "x"
, lat = "y"
, fill_colour = "letter"
, radius = 5
, fill_opacity = 50
, layer_id = "dots"
)

Drawing plots interactively in a web app

I am looking for a library preferably in JavaScript, that will allow a user to draw a plot (simple one consisting of vertical and horizontal steps) like this one:
The idea is that when the user is done with the plot I can generate data points from the graph and process them.
I don't know where to start, I am looking to start learning to do this within a JS based framework (meteor) but I can't find a library that allows for something like this. The closest library I found is d3.js but I couldn't find any example that allows for this.
Would anyone be able to point out to me a sample example to start from? Would you know of a better suited library to accomplish what I am asking for?

Here is a relatively simple fiddle which accomplishes some of what you asked for, excluding axis (which are relatively easy and has plenty of examples). It uses D3 for all the drawing and mouse event handling. On click it simply executes svg.append("circle").attr("r", 5), and if it's not the first click (i.e. linking points) then it also will create a path element using the previous mouse click coordinates:
svg.insert("path", "circle").attr("d", function () {
return [
"M", prevClickLoc[0], prevClickLoc[1],
"L", prevClickLoc[0], y,
"L", x, y].join(" ");
})
Where x and y are the current mouse coordinates. Also has an export button that will output a list in the form of cx,cy,cx,cy,... :: d,d,d,d,.... On import, you could easily split this array into two using indexOf("::") or whatever you choose if you want to change the formatting. Then just exectue for (x in circles) {svg.append("circle").attr("cx", function...).attr("cy", function...);} and do something similar for paths for (y in paths) {svg.append("path").attr("d", function(){return paths[y];});}. It would be even easier if on export you made the cxcy array in the format cx;cy,cx;cy since then you could simply split the array at each comma and then split each index of the resulting array at the semicolon for a nice nested array.
Small update in this version, you can only place points if the current mouse x is greater than the previous x coordinate, and it also has the line d3.event.stopPropagation(); which prevents accidental highlighting of the page.

Embed text into PNG

I'm looking for a tool that'd let me embed json-formatted information inside a PNG -file.
This far it's been quite quiet. Do I have to write it myself?
I'd be especially interested of doing it with javascript. Into an image I extract from a canvas with toDataURL -method.

If you want to embed text you may want to look at this in particular, which comes from the PNG specification. It seems a little on the complicated side.
The "easy" steganographic method looks a little more simple.
What may actually be better suited for your purpose is to create a javascript object that contains the image data and the JSON data - then just pass that object around wherever you need it.

I am completely unfamiliar with Python, however if you can access any of the prominent image processing libraries it is possible.
Take a look here for ImageMagick<->Python solutions.
Edit
You may wish to take a look at this blog post for information regarding steganography (hiding information within an image) and an offshoot of the ImageMagick library. It uses C++ here but I'm sure you could figure out a way to incorporate the base processes in Python.

Hmm, here’s a partial libpng implementation in JS: http://www.xarg.org/download/pnglib.js.

Copying another answer of mine in another question:
Here's an old not too-fancy module I did for a friend once (Python 2.x code):
the code
from __future__ import division
import math, os, array, random
import itertools as it
import Image as I
import sys
def encode(txtfn, imgfn):
with open(txtfn, "rb") as ifp:
txtdata= ifp.read()
txtdata= txtdata.encode('zip')
img= I.open(imgfn).convert("RGB")
pixelcount= img.size[0]*img.size[1]
## sys.stderr.write("image %dx%d\n" % img.size)
factor= len(txtdata) / pixelcount
width= int(math.ceil(img.size[0]*factor**.5))
height= int(math.ceil(img.size[1]*factor**.5))
pixelcount= width * height
if pixelcount < len(txtdata): # just a sanity check
sys.stderr.write("phase 2, %d bytes in %d pixels?\n" % (len(txtdata), pixelcount))
sys.exit(1)
## sys.stderr.write("%d bytes in %d pixels (%dx%d)\n" % (len(txtdata), pixelcount, width, height))
img= img.resize( (width, height), I.ANTIALIAS)
txtarr= array.array('B')
txtarr.fromstring(txtdata)
txtarr.extend(random.randrange(256) for x in xrange(len(txtdata) - pixelcount))
newimg= img.copy()
newimg.putdata([
(
r & 0xf8 |(c & 0xe0)>>5,
g & 0xfc |(c & 0x18)>>3,
b & 0xf8 |(c & 0x07),
)
for (r, g, b), c in it.izip(img.getdata(), txtarr)])
newimg.save(os.path.splitext(imgfn)[0]+'.png', optimize=1, compression=9)
def decode(imgfn, txtfn):
img= I.open(imgfn)
with open(txtfn, 'wb') as ofp:
arrdata= array.array('B',
((r & 0x7) << 5 | (g & 0x3) << 3 | (b & 0x7)
for r, g, b in img.getdata())).tostring()
findata= arrdata.decode('zip')
ofp.write(findata)
if __name__ == "__main__":
if sys.argv[1] == 'e':
encode(sys.argv[2], sys.argv[3])
elif sys.argv[1] == 'd':
decode(sys.argv[2], sys.argv[3])
the algorithm
It stores a byte of data per image pixel using: the 3 least-significant bits of the blue band, the 2 LSB of the green one and the 3 LSB of the red one.
encode function: An input text file is compressed by zlib, and the input image is resized (keeping proportions) to ensure that there are at least as many pixels as compressed bytes. A PNG image with the same name as the input image (so don't use a ".png" filename as input if you leave the code as-is :) is saved containing the steganographic data.
decode function: The previously stored zlib-compressed data are extracted from the input image, and saved uncompressed under the provided filename.
I verified the old code still runs, so here's an example image containing steganographic data:
You'll notice that the noise added is barely visible.

You can use the recently released JavaScript library steganography.js. Have a look at the showcase and the documentation for the basic usage.
The library makes use of a HTML canvas and embeds the information in the alpha channel of the image. Basically you just need to use steg.encode(message, image) and pass the JSON-data (as string) as message and the image as dataURL. To read this information from the image use steg.decode(image) and pass the image as dataURL again.
After reading the information again you get the JSON-data as string so you will have to parse it to a JavaScript object again. So besides the parsing from/to string I hope it fits your requirements.

The answers here have seriously over-engineered the question. I am sorry you had to wait 11 years for the answer. The ability to do this is standard and part of the PNG spec. Tons of apps use this today.
You can drop JSON or any other data right into PNGs. You can do this in like 3 lines of JavaScript with the npm "png-chunk-text" (https://www.npmjs.com/package/png-chunk-text)
Now you can read and write JSON inside of PNGs at lightning speed and it took you 30 seconds to develop it, and the process is standard, i.e., other tools can read it too.

Someone has already attempted to embed a Mario-like platformer game in a PNG image. Using getImageData is the key to reading out the data; the object that that method returns gives a one-dimensional array holding the individual RGBA values of each pixel.
Note that this does not "hide" the data within the area that the user sees, but you could probably use a CSS sprite like technique (or "crop" the image using multiple canvases) to show only the part of a combined image you want to.

Why not just use this tool?
http://www.jsclasses.org/package/324-JavaScript-Embed-encoded-text-in-images-using-steganography.html
and here on github:
https://github.com/blauharley/LoremImageCryptonator
It's a normal object that can be used for embedding text into each color-channel(red, green...).
When embedding text into the alpha-channel there should not be as much noise as embedding text on other color-channels, so you should not see any difference before and after inserting text into an image.