Given the following db structure https://drawsql.app/sensor_network/diagrams/db, I would like to get all sensor_data from a certain location, while grouping the response using station.location, sensor_data.time.
I have the following query:
select
station.location_name,
sensor_data.time,
sensor.type,
sensor_data.value,
sensor.id
from station
inner join
(sensor inner join sensor_data on sensor.id = sensor_data.sensor_id)
on station.sensor_id = sensor.id
where station.location = ?
which gives me the rows I want, however, I would like to format the response in a way where all the rows which share location and time are in the same row. an example output is
location
time
type
value
id
city1
2022-01-01 08:00:00
temperature
290
1
city1
2022-01-01 09:00:00
temperature
292
1
city1
2022-01-01 08:00:00
ph
7
2
city1
2022-01-01 09:00:00
ph
8
2
which I would like to format either like (or similar to)
[
{"location": "city1", "time": "2022-01-01 08:00:00", "temperature": 290, "ph": 7},
{"location": "city1", "time": "2022-01-01 09:00:00", "temperature": 292, "ph": 8},
]
or
[
{"location": "city1", "time": "2022-01-01 08:00:00", "sensors": [{"id": 1, "type": "temperature", "value": 290}, {"id": 2, "type": "ph", "value": 7}]},
{"location": "city1", "time": "2022-01-01 09:00:00", "sensors": [{"id": 1, "type": "temperature", "value": 292}, {"id": 2, "type": "ph", "value": 8}]}
]
Currently, I am doing this formatting in javascript using array functions, but this has proven to be too slow. I have tried using group by query but to be fair, I am struggling to understand how to use that when you don't want to use methods like count etc.
I am using nodejs and its mysql package, the database is mysql:8 and tables are running with innodb engine
You can achieve this with subqueries.
SELECT
st.location,
sd.time,
(
SELECT sd2.value
FROM sensor_data sd2
INNER JOIN sensor s2
ON s2.id = sd2.sensor_id
WHERE sd2.time = sd.time
AND s2.type = "temperature"
) AS temperature,
(
SELECT sd2.value
FROM sensor_data sd2
INNER JOIN sensor s2
ON s2.id = sd2.sensor_id
WHERE sd2.time = sd.time
AND s2.type = "ph"
) AS ph
FROM station st
INNER JOIN sensor s
ON st.sensors = s.id
INNER JOIN sensor_data sd
ON s.id = sd.sensor_id
WHERE st.location = ?
GROUP BY sd.time
Obviously, this will work only if you know the list of types (temperature, ph) in advance and thus you can write separate subqueries for each of them.
If you don't want to build separate subquery for each type, you can concat the values into a single subquery column.
SELECT
st.location,
sd.time,
(
SELECT
GROUP_CONCAT(
CONCAT(s2.type, ':', sd2.value)
SEPARATOR ','
)
FROM sensor_data sd2
INNER JOIN sensor s2
ON s2.id = sd2.sensor_id
WHERE sd2.time = sd.time
GROUP BY sd2.time
) AS sensors
FROM station st
INNER JOIN sensor s
ON st.sensors = s.id
INNER JOIN sensor_data sd
ON s.id = sd.sensor_id
WHERE st.location = ?
GROUP BY sd.time;
Related
I am new to SQL and wonder how to select nested tables.
I have two tables like this:
sensors
sensor_id
model_no
location_id
int
varchar
int
locations
location_id
name
location
radius
int
varchar
point
int
They are linked with foreign key. Currently, I select using
SELECT sensors.*, locations.*
FROM sensors INNER JOIN locations
ON sensors.location_id = locations.location_id;
to get the data from both like this:
{
"sensor_id": 1,
"model_no": "some string",
"location_id": 2,
"name": "Berlin",
"location": {
"x": 3,
"y": 3
},
"radius": 1000
}
I wonder if there is any way I can keep the location data grouped as its own object like this:
{
"sensor_id": 1,
"model_no": "some string",
"location": {
"name": "Berlin",
"location": {
"x": 3,
"y": 3
},
"radius": 1000
}
}
I am using MySQL 8 with mysql npm package to execute the queries. I know I can modify the response using javascript but wonder if it can be done directly in the query, and if so, is it better or worse for performance?
SELECT JSON_OBJECT(
'sensor_id', sensor_id,
'model_no', model_no,
'location', JSON_OBJECT(
'name', name,
'location', JSON_OBJECT(
'x', CAST(ST_X(location) AS SIGNED),
'y', CAST(ST_Y(location) AS SIGNED)
),
'radius', radius
)
) as JSON_output
FROM sensors
JOIN locations USING (location_id);
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=b17dfa3069b4bb9345a9e99e8b893121
You must manually label each column; you cannot get the desired result and use *.
A small consolation prize: If you change
ON sensors.location_id = locations.location_id;
to
USING(location_id)
There will be only one location_id column.
In the frontend there is a list of game titles that consist of strings:
id, game name, date, price
The search shall accept multiple keywords, e.g. user might type: 1998 Streetfighter 2 or Streetfighter 1998 Currently I create an array separated by empty space, that creates 3 keywords: [1998, Streetfighter , 2 ] Then I go through the collection of game titles to filter matches. unfortunately it also gives back any title that includes "2" because there is no pattern recognition that identifies "Streetfighter 2" belongs together. Is there a simple algorithm to provide a pattern search?
const allGames = [
"Streetfighter 1, 1992, 20",
"Streetfighter 2, 1998, 20",
"pokemon, 2016, 20",
"Diablo 3, 2015, 40",
"Super mario, 1995, 20",
"The Witcher, 2012, 20",
]
Your search query looks advanced enough to justify using a search engine. Just don't create one yourself (it's harder than you may think).
In this answer I'll be using Lunr.js
Here's a 2min crash course:
Transform your data into documents. (I've already converted your initial allGames array.)
Then create a search index where you:
Specify which property of a document holds a unique identifier. (In our case title.)
Define which properties should be indexed. (In our case all of them.)
Define a boost score for each property. (i.e a match in the title has a higher relevance score than a match in the price.)
Add all documents to the search index.
Search! ;)
📢 Search Query FTW!
Notice the last search, I'm using a wildcard in the search string (street*) to find the two Street Fighter titles!
const createLunrIndex = docs =>
lunr(function () {
this.ref('title');
this.field('title', 5);
this.field('date', 3);
this.field('price', 1);
for (doc of docs) this.add(doc);
});
const search = (lunrIndex, term) =>
lunrIndex
.search(term)
.map(res => res.ref)
const gamesIndex = createLunrIndex(allGames);
console.log(
search(gamesIndex, '1998 Streetfighter 2')
);
console.log(
search(gamesIndex, 'Streetfighter 1998')
);
console.log(
search(gamesIndex, 'street*')
);
<script src="https://unpkg.com/lunr/lunr.js"></script>
<script>
const allGames =
[ { "title": "Streetfighter 1"
, "date": "1992"
, "price": "20"
}
,
{ "title": "Streetfighter 2"
, "date": "1998"
, "price": "20"
}
,
{ "title": "pokemon"
, "date": "2016"
, "price": "20"
}
,
{ "title": "Diablo 3"
, "date": "2015"
, "price": "40"
}
,
{ "title": "Super mario"
, "date": "1985"
, "price": "20"
}
,
{ "title": "The Witcher"
, "date": "2012"
, "price": "20"
}
]
</script>
The keys and values are separated in the Json object that I get from an api call. I have tried finding a solution It looks like the following:
{
"range": "'1'!A1:AM243",
"majorDimension": "ROWS",
"values":
[
"DeptID",
"DeptDescr",
"VP Area",
"VP Descr",
"HR Category",
"Employee Relations1",
"ER1Title",
"ER1Phone",
"ER1Email",
"Employee Relations2",
"ER2Title",
"ER2Phone",
"ER2Email",
"Compensation1",
"Comp1Title",
"Comp1Phone",
"Comp1Email",
"Compensation2",
"Comp2Title",
"Comp2Phone",
"Comp2Email",
"Employment1",
"E1Title",
"E1Phone",
"E1Email",
"Employment2",
"E2Title",
"E2Phone",
"E2Email",
"Employee Pay Services1",
"EPS1Title",
"EPS1Phone",
"EPS1Email",
"Employee Pay Services2",
"EPS2Title",
"EPS2Phone",
"EPS2Email"
],
[
"20734",
"Academic Success Centers",
"VES",
"VP Enroll Mgmt & Student Aff",
"Administrative",
"Brian Schmidt",
" Employee Relations Consultant",
"(928)523-6139",
"Brian.Schmidt#nau.edu",
"Marcia Warden",
"Assistant Director, Employee Relations",
"(928)523-9624",
"Marcia.Warden#nau.edu",
"Nicole Christian",
"Employment & Compensation Analyst",
"(928)523-6127",
" Nicole.Christian#nau.edu",
"Cathy Speirs",
"Associate Director",
"(928)523-6136",
"Cathy.Speirs#nau.edu",
"Nicole Christian",
"Employment & Compensation Analyst",
"(928)523-6127",
" Nicole.Christian#nau.edu",
"Cathy Speirs",
"Associate Director",
"(928)523-6136",
"Cathy.Speirs#nau.edu",
"Katherine Kurpierz",
"Payroll Specialist",
"(928)523-6129",
"Katherine.Kurpierz#nau.edu",
"Cheryl Brothers",
"Assistant Director - HR Payroll Services",
"(928)523-6085",
"Cheryl.Brothers#nau.edu"
], etc.
But I need it to look like:
[
{
"DeptID": 20734,
"DeptDescr": "Academic Success Centers",
"VP Area": "VES",
"VP Descr": "VP Enroll Mgmt & Student Aff",
"HR Category": "Administrative",
"Employee Relations1": "Brian Schmidt",
"Employee Relations2": "Marcia Warden",
"Compensation1": "Nicole Christian",
"Compensation2": "Cathy Speirs",
"Employment1": "Nicole Christian",
"Employment3": "Cathy Speirs",
"Employee Pay Services1": "Katherine Kurpierz",
"Employee Pay Services2": "Cheryl Brothers"
},etc
I am trying to use the data to populate a drop down using javascript and ajax. Any help is really appreciated.
The object your API returns is not a valid JSON. Was that API made by you or can you get that fixed somehow?
There are 2 things you could do to make it work
-One is change it to return exactly what you want;
-Two is to fix what it returns so that it is a valid JSON;
Going for what is wrong with the file you initially posted, let's remove the contents of the arrays so it's easier to spot the problem:
Your original data looks roughly like this:
{ "range": "'1'!A1:AM243",
"majorDimension": "ROWS",
"values": [],[]
}
To be valid you would need it to look like this:
{ "range": "'1'!A1:AM243",
"majorDimension": "ROWS",
"values": {
"keys": [],
"data": [],
}
}
Notice that I wrapped the two arrays of "values" with { } because it has to be an object if you want it to contain two arrays in it.
Then I gave each array a key with which you can call them. With that you'd be able to get what you want from your "values", so that for each item in the "keys" array you have something in that "data" array.
Hope this helps.
Well let's have a look;
Suppose this is a short version of the response data you got:
var res = `
{
"range": "'1'!A1:AM243",
"majorDimension": "ROWS",
"values": [
"DeptID",
"DeptDescr",
"VP Area"
],
[
"20734",
"Academic Success Centers",
"VES"
],
[
"345543",
"Academic Fails Centers",
"OK"
]
}
`;
As we can see by the first data, looks like a dump from a spreadsheet of sorts, and someone maybe scripted a way to export this data in a JSON-ish way. The values "Array" are the rows of this "spreadsheet".
We will clean it up, and get only the chunks that looks like ["value", "another value", "etc"]
// clean tabs and returns
res = res.replace(/\t/g, '').replace(/\n/g, '');
// get the array-ish chunks
rows = res.match(/\[(((["'])(?:(?=(\\?))\4.)*?\3),*)+\]/gm)
now let's make them real arrays:
var data = rows.map(function (row) {
return JSON.parse(row);
});
Now we have an array of arrays of strings. that means, an array of "rows" that contains the values of the "cells". The first one looks like the header row (the one with the names of the fields)
Lets make objects using each row of data except the first one. The first will serve us as the keys (we match the position (index) of the value from rows[n] from the value on rows[0] to get a key-value pair)
// Here we will define an object to store data
var data_object = { values: [] };
// for each row except the first
for(var i = 1; i < data.length; i++) {
var my_data = {};
//for each element of this row
for(var j = 0; j < data[i].length; j++) {
my_data[data[0][j]] = data[i][j];
}
data_object.values.push(my_data);
}
We have our object, let's suppose you need it in JSON format now:
var json_data = JSON.stringify(data_object);
// let's look what we have here
console.log('json_data:', json_data);
We will look at something like this as a result:
json_data: {"values":[{"DeptID":"20734","DeptDescr":"Academic Success Centers","VP Area":"VES"},{"DeptID":"345543","DeptDescr":"Academic Fails Centers","VP Area":"OK"}]}
NOW A WARNING:
This is what you DON'T want to do if you can fix the API you are getting this data from first. If any inconsistency appears, things will break. and in this example i'm not managing any edge case or exception, neither checking boundaries of arrays or wrapping things in try-catch blocks.
trying to locate the word 'London' from a generated JSON array entered from a user.
response 200 (API example):
{
"latitude": 52.24593734741211,
"longitude": -0.891636312007904,
"addresses":
["10 Watkin Terrace, , , , , Northampton, Northamptonshire",
"12 Watkin Terrace, , , , , Northampton, Northamptonshire",
i wrote a function:
request.onload = function() {
var data = JSON.parse(this.response);
if(request.status >= 200 && request.status < 400) {
var city = data.addresses;
var london = 'London';
var check = city.includes(london);
if(check) {
console.log(city);
} else {
console.log('not in london');
var show = document.getElementById('incorrect_postcode').style.visibility='visible';
I'm unsure if I misunderstand the API or if somethings wrong here... the statement is always false.
Also tried for loop to search the length of the array to locate 'London', to no avail
Thanks.
The addresses array is a list of strings, each string being an address. So, none of the strings will exactly equal 'London', because there is more to an address than the city.
You could loop through the array of addresses and check if each string contains 'London', but that would also match any addresses where London is in the street name.
The better way would be to loop through the array of addresses and parse each address string to pull out the city value. Then compare the city value to 'London'.
Note: I am assuming the second-to-last value is the city.
const data = {
"latitude": 52.24593734741211,
"longitude": -0.891636312007904,
"addresses": [
"10 Watkin Terrace, , , , , Northampton, Northamptonshire",
"12 Watkin Terrace, , , , , Northampton, Northamptonshire",
"221B Baker Street, , , , , London, "
]
};
let desiredCity = 'London';
// array index of city after splitting the addresses
const CITY_POSITION = 5;
console.log('Addresses in ' + desiredCity + ':');
for(let i = 0; i < data.addresses.length; i++) {
// split the address on the delimiter, ', '
let addr = data.addresses[i].split(', ');
if(addr[CITY_POSITION] === desiredCity) {
console.log(data.addresses[i]);
}
}
If you simply want true/false check, you can go, like:
var input = {
"latitude": 52.24593734741211,
"longitude": -0.891636312007904,
"addresses":[
"10 Watkin Terrace, Northampton, Northamptonshire",
"12 Watkin Terrace, Northampton, Northamptonshire",
//"Baker Street, London, England",
"Londonderry, Ireland"
]
};
const isLondon = obj => obj.addresses.some(addr => addr.match(/, London,/));
console.log(isLondon(input));
Welcome to SO
If we assume the JSON request is working, then no need to include that part.
The reason your example does not work, is that includes is looking for the complete string
Instead loop over the array and look at each string
NOTE: If you use search or indexOf, you may find London in Londonderry
var response = `{
"latitude": 52.24593734741211,
"longitude": -0.891636312007904,
"addresses":
[
"10 Watkin Terrace, , , , , Northampton, Northamptonshire, NN1 1ED",
"Bishop Street, , , , , Londonderry, BT48 6PQ",
"12 Watkin Terrace, , , , , Northampton, Northamptonshire, NN1 1ED",
"10 Downing Street, , , , , LONDON, SW1A 2AA"
]
}`
var data = JSON.parse(response);
var addrs = data.addresses;
console.log(addrs)
var city = 'London'.toLowerCase();
var check = addrs.filter(function(addr) {
var thisCity = addr.split(",")[5].toLowerCase().trim()
console.log(city,thisCity)
return thisCity === city
})
if (check.length > 0) {
console.log("Yes, in "+city,check);
} else {
console.log('not in london');
}
Here is a simple search
var data ={
"latitude": 52.24593734741211,
"longitude": -0.891636312007904,
"addresses":
["10 Watkin Terrace, , , , , Northampton, Northamptonshire",
"12 Watkin Terrace, , , , , Northampton, Northamptonshire",
"london",
"londonderry, Ireland",
"London Road"
]}
var searchFor = "london";
console.log(data.addresses.filter((item) => {
return item.toLowerCase().split(" ").indexOf(searchFor.toLowerCase()) != -1
}))
I am new to JavaScript, so please excuse my ignorance of basics and proper vocabulary.
I have a list of objects all containing specific information/variables/elements (latitde and longitude) which I would like to have in separate arrays.
var data = [{
"0": "44",
"latitude": "44",
"1": "11",
"longitude": "11"
}, {
"0": "45",
"latitude": "45",
"1": "12",
"longitude": "12"
}, {
"0": "46",
"latitude": "46",
"1": "13",
"longitude": "13"
}, {
"0": "47",
"latitude": "47",
"1": "14",
"longitude": "14"
}];
I know already that I can access specific values easily:
data[1].latitude
data[1].longitude
But how do I put them together to get something like this? :
var latitude = [44, 45, 46, 47]
var longitude = [11, 12, 13, 14]
You could loop through the elements of the data array and add the desired values to other arrays that will hold the result:
var latitudes = [];
var longitudes = [];
for (var i = 0; i < data.length; i++) {
latitudes.push(data[i].latitude);
longitudes.push(data[i].longitude);
}
// at this stage the latitudes and longitudes arrays
// will contain the desired values
Also in your data array, the latitudes and longitudes are strings whereas in your expected arrays you want integers. So you might need to parse the values using the parseInt function before adding them to the resulting arrays.
latitudes.push(parseInt(data[i].latitude, 10));
longitudes.push(parseInt(data[i].longitude, 10));
or with the parseFloat function if those strings could represent decimal numbers (which is more realistic for latitudes and longitudes).
Very similar to #Darin Dimitrov, but using .map() of array
Array.prototype.map() Creates a new array with the results of calling a provided function on
every element in this array.
Updates: It should have been like this
var lat = data.map( function(item) {
return item.latitude;
});
var lon = data.map( function(item) {
return item.longitude;
});
JSFiddle