How can I load and process a 3D image volume, saved as an hdf5 file, directly in the frontend using JS?
This can be achieved using jsfive and numjs.
The following code cuts an 124x124 image, at z=10, y0=0, y1=124, x0=0, x1=124, from a volume of dimension 20x1400x700 (z,y,x). The 3D volume is stored in the h5 file under the key 'main'. The implementation uses a callback that provides more flexibility and makes the h5 file available outside the async function.
$(document).ready(function() {
$("#datafile").change(async function loadData() {
const reader = new FileReader();
let file = $("#datafile")[0].files[0]
let file_name = file.name
reader.readAsArrayBuffer($("#datafile")[0].files[0]);
reader.onload = () => storeResults(reader.result, file_name);
})
// callback function
function storeResults(result, file_namet) {
f = new hdf5.File(result, file_namet);
let array = f.get('main').value
// jsfive can only return 1D arrays from a read operation -> use numjs to reconstruct the 3D volume
array = nj.array(array).reshape(20, 1400, 700)
// slicing a 124x124 image from the volume
// use reshape to drop the channel dimension 1x124x124 -> 124x124
let img = array.slice([10, 11], [0, 124], [0, 124]).reshape(124, 124)
// convert to image and save to canvas
let resized = nj.images.resize(img, 124, 124)
let $original = document.getElementById('original');
$original.width = 124;
$original.height = 124;
nj.images.save(resized, $original);
console.log("done")
}
})
<!DOCTYPE html>
<html lang="eng">
<head>
<!-- Import JQuery -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<!-- Import JSFive -->
<script src="https://cdn.jsdelivr.net/npm/jsfive#0.3.7/dist/hdf5.js"></script>
<!-- Import NumJs-->
<script src="https://cdn.jsdelivr.net/gh/nicolaspanel/numjs#0.15.1/dist/numjs.min.js"></script>
</head>
<body>
<div>
<h3>Original image (h<span id="h"></span>, w<span id="w"></span>)</h3>
<canvas id="original" width=64 height=64></canvas>
</div>
<input type="file" id="datafile" name="file">
<!-- Import main JS -->
<script src="app.js"></script>
</body>
</html>
I am working on Yolov3, OpenCV,python and Flask. Here, I explain the details of usage for each item.
Yolov3 - To detect and recognize object in video input
Opencv - To capture images of detected object in video input.
Flask - As a web server since it's support python language.
My objective
To develop application which able to captured the image of the object and update in the flask web directly or real-time
For your information, currently my system able to capture image and save in one folder which name as images by using OpenCv and python. you can refer the code below.
opencv and python code to capture image of object detected
for i in range(len(boxes)):
if i in indexes:
x,y,w,h = boxes[i]
label = str(LABELS[class_ids[i]])
confidence= confidences[i]
color = colors[class_ids[i]]
crop_img = image[y:y + h, x:x + w]
imagesPath = "images/file_%d.jpg"%self.d
cv2.imwrite(imagesPath, crop_img)
self.d+=1
cv2.rectangle(image,(x,y),(x+w,y+h),color,2)
cv2.putText(image,label+" "+str(round(confidence,2)),(x,y+30),font,1,(255,255,255),2)
When the images successfully save in images folder. I convert all the images in to base64 and return as json in flask. Here I attach the python code in flask.
Covert image to base64 and render in html templates
#app.route("/images")
def get_images():
directory = os.listdir('C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images')
os.chdir('C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images')
image_list= list()
for file in directory:
data = dict()
base = os.path.basename(file)
data["label"] = base
open_file = open(file,'rb')
image_read = open_file.read()
image_64_encode = base64.encodebytes(image_read)
data["data"] = image_64_encode.decode('ascii')
image_list.append(data)
final_data = {'files':image_list}
return render_template('images.html', final_data=final_data)
images.html
<!DOCTYPE html>
<html>
<head>
<!DOCTYPE html>
<html lang="en">
<head>
<title>yolo</title>
</head>
<body>
<h1 class="logo">Results</h1>
<ul>
{% for data in final_data.files %}
<li>{{data.label}}</li>
<img alt="embedded" src="data:image/jpg;base64,{{data.data}}"/>
{% endfor %}
</ul>
</body>
</html>
Problem and question
I have two major problem here.
1) When i run my application, my application captured the images welly. But it's automatically stop when image.html page is open to display the picture. Why it's happen and how to solve this?
2) If the application able to capture image even i open the image.html page. What should i do to update my web directly or real-time?
Here I attach the full code. Because I have been works for many weeks but still not found the solution. Hope someone can helps. Let's me know if u need more information
code
App.py
from flask import Flask, render_template, Response,jsonify
from camera import VideoCamera
import numpy as np
import os
import time
import detect as dt
from PIL import Image
import cv2
import base64
import json
from pprint import pprint
app = Flask(__name__)
def success_handle(output, status=200, mimetype='application/json'):
return Response(output, status=status, mimetype=mimetype)
#app.route("/images")
def get_images():
directory = os.listdir('C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images')
os.chdir('C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images')
flist = list()
for file in directory:
data = dict()
base = os.path.basename(file)
data["label"] = base
open_file = open(file,'rb')
image_read = open_file.read()
image_64_encode = base64.encodebytes(image_read)
data["data"] = image_64_encode.decode('ascii')
flist.append(data)
final_data = {'files':flist}
return render_template('images.html', final_data=final_data)
#app.route('/')
def index():
return render_template('index.html')
def gen(camera):
frame_id = 0
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
#app.route('/video_feed')
def video_feed():
return Response(gen(VideoCamera()),
mimetype='multipart/x-mixed-replace; boundary=frame')
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
camera.py
import cv2
import time
import os
import numpy as np
font = cv2.FONT_HERSHEY_PLAIN
starting_time= time.time()
frame_id = 0
count = 0
d = 0
labelsPath = os.path.sep.join(["yolo-coco", "coco.names"])
weightsPath = os.path.sep.join(["yolo-coco", "yolov3.weights"])
configPath = os.path.sep.join(["yolo-coco", "yolov3.cfg"])
#imagesPath = 'C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images/file_%d.jpg"%d'
labelsPath = os.path.sep.join(["yolo-coco", "coco.names"])
VideoPath = os.path.sep.join(["videos", "highway.mp4"])
LABELS = open(labelsPath).read().strip().split("\n")
net = cv2.dnn.readNet(configPath, weightsPath)
layer_names = net.getLayerNames()
outputlayers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]
colors= np.random.uniform(0,255,size=(len(LABELS),3))
class VideoCamera(object):
def __init__(self):
# Using OpenCV to capture from device 0. If you have trouble capturing
# from a webcam, comment the line below out and use a video file
# instead.
self.video = cv2.VideoCapture(VideoPath)
self.frame_id = 0
self.d = 0
# If you decide to use video.mp4, you must have this file in the folder
# as the main.py.
# self.video = cv2.VideoCapture('video.mp4')
def __del__(self):
self.video.release()
def get_frame(self):
success, image = self.video.read()
# We are using Motion JPEG, but OpenCV defaults to capture raw images,
# so we must encode it into JPEG in order to correctly display the
# video stream.
self.frame_id+=1
#print(frame_id)
height,width,channels = image.shape
#print (frame.shape)
#detecting objects
blob = cv2.dnn.blobFromImage(image,0.00392,(320,320),(0,0,0),True,crop=False) #reduce 416 to 320
net.setInput(blob)
outs = net.forward(outputlayers)
#print(outs)
print(outs[1])
#Showing info on screen/ get confidence score of algorithm in detecting an object in blob
class_ids=[]
confidences=[]
boxes=[]
for out in outs:
#print(out)
for detection in out:
scores = detection[5:]
class_id = np.argmax(scores)
confidence = scores[class_id]
print(confidence)
if confidence > 0.8:
#object detected
center_x= int(detection[0]*width)
center_y= int(detection[1]*height)
w = int(detection[2]*width)
h = int(detection[3]*height)
#cv2.circle(img,(center_x,center_y),10,(0,255,0),2)
#rectangle co-ordinaters
x=int(center_x - w/2)
y=int(center_y - h/2)
#cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
boxes.append([x,y,w,h]) #put all rectangle areas
confidences.append(float(confidence)) #how confidence was that object detected and show that percentage
class_ids.append(class_id) #name of the object tha was detected
indexes = cv2.dnn.NMSBoxes(boxes,confidences,0.4,0.6)
# result = open('C:/Users/HP/Miniconda3/envs/count_vechicle/coding/images/frame%04d.txt'%(count), 'w')
for i in range(len(boxes)):
if i in indexes:
x,y,w,h = boxes[i]
label = str(LABELS[class_ids[i]])
# cv2.imwrite(label, crop_img)
confidence= confidences[i]
color = colors[class_ids[i]]
crop_img = image[y:y + h, x:x + w]
imagesPath = "images/file_%d.jpg"%self.d
cv2.imwrite(imagesPath, crop_img)
self.d+=1
cv2.rectangle(image,(x,y),(x+w,y+h),color,2)
cv2.putText(image,label+" "+str(round(confidence,2)),(x,y+30),font,1,(255,255,255),2)
elapsed_time = time.time() - starting_time
fps=frame_id/elapsed_time
cv2.putText(image,"FPS:"+str(round(fps,2)),(10,50),font,2,(0,0,0),1)
# cv2.imshow("Image",image)
# key = cv2.waitKey(1) #wait 1ms the loop will start again and we will process the next frame
# if key == 27: #esc key stops the process
# break;
ret, jpeg = cv2.imencode('.jpg', image)
return jpeg.tobytes()
index.html
<html>
<head>
<title>Object Detection</title>
</head>
<body>
<h1>Video Streaming Demonstration</h1>
<img src="{{ url_for('video_feed') }}">
</body>
</html>
images.html
<!DOCTYPE html>
<html>
<head>
<!DOCTYPE html>
<html lang="en">
<head>
<title>yolo</title>
</head>
<body>
<h1 class="logo">Results</h1>
<ul>
{% for data in final_data.files %}
<li>{{data.label}}</li>
<img alt="embedded" src="data:image/jpg;base64,{{data.data}}"/>
{% endfor %}
</ul>
</body>
</html>
I am trying to run a python script. I have an input field in html which will pass the values to js file. Then I used python-shell to pass the values in a form of arguments to the python script to execute them. I am using express.js for scripting
Here is my temp.ejs file
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<script src="./index.js"></script>
<title>Document</title>
</head>
<body>
<img src="./my_plot.png">
<form>
<input type="text" id="source" />
<button onclick="get_data();">Go!</button>
</form>
</body>
</html>
Index.js
function get_data() {
var source = document.getElementById("source").value;
alert(source);
var options = {
args: [source],
pythonPath: '/usr/bin/python3'
};
var tests = new PythonShell('hello.py', options);
tests.on('message', function (message) {
// received a message sent from the Python script (a simple "print" statement)
console.log(message);
});
}
python file
import matplotlib.pyplot as plt
import numpy as np
import wave
import sys
CHANNELS = 2
swidth = 4
Change_RATE = 2
T = int(sys.argv[1])
#print(source)
q = 0.5
Q = 1/10
A = 1 # amplitude of signal # quatization stepsize
N = 2000
spf = wave.open('sample.wav', 'rb')
RATE=spf.getframerate()
Byte = spf.getsampwidth()
signal = spf.readframes(1024)
signal = np.fromstring(signal, 'Int16')
def uniform_midtread_quantizer(x, Q):
# limiter
x = np.copy(x)
idx = np.where(np.abs(x) >= 1)
x[idx] = np.sign(x[idx])
# linear uniform quantization
xQ = Q * np.floor(x/Q + 1/2)
return xQ
def plot_signals(x, xQ, T):
e = xQ - x
plt.figure(figsize=(10,6))
plt.plot(signal, label=r'quantized signal $x_Q[k]$')
plt.xlabel(r'$k$')
plt.axis([0, N, -T, T])
plt.grid()
plt.savefig('my_plot.png')
# generate signal
x = signal
# quantize signal
xQ = uniform_midtread_quantizer(x, Q)
# plot signals
plot_signals(x, xQ, T)
When I am doing node index.js in terminal then I am getting the desired output but not when I pass the value through html (so no error with python and js file)
Express.js is not causing the error as i have used vanilla ja also but same error occured
I try to draw circular gauge using jQuery and circularchart and I'm able to make it.
I want the setInterval() function of javascript to auto-refresh the value so that the gauge value keep update by itself without manual refresh.
But the setinterval() function is not working at all.
I don't want to refresh the whole page or the body of html.
I just want to refresh the particular circleChart#0 function.
Your help is needed.
This is circle.html
<body>
<div class="circleChart" id="0"></div>
<div class="circleChart" id="1" data-value="77"></div>
<script src="{{ url_for('static', filename='jquery-1.12.4.min.js') }}"></script>
<script src="{{ url_for('static', filename='circleChart.js') }}"></script>
<script>
$(".circleChart#1").circleChart();
$(".circleChart#0").circleChart({
size: 200,
value: {{temperature}},
text: 0,
onDraw: function(el, circle) {
circle.text(Math.round(circle.value) + "'C");
}
});
setInterval(function() {
$("#0").circleChart({
value: {{temperature}},
onDraw: function(el, circle) {
circle.text(Math.round(circle.value) + "'C");
}
});
}, 2000);
</script>
</body>
</html>
This is my python code (main.py)
#!/usr/bin/python
from flask import Flask, render_template
app = Flask(__name__)
import os
import glob
import time
os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')
base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')[0]
device_file = device_folder + '/w1_slave'
def read_temp_raw():
f = open(device_file, 'r')
lines = f.readlines()
f.close()
return lines
def read_temp():
lines = read_temp_raw()
while lines[0].strip()[-3:] != 'YES':
time.sleep(0.2)
lines = read_temp_raw()
equals_pos = lines[1].find('t=')
if equals_pos != -1:
temp_string = lines[1][equals_pos+2:]
temp_c = float(temp_string) / 1000.0
temp_f = temp_c * 9.0 / 5.0 + 32.0
return temp_c, temp_f
#app.route("/")
def main():
temperature , humidity = read_temp()
templateData = {
'temperature' : temperature,
'humidity': humidity
}
return render_template('circle.html', **templateData)
if __name__ == "__main__":
app.run(host='0.0.0.0', port=80, debug=True)
Assuming {{temperature}} is some kind of template variable, those are (typically) only evaluated once when generating the page from your template.
You will need some kind of AJAX call to fetch the updated temperature value.
I am trying to make auto filling graph with pygal on flask, I have tried the setInterval from jQuery in Javascript and text values are refreshing correctly, but still can't make autorefresh of the graph. Here are my sources:
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script type=text/javascript>
setInterval(
function()
{
$.getJSON(
'/_data',
{},
function(data)
{
$("#x-value").text('X: '+data.xval);
$("#y-value").text('Y: '+data.yval);
$("#z-value").text('Z: '+data.zval);
$("#chart").text(data.chart)
});
},
500);
</script>
...
<div id=x-value>X</div>
<div id=y-value>Y</div>
<div id=z-value>Z</div>
</div>
<div class="col-lg-6">
<object id="chart" type="image/svg+xml"></object>
</div>
I have tried many combinations of the "Object" and div tags, nothings's working..
Here is my python:
def data():
data = get_last()
charts = chart()
return jsonify(xval=data['X'],
yval=data['Y'],
zval=data['Z'],
chart=charts)
def chart():
data = get_last_hun()
chart = line_route(data)
return chart
def get_last():
vals = db.get_last()
return vals
def get_last_hun():
arr = {'X':[],'Y':[],'Z':[]}
latest = db.get_last()['id']
start = 1
if latest > 100:
start = latest-100
while start<=latest:
arr['X'].append(db.get_val(start)['X'])
arr['Y'].append(db.get_val(start)['Y'])
arr['Z'].append(db.get_val(start)['Z'])
start = start + 1
return arr
def line_route(data):
chart = pygal.Line()
chart.add('X', data['X'])
chart.add('Y', data['Y'])
chart.add('Z', data['Z'])
chart.render(is_unicode=True)
return render_template('home.html',chart = chart)
Do you know maybe some other way how to make it working, please?
Thank you very much!