Develop Reference

Download file using nestjs from server / observable

I am using a BFF as a proxy.
I need get a excel (and pdf) file from server and forward to download from front app (that are using reactjs).
The code:
monthlyGet(): Observable<AxiosResponse<any>> {
const req: Request = RequestContext.currentContext.req;
const headers = req.headers as AxiosRequestHeaders;
const api = this.configService.get<string>('REPORTS_API');
const path = req.path;
const query = req.url.split('?')[1];
const url = `${api}${path}${query ? `?${query}` : ''}`;
delete headers.host;
return this.httpService
.get(url, { headers })
.pipe(
catchError((e) => {
throw new HttpException(e.response?.data, e.response?.status || 500);
}),
)
.pipe(map((response) => response.data));
}
The result is a corrupted file.
I've tried several ways using stream, buffer... unsuccessfully.

How to set a key in Redis and get the value (I'm building a url shortener)

I'm kind of new to Redis and I'm currently experiencing a project stand-still because I don't know any other way to set and get in Redis.
My problem is I'm building a url shortener and when the user posts (a POST request) a url to the server, I'm setting the url as the key and a nanoid generated code as the value and sending back the nanoid code to the user. But when the user sends a GET request with the url code to the server I have to check if the url is already cached and redirect the user to the url but I can't because the actual url as been set as the key not the url code so it will always return undefined. Please can you help me with this problem? Is there some other to do this? Many thanks in advance! Here is the code:
import redis from 'redis';
import http from 'http';
import express from 'express';
import { Router } from 'express';
import { promisify } from 'util';
import { nanoid } from 'nanoid';
interface Handler {
(req: Request, res: Response, next: NextFunction): Promise<void> | void;
}
interface Route {
path: string;
method: string;
handler: Handler | Handler[];
}
const { PORT = 8080} = process.env;
// I'm using a docker container
const { REDIS_URL = 'redis://cache:6379' } = process.env;
const redisClient = redis.createClient({
url: REDIS_URL
});
const initCache = async () =>
new Promise((resolve, reject) => {
redisClient.on('connect', () => {
console.log('Redis client connected');
resolve(redisClient);
});
redisClient.on('error', error => reject(error));
});
async function getShortenedURL(url: string) {
const urlCode = nanoid(7);
redisClient.setex(url, 3600, urlCode);
return urlCode;
}
const getAsync = promisify(redisClient.get).bind(redisClient);
async function getFromCache(key: string) {
const data = await getAsync(key);
return data;
}
const routes = [
{
path: '/:url',
method: 'get',
handler: [
async ({ params }: Request, res: Response, next: NextFunction) => {
try {
const { url } = params;
const result = await getFromCache(url);
if (result) {
res.redirect(301, result);
} else {
throw new Error('Invalid url');
}
} catch (error) {
console.error(error);
}
}
]
},
{
path: '/api/url',
method: 'post',
handler: [
async ({ body }: Request, res: Response, next: NextFunction) => {
const { url } = body;
const result = await getFromCache(url);
result ? res.status(200).send(`http://localhost:${PORT}/${result}`) : next();
},
async ({ body }: Request, res: Response) => {
const result = await getShortenedURL(body.url as string);
res.status(200).send(result);
}
]
}
];
const applyRoutes = (routes: Route[], router: Router) => {
for (const route of routes) {
const { method, path, handler } = route;
(router as any)[method](path, handler);
}
};
const router = express();
applyRoutes(routes, router);
const server = http.createServer(router);
async function start() {
await initCache();
server.listen(PORT, () => {
console.log(`Server is running on http://localhost:${PORT}...`)
}
);
}
start();

As I understand, you need to make sure that you do not shorten and store any given url twice.
You could encode the url and use it as the sort version and as a key at the same time. E.g.
www.someurltoshorten.com -> encoded value ->
{key: value} -> encoded value: www.someurltoshorten.com
If a user wants to shorten a url, you encode it first and you should get the exact same hash for the exact same url.
Once you get the encoded value, you can use the SET command with a "GET" option. You can also use the expire (EXAT) option to clean up old urls (those that nobody is looking for anymore) using the feature that is built in Redis.
It will do the following for you:
Set key to hold the string value (the key is the short version of the url and the value is the url itself)
If the value exists, it will overwrite it and reset (extend) the TTL (time to live) if you set it.
And the "GET" option will return the old value if it exists or null.
With one command you will be able to:
Create a value in Redis
Get the value if it already exists resetting the TTL (it makes sense to extend it) and all of the without any extra code with one command only!!!
The flow may look as follows:
A user inputs a url to be shortened:
you encode the url
you store it in Redis using the SET command where the key is the encoded value and the value is the url.
you return the encoded value which you already now. There is no need to check whether the url has already been shortened once because the SET command will either create a new entry or update the existing once.
A user inputs a shortened url
you encode the url
you store it in Redis using the SET command where the key is the encoded value and the value is the url.
you get the url from the value that was returned by the SET command thank to the "GET" option.
The only difference between the two cases is in whether you return the shortened url or the normal url
Basically, you need one Redis command for all of that to work.
I did not test the encoding/hashing of the url and it may not work with all types of url. You need to check which encoding would cover all cases.
But the idea here is the concept itself. It's similar to how we handle passwords. When you register, it's hashed. Then, when you log in and provide the same password, we can hash it again and compare hashes. Secure hashing with bycript, as an example, can be expensive (can take a lot of time).
For urls you need to make sure that encoding/hashing always produces the same result for the same url.
Keep in mind the length of the keys as describe here https://redis.io/topics/data-types-intro#redis-keys

you should use the HashCode generate for the URL as the Key for your dictionary since you intend to lookup by the shortened URL later.
Post--> Hash the URL, Encode it as per your need for length restrictions return the shortened Key as shortened URL and put <Hash,URL> in your map
Get--> User gives the shortened Key, Dictionary lookup for shortened Key and return the actual URL.

GraphQL Client with React frontend subscription not working

I started to add a websocket to my React app and I managed to handle the back-end part with nodejs just fine, working perfectly with GraphQL playground. However, I can't get my subscriptions to work from the client side.
I did everything the documentation told me to, set up my websocket and my websocket connect without any trouble.
const wsLink = new WebSocketLink({
uri: ws://myUrl,
options: {
connectionParams
}
);
function splitByOperation({ query }) {
const { kind, operation } = getMainDefinition(query);
return kind === 'OperationDefinition' && operation === 'subscription';
}
let link = ApolloLink.from([
authLink,
errorLink,
uploadLink,
stateLink,
ApolloLink.split(
// split based on operation type
splitByOperation,
wsLink,
httpLink
)
]);
const Client = new ApolloClient({
uri: '/graphql',
cache,
resolvers,
link
});
which result in :
Request URL: ws://myUrl:3000/subscription
Request Method: GET
Status Code: 101 Switching Protocols
in my console.
My main problem is : when using "useSubscription" like that :
const COMMENTS_SUBSCRIPTION = gql`
subscription OnCommentAdded {
commentAdded {
id
content
}
}
`;
const { data: { commentAdded }, loading } = useSubscription(
COMMENTS_SUBSCRIPTION
);
My data commentAdded is always null, I strongly suspect the useSubscription hook doesn't subscribe at all since my subscription resolve in the backend act like there's no one to send the data to.
Also, in the network part of my chrome dev tools there's nothing except my first interaction with the websocket when setting up apollo client.
I have no error to work with and I'm pretty much lost at this point.
The names and field requested match perfectly, nothing wrong when using my backend playground, still in my frontend playground I get this when trying to use the subscription :
Could not connect to websocket endpoint ws://myUrl/subscription. Please check if the endpoint url is correct.
I'm using apollo-client 2.6.4.

finally figured this out.
const wsLink = new WebSocketLink({
uri: 'ws://localhost:3000/subscription',
options: {
connectionParams
}
});
function splitByOperation({query}) {
const definition = getMainDefinition(query);
return (
definition.kind === 'OperationDefinition' &&
definition.operation === 'subscription'
);
}
const link = ApolloLink.from([ApolloLink.split(
// split based on operation type
splitByOperation,
wsLink,
authLink.concat(httpLink)
),
uploadLink,
stateLink,
errorLink]);
// apollo client definition
const Client = new ApolloClient({
uri: '/graphql',
cache,
resolvers,
link
});
correct syntaxe :
function splitByOperation({query}) {
const definition = getMainDefinition(query);
return (
definition.kind === 'OperationDefinition' &&
definition.operation === 'subscription'
);
}
const link = ApolloLink.from([ApolloLink.split(
// split based on operation type
splitByOperation,
wsLink,
authLink.concat(httpLink)
),
uploadLink,
stateLink,
errorLink]);
const Client = new ApolloClient({
uri: '/graphql',
cache,
resolvers,
link
});
i was using HTTP all along.

Path must be a string (require url with node.js http module)

So im trying to make an update checker that doesnt actually download the update but nvm and all im trying to is check if the version on the package.json on github is the same as the one in the app (im making it in electron)
And with this code i get a "path must be a string error" (heres an image https://gyazo.com/7b55a1cbe96b2719bb588a6591855839 )
also i did look it up many times and in fact the code to get the package.json from github with the http module was from how to require from URL in Node.js
function checkforupdate() {
var http = require('http')
const file_url = "https://raw.githubusercontent.com/FloffahDevelopments/FloffahsHub/master/package.json";
const oldpackagejson = require("package.json");
document.getElementById("checkupdate").innerHTML = 'Checking for updates</br>Please wait</br><img src="./assets/icons/gif/loading.gif" alt="loading" style="width: 10%; left: 45%;">'
http.get(file_url).then(res => res.json()).then(pack => {
if (pack.version !== oldpackagejson.version) {
document.getElementById("checkupdate").innerHTML = 'Update available!'
} else {
document.getElementById("checkupdate").innerHTML = 'No update available!'
}
});
}

This will make the request you want:
var https = require('https')
const file_url = "https://raw.githubusercontent.com/FloffahDevelopments/FloffahsHub/master/package.json"
const oldpackagejson = require("./package.json");
https.get(file_url, (res) => {
res.on('data', (d) => {
process.stdout.write(d)
})
}).on('error', (e) => {
console.log(e)
})
Some mistakes you made were: Treating http.get as a promise when it's not. It could by with the inclusion of a module like bluebird, though. You used the http module to make an https request. You did not give http.get it's parameters correctly, that is, your syntax was incorrect. You're trying to update the DOM on server-side code, you should separate client and server logic. res => res.json() does not change res to json, you'll need JSON.parse.

How to handle authorization token

I would like to add auth token to http request header every time a http request sent and if authorization fails, I want to redirect user to the login. Should I decorate Http Driver or is there a better way to do it?
I came with a solution that decorates http driver. But I'm not sure this is the correct way of doing it. Here's the code so far I have written:
import Rx from 'rx';
import {makeHTTPDriver} from '#cycle/http';
function makeSecureHTTPDriver({eager = false} = {eager: false}) {
return function secureHTTPDriver(request$) {
const httpDriver = makeHTTPDriver(eager);
const securedRequest$ = request$
.map(request => {
const token = localStorage.getItem('token');
if (token) {
request.headers = request.headers || {};
request.headers['X-AUTH-TOKEN'] = token;
}
return request;
});
const response$ = httpDriver(securedRequest$);
//todo: check response and if it fails, redirect to the login page
return response$;
}
}
export default makeSecureHTTPDriver;
Here is the code how I use makeSecureHttpDriver
const drivers = {
DOM: makeDOMDriver('#app'),
HTTP: makeSecureHttpDriver()
};

This is a little late, I don't frequent SO very much. I'd suggest using other drivers instead to avoid placing any logic in your drivers.
import storageDriver from '#cycle/storage'
import {makeHTTPDriver} from '#cycle/http'
function main(sources) {
const {storage, HTTP} = sources
const token$ = storage.local.getItem('token')
.startWith(null)
const request$ = createRequest$(sources)
const secureRequest$ = request$.withLatestFrom(token$,
(request, token) => token ?
Object.assign(request, {headers: {'X-AUTH-HEADER' : token }) :
request
)
return {HTTP: secureRequest$, ...}
}
Cycle.run(main, {
...
storage: storageDriver,
HTTP: makeHTTPDriver()
})

I'm not sure if this will help but HTTP driver is superagent under the hood so you can pass it an object like with required info like here.
But in regards to your issue I think that the HTTP driver might need this option added to the driver it self so you can dictate if the driver should be secure or not eg:
const drivers = {
DOM: makeDOMDriver('#app'),
HTTP: makeSecureHttpDriver({secure:true})
};
Because your implementation looks ok to me, it might be worth having it in the driver itself.
I'd create an issue in the HTTP driver repo and see what the community think, you can also ask people to interact via the gitter channel :-)