I'm trying to argue the fact as to a project needs to capture device type into an eVar/sProp. Obviously, the technology report via SiteCat will pull this information, and I can see some use for it; however, it's still requested.
The question ==
I can pull the user-agent but I need it to be more granular, other than the entire UA string - looking specifically at the device type.
I'm not able to find any information around dong this using the dynamic variable (e.g. D=device type).
Has anyone evert tried capturing this?
Mobile Device and Device Type, as you see it in Adobe Analytics, uses a lookup table from DeviceAtlas for display in reporting. The specifics on what you're looking for is exactly why DeviceAtlas exists; to map user agents into actual readable devices, which is maintained and updated by them.
Unfortunately there's no value to 'piggyback' onto - however, there's nothing stopping you from independently subscribing to DeviceAtlas's services to obtain access to that data.
Related
Right now we have javascript application that gets pulled into a WKWebView. This application fires out messages via events. We wrote an intermediary layer that listens for those events, does some parsing and delivers the result to iOS to do some native work.
This works pretty well; however, we are a little worried (and have already seen) type safety issues arise. If the JS API changes then iOS needs to adapt and there is no contract in place really to confirm everything will remain working.
We came up with 2 ideas:
more unit tests to enforce the contract
using some sort of protocol buffer type strategy to ensure schema
Is there a better way? Is there a good protocol buffer example that does something like this?
One thing you could try is send all primitives as strings, and then parse them based on the key name. e.g. numUnits will always be an int, or maybe make some naming convention that would make it obvious what type the value is.
You could even make each value consist of 2 key value pairs, one for the value, one for the type. That would bloat the JSON but would make it type safe.
e.g. {"numUnits":{"value":"5", "type":"int"}}
I've got a web app which gets a couple dozen items at boot. All these items are JSON and are smaller then 1kb.
Now there are a number of storage options as seen in the Question.
I was thinking of just storing these objects inside a variable in the browser JS. I don't really see why I would want to use any of these browser storages?
So what would be reasons to use any of the browser-based storage instead of a variable inside JS.
Could be that from a certain data size it is preferable to use browser storage, e.g. from 100kb onwards it's better to not use a JS variable.
var myModel = {}
NOTE
Every time the user enters the app he will get fresh content from the server. The content is too realtime for caching.
`
localStorage , globalStorage and sessionStorage:
These features are ready in browsers that have implemented the "Web Storage", they all refer to a kind of HashMap, a map between string keys and string values. but the life is different. once the active page is closed sessionStorage would be cleaned but the localStorage is permanent.(MDN DOM Storage guide)
There is a point about globalStorage, which is its being obsolete since Gecko 1.9.1 (Firefox 3.5) and unsupported since Gecko 13 (Firefox 13), since then we should use localStorage. the difference between these 2 was just the HTML5 scope support(scheme + hostname + non-standard port).
These could be useful for you to:
-Share your objects between your different pages, in your site.
-Offline programming.
-Caching large object
-Or whenever you need to a local persistent storage.
IndexedDB:
IndexedDB is useful for applications that store a large amount of data (for example, a catalog of DVDs in a lending library) and applications that don't need persistent internet connectivity to work (for example, mail clients, to-do lists, and notepads)
based on this quote from MDN you can easily find your answer out, regarding using IndexedDB, if you don't know whether IndexedDB is useful for you or not, just answer these questions:
Do you store a large amount of data on client? if yes, so consider using it.
Does your app need to be offline enabled? if yes, so consider using IndexedDB.
Does your app need to a persistent internet connectivity? If yes, it stays still an option, based on the other factors.
So other than working offline as far as you don't need it, I guess, because as you said:
The content is too realtime for caching.
These have some features like sharing objects, and managing large amount of data, which you should be the one to decide.
localStorage and sessionStorage are solving a caching problem; think of them as cookies. You've said you don't want caching, so you can ignore them.
JavaScript objects behave basically like O(1) lookup tables (see How is a JavaScript hash map implemented?, and make sure you read both the top two answers, as both have something useful to say), and there is no maximum memory limit that I am aware of, or a point where another solution becomes a better choice
The only reason I can think of that you should bother with the extra step of inserting the data in an IndexedDB is if you need O(1) lookups on a field that is not the object key you are using.
I have a question about how to approach a certain scenario before I get halfway through it and figure out it was not the best option.
I work for a large company that has a team that creates tools for the team mates to use that aren’t official enterprise tools. We have no access to the database directly, just access to an internal server to store our files to run and be able to access the main site with javascript etc (same domain).
What I am working on is a tool that has a ton of options in it that allow you to select that I will call “data points” on a page.
There are things like “Account status, Balance, Name, Phone number, email etc” and have it save those to an excel sheet.
So you input account numbers, choose what you need and then using IE Objects it navigates to the page and scrapes data you request.
My question is as follows..
I want to make the scraping part pretty Dynamic in the way it works. I want to be able to add new datapoints on the fly.
My goal or idea is so store the regular expression needed to get the specific piece of data in the table with the “data point option”.
If I choose “Name” it knows the expression for name in the database to run again the DOM.
What would be the best way about creating that type of function in Javascript / Jquery?
I need to pass a Regex to a function, have it run against the DOM and then return the result.
I have a feeling that there will be things that require more than 1 step to get the information etc.
I am just trying to think of the best way to approach it without having to hardcode 200+ expressions into the file as the page may get updated and need to be changed.
Any ideas?
IRobotSoft scraper may be the tool you are looking for. Check this forum and see if questions are similar to what you are doing: http://irobotsoft.org/bb/YaBB.pl?board=newcomer. It is free.
What it uses is not regular expression but a language called HTQL, which may be more suitable for extracting web pages. It also supports regular expression, but not as the main language.
It organizes all your actions well with a visual interface, so you can dynamically compose actions or tasks for changing needs.
im building a web app in html5.. basically a form with a time counter and questions and answers.
im looking for a way that the user cannot change the score (that is calculated from the time took to answer the question) via browser debugger or etc.
encrypting the raw data sounds like an options.. but when the data is at dom, the user can change it.
i added some "time checking" in server side.. but still i would prefer some client side protection as well.
any suggestions? thanks
I'm no web pro, but I'd say just stick all the validation on the server side. From what I know about people exploiting MMORPGs, there is always a way to access/change client side data.
What you're asking for is impossible. No matter how you implement it, the user can use debugging tools to alter how the code runs in their browser - or, ultimately, just generate the HTTP POST request themselves, independent of your code.
Well, since you're saying you're using html5, why don't you just use the storage support?
e.g:
var store = sessionStorage.question= new Array();
store[0]="10s";
store[1]="5s";
Now just set that programmatically! It will last for the whole session
Put that in a file and import it and the better-than-average user wont know where to look!
You can also check This Link for a more robust solution
As Nick says, a determined user will be able to get round any encryption scheme you use on the client machine. At most you can make it difficult for them to break. You need to do two things, 1) encrypt so as to make tampering difficult and 2) try to detect any tampering that does occur.
I don't know what is available off the shelf for Javascript, if available then use AES for encryption and HMAC to detect tampering. If you have to write your own then use RC4 for encryption (not as strong as AES but much simpler to code) and a checksum to detect tampering.
One thing you can do to make it more difficult for an attacker to find your encryption key and HMAC key is not to store them in one place. Have two arrays such that the real key is array1 XOR array2. That way the actual key is not explicitly in code anywhere.
I have an old customer list of 4,000 businesses. I want to determine if the phone numbers associated with each listing are still working (and therefore the business is probably still open).
I can put each number in whitepages.com and check them one by one... but want to automate the results. I have looked at their API and can't digest it. I can form the correct query URL, but trying things like cURL -O doesn't work.
I have access to Mac tools, Unix tools, and could try various javascript stuff if anyone could point me in the right direction... would even pay. Help?
Thx
As per Pekka's comment, most companies with a public API don't allow scraping in their terms of service, so it's quite possible that performing 4k GET requests to their website will flag you as a malicious user and get you blacklisted!
Their API is RESTful and seems simple and pretty well documented, definitely try to get that working instead of going the other way. A good first attempt after getting your API key would be to write a UNIX script to perform a reverse phone number lookup. For example, suppose you had all 4000 10-digit phone numbers in a flat text file, one per line with no formatting, you could write a simple bash script as follows:
#!/bin/bash
INPUT_FILE=phone_numbers.txt
OUTPUT_DIR=output
API_KEY='MyWhitePages.comApiKey'
BASE_URL='http://api.whitepages.com'
# Perform a reverse lookup on each phone number in the input file.
for PHONE in $(cat $INPUT_FILE); do
URL="${BASE_URL}/reverse_phone/1.0/?phone=${PHONE};api_key=${API_KEY}"
curl $URL > "${OUTPUT}/result-${PHONE}.xml"
done
Once you've retrieved all the results you can either parse the XML to analyze the matching businesses, or if you're just interested in existence you could simply grep each output file for the string The search did not find results which, from the WhitePages.com API, indicates no match. If the grep succeeds then the business doesn't exist (or changed its phone number), otherwise it's probably still around (or another business exists with that phone number).
As others have noted, it is a tos violation to scrape our website or to store the data returned from the api. However, you can get the data you want from our pro service at:
https://pro.whitepages.com/list-update/upload_file
Dan
Whitepages API lead.
you can scrape the website. they have limits if you keep coming from the same ip, plus captcha. it's easy enough to get around if you know what you're doing. also, while it might violate the TOS, it's certainly not illegal. You can't copyright phone numbers and addresses says the law, so you don't have much to worry about.