we have an existing silverlight app which runs in browser + on hardware.
we want to rewrite this app using angular js and html5.
one of the key requirements with new system is support of internationalization and localization. and target countries are usa, brazil, italy for now.
Am new to this area and have lot of basic questions.
does existing database needs to be redesigned to support same ? i mean to identify columns (product_name/customer_name etc) that needs to have locale specific data and then store data for each locale and modify sprocs and webapi to accept language parameter and then get content based on that. ?
I believe we need to user nvarchar for such columns.
what will happen to currency and date time columns in db ? say there is quantity column then what should be data type of this column in db ? if current locale is Portuguese then will qty stored in Portuguese number.
what is the best practices for storing and retrieving currency column
based on locale.
what is the best practices for storing and retrieving date column
based on locale.
how to handle string checks, numeric checks in webapi methods ?
how to do comparison and checks in javascript for string,number,datetime
please share link to some good pointers which could help.
so in short right from javascript to .net webapi to database (sql) how should we take care of locale dependent logic and fields
thanks.
A lot of questions, let's see if I can answer those.
If your existing application is properly internationalized, I don't think there is any need to modify the database. Just make sure it is able to handle international characters (NCHAR, NVARCHAR, NTEXT in MS SQL, valid character encodings in others).
As for DB design, it is good to keep things locale-independent as long as you can. For instance it is better to store keys in the database and resolve them at runtime. However, if your data is dynamic (i.e. you have product names and their descriptions that changes often), the only way to go is to have translation table and look the data up using valid locale. It's quite complex in relational world (i.e. joins), but it could be done.
2,3. All the numeric columns should be kept locale-independent and formatted on the UI side. The more problematic would be prices and sales orders - you would need an additional column to store the currency code (i.e. 12.34 | USD). On the UI side you would need to pass the code to the Angular currency filter. The only gotcha here is, Angular does not support easy locale context switching, so you would need to use a hacky library like Angular Dynamic Locale to load the formats for you.
Similar. Keep it locale-independent. DB built-in types should automatically handle that for you and give you nice DateTime/DateTimeOffset (in a .Net world) back. The only gotcha would be the time zone - it may make sense to use DATETIMEOFFSET MS SQL type, as others does not store time zone.
There is an alternative way to store date and times in the database - you may decide to store it as a number of milliseconds since January 1, 1970 UTC - as BIGINT type. Especially if you are going to read this directly to JS, you will be able to easily re-create JS Date object (should you need this for calculations or something) in a valid time zone (it works the other way round as well). All you have to do to format date is to use this number (not date, that is AFAIR) and Angular's date filter with UTC as a parameter.
I don't think I understand what you're asking exactly. I guess the question is about validation of user input, rather than API. Well, beware of using Regular Expressions, because JavaScript doesn't handle Unicode well (at least in this area). You'd need to ask more precise question.
Assuming that you have Number and Date objects (i.e. typeof o == 'number') it is straightforward (as in obj1 === obj2).
As far as strings are concerned... Well, str1 === str2 will give you valid answer if you want to be exact. If you want to sort them, modern web browsers (Chrome 14+, Firefox 29+, IE11+) implement EcmaScript 402 Internationalization API so you can do something like str1.localeCompare(str2, locale), see this article.
The real problem occurs when you want to compare two strings case insensitive and accent insensitive for equality (as oppose for ordering like in case of sorting). Basically, there is no way (and this is true even in "big" programming languages like Java or C#).
Related
In MongoDB, I only need to make date range queries. But the data set is huge (9 M) and coverting a string to DateTime object (I use Perl script) and then inserting them into MongoDB is very time consuming. If I just store the dates as strings "YYYY-MM-DD", would not the range query gt:"2013-06-01" and lt:"2013-08-31" still give me the same results as if they were of datetime type? Are they the same in this scenario? If so, what would be the advantage of storing as a DateTime object.
Thanks.
If you don't care about time-zone support in your application, then using strings for basic queries in MongoDB should work fine (but if it does matter, you'll want a real Date type).
However, if you later want to do date math or use the Aggregation Framework with your date field, it's necessary that the field is actually a Date type:
http://docs.mongodb.org/manual/reference/aggregation/#date-operators
For example, you could use the $dayOfWeek function on the Date typed field.
You could likely do some simple things like group on year by using $substr (doc) in MongoDB, but the resulting code will not be as clear (nor likely perform as well).
While it's not a huge difference, I'd recommend storing them as Date types if possible generally.
I see in the docs for the Perl driver that developers are warned against using the DateTime due to the fact that it is very slow, so maybe if you use Perl regularly, and the Aggregation Framework isn't a big issue, you'd be better off storing them as either numbers or as strings, and converting them as needed in Perl.
If space is an issue, remove unnecessary characters (such as the -):
20130613 ->
4 bytes for length of string
8 bytes encoded as UTF-8
NULL character
That would be 13 characters. A DateTime value in BSON/MongoDB requires 8 bytes on the other hand (as would the Perl $time function).
(I'd strongly recommend you do a bit of performance testing to find out if the performance impact of using a Date type in MongoDB with Perl will impact your typical workflows.)
The advantage of DateTime is a few bytes less on disk. bson stores DateTime as an integer, but "2013-08-31" is a string, at 20 bytes right there.
ISO-8601 (http://www.w3.org/QA/Tips/iso-date) is meant for being able to sort quickly.
In this case, I would always store as datetime.
edit: How time-consuming are you seeing this string-to-datetime conversion? Are you sure that is your bottleneck? I have a hard time believing the conversion is taking as long as you claim.
I have a number being returned from a database with an associated format string. For the purposes of this example think 2 columns.
Value FormatString ----> DESIRED OUTPUT FROM JAVASCRIPT/JQUERY
----- ------------ -------------------------------------
100 #0.00 100.00
100000 #0,.0K 100.0K
.193 #0.0% 19.3%
Formatting options include # of decimal points, thousands separator or not, magnitude (K,M,B), percent, currency, etc. The format strings work just fine in C# and VBA (and I believe Java uses the same format strings).
I am looking for a javascript way (using whatever library is applicable, jQuery preferred) to provide the raw value and the format string and have it return a formatted number.
We are trying to re-write an app and this is what the database is giving us...can't change the database (and not sure I would want to either as the database shouldn't care about formatting the numbers for display).
Worst case: we have to change all the format strings to something javascript likes...but even then I have yet to find a usable solution that accepts a format string.
EDIT
I have looked into the following libraries/plugins/tools.
Numeral.js
DecimalFormat.js
jquery-numberformatter
All have varying levels of "correctness" (as it applies to the C#/VBA format strings). I could probably hack something together using 1 of the above as a starting point but want to see if there is something already out there.
Numeral.js feels like the most promising but it doesn't handle magnitude the way I need (it wants to auto determine the magnitude for you based on the value passed in). This feels like the "easiest" one to alter if that is the ultimate solution.
And after sleeping on it and trying to google again the morning I believe I have found what I need.
flexible-js
This appears to be working as I would expect. Don't know if it handles every use case perfectly but based on initial testing it looks to be working as I need.
I'm editing a web based program at the moment. It is used all over the globe. I'm adding a number based field. I want to be able to allow the end user to enter in the number the way they want in their local locale. I see that there is a function called Number.toLocaleString() that will give me what I need. However, I can't seem to find an inverse function.
Take the string "1,000" for example. If my user's locale is en-US, I want it to be interpreted as 1000. If my user's locale is de-DE then it should be interpreted as 1. What is the standard way of doing this in JavaScript?
Javascript has pretty much no locale support beyond just detecting what is set. For that, see the navigator object, and this other question.
If you want locale specific parsing, you'll need to use one of the many libraries out there. Checkout localplanet. It has a formatter for integers and decimals.
the data in the DB look like this
2011-09-07 14:43:22.520
But my Web API outputs the data and replace the space with the letter T
2011-09-07T14:43:22.520
I can replace the letter T with a space again in jquery, but can I fix this problem from the Web API (make the web api output the original data?)
I also do not want the miliseconds at the end. How can I get rid of them?
The format of how you see the date in the database is usually irrelevant, because it should be passed into .Net as a DateTime - not as a string. (If you are storing it as a varchar in the database, you have a bigger problem.)
ASP.Net WebAPI is returning the value in format defined by ISO8601 and RFC3339. This is a good thing, as it is a recognized machine-readable format. You probably don't want to change it.
If you really want to change it, you would need to implement a custom JSON.Net JsonConverter, deriving from DateTimeConverterBase. This is discussed here and here.
But instead, you should consider how you are using the actual result in your client application. You mentioned jQuery, so I will assume your consumer is JavaScript. In many browsers, the ISO8601 value that you have is already recognized by the JavaScript Date constructor, so you might be able to just do this:
var dt = new Date("2011-09-07T14:43:22.520");
But this won't work in all browsers. And Date doesn't have a whole lot of flexibility when it comes to formatting. So instead, you might want to consider a library such as moment.js. With that in place, you can do this:
var m = moment("2011-09-07T14:43:22.520");
var s = m.format("YYYY-MM-DD HH:mm:ss"); // output: "2011-09-07 14:43:22"
Please note that the format string here conforms to moment.js, not to .NET. There are differences in case sensitivity. Please refer to the moment.js documentation for details.
One other thing - since the value you provided doesn't have either a Z at the end, nor does it have an offset such as -07:00, then I assume it came from a DateTime whos .Kind value is DateTimeKind.Unspecified. You should be aware that when this gets sent into JavaScript (or anywhere else for that matter), there is no information about what time zone is represented. JavaScript will assume the local time zone of the browser.
If that's not what you had intended, then you need to store UTC values in your database, and make sure they have DateTimeKind.Utc so they get serialized with a Z at the end. JavaScript will normalize this to the browser's time zone, but you will still be talking about the same moment in time.
Alternatively, you could use a DateTimeOffset type - which would serialize with the specific offset. JavaScript will still normalize this to the user's time zone.
Let's say I have a date that I can represent in a culture-invariant format (ISO 8601).
I'll pick July 6, 2009, 3:54 pm UTC time in Paris, a.k.a. 5:54 pm local time in Paris observing daylight savings.
2009-07-06T15:54:12.000+02:00
OK... is there any hidden gem of markup that will tell the browser to convert that string into a localized version of it?
The closest solution is using Javascript's Date.prototype.toLocaleString(). It certainly does a good job, but it can be slow to iterate over a lot of dates, and it relies on Javascript.
Is there any HTML, CSS, XSLT, or otherwise semantic markup that a browser will recognize and automatically render the correct localized string?
Edit:
The method I am currently using is replacing the text of an HTML element with a localized string:
Starting with:
<span class="date">2009/07/06 15:54:12 GMT</span>
Using Javascript (with jQuery):
var dates = $("span.date", context);
// use for loop instead of .each() for speed
for(var i=0,len=dates.length; i < len; i++) {
// parse the date
var d = new Date(dates.eq(i).text());
// set the text to the localized string
dates.eq(i).text(d.toLocaleString());
}
From a practical point of view, it makes the text "flash" to the new value when the Javascript runs, and I don't like it.
From a principles point of view, I don't get why we need to do this - the browser should be able to localize standard things like currency, dates, numbers, as long as we mark it up as such.
A follow up question: Why do browsers/the Web not have such a simple feature - take a standard data item, and format it according to the client's settings?
I use toLocaleString() on my site, and I've never had a problem with the speed of it. How are you getting the server date into the Date object? Parsing?
I add a comment node right before I display the date as the server sees it. Inside the comment node is the date/time of that post as the number of milliseconds since epoch. In Rails, for example:
<!--<%= post.created_at.to_i * 1000 %>-->
If they have JS enabled, I use jQuery to grab those nodes, get the value of the comment, then:
var date = new Date();
date.setTime(msFromEpoch);
// output date.toLocaleString()
If they don't have JS enabled, they can feel free to do the conversion in their head.
If you're trying to parse the ISO time, that may be the cause of your slowness. Also, how many dates are we talking?
Unfortunately, there is not.
HTML & CSS are strictly used for presentation, as such, there is no "smarts" built in to change the way things are displayed.
Your best bet would be to use a server side language (like .NET, Python, etc.) to emit the dates into the HTML in the format you want them shown to your user.
It is not possible to do this with HTML, it has no smart tags that can make any kind of decisions like this. It is strictly presentational. I do wonder, though, if HTML5 perhaps has a tag for something like this...
Anyways, the way I see it, you have 3 options:
Stick to the Javascript way. There's questions with more details on it on this website, such as How do I display a date/time in the user’s locale format and time offset? and How can I determine a web user’s time zone?
Try to use geolocation. That is, your server side script fires off a request to one of the many geolocator services out there on the user's first page visit to try and guess where the user is. The downside of this is that it will be wrong about 10% of the time, so it's not that much better than the market share Javascript is going to get you.... (all in all, then, not a very good method...)
Ask the user! You will see that most websites that want to display a tailored experience for you will ask you this sort of thing because it's just not possible to know. As a neat fallback, you could wrap the question around <noscript> tags so you only ask those with Javascript disabled while offering the Javascript experience to those that have it.
Dojo has some pretty good localizations for dates and currencies. Using this method also allows you to pick different formats (e.g.: short date vs long date) and force locales.
The language and the user's locale should be sent on the HTTP header. You can use those to create the correct date format server-side to be displayed to the user. However, this is often undesirable because many users completely ignore their locale settings in their OS and/or browser. So, you may be feeding USA style timestamps to New Zealanders.
I liked the trick posted in the comment above, but it sounds like a QA headache, since you could be dealing with a large number of clients that implement timestamps in very different ways.
The most effective solution I have seen, is to simple provide a panel to allow your users to choose what time format they like. Some users even ****gasp**** like ISO formats. Then you do the time format conversion server side. If your application language does not have good locale to timezone formatting mapping, check your database. Many databases provide locale-based customized timezone formatting as well.
Because this anwser still popups in google I share that this is now possible to do by using a readonly datetime-local input (see below) and you can then style the input the way you want:
<input type="datetime-local" value="2018-06-12T19:30" readonly />
For more information see: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/datetime-local