Develop Reference

Duplicate email address in MongoDB

Gmail treats any email with dot (.) in it as the same as without dots. For example mail sent to hello#gmail.com or he.ll.o#gmail.com or he.llo#gmail.com will be sent to the same address. How will you store this in the database so that you avoid duplicate usage of emails with dots in it. For example if a user registers first with salman.khan#gmail.com , he should not be able to use any variation of the same email like sal.mankhan#gmail.com or salman.khan#gmail.com . (Assume the database have 10M of users).

You could remove .'s prior to saving them on the server, eg...
let emailone = "hello#gmail.com"
let emailtwo = "hel.lo#gmail.com"
const normaliseEmails = email => email.split('#')[0].replaceAll(".", "") + "#" + email.split('#')[1]
console.log(normaliseEmails(emailone))
console.log(normaliseEmails(emailtwo))

It's good you're thinking about this, as it can bite you later. Unless you are never going to use or reflect back the email address of a user, you'll likely want to keep the dots so you can show them. So what to do?
If this is important problem to solve, I'd add a second column to your database, a canonical email. This would be the email addresses without the dots. Then, when looking up email addresses, "canonize" the input by removing the periods, and then look it up using this column.
Your database may be able to do this dynamically for you, depending on its features. Be cognizant that this might not be performant as the database grows, but may be okay short term.
You may also want consider other "features" of email addresses, such as the ability to add optional stuff before the #, as in joe+duplicate#example.com, which gets routed to joe#example.com. This is likely fine, but you may want to guard against it as well.

ArrayCollection (Collection of forms) index collision in Symfony 2

I am using Symfony2 to build up my page.
When I try to update a collection of forms (like described in the cookbook entry "How to Embed a Collection of Forms"), i get a collision of the indexes of the frontend and the indexes of the ArrayCollection in the backend.
I've got the relation User <-> Address (OneToMany). A user wants to create/update/delete his addresses, therefore he can add / delete in the frontend with the help of the javascript part new address elements. He does the following:
(1) Adds new address (has index: 0)
(2) Adds new address (has index: 1) and instantly removes this address again
(3) Adds new address (has index: 2).
When he clicks on save button, the following code saves/updates the user (and its addresses):
$this->em->persist($user);
$this->em->flush();
New addresses for example are then correctly persisted to the database.
Now the user wants to update the address e.g. with index 0.
When he now clicks on the save button, it updates the adress with "index 0", but at the same time, it adds again the address with "index 2" to the database (object).
To better understand the problem, i've drawn a small illustration (handmade, sorry for my bad art skills):
Now , i've got two times the address with "index 1" within my object / database.
I know why this happens, it's because the first "index 1" address gets mapped to the ArrayCollection element "number 1", and the second gets mapped to "number 2 "(because of the frontend name "index 2").
You can say: "it just fills up the addresses, until it reaches the frontend index in the backend"..
But how can I fix this behaviour ?
Site note:
This behaviour occurs using ajax requests, because if you would reload the page after clicking "save button", it would reindex the addresses in the frontend correctly with the indexes in the backend.
My suggestion to handle that situation:
Reindexing the frontend indexes after clicking save with the server side
indexes. Is this a clear / the only solution for my problem?

Yes, this is problem of Symfony form collection and it has no easy solution imho. But I have to ask why don't you do exactly the same thing what page refresh does? You can refresh only html snippet with collection. HTML code for snippet can come from server-side. Back to your question - yes, reindexing is good solution until you do not want to try write custom collection type on your own.
symfony/symfony/issues/7828
There is similar problem with validating in collection - symfony/symfony/issues/7468.
Well I think default collection type and the tutorial in Symfony docs has the some drawbacks. Hope that's help.

I have come round this issue on the client side by modifying the Javascript/Jquery code given in the Symfony Documentation.
Instead of numbering the new elements by counting the sub-elements, I am looking at the last element's id and extracting its index with a regular expression.
When adding an element, I am incrementing the last index by 1. That way, I never use the same index.
Here is my code :
// Initializing default index at 0
var index = 0;
// Looking for collection fields in the form
var $findinput = $container.find(':input');
// If fields found then looking for last existing index
if ( $findinput.length > 0 ) {
// Reading id of last field
var myString = $findinput.last().attr('id')
// Setting regular expression to extract number from id containing letters, hyphens and underscores
var myRegex = /^[-_A-Za-z]+([0-9]+)[-_A-Za-z]*$/
// Executing regular expression on last collection field id
var test = myRegex.exec(myString);
// Extracting last index and incrementing by 1
if (test.length > 0) index = parseInt(test[1]) + 1;
}

I ran into this problem a couple of times during the past two years. Usually, following the Symfony tutorial How to Embed a Collection of Forms does the job just fine. You need to do a little bit javascript coding to add the "edit/update" functionality, but other than that - you should be just fine using this approach.
If, on the other hand, you have a really complex form which uses AJAX to validate/save/calculation/business logic/etc, I've found it's usually a better to store the final data into an array in the session. After submitting the form, inside the if($form->isValid()){...} block, you would have
$collection = new ArrayCollection($mySessionPlainArray);
$user->setAddress($collection);
I would like to warn you to be careful with the serialization of your data - you might get some awkward exceptions or misbehavior if you're using entities (see my question).
I'm sorry I can't provide more code, but the solution to this problem sometimes is quite complex.

Safely using eval to use variable as an object name

As shown in this example
javascript-use-variable-as-object-name
I am using eval to use a DOM attribute to select an element from an array. Though there is no direct way for the user to change the input, I want to be as secure as possible and make sure that the variable is indeed an integer before I evaluated it.
Which of the following would be the best, most secure, way?
$(".listitem").click(function(){
var id = $(this).attr("record-id");
if(!isNaN(new Number(id))){
Storage.search.nearby.currec = rowsHolder[eval(id)];
}else{
// send email to admin, shut down
}
});
or
$(".listitem").click(function(){
var id = $(this).attr("record-id");
if(parseInt(id)){
Storage.search.nearby.currec = rowsHolder[eval(id)];
}else{
// send email to admin, shut down
}
});
More, but not required info:
Basically I am pulling down a large JSON string from online, containing an array of records. Upon building a table from the info using a for statement ( for(i in array) ), I push each row into an array called rowsHolder and give the tr an attribute of record-id="i". Then when the user clicks the row, I call the method you see above. I am using PhoneGap with JQuery Mobile.
As always, thanks for the input
-D

There is absolutely no reason to use eval here.
If your id is kind of a number, use parseFloat(id) to get it. Unnecessary as it would be converted back to a string when used as a property name, though.
If your id is an integer, use parseInt(id, 10) to get it. Unnecessary as it would be converted back to a string when used as a property name, though.
If your id is a string, just let it be a string. The property name you use it for would be one anyway.

Javascript function taking too long to complete?

Below is a snipet of code that I am having trouble with. The purpose is to check duplicate entries in the database and return "h" with a boolean if true or false. For testing purposes I am returning a true boolean for "h" but by the time the alert(duplicate_count); line gets executed the duplicate_count is still 0. Even though the alert for a +1 gets executed.
To me it seems like the function updateUserFields is taking longer to execute so it's taking longer to finish before getting to the alert.
Any ideas or suggestions? Thanks!
var duplicate_count = 0
for (var i = 0; i < skill_id.length; i++) {
function updateUserFields(h) {
if(h) {
duplicate_count++;
alert("count +1");
} else {
alert("none found");
}
}
var g = new cfc_mentoring_find_mentor();
g.setCallbackHandler(updateUserFields);
g.is_relationship_duplicate(resource_id, mentee_id, section_id[i], skill_id[i], active_ind,table);
};
alert(duplicate_count);

There is no reason whatsoever to use client-side JavaScript/jQuery to remove duplicates from your database. Security concerns aside (and there are a lot of those), there is a much easier way to make sure the entries in your database are unique: use SQL.
SQL is capable of expressing the requirement that there be no duplicates in a table column, and the database engine will enforce that for you, never letting you insert a duplicate entry in the first place. The syntax varies very slightly by database engine, but whenever you create the table you can specify that a column must be unique.
Let's use SQLite as our example database engine. The relevant part of your problem is right now probably expressed with tables something like this:
CREATE TABLE Person(
id INTEGER PRIMARY KEY ASC,
-- Other fields here
);
CREATE TABLE MentorRelationship(
id INTEGER PRIMARY KEY ASC,
mentorID INTEGER,
menteeID INTEGER,
FOREIGN KEY (mentorID) REFERENCES Person(id),
FOREIGN KEY (menteeID) REFERENCES Person(id)
);
However, you can make enforce uniqueness i.e. require that any (mentorID, menteeID) pair is unique, by changing the pair (mentorID, menteeID) to be the primary key. This works because you are only allowed one copy of each primary key. Then, the MentorRelationship table becomes
CREATE TABLE MentorRelationship(
mentorID INTEGER,
menteeID INTEGER,
PRIMARY KEY (mentorID, menteeID),
FOREIGN KEY (mentorID) REFERENCES Person(id),
FOREIGN KEY (menteeID) REFERENCES Person(id)
);
EDIT: As per the comment, alerting the user to duplicates but not actually removing them
This is still much better with SQL than with JavaScript. When you do this in JavaScript, you read one database row at a time, send it over the network, wait for it to come to your page, process it, throw it away, and then request the next one. With SQL, all the hard work is done by the database engine, and you don't lose time by transferring unnecessary data over the network. Using the first set of table definitions above, you could write
SELECT mentorID, menteeID
FROM MentorRelationship
GROUP BY mentorID, menteeID
HAVING COUNT(*) > 1;
which will return all the (mentorID, menteeID) pairs that occur more than once.
Once you have a query like this working on the server (and are also pulling out all the information you want to show to the user, which is presumably more than just a pair of IDs), you need to send this over the network to the user's web browser. Essentially, on the server side you map a URL to return this information in some convenient form (JSON, XML, etc.), and on the client side you read this information by contacting that URL with an AJAX call (see jQuery's website for some code examples), and then display that information to the user. No need to write in JavaScript what a database engine will execute orders of magnitude faster.
EDIT 2: As per the second comment, checking whether an item is already in the database
Almost everything I said in the first edit applies, except for two changes: the schema and the query. The schema should become the second of the two schemas I posted, since you don't want the database engine to allow duplicates. Also, the query should be simply
SELECT COUNT(*) > 0
FROM MentorRelationship
WHERE mentorID = #mentorID AND menteeID = #menteeID;
where #mentorID and #menteeID are the items that the user selected, and are inserted into the query by a query builder library and not by string concatenation. Then, the server will get a true value if the item is already in the database, and a false value otherwise. The server can send that back to the client via AJAX as before, and the client (that's your JavaScript page) can alert the user if the item is already in the database.

How can I reformat form input before POST using Javascript (or Rails)?

I have a field in my form labeled "Name" that will contain both the First & Last name.
Our existing dynamic server (to which the form is being POSTed to), expects two separate fields (first name, last name).
Can I use Javascript to split the user input into two separate variables before the form is posted to the server? How would I do this?

You should not rely on client side parsing whenever possible. If you are sending this form to an app you can't modify, use the Javascript method mentioned above because you have no control over it (but then why not just have a first and last name field). But if you are controller the backend app, perform all your massaging and data validation there.
Javascript should only be used to enhance the UI experience, not perform import data manipulation, it can be both a security hole and a point of failure if use Javascript for these important tasks.
Also, when manipulating names, keep in mind all the different kinds of formats you will get, such as:
John Smith Jr
Dr John Smith
John Smith Esq.
John Smith IV
John A Smith
So be careful, massaging names is very messy business and the public will enter whatever they want, at the very least, add a small label and ask them to only enter "first and last name" and pray for the best.

I would process this on the server end to make sure the data that is passed is accurate from what was posted. It's relatively easy programmatically to split the name, but the problem is how do you define your separator. Most would agree to split it wherever there is a white-space character, but what if they enter a three word name such as Don Juan DiMarco? If you decide to split the name based on whitespace, you'll have to determine how to assign the first and last names.
$arrNames = preg_split('/\s+/', $_POST['name']);
The above will give you (in PHP) an array of values split by white space. Running that on the string Don Juan DiMarco would give you an array like:
Array([0] => "Don", [1] => "Juan", [2] => "DiMarco")
From there you have to determine which ones are the first name, and which are a middle, and which are a last name. It gets even harder if you have 4 or 5 names entered, which is entirely realistic for certain cultures. All of this is guesswork, which is why I would recommend simply adding a Last Name input field on the front-end. This would eliminate all guess work as to which name is the first name and which is the last name.

If it needs to be handled on client-side, Use the Javascript split method to parse the name field value. Then append the two names to the form's url or create a couple of hidden fields.

You can do something like this:
var yourForm = document.getElementById('yourFormId');
yourform.onsubmit = new function()
{
var nameBox = document.getElementById('nameBox');
var fname = document.createElement('INPUT');
var lname = document.createElement('INPUT');
fname.type = 'HIDDEN';
lname.type = 'HIDDEN';
fname.name = 'fname';
lname.name = 'lname';
var tokens = nameBox.value.split(' ');
// Note there are a million ways to break this parser, demonstration only
fname.value = tokens[0];
lname.value = tokens[1];
yourForm.appendChild(fname);
yourForm.appendChild(lname);
}

For me, the reason for having a single field is for form simplicity, which will get higher conversions.
Getting PHP to split name into firstname and lastname, then getting PHP to resubmit the form to another server might be a bit tricky...