Develop Reference

ArrayCollection (Collection of forms) index collision in Symfony 2

I am using Symfony2 to build up my page.
When I try to update a collection of forms (like described in the cookbook entry "How to Embed a Collection of Forms"), i get a collision of the indexes of the frontend and the indexes of the ArrayCollection in the backend.
I've got the relation User <-> Address (OneToMany). A user wants to create/update/delete his addresses, therefore he can add / delete in the frontend with the help of the javascript part new address elements. He does the following:
(1) Adds new address (has index: 0)
(2) Adds new address (has index: 1) and instantly removes this address again
(3) Adds new address (has index: 2).
When he clicks on save button, the following code saves/updates the user (and its addresses):
$this->em->persist($user);
$this->em->flush();
New addresses for example are then correctly persisted to the database.
Now the user wants to update the address e.g. with index 0.
When he now clicks on the save button, it updates the adress with "index 0", but at the same time, it adds again the address with "index 2" to the database (object).
To better understand the problem, i've drawn a small illustration (handmade, sorry for my bad art skills):
Now , i've got two times the address with "index 1" within my object / database.
I know why this happens, it's because the first "index 1" address gets mapped to the ArrayCollection element "number 1", and the second gets mapped to "number 2 "(because of the frontend name "index 2").
You can say: "it just fills up the addresses, until it reaches the frontend index in the backend"..
But how can I fix this behaviour ?
Site note:
This behaviour occurs using ajax requests, because if you would reload the page after clicking "save button", it would reindex the addresses in the frontend correctly with the indexes in the backend.
My suggestion to handle that situation:
Reindexing the frontend indexes after clicking save with the server side
indexes. Is this a clear / the only solution for my problem?

Yes, this is problem of Symfony form collection and it has no easy solution imho. But I have to ask why don't you do exactly the same thing what page refresh does? You can refresh only html snippet with collection. HTML code for snippet can come from server-side. Back to your question - yes, reindexing is good solution until you do not want to try write custom collection type on your own.
symfony/symfony/issues/7828
There is similar problem with validating in collection - symfony/symfony/issues/7468.
Well I think default collection type and the tutorial in Symfony docs has the some drawbacks. Hope that's help.

I have come round this issue on the client side by modifying the Javascript/Jquery code given in the Symfony Documentation.
Instead of numbering the new elements by counting the sub-elements, I am looking at the last element's id and extracting its index with a regular expression.
When adding an element, I am incrementing the last index by 1. That way, I never use the same index.
Here is my code :
// Initializing default index at 0
var index = 0;
// Looking for collection fields in the form
var $findinput = $container.find(':input');
// If fields found then looking for last existing index
if ( $findinput.length > 0 ) {
// Reading id of last field
var myString = $findinput.last().attr('id')
// Setting regular expression to extract number from id containing letters, hyphens and underscores
var myRegex = /^[-_A-Za-z]+([0-9]+)[-_A-Za-z]*$/
// Executing regular expression on last collection field id
var test = myRegex.exec(myString);
// Extracting last index and incrementing by 1
if (test.length > 0) index = parseInt(test[1]) + 1;
}

I ran into this problem a couple of times during the past two years. Usually, following the Symfony tutorial How to Embed a Collection of Forms does the job just fine. You need to do a little bit javascript coding to add the "edit/update" functionality, but other than that - you should be just fine using this approach.
If, on the other hand, you have a really complex form which uses AJAX to validate/save/calculation/business logic/etc, I've found it's usually a better to store the final data into an array in the session. After submitting the form, inside the if($form->isValid()){...} block, you would have
$collection = new ArrayCollection($mySessionPlainArray);
$user->setAddress($collection);
I would like to warn you to be careful with the serialization of your data - you might get some awkward exceptions or misbehavior if you're using entities (see my question).
I'm sorry I can't provide more code, but the solution to this problem sometimes is quite complex.

Compound Query JS SDK paRse.com

I have one class Messages with 3 principal fields:
id FromUser ToUser
I do have a query where the To = Value field and the From field is not repeated. I mean, get all FROMUSER who sent me a message.
Any Idea?
Thanks!

As #Fosco says, "group by" or "select distinct" are not supported yet in Parse.com.
Moreover keep in mind the restriction on the selection limit (max 1000 results for query) and timeout request call ( 3 seconds in the before save events, 7/10 seconds in the custom functions ). For the "count" selection, the restriction is the timeout request call.
I'm working on Parse.com too, and i've changed a lot the structure of my db model, often adding some inconsistent columns in several classes, keeping them carefully updated for each necessary query.
For cases like yours, i suggest to make a custom function, that keep in input two parameter ( we can say, "myLimit" and "myOffset" ) for the lazy loading, then select the slices, and programmatically try to filter the resulting array item list (with a simple search using for..loop, or using some utility of UnderscoreJS). Start with small slices ( eg: 200-300 records maximum for selection ) until the last selection returns zero results ( end reached). You could count all items before start all of this, but the timeout limitation could cause you problems. If this not works as expected try to make the same, client side.
You could also make a different approach, so creating another table/class, and for each new message, adding the FromUser in that table ONLY if it doesn't already exist, for that specified ToUser.
Hope it helps

Parsing and constructing filtering queries similiar to SQL WHERE clause in Python/JavaScript

I am building a query engine for a database which is pulling data from SQL and other sources. For normal use cases the users can use a web form where the use can specify filtering parameters with select and ranged inputs. But for advanced use cases, I'd like to to specify a filtering equation box where the users could type
AND, OR
Nested parenthesis
variable names
, <, =, != operators
So the filtering equation could look something like:
((age > 50) or (weight > 100)) and diabetes='yes'
Then this input would be parsed, input errors detected (non-existing variable name, etc) and SQL Alchemy queries built based on it.
I saw an earlier post about the similar problem https://stackoverflow.com/a/1395854/315168
There seem to exist several language and mini-language parsers for Python http://navarra.ca/?p=538
However, does there exist any package which would be out of the box solution or near solution for my problem? If not what would be the simplest way to construct such query parser and constructor in Python?

Have a look at https://github.com/dfilatov/jspath
It's similar to xpath, so the syntax isn't as familiar as SQL, but it's powerful over hierarchical data.

I don't know if this is still relevant to you, but here is my answer:
Firstly I have created a class that does exactly what you need. You may find it here:
https://github.com/snow884/filter_expression_parser/
It takes a list of dictionaries as an input + filter query and returns the filtered results. You just have to define the list of fields that are allowed plus functions for checking the format of the constants passed as a part of filter expression.
The filter expression it ingests has to have the following format:
(time > 45.34) OR (((user_id eq 1) OR (date gt '2019-01-04')) AND (username ne 'john.doe'))
or just
username ne 'john123'
Secondly it was foolish of me to even create this code because dataframe.query(...) from pandas already does almost exactly what you need: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html

Javascript function taking too long to complete?

Below is a snipet of code that I am having trouble with. The purpose is to check duplicate entries in the database and return "h" with a boolean if true or false. For testing purposes I am returning a true boolean for "h" but by the time the alert(duplicate_count); line gets executed the duplicate_count is still 0. Even though the alert for a +1 gets executed.
To me it seems like the function updateUserFields is taking longer to execute so it's taking longer to finish before getting to the alert.
Any ideas or suggestions? Thanks!
var duplicate_count = 0
for (var i = 0; i < skill_id.length; i++) {
function updateUserFields(h) {
if(h) {
duplicate_count++;
alert("count +1");
} else {
alert("none found");
}
}
var g = new cfc_mentoring_find_mentor();
g.setCallbackHandler(updateUserFields);
g.is_relationship_duplicate(resource_id, mentee_id, section_id[i], skill_id[i], active_ind,table);
};
alert(duplicate_count);

There is no reason whatsoever to use client-side JavaScript/jQuery to remove duplicates from your database. Security concerns aside (and there are a lot of those), there is a much easier way to make sure the entries in your database are unique: use SQL.
SQL is capable of expressing the requirement that there be no duplicates in a table column, and the database engine will enforce that for you, never letting you insert a duplicate entry in the first place. The syntax varies very slightly by database engine, but whenever you create the table you can specify that a column must be unique.
Let's use SQLite as our example database engine. The relevant part of your problem is right now probably expressed with tables something like this:
CREATE TABLE Person(
id INTEGER PRIMARY KEY ASC,
-- Other fields here
);
CREATE TABLE MentorRelationship(
id INTEGER PRIMARY KEY ASC,
mentorID INTEGER,
menteeID INTEGER,
FOREIGN KEY (mentorID) REFERENCES Person(id),
FOREIGN KEY (menteeID) REFERENCES Person(id)
);
However, you can make enforce uniqueness i.e. require that any (mentorID, menteeID) pair is unique, by changing the pair (mentorID, menteeID) to be the primary key. This works because you are only allowed one copy of each primary key. Then, the MentorRelationship table becomes
CREATE TABLE MentorRelationship(
mentorID INTEGER,
menteeID INTEGER,
PRIMARY KEY (mentorID, menteeID),
FOREIGN KEY (mentorID) REFERENCES Person(id),
FOREIGN KEY (menteeID) REFERENCES Person(id)
);
EDIT: As per the comment, alerting the user to duplicates but not actually removing them
This is still much better with SQL than with JavaScript. When you do this in JavaScript, you read one database row at a time, send it over the network, wait for it to come to your page, process it, throw it away, and then request the next one. With SQL, all the hard work is done by the database engine, and you don't lose time by transferring unnecessary data over the network. Using the first set of table definitions above, you could write
SELECT mentorID, menteeID
FROM MentorRelationship
GROUP BY mentorID, menteeID
HAVING COUNT(*) > 1;
which will return all the (mentorID, menteeID) pairs that occur more than once.
Once you have a query like this working on the server (and are also pulling out all the information you want to show to the user, which is presumably more than just a pair of IDs), you need to send this over the network to the user's web browser. Essentially, on the server side you map a URL to return this information in some convenient form (JSON, XML, etc.), and on the client side you read this information by contacting that URL with an AJAX call (see jQuery's website for some code examples), and then display that information to the user. No need to write in JavaScript what a database engine will execute orders of magnitude faster.
EDIT 2: As per the second comment, checking whether an item is already in the database
Almost everything I said in the first edit applies, except for two changes: the schema and the query. The schema should become the second of the two schemas I posted, since you don't want the database engine to allow duplicates. Also, the query should be simply
SELECT COUNT(*) > 0
FROM MentorRelationship
WHERE mentorID = #mentorID AND menteeID = #menteeID;
where #mentorID and #menteeID are the items that the user selected, and are inserted into the query by a query builder library and not by string concatenation. Then, the server will get a true value if the item is already in the database, and a false value otherwise. The server can send that back to the client via AJAX as before, and the client (that's your JavaScript page) can alert the user if the item is already in the database.

Creating a BIRT Data Set with Dynamic Data - ORA-01722

Having some trouble getting BIRT to allow me to create a Data Set with Parameters that are set at run time.
The SQL that is giving me the error is:
...
FROM SPRIDEN, SPBPERS P, POSNCTL.NBRJOBS X, NHRDIST d1
where D1.NHRDIST_PAYNO between '#PAYNO_BEGIN' and '#PAYNO_BEGIN'
AND D1.NHRDIST_YEAR = '#YEAR'
...
I have my Report Parameters defined as PaynoBegin, PaynoEnd, Year
I also have a Data Set script set for beforeOpen as follows:
queryText = String (queryText).replace ("#PAYNO_END", Number(params["PaynoEnd"]));
queryText = String (queryText).replace ("#PAYNO_BEGIN", Number(params["PaynoBegin"]));
queryText = String (queryText).replace ("#YEAR", Number(params["Year"]));
The problem seems to be that the JDBC can't get the ResultSet from this, however I have 10 other reports that work the same way. I have commented out the where clause and it will generate the data set. I also tried breaking the where clause out into two and clauses with <= and >=, but it still throws a ORA-01722 invalid number error on the line.
Any thoughts on this?

Two quite separate thoughts:
1) You have single quotes around each of your parameters in the query, yet it appears as though each one is a numeric - try removing the single quotes, so that the where clause looks like this:
where D1.NHRDIST_PAYNO between #PAYNO_BEGIN and #PAYNO_BEGIN
AND D1.NHRDIST_YEAR = #YEAR
Don't forget that all three parameters should be required. If the query still returns an error, try replacing #PAYNO_BEGIN, #PAYNO_BEGIN and #YEAR with hardcoded numeric values in the query string, and see whether you still get an error.
2) You are currently using dynamic SQL - amending query strings to replace specified markers with the text of the entered parameters. This makes you vulnerable to SQL Injection attacks - if you are unfamiliar with the term, you can find a simple example here.
If you are familiar with the concept, you may be under the impression that SQL Injection attacks cannot be implemented with numeric parameters - Tom Kite has recently posted a few articles on his blog about SQL Injection, including one that deals with a SQL Injection flaw using NLS settings with numbers.
Instead, you should use bind parameters. To do so with your report, amend your query to include:
...
FROM SPRIDEN, SPBPERS P, POSNCTL.NBRJOBS X, NHRDIST d1
where D1.NHRDIST_PAYNO between ? and ?
AND D1.NHRDIST_YEAR = ?
...
instead of the existing code, remove the queryText replacement code from the beforeOpen script and map the three dataset parameters to the PaynoBegin, PaynoEnd and Year report parameters respectively in the Dataset Editor. (You should also change any other replaced text in your query text to bind parameter markers (?) and map dataset parameters to them as required.)