Develop Reference

How Do I Search A Sharepoint List By Using Prefix And Suffix Wildcards And Pagination?

I am trying to develop a html/css/js frontend that allows me to search my SharePoint online list as I would a normal list. This turned out surprisingly impossible for MS Sharepoint.
By trial and error and reading through the sparse documentation, I did the following:
I tried using Search Rest API, but it doesn't allow suffix matching, so *arepoint won't return items that have words like sharepoint.
I tried using CAML query, but it doesn't allow me to jump to a page because I can't define a start row.
I tried using list filtering, but its "substringof" filter doesn't allow me to search multiline text.
Basically all three of them seem to have at least one glaring issue that impedes normal usage.
I'd like to know if anybody found a way to get around the issues of any one of the three methods.
I didn't paste the code because the issue I'm having is with the API's themselves, but I could post them if needed. MS Sharepoint just flat out said it doesn't support suffix matching with no further explanation like a punch in the face.
Edit:
The workaround that I have so far is to get all the items as one giant JSON and do the searching and filtering client side, but that's already slow with my 900 row list.
Thanks,
Update:
Wow, I've been trying and errorring with CAML since it seemed the best of the three, but also the hardest.
So my best workaround so far is a two stage simultaneous search...
One function searches the first 20-30 rows and returns the results pretty quickly, so the user can scroll asap.
Another function starts at the same time and searches the whole list. It returns only IDs, just to build the pagination buttons. It's pretty slow but I'm still pretty stubborn about not using the left and right button without paging because ew.
Update: Here are my sharepoint search functions:
git
You can use the same sp_listSearch to search for the first 30 items, then use sp_getRecordsFromList to get the whole details of the 30 items.
Run the sp_listSearch again without row limit so that it will return all the ids of all matches (might take long) and then program your own page loading functionality for all those ids... like maybe a put every 30 ids in a button named page1 and so on and then run sp_getRecordsFromList on button click.

I guess you are using old Sharepoint because you tried caml query the same I did. I have a partial solution but if u want to search on all your queries while they are shown to the user u cant because caml query doesn't have an option to tell it the first 100 rows and then from 101 to 200.
You can filter your caml query with a search input, and after that, u can apply datatables jquery to search on that result and pagination too. like this sample, u introduce the filter parameter and get an exact result just like SQL. Just send your view data to the model and from the model to the query. it's on c#.
camlQuery.ViewXml = #"<View>
<Query>
<Where>
<Eq>
<FieldRef Name='CI' />
<Value Type='Text'>" + model.Carnet +"</Value></Eq></Where></Query></View>";
this returns exactly and specified results so you won't get 900 rows
try saketa caml query builder for testing your queries.
you can limit your query with a row limit, but the fasted solution to queries fast and not get 100000 rows is to limit your query with a search parameter like before. further, then that I didn't find a solution either to stop 100000 row queries.
this is the datatables that let you paginate and search but it won't limit the query rows. all docs are on datatables.net
<script type="text/javascript" src="~/Scripts/jquery.dataTables.min.js"></script>
<script type="text/javascript" src="~/Scripts/dataTables.bootstrap5.min.js"></script>
<script type="text/javascript">
$(document).ready(function () {
$('#tblResultados').dataTable({
"language": {
"url": "//cdn.datatables.net/plug-ins/1.10.16/i18n/Spanish.json"
},
order: [[0, 'desc']],
"iDisplayLength": 25,
"lengthMenu": [25, 50, 100]
});
});
</script>

jqGrid onCellSelect Return Property from Local Data

I've run into an issue using jqGrid 5.2.1.
I'd like to retrieve a property from a row of data when a user clicks on a cell. The property is in the original data set, but it isn't displayed in the grid. Also, I'm using the scroll feature, so the row ID's are coming back like "jqg20".
I've tried using getRowData, but that only returns the data that is displayed in that row. getLocalRow will not accept the row ID with "jqg" in it. Is there another way that I can access the same row in the local data that the user clicked in the grid to pull out a property? The property 'attr' is what I want to pull out in the example:
var testData = [
{col1:10, col2:20, col3:30, col4:'TEST', col5:50, col6:60, col7:70, col8:80, col9:90, col10:100, attr: {property: "this is column 1"}}
]
Here is the jsfiddle that I've created to show the problem:
https://jsfiddle.net/rhv247q7/

Before to answer of the direct problems I need to do some notes.
It is recommend to use the jqGrid version where the problem persist. You tell us for version 5.2.1, but you use 4.6, which we think can come to some uncomfortable situations.
It is always good idea to set id row – this can be done either in the description of the colModel (key : true) or with the appropriate reader in this case localReader. When you set this you will be sure that there will be not a problems getting certain row and some other important commands. It is not good idea to let the grid create the ids.
Now to the problem : to do what you want you should use the getLocalRow, which return the data as it comes to the grid – i.e the original data.
In your case there was a bug when scroll is on and no id is set from the developer (i.e. the grid creates internally the id). The problem is fixed in GitHub and you can test it.

Google Custom Search API javascript

I'm writing an AngularJS application which should perform image search through entire web using google custom search api.
I've used the query as follows:
'https://www.googleapis.com/customsearch/v1?key='+ googleApiKey + '&cx=014766414461901118935:knbecyxzx4u&searchType=image&startIndex=41&imgType=photo&q=' + query;
As a result I get an array which consists of 10 elements. The documentation says that it can return up to the first 100 results. How can I load all the rest of results beside those 10 which come from the query above?
Note: Besides the array of images I also get the query object which has the nextPage object which holds the metadata describing the query to use for the next page of results, but apparently I have no idea how to use it as the startIndex value is always 11

I believe that you use &start instead of &startIndex

To incorporate find dialog similar to ms acess in my java data table

I have a data table in my application built using JSF and primefaces 3.5 which populates the data from database.Since it is a large data set i have implemented live scrolling to fetch rows as and when end of scrolling is reached.In this data table i have incorporated the search functionality for each column of the table as shown in the showcase here.I am developing an application as part of re-platforming of one of MS access application to java web application.The customer has now changed the requirement where instead of the find text box for each column they want the find dialog similar to the one used in MS access as shown in the image below.
The dialog should contain all matching criteria as shown in the image.
Can someone please suggest how to incorporate the above feature by any means.It can be either using JSF or any other front-end technologies.
Thanks in advance.

There are basically two options for this with the primefaces datatable:
You could use a lazy table, providing the required filtering when executing the lazy statements.
You could setup a custom filter that adapts to the constraints selected in the dialog.
The first approach would be preferable for huge amounts of data, because your live-scroll would just load entries that are matching the filter defined on the LazyModel.
The second approach would be faster to implement, but suitable for regular dataset-sizes I'd say.
Lazy Approach
http://www.primefaces.org/showcase/ui/data/datatable/lazy.xhtml
You would need to implement your own LazyDataModel and handle the conditions in the load() Method. The concrete implementation of this completly depends on whether you are using sql-queries, or some sort of Entitymanager to load the data required. The load method just needs to take care to take the filters into account along with the required pageSize determined by your live-scrolling logic.
custom filter
This way would allow you to keep your current "logic" of fetching entities and completly work "on top" of the data.
looking at the example you already linked http://www.primefaces.org/showcase/ui/data/datatable/filter.xhtml (last column) you could solve this using some javascript along with the proper custom filter:
setup the same custom filter for all columns of your table.
hide the primefaes searchfields using some css (you are using them, customer just does not see them)
Whenever the filter conditions on your dialog are changing, set the corresponding filter-input to a value (based on your conventions) and call PF('table').filter() afterwards.
i.e. "Look In":
"Current Field": Update only the respective (hidden) input element. (Set others to null or empty string)
"All Fields": Update the global (hidden) input element.
i.e. "Match":
WholeField: set a regular String : kitchen to the fields of the required "Look In".
StartOfField: use some Wildcard-Identifier behind: kitchen%.
EndOfField: use some Wildcard-Identifier behind: %kitchen.
AnyPartOfField: use Wildcards on both sites: %kitchen%.
ps.: Use some Asci-Character no body would search like ❤ :P
After calling PF('table').filter(), your custom filter will be invoked, which could look roughly like this:
public boolean filterColumn(Object value, Object filter, Locale locale) {
String filterText = (filter == null) ? null : filter.toString().trim();
//no filter?
if(filterText == null||filterText.equals("")) {
return true;
}
//cell empty?
if(value == null) {
return false;
}
String valueText = (String)value; //assuming only strings
//"contains"
if (filterText.startsWith("%") && filterText.endsWith("%")){
return valueText.contains(filterText);
}
//"endsWith" - Yes wildcard at the BEGINNING is a endsWith!
if (filterText.startsWith("%")){
return valueText.endsWith(filterText);
}
//"startsWith"
if (filterText.endsWith("%")){
return valueText.startsWith(filterText);
}
//no wildcards? "exact match"!
return valueText.equals(filterText);
}
Aww, just saw you are on 3.5 - I think the custom Filter is available in 5.0+ only. But maybe it gives you a hint, or you can update PF.

Most Efficient way of Filtering an Html Table?

I have an ajax function which call a servlet to get list of products from various webservices, the number of products can go up to 100,000. I need to show this list in a html table.
I am trying to provide users an interface to filter this list based on several criteria. Currently I am using a simple jQuery plugin to achieve this, but I found it to hog memory and time.
The Javascript that I use basically uses regex to search and filter rows matching the filtering criteria.
I was thinking of an alternate solution wherein I filter the JSON array returned by my servlet and bind the html table to it. Is there a way to achieve this, if there is, then is it more efficient than the regex approach.

Going through up to 100,000 items and checking if they meet your criteria is going to take a while, especially if the criteria might be complex (must be CONDO with 2 OR 3 bedrooms NOT in zip code 12345 and FIREPLACE but not JACUZZI).
Perhaps your servlet could cache the data for the 100,000 items and it could do the filtering, based on criteria posted by the user's browser. It could return, say, "items 1-50 of 12,456 selected from 100,000" and let the user page forward to the next 50 or so, and even select how many items to get back (25, 50, all).
If they select "all" before narrowing down the number very far, then a halfway observant user will expect it to take a while to load.
In other words, don't even TRY to manage the 100,000 items in the browser, let the server do it.

User enters filter and hits
search.
Ajax call to database, database has indexes on appropriate
columns and the database does the filtering.
Database returns result
Show result in table. (Probably want it to be paged to
only show 100-1000 rows at a time
because 100,000 rows in a table can
really slow down your browser.
Edit: Since you don't have a database, the best you're going to be able to do is run the regex over the JSON dataset and add results that match to the table. You'll want to save the JSON dataset in a variable in case they change the search. (I'm assuming that right now you're adding everything to the table and then using the jquery table plugin to filter it)

I'm assuming that by filtering you mean only displaying a subset of the data; and not sorting.
As you are populating the data into the table add classes to each row for everything in that row you want to filter by. e.g.:
<tr class="filter1 filter2 filter3">....
<tr class="filter1 filter3">....
<tr class="filter2">....
<tr class="filter3">....
Then when you want to apply a filter you can do something like:
$('TR:not(.filter1)').hide();

I agree with Berry that 100000 rows in the browser is bit of a stretch, but if there's anything that comes close to handling something of that magnitude then it's jOrder. http://github.com/danstocker/jorder
Create a jOrder table based on your JSON, and add the most necessary indexes. I mean the ones that you must at all cost filter by.
E.g. you have a "Name" field with people's names.
var table = jOrder(json)
.index('name', ['Name'], { sorted: true, ordered: true });
Then, for instance, this is how you select the records where the Name field starts with "John":
var filtered = table.where([{ Name: 'John' }], { mode: jOrder.startof, renumber: true });
Later, if you need paging in your table, just feed the table builder a filtered.slice(...).

If you're getting back xml, you could just use jQuery selection
$('.class', context) where context is your xml response.
From this selection, you could just write the xml to the page and use CSS to style it. That's where I'd start at first, at least. I'm doing something similar in one of my applications, but my dataset is smaller.

I don't know what you mean by "bind"? You can parse JSON and then use for loop (or $.each()) to populate ether straight HTML or by using grid plugin's insert/add