Posts Tagged With: Excel

Exception avoided in FTSearchSorted in #XPages

Regular readers will remember that I’ve been playing around with using FTSearchSorted in order to create Excel spreadsheets from Notes databases in XPages. I’d had that in my sample database for a while, but hadn’t used it our production environment yet. I worried that my experience with crashes was going to be repeated and that it would take forever to integrate it into the design. Fortunately, I got over my concern about crashes (that’s what an admin team is for, right?) since my coding and the versioning all make it seem stable. The integration of the changes (modify one form, one button on a custom control and a few lines in a script library) was relatively quick. So, in the spirit of the 442nd, I decided to “Go For Broke”.

Having pushed the changes to my development environment, I set up a new report with a querystring and a sort column. My brilliant code would soon return a report of all contacts with their city = “Bethesda”, sorted by… Position since that was a column in the view and I was being random. It didn’t work. I played around with the querystring and finally decided to use FTSearch without using sort or my sort column. That worked. So, I tried FTSearchSorted, with the column name in parentheses, so it would definitely be string. Then, a number for the column instead. Then, I decided, I wouldn’t provide a sort column at all. That worked. So it wasn’t the method and it wasn’t the values I was passing to it. It just returned a silly null object every time.

So, I turned to Stack Overflow. Sure enough, someone else had the problem. The answer provided (too short in the Stack Overflow admins’ minds) was from Thomas Adrian, “make sure the view is user sortable”. Aha! I went back to my form, on which I had placed useful help text that I didn’t bother to read:

A column may only be used for sorting if it has been designed to allow “Click on column header to sort” on the Sorting tab of the column properties. The relevant options are Ascending, Descending, and Both. Trying to sort a column in an unsupported direction throws an exception.

I even bolded the important part of the text on my form. I hadn’t read it when I picked my sort column, so it threw an exception every time I used one of those unsorted columns.

Since I’ve demonstrated that even I won’t read my help text, I’ve changed the code on the button that allows you to select which column to use as your sort column so that you can’t choose one that isn’t sorted either ascending or descending. It will only list choices that are click to sort one way or the other (or both).

Forall columns In columnArray
	If ( columns.Isresortascending Or columns.Isresortdescending ) Then
		' add each view name to an array if it is eligible for click to sort
		Redim Preserve headerArray ( count )
		headerArray ( count ) = columns.ItemName
		Print headerArray ( count )
		count = count + 1
	End If
End Forall

I’ve added a clearer explanation to Thomas’ answer, but that short sentence was all the answer I needed. Once my expansion of his answer clears the editors, it will look nicer up there, but since you’ve already read this blog post, you won’t need it, eh?

I’ve updated the sample database, so you can download it to see the rest of how it works if the code sample is not enough. I’ll be at MWLUG starting tomorrow afternoon, so make sure to say “Hello”!

Advertisements
Categories: Server-Side Javascript, Xpages | Tags: , , , , , , , | Leave a comment

Exporting to Excel using Queries in #Xpages

In Paul Calhoun’s Jump Start presentation (JMP101) at IBM Connect, he was talking about using Apache POI for exporting data to Excel. I’d gotten interested when he spoke about this before, which led me to developing my own two-part posting on exporting data to Excel without having Excel (part 1, part 2). What intrigued me was his discussion of both selected documents and ad hoc queries of Notes views. My immediate thought was that I could enhance capabilty and avoid the creation of multiple new views by configuring new exports for the users via queries of our own. Once I have that working, I’ll look at allowing the users to select documents and then, perhaps, allow ad hoc queries entered by users. It’s not that I don’t want to provide the capability, I just want to be cautious about it.

In our implementation, I’ve got our development team configuring reports for the users in documents, with the end users having no real-time input into the contents of the report. The thing I’d most like to change is to move the document selection from the view selection formula into those configuration documents.

Writing a Query

The first hurdle that I have is that despite 20 years of Notes development experience, I’ve never written a full-text query. I don’t usually access databases as an end user, so it never really comes up for me. Way back in the day, when I was at Exxon-Mobil, there were some full-text indexes and queries, but as the most junior developer on the staff, I never needed to be involved in designing the interface for that or in explaining how it worked. So, here in 2014, I’ve got to learn the syntax so we can create some pre-configured queries for our users.

I found the search operators help document relatively easily. While I don’t know SQL queries either, I was hoping that it would use the same syntax so that learning this would be useful later, so that getting help from outside our community would be easy and so that people from outside (including my end users) would be able to use it easily as well. At first glance, I thought they were similar, with CAPITALIZED words and logical operators (AND, OR, etc), but the similarity ends very quickly. The Notes search query is not at all like SQL queries.

Operator
Description and examples
Where operator works
CONTAINScontains= Requires that the field before it must contain the text after it. There should be spaces between ‘CONTAINS’ and words surrounding it.Example

[Projectname] CONTAINS top secret

Finds documents whose Projectname field contains the words ‘top secret.’

Notes view – Yes
Notes domain – Yes
Web view/domain – No
IS PRESENT Requires that the field before it must be non-blank. There should be spaces between ‘IS PRESENT’ and words surrounding it. This operator can be used to find all documents that do (or do not) contain a value in a particular field.Example

[Subject] IS PRESENT

Finds documents whose Subject field is non-blank.

Notes view – Yes
Notes domain – Yes
Web view/domain – No
field fieldnameFIELD fieldname[fieldname] (use square brackets) Means ‘search this field.’ You must specify the field to search by using its field name.ExampleEach of the following are equivalent ways to find documents whose Projectname field contains the phrase top secret:

FIELD Projectname CONTAINS top secret

field Projectname CONTAINS top secret

[Projectname] CONTAINS top secret

Note Notes stores file attachments in the $FILE field.

Notes view – Yes
Notes domain – No
Web view/domain – Yes, but in view searches only
( ) [parentheses] Determines the order in which sections of your query are processed. A part of the query enclosed in parentheses will be processed before parts outside the parentheses.Example

(FIELD Projectname CONTAINS top secret OR FIELD Title CONTAINS tuna) AND (Field body CONTAINS cat OR Field revdate > 01/01/2004)

Return documents whose Projectname field contains the words ‘top secret’ or whose title field contains the word ‘tuna’; and either the body field contains the word cat or the revdate field contains a date greater than 01/01/2004.

Notes view – Yes
Notes domain – No
Web view/domain – Yes
andAND& Finds documents containing all the conditions or words linked by AND.Example

cat AND dog AND fish

Finds documents containing all three of these words.

Notes view – Yes
Notes domain – Yes
Web view/domain – Yes except ‘&’ does not work
orOR|ACCRUE, (comma) Finds documents containing either of the conditions or words and returns them ranked by number of appearances in the document.Example

cat OR dog OR fish

Finds documents containing at least one of these words.

Note ACCRUE works slightly better than OR when sorting results by relevance.

Notes view – Yes
Notes domain – Yes
Web view/domain – Yes
NOTnot! Excludes documents containing the term that appears after the operator.ExamplesYou can put NOT between words:

  • cat AND NOT dog
    Finds documents containing the word cat, but not if they also contain the word dog.
  • (cat OR dog) AND NOT (fish OR frogs)
    Finds documents containing the word cat or dog, but not if they also contain the word fish or frog.
  • (cat AND dog) AND NOT fish
    Finds documents containing the words cat and dog, but not if they also contain the word fish.

You can put NOT before any field name:

  • NOT [Projectname] CONTAINS top secret
    Finds documents whose Projectname field does not contain the words ‘top secret.’

You cannot put NOT after the math symbols =, <, >, <=, or >=; and before a date or number:

  • [date1] = NOT 12/25/2002
    is not an appropriate query.
Notes view – Yes
Notes domain – Yes
Web view/domain – Yes
” “ Requires that only documents containing the exact phrase are returned. Placing double quotes around operators (like AND, OR, CONTAINS etc.) allows them to be read as normal words.Example

“rock and roll”

Finds documents containing the phrase rock and roll.

Notes view – Yes
Notes domain – Yes
Web view/domain – Yes
PARAGRAPHparagraph Finds documents in which the words surrounding PARAGRAPH are in the same paragraph, and ranks them by how close they are.Example

car PARAGRAPH wheels

Finds documents in which ‘car’ and ‘wheels’ appear in the same paragraph and ranks them by how close the words are within the paragraph.

Notes view – Yes(If the application’s full-text index has been created with the “Index sentence and paragraph breaks” option selected. For more information, see Full-text index options.)
Notes domain – Yes
Web view/domain – No
SENTENCEsentence Finds documents in which the words surrounding SENTENCE are in the same sentence, and ranks them by how close they are.Example

car SENTENCE wheels

Finds documents in which ‘car’ and ‘wheels’ appear in the same sentence and ranks them by how close the words are within the sentence.

Notes view – Yes(If the application’s full-text index has been created with the “Index sentence and paragraph breaks” option selected. For more information, see Full-text index options.)Notes domain – Yes
Web view/domain – No
? A wildcard that can represent any single letter. It does not work with dates or numbers.Examples

?one

Finds documents containing bone, cone, done, gone (and any other four-letter words that end with ‘one’)

???ck

Finds documents containing stack, clock, stick, truck; rack, rick, rock

Notes view -Yes
Notes domain – Yes
Web view/domain – Yes
* A wildcard that can represent any extension of letters. It does not work with dates or numbers.Examples

*one

Finds documents containing bone, cone, clone, crone, done, drone, gone, telephone (and any other words of any length that end with ‘one’).

Also,

*one*

Finds documents containing bone, cone, clone, lonely, phoned, stoned, pardoned.

Notes view – Yes
Notes domain – Yes
Web view/domain – Yes
TERMWEIGHTtermweight Gives importance, or “weight,” to search words. You can use any value from 0 through 65537 to assign weight.Example

TERMWEIGHT 25 photo or TERMWEIGHT 75 audio or TERMWEIGHT 50 video

Finds documents containing at least one of the words. ‘Audio’ is most important, ‘video’ is next, and ‘photo’ is least important. Notes ranks results accordingly. You need an AND or OR between first TERMWEIGHT and subsequent ones.

Notes view – Yes
Notes domain – Yes
Web view/domain – No
EXACTCASEexactcase Search for the exact case of the word sepcified after the operator.Example

exactcase Apple

Finds documents containing ‘Apple,’ but not ‘APPLE’ or ‘apple.’

Notes view – Yes(If the application’s full-text index has been created with the “Enabled case sensitive searches” option selected. For more information, see Full-text index options.)Notes domain – NoWeb view/domain – No
= (equal)< (less than)> (greater than)<= (less than or equal)>= (greater than or equal) Search for numbers or dates in numeric or date fields only.Example

FIELD date1<12/25/98

Finds documents whose ‘date1’ field contains any date before 12/25/98.

Notes view – Yes
Notes domain – Yes
Web view/domain – No
– (hyphen) Finds documents with the hyphenated word pair.Example

full-text

Finds documents containing “full-text.”

Notes view – Yes
Notes domain – Yes
Web view/domain – Yes

Intermission

What? An intermission? Yeah, well, I started this post DURING IBM Connect in January. When I got back and tried to implement using query strings in my sample database, I crashed the development server 4 times in a row. The server was still on 8.5.3 and my need for query strings was not high. Fortunately, I’m on vacation at the beach as I write this. Yes, during a week of gorgeous weather in the Outer Banks of North Carolina, I am reading Mastering XPages and adding an entry to my blog. Some people read trashy novels at the beach, but I follow the example of Montgomery Scott and read technical manuals. Ok, so back to the topic….

Coding the call

Passing the query string from our report configuration documents to our function really isn’t that hard. I worried a lot about those crashes, so I started coding defensively, though I didn’t go so far as to add a try-catch block into the code. That will likely be inserted in the next version (after all, I expect to return to this as I get more savvy with POI’s Excel tools).

All we have to change in our code is to pull both the query string and sort column name from the report configuration document and append those string values to our function call.

	var queryString = doc.getItemValue("QueryString");
    var sortColumn = doc.getItemValue("SortColumnName");
	postValidationError(control,"queryString length: " + queryString.length );
	if ( queryString.length == 0 ) {
		createWorkbookStreamWithLabels(wbName,sheetName,colFields,dbName[0],viewName[0],labels,totalLabels,"","");
		return;
	}
	if ( queryString == null ) {
		postValidationError(control,"queryString null");
		createWorkbookStreamWithLabels(wbName,sheetName,colFields,dbName[0],viewName[0],labels,totalLabels,"","");
	} else {
		if ( queryString [0] == "" ) {
			createWorkbookStreamWithLabels(wbName,sheetName,colFields,dbName[0],viewName[0],labels,totalLabels,"","");
			postValidationError(control,"queryString empty");
		} else {
			createWorkbookStreamWithLabels(wbName,sheetName,colFields,dbName[0],viewName[0],labels,totalLabels,queryString[0],sortColumn[0]);
		};
	}

Applying the QueryString

Now, all we have to do is insert the query into our Excel sheet creation function. It’s actually very simple, as FTSearch and FTSearchSorted both modify the view object. You can simply drop the code in anywhere after you get a handle to the view and before you start processing documents.

Interestingly, after you apply either FTSearch or FTSearchSorted, the view remains sorted in original view order ONLY if the database is not full-text indexed, otherwise, it is sorted by relevance. In FTSearchSorted, you can override either behavior by selecting one column to sort instead (ascending or descending, of course). FTSearchSorted also allows you to use either a single query or a vector of queries.

Both methods return an integer count of the number of documents found by the search, so, in my code below, I simply return if there are no documents found. Yes, it ought to be fancied up, but I want to get this code out, so functionality is all I’m after today (There is Provencal rose’ chilling in the refrigerator, after all.) Here’s our code:

    //apply the queryString
    postValidationError(control,"Entries: " + myview.getEntryCount());
    if ( queryString != "" ) {
         postValidationError(control,"Sort Column: " + sortColumn);
        // max number of documents return is set as myview.getEntryCount() so that all view entries could be returned
        // default sort order is ascending
        var docCount = myview.FTSearchSorted(queryString,myview.getEntryCount(),sortColumn);
        postValidationError(control,"Query: " + queryString);
        postValidationError(control,"Query count: " + docCount);
        if ( docCount == 0) return;
    }
    postValidationError(control,"Queried view entry count: " + myview.getEntryCount());

Syntax for FTSearchSorted

FTSearchSorted(query:string) : int

FTSearchSorted(query:string, maxdocs:int) : int

FTSearchSorted(query:string, maxdocs:int, column:string) : int

FTSearchSorted(query:string, maxdocs:int, column:string, ascending:boolean, exact:boolean, variants:boolean, fuzzy:boolean) : int

FTSearchSorted(query:string, maxdocs:int, column:int) : int

FTSearchSorted(query:string, maxdocs:int, column:int, ascending:boolean, exact:boolean, variants:boolean, fuzzy:boolean) : int

FTSearchSorted(query:java.util.Vector) : int

FTSearchSorted(query:java.util.Vector, maxdocs:int) : int

FTSearchSorted(query:java.util.Vector, maxdocs:int, column:string) : int

FTSearchSorted(query:java.util.Vector, maxdocs:int, column:string, ascending:boolean, exact:boolean, variants:boolean, fuzzy:boolean) : int

FTSearchSorted(query:java.util.Vector, maxdocs:int, column:int) : int

FTSearchSorted(query:java.util.Vector, maxdocs:int, column:int, ascending:boolean, exact:boolean, variants:boolean, fuzzy:boolean) : int

Parameter Description
query The full-text query or the intersection of multiple queries. See below for the query syntax.
maxdocs The maximum number of documents you want returned from the search. If you want to receive all documents that match the query, specify 0. Defaults to 0.
column The name or 0-based index of a sorted column. A specification of NotesView.VIEW_FTSS_RELEVANCE_ORDER (512) returns results in relevance order while honoring the use of the extended flags for exact case, variants, and fuzzy search.
ascending Sorts column data in ascending order if true, descending order if false. Defaults to true. Ignored if NotesView.VIEW_FTSS_RELEVANCE_ORDER is in effect.

The availability of a column to be sorted in ascending or descending order is determined by “Click on column header to sort” on the Sorting tab of the column properties. The relevant options are Ascending, Descending, and Both. Trying to sort a column in an unsupported direction throws an exception.

exact Applies exact case to the search if true. Defaults to false.
variants Returns word variants in the search results if true. Defaults to false.
fuzzy Returns misspelled words in the search results if true. Defaults to false.

Epilogue

So, using the code above, you should be able to create report configurations that produce sorted Excel sheets to any device. My favorite is to create on on my Motorola Xoom tablet and show people data from our databases right there. I’ll keep working on this because I’m sure I will need to sort in multiple columns. I know that if I put each of the entries into an array list, I could use multi-value sorting from Java, so I might pursue that.

I’ve updated the file on dropbox.

Categories: Server-Side Javascript, Utilities, Xpages | Tags: , , , , , | 3 Comments

Sample database for #Excel exporting from #XPages

Folks had asked if I could provide a sample database beyond just the coding examples I’d provided (part 1 and part 2), so I got ready to upload one. Then, I thought, “I bet I can add full-text searching to that real easily.” Unfortunately, I hit snag after snag on adding the full-text searching, so I decided I’d post what I have working.

When I do finish up the full-text search capability, I will not only write about how I coded that, but update the sample database as well. Until then, enjoy the sample!

Categories: Utilities, Xpages | Tags: , | 7 Comments

Exporting from #XPages to Excel without Excel, Part 2

Yesterday, I posted about exporting to Excel without having Excel on the destination device, which I found particularly useful when using my Android tablet and the Chrome browser. It was a relatively long post, necessitating a two-part post.

Today, I want to spend some time showing the server-side javascript library that I’ve developed (based on the one Russ Maher presented at the AdminDev 2012 conference). So, we’re going to delve into some Apache POI, examining how we pass the configuration information in and which lines of code implement that. I expect this examination will be useful for me as well, since it will open my eyes wider to the potential within the code. Russ assures me that while he dabbles in POI, Bill Buchan actually speaks POI, so there is at least one Notes genius who knows far more than both of us….

The Function Call

Before we delve into the code, lets first examine what we’re passing to the function. Each of these values is acquired either from the XPage or from the report configuration. Using these, we have a lot of flexibility to create and format our spreadsheet.

function createWorkbookStreamWithLabels(workbookName,sheetName,fieldList,viewName,colLabels,totalLabels)

workbookName: From the XPage, which defaults the value to the report name selected. It will be used as the filename for our spreadsheet.

sheetName: Also from the XPage, defaulting to ‘Report’, but editable by the user to place as the worksheet name for the single worksheet of our spreadsheet.

fieldList: From the report configuration. This is a list of the field names, in order, to be used as values in the columns. This implementation uses field values instead of column values, but it could easily be modified to use those instead.

viewName: From the report configuration. View containing the documents to be exported.

colLabels: From the report configuration. These will end up in the first row of the spreadsheet and be styled for emphasis.

totalLabels: From the report configuration. These are used to identify which columns will get totaled. This was clearer and more reliable than using column numbers.

Creating the Stream

I actually want to start by examining the last piece of code in the library, the download of the file to the client. To me, the great strength of this whole undertaking is that I can get a spreadsheet to my device without any Excel software being loaded there. As I noted yesterday, when I used the XPage and this script library to create a spreadsheet on my tablet, the usefulness skyrocketed in the estimation of my office-mate, Ariwan. “You could download the inventory to a tablet, walk around, making notes on your tablet and then return and update the database.” Of course, I started thinking one step further and wanting to import the marked up spreadsheet right back into the database, but that code will have to wait for a while. Nonetheless, we see an immediate impact beyond just creating spreadsheets.

By using the OutputStream, we’re able to take the export that we build in the workbook, wb, and simply prompt the user to download the file as an output stream. On my tablet, this downloaded the file without prompting, which was exactly what I wanted. In a quick test, I tried to use .xlsx, but it refused to work – that might be fixed in the later releases of the POI library (I’m using 3.6 currently, though 3.9 has already been released).

//Create the filename for the spreadsheet
var fileName = workbookName+".xls";

// The Faces Context global object provides access to the servlet environment via the external content
var extCont = facesContext.getExternalContext();
// The servlet's response object provides control to the response object
var pageResponse = extCont.getResponse();
//Get the output stream to stream binary data
var pageOutput = pageResponse.getOutputStream();

// Set the content type and headers
pageResponse.setContentType("application/x-ms-excel");
pageResponse.setHeader("Cache-Control", "no-cache");
pageResponse.setHeader("Content-Disposition","inline; filename=" + fileName);
//Write the output, flush the buffer and close the stream
wb.write(pageOutput);
pageOutput.flush();
pageOutput.close();

Error-handling

I found myself trying to trouble-shoot and, as always, being frustrated with XPages because I don’t know how to do that very well. I got real used to being able to step through my LotusScript, to being able to look at field values on a document using properties, to using print statements and popups to track what was going on, but I haven’t yet filled my toolkit with ways to trouble-shoot XPages. So, I borrowed Don Mottolo’s postValidationError function from his SSJS Form Validation XSnippet. Whenever I want to return an error, I just pass the control and some text to that function and it should post the message in my display errors control. I’ve only got a few of these in the library because it started working properly before I added all the potential error-trapping and who has the patience to add all that? (Yeah, I’ll add more later, since I’m sure I’ll run into bugs in production. Idiot-proof code is impossible because idiots are just too darn inventive!)

function postValidationError(control, msg) {
if ((typeof msg) != "string")
	return;
var msgObj = new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, msg, msg);
facesContext.addMessage(control.getClientId(facesContext), msgObj);
control.setValid(false);
}

Create some cell styles

As I mentioned yesterday, we deal with a lot of international date formats and, in any reporting, dates are all over the place. So, I added a date format to be applied to those cells. Similarly, as mentioned above, I want my headers to get some styling to emphasize them.

I’ve also designated my numeric format, using two digits after the decimal. Note that if the user has their machine set to use European number formatting, with periods separating thousands and a comma to show the decimal, the export won’t care – it passes the values and the user’s local settings display it according to their preferences. I’m not certain if the preferences at the server (if any) would have any effect on how you must code it, so am curious what experience others have there.

I’m sure that if we delve into the POI documentation, we can learn all kinds of ways to format the cells and I expect that once this deploys, we’ll be working on learning those. We start, however, with just these three.

//Create helper class and styles for dates
var createHelper:HSSFCreationHelper = wb.getCreationHelper();
var dateStyle:HSSFCellStyle = wb.createCellStyle();
dateStyle.setDataFormat(HSSFDataFormat.getBuiltinFormat("m/d/yy"));

var headerStyle:HSSFCellStyle = wb.createCellStyle();
var headerFont:HSSFFont = wb.createFont();
headerFont.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
headerStyle.setFont(headerFont);

// set number formatting to use a thousands separator and two digits
// note that it will display using the user's thousands separator and decimal notation
// rather than forcing US-standard settings
var format:DataFormat = wb.createDataFormat();
style.setDataFormat(format.getFormat("#,##0.00"));

Handle various data types

The next important issue is that when passing the data through, you must make sure the Excel file understands the data type.

//depending on the type of value, use proper method to retrieve
if (valueType.endsWith("DateTime")) {
	var reportValue = (String) (itemValue);
} else if (valueType.endsWith("number")) {
	var reportValue = Number (itemValue);
} else if (valueType.endsWith("string")) {
	var reportValue = itemValue;
} else if (valueType.endsWith("Vector")) {
	var reportValue = (String) (itemValue); // multi-value field
	reportValue = reportValue.replace("[",""); // remove opening bracket
	reportValue = reportValue.replace("]",""); // remove closing bracket
} else {
	var reportValue = itemValue.toString;
}
if ( reportValue == null ) {
	reportValue = "Value not found"
}

The Whole Library

Everything else in here should be pretty self-explanatory. We simply walk through the view, almost using LotusScript, moving from document to document, getting the appropriate field values and passing them to the Excel file. When it works, it’s lovely.

function createWorkbookStreamWithLabels(workbookName,sheetName,fieldList,viewName,colLabels,totalLabels){
//import the appropriate java packages
importPackage(java.lang);
importPackage(org.apache.poi.hssf.usermodel);
importPackage(org.apache.poi.hssf.util);

var control = getComponent("comboBox1");

//Find the database connection document
var dbConView:NotesView = database.getView("TSDbConnectionLU");
var doc:NotesDocument = dbConView.getDocumentByKey("TAMIS II main db");
//Get the maindb object
if (doc != null){
	var maindb = session.getDatabase(doc.getItemValueString("DbServer"),doc.getItemValueString("DbPath"), false);
}
if ( maindb == null ) {
	postValidationError(control,"Main db Not Found");
}
var view:NotesView=maindb.getView(viewName);
if ( view == null ) {
	postValidationError(control,"View Not Found");
}

//Create placeholders for the notes document and temperary document
var doc:NotesDocument;
var ndoc:NotesDocument;

//Create a new workbook object from the poi library
var wb:HSSFWorkbook = new HSSFWorkbook();
//Create additional sheets using same sytnax and different sheet name
var sheet1:HSSFSheet = wb.createSheet(sheetName);

//Create helper class and styles for dates
var createHelper:HSSFCreationHelper = wb.getCreationHelper();
var dateStyle:HSSFCellStyle = wb.createCellStyle();
dateStyle.setDataFormat(HSSFDataFormat.getBuiltinFormat("m/d/yy"));

var headerStyle:HSSFCellStyle = wb.createCellStyle();
var headerFont:HSSFFont = wb.createFont();
headerFont.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
headerStyle.setFont(headerFont);

//Create the Column Header Rows
var row:HSSFRow = sheet1.createRow(0);
for( i = 0 ; i <= colLabels.length-1 ; i++){
	var hCell:HSSFCell = row.createCell((java.lang.Integer)(i));
	hCell.setCellValue(colLabels[i]);
	hCell.setCellStyle(headerStyle);
}

// Style the cell with borders all around. GREY 25% - 22, GREY 40% - 55, GREY 50% - 23, GREY 80% - 63
var style:HSSFCellStyle= wb.createCellStyle();
style.setBorderBottom(HSSFCellStyle.BORDER_THIN);
style.setBottomBorderColor(55);
style.setBorderLeft(HSSFCellStyle.BORDER_THIN);
style.setLeftBorderColor(55);
style.setBorderRight(HSSFCellStyle.BORDER_THIN);
style.setRightBorderColor(55);
style.setBorderTop(HSSFCellStyle.BORDER_THIN);
style.setTopBorderColor(55);

// set number formatting to use a thousands separator and two digits
// note that it will display using the user's thousands separator and decimal notation
// rather than forcing US-standard settings
var format:DataFormat = wb.createDataFormat();
style.setDataFormat(format.getFormat("#,##0.00"));

//initialize the row and cell counters
var rowCount=0;
var cellCount=0;

//Get the first document in the view and then cycle through
//the view to create the detail rows.
doc = view.getFirstDocument();

while(doc!=null){
var row:HSSFRow = sheet1.createRow(++rowCount);
for( f = 0 ; f <= fieldList.length-1 ; f++){
	var itemValue:NotesItem = doc.getColumnValues().elementAt(Number (fieldList[f]) - 1 );
	var valueType = typeof(itemValue);

	if (itemValue != null) {
		var reportValue;
		//depending on the type of value, use proper method to retrieve
		if (valueType.endsWith("DateTime")) {
			var reportValue = (String) (itemValue);
		} else if (valueType.endsWith("number")) {
			var reportValue = Number (itemValue);
		} else if (valueType.endsWith("string")) {
			var reportValue = itemValue;
		} else if (valueType.endsWith("Vector")) {
			var reportValue = (String) (itemValue); // multi-value field
			reportValue = reportValue.replace("[",""); // remove opening bracket
			reportValue = reportValue.replace("]",""); // remove closing bracket
		} else {
			var reportValue = itemValue.toString;
		}
		if ( reportValue == null ) {
			reportValue = "Value not found"
		}
		var dataCell:HSSFCell = row.createCell((java.lang.Integer)(f));
		dataCell.setCellValue(reportValue);
		dataCell.setCellStyle(style);
	}
}
doc = view.getNextDocument(doc);
}

// add a row for totaling and put the formula in where it is useful
var row:HSSFRow = sheet1.createRow(++rowCount);

// cycle through totalLabels, find match in colLabels, make a totalling cell
for( i = 0 ; i <= totalLabels.length-1 ; i++){
for( j = 0 ; j <= colLabels.length-1 ; j++){
if ( totalLabels[i] == colLabels[j] ) {
var summaryCell:HSSFCell = row.createCell((java.lang.Integer)(j));
// add the SUM formula to that cell
summaryCell.setCellFormula("SUM(INDIRECT(CONCATENATE(ADDRESS(2,COLUMN()),\":\")&ADDRESS("+rowCount+",COLUMN())))");
}
}
}

// auto size all the columns
for( j = 0 ; j <= colLabels.length-1 ; j++){
sheet1.autoSizeColumn (j);
}

//Create the filename for the spreadsheet
var fileName = workbookName+".xls";

// The Faces Context global object provides access to the servlet environment via the external content
var extCont = facesContext.getExternalContext();
// The servlet's response object provides control to the response object
var pageResponse = extCont.getResponse();
//Get the output stream to stream binary data
var pageOutput = pageResponse.getOutputStream();

// Set the content type and headers
pageResponse.setContentType("application/x-ms-excel");
pageResponse.setHeader("Cache-Control", "no-cache");
pageResponse.setHeader("Content-Disposition","inline; filename=" + fileName);
//Write the output, flush the buffer and close the stream
wb.write(pageOutput);
pageOutput.flush();
pageOutput.close();

//  Terminate the request processing lifecycle.
facesContext.responseComplete();
}
function postValidationError(control, msg) {
if ((typeof msg) != "string")
return;
var msgObj = new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, msg, msg);
facesContext.addMessage(control.getClientId(facesContext), msgObj);
control.setValid(false);
}

The sample database is available, as is the first post on this topic.

Categories: Server-Side Javascript, Xpages | Tags: , , , , , , , , , , , | 15 Comments

Exporting from #XPages to Excel without Excel, Part 1

As much as I’d love my users to simply access their data in my Notes databases, I know they’re going to need to export it. Mostly, they want to be able to manipulate it to analyze it, but they also want to create static reports and to send those reports to external parties that we don’t want accessing our data live. In old Notes, I could force train people to use the export functions of Notes, or with the advent of Copy As Table, scream at teach them to cut-and-paste. Back when I was at a government agency, helping them email their order via spreadsheet from an outside vendor in a semi-automated process, it required the user to have Excel on their machine (initially, only specific versions of Excel, but I learned to get beyond that). Fortunately, Russ Maher taught me that you don’t have to do it that way in his talk Extending Your XPages Applications with Java at AdminDev 2012.

My favorite part of his talk, since I’d done the aforementioned Notes-to-Excel export, was when he talked about creating Excel files even if the user didn’t have Excel. Apache POI is a Java API for Microsoft documents (download page for the JAR files) and by loading the JAR file into your Notes databases, you can use all of the functionality in your XPages applications. Thus, I can use my Motorola XOOM tablet to access the project database, select the inventory report, generate the Excel file and download it directly to my tablet without having Excel on the tablet. I showed the result to office mate and he came up with the inventory idea — no need to print out the inventory and you can make notes on your tablet as you walk around. Heck, I’m sure I could also write something that would process changes back into the database there as well, but that remains for a future date.

Architectural Information

Before we plunge into the code, let me first explain the architecture and use case involved here. Each of our projects around the world uses one of our applications, which we call TAMIS. In the past, this meant one Notes database for each project,  with perhaps 70 projects active at a time. Each database would start with the same design and then be customized for the project. This has usually meant design changes rather than just configuration, but both methods are used. So, there will be some configuration that is done by local staff and some by the development team. Configuration of these Excel reports is something that has always been done by the development team.

In the XPages version our application, there are actually four databases: Shared Resources, Main, Attachments and Workflow. The design work was done by Scott Good’s folks at Teamwork Solutions, so it bears many marks of their Process It! workflow engine. Shared Resources contains all the XPage design elements, configuration documents and some general information. Main is where the data goes. Attachments is obvious, but Workflow not as much – it contains both the workflow configuration and the workflow tracking documents.

Given all of that, we will have our code in Shared Resources and our data over in Main. Thus, we want to launch from one database, grab data from another and return it all to the user as a seamless download that pops into his machine just by clicking one button on our XPage.

So, how do we do this?

POI JarAdd JAR to database

Well, first thing to do is to put the JAR file into your Notes database. I’m still on 8.5.3, so I have to put it in \WEB-INF\lib, while you can bring it in directly as a design element in Notes 9. As you can see in the image here, I’m using version 3.6 of the JAR file, despite the fact that 3.9 is the current release, but I don’t really mind being behind the times a little bit, as long as it works. It looks like 3.7 and 3.8 added some fixes for handling dates and numbers better, so I will probably upgrade after our next pilot rolls out. We have seen occasional issues with dates, being a very international company.

ReportKeywordCreate the Configuration Form

The was arguably my favorite part of the process. Well, until it actually started spitting out spreadsheets. Why did I enjoy it so much? Is it because it’s ‘old Notes’? I don’t think so. While the form did build off the keyword documents that my old development mentors (Elvis Lezcano and John Mirza) created back in the 1990s when we were all at Exxon-Mobil, the neat part was utilizing some of the knowledge I’d gained about navigating design elements in simple, front-end LotusScript. Basically, the user selects the view (via name or alias) and then can choose the columns right from the view design rather than requiring any pre-configuration by me. While I do need to work on the configuration form to add more choices about sorting and styling, I got pretty happy with the results.

The configuration form supplies the view name, the column headers and numbers, and identifies which columns to total. This gets used by the Export Stream Library when the user initiates the export.

Create your Export XPage

This was the simplest design element, though also, since I’m still so new to XPages, the most challenging. With the configuration document, I was working with LotusScript, so the new wrinkles were just fun extensions of my knowledge. The Export Stream Library was initially just something I took wholesale from Russ’ presentation. While I did start with Russ’ XPage, I ended up, because of the configuration document and the UI things I wanted to do, making myself jump through a few hurdles. Worse yet, when I went to move it from it’s original implementation, I stumbled several times on the fact that the button was…. disabled. Nothing like clicking and checking your code 10 different ways and then examining it using ‘Inspect Element’ to realize the problem is that the disabled attribute simply computes as true every time.

Our Export Xpage allows the user to select from all of the report configurations we’ve created and provide both the filename (to which it will append .xls) and the name for the worksheet. I’m sure that it wouldn’t take much additional work to export multiple worksheets into a single file for a more complex and useful product, but we’re starting pretty simple.

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core"
	xmlns:xp_1="http://www.ibm.com/xsp/coreex">
	<xp:this.resources>
		<xp:script src="/CreateExcelWorkbookStream.jss" clientSide="false">
		</xp:script>
	</xp:this.resources>
	<xp:label id="label1" value="Create Reports"></xp:label>
	<xp:br></xp:br>
	<xp:br></xp:br>
	<xp:table>
		<xp:tr>
			<xp:td>
				<xp:label value="Choose Report:" id="label2"></xp:label>
			</xp:td>
			<xp:td>
				<xp:comboBox id="comboBox1">
					<xp:selectItems id="selectItems1">
						<xp:this.value><![CDATA[#{javascript:var noval = [];
						noval[0] = "Please select a report";
						var forms = @DbColumn(@DbName(),"ExcelReports",1);
						var vals = noval.concat(forms);
						return vals;}]]></xp:this.value>
					</xp:selectItems>
					<xp:eventHandler event="onchange" submit="true"
						refreshMode="complete" refreshId="wbName" id="eventHandler1">
						<xp:this.script><![CDATA[var x= '#{javascript:getClientId("comboBox1")}'; var z= '#{javascript:getClientId("wbName")}'; var tmp = document.getElementById(x).value; document.getElementById(z).value=tmp.replace(/ /g,"_"); ]]></xp:this.script>
					</xp:eventHandler>
				</xp:comboBox>
			</xp:td>
		</xp:tr>
		<xp:tr>
			<xp:td>
				<xp:label value="File Name:" id="label4" for="wbName">
				</xp:label>
			</xp:td>
			<xp:td>
				<xp:inputText id="wbName"
					disableClientSideValidation="true" required="true"
					defaultValue="Sample">
					<xp:this.validators>
						<xp:validateRequired>
							<xp:this.message><![CDATA[#{javascript:return getLabelFor(this).getValue() + " is a required field.";}]]></xp:this.message>
						</xp:validateRequired>
					</xp:this.validators>
				</xp:inputText>
			</xp:td>
		</xp:tr>
		<xp:tr>
			<xp:td>
				<xp:label value="Sheet Name:" id="label5"
					for="sheetName">
				</xp:label>
			</xp:td>
			<xp:td>
				<xp:inputText id="sheetName"
					disableClientSideValidation="true" required="true"
					defaultValue="Report">
					<xp:this.validators>
						<xp:validateRequired>
							<xp:this.message><![CDATA[#{javascript:return getLabelFor(this).getValue() + " is a required field.";}]]></xp:this.message>
						</xp:validateRequired>
					</xp:this.validators>
				</xp:inputText>
			</xp:td>
		</xp:tr>
		<xp:tr>
			<xp:td>

			</xp:td>
			<xp:td>
				<xp:messages id="messages1"></xp:messages>
			</xp:td>
		</xp:tr>
	</xp:table>
	<xp:br></xp:br>
	<xp:br></xp:br>
	<xp:button value="Create Report" id="button1">
		<xp:this.disabled><![CDATA[#{javascript:getComponent("comboBox1").getValue() == "Please select a report";}]]></xp:this.disabled>
		<xp:eventHandler event="onclick" submit="true"
			refreshMode="complete">
			<xp:this.action><![CDATA[#{javascript:
function postValidationError(control, msg) {
    if ((typeof msg) != "string")
            return;
    var msgObj = new javax.faces.application.FacesMessage(javax.faces.application.FacesMessage.SEVERITY_ERROR, msg, msg);
    facesContext.addMessage(control.getClientId(facesContext), msgObj);
    control.setValid(false);
}
var control = getComponent("comboBox1");

var wbName = getComponent("wbName").getValue();
var sheetName = getComponent("sheetName").getValue();
var formName = getComponent("comboBox1").getValue();
var fpapp:NotesDatabase = session.getDatabase(@DbName()[0],@DbName()[1]);
var lkey:java.util.Vector = new java.util.Vector;
lkey.addElement(formName);
var lview = fpapp.getView("ExcelReports");
var doc:NotesDocument = lview.getDocumentByKey(lkey,true);
if ( !@IsNull (doc) ) {
	var viewName = doc.getItemValue("ViewName");
	postValidationError(control,"ViewName: " + viewName);
	var columns = doc.getItemValue("columnNumbers");
	postValidationError(control,"columnNumbers: " + columns);
	var colFields = [];
	for(var i=0;i<=columns.length-1;i++){
		colFields.push(columns[i]);
	}
	labelList = doc.getItemValue("Value");
	var labels = [];
	for(var i=0;i<=labelList.length-1;i++){
		labels.push(labelList[i]);
	}
	var columnsWithTotals = doc.getItemValue("ColumnsWithTotals");
	var totalLabels = [];
	for(var i=0;i<=columnsWithTotals.length-1;i++){
		totalLabels.push(columnsWithTotals[i]);
	}
	createWorkbookStreamWithLabels(wbName,sheetName,colFields,viewName[0],labels,totalLabels);
}
}]]></xp:this.action>
		</xp:eventHandler>
	</xp:button>
	<xp:br></xp:br>
</xp:view>

Create your Export Stream Library

Due to the length of this post, the library is in a second post. I’ve also added a sample database.

Categories: Java, Server-Side Javascript, Xpages | Tags: , , , , , | 7 Comments

Blog at WordPress.com.

%d bloggers like this: