Large PDF Documents with Lasso PDF Tags

Note this article is relevant for LP8x, although improvements in 8.6 can be observed the CentOS version.

This article explains how to address a problem with the PDF tags when trying to create PDF files larger than 20MB. It is due to Java running out of memory, and the solution is to have Lasso allocate more memory to the Java Virtual Machine (JVM).

The Lasso PDF tags have been very useful to some of us Lasso Developers, but until now there has been one recurring problem that threw a spanner in the works: Lasso cannot create PDF documents larger than 20MB - or so it seems. This file size limit has cropped up several times on LassoTalk, and there have been suggestions such as optimising images where images are used, or using command line apps to stitch large documents together. Following the latter example, I used pdftk (http://freshmeat.net/projects/pdftk/) for some time, but recently discovered that the version I'm using does not support the most recent version of PDF, causing errors yet again. Anyway, back to the Lasso tags - and solving the problem.

Java Virtual Machine

The PDF tags utilise the Java Virtual Machine (JVM) which is part of the Mac OS X standard installation. Java is loaded when Lasso starts up, so this is where the memory issue arises. When Java runs out of memory, you might see errors like this one (when running Lasso in console mode):

Exception in thread "Thread-3" java.lang.OutOfMemoryError: Java heap space
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
	at com.lowagie.text.pdf.OutputStreamCounter.write(Unknown Source)
	at com.lowagie.text.pdf.PRStream.toPdf(Unknown Source)
	at com.lowagie.text.pdf.PdfIndirectObject.writeTo(Unknown Source)
	at com.lowagie.text.pdf.PdfWriter$PdfBody.add(Unknown Source)
	at com.lowagie.text.pdf.PdfWriter$PdfBody.add(Unknown Source)
	at com.lowagie.text.pdf.PdfWriter.addToBody(Unknown Source)
	at com.lowagie.text.pdf.PdfReaderInstance.writeAllVisited(Unknown Source)
	at com.lowagie.text.pdf.PdfReaderInstance.writeAllPages(Unknown Source)
	at com.lowagie.text.pdf.PdfWriter.addSharedObjectsToBody(Unknown Source)
	at com.lowagie.text.pdf.PdfWriter.close(Unknown Source)
	at com.lowagie.text.pdf.PdfDocument.close(Unknown Source)
	at com.lowagie.text.Document.close(Unknown Source)
	at com.omnipilot.tags.PDFDoc.closeDoc(Unknown Source)
	at PDFTag.myCloseDocFunc(Unknown Source)

Time to pay a visit to Lasso Admin.

Lasso Site Admin

This Java memory issue must be fixed on a site-by-site basis. Even though it is possible to make changes to the Lasso internal database from the Server Admin interface, this appears to have no effect on Java's behaviour in individual Lasso sites.

So, log in to the Site Admin page for each site where you need to be able to create large PDF files, and do the following:

1. Go to Lasso Site Admin > Utility > SQL > Browser
2. In the "SQL Statement" panel, select:

• Datasource: Lasso Internal
• Host: Internal
• Database: lasso_internal
• Table: global_prefs

3. In the SQL Statement field, enter the following:

insert into global_prefs (`store_key`,`data`) values ("java_vm_options","-Xms32m -Xmx256m");

and click the "Issue SQL Statement" button. This will increase the default memory allocated to Java to 32MB and the max memory to 256MB. Obviously you can change those numbers to suit your own requirements. To see the result of what you have just inserted, type the following in the SQL Statement field and click the Issue button again:

SELECT * FROM global_prefs;

4. Restart the Lasso Site.

That's it, you're done. If all goes well, you will now be able to create PDF documents with Lasso's PDF tags that exceed the 20MB size limit. Happy days :-)

5. Change the settings

I you need to change the settings, here's the SQL request :

UPDATE global_prefs SET data = "-Xms32m -Xmx256m" WHERE store_key = "java_vm_options" ;

Credits from original author Pier Kuipers

Many thanks to Steffan Cline and Bil Corry for putting me on the right track and suggesting that I read the manual :-/

Speaking of the manual, I refer to the Lasso 8.5 Language Guide:

Lasso Professional 8 allows you to specify Java Virtual Machine options used for launching JVM upon Lasso startup. These options are stored in the lasso_internal.global_prefs table as java_vm_options in the store_key field.

Annex from Dominique Guardiola Falco

Memory-related JVM parameters

-Xms size in bytes
Sets the initial size of the Java heap.
The default size is 2097152 (2MB).
The values must be a multiple of, and greater than, 1024 bytes (1KB).
(The -server flag increases the default size to 32M.)

-Xmn size in bytes
Sets the initial Java heap size for the Eden generation.
The default value is 640K.
(The -server flag increases the default size to 2M.)

-Xmx size in bytes
Sets the maximum size to which the Java heap can grow.
The default size is 64M.
(The -server flag increases the default size to 128M.)
The maximum heap limit is about 2 GB (2048MB).

-XX:StackShadowPages
The size of the shadow pages is between 3 and 20 pages, depending on the platform. This distance is tunable, so that applications with native code needing more than the default distance can increase the shadow page size. The option to increase shadow pages is -XX:StackShadowPages=n, where n is greater than the default stack shadow pages for the platform.
If you increase the value of StackShadowPages, you might also need to increase the default thread stack size using the -Xssparameter. Increasing the default thread stack size might decrease the number of threads that can be created, so be careful in choosing a value for the thread stack size. The thread stack size varies by platform from 256k to 1024k.

-Xss
the stack size for each thread
Each thread in the VM get's a stack. The stack size will limit the number of threads that you can have, too big of a stack size and you will run out of memory as each thread is allocated more memory than it needs.
 

Author: Pier Kuipers
Created: 27 Feb 2009
Last Modified: 28 Jul 2011

Updated with Annex by:

Author: Dominique Guardiola Falco
Created: 27 Feb 2009
Last Modified: 28 Jul 2011