Re: Compiling Large Book Files Without Running out of Memory

[Jim Stauffer <jims@xxxxxxxxxxxxx>]

Another trick (I theorize) to save memory during compiling if your
document has lots of graphics imported by reference: Turn the
graphics off (View>Options) during the compiling process. It also
seems to speed up the open/close time of the files.

Jim Stauffer
ArrayComm, Inc
San Jose, CA


Dan Emory wrote:
> 
> Using FrameMaker, I successfully finished today a database publishing job
> for a 920-page directory of 28,000 specialist physicians who are members of
> an association (this is the second year in a row I've produced it). It
> consists of:
> 
> 1.  An auto-generated Table of Contents (2-column layout)
> 
> 2. 750+ pages of database-derived member listings (2-column layout) sorted
> by geographical location (Country/State/City), with a 3-15 line
> who's-who-type biography of each physician.
> 
> 3. A 136-page autogenerated alphabetical index (4-column layout), in which
> each physician's name (index level 1) includes up to two (index level 2)
> locations (City/State or City/Country) The page number(s) where the
> physician's biographical information is located is indicated for each listed
> location.
> 
> 4. A 25-page database-derived Resource Guide, interspersed with display ads,
> listing commercial companies who provide resources for the association
> members. The Guide consists of an alphabetical listing of the companies
> (name, address, fax, phone, URL, email) followed by a listing of the same
> companies by product/service category.
> 
> The book file for all of this information contained 14 FrameMaker files
> (26MB total, including the 3.5MB index).
> 
> All of the work to produce this heavy tome was done on a 32MB Windows
> platform, using UniMerge and FM+SGML 5.1.1 (FM+SGML was used not because the
> book is structured, but because it is much more stable than FrameMaker, and
> releases memory better).
> 
> The challenge was to compile the alphabetical index from the book file
> without running out of memory. Obviously, a huge chunk of memory is required
> for compiling and sorting a 2-level index of 28,000 names. During this
> process, FrameMaker must open, scan, paginate, and save each file in the
> book several  times, storing each file in memory while it is being scanned
> and paginated. If the combination of the individual file size being scanned
> and the growing chunk of memory reserved for compiling the index exceeds the
> available memory, FrameMaker puts up the dreaded "Can't open file xxx"
> message, and aborts the Generate/Update execution.
> 
> The secret, obviously, is conservation of  memory.  There are three
> techniques for doing this, and I use all of them:
>         a. If the book contains two or more generated lists/indexes, update
>            only one at a time. For example, if the book file contains both
>            a generated TOC and a generated index, execute  one Generate/Update
>            pass to compile the index only, and then immediately follow with
>            another Generate/Update pass to compile only the TOC.
>         b. Be sure that none of the individual document files in the book
>            exceed about3MB (a 2.5MB limit is safer). This is the most
>            vital of the three techniques.
>         c. Although it is probably too obvious to mention, shut down all
>            other applications that use up memory.
> 
> So far, I haven't built a large book I couldn't compile successfully on my
> 32MB platform (but I am getting more memory just in case). Recently, I did
> (for the third year in a row) a database-derived 1216-page World Wide Web
> Yellow Pages book containing complete descriptions and ratings, interspersed
> with home-page screen shots, of 10,000 web sites sorted alphabetically under
> each of 187 categories. That one had three different generated indexes (250
> pages total), which all compiled successfully using the techniques described
> above.
> 
> Dan Emory
> Dan Emory & Associates
> FrameMaker/FrameMaker+SGML Document Design
> and Database Publishing Specialists
> 
> Voice/Fax: 949-722-8971
> E-Mail: danemory@primenet.com
> 10044 Adams Ave. #208
> Huntington Beach, CA 92646
> 
> ** To unsubscribe, send a message to majordomo@omsys.com **
> ** with "unsubscribe framers" (no quotes) in the body.   **

** To unsubscribe, send a message to majordomo@omsys.com **
** with "unsubscribe framers" (no quotes) in the body.   **