ImageEn for Delphi and C++ Builder ImageEn for Delphi and C++ Builder

 

ImageEn Forum
Profile    Join    Active Topics    Forum FAQ    Search this forumSearch
Forum membership is Free!  Click Join to sign-up
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 ImageEn Library for Delphi, C++ and .Net
 ImageEn and IEvolution Support Forum
 Best Storage and Compression for large documents, using Snapshots 100+ pages
 New Topic  Reply to Topic
Author Previous Topic Topic Next Topic  

brandonbrown

USA
37 Posts

Posted - Feb 01 2022 :  09:43:44  Show Profile  Reply
Good morning all,

After moving up to 64-bit, and now using snapshots, I am starting to run into some technical issues with what I think is large TIFF files.

My application uses a single TIFF file as a scanned 'batch' of documents, so in theory, the page counts can sometimes run as high as 1000 pages.

Right now the thought was to use 24-bit, full color images as the source image, and then downgrade the image behind the scenes to read barcodes, which is a requirement I have. The reason for not converting to Group4 TIFF and bitonal images as the 'base' for the batches was that some documents need to have their colors retained for other potential processing, and for nice looking PDFs stored.

I've tried almost all of the compression behind the TIFF file, which some work better than others, however, all of them break the snapshot capabilities, even on a 64-bit bit machine.

I think the limit of a snapshot is somewhere around 2gig, uncompressed. If I use LZW compression on the SaveSnapShot function, it simply crashes on anything larger than 100 pages for me.

Starting with the best compression, JPEG, I'm getting this for TIFF file size and snapshot size:

Source File Size Snapshot Size
100 pages: 63mb 1.1g
200 pages: 128mb 2.2g * - Won't load
400 pages: 256mb 4.4g * - Won't load
500 pages: 320mb 9.9g * - Won't Load

So in closing, what are all of you using for the display and processing of 24bit color images for 100+ pages in a single TIFF file?

is there a better way? JPEG is the smallest, but it seems to have broken all of my base functionality.

xequte

38194 Posts

Posted - Feb 01 2022 :  16:29:12  Show Profile  Reply
Hi Brandon

Sorry, just to clarify, when you refer to snapshots, do you mean the LoadSnapshot/SaveSnapshot functionality of IEImageList, or simply that you are saving a batch of images to TIFF?

Nigel
Xequte Software
www.imageen.com
Go to Top of Page

brandonbrown

USA
37 Posts

Posted - Feb 01 2022 :  18:38:44  Show Profile  Reply
LoadSnapshot/SaveSnapshot you told me about on another post. I tried using the LZW compression and other options in the boolean,boolean,boolean configuration to no avail.
Go to Top of Page

xequte

38194 Posts

Posted - Feb 01 2022 :  19:52:40  Show Profile  Reply
Hi

It sounds like there is an 32bit integer limitation occurring there, though I can't spot one in the code. We'll investigate large file support for a coming update. I'm afraid you should use a different model for storing your state. For data of that size, I don't think snapshots would be ideal anyway. Either use a database or just load files as they are needed (e.g. with a cache of their thumbnail state).





Nigel
Xequte Software
www.imageen.com
Go to Top of Page

brandonbrown

USA
37 Posts

Posted - Feb 01 2022 :  19:54:25  Show Profile  Reply
OK, will do. I think it is actually a 2g limitation if that helps.
Go to Top of Page

xequte

38194 Posts

Posted - Feb 02 2022 :  22:11:35  Show Profile  Reply
Hi

We have fixed the issue in ImageEn with loading snapshots >2GB. This should be included in the next IEvolution release too.

Nigel
Xequte Software
www.imageen.com
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
Jump To: