Author |
Topic  |
|
Karl_H
USA
9 Posts |
Posted - Mar 16 2012 : 12:25:16
|
Last week I posted about having problems running two image processing threads in parallel. The issue is that we have a foreground thread that captures images from a video stream and writes them as BMP files. The user can optionally select that captured images also be printed. The user may also opt to have the images printed be subject to various image enhancement functions, such as HistogramEqualize, GammaCorrect, etc. The image enhancement functions have been done serially with the capture, but the complaint we get from users is that the processing time is too long, and they are unable to capture multiple images in a row rapidly enough. That is, the next capture is delayed by the image enhancement.
I have been struggling to relegate the image enhancement to a background thread. I have tried several attacks. The latest indicates unambiguously that if the image enhancement process is interrupted to capture a fresh image, not only does the enhancement fail, but it can do so in a disasterous way. I'd be happy if it just threw an exception (which it often does). I could catch that and rerun the enhancement. But often the background thread simply dies shortly after returning from the enhancement function. The trace shows it executed one line of code and not the next -- and not always in the same place. Replacing the enhancement function with a dummy function makes the problem go away.
I have gone to great lengths to make sure that foreground and background do not share any TImageEnProc objects. Yet the problem still happens. It seems that the capture and the image enhancement share some fixed resource that would have to be mutex'd. But putting a mutex on the capture and image enhancement defeats the whole objective, since now the user once again has to wait for the enhancement to complete before the next capture can begin. I can delay the enhancement if a capture is underway, but I can't delay a capture if an enhancement is underway.
This afternoon I have told management that this performance issue does not appear to be solvable with our existing HW configuration, and users will have to live with the capture delays. But if anybody has any thoughts what the shared resource is and how I can protect it and still solve the performance issue, I'd love to hear from you.
Thanks.
|
|
fab
   
1310 Posts |
Posted - Mar 16 2012 : 12:53:48
|
I don't know how your code is organized and what actually does, but you should take into consideration that TImageEnMView (just an example) loads and resamples images using a TImageEnProc (and a TImageEnIO) for each thread. Synchronization is needed only when these images need to be displayed. Other than recheck your code I can only suggest to avoid to use TBitmap as location for TIEBitmap objects. For example, you could execute this in a thread:
var
io:TImageEnIO;
proc:TImageEnProc;
bmp:TIEBitmap;
begin
bmp := TIEBitmap.Create();
bmp.Location := ieMemory;
io := TImageEnIO.CreateFromBitmap(bmp);
proc := TImageEnProc.CreateFromBitmap(bmp);
try
io.LoadFromFile(....);
proc.HistAutoEqualize();
io.SaveToFile(...);
finally
proc.Free();
io.Free();
bmp.Free();
end;
end; |
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 19 2012 : 06:57:41
|
Thank you so much, Fabrizio, for your response. The code I am working on I inherited from another developer. His use of HiComponents packages so far have been the only examples I've had to get up to speed on it. His code does not use TIEBitmap anywhere (indeed his inclusion of externals, imageenproc, hyiedefs, imageenio, does not include a definition of TIEBitmap). The time has come that I have to read the manual to find out in more detail how ImageEn works.
I am enclosing the relevant snippets of the existing code:
In the forground, capturing images:
aBitmap: TBitmap;
Stream: TMemoryStream;
VideoWindow: TVideoWindow; // active video display to this obj
// already estblished
aBitmap := TBitmap.Create;
Stream := TMemoryStream.Create;
VideoWindow.VMRGetBitmap(Stream);
aBitmap.LoadFromStream(Stream);
Stream.Free;
aBitmap.PixelFormat := pf24bit;
aBitmap.SaveToFile(FileName);
Later the background thread finds the captured image by filename and processes it:
Pic: TImage; // Scope: local
ImageIO: TImageEnIO; // Scope: attached to a form
// but is used nowhere else
ImageProcessEnh: TImageEnProc; // Scope: ditto the above
Pic := TImage.Create(Paper); // Paper is the container to be later
Pic.Parent := Paper; // delivered to the print function
ImageIO.AttachedTImage := Pic;
ImageIO.LoadFromFileBMP(FileName);
ImageProcessEnh.AttachedBitmap := Pic.Picture.Bitmap;
ImageProcessEnh.HistEqualize(LoThresh, HiThresh);
ImageProcessEnh.GammaCorrect( ... );
// etc.
If I declare a new var of type TIEBitmap and assign Pic.Picture.Bitmap to it, then do the enhancments, then assign it back, is there a chance that will solve my problem? i.e.,
EnhBitmap: TIEBitmap;
EnhBitmap.Assign(Pic.Picture.Bitmap);
ImageProcessEnh.AttachedBitmap := EnhBitmap;
ImageProcessEnh.GammaCorrect( ... ); // etc.
Pic.Picture.Bitmap.Assign(EnhBitmap);
|
 |
|
fab
   
1310 Posts |
Posted - Mar 19 2012 : 07:18:31
|
In my knowledge VCL (and then TBitmap) is not re-entrant: I don't know if VCL released with recent Delphi version is reentrant (Delphi XE, XE2?), but my suggest is to avoid to use TBitmap and TImage components at all, inside a thread. About your code, what is TVideoWindow?
|
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 19 2012 : 07:50:06
|
TVideoWindow is a Direct X object. It enables you to display a live video stream from a Direct X filter graph to a display Window. The image capture extracts a single frame of that stream to a bitmap image.
I have to compile this code under Delphi 5. If what you say about TBitmap being non-reentrant, it seems it will be necessary for me to restructure the background thread to use TIEBitmap throughout. That's likely to take me some time.
Is there such a thing at TIEImage, that is the analog of TImage, in which the bitmap will be intrinsically of type TIEBitmap?
|
 |
|
fab
   
1310 Posts |
Posted - Mar 19 2012 : 08:07:44
|
quote: Is there such a thing at TIEImage, that is the analog of TImage, in which the bitmap will be intrinsically of type TIEBitmap?
You could replace TImage with TImageEnView. It encapsulates a TIEBitmap, but also a TBitmap. To avoid it (use TIEBitmap instead of TBitmap) write:
ImageEnView.LegacyBitmap := false;
|
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 19 2012 : 11:13:22
|
My first attempt to use the TImageEnView object. I have OriginalImage and AfterImage that are both TImage types that are both images on a form. The objective is to apply, say, GammaAdjust to OriginalImage and display it in the AfterImage position. I tried the following code:
locImage: TImageEnView;
locImage := TImageEnView.Create(nil); //1
locImage.LegacyBitmap := false; //2
locImage.Assign(OriginalImage.Picture); //3
ImageProcessEnh.AttachedImageEn := locImage;
ImageProcessEnh.GammaAdjust( ... ); //4
AfterImage.Picture.Assign(locImage); //5
locImage.free; //6
I've tried several variations of the above. The one shown here crashes on //5. Other variations have ended up with a blank displayed in AfterImage. Another variation is to do the Assign function on Bitmap properties only. No success there either.
On //3, is this the correct way of copying from a TImage to a TImageEnView? The TImageEnView does not contain a Picture property, so this is unclear. Also should I be doing //2 before or after //3? Also, in //5, what is the proper way to copy the content of a TImageEnView back to a TImage?
Added 18:30 UT: Here's another variation I've tried -- this one doesn't crash but presents a blank image:
locImage: TImageEnView;
locImage := TImageEnView.Create(nil); //1
locImage.LegacyBitmap := false; //2
locImage.IEBitmap.Assign(OriginalImage.Picture.Bitmap); //3
ImageProcessEnh.AttachedImageEn := locImage;
ImageProcessEnh.GammaAdjust( ... ); //4
AfterImage.Picture.Bitmap.Assign(locImage.Bitmap); //5
locImage.IEBitmap.Assign(nil);
locImage.free; //6
|
 |
|
fab
   
1310 Posts |
Posted - Mar 19 2012 : 11:16:14
|
You cannot assign a TImageEnView to a TPicture. Replace AfterImage (and OriginaImage) with another TImageEnView. |
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 19 2012 : 13:17:52
|
Thanks again for the help. I did as you suggested, replacing OriginalImage and AfterImage with ImageEnView types in the form. I loaded them both from a file on start-up, and the image enhancement functions (e.g., GammaCorrection) worked as expected.
The only thing is an apparent licensing issue. I'm a babe in the woods with regard to this. I believe our organization (we are a small shop and I joined only six months ago) has a license for ImageEn from circa 2008. The person from whom I inherited the code also gave me instructions for configuring my development system so it would be a virtual clone of his. Is the license required on a per-development system basis? The reason I ask is because the images displayed using the ImageEnView object on the form contain a top-strip indicating that ImageEn is unregistered (with a copyright notice from 2008). Note that this is what appears on target system, and the target system is not the same as the development system. The notice did not appear doing the image-enhancements the old way using TBitmap. Please clarify the licensing issues that I've exposed here and what we have to do to resolve them, and I will take that info to management. I'm sure they will do the right thing. |
 |
|
xequte
    
39061 Posts |
Posted - Mar 20 2012 : 01:26:10
|
Hi Karl
Licensing is handled on a per developer basis, so if you are the only one working on the application then your existing license is sufficient. If you email me your details I will clarify your license status.
Nigel Xequte Software www.xequte.com nigel@xequte.com
|
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 20 2012 : 08:48:58
|
As I mentioned yesterday, I was able to get a small example to work using TImageEnView in place of TImage. I find, though, that in the 2008 version I am using, the TImageEnView.IO property does not seem to exist. I'm not sure we can update to your latest version for the next release of our SW because that would involve our having to re-validated it (which is costly in both time and $).
Using the TImageEnView object throughout the code presents some problems for me. These arise because it seems that there is no path by which an existing TBitmap or TImage can be used as the source for populating a TImageEnView. It appears its image can be populated only by loading it from a file. Can the LoadFromBuffer method be used to populate the image from an existing TBitmap?
The reason for this is that the legacy code has some images (like the logo) that are loaded on start-up. If the user has not configured a logo file, then we use our own logo, which is hard-coded into a form. In the background print thread, the logo is reproduced onto the print-paper container. Since the TImage is not thread-safe, I'd like to be able to attach the logo to the print-paper as a TImageEnView. Is there a way to convert it?
|
 |
|
fab
   
1310 Posts |
Posted - Mar 20 2012 : 14:19:05
|
quote: Using the TImageEnView object throughout the code presents some problems for me. These arise because it seems that there is no path by which an existing TBitmap or TImage can be used as the source for populating a TImageEnView. It appears its image can be populated only by loading it from a file. Can the LoadFromBuffer method be used to populate the image from an existing TBitmap?
You can assign (copy) a TBitmap to TImageEnView with...
ImageEnView.Assign(MyTBitmap);
...or...
if (MyTBitmap.PixelFormat <> pf24bit) or (MyTBitmap.PixelFormat <> pf1bit) then MyTBitmap.PixelFormat := pf24bit; ImageEnView.IEBitmap.CopyFromTBitmap(MyTBitmap); ImageEnView.Update();
|
 |
|
Karl_H
USA
9 Posts |
Posted - Mar 21 2012 : 07:42:23
|
Fabrizio, I can't thank you enough. This morning I have it working, doing the enhancement as a background thread.
My approach was to define the following:
TImageExt = class(Timage)
public
aIEBitmap: TIEBitmap;
destructor Free; // free aIEBitmap if it's assigned
end;
The TImageExt are used to populate the print-paper container in the forground (it used to populate it with TImage):
Pic: TImageExt;
Pic := TImageExt.Create(Paper);
... read an image file into Pic using VCL method ...
Pic.aIEBitmap := TIEBitmap.Create;
Pic.aIEBitmap.CopyFromTBitmap(Pic.Picture.Bitmap);
The background does the image enhancements on aIEBitmap, and then the foreground, upon finding the completed enhancements (there's a state variable the background sets that I'm not showing that indicates this) collects them using:
Pic.aIEBitmap.CopyToTBitmap(Pic.Picture.Bitmap);
prior to sending Paper container to the printer.
So far there have been no anomalies or lost images, so it looks as though all the thread-safe TIEBitmap activity is performing as you said it would.
I have been struggling with this enhancement-in-the-background issue for weeks now, and it feels wonderful to see it working reliably.
Again, a thousand thanks.
Karl
|
 |
|
|
Topic  |
|
|
|