- This topic has 17 replies, 3 voices, and was last updated February 20 by
.
-
Topic
-
Our goal is the development of a UNet with Haibal for the automatic segmentation of lungs in computed tomographic (CT) images. First as a 2D UNet to segment the lung into individual CT slices. The image data are individual slices with a slice thickness of 0.6 mm to 5 mm. The complete lung can be reconstructed and rendered from a complete stack of images (with a slice thickness of 5 mm, that is about 50 – 60 slices). That’s why we would like to program a 3D UNet with Haibal (which is not possible with DeepLTK) in order to use the full information of the volume for segmentation. Predicted lung masks and ground truth (manually created masks) should be compared with the intersection over union (IoU) metric. Also known as the Jaccard index.
How can we realize a UNet with Haibal?
Previous work on Ngene’s Deep Learning Toolkit
The network we used is based on the U-Net architecture. The U-Net was programmed with the graphical programming language LabVIEW , with which we had many years of experience in the development of software for image analysis, and the Deep Learning Toolkit for LabVIEW (Ngene, Armenia).The unique concept of U-Net is that it is able to generate a new, altered image as the output from an input image, after appropriate processing. This is very useful for generating segmentation images. The U-Net is a so-called fully convolutional network. Our U-Net programmed with LabVIEW is shown in Figure 1. The architecture has a symmetric “U” shape and consists of two major parts: a contraction path (left side) and an expansion path (right side). The path follows the typical architecture of a convolution neural network. It consists of the repeated application of two convolution layers, each layer with batch normalization, followed by an activation function. In all convolution layers we use a filter kernel size of 3 × 3 pixels. For each convolution we used the so called “SAME” padding type, which means there is automatically enough padding that the output image of the convolution layer has the same dimensions as the input image. For downsampling we chose a stride of 2 to halve the size of the input image. For upsampling we use an upsampling layer. This layer increases the dimensionality (rows and columns) of output feature maps by doubling the values (stride = 2).
The DeepLTK still has a number of limitations: IOU and Dice coefficient are the only metrics so far. More will be added in the next releases. Shape quality performance metrics like ASSD or BF-Score are not yet supported. The only optimization algorithms are SGD and Adam. Further algorithms such as Adagrad, AdaDelta, RMSProp, Nesterov … are being developed for the next releases. A 3D semantic segmentation architecture is still not possible with the DeepLTK.
Figure 1: Our used U-Net architecture. Each green or blue box corresponds to a multi-channel feature map. The number of channels is shown above the box. The specifications 512 × 512 to 16 × 16 (in the lowest resolution) show the x, y dimensions in pixels of the input and output images (or feature maps).
References:
-
Replies
-
No Youssef, not on Keras.
My model was only developed with Ngene´s Deep Learning Toolkit.
I would like to program the same UNet architecture with Haibal. Then train the network and save the trained network (weights, configuration, etc.). I would then like to reload the trained and saved network in Haibal, segment unknown lung CTs and quantify them with my image processing software (which was also programmed with LabVIEW).
Example:
The user interface that I programmed.
UNet with downsampling path (Image size from 256×256 to 16×16 pixel, no of convolution filters from 64 to 1024) and upsampling path (upsampling and concatenation. Image size increase from 16×16 to 256×256 pixel and no of conv. filters decrease from 1024 to 64).
Downsampling VI:
Upsampling and Concatenation VI:
Hi @PietH , it looks like you want to add HTML content to your reply. This is an option we did not have enabled but now it is possible.
To do this, simply add your html code through the Text tab of the response field (see screenshoot)
You can also use “BBcode” to partially format your text in the future.
I invite you to edit your message so that it will be displayed properlyFor more information, you can consult the “Formatting” part of this article.
Hi Julien,
I used the html tags only in the Text tab. Switched back to Visual and it looks good.
The html formatting was only gone after it was activated by the moderator.
Unfortunately, I can no longer edit my posts afterwards.
Hello again,
I just checked and indeed a user could only edit his answer for a limited time. Now you can edit your answer anytime.
Dear Peter,
Making your model architecture and run it with HAIBAL will not be a problem.
If you already trained your model with another framework and you can exctract weights of the differents layers, it’s possible to reinject it in HAIBAL. There is a powerfull Weight editor available in HAIBAL. (Get / Set Weights functionalities).
Dear Youssef,
Since the previous model was not that efficient, I would like to program a UNet from scratch with HAIBAL.
That was also the main reason why I chose HAIBAL. There are now so many modifications that are simply not feasible with the DeepLTK, such as 3D UNet for exampleIn our research project, the goal for me is to program a multiclass segmentation. So to segment different organs in the CT image or in the MRT (magnetic resonance imaging) image. e.g. lungs, trachea, heart, liver etc.
But first I have to master the basics in HAIBAL. I need your help for the first basic steps in developing a UNet with HAIBAL.
Maybe the following Keras examples will help you
UNet (Kera’s example: https://pyimagesearch.com/2022/02/21/u-net-image-segmentation-in-keras/)
SegNet (an encoder-decoder architecture, https:/ /github.com/0bserver07/Keras-SegNet-Basic).Ok, we will work on a basic encoder-decoder CNN (SegNet) and a basic U-Net examples for the future release to help you.
Dear Youssef,
I would like to program a UNet from scratch with HAIBAL in order to train it with new data.
My goal within our research project is a multiclass segmentation, i.e. not just one class (the lungs), but several classes (lungs, heart, trachea, etc.). There are now also very interesting modifications of UNet to optimize the segmentation process (e.g. 3D U-Net, Attention U-Net, Inception U-Net, 5 Residual U-Net, Recurrent Convolutional Network, Dense U-Net, U-Net++, Adversarial U-Net), which cannot be programmed with DeepLTK.
In order to program these new U-Net modifications, however, I first have to master the basics in HAIBAL!
1.) Development of an encoder-decoder (SegNet) with HAIBAL and
2.) Development of a U-Net with HAIBAL
Youssef, I need your help for this.Maybe the examples in Keras will help:
UNet: https://pyimagesearch.com/2022/02/21/u-net-image-segmentation-in-keras/
https://github.com/divamgupta/image-segmentation-keras
https://keras.io/examples/vision/oxford_pets_image_segmentation/
Ok let me check all of this.
I’ll first fix your licence problem in priority and then we will start to work on examples helping you.
Dear Peter,
A complete example of UNET is now available on the last release of HAIBAL. 🙂
Have a fun !
Dear Youssef,
Oh great thank you. I will have a look.
In the meantime I had already dealt with the previous example.
I modified the MedUNet VI a bit. e.g. programmed the original UNet. Reduced the images to 256×256 with Imaq Vision…
In the training loop, I passed the 4D validation data (you call it test data) directly to the poly_model.vi (Forward Input 4D). So not from a parallel loop via a global variable.
The data type is exactly the same as in your example. However, I get an error message:
“Our model expects 4 input(s), but it receives 1 input array”.
Can you tell me what the error message means?Hi Peter,
Could you send me the summary of your model please.
I need to see the modification you did. :).
it seems you setup some inputs layers.Thank you,
Hi Youssef,
Yes, I know that the summary should actually open up.
I controlled the workstation remotely via VPN and the summary didn’t pop up.If I copy everything to my computer at home, the summary pops up.
I’ll try it next week on the workstation directly.
I saved the summary. However, the order of the layers is a bit confusing for me.
I will email you the UNet model.Maybe you can take a look why the summary doesn’t show the correct order of the layers.
Thank you
Peter
- You must be logged in to reply to this topic.