AI4eLIFE: Easying local image feature extraction using AI.¶
A data-centric AI for tumor segmentation from [18F]FDG-PET images.
-This jupyter notebook is designed to explain how to run the proposed data-centric AI. It also illustrates how to visualize and interpret the results.
- -# import important libraries
-import os
-import numpy as np
-import pandas as pd
-import matplotlib.pyplot as plt
-
-# python script
-import visualization as view
-import warnings
-warnings.filterwarnings('ignore')
-# check the folder structure of the main folder for the input and output data.
-# function to print the folders, subfolders, and files in a given path "main_folder_path".
-def list_folders_files(main_folder_path):
- for root_dir, dirs, files in os.walk(main_folder_path):
- folder_level = root_dir.replace(main_folder_path, '').count(os.sep)
- indentation = "-"*4*(folder_level) # tabe
- print(f'{indentation}{os.path.basename(root_dir)}')
-
- sub_indentation = "-"*4*(folder_level + 1)
-
- for get_file in files:
- print(f'{sub_indentation}{get_file}')
-
-main_directory = "E:/data"
-list_folders_files(main_directory)
-We have the two main folders, input and output folders. Under the input folder, we do have the structured data.
Step 2: Download/Clone the repository¶
-# Let us clone the repository to a folder named "E:/step-by-step-test"
-# create the directory
-os.mkdir("E:/step-by-step-test")
-# change working directory
-os.chdir("E:/step-by-step-test/")
-print(f"You are at {os.getcwd()} directory !")
-# clone or download the directory to the created folder:
-!git clone https://github.com/KibromBerihu/ai4elife.git
-os.chdir('E:/step-by-step-test/ai4elife/')
-If you choose option one installation using a virtual environment, activate the virtual environment: conda activate myenv
Step 4: Run the following command¶
Make sure you activate the virtual environment if you choose virtual environment-based testing.
- -# Example
-!python test_env.py --input_dir "E:/data/input/" --output_dir "E:/data/output/"
-Run the the following if you choose Docker-based testing:
-Option 1:run_docker_image.bat E:/data/input/ E:/data/output DockerName Latest ai_1
Option 2: docker run -it --rm --name ai_1 -v E:/data/input/:/input -v E:/data/output/:/output DockerName:Latest
Step 5: Result visualization and interpretation: examine the output folder¶
-# Let us check the output directory.
-main_directory = "E:/data/output/"
-list_folders_files(main_directory)
--
-
Given N patients in the input directory, the system produces N resized (4x4x4 voxel size) and cropped (128x128x256 resolution) 3D data under the folder name
-data_default_3d_dir_. -
-It generates corresponding maximum intensity projections (MIPs) and is saved under the folder name
-data_default_MIP_dir.
-
-The LBFNet uses the generated MIP images under the folder
-data_default_MIP_dirto predict the lymphoma tumor regions. -
-The predicted images are saved under the folder name
-predicted_data.-
-Two excel files are generated.
--
-
The first excel file is with the name
-surrogate_ground_truth.csvfor surrogate biomarker features calculated from the ground truth segmentations (lymphoma tumor regions from an expert), if available. If there is no ground truth data provided along with the pet images, this file will have all its values with zero.
-The second CSV file,
-surrogate_predicted.csv, contains the surrogate biomarkers predicted by the AI algorithm.
-
-
Step 6: Excel or csv file interpretation¶
-# read the csv files
-df_predicted = pd.read_csv("E:/data/output/surrogate_predicted.csv", encoding='latin1')
-df_predicted.head()
-# Simple statistical description of the data.
-df_predicted.describe()
-Note: PID: Unique patient identifier, sTMTV: surrogate total metabolic tumor volume in pixels. _coronal and _sagittal indicate that the values were computed from the coronal and sagittal views, respectively. Sagittal_xy and coronal_xy indicate the dissemination along the width of the 2D sagittal and coronal PET MIP images, and the Sagittal_z and coronal_z indicate the dissemination along with the height of the 2D sagittal and coronal PET MIP images. The following figure illustrates computing the dissemination from the coronal view.
-
The x, y, and z are the voxel spacing of the images.
- -# Visualize the distribution of the computed biomarkers.
-plot_biomarker = ["sTMTV_(mm²)", "sDmax_(mm)"]
-for biomarker in plot_biomarker:
- df_predicted[biomarker].hist()
- plt.title(f"Histogram of {biomarker}")
- plt.show()
-
-
-Step 7: Visualization of the PET and their segmentation results¶
-# data path
-path = "E:/data/output/predicted_data//"
-view.read_predicted_images(path)
-
-
-
-
-
-
-
-
-Note that if no ground truth is provided it will only display the predicted images.
- -Good job!
- -