Explainable-Neural-Networks

Introduction

Following is the initial file structure.

Folders/File structure

.
├── data
│   ├── test
│   │   ├── normal
│   │   ├── bacteria
│   │   └── virus
│   ├── train
│   │   ├── normal
│   │   ├── bacteria
│   │   └── virus
│   └── val
│       ├── normal
│       ├── bacteria
│       └── virus
├── conda-env.yaml
├── src
├── LICENSE
├── notebooks
└── README.md

Getting Started

1. Data

Curl down and unzip the data to the existing repositry using the command:

# download using
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1VuaSBUw2MFTbobZ2ZcVjugVx-ey88xkF' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1VuaSBUw2MFTbobZ2ZcVjugVx-ey88xkF" -O pneumonia.zip && rm -rf /tmp/cookies.txt
# Unzip using
unzip -q pneumonia.zip .

Alternatively download the dataset from here.

2. Environment

Please ensure dependencies adhere to python 3.10

conda env create -f conda-env.yaml
conda activate xnn

1. Pre-commit hook

More information on precommit hook here.

pre-commit install

FiftyOne

FiftyOne is a widely used solution to quickly view multiple images for computer vision tasks. Please run the following commands in docker to view the data, data has already been tagged with both "normal", "bacteria" and "virus" images for quick comparison.

1. Build the service using the command:

docker build -f docker/fiftyone.Dockerfile \
    -t fiftyone:0.1.0 \
    --platform linux/amd64 .  

2. Run the service using the command:

docker run -p 5151:5151 \
    --name fiftyone \
    -v ~/Documents/coding/personal/Explainable-Neural-Networks/data:/data \
    fiftyone:0.1.0    

3. Access the fiftyone dashboard using the link http://localhost:5151 on your local computer

4. Stop the service using the command:

docker container stop fiftyone   

5. To remove the container use the command:

docker rm fiftyone   

t-SNE

t-Distributed Stochastic Neighbor Embedding (t-SNE) is an unsupervised, non-linear technique primarily used for data exploration and visualizing high-dimensional data. In this project, a t-SNE vizualization has been created to better allow users to understand the images.

(Above) Image sprite showing the images being classified using RESNET embeddings

(Above) Tensorboard showing t-SNE clustering according to the categories "normal", "bacteria" and "virus".

Contributing

• Outstanding features are listed in the project's kanban board here!
• Dont see an issue you want? raise an issue here

Commit msg

<type>: <short summary>
  │            │
  │            └─⫸ Summary in present tense. Not capitalized. No period at the end.
  │
  └─⫸ Commit Type: build|cicd|docs|feat|fix|node|refactor|test