done migrating previous docs

v1.1.0 almost complete and created workflow

Author: zhaozewang

.DS_Store

@@ -0,0 +1,64 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# Sample workflow for building and deploying a Jekyll site to GitHub Pages
+name: Deploy Jekyll site to Pages
+
+on:
+  # Runs on pushes targeting the default branch
+  push:
+    branches: ["main"]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
+# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  # Build job
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Setup Ruby
+        uses: ruby/setup-ruby@8575951200e472d5f2d95c625da0c7bec8217c42 # v1.161.0
+        with:
+          ruby-version: '3.2.2' # Not needed with a .ruby-version file
+          bundler-cache: true # runs 'bundle install' and caches installed gems automatically
+          cache-version: 0 # Increment this number if you need to re-download cached gems
+      - name: Setup Pages
+        id: pages
+        uses: actions/configure-pages@v5
+      - name: Build with Jekyll
+        # Outputs to the './_site' directory by default
+        run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}"
+        env:
+          JEKYLL_ENV: production
+      - name: Upload artifact
+        # Automatically uploads an artifact from the './_site' directory by default
+        uses: actions/upload-pages-artifact@v3
+
+  # Deployment job
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.github/workflows/jekyll.yml

@@ -14,5 +14,13 @@ Artificial Neural Networks (ANNs) trained with backpropagation, despite being bi
 
 This project implements Recurrent Neural Networks (RNNs) and Multilayer Perceptrons (MLPs) designed for a parametrized and granular control over network modularity, synaptic plasticity, and other constraints to enable biologically feasible modeling of brain regions.
 
+[GitHub Repository](https://github.com/NN4Neurosim/nn4n)
+
 #### Acknowledgement
 Immense thanks to [Dr. Christopher J. Cueva](https://www.metaconscious.org/author/chris-cueva/) for his mentorship in developing this project. This project can't be done without his invaluable help.
+
+#### License
+This project is licensed under the terms of the MIT license. See the [LICENSE](https://opensource.mit.edu/license) file for details.
+
+#### Template
+The project documentation is based on [Jekyll](https://jekyllrb.com/) and uses [Jekyll Gitbook](https://github.com/sighingnow/jekyll-gitbook) theme developed by [sighingnow](https://github.com/sighingnow).

README.md

@@ -7,18 +7,18 @@ layout: post
 order: 1
 ---
 
-### Install using pip
+# Install using pip
 ```
 pip install nn4n
 ```
 
-### Install from GitHub
+# Install from Source
 #### Clone the repository
 ```
-git clone https://github.com/zhaozewang/NN4Neurosci.git
+git clone https://github.com/NN4Neurosim/nn4n.git
 ```
 #### Navigate to the NN4Neurosci directory
 ```
-cd NN4Neurosci/
+cd nn4n/
 pip install -e .
 ```

_collection_1/installation.md

@@ -7,4 +7,62 @@ layout: post
 order: 2
 ---
 
+## Initialize a Continuous-Time RNN
+```python
+import torch
+from nn4n.model import CTRNN
 
+rnn = CTRNN(input_dim=1, hidden_size=10, output_dim=1)
+optimizer = torch.optim.Adam(rnn.parameters(), lr=0.001)
+```
+
+
+## Define a Task
+The input/output signals to train the RNN can be any time series data. Let $X$ be the input signal and $Y$ be the output signal. $X$ should of shape `(n_timesteps, batch_size, input_dim)` and $Y$ should be of shape `(n_timesteps, batch_size, output_dim)`. These signals could be representations of many cognitive tasks, such as working memory, decision making, or motor control, etc. Here we use a simple sin wave as an example.
+```python
+import numpy as np
+import matplotlib.pyplot as plt
+
+# predict sin wave
+inputs = np.sin(np.linspace(0, 10, 1000))
+inputs = torch.from_numpy(inputs).float().unsqueeze(1).unsqueeze(1)
+labels = inputs[1:]
+inputs = inputs[:-1]
+
+plt.plot(inputs.squeeze(1).squeeze(1).numpy())
+plt.plot(labels.squeeze(1).squeeze(1).numpy())
+plt.show()
+```
+
+<p align="center">
+  <img src="{{ '/assets/images/results/sin_wave.png' | relative_url }}" width="400" alt="Sin Wave">
+</p>
+
+## Train the RNN
+```python
+losses = []
+for epoch in range(500):
+    outputs, states = rnn(inputs)
+    loss = torch.nn.MSELoss()(outputs, labels)
+    optimizer.zero_grad()
+    loss.backward()
+    optimizer.step()
+    losses.append(loss.item())
+
+    if epoch % 50 == 0:
+        print(f'Epoch {epoch} Loss {loss.item()}')
+```
+
+##### Output:
+```
+Epoch 0 Loss 0.3866065442562103
+Epoch 50 Loss 0.20944912731647491
+Epoch 100 Loss 0.03360378369688988
+Epoch 150 Loss 0.016431370750069618
+Epoch 200 Loss 0.013084247708320618
+Epoch 250 Loss 0.010527823120355606
+Epoch 300 Loss 0.007640092633664608
+Epoch 350 Loss 0.005286946427077055
+Epoch 400 Loss 0.003560091834515333
+Epoch 450 Loss 0.0028597351629287004
+```

_collection_1/quickstart.md

@@ -1,13 +1,12 @@
 ---
-title: Excitatory-Inhibitory RNN
+title: 'Example: Excitatory-Inhibitory RNN'
 author: Zhaoze Wang
 date: 2024-06-16
 category: docs
 layout: post
-order: 4
+order: 11
 ---
 
-To train a Excitatory-Inhibitory Constrained RNN (EIRNN)
 ```python
 import nn4n
 from nn4n.model import CTRNN

_collection_2/eirnn.md

@@ -1,17 +1,17 @@
 ---
-title: RNN Introduction
+title: Introduction
 author: Zhaoze Wang
 date: 2024-06-14
 category: docs
 layout: post
 order: 1
 ---
 
-## Mathematical Formulation
+# Mathematical Formulation
 
 The terms 'nodes', 'neurons', and 'units' are used interchangably when referring nodes in the RNN/MLP models. Whereas the biological neurons are only referred to as neurons.
 
-### Simplified Network Model
+## Simplified Network Model
 
 The firing rate of a single neuron is described by the following equation:
 
@@ -36,7 +36,7 @@ $$ \mathbf{r}^t = f\left( \mathbf{v}^t \right) $$
 - $ \mathbf{W} $: Weight matrix.
 - $ \mathbf{v}^{t-1} $: Vector of membrane potentials for all neurons at time $ t-1 $.
 
-### RNN Dynamics
+## RNN Dynamics
 At every timestep, we assume that the neurons in the modeled brain region receive external inputs and signals from their neighboring neurons. These signals are then non-linearly integrated to produce an output. The dynamics of the Recurrent Neural Network (RNN) can be described by the following differential equation:
 
 $$\tau \frac{d \mathbf{v}}{dt} = -\mathbf{v}^t + \mathbf{W}_{hid} f(\mathbf{v}^t) + \mathbf{W}_{in} \mathbf{u}^t + \mathbf{b}_{hid} + \epsilon_t$$
@@ -62,7 +62,18 @@ $$
 \Delta \mathbf{v}^t = \gamma (-\mathbf{v}^t + \mathbf{W}_{hid} f(\mathbf{v}^t) + \mathbf{W}_{in} \mathbf{u}^t + \mathbf{b}_{hid} + \epsilon_{t}) + \xi_{t}
 $$
 
-## Vanilla CTRNN
+
+# Model Structure
+```
+├── CTRNN
+│   ├── RecurrentLayer
+│   │   ├── InputLayer (class LinearLayer)
+│   │   ├── HiddenLayer
+│   ├── Readout_areas (class LinearLayer)
+```
+
+
+# Vanilla CTRNN
 A simplistic CTRNN contains three layers, an input layer, a hidden layer, and an readout layer as depicted below.
 
 <p align="center">
@@ -71,13 +82,13 @@ A simplistic CTRNN contains three layers, an input layer, a hidden layer, and an
 
 The yellow nodes represent neurons that project input signals to the hidden layer, the green neurons are in the hidden layer, and the purple nodes represent neurons that read out from the hidden layer neurons. Both input and readout neurons are 'imagined' to be there. I.e., they only project or receives signals and therefore do not have activations and internal states.
 
-## Excitatory-Inhibitory Constrained CTRNN
-The implementation of CTRNN also supports Excitatory-Inhibitory constrained continuous-time RNN (EIRNN). EIRNN is proposed by H. Francis Song, Guangyu R. Yang, and Xiao-Jing Wang in [Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework](https://doi.org/10.1371/journal.pcbi.1004792)
+# Excitatory-Inhibitory Constrained CTRNN
+The implementation of CTRNN also supports Excitatory-Inhibitory constrained continuous-time RNN (EIRNN) similar to what was proposed by H. Francis Song, Guangyu R. Yang, and Xiao-Jing Wang in [Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework](https://doi.org/10.1371/journal.pcbi.1004792)
 
 A visual illustration of the EIRNN is shown below.
 
 <p align="center">
   <img src="{{ '/assets/images/basics/EIRNN_structure.png' | relative_url }}" width="400" alt="Description of Exciatory-Inhibitory RNN Structure">
 </p>
 
-The yellow nodes denote nodes in the input layer. The middle circle denotes the hidden layer. There are blue nodes and red nodes, representing inhibitory neurons and excitatory neurons, respectively. The depicted network has an E/I ratio of 4/1. The purple nodes are ReadoutLayer neurons.
+The yellow nodes denote nodes in the input layer. The middle circle denotes the hidden layer. There are blue nodes and red nodes, representing inhibitory neurons and excitatory neurons, respectively. The depicted network has an E/I ratio of 4/1. The purple nodes are ReadoutLayer neurons.