diff --git a/.gitignore b/.gitignore index 45c8c19..e6bf6ce 100644 --- a/.gitignore +++ b/.gitignore @@ -11,7 +11,9 @@ test-trainer bert_dataset trained_model results +results_qlora output +qlora_final_model src tmp diff --git a/README.md b/README.md index 9091e98..5670640 100644 --- a/README.md +++ b/README.md @@ -16,4 +16,4 @@ A sandbox environment to experiment with large language models and various NLP t - **Fine-tuning Series**: Develop a series of proof-of-concept (POC) repositories for fine-tuning LLMs, hosted on GitHub (e.g.,[ft2](https://github.com/locchh/ft2), [ft3](https://github.com/locchh/ft3), etc.). 🔥 - **LLM Blog**: Write and publish insightful blog posts on LinkedIn, sharing perspectives and research related to LLMs (e.g.,[Artificial Intelligence For Programming](https://www.linkedin.com/posts/chuong-loc_artificialintelligence-softwaredevelopment-activity-7278039943480856576-2o3E?utm_source=share), [The Limitation of Context](https://www.linkedin.com/posts/chuong-loc_%F0%9D%90%93%F0%9D%90%A1%F0%9D%90%9E-%F0%9D%90%8B%F0%9D%90%A2%F0%9D%90%A6%F0%9D%90%A2%F0%9D%90%AD%F0%9D%90%AC-%F0%9D%90%A8%F0%9D%90%9F-%F0%9D%90%86%F0%9D%90%A2%F0%9D%90%AF%F0%9D%90%A2%F0%9D%90%A7%F0%9D%90%A0-%F0%9D%90%82%F0%9D%90%A8%F0%9D%90%A7%F0%9D%90%AD%F0%9D%90%9E%F0%9D%90%B1%F0%9D%90%AD-activity-7274037081000034304-tTY4?utm_source=share), etc.). 📝🌐 - **POC Repositories**: Create and share repositories showcasing new technologies and quick demonstrations for easy exploration (e.g.,[lr4lg](https://github.com/locchh/lr4lg), [debugger](https://github.com/locchh/debugger), etc.). 📂 -- **Build Continuously Growing Repositories**: Develop and maintain long-term, evolving repositories for ongoing projects (e.g., [Synthflow](https://github.com/locchh/synthflow), [Antflow](https://github.com/locchh/antflow), [Antelligence](https://github.com/locchh/antelligence), etc.). 🔄 +- **Build Continuously Growing Repositories**: Develop and maintain long-term, evolving repositories for ongoing projects (e.g., [Synthflow](https://github.com/locchh/synthflow), [Antflow](https://github.com/locchh/antflow), [Antelligence](https://github.com/locchh/antelligence), etc.). 🔄 \ No newline at end of file diff --git a/docs/render.html b/docs/generative_AI _engineering_and_fine-tuning_transformers.html similarity index 100% rename from docs/render.html rename to docs/generative_AI _engineering_and_fine-tuning_transformers.html diff --git a/docs/instruction_tuning_best_practices.md b/docs/instruction_tuning_best_practices.md new file mode 100644 index 0000000..b5fd7a5 --- /dev/null +++ b/docs/instruction_tuning_best_practices.md @@ -0,0 +1,43 @@ + +# Best Practices for Instruction-Tuning Large Language Models + +Instruction-tuning is a powerful fine-tuning approach that adapts large language models (LLMs) to follow specific instructions more effectively, enhancing their usefulness in practical applications. Below, we outline best practices for optimizing instruction-tuning in LLMs. + +## Data Selection for Instruction-Tuning + +High-quality data is crucial for effective instruction-tuning. The selected data should reflect diverse instructions and responses to help the model generalize and respond accurately across varied scenarios. + +- **Diverse Dataset Collection**: Use datasets that cover a wide range of topics, contexts, and instructions. Including different prompt types and response styles helps the model handle a broader set of instructions. +- **Balance of Specialized and General Data**: While it's beneficial to include domain-specific instructions, balancing this with general data improves versatility, allowing the model to perform well across various domains. + +## Optimize Prompt Engineering + +Effective prompt engineering enables the model to understand and respond appropriately to different instructions. + +- **Contextual Prompt Design**: Design prompts that reflect real-world use cases and specific contexts the model might encounter. For instance, instructions could vary in formality, complexity, or specificity, helping the model adapt to different audiences. +- **Testing Prompt Variability**: Experiment with different prompts to assess how well the model generalizes to unseen instructions. This helps ensure that the model doesn't overly rely on specific patterns or structures. + +## Measure Response Consistency + +Consistency in response quality is key to creating a reliable model. + +- **Evaluate Accuracy and Consistency**: Regularly test the model with similar instructions to measure consistency. Consistent and accurate responses to repeated instructions indicate a well-tuned model. +- **Monitor Task-Specific Performance**: If the model is tuned for a specialized application, evaluate its performance across task-specific scenarios to ensure consistency within that context. + +## Limit Overfitting on Instruction Style + +Overfitting on specific instruction styles or tones can reduce the model’s adaptability. + +- **Style Variety in Instructions**: Include a variety of tones and structures in the instruction dataset to avoid making the model too reliant on specific formats. +- **Balance Precision and Flexibility**: Fine-tune the model to be precise in its responses without limiting its ability to adapt to different instruction types. This balance helps create a model that is accurate yet flexible in understanding various instructions. + +## Implement Regular Evaluation Metrics + +Regular evaluation of the fine-tuned model ensures it meets the desired quality standards. + +- **Use Metrics for Instruction Adherence**: Implement metrics that evaluate how closely the model's responses align with provided instructions. +- **Human Review and Quality Checks**: Regular human review of model responses provides insights that are difficult to capture with automated metrics, adding another layer of evaluation for adherence and appropriateness. + +## Conclusion + +Following these best practices for instruction-tuning can significantly enhance an LLM's performance, enabling it to respond more accurately and flexibly to a wide array of instructions. By focusing on quality data, diverse prompt engineering, and regular evaluation, you can create an instruction-tuned model that is both effective and reliable in real-world applications. diff --git a/docs/peft.md b/docs/peft.md new file mode 100644 index 0000000..c64ae06 --- /dev/null +++ b/docs/peft.md @@ -0,0 +1,136 @@ + +### **LoRA (Low-Rank Adaptation):** + +- **Core Idea:** + LoRA introduces trainable low-rank matrices into the model, targeting specific weight matrices (e.g., attention layers). Instead of fine-tuning the full model, LoRA adds a low-rank decomposition to certain layers and fine-tunes only these low-rank matrices. + +- **Implementation:** + - Adds two small low-rank matrices \( A \) and \( B \) to the original weights \( W \) of the model, where \( \Delta W = AB^T \). + - The original pre-trained weights \( W \) remain frozen during training, and only \( A \) and \( B \) are updated. + +- **Advantages:** + - Memory efficient: Fewer parameters are updated. + - Doesn't require modifying the model architecture significantly. + - Easy to integrate into transformer-based models. + +- **Use Case:** + - Primarily used for fine-tuning language models (e.g., LLaMA, GPT). + - Common in NLP and generative tasks. + +--- + +### **Adapters:** + +- **Core Idea:** + Adapters are small trainable modules inserted into the layers of a pre-trained model. These modules are trained while keeping the original model weights frozen. + +- **Implementation:** + - Adapters are typically lightweight neural network modules (e.g., feedforward layers) inserted between layers of a model, such as attention or feedforward blocks. + - During training, only the parameters of the adapters are updated, while the rest of the model remains fixed. + +- **Advantages:** + - Modular: Different tasks can have separate adapters without modifying the original model. + - Memory efficient: Reduces the need to fine-tune the entire model. + - Easy task-switching by replacing adapters. + +- **Use Case:** + - Popular in multi-task learning and scenarios requiring task-specific fine-tuning. + - Useful in both NLP and multimodal applications. + +--- + +### **Key Differences:** + +| Feature | LoRA | Adapters | +|-----------------------|-------------------------------|--------------------------------| +| **Architecture** | Adds low-rank matrices to weight updates. | Inserts small modules between layers. | +| **Frozen Parameters** | Original model weights are frozen. | Original model weights are frozen. | +| **Parameter Updates** | Updates low-rank matrices \( A, B \). | Updates adapter parameters only. | +| **Overhead** | Minimal: modifies specific weight matrices. | Moderate: introduces new modules into the model. | +| **Use Cases** | NLP fine-tuning, generative tasks. | Multi-task learning, task-specific adaptation. | + +--- + +Both methods are highly efficient and suitable for scenarios where training large models directly is infeasible. However, the choice depends on your use case, such as the number of tasks, modularity requirements, and computational constraints. + + +### **1. LoRA for Fine-Tuning** + +#### **Advantages:** + +1. **Efficiency:** + - LoRA modifies only a small subset of trainable parameters (the low-rank matrices), making it highly parameter-efficient. +2. **Minimal Overhead:** + - No additional modules are inserted into the architecture; only the weight matrices of certain layers are modified. +3. **Better for Single Tasks:** + - Works well for scenarios where you want to fine-tune on a single or specific domain/task without modularity concerns. +4. **Speed:** + - Training and inference are faster compared to adapters due to the lightweight nature of LoRA’s modifications. + +#### **Use Cases:** +- Single-task fine-tuning. +- Resource-constrained environments (e.g., limited GPU memory). +- Generative tasks such as text generation or summarization. + +#### **Limitations:** +- Less modular: Difficult to manage multiple tasks or transfer fine-tuned components across models. +- Potentially less effective when task-switching is required. + +--- + +### **2. Adapters for Fine-Tuning** + +#### **Advantages:** + +1. **Modularity:** + - Adapters are ideal for multi-task setups since each task can have its own adapter, enabling seamless task switching. +2. **Transfer Learning:** + - Adapters trained on one task can be reused or adapted for related tasks, making them versatile. +3. **Isolation:** + - Fine-tuning with adapters avoids interference between tasks, which is especially useful in multi-task or federated learning setups. + +#### **Use Cases:** +- Multi-task learning or scenarios requiring task switching. +- Incremental fine-tuning on new tasks/domains. +- Applications where modularity and reusability of components are important. + +#### **Limitations:** +- Higher computational overhead compared to LoRA due to added modules. +- Slightly more complex integration into existing architectures. + +--- + +### **Comparison Table:** + +| Feature | LoRA | Adapters | +|--------------------------|---------------------------------|---------------------------------| +| **Parameter Efficiency** | High | Moderate | +| **Modularity** | Low | High | +| **Fine-Tuning Overhead** | Minimal | Moderate | +| **Use Case** | Single-task fine-tuning | Multi-task or modular setups | +| **Scalability** | Limited for multiple tasks | Scalable for multi-task setups | +| **Inference Efficiency** | Higher | Lower due to added modules | + +--- + +### **What’s Better?** + +- **LoRA** is generally better if: + - You are working on a single task or domain. + - You prioritize efficiency and minimal computational overhead. + - You have limited resources (e.g., GPU memory). + +- **Adapters** are better if: + - You need to fine-tune on multiple tasks or domains. + - Task modularity and reusability are important. + - You want a framework that supports incremental learning. + +--- + +### **Best Practices:** +- **Experimentation:** + If resources allow, experiment with both methods to see which performs better for your specific task. +- **Hybrid Approaches:** + Recent work combines both methods for more efficient and effective fine-tuning. +- **Task-Specific Considerations:** + Consider the complexity of the task, the expected level of generalization, and memory constraints. \ No newline at end of file diff --git a/docs/render.pdf b/docs/soft_prompts.pdf similarity index 100% rename from docs/render.pdf rename to docs/soft_prompts.pdf diff --git a/notebooks/LLM_Specialization/Adapters in PyTorch.ipynb b/notebooks/LLM_Specialization/Adapters in PyTorch.ipynb deleted file mode 100644 index 7f236d0..0000000 --- a/notebooks/LLM_Specialization/Adapters in PyTorch.ipynb +++ /dev/null @@ -1,3389 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "b75e0832-1651-4d74-85dd-14fd49ef56e3", - "metadata": {}, - "source": [ - "

\n", - " \n", - " \"Skills\n", - " \n", - "

\n", - "\n", - "# **Adapters in PyTorch**\n", - "\n", - "Estimated time needed: **45** minutes\n", - "\n", - "**_Note to advanced users_: If you are already familiar with classical fine-tuning and you only want to see the section that relates to adapters, skip forward to Adapters and run all of the cells above that section by going to _Run --> Run All Above Selected Cell_**\n", - "\n", - "You can fine-tune a neural network in several ways. Common strategies include adjusting only the final layer or fine-tuning all layers. However, these methods have their drawbacks: fine-tuning just the final layer often leads to less than optimal results, while fine-tuning all layers can be very time-consuming.\n", - "\n", - "To address these issues, researchers have developed various parameter efficient fine-tuning (PEFT) techniques. One such technique involves the use of adapters. Adapters enable modular training, where small, task-specific modules are trained within the model without changing the pre-existing pretrained parameters. This approach efficiently tailors the model to new tasks with a reduced risk of overfitting. However, adapters are not a cure-all solution. While they are less likely to overfit and are computationally efficient, they might not always reach the same level of accuracy as full model fine-tuning, particularly if the task necessitates substantial changes from the pretrained model's original capabilities.\n", - "\n", - "In this hands-on lab, you learn how to apply an adapter to a transformer-based neural network that has been trained on the AG News data set, with the aim of using this model on the IMDB data set. You also evaluate and compare the performance of this method with that of a fully fine-tuned model and a model where only the last layer is fine-tuned.\n", - "\n", - "---\n", - "\n", - "# __Table of contents__\n", - "\n", - "
    \n", - "
  1. Objectives
    2. \n", - "
  2. \n", - " Setup\n", - "
      \n", - "
    1. Install required libraries
      2. \n", - "
    2. Import required libraries
      3. \n", - "
    3. Defining helper functions
      4. \n", - "
    \n", - "
    3. \n", - "
  3. Positional encodings
    4. \n", - "
  4. Import IMDB data set
    5. \n", - "
      \n", - "
    1. IMDB data set overview
      2. \n", - "
        \n", - "
      1. Data set composition
        2. \n", - "
      2. Applications
        3. \n", - "
      3. Challenges
        4. \n", - "
      4. Data set splits
        5. \n", - "
      5. Data loader
        6. \n", - "
      6. Neural network
        7. \n", - "
      \n", - "
    \n", - "
  5. \n", - " Training\n", - "
      \n", - "
    1. Train IMDB
      2. \n", - "
    2. Fine-tune a model pretrained on the AG News data set
      3. \n", - "
    3. Fine-tune the final layer only
      4. \n", - "
    \n", - "
    6. \n", - "
  6. \n", - " Adapters\n", - "
      \n", - "
    1. Benefits of using adapters in neural networks
      2. \n", - "
    \n", - "
    7. \n", - "
  7. \n", - " Exercise: Adapt linear layers in a different network\n", - "
    8. \n", - "
\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9e4d071d-35d8-4d69-9168-a40d25df2205", - "metadata": {}, - "source": [ - "# Objectives\n", - "\n", - "After completing this lab, you are able to:\n", - "\n", - "- Define and pretrain a transformer-based neural network using PyTorch for a classification task [Optional]\n", - "- Fully fine-tune the pretrained model for a different classification task [Optional]\n", - "- Compare results by fine-tuning only the last layer of the pretrained model [Optional]\n", - "- Understand how adapters work\n", - "- Apply adapters to linear layers in a neural network\n", - "- Train a neural network in a parameter efficient way by training just the adapted layers\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "35a85867-b5fa-4780-b512-35bdf829b33c", - "metadata": {}, - "source": [ - "# Setup\n", - "\n", - "### Install required libraries\n", - "\n", - "For this lab, you use the following libraries, which are __not__ preinstalled in the Skills Network Labs environment. __You must run the code in the following cell__ to install them.\n", - "\n", - "```bash\n", - "!pip install --upgrade portalocker==2.8.2 torchtext==0.17.0 torchdata==0.7.1 pandas==2.2.2 matplotlib==3.9.0 scikit-learn==1.5.0 torch==2.2.0 numpy==1.26.4\n", - "```\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "a2d7c12f-2141-4af9-91a8-b0c1c8088d66", - "metadata": {}, - "source": [ - "### Import required libraries\n", - "\n", - "The following code imports the required libraries.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "81927908-a8c5-42d6-b24c-97b8b13a42e6", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "True\n", - "Tesla P40\n", - "Import Successfully!\n" - ] - } - ], - "source": [ - "# Environment setup\n", - "import os\n", - "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n", - "\n", - "# Suppress warnings\n", - "import warnings\n", - "warnings.filterwarnings('ignore')\n", - "def warn(*args, **kwargs):\n", - " pass\n", - "warnings.warn = warn\n", - "\n", - "# PyTorch and related libraries\n", - "import torch\n", - "from torch import nn\n", - "from torch.utils.data import DataLoader, Dataset\n", - "from torch.utils.data.dataset import random_split\n", - "from torch.nn.utils.rnn import pad_sequence\n", - "\n", - "# TorchText for NLP tasks\n", - "from torchtext.datasets import AG_NEWS, IMDB\n", - "from torchtext.data.utils import get_tokenizer\n", - "from torchtext.vocab import build_vocab_from_iterator, GloVe, Vectors\n", - "from torchtext.data.functional import to_map_style_dataset\n", - "\n", - "# Utility libraries\n", - "import time\n", - "from itertools import accumulate\n", - "import math\n", - "import pickle\n", - "import io\n", - "from urllib.request import urlopen\n", - "import tarfile\n", - "import tempfile\n", - "\n", - "# Data manipulation and visualization\n", - "import numpy as np\n", - "import pandas as pd\n", - "import matplotlib.pyplot as plt\n", - "from tqdm import tqdm\n", - "\n", - "# Jupyter Notebook utilities\n", - "from IPython.display import Markdown as md\n", - "\n", - "# PyTorch-specific configurations\n", - "torch.set_num_threads(1)\n", - "\n", - "# CUDA-related checks\n", - "print(torch.cuda.is_available())\n", - "print(torch.cuda.get_device_name())\n", - "\n", - "print(\"Import Successfully!\")" - ] - }, - { - "cell_type": "markdown", - "id": "5e710480-51b4-46c7-a3b4-5c80021007b2", - "metadata": {}, - "source": [ - "### Define helper functions\n", - "\n", - "The following code shows some helper functions to help with plotting, saving, and loading files. These functions are not the main focus of this lab, so you do not have to dwell on these too long. However, do run the cells in this section to define these helper functions.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "57d49c2a-d309-4608-8e45-b9fa8ec63cbd", - "metadata": {}, - "outputs": [], - "source": [ - "def plot(COST,ACC):\n", - "\n", - " fig, ax1 = plt.subplots()\n", - " color = 'tab:red'\n", - " ax1.plot(COST, color=color)\n", - " ax1.set_xlabel('epoch', color=color)\n", - " ax1.set_ylabel('total loss', color=color)\n", - " ax1.tick_params(axis='y', color=color)\n", - "\n", - " ax2 = ax1.twinx()\n", - " color = 'tab:blue'\n", - " ax2.set_ylabel('accuracy', color=color) # you already handled the x-label with ax1\n", - " ax2.plot(ACC, color=color)\n", - " ax2.tick_params(axis='y', color=color)\n", - " fig.tight_layout() # otherwise the right y-label is slightly clipped\n", - "\n", - " plt.show()\n", - "\n", - "\n", - "def save_list_to_file(lst, filename):\n", - " \"\"\"\n", - " Save a list to a file using pickle serialization.\n", - "\n", - " Parameters:\n", - " lst (list): The list to be saved.\n", - " filename (str): The name of the file to save the list to.\n", - "\n", - " Returns:\n", - " None\n", - " \"\"\"\n", - " with open(filename, 'wb') as file:\n", - " pickle.dump(lst, file)\n", - "\n", - "\n", - "def load_list_from_file(filename):\n", - " \"\"\"\n", - " Load a list from a file using pickle deserialization.\n", - "\n", - " Parameters:\n", - " filename (str): The name of the file to load the list from.\n", - "\n", - " Returns:\n", - " list: The loaded list.\n", - " \"\"\"\n", - " with open(filename, 'rb') as file:\n", - " loaded_list = pickle.load(file)\n", - " return loaded_list" - ] - }, - { - "cell_type": "markdown", - "id": "fb4827b6-33c9-4370-bfbf-983d89623c98", - "metadata": {}, - "source": [ - "---" - ] - }, - { - "cell_type": "markdown", - "id": "0d6f6a86-020b-4ed5-8c52-5fa69dceca97", - "metadata": {}, - "source": [ - "# Positional encodings\n", - "\n", - "Positional encodings play a pivotal role in transformers and various sequence-to-sequence models, aiding in conveying critical information regarding the positions or sequencing of elements within a given sequence. To illustrate, let's examine the sentences: \"He painted the car red\" and \"He painted the red car.\" Despite their distinct meanings, it's worth noting that the embeddings for these sentences remain identical in the absence of positional encodings. The following class defines positional encodings by inheriting from PyTorch's `Module` class.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "88c9cebf-7dbe-46d0-81d2-5a116120c374", - "metadata": {}, - "outputs": [], - "source": [ - "class PositionalEncoding(nn.Module):\n", - " \"\"\"\n", - " https://pytorch.org/tutorials/beginner/transformer_tutorial.html\n", - " \"\"\"\n", - "\n", - " def __init__(self, d_model, vocab_size=5000, dropout=0.1):\n", - " super().__init__()\n", - " self.dropout = nn.Dropout(p=dropout)\n", - "\n", - " pe = torch.zeros(vocab_size, d_model)\n", - " position = torch.arange(0, vocab_size, dtype=torch.float).unsqueeze(1)\n", - " div_term = torch.exp(\n", - " torch.arange(0, d_model, 2).float()\n", - " * (-math.log(10000.0) / d_model)\n", - " )\n", - " pe[:, 0::2] = torch.sin(position * div_term)\n", - " pe[:, 1::2] = torch.cos(position * div_term)\n", - " pe = pe.unsqueeze(0)\n", - " self.register_buffer(\"pe\", pe)\n", - "\n", - " def forward(self, x):\n", - " x = x + self.pe[:, : x.size(1), :]\n", - " return self.dropout(x)" - ] - }, - { - "cell_type": "markdown", - "id": "aadc814f-d060-47be-963f-c28cfd0618e4", - "metadata": {}, - "source": [ - "# Import IMDB data set\n", - "\n", - "The following code loads the IMDB data set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "c479b278-01b9-4863-9821-528b1607a74b", - "metadata": {}, - "outputs": [], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/35t-FeC-2uN1ozOwPs7wFg.gz')\n", - "tar = tarfile.open(fileobj=io.BytesIO(urlopened.read()))\n", - "tempdir = tempfile.TemporaryDirectory()\n", - "tar.extractall(tempdir.name)\n", - "tar.close()" - ] - }, - { - "cell_type": "markdown", - "id": "ec7531bd-8b33-45d9-8061-9de43bd188b8", - "metadata": {}, - "source": [ - "## IMDB data set overview\n", - "\n", - "The **IMDB data set** contains movie reviews from the Internet Movie Database (IMDB) and is commonly used for binary sentiment classification tasks. It's a popular data set for training and testing models in natural language processing (NLP), particularly in the context of sentiment analysis.\n", - "\n", - "### Data set composition\n", - "\n", - "- **Reviews**: The data set consists of 50,000 movie reviews, divided evenly into 25,000 training and 25,000 testing samples.\n", - "- **Sentiment labels**: Each review is labeled as either positive or negative, indicating the sentiment expressed in the review. The data set is balanced, with an equal number of positive and negative reviews in both the training and testing sets.\n", - "- **Text content**: Reviews are presented as plain text and have been preprocessed to some extent. For example, HTML tags are removed, but the text retains its original punctuation and capitalization.\n", - "- **Usage**: The data set is commonly used to train models for binary sentiment classification, where the goal is to predict whether a given review is positive or negative based on its text content.\n", - "\n", - "### Applications\n", - "\n", - "- **Sentiment analysis**: The primary application of the IMDB data set is in sentiment analysis, where it serves as a benchmark for various text classification algorithms.\n", - "- **Natural language processing**: The data set is widely used in NLP research and applications, providing a basis for testing the effectiveness of different models and approaches in understanding human language.\n", - "\n", - "### Challenges\n", - "\n", - "The data set is small, so it's hard to train a model from scratch.\n", - "\n", - "The following class is defined to traverse the IMDB data set. The need to define this class arises from the fact that the IMDB data set is split across a large number of files.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "51bc66a7-506d-4ac4-aa0b-1813c6a0e4c5", - "metadata": {}, - "outputs": [], - "source": [ - "class IMDBDataset(Dataset):\n", - " def __init__(self, root_dir, train=True):\n", - " \"\"\"\n", - " root_dir: The base directory of the IMDB dataset.\n", - " train: A boolean flag indicating whether to use training or test data.\n", - " \"\"\"\n", - " self.root_dir = os.path.join(root_dir, \"train\" if train else \"test\")\n", - " self.neg_files = [os.path.join(self.root_dir, \"neg\", f) for f in os.listdir(os.path.join(self.root_dir, \"neg\")) if f.endswith('.txt')]\n", - " self.pos_files = [os.path.join(self.root_dir, \"pos\", f) for f in os.listdir(os.path.join(self.root_dir, \"pos\")) if f.endswith('.txt')]\n", - " self.files = self.neg_files + self.pos_files\n", - " self.labels = [0] * len(self.neg_files) + [1] * len(self.pos_files)\n", - " self.pos_inx=len(self.pos_files)\n", - "\n", - " def __len__(self):\n", - " return len(self.files)\n", - "\n", - " def __getitem__(self, idx):\n", - " file_path = self.files[idx]\n", - " label = self.labels[idx]\n", - " with open(file_path, 'r', encoding='utf-8') as file:\n", - " content = file.read()\n", - " return label, content" - ] - }, - { - "cell_type": "markdown", - "id": "6bb31d42-b20e-413d-96f6-7d680eb83bb2", - "metadata": {}, - "source": [ - "The following code uses the `IMDBDataset` class previously defined to create iterators for the train and test data sets. In the latter part of the cell, you can return 20 examples from the train set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "01513044-f657-4203-b933-bad10ebeb4c8", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(0, 'It is important not to be insulted by lack of logic or common sense and those who have any \"gray matters\" will agree that this movie just doesn\\'t work.

The problems lay in the direction, cast selections and lack of depth in the character building. The word comedy was very hard thing to say when i expect to laugh when these words are used. Let\\'s look at the problems in direction/script.

Brother and sister both in their mid 30\\'s seem to be well adjusted. They meet a complete stranger at a park and Heather Graham character walks up to her and asks the most intimate questions that even half sane person would be running the other way or at least scream for a police officer. He then awkwardly walks over and makes some stupid statements and she falls for him. Then after ONE date were they all go out together he falls in love with her and decides to get married in Vegas in a week\\'s time???? Hello does anyone feel stupid yet? He goes out with thousands of women and he meets this one person who says about 10 words that WE see on the screen and he wants to marry her. Not only was there no chemistry it just doesn\\'t make sense. Sure it\\'s a romantic comedy and I want to believe it could, but the direction made it completely flat.

Now Heather falls head over heels with her too and when Heather Graham and Bridget Moynahan (very shallow character) kiss or more to the point it was sloppiest kiss ever that chemistry MIGHT be there. I found it unromantic and unfunny and while many say Heather cannot act i think the reality is Heather was clearly the wrong person for this role.

This was Sue Kramer debut as a director and to me it was just too much for her to chew. It would take a lot of craft to make this movie work and IMHO it could be done with better writers and casting and direction.')\n", - "(0, \"THE SCREAMING SKULL (1 outta 5 stars) This movie boasts some pretty cool opening credits (an offscreen narrator warning that movie patrons will be offered a free burial if they die of fright watching this movie, a scary shot of a skull emerging from a placid pool and the ubiquitous scary music) but, sadly, the movie is all downhill from there. A widowed man takes his new bride to his secluded mansion... admonishing his servants and friends that the new Mrs. has a very fragile disposition due to a tragedy in her past. Well, in no time at all she begins to see and hear mysterious things that no one else can. Her husband assures her that it's all merely in her mind and... well, you can probably see where this all is going. You will have figured out what's going on long before our hapless heroine... because you have probably seen the exact same plot in hundreds of other movies and TV shows (and done better, too). To add to the movie's myriad transgressions, most cuts of this movie (on numerous cheap DVD compilations) seem to be missing a few key scenes. You see the heroine slowly walking towards the window... she goes to open it... you know she is going to see something scary... and then... suddenly the scene cuts to her sobbing in her husband's arms. So what did she see??? I guess we'll never know.\")\n", - "(0, \"

Whether any indictment was intended must be taken into consideration. If in the year 2000 there were still rifts of feeling between Caucasian and Afro-Americans in Georgia, such as shown in this film, obviously there remains a somewhat backward mentality among a lot of people out there. It is rather hypocritical, to say the least, if everyone adores Halle Berry, Whoopie Goldberg, Beyoncé, Noemi Campbell, Denzel Washington, Will Smith, et. al., whilst out in the backs there persist manifest racial divides.

White grandmother suddenly gets black grand-daughter thrust upon her, only to meet up with black grandfather in a very white social backwater. The story is sweet, not lacking tragic overtones, and eminently predictable as in most of these kinds of TV films, though the final scene has you guessing............ will he? won't he.......?

Gena Rowlands in her typical style offers a sincere rendering, and Louis Gossett is a good match for her; the little Penny Bae fortunately does not steal the show.

A `nice' way of relaxing after Sunday lunch without having to force your mind too much, though you might just find yourself having a little siesta in the middle of it.\")\n", - "(0, 'First, it takes a full half hour to get Hackman out of jail and to start doing the job. What a waste of time, we all know Hackman is getting out to do some job for his masters, why waste almost a third of the movie on these sequences. Then Hackman stays in a hotel and the story arc again goes nowhere, simply proving to us that Hackman is under close watch and anything he says or does is know by the masters. Again, another 20 minutes. Then more wasted time showing the reunion with his wife. All of this should have taken 10-15 minutes at most simply as a set-up for the real action, intrigue and plot twists. By the time the real action gets going, I was so bored that I just wanted the movie to end. Hackman is great as usual, and the other actors as well, but this is a dud of the first magnitude.')\n", - "(0, \"Banned as a 'Video Nasty' in the UK, Unhinged has naturally gained quite a bit of notoriety. However, the most shocking thing I found about the film was its amateurishness in all departments. The bloodletting I could handle: the terrible acting, shoddy editing, awful direction, lousy script and abysmal soundtrack were much harder to take.

Three girls on their way to a music festival crash into a ravine during a storm. They are rescued by a friendly stranger who takes them to a nearby house. The owner of the house, a batty old lady, and her spinster daughter, welcome the girls in, allowing them to stay for a few days in order to recuperate. However, someone doesn't want the girls to leave\\x97ever! One by one they fall victim to an unseen assailant.

Taking a long time to get going and featuring some of the worst performances ever in a horror film (and that takes some doing), Unhinged is a truly awful film. The music is a total mess (it sounds like a three year old has been let loose on a synthesiser) and as such, it complements the movie perfectly. Only a couple of bloody scenes towards the end and a bit of gratuitous nudity save Unhinged from getting the lowest possible score.

If you are a horror completist (and unfortunately, I am), you will want to see this in order to tick it off the Video Nasty watch-list. But be warned\\x97it is really, really bad.\")\n", - "(0, \"When great director/actor combinations are talked about the team of J. Lee Thompson and Charles Bronson is not usually mentioned. Probably because the output of nine joint ventures between the two of them runs the gamut from the really good action entertainment to the mediocre. Unfortunately Kinjite: Forbidden Subjects falls in the latter.

That's sad because Kinjite could have been a whole lot better. But for the life of me I don't understand why it was necessary to make the father of the missing Japanese girl, a guy used to getting some cheap jollies because the romance in his marriage has run out. That might have been good for another film altogether, but it served no purpose here.

A straightforward cop drama with Charles Bronson as a vice cop who's seen a bit too much in his line of work and has a strong prejudice against orientals. That part could also have used a little explaining as well. But he's going to have to overcome it if he and patient partner Perry Lopez are going to locate a captured Japanese school girl.

Bronson's time in the vice squad have told him exactly where to look for the kidnapper. A stylish, murderous pimp played by Jaime Fernandez is the guy and he and Bronson have some history. In fact in the film's best scene, Bronson made him eat an expensive rolex watch and set his car on fire.

At one point Fernandez happens to spot Bronson and Lopez in an all night delicatessen and this being after his rolex snack, he sprays the place with an Uzi killing everyone, but Bronson and Lopez. I really think that little incident would have had more than a couple vice cops from the LAPD after Fernandez. But that's another terribly big hole in the plot.

Still there is a very rough justice in the end for Fernandez. I wish the whole film had been better though. This was the last film of the Bronson-Thompson team and J. Lee Thompson's last as a director. He should have gone out with something better.\")\n", - "(0, 'First off, let me say that I am a great believer in Fanpro stuff. I see it as a way to continue a good show long after it has been cancelled. Star Trek Voyages and Star Wars Revelations are examples of decent efforts. So I have a soft-spot for fanpro stuff that means I\\'ll overlook things that I would ordinarily slate badly.

So on to ST: HF. Well, first off the good things. Enthusiasm is a major part of making any show believable and, for the most part, the crew of the various ships all seem to be having a good time with their roles. Next, the effects aren\\'t bad for a home-brew effort, with nothing to make you really wince. The stories aren\\'t too bad either. Nothing particularly innovative, but solid enough stuff and at least there are ongoing story-arcs.

But it has a lot of faults.

First off, although they quite obviously HAVE to rip-off Star Trek footage, set backdrops, music and effects, I see no reason why they proceeded to rip off virtually every other sci-fi musical score ever made. Everything from Aliens to Starship Troopers rears it orchestral head at one point or another. Likewise, much of the footage is from other movies, dutifully CGI\\'d over to make it look different. The Grey warships, for instance, though disguised, are quite obviously Star Destroyers from Star Wars. And the station is also rather obviously Fleet Battle Station Ticonderoga from Starship Troopers. Likewise, sound effects from various Star Wars movies appear in space battles between fighters, as does animated over footage. In one scene in either first or second season, I think, you even see two TIE fighters fly past during a battle, which hardly does your suspension of disbelief any favours.

Acting varies from the reasonable to the hideously painful to watch. Everyone does improve as the seasons progress, though, but expect to grimace at the screen a lot, especially in the early seasons. They\\'ve also made some interesting acting choices. Let\\'s just say that the food replicators on this show seem permanently set to \"cake\" and leave it at that.

Make-up effects are generally quite effective on the whole. But they really ought to mercilessly club to death the person who decided to use cheap Ferengi and Cardassian masks for anything other than background use or \"passing\" shots. They are just beyond unrealistic. Every time I saw one of these (apart from trying not to laugh too much) I kept expecting the unfortunate soul wearing it to pull out a gun and announce that \"This is a stick-up!\" In one scene a \"Cardassian\" actually talks whilst wearing one of these. Not only do the lips not move, but the mask doesn\\'t even have an opening where the mouth should be. Someone needs to be slapped hard for that. Couldn\\'t they have taken a craft knife to it, for goodness\\' sake! There are also some well-done, but unintentionally funny make-up jobs, such as the Herman Munster look alike.

The writing, though coherent, is nothing new. Instead the script runs like a continuation of DS9, with the ships heading out from DS12 on various missions. The new enemy, \"The Grey\" aren\\'t very menacing and the plot line involving them is effectively a reworking of the Borg threads. i.e. Starfleet meet the Grey, the Grey are hugely powerful, Starfleet barely escape with their lives, then through technology they begin to find ways to combat the enemy etc etc. All done before with the Borg.

Another bone of contention is the dialogue. Star Trek writers have long had the ability to write \"insert technobabble here\" into a script. It usually means an exposition of the latest plan to combat the enemy using \"quantum phase discriminators\" or \"isolytic charges\" etc. In other words, nonsense that tells you that they are on the case and a resolution is at hand.

The words are just gibberish really. I\\'ve no problem with this, but where ST:HF makes a mess of it is where they include real-world comments into this concept.

Tactical advice such as \"We need to regroup\" sounds good, but not when uttered by trio of characters already standing in a group. Likewise when asked what the situation is, a tactical officer is heard to reply \"We count three battleships\". He actually needed to count them? C\\'mon! I expected the questioner to ask him \"Are you sure?\" or \"Can you double check\". But my all-time favourite comment is this:

Captain: \"Can we establish two-way communication?\"

Comms officer: \"No, we can only send and receive..\"

Well, duh!.....

Having said all the above, the show does improve as it goes along. Seasons 1 and 2 are pretty bad, 3 shows an improvement but 4 & 5 are where it starts to get noticeably better. Season 6 so far looks quite reasonable.

I do have a problem with their choice of media for the shows though. Quicktime sucks, quite frankly and the sooner they move to divx/avi format the better. Some of us like to actually take our downloaded shows and watch them on decent size screen and not peer at a tiny QT window on a computer monitor. Not only does Quicktime make this difficult, but the 320x180 resolution the shows are in does not scale at all well. In fact, it makes the shows pretty unwatchable, like they were a tenth-generation VHS tape copy. The least they could do was to include a hi-res downloadable option.

Anyway, the show has promise, and I\\'m even beginning to like some of the characters. But that\\'s 40 episodes on, so I\\'m not sure this says that much about character development at all.

But what can you say, it\\'s free....

PS: Out of 28 votes, 19 people rated this show as a 9 or 10. Hmmmm... were we watching the same show? Or are you 19 all three year olds?')\n", - "(0, 'I read in the papers that W.Snipes was broke so no wonder he would take parts in low budget projects like The Contractor.He is just the next action star to join a growing club:the penniless action stars of the 90s (Van Damme,Segal,Lundgren,Snipes). Here he stars the lead in a cheap action flick which was shot in Bulgaria( we are supposed to believe that the location is London, like only a complete moron would buy that)The story is the one of 1000 other movies: retired special forces good guy gets hired by the government again to do a wet job- after that government wants to get rid of him- good guy gets away after killing bad guys (was that a spoiler? guess not!) The star of the movie: the little girl (Eliza Bennett) outperforms everybody else of the cast!!!One star is for her plus one star for eye candy Lena Headey, makes 2 stars. Only for die hard Snipes fans!Everybody else:avoid!')\n", - "(0, \"Did HeidiJean really see this movie? A great Christmas movie? Not even close. Dull, bland and completely lacking in imagination and heart. I kept watching this movie wondering who the hell thought that Carly Pope could play the lead in this movie! The woman has no detectable personality and gives a completely lackluster performance. Baransky was great as usual and provided the only modicum of interesting the whole thing. Probably her involvement was the only reason this project was green lighted to begin with. Maybe I'm expecting too much for a Lifetime movie played 15 days from Christmas but I sat through this thing thinking that with a different director and a recasting JJ with an actress that at least could elicit sympathy this could have been quite a cute little movie.\")\n", - "(0, 'Band Camp was awful, The Naked Mile was a little better, and this third straight to DVD in the American Pie franchise seems the same quality as the predecessor. Basically Erik Stifler (John White) split from his girlfriend after losing his virginity, and now him and Mike \\'Cooze\\' Coozeman (Jake Siegel) are joining Erik\\'s cousin Dwight (Steve Talley) at college. With the promise of many parties, plenty of booze, and enough hot chicks at the Beta House, they only have fifty listed tasks to carry out to become official privileged members. But a threat comes into sight with the rivals, GEK (\"Geek\") House, led by power-hungry nerd (and sheep shagger) Edgar (Tyrone Savage) offering bigger and better than what Beta have. To settle it once and for all, Beta and Gek go into battle with the banned, for forty years, Greek Games to beat each other in, with the loser moving out. The last champion of the games, Noah Levenstein aka Jim\\'s Dad (the only regular Eugene Levy) runs the show, which sees the people unhooking bras, a gladiator duel floating on water, catching a greased pig, Russian Roulette in the mouth with cartridges of aged horse spunk, wife carrying and drinking a full keg of alcohol (with puking not disqualifying). It all comes to the sudden death, with a guy getting stripper lap dancing, and they have to resist cumming, Beta House win when Edgar cums with a girl dressed as a sheep on his lap. Also starring Flubber\\'s Christopher McDonald as Mr. Stifler, Meghan Heffern as Ashley, Dan Petronijevic as Bull, Nic Nac as Bobby, Christine Barger as Margie, Italia Ricci as Laura Johnson, Moshana Halbert as Sara Coleman, Sarah Power as Denise, Andreja Punkris as Stacy and Jordan Prentice as Rock. The nudity amount is very slightly increased, as is the grossness of the jokes, and I could guess it being rated one star out of five, but I like it. Adequate!')\n", - "(1, \"I saw this recently on a cable channel. The movie is great; it's one of the few musicals I have seen that doesn't shy away from the light and dark. It portrays some of the splendour of the age along with a lot of the squalor. Some of the set piece dance sequences so much is going on, I didn't know where to look next. One day I shall go and see this on the big screen, just so that I see what's happening. But what really lifts this to another level is Oliver Reed's performance as Bill Sykes. Not only is a thoroughly mean and menacing man but there is something else, some inner demons. He gave me the impression that if you pushed him into a corner, he was capable of anything. It was almost as if the Sykes character was on the edge of madness, just awaiting the trigger. I have seen the Robert Newton's Bill Sykes from the 1948 movie, and I thought he was 'just' a bad egg, but Oliver Reed's performance intimidated me in my own living room.\")\n", - "(1, '\"The Gingerbread Man is the first thriller I\\'ve ever done!\" \\x96 Robert Altman

In 1955 Charles Laughton directed \"The Night of the Hunter\", a spooky slice of Southern Gothic in which Robert Mitchum plays a scary serial killer. One of the film\\'s more famous sequences consists of two kids escaping from Mitchum on a rowboat, the kids frantically paddling whilst Mitchum wades after them like a monster.

Seven years later Mitchum played an equally spooky killer in \"Cape Fear\", another film set in the American South. That film featured a local attorney trying to protect his family and likewise ended with Mitchum terrorising folks on a boat. In 1991 Martin Scorsese, trying to branch out and tackle something more mainstream, remade \"Cape Fear\", boat scene and all.

Now we have Robert Altman\\'s \"The Gingerbread Man\", another slice of small town Southern Gothic. Altman says he consulted \"The Night of the Hunter\" for inspiration and tackled such a mainstream film purely because he wanted to \"spread his wings and try a popcorn picture\", but what he\\'s secretly attempting to do here is deconstruct the canonical films of the Southern Gothic genre.

So instead of a showdown on small boat, we get a showdown on a giant ship. Instead of two kids being kidnapped, we get two kids being safely returned to the police. Instead of money being hidden, we have money being readily given via a last will and testament. Instead of the righteous attorney of the 1961 film and the deplorable attorney of the 1991 remake, we get a rather three-dimensional lawyer in Kenneth Branagh. Instead of the monster chasing the family we get the hero chasing the bad guys. Instead of the monster breaking into the family\\'s house boat, we have the hero hunting the monster on board the monster\\'s \"house ship\". Similarly, instead of a murderous serial killer we get an innocent weirdo played by Robert Duvall. . .etc etc etc.

Altman goes on and on, reversing everything just a little slightly, pulling at the edges and doing his own thing. His touch is most apparent during the film\\'s first half-hour, the film existing in an uneasy space between conventional plot-driven movie storytelling and Altman\\'s fondness for overlapping dialogue, casual narratives, prowling camera movement and the way that characters aren\\'t so much introduced as they are simply part of what\\'s going on.

Still, despite Altman\\'s best intentions, the film never rises above mediocrity. Altman\\'s too bound to the conventions of the \"thriller format\" to do much damage, his style is too lethargic to generate tension and the film is simply not radical enough to counterpoint other canonical films in the genre. \"Gingerbread Man\" is thus too mainstream to work as a more pure Altman film and too Altman to work as a mainstream thriller.

The film\\'s not a complete waste, though. Robert Downey Junior, Kenneth Branagh and the usually intolerable Daryl Hannah, all turn in juicy performances. The film also has a nice atmosphere, set against a approaching hurricane, and the final act contains some interesting twists and turns. While it\\'s not the complete disaster that Scorsese\\'s \"Cape Fear\" was, the film still never amounts to anything special.

7/10 \\x96 In the late 90s Altman made 3 successive films set in the American South: \"Kansas City\", \"Gingerbread Man\" and \"Cookie\\'s Fortune\". Unlike \"Gingerbread Man\", both \"Kansas City\" and \"Cookie\\'s Fortune\" tackle the genre on the broader, more looser canvases that Altman was most comfortable with.

\"Kansas City\" is the more important of these two films, its hierarchies of class, politics and crime, and its desire to break radically away from typical gangster genre frameworks, would prove influential on all serious 21st century film crime writers (see, for example, \"The Wire\"). That said, \"Cookie\\'s Fortune\", while a much slighter tale, is perhaps the better picture.

Note: Altman claims that this is his first thriller, but he directed \"Images\", an art house thriller, in 1972.

Worth one viewing.')\n", - "(1, '\"Read My Lips (Sur mes lèvres)\" (which probably has different idiomatic resonance in its French title) is a nifty, twisty contemporary tale of office politics that unexpectedly becomes a crime caper as the unusually matched characters slide up and down an ethical and sensual slippery slope.

The two leads are magnetic, Emmanuelle Devos (who I\\'ve never seen before despite her lengthy resume in French movies) and an even more disheveled than usual Vincent Cassel (who has brought a sexy and/or threatening look and voice to some US movies).

The first half of the movie is on her turf in a competitive real estate office and he\\'s the neophyte. The second half is on his turf as an ex-con and her wrenching adaptation to that milieu.

Writer/director Jacques Audiard very cleverly uses the woman\\'s isolating hearing disability as an entrée for us into her perceptions, turning the sound up and down for us to hear as she does (so it\\'s even more annoying than usual when audience members talk), using visuals as sensory reactors as well.

None of the characters act as anticipated (she is not like that pliable victim from \"In the Company of Men,\" not in individual interactions, not in scenes, and not in the overall arc of the unpredictable story line (well, until the last shot, but heck the audience was waiting for that fulfillment) as we move from a hectic modern office, to a hectic disco to romantic and criminal stake-outs.

There is a side story that\\'s thematically redundant and unnecessary, but that just gives us a few minutes to catch our breaths.

This is one of my favorites of the year!

(originally written 7/28/2002)')\n", - "(1, \"This was a pretty decent movie. This movie is good to just sit down and watch and be entertained. Just a typical Hollywood film. This movie will never win an Oscar or anything and definitely doesn't deserve one, but I thought it was pretty good. It's kind of like the show 24 but set into movie format. If you like the whole we've got to stop the terrorist from killing the president kind of movie then you will enjoy this flick. I personally think that storyline has been done WAY too much, but The Sentinel does add a little twist with the mole in the Secret Service. All in all, this movie won't leave your jaw to the floor or change your life, but who says every single movie has to be like that to be good?\")\n", - "(1, 'Actually they could not have chosen a better diversified actor to portray Little Richard than Leon. He captures Little Richard to a most believable essence. The outfits where wonderful and any person watching this movie will definitely keep a smile on their face through the entire movie. Although the movie is a little long, it keeps your attention with the personality and outfits of Little Richard in mind. The ending should have taken a direction of moving Little Richard more into the present where you could see him as he has aged into this new millennium. He will always be the King of Rock-N-Roll as far as I am concerned regardless of what the other media says.')\n", - "(1, \"If you are a fan of Altman's large ensemble casts, as evidenced in major films like M.A.S.H., Nashville, Gosford Park, and lesser seen films like A Wedding, then you will no doubt be entertained by HealtH. Centered around a Health Convention where two women are running for President, HealtH contains many of Altman's latter 70s regulars like Paul Dooley (who helped write the film), Carol Burnett, and Henry Gibson, while also including top star Altman newcomers like Lauren Bacall, James Garner, and Glenda Jackson. Like a lot of Altman ensemble films there are numerous subplots in this film, but it is not nearly as overwhelming as films like Nashville or A Wedding, rather it has a more centered feel, perhaps like M.A.S.H. or Gosford Park. The whole thing is an obvious satire on the Health movement, filled with over-top, outlandish, contradictive characters, with guest stars like Dick Cavett providing a wry commentary on the whole thing. Underlining the whole election process is Altman's characteristic pessimism about politics and public appeal but what is most appealing about this film is the sheer fun most people seem to be having. This would be one of Altman's last films like this for a while!\")\n", - "(1, 'I saw the movie with two grown children. Although it was not as clever as Shrek, I thought it was rather good. In a movie theatre surrounded by children who were on spring break, there was not a sound so I know the children all liked it. There parents also seemed engaged. The death and apparent death of characters brought about the appropriate gasps and comments. Hopefully people realize this movie was made for kids. As such, it was successful although I liked it too. Personally I liked the Scrat!!')\n", - "(1, \"This show has been my escape from reality for the past ten years. I will sadly miss it. Although Atlantis has filled the hole a small bit.

The last ever episode of SG1(on television anyway)was beautifully done. Robert wrote something that felt close to reality. As though he was trying to explain what it was like on the set of the show. (Everyone working closely together for such a long time there are bound to up's and downs. But over the years they've turned into a family). I thought this was a wonderful way to end despite anyone else's criticisms.

SG1 was something special and time and time again it took me across thresholds of disbelief and amazement. The wonderful characters, stories, directors, writers. From episode one I was hooked. The blend of action, science, drama and especially comedy worked so well that made me keep wanting more.

There are no real words in which to completely express what this show meant to me. I can only thank those who kept the show so fresh and entertaining for so many years. It has inspired me to do many things that I thought was impossible.

I look forward to the movies next year and I really hope there will be a number of them. I never want the show to die.

Stargate SG1 - 1997 - 2007?\")\n", - "(1, 'For a long time it seemed like all the good Canadian actors had headed south of the border and (I guessed) all the second rank ones filled the top slots and that left the dregs for the sex comedies.

This film was a real surprise: despite the outlandish plots that are typical of farces, the actors seemed to be trying to put something into their characters and what we, the viewer, got back was almost true suspension of belief. When the extras from the music video attacked the evicting police, you almost believed it was possible.

If you are a fan of some of the better sex farces (Canadian or not) you should definitely seek this one out. And the big surprise, this sex farce is also loaded with some very good nudity.')\n", - "(1, 'Thank God this wasn\\'t based on a true story, because what a story it is. Populated by despicable characters whose depravity knows no bounds, Before The Devil is a mesmerizing, jaw-dropping excursion into perversion which would be laughable (and sometimes is, even with - or perhaps because of - the sickeningly tragic undercurrent of human dysfunction throughout) if it weren\\'t carried out with such magnificent, overwhelming conviction by its stars. The excellent script by Kelly Masterson and superb direction by none other than Sidney Lumet doesn\\'t hurt either.

The main dysfunction here is of a family nature, with the two majorly screwed up brothers (brilliant portrayals from Philip Seymour Hoffman and Ethan Hawke) deciding to rob their own parents\\' jewelry store, an attempt that goes pathetically awry.

The story is told with time-shifts (which are noted on screen, such as: \"Charlie: Two Days Before The Robbery\", so no one should be confused); some people have said they didn\\'t like this device but I thought it worked perfectly, adding to the skeweredness of the whole affair, considering that the two brothers in question are hardly playing with full decks - between them you couldn\\'t make a decent poker hand to save your life. Throw in these cheesy extra tidbits: one of the brothers is a drug addict, married to Gina (Marisa Tomei, also excellent), who is having an affair with the other brother, toss in some monumental sibling rivalry, along with the fact that said drug addict brother hates his father (a wrenching performance from Albert Finney), who has apparently caused him serious past pain, and you\\'ve got a Shakespearean/Greek tragedy on your hands. Proceed with caution.')\n" - ] - } - ], - "source": [ - "root_dir = tempdir.name + '/' + 'imdb_dataset'\n", - "train_iter = IMDBDataset(root_dir=root_dir, train=True) # For training data\n", - "test_iter = IMDBDataset(root_dir=root_dir, train=False) # For test data\n", - "\n", - "start=train_iter.pos_inx\n", - "for i in range(-10,10):\n", - " print(train_iter[start+i])" - ] - }, - { - "cell_type": "markdown", - "id": "1a6c647b-b8ab-49df-8434-becaa0dea775", - "metadata": {}, - "source": [ - "The following code defines the mapping of numeric labels to positive and negative reviews.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "3fca543a-ffc0-4079-bc30-7e3e05576623", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'positive review'" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "imdb_label = {0: \" negative review\", 1: \"positive review\"}\n", - "imdb_label[1]" - ] - }, - { - "cell_type": "markdown", - "id": "ac4da5eb-a679-45d2-90bd-cfde475e0f8e", - "metadata": {}, - "source": [ - "The following code checks to ensure that there are exactly two classes in the train data set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "e78c80a0-0dee-4236-a382-dfe9f75a8fea", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "2" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "num_class = len(set([label for (label, text) in train_iter]))\n", - "num_class" - ] - }, - { - "cell_type": "markdown", - "id": "9c35ca39-77a1-40d9-9f16-fddb3259fd64", - "metadata": {}, - "source": [ - "The following code loads a basic English tokenizer and defines a function called ```yield_tokens``` that uses the tokenizer to break down text data yielded by an iterator into tokens.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "6a41ea50-7e8b-423b-9de8-36a88e97e96b", - "metadata": {}, - "outputs": [], - "source": [ - "tokenizer = get_tokenizer(\"basic_english\")\n", - "\n", - "def yield_tokens(data_iter):\n", - " \"\"\"Yield tokens for each data sample.\"\"\"\n", - " for _, text in data_iter:\n", - " yield tokenizer(text)" - ] - }, - { - "cell_type": "markdown", - "id": "e1853ae6-c596-4fac-a2f4-185c0a354510", - "metadata": {}, - "source": [ - " The following code loads a pretrained word embedding model called GloVe into a variable called `glove_embedding`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "0cbe5e7e-13d0-4d64-8c0c-86b74954c9b2", - "metadata": {}, - "outputs": [], - "source": [ - "# Note that GloVe embeddings are typically downloaded using:\n", - "#glove_embedding = GloVe(name=\"6B\", dim=100)\n", - "# However, the GloVe server is frequently down. The code below offers a workaround\n", - "\n", - "\n", - "class GloVe_override(Vectors):\n", - " url = {\n", - " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", - " }\n", - "\n", - " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", - " url = self.url[name]\n", - " name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", - " #name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", - " super(GloVe_override, self).__init__(name, url=url, **kwargs)\n", - "\n", - "class GloVe_override2(Vectors):\n", - " url = {\n", - " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", - " }\n", - "\n", - " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", - " url = self.url[name]\n", - " #name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", - " name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", - " super(GloVe_override2, self).__init__(name, url=url, **kwargs)\n", - "\n", - "try:\n", - " glove_embedding = GloVe_override(name=\"6B\", dim=100)\n", - "except:\n", - " try:\n", - " glove_embedding = GloVe_override2(name=\"6B\", dim=100)\n", - " except:\n", - " glove_embedding = GloVe(name=\"6B\", dim=100)" - ] - }, - { - "cell_type": "markdown", - "id": "e81eda58-67d1-4414-a91a-c238a1434729", - "metadata": {}, - "source": [ - "The following code builds a vocabulary object from a pretrained GloVe word embedding model and sets the default index to the token.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "5a3dbfda-7d7f-4256-937c-af441df1ee1b", - "metadata": {}, - "outputs": [], - "source": [ - "from torchtext.vocab import GloVe,vocab\n", - "# Build vocab from glove_vectors\n", - "vocab = vocab(glove_embedding .stoi, 0,specials=('', ''))\n", - "vocab.set_default_index(vocab[\"\"])" - ] - }, - { - "cell_type": "markdown", - "id": "e1fc78f6-0d04-4b35-afdd-a6193c66166e", - "metadata": {}, - "source": [ - "Let's count the number of words in the vocab:\n" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "bc7de385-2867-4836-a6dc-f87a9f0c38f5", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "400002" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "vocab_size=len(vocab)\n", - "vocab_size" - ] - }, - { - "cell_type": "markdown", - "id": "aa4eca48-cfb4-4fb4-8ca3-180319852e0d", - "metadata": {}, - "source": [ - "Let's test the ```vocab``` function:\n" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "59689af6-a6ed-49eb-a12e-51303c1eb9da", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[20]" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "vocab(['he'])" - ] - }, - { - "cell_type": "markdown", - "id": "6ca4130b-f548-4b92-a68f-b2d83b6b19a5", - "metadata": {}, - "source": [ - "### Data set splits\n", - "\n", - "The following converts the data set into map-style data sets and then performs a random split to create separate training and validation data sets. The training data set will contain 95% of the samples in the original training set, while the validation data set will contain the remaining 5%. These data sets can be used for training and evaluating a machine learning model for text classification on the IMDB data set. The final performance of the model will be evaluated on the hold-out test set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "be0403dd-b319-406e-96ea-d215c0a42f6c", - "metadata": {}, - "outputs": [], - "source": [ - "# Convert the training and testing iterators to map-style datasets.\n", - "train_dataset = to_map_style_dataset(train_iter)\n", - "test_dataset = to_map_style_dataset(test_iter)\n", - "\n", - "# Determine the number of samples to be used for training and validation (5% for validation).\n", - "num_train = int(len(train_dataset) * 0.95)\n", - "\n", - "# Randomly split the training dataset into training and validation datasets using `random_split`.\n", - "# The training dataset will contain 95% of the samples, and the validation dataset will contain the remaining 5%.\n", - "split_train_, split_valid_ = random_split(train_dataset, [num_train, len(train_dataset) - num_train])" - ] - }, - { - "cell_type": "markdown", - "id": "82b6143c-d493-49b1-a255-b5aae8f9280a", - "metadata": {}, - "source": [ - "Be aware that the Skills Network currently does not offer GPU access to learners. As a result, training on the full data set could be time-consuming. To address this, you further reduce the size of the training set. This approach helps you mimic the training process as if a GPU were available. However, if you want to train using the full IMDB data set, you must either comment out or remove the two lines in the following code block.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "0b86c021-06f7-463d-95d7-c83f45de8605", - "metadata": {}, - "outputs": [], - "source": [ - "num_train = int(len(train_dataset) * 0.05)\n", - "split_train_, _ = random_split(split_train_, [num_train, len(split_train_) - num_train])" - ] - }, - { - "cell_type": "markdown", - "id": "e2e0359b-b92d-4aae-9916-aaadbd206ddb", - "metadata": {}, - "source": [ - "The following code checks to see if a CUDA-compatible GPU is available in the system using PyTorch, a popular deep learning framework. If a GPU is available, it assigns the device variable to \"cuda\" (which stands for CUDA, the parallel computing platform and application programming interface model developed by NVIDIA). If a GPU is not available, it assigns the device variable to \"cpu\" (which means the code will run on the CPU instead).\n" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "id": "3e37cbbd-1c52-4b02-b26c-c7f4b694e944", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "device(type='cuda')" - ] - }, - "execution_count": 21, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "device" - ] - }, - { - "cell_type": "markdown", - "id": "9c0631e5-5eb9-4e91-a84f-315381189cb9", - "metadata": {}, - "source": [ - "### Data loader\n", - "\n", - "The following code prepares the text processing pipeline with the tokenizer and vocabulary. The text pipeline is used to process the raw data strings from the data set iterators.\n", - "\n", - "The function **```text_pipeline```** first tokenizes the input text, then **```vocab```** is applied to get the token indices.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "6b1807bd-4548-435d-b763-cc1641c95c45", - "metadata": {}, - "outputs": [], - "source": [ - "def text_pipeline(x):\n", - " return vocab(tokenizer(x))" - ] - }, - { - "cell_type": "markdown", - "id": "58976c04-a071-4f9f-8e55-b0207f39a7d1", - "metadata": {}, - "source": [ - "In PyTorch, the **`collate_fn`** function is used in conjunction with data loaders to customize the way batches are created from individual samples. The provided code defines a `collate_batch` function in PyTorch, which is used with data loaders to customize batch creation from individual samples. It processes a batch of data, including labels and text sequences. It applies the `text_pipeline` function to preprocess the text. The processed data is then converted into PyTorch tensors and returned as a tuple containing the label tensor, text tensor, and offsets tensor representing the starting positions of each text sequence in the combined tensor. The function also ensures that the returned tensors are moved to the specified device (for example, GPU) for efficient computation.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "442be80e-ae39-4001-98f7-0a0c5266bb65", - "metadata": {}, - "outputs": [], - "source": [ - "from torch.nn.utils.rnn import pad_sequence\n", - "\n", - "def collate_batch(batch):\n", - " label_list, text_list = [], []\n", - " for _label, _text in batch:\n", - "\n", - " label_list.append(_label)\n", - " text_list.append(torch.tensor(text_pipeline(_text), dtype=torch.int64))\n", - "\n", - " label_list = torch.tensor(label_list, dtype=torch.int64)\n", - " text_list = pad_sequence(text_list, batch_first=True)\n", - "\n", - " return label_list.to(device), text_list.to(device)" - ] - }, - { - "cell_type": "markdown", - "id": "fb6f5466-b4b7-4772-9f09-9836249d4279", - "metadata": {}, - "source": [ - "You can convert the data set objects to data loaders by applying the `collate` function.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "95d54253-4781-4607-9f60-f4727117a520", - "metadata": {}, - "outputs": [], - "source": [ - "BATCH_SIZE = 32\n", - "\n", - "train_dataloader = DataLoader(\n", - " split_train_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", - ")\n", - "valid_dataloader = DataLoader(\n", - " split_valid_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", - ")\n", - "test_dataloader = DataLoader(\n", - " test_dataset, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "3cc82919-3aa8-4f02-9be6-696a71335672", - "metadata": {}, - "source": [ - "Let's check to see what these data loaders generate.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "id": "0e6a5049-2b5e-4661-ab9a-30b6072da1bb", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "(tensor([0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0,\n", - " 1, 0, 1, 1, 0, 1, 0, 0], device='cuda:0'),\n", - " tensor([[ 0, 3542, 0, ..., 0, 0, 0],\n", - " [ 40, 663, 838, ..., 0, 0, 0],\n", - " [ 39, 16, 2, ..., 0, 0, 0],\n", - " ...,\n", - " [16307, 0, 59, ..., 0, 0, 0],\n", - " [ 9, 12389, 1608, ..., 0, 0, 0],\n", - " [ 193, 44724, 144, ..., 0, 0, 0]], device='cuda:0'))" - ] - }, - "execution_count": 22, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "label,seqence=next(iter(valid_dataloader))\n", - "label,seqence" - ] - }, - { - "cell_type": "markdown", - "id": "1f0d739f-47a7-44e6-b61d-6ce1eecda895", - "metadata": {}, - "source": [ - "### Neural network\n", - "\n", - "This code defines a class called Net that represents a text classifier based on a PyTorch TransformerEncoder.\n", - "The constructor takes the following arguments:\n", - "\n", - "- `num_class`: The number of classes to classify\n", - "- `vocab_size`: The size of the vocabulary\n", - "- `freeze`: Whether to freeze the embedding layer\n", - "- `nhead`: The number of heads in the transformer encoder\n", - "- `dim_feedforward`: The dimension of the feedforward layer in the transformer encoder\n", - "- `num_layers`: The number of transformer encoder layers\n", - "- `dropout`: The dropout rate\n", - "- `activation`: The activation function to use in the transformer encoder\n", - "- `classifier_dropout`: The dropout rate for the classifier\n", - "\n", - "**Attributes:**\n", - "\n", - "- `emb`: An embedding layer that maps each word in the vocabulary to a dense vector representation\n", - "- `pos_encoder`: A positional encoding layer that adds positional information to the word vectors\n", - "- `transformer_encoder`: A transformer encoder layer that processes the sequence of word vectors and extracts high-level features\n", - "- `classifier`: A linear layer that maps the output of the transformer encoder to the desired number of classes\n", - "\n", - "---\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "id": "7f290742-5806-47d8-8533-f3d92709eba9", - "metadata": {}, - "outputs": [], - "source": [ - "class Net(nn.Module):\n", - " \"\"\"\n", - " Text classifier based on a pytorch TransformerEncoder.\n", - " \"\"\"\n", - " def __init__(\n", - "\n", - " self,\n", - " num_class,vocab_size,\n", - " freeze=True,\n", - " nhead=2,\n", - " dim_feedforward=128,\n", - " num_layers=2,\n", - " dropout=0.1,\n", - " activation=\"relu\",\n", - " classifier_dropout=0.1):\n", - "\n", - " super().__init__()\n", - "\n", - " #self.emb = embedding=nn.Embedding.from_pretrained(glove_embedding.vectors,freeze=freeze)\n", - " self.emb = nn.Embedding.from_pretrained(glove_embedding.vectors,freeze=freeze)\n", - " embedding_dim = self.emb.embedding_dim\n", - "\n", - "\n", - " self.pos_encoder = PositionalEncoding(\n", - " d_model=embedding_dim,\n", - " dropout=dropout,\n", - " vocab_size=vocab_size,\n", - " )\n", - "\n", - " encoder_layer = nn.TransformerEncoderLayer(\n", - " d_model=embedding_dim,\n", - " nhead=nhead,\n", - " dim_feedforward=dim_feedforward,\n", - " dropout=dropout,\n", - " )\n", - " self.transformer_encoder = nn.TransformerEncoder(\n", - " encoder_layer,\n", - " num_layers=num_layers,\n", - " )\n", - " self.classifier = nn.Linear(embedding_dim, num_class)\n", - " self.d_model = embedding_dim\n", - "\n", - " def forward(self, x):\n", - " x = self.emb(x) * math.sqrt(self.d_model)\n", - " x = self.pos_encoder(x)\n", - " x = self.transformer_encoder(x)\n", - " x = x.mean(dim=1)\n", - " x = self.classifier(x)\n", - "\n", - " return x" - ] - }, - { - "cell_type": "markdown", - "id": "68664b81-817c-4633-bc71-9b32f95ee7f3", - "metadata": {}, - "source": [ - "The model can then be trained on labeled data from the IMDB data set with two classes.\n", - "Let's create the model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 24, - "id": "545ebebe-2206-47ce-81ab-9f168a80e4bd", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Net(\n", - " (emb): Embedding(400000, 100)\n", - " (pos_encoder): PositionalEncoding(\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " )\n", - " (transformer_encoder): TransformerEncoder(\n", - " (layers): ModuleList(\n", - " (0-1): 2 x TransformerEncoderLayer(\n", - " (self_attn): MultiheadAttention(\n", - " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", - " )\n", - " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", - " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (dropout1): Dropout(p=0.1, inplace=False)\n", - " (dropout2): Dropout(p=0.1, inplace=False)\n", - " )\n", - " )\n", - " )\n", - " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", - ")" - ] - }, - "execution_count": 24, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "model = Net(num_class=2,vocab_size=vocab_size).to(device)\n", - "model" - ] - }, - { - "cell_type": "markdown", - "id": "18ece646-3872-41e1-bedb-be1c7d8a37cc", - "metadata": {}, - "source": [ - "The following **`predict`** function takes in a text, a text pipeline, and a model as inputs. It uses a pretrained model passed as a parameter to predict the label of the text for text classification on the IMDB data set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "id": "4cde3566-d2df-40b8-8fd1-944018d4b0ab", - "metadata": {}, - "outputs": [], - "source": [ - "def predict(text, text_pipeline, model):\n", - " with torch.no_grad():\n", - " text = torch.unsqueeze(torch.tensor(text_pipeline(text)),0).to(device)\n", - " model.to(device)\n", - " output = model(text)\n", - " return imdb_label[output.argmax(1).item()]" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "id": "c7388103-5814-402d-a7de-ede861dc0927", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "' negative review'" - ] - }, - "execution_count": 26, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "predict(\"I like sports and stuff\", text_pipeline, model)" - ] - }, - { - "cell_type": "markdown", - "id": "40449174-d493-4521-8417-7950dafb0072", - "metadata": {}, - "source": [ - "You can create a function to evaluate the model's accuracy on a data set. Here, you define two nearly identical evaluation functions, one that provides a `tqdm` progress bar, and one that does not.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "id": "1171efb1-cf5c-4a25-a0e0-91feb88e8ab7", - "metadata": {}, - "outputs": [], - "source": [ - "def evaluate(dataloader, model_eval):\n", - " model_eval.eval()\n", - " total_acc, total_count= 0, 0\n", - "\n", - " with torch.no_grad():\n", - " for label, text in tqdm(dataloader):\n", - " label, text = label.to(device), text.to(device)\n", - " output = model_eval(text)\n", - " predicted = torch.max(output.data, 1)[1]\n", - " total_acc += (predicted == label).sum().item()\n", - " total_count += label.size(0)\n", - " return total_acc / total_count" - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "id": "571400ae-320b-4d1a-a103-91592812d1c1", - "metadata": {}, - "outputs": [], - "source": [ - "def evaluate_no_tqdm(dataloader, model_eval):\n", - " model_eval.eval()\n", - " total_acc, total_count= 0, 0\n", - "\n", - " with torch.no_grad():\n", - " for label, text in dataloader:\n", - " label, text = label.to(device), text.to(device)\n", - " output = model_eval(text)\n", - " predicted = torch.max(output.data, 1)[1]\n", - " total_acc += (predicted == label).sum().item()\n", - " total_count += label.size(0)\n", - " return total_acc / total_count" - ] - }, - { - "cell_type": "markdown", - "id": "0092f33b-ac39-41d2-8525-443d2759b55e", - "metadata": {}, - "source": [ - "The following code evaluates the performance of your model. Note that this can take approximately 4 minutes on a CPU. **For efficiency, let's not run this cell now, but trust us that the performance of the untrained model is no better than average. If you wish to confirm yourself of this fact, you are free to uncomment this cell**:\n" - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "id": "6b48b713-72a9-4267-b32e-dfab588c659c", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|█████████████████████████████████████████| 782/782 [00:09<00:00, 82.88it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.5" - ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader, model)" - ] - }, - { - "cell_type": "markdown", - "id": "b855a1d8-5e52-4fa2-a507-d7383b2ea73d", - "metadata": {}, - "source": [ - "Note that the current performance of the model is no better than average. This outcome is expected, considering that the model has not undergone any training yet.\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "8a28c3ce-3828-4b70-8442-ae9b7757805c", - "metadata": {}, - "source": [ - "# Training\n", - "\n", - "The following coe defines the training function used to train your model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "id": "376d4c30-7f50-4c97-9fa2-667f3c1a47eb", - "metadata": {}, - "outputs": [], - "source": [ - "def train_model(model, optimizer, criterion, train_dataloader, valid_dataloader, epochs=1000, save_dir=\"\", file_name=None):\n", - " cum_loss_list = []\n", - " acc_epoch = []\n", - " acc_old = 0\n", - " model_path = os.path.join(save_dir, file_name)\n", - " acc_dir = os.path.join(save_dir, os.path.splitext(file_name)[0] + \"_acc\")\n", - " loss_dir = os.path.join(save_dir, os.path.splitext(file_name)[0] + \"_loss\")\n", - " time_start = time.time()\n", - "\n", - " for epoch in tqdm(range(1, epochs + 1)):\n", - " model.train()\n", - " #print(model)\n", - " #for parm in model.parameters():\n", - " # print(parm.requires_grad)\n", - " \n", - " cum_loss = 0\n", - " for idx, (label, text) in enumerate(train_dataloader):\n", - " optimizer.zero_grad()\n", - " label, text = label.to(device), text.to(device)\n", - "\n", - " predicted_label = model(text)\n", - " loss = criterion(predicted_label, label)\n", - " loss.backward()\n", - " #print(loss)\n", - " torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)\n", - " optimizer.step()\n", - " cum_loss += loss.item()\n", - " print(f\"Epoch {epoch}/{epochs} - Loss: {cum_loss}\")\n", - "\n", - " cum_loss_list.append(cum_loss)\n", - " accu_val = evaluate_no_tqdm(valid_dataloader,model)\n", - " acc_epoch.append(accu_val)\n", - "\n", - " if model_path and accu_val > acc_old:\n", - " print(accu_val)\n", - " acc_old = accu_val\n", - " if save_dir is not None:\n", - " #pass\n", - " print(\"save model epoch\",epoch)\n", - " torch.save(model.state_dict(), model_path)\n", - " save_list_to_file(lst=acc_epoch, filename=acc_dir)\n", - " save_list_to_file(lst=cum_loss_list, filename=loss_dir)\n", - "\n", - " time_end = time.time()\n", - " print(f\"Training time: {time_end - time_start}\")" - ] - }, - { - "cell_type": "markdown", - "id": "353d2098-4d5d-4a06-aa6a-c8866a6ecb0f", - "metadata": {}, - "source": [ - "### Train IMDB\n", - "\n", - "The following code sets the learning rate (LR) to 1, which determines the step size at which the optimizer updates the model's parameters during training. The CrossEntropyLoss criterion is used to calculate the loss between the model's predicted outputs and the ground truth labels. This loss function is commonly employed for multiclass classification tasks.\n", - "\n", - "The chosen optimizer is Stochastic Gradient Descent (SGD), which optimizes the model's parameters based on the computed gradients with respect to the loss function. The SGD optimizer uses the specified learning rate to control the size of the weight updates.\n", - "\n", - "Additionally, a learning rate scheduler is defined using StepLR. This scheduler adjusts the learning rate during training, reducing it by a factor (gamma) of 0.1 after every epoch (step) to improve convergence and fine-tune the model's performance. These components together form the essential setup for training a neural network using the specified learning rate, loss criterion, optimizer, and learning rate scheduler.\n", - "\n", - "For the sake of time efficiency, **the following lines are commented out and the model is not actually trained**. If you would like to get a glimpse of what training would look like, uncomment the following code block to train the model for 2 epochs. If you were to train this model in a real-world scenario, you would likely increase the number of epochs to a larger figure, such as 100 or more. Given the reduced training set defined earlier, it takes approximately 2 minutes to complete 2 epochs of training.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 32, - "id": "03ab8dc6-6dda-44fa-b86c-1ad67e651463", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - " 0%| | 0/2 [00:00" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/sybqacL5p1qeEO8d4xRZNg/model-IMDB%20dataset%20small2-acc')\n", - "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/eOt6woGoaOB565T0RLH5WA/model-IMDB%20dataset%20small2-loss')\n", - "acc_epoch = pickle.load(acc_urlopened)\n", - "cum_loss_list = pickle.load(loss_urlopened)\n", - "plot(cum_loss_list,acc_epoch)" - ] - }, - { - "cell_type": "markdown", - "id": "37fe982b-b1a7-4d31-a5fc-21481d97fa4e", - "metadata": {}, - "source": [ - "The following code loads your pretrained model and evaluates its performance on the test set. **For efficiency, let's not run the evaluation because it can take approximately 4 minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, you are free to uncomment the last line in the following code block.**\n" - ] - }, - { - "cell_type": "code", - "execution_count": 39, - "id": "ca827b34-bd27-413f-ba76-8fde3d73c570", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 39, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/q66IH6a7lglkZ4haM6hB1w/model-IMDB%20dataset%20small2.pth')\n", - "model_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", - "model_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "code", - "execution_count": 40, - "id": "f7421582-5341-4554-bb43-93eaeec31de6", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|█████████████████████████████████████████| 782/782 [00:10<00:00, 71.81it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.83208" - ] - }, - "execution_count": 40, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader, model_)" - ] - }, - { - "cell_type": "markdown", - "id": "a91438ab-0a67-47b4-908c-2114cedb29e2", - "metadata": {}, - "source": [ - "As you can see, the pretrained model achieved an accuracy of approximately 83% on the test data.\n" - ] - }, - { - "cell_type": "markdown", - "id": "d8f01fc5-871b-4978-a2dd-724efa504014", - "metadata": {}, - "source": [ - "### Fine-tune a model pretrained on the AG News data set\n", - "\n", - "Rather than training a model on the IMDB data set as you did earlier, you can fine-tune a model that has been pretrained on the AG News data set, which is a collection of news articles. The goal of the AG News data set is to categorize news articles into one of four categories: Sports, Business, Sci/tech, or World. You’ll start training a model from scratch on the AG News data set. To save time, you can do this in just one cell. Also, for efficiency, ** comment out the training bit**. If you want to train the model for 2 epochs on a smaller data set to demonstrate what the training process would look like, uncomment the part that says `### Uncomment to Train ###` before running the cell. Training for 2 epochs on the reduced data set can take approximately 3 minutes.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 36, - "id": "4db8b115-13e1-4d42-99e0-fd09dc0527c6", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Net(\n", - " (emb): Embedding(400000, 100)\n", - " (pos_encoder): PositionalEncoding(\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " )\n", - " (transformer_encoder): TransformerEncoder(\n", - " (layers): ModuleList(\n", - " (0-1): 2 x TransformerEncoderLayer(\n", - " (self_attn): MultiheadAttention(\n", - " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", - " )\n", - " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", - " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (dropout1): Dropout(p=0.1, inplace=False)\n", - " (dropout2): Dropout(p=0.1, inplace=False)\n", - " )\n", - " )\n", - " )\n", - " (classifier): Linear(in_features=100, out_features=4, bias=True)\n", - ")" - ] - }, - "execution_count": 36, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "train_iter_ag_news = AG_NEWS(split=\"train\")\n", - "\n", - "num_class_ag_news = len(set([label for (label, text) in train_iter_ag_news ]))\n", - "num_class_ag_news\n", - "\n", - "# Split the dataset into training and testing iterators.\n", - "train_iter_ag_news, test_iter_ag_news = AG_NEWS()\n", - "\n", - "# Convert the training and testing iterators to map-style datasets.\n", - "train_dataset_ag_news = to_map_style_dataset(train_iter_ag_news)\n", - "test_dataset_ag_news = to_map_style_dataset(test_iter_ag_news)\n", - "\n", - "# Determine the number of samples to be used for training and validation (5% for validation).\n", - "num_train_ag_news = int(len(train_dataset_ag_news) * 0.95)\n", - "\n", - "# Randomly split the training dataset into training and validation datasets using `random_split`.\n", - "# The training dataset will contain 95% of the samples, and the validation dataset will contain the remaining 5%.\n", - "split_train_ag_news_, split_valid_ag_news_ = random_split(train_dataset_ag_news, [num_train_ag_news, len(train_dataset_ag_news) - num_train_ag_news])\n", - "\n", - "# Make the training set smaller to allow it to run fast as an example.\n", - "# IF YOU WANT TO TRAIN ON THE AG_NEWS DATASET, COMMENT OUT THE 2 LINEs BELOW.\n", - "# HOWEVER, NOTE THAT TRAINING WILL TAKE A LONG TIME\n", - "num_train_ag_news = int(len(train_dataset_ag_news) * 0.05)\n", - "split_train_ag_news_, _ = random_split(split_train_ag_news_, [num_train_ag_news, len(split_train_ag_news_) - num_train_ag_news])\n", - "\n", - "\n", - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "device\n", - "\n", - "def label_pipeline(x):\n", - " return int(x) - 1\n", - "\n", - "from torch.nn.utils.rnn import pad_sequence\n", - "\n", - "def collate_batch_ag_news(batch):\n", - " label_list, text_list = [], []\n", - " for _label, _text in batch:\n", - " label_list.append(label_pipeline(_label))\n", - " text_list.append(torch.tensor(text_pipeline(_text), dtype=torch.int64))\n", - "\n", - "\n", - " label_list = torch.tensor(label_list, dtype=torch.int64)\n", - " text_list = pad_sequence(text_list, batch_first=True)\n", - "\n", - "\n", - " return label_list.to(device), text_list.to(device)\n", - "\n", - "BATCH_SIZE = 32\n", - "\n", - "train_dataloader_ag_news = DataLoader(\n", - " split_train_ag_news_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", - ")\n", - "valid_dataloader_ag_news = DataLoader(\n", - " split_valid_ag_news_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", - ")\n", - "test_dataloader_ag_news = DataLoader(\n", - " test_dataset_ag_news, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", - ")\n", - "\n", - "\n", - "model_ag_news = Net(num_class=4,vocab_size=vocab_size).to(device)\n", - "model_ag_news.to(device)" - ] - }, - { - "cell_type": "code", - "execution_count": 37, - "id": "1ee6601d-69a0-4274-a237-38f981a6153b", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - " 0%| | 0/2 [00:00" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/bQk8mJu3Uct3I4JEsEtRnw/model-AG%20News%20small1-acc')\n", - "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/KNQkqJWWwY_XfbFBRFhZNA/model-AG%20News%20small1-loss')\n", - "acc_epoch = pickle.load(acc_urlopened)\n", - "cum_loss_list = pickle.load(loss_urlopened)\n", - "plot(cum_loss_list,acc_epoch)" - ] - }, - { - "cell_type": "markdown", - "id": "a17edde1-7268-468d-ad53-d1880848efc6", - "metadata": {}, - "source": [ - "The following code loads the pretrained model and evaluates its performance on the AG News test set. **For efficiency, let's not run the evaluation because it can take a few minutes. Instead, claim that the pretrained model works well on the AG News dataset. If you would like to confirm the result for yourself, feel free to uncomment the last line in the following code block.**\n" - ] - }, - { - "cell_type": "code", - "execution_count": 43, - "id": "e98831cf-bdb6-4e9c-9932-8ee7e6897450", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 43, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", - "model_ag_news_ = Net(vocab_size=vocab_size, num_class=4).to(device)\n", - "model_ag_news_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "code", - "execution_count": 44, - "id": "346d1f7a-2331-4010-a979-ae71efb00fcc", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|████████████████████████████████████████| 238/238 [00:00<00:00, 323.29it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.9046052631578947" - ] - }, - "execution_count": 44, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader_ag_news, model_ag_news_)" - ] - }, - { - "cell_type": "markdown", - "id": "49ecfc70-585f-4fd5-a2d3-3eb558705103", - "metadata": {}, - "source": [ - "As you can see, the pretrained model worked extremely well on the AG News data set. However, can this model be fine-tuned to perform well on the IMDB data set as well? Let's find out! You can begin by loading the pretrained AG News model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "id": "92336349-f915-4471-a3d2-64c90ea31171", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 46, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", - "model_fine1 = Net(vocab_size=vocab_size, num_class=4).to(device)\n", - "model_fine1.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))\n" - ] - }, - { - "cell_type": "markdown", - "id": "80e739c4-0b6b-44fa-8d5d-456c52576dd6", - "metadata": {}, - "source": [ - "The IMDB dataset is a binary classification task with only two classes (positive and negative reviews). Therefore, the output layer of the AG NEWS model should be adjusted to have just two output neurons to reflect the binary nature of the IMDB dataset. This adjustment is essential for the model to accurately learn and predict the sentiment of movie reviews in the IMDB dataset.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 47, - "id": "1190f1bf-a030-43b1-b146-5f27715ce6a4", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Original final layer: Linear(in_features=100, out_features=4, bias=True)\n", - "Input dimention final layer: 100\n" - ] - } - ], - "source": [ - "model_fine1.classifier\n", - "in_features = model_fine1.classifier.in_features\n", - "print(\"Original final layer:\", model_fine1.classifier)\n", - "print(\"Input dimention final layer:\", in_features)" - ] - }, - { - "cell_type": "markdown", - "id": "682f6a52-d8d8-4ed7-9162-1bf74dee5542", - "metadata": {}, - "source": [ - "You can change the final layer into a two-class problem.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 48, - "id": "46da3876-5b30-4174-bcf1-2702836f3d6d", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Net(\n", - " (emb): Embedding(400000, 100)\n", - " (pos_encoder): PositionalEncoding(\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " )\n", - " (transformer_encoder): TransformerEncoder(\n", - " (layers): ModuleList(\n", - " (0-1): 2 x TransformerEncoderLayer(\n", - " (self_attn): MultiheadAttention(\n", - " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", - " )\n", - " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", - " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (dropout1): Dropout(p=0.1, inplace=False)\n", - " (dropout2): Dropout(p=0.1, inplace=False)\n", - " )\n", - " )\n", - " )\n", - " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", - ")" - ] - }, - "execution_count": 48, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "model_fine1.classifier = nn.Linear(in_features, 2)\n", - "model_fine1.to(device)" - ] - }, - { - "cell_type": "markdown", - "id": "672f77ee-1134-4656-9e85-c158cb4a64ea", - "metadata": {}, - "source": [ - "The following code shows the layers that are frozen (`requires_grad == False`) and unfrozen (`requires_grad == True`) in the model. The unfrozen layers will have their weights updated during fine-tuning.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 49, - "id": "9afe1e34-b1e3-42a8-b67d-5e99e761dbab", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "emb.weight requires_grad: False\n", - "transformer_encoder.layers.0.self_attn.in_proj_weight requires_grad: True\n", - "transformer_encoder.layers.0.self_attn.in_proj_bias requires_grad: True\n", - "transformer_encoder.layers.0.self_attn.out_proj.weight requires_grad: True\n", - "transformer_encoder.layers.0.self_attn.out_proj.bias requires_grad: True\n", - "transformer_encoder.layers.0.linear1.weight requires_grad: True\n", - "transformer_encoder.layers.0.linear1.bias requires_grad: True\n", - "transformer_encoder.layers.0.linear2.weight requires_grad: True\n", - "transformer_encoder.layers.0.linear2.bias requires_grad: True\n", - "transformer_encoder.layers.0.norm1.weight requires_grad: True\n", - "transformer_encoder.layers.0.norm1.bias requires_grad: True\n", - "transformer_encoder.layers.0.norm2.weight requires_grad: True\n", - "transformer_encoder.layers.0.norm2.bias requires_grad: True\n", - "transformer_encoder.layers.1.self_attn.in_proj_weight requires_grad: True\n", - "transformer_encoder.layers.1.self_attn.in_proj_bias requires_grad: True\n", - "transformer_encoder.layers.1.self_attn.out_proj.weight requires_grad: True\n", - "transformer_encoder.layers.1.self_attn.out_proj.bias requires_grad: True\n", - "transformer_encoder.layers.1.linear1.weight requires_grad: True\n", - "transformer_encoder.layers.1.linear1.bias requires_grad: True\n", - "transformer_encoder.layers.1.linear2.weight requires_grad: True\n", - "transformer_encoder.layers.1.linear2.bias requires_grad: True\n", - "transformer_encoder.layers.1.norm1.weight requires_grad: True\n", - "transformer_encoder.layers.1.norm1.bias requires_grad: True\n", - "transformer_encoder.layers.1.norm2.weight requires_grad: True\n", - "transformer_encoder.layers.1.norm2.bias requires_grad: True\n", - "classifier.weight requires_grad: True\n", - "classifier.bias requires_grad: True\n" - ] - } - ], - "source": [ - "for name, param in model_fine1.named_parameters():\n", - " print(f\"{name} requires_grad: {param.requires_grad}\")" - ] - }, - { - "cell_type": "markdown", - "id": "2f1597ed-a727-42d5-bee1-7aaabd6c7681", - "metadata": {}, - "source": [ - "The following code block simulates fine-tuning on the shortened training set for just 2 epochs. **For the sake of time efficiency, this code block has been commented out**. If you want to see what training looks like, uncomment the following code block, but remember that this code could take approximately 2 minutes to run.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 50, - "id": "3d5c5154-4b9d-4360-83d5-fc21a6ba930b", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - " 0%| | 0/2 [00:00" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/3LEJw8BRgJJFGqlLxaETxA/model-fine1-acc')\n", - "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/-CT1h97vjv0TolY82Nw29g/model-fine1-loss')\n", - "acc_epoch = pickle.load(acc_urlopened)\n", - "cum_loss_list = pickle.load(loss_urlopened)\n", - "plot(cum_loss_list,acc_epoch)" - ] - }, - { - "cell_type": "markdown", - "id": "85866d1f-1a0d-487c-a426-9da326a06c1f", - "metadata": {}, - "source": [ - "The following line loads a prefine-tuned model that was trained for 100 epochs on the full IMDB training set and evaluates its performance on the IMDB test set. **For the sake of efficiency, let's not run the evaluation because it can take a few minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, feel free to uncomment the last line in the code block.**\n" - ] - }, - { - "cell_type": "code", - "execution_count": 54, - "id": "cda6a606-1a87-4846-9926-54edc577879a", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 54, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/e0WOHKh5dnrbC2lGhpsMMw/model-fine1.pth')\n", - "model_fine1_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", - "model_fine1_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "code", - "execution_count": 55, - "id": "e75cfe66-a265-4e7a-8d5f-a7c68f17c6e3", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|█████████████████████████████████████████| 782/782 [00:10<00:00, 77.06it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.8604" - ] - }, - "execution_count": 55, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader, model_fine1_)" - ] - }, - { - "cell_type": "markdown", - "id": "c9b127d6-b41d-4195-ac5a-3da2d8d70b33", - "metadata": {}, - "source": [ - "This model demonstrated notable improvement, exhibiting a remarkable achievement with an accuracy of 86% on the test data. This is higher than the 83% achieved by the model trained from scratch on the IMDB dataset. Although the training process was time-intensive (The fine-tuning was as time-intensive as training the model from scratch), the enhanced performance underscores the fine-tuned model's effectiveness and superiority over the model trained from scratch. Much of the computational effort was devoted to updating the transformer layers. To expedite the training process, one viable strategy is to focus on training the final layer only, which can significantly reduce the computational load but might compromise the model's accuracy.\n" - ] - }, - { - "cell_type": "markdown", - "id": "55331d63-1150-465b-b0bc-d13bbd24fb7c", - "metadata": {}, - "source": [ - "### Fine-tune the final layer only\n", - "\n", - "Fine-tuning the final output layer of a neural network is similar to fine-tuning the whole model. You can begin by loading the pretrained model that you would like to fine-tune. In this case, it is the same model pretrained on the AG News data set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 59, - "id": "c2ffcf34-695f-4b9d-b6c2-5529a74d568a", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 59, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", - "model_fine2 = Net(vocab_size=vocab_size, num_class=4).to(device)\n", - "model_fine2.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "markdown", - "id": "5fa5ba41-8d73-4649-b459-cbc321ca26e2", - "metadata": {}, - "source": [ - "Now, the key difference. You iterate through all of the parameters in the `model_fine2` model and set the `requires_grad` attribute of each parameter to `False`. This effectively freezes all of the layers in the model, meaning that their weights are to be updated during training.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 60, - "id": "000978e5-6244-4be2-8fd5-3e0c7ed1c619", - "metadata": {}, - "outputs": [], - "source": [ - "# Freeze all layers in the model\n", - "for param in model_fine2.parameters():\n", - " param.requires_grad = False" - ] - }, - { - "cell_type": "markdown", - "id": "54b17c25-96f4-43b9-b0a8-963085cf5638", - "metadata": {}, - "source": [ - "Replace the final layer to reflect the fact that you are solving a two-class problem. Note that the new layer will be unfrozen.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 61, - "id": "f0c29db4-3051-4247-8459-b7adab12fa45", - "metadata": {}, - "outputs": [], - "source": [ - "dim=model_fine2.classifier.in_features" - ] - }, - { - "cell_type": "code", - "execution_count": 62, - "id": "b79d7ed1-7a83-4140-ad93-b5e28cdab784", - "metadata": {}, - "outputs": [], - "source": [ - "model_fine2.classifier = nn.Linear(dim, 2)" - ] - }, - { - "cell_type": "code", - "execution_count": 63, - "id": "9387fde1-0a71-45b5-89cf-a2c90ff663d1", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Net(\n", - " (emb): Embedding(400000, 100)\n", - " (pos_encoder): PositionalEncoding(\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " )\n", - " (transformer_encoder): TransformerEncoder(\n", - " (layers): ModuleList(\n", - " (0-1): 2 x TransformerEncoderLayer(\n", - " (self_attn): MultiheadAttention(\n", - " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", - " )\n", - " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", - " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (dropout1): Dropout(p=0.1, inplace=False)\n", - " (dropout2): Dropout(p=0.1, inplace=False)\n", - " )\n", - " )\n", - " )\n", - " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", - ")" - ] - }, - "execution_count": 63, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "model_fine2.to(device)" - ] - }, - { - "cell_type": "markdown", - "id": "42123afb-6a78-4f98-87dd-e87c2cfa7c4f", - "metadata": {}, - "source": [ - "The following block simulates fine-tuning on the shortened training set for just 2 epochs. **For the sake of time efficiency, this code block has been commented out**. The following code should take a shorter amount of time to train than the full fine-tuning conducted previously because only the final layer is unfrozen.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 64, - "id": "3114fa2b-90ee-4955-8459-e01c1db83f2d", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - " 0%| | 0/2 [00:00" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/UdR3ApQnxSeV2mrA0CbiLg/model-fine2-acc')\n", - "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/rWGDIF-uL2dEngWcIo9teQ/model-fine2-loss')\n", - "acc_epoch = pickle.load(acc_urlopened)\n", - "cum_loss_list = pickle.load(loss_urlopened)\n", - "plot(cum_loss_list,acc_epoch)" - ] - }, - { - "cell_type": "markdown", - "id": "3043392c-ee76-4103-a758-83e90f594604", - "metadata": {}, - "source": [ - "The following line loads the pretrained model and evaluates its performance on the test set. **For efficiency, let's not run the evaluation because it can take a few minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, feel free to uncomment the last line in the following code block.**\n" - ] - }, - { - "cell_type": "code", - "execution_count": 68, - "id": "14e51cf2-4e3f-4adf-8c58-f65da0e74f65", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 68, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/B-1H6lpDg-A0zRwpB6Ek2g/model-fine2.pth')\n", - "model_fine2_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", - "model_fine2_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "code", - "execution_count": 69, - "id": "d7cf2dfa-3705-4f3e-a114-b4c56368cc77", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|█████████████████████████████████████████| 782/782 [00:09<00:00, 85.13it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.64144" - ] - }, - "execution_count": 69, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader, model_fine2_)" - ] - }, - { - "cell_type": "markdown", - "id": "dbd9b301-88ba-42d4-b8ca-6e00379fb3b4", - "metadata": {}, - "source": [ - "The previous code indicates that although fine-tuning the final layer takes a significantly smaller amount of time than fine-tuning the whole model, the performance of the model with just the last layer unfrozen is significantly worse (64% accuracy) than the fine-tuned model with all layers unfrozen (86% accuracy).\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "d23b9fe8-1655-4c4c-8ee2-f78db934d5f2", - "metadata": {}, - "source": [ - "# Adapters\n", - "FeatureAdapter is a neural network module that introduces a low-dimensional bottleneck in a transformer architecture to allow fine-tuning with fewer parameters. It compresses the original high-dimensional embeddings into a lower dimension, applies a non-linear transformation, and then expands it back to the original dimension. This process is followed by a residual\n", - "connection that adds the transformed output back to the original input to preserve information and\n", - "promote gradient flow.\n", - "\n", - "## Benefits of using adapters in neural networks\n", - "\n", - "- **Efficient fine-tuning**: Adapters allow for targeted updates to specific parts of the model, reducing the need to retrain large sections of the network.\n", - "\n", - "- **Parameter efficiency**: By adding only a few parameters, adapters make it feasible to modify large models without substantial computational overhead.\n", - "\n", - "- **Preservation of pretrained features**: Adapters enable the modification of a model while retaining the valuable features learned during extensive pretraining.\n", - "\n", - "- **Modularity and flexibility**: They enhance the modularity of models, allowing easy adaptation to various tasks without altering core architecture.\n", - "\n", - "- **Task-specific adaptation**: Adapters can be tailored to improve performance on particular tasks, optimizing the model’s effectiveness.\n", - "\n", - "- **Transfer learning and domain adaptation**: They facilitate the adaptation of models to new domains, bridging gaps between different data distributions.\n", - "\n", - "- **Continual learning**: Adapters support the model's ability to learn new information continuously without forgetting previous knowledge.\n", - "\n", - "- **Reduced risk of overfitting**: With fewer trainable parameters, adapters help prevent overfitting, especially on smaller data sets.\n", - "\n", - "The following code shows an adapter model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 70, - "id": "c46c86b1-eb04-4976-9fa7-c3fc06339bae", - "metadata": {}, - "outputs": [], - "source": [ - "class FeatureAdapter(nn.Module):\n", - " \"\"\"\n", - " Attributes:\n", - " size (int): The bottleneck dimension to which the embeddings are temporarily reduced.\n", - " model_dim (int): The original dimension of the embeddings or features in the transformer model.\n", - " \"\"\"\n", - " def __init__(self, bottleneck_size=50, model_dim=100):\n", - " super().__init__()\n", - " self.bottleneck_transform = nn.Sequential(\n", - " nn.Linear(model_dim, bottleneck_size), # Down-project to a smaller dimension\n", - " nn.ReLU(), # Apply non-linearity\n", - " nn.Linear(bottleneck_size, model_dim) # Up-project back to the original dimension\n", - " )\n", - "\n", - " def forward(self, x):\n", - " \"\"\"\n", - " Forward pass of the FeatureAdapter. Applies the bottleneck transformation to the input\n", - " tensor and adds a skip connection.\n", - "\n", - " Args:\n", - " x (Tensor): Input tensor with shape (batch_size, seq_length, model_dim).\n", - "\n", - " Returns:\n", - " Tensor: Output tensor after applying the adapter transformation and skip connection,\n", - " maintaining the original input shape.\n", - " \"\"\"\n", - " transformed_features = self.bottleneck_transform(x) # Transform features through the bottleneck\n", - " output_with_residual = transformed_features + x # Add the residual connection\n", - " return output_with_residual" - ] - }, - { - "cell_type": "markdown", - "id": "a5c12e9a-5bf1-4a4f-b792-25a264253d28", - "metadata": {}, - "source": [ - "The adapted class wraps this adapter functionality around any specified linear layer, enhancing its output with the non-linearity of a ReLU activation function. This setup is particularly useful for experimenting with subtle architectural modifications in deep learning models, facilitating fine-tuning and potentially improving model performance on complex tasks.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 71, - "id": "97d59ae0-610e-49bb-a9c4-1020c2dc468d", - "metadata": {}, - "outputs": [], - "source": [ - "class Adapted(nn.Module):\n", - " def __init__(self, linear,bottleneck_size=None):\n", - " super(Adapted, self).__init__()\n", - " self.linear = linear\n", - " model_dim = linear.out_features\n", - " if bottleneck_size is None:\n", - " bottleneck_size = model_dim//2 # Define default bottleneck size as half the model_dim\n", - "\n", - " # Initialize FeatureAdapter with calculated bottleneck_size and model_dim\n", - " self.adaptor = FeatureAdapter(bottleneck_size=bottleneck_size, model_dim=model_dim)\n", - "\n", - " def forward(self, x):\n", - " # First, the input x is passed through the linear layer\n", - " x=self.linear(x)\n", - " # Then it's adapted using FeatureAdapter\n", - " x= self.adaptor(x)\n", - " return x" - ] - }, - { - "cell_type": "markdown", - "id": "d8cfb80d-eb8c-4342-8066-33134c5b456d", - "metadata": {}, - "source": [ - "You load the pretrained transformer model that was trained on the AG News dataset.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 74, - "id": "01070ca9-5037-470e-b8f7-ceddfc2f279b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 74, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", - "model_adapters = Net(vocab_size=vocab_size, num_class=4).to(device)\n", - "model_adapters.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "markdown", - "id": "e1df412e-51c5-4e8c-9dbe-c6d38f11a0cf", - "metadata": {}, - "source": [ - "\n", - "First, freeze the parameters of a model named model_adapters to prevent them from being updated during training. Then, retrieve the number of input features for the classifier, and replace the classifier with a new linear layer that outputs to two classes.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 75, - "id": "1820db89-b3b0-4cb2-b42f-1826ba1ec6d0", - "metadata": {}, - "outputs": [], - "source": [ - "for param in model_adapters.parameters():\n", - " param.requires_grad = False\n", - "\n", - "dim= model_adapters.classifier.in_features\n", - "\n", - "model_adapters.classifier = nn.Linear(dim, 2)" - ] - }, - { - "cell_type": "markdown", - "id": "905ad98d-2f62-475a-a7e0-a2f7119b5081", - "metadata": {}, - "source": [ - "Let's explore how to apply the adapted object to a linear layer to obtain the first output. You can obtain the unadapted linear layer for the first output by:\n" - ] - }, - { - "cell_type": "code", - "execution_count": 76, - "id": "cbdcd100-7fda-45ee-a823-9233b7e68ca6", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Linear(in_features=100, out_features=128, bias=True)\n" - ] - } - ], - "source": [ - "my_example_layer=model_adapters.transformer_encoder.layers[0].linear1\n", - "print(my_example_layer)" - ] - }, - { - "cell_type": "markdown", - "id": "1cf2017f-bb26-4b9c-86ea-f32cc5ce4450", - "metadata": {}, - "source": [ - "In the following code, you copy the linear layer and add an adapter layer to it.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 77, - "id": "83fd78c1-111d-45f6-9adf-3ab5c7264a8b", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=64, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=64, out_features=128, bias=True)\n", - " )\n", - " )\n", - ")\n" - ] - } - ], - "source": [ - "my_adapeted_layer=Adapted(my_example_layer)\n", - "print(my_adapeted_layer)" - ] - }, - { - "cell_type": "markdown", - "id": "50e35b98-bcbd-4aec-b039-464e87ddd6fb", - "metadata": {}, - "source": [ - "You can print the adapted layer and show that the new layers have their requires_grad attribute set to True, indicating that these layers will be updated during training.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 78, - "id": "d23c9515-4303-437d-8f02-99a3723e8b89", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "False\n", - "False\n", - "True\n", - "True\n", - "True\n", - "True\n" - ] - } - ], - "source": [ - "for parm in my_adapeted_layer.parameters():\n", - " print(parm.requires_grad)" - ] - }, - { - "cell_type": "markdown", - "id": "1a6bc1b1-f1bb-4c01-99ee-c2e16e5090c2", - "metadata": {}, - "source": [ - "You can set a layer in the model to the adapter layer, as shown in the following code in the commented-out line. However, because there are many layers, a more systematic approach would be to traverse the model and replace specific layers with the adapter layer. Note that you should set the bottleneck size to 24, ensuring that there are fewer parameters to train than during a full fine-tuning.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 79, - "id": "f49d7836-5487-4cb4-a177-797f4eb75cee", - "metadata": {}, - "outputs": [], - "source": [ - "# Adapt a specific layer\n", - "#model_adapters.transformer_encoder.layers[0].linear1=Adapted(my_example_layer)" - ] - }, - { - "cell_type": "code", - "execution_count": 82, - "id": "bc5a4706-ef6f-4d1d-9b44-4e6e3cfba925", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "2" - ] - }, - "execution_count": 82, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Find number of layers\n", - "N_layers=len(model_adapters.transformer_encoder.layers)\n", - "N_layers" - ] - }, - { - "cell_type": "code", - "execution_count": 83, - "id": "6af0d8e7-f38c-47a6-9ba4-7a74994e7f33", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " before linear1\n", - "Linear(in_features=100, out_features=128, bias=True)\n", - " after linear1\n", - "Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=128, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear2\n", - "Linear(in_features=128, out_features=100, bias=True)\n", - " after linear2\n", - "Adapted(\n", - " (linear): Linear(in_features=128, out_features=100, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=100, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=100, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear1\n", - "Linear(in_features=100, out_features=128, bias=True)\n", - " after linear1\n", - "Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=128, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear2\n", - "Linear(in_features=128, out_features=100, bias=True)\n", - " after linear2\n", - "Adapted(\n", - " (linear): Linear(in_features=128, out_features=100, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=100, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=100, bias=True)\n", - " )\n", - " )\n", - ")\n" - ] - } - ], - "source": [ - "# Traverse model and adapt\n", - "for n in range(N_layers):\n", - " encoder=model_adapters.transformer_encoder.layers[n]\n", - " if encoder.linear1:\n", - " print(\" before linear1\")\n", - " print(encoder.linear1)\n", - " model_adapters.transformer_encoder.layers[n].linear1=Adapted(encoder.linear1, bottleneck_size=24)\n", - " print(\" after linear1\")\n", - " print(model_adapters.transformer_encoder.layers[n].linear1)\n", - "\n", - " if encoder.linear2:\n", - " print(\" before linear2\")\n", - " print(model_adapters.transformer_encoder.layers[n].linear2)\n", - " model_adapters.transformer_encoder.layers[n].linear2=Adapted(encoder.linear2, bottleneck_size=24)\n", - " print(\" after linear2\")\n", - " print(model_adapters.transformer_encoder.layers[n].linear2)" - ] - }, - { - "cell_type": "markdown", - "id": "1ea3e84d-649a-462d-8c21-ce1d4a915ce2", - "metadata": {}, - "source": [ - "The following code sends the model to the device.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 84, - "id": "67b7450d-b1f4-42c6-9995-b13da7a8a7d7", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Net(\n", - " (emb): Embedding(400000, 100)\n", - " (pos_encoder): PositionalEncoding(\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " )\n", - " (transformer_encoder): TransformerEncoder(\n", - " (layers): ModuleList(\n", - " (0-1): 2 x TransformerEncoderLayer(\n", - " (self_attn): MultiheadAttention(\n", - " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", - " )\n", - " (linear1): Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=128, bias=True)\n", - " )\n", - " )\n", - " )\n", - " (dropout): Dropout(p=0.1, inplace=False)\n", - " (linear2): Adapted(\n", - " (linear): Linear(in_features=128, out_features=100, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=100, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=100, bias=True)\n", - " )\n", - " )\n", - " )\n", - " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", - " (dropout1): Dropout(p=0.1, inplace=False)\n", - " (dropout2): Dropout(p=0.1, inplace=False)\n", - " )\n", - " )\n", - " )\n", - " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", - ")" - ] - }, - "execution_count": 84, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Send model to device\n", - "model_adapters.to(device)" - ] - }, - { - "cell_type": "markdown", - "id": "d233586b-c7ef-49e6-b684-b8d24997db80", - "metadata": {}, - "source": [ - "Finally, the following code simulates training of the adapted model by training on a shortend IMDB train set for 2 epochs.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 85, - "id": "f275d5f0-ee19-469b-a957-98bf72d3f3c3", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - " 0%| | 0/2 [00:00" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/D49zrrMPWO_ktwQo7PSHIQ/model-adapters-acc')\n", - "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/RXWlmyaco695RiaoU7QsnA/model-adapters-loss')\n", - "acc_epoch = pickle.load(acc_urlopened)\n", - "cum_loss_list = pickle.load(loss_urlopened)\n", - "plot(cum_loss_list,acc_epoch)" - ] - }, - { - "cell_type": "markdown", - "id": "f7e5392a-549b-4b68-968a-8c8d70dbf05d", - "metadata": {}, - "source": [ - "The following code loads the adapted model fine-tuned for 100 epochs on the full IMDB train set and evaluates its performance on the IMDB test set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 89, - "id": "5caaa3f7-5c89-4f27-8ef0-3d3d29f79b66", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " before linear1\n", - "Linear(in_features=100, out_features=128, bias=True)\n", - " after linear1\n", - "Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=128, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear2\n", - "Linear(in_features=128, out_features=100, bias=True)\n", - " after linear2\n", - "Adapted(\n", - " (linear): Linear(in_features=128, out_features=100, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=100, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=100, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear1\n", - "Linear(in_features=100, out_features=128, bias=True)\n", - " after linear1\n", - "Adapted(\n", - " (linear): Linear(in_features=100, out_features=128, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=128, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=128, bias=True)\n", - " )\n", - " )\n", - ")\n", - " before linear2\n", - "Linear(in_features=128, out_features=100, bias=True)\n", - " after linear2\n", - "Adapted(\n", - " (linear): Linear(in_features=128, out_features=100, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=100, out_features=24, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=24, out_features=100, bias=True)\n", - " )\n", - " )\n", - ")\n" - ] - } - ], - "source": [ - "model_adapters_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", - "for n in range(N_layers):\n", - " encoder=model_adapters_.transformer_encoder.layers[n]\n", - " if encoder.linear1:\n", - " print(\" before linear1\")\n", - " print(encoder.linear1)\n", - " model_adapters_.transformer_encoder.layers[n].linear1=Adapted(encoder.linear1, bottleneck_size=24)\n", - " print(\" after linear1\")\n", - " print(model_adapters_.transformer_encoder.layers[n].linear1)\n", - "\n", - " if encoder.linear2:\n", - " print(\" before linear2\")\n", - " print(model_adapters_.transformer_encoder.layers[n].linear2)\n", - " model_adapters_.transformer_encoder.layers[n].linear2=Adapted(encoder.linear2, bottleneck_size=24)\n", - " print(\" after linear2\")\n", - " print(model_adapters_.transformer_encoder.layers[n].linear2)\n", - "\n", - "model_adapters_.to(device)\n", - "for param in model_adapters_.parameters():\n", - " param.requires_grad = False\n" - ] - }, - { - "cell_type": "code", - "execution_count": 90, - "id": "d582b1d3-0fcc-473e-82de-58f9ab1443ef", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 90, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/PGhd5G_NVrWNH-_jdjwNlw/model-adapters.pth')\n", - "model_adapters_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" - ] - }, - { - "cell_type": "code", - "execution_count": 91, - "id": "df69e146-1a1f-4267-9372-9834fcf6616d", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|█████████████████████████████████████████| 782/782 [00:11<00:00, 66.29it/s]\n" - ] - }, - { - "data": { - "text/plain": [ - "0.85608" - ] - }, - "execution_count": 91, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "evaluate(test_dataloader, model_adapters_)" - ] - }, - { - "cell_type": "markdown", - "id": "08262362-924a-42d0-b928-0cf6ffe5331c", - "metadata": {}, - "source": [ - "As you can see, the performance of the fine-tuned adapted model is nearly identical to the fully fine-tuned model, with both models achieving a roughly 86% accuracy. This is an especially surprising result because a significantly smaller number of weights were updated for the adapted model than the fully fine-tuned model. Note that only the adapter layers with a bottleneck size of 24 and the final classifier layer are unfrozen.\n", - "\n", - "The above shows that adapters can be used for parameter efficient fine-tuning (PEFT) and that the performance of a model fine-tuned using adapters can be almost as good as a fully fine-tuned model with all of the layers unfrozen!\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "3b255d65-8f64-4c95-8fd6-ac69d1a67a5a", - "metadata": {}, - "source": [ - "## Exercise: Adapt linear layers in a different network\n", - "\n", - "The following code defines a neural network called `NeuralNetwork`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 92, - "id": "82730ce9-d9a9-483f-8d00-e0cbd78de764", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "NeuralNetwork(\n", - " (flatten): Flatten(start_dim=1, end_dim=-1)\n", - " (linear_relu_stack): Sequential(\n", - " (0): Linear(in_features=784, out_features=512, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=512, out_features=512, bias=True)\n", - " (3): ReLU()\n", - " (4): Linear(in_features=512, out_features=10, bias=True)\n", - " )\n", - ")\n" - ] - } - ], - "source": [ - "class NeuralNetwork(nn.Module):\n", - " def __init__(self):\n", - " super().__init__()\n", - " self.flatten = nn.Flatten()\n", - " self.linear_relu_stack = nn.Sequential(\n", - " nn.Linear(28*28, 512),\n", - " nn.ReLU(),\n", - " nn.Linear(512, 512),\n", - " nn.ReLU(),\n", - " nn.Linear(512, 10),\n", - " )\n", - "\n", - " def forward(self, x):\n", - " x = self.flatten(x)\n", - " logits = self.linear_relu_stack(x)\n", - " return logits\n", - "\n", - "exercise_model = NeuralNetwork()\n", - "\n", - "exercise_model.to(device)\n", - "for param in exercise_model.parameters():\n", - " param.requires_grad = False\n", - "\n", - "print(exercise_model)" - ] - }, - { - "cell_type": "markdown", - "id": "ed4e3b4b-a5df-4543-8d35-adac4ff4b093", - "metadata": {}, - "source": [ - "`NeuralNetwork` is a neural network that uses the `Sequential` container from PyTorch. Adapt the first two linear layers in the `Sequential` container by using the bottleneck adapter with a bottleneck size of 30. Also, change the last linear layer to a layer that has 5 outputs.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 94, - "id": "2fde3cfa-d77d-4ad3-82e1-a3f7eb1d0cf0", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "NeuralNetwork(\n", - " (flatten): Flatten(start_dim=1, end_dim=-1)\n", - " (linear_relu_stack): Sequential(\n", - " (0): Adapted(\n", - " (linear): Linear(in_features=784, out_features=512, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=512, out_features=30, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=30, out_features=512, bias=True)\n", - " )\n", - " )\n", - " )\n", - " (1): ReLU()\n", - " (2): Adapted(\n", - " (linear): Linear(in_features=512, out_features=512, bias=True)\n", - " (adaptor): FeatureAdapter(\n", - " (bottleneck_transform): Sequential(\n", - " (0): Linear(in_features=512, out_features=30, bias=True)\n", - " (1): ReLU()\n", - " (2): Linear(in_features=30, out_features=512, bias=True)\n", - " )\n", - " )\n", - " )\n", - " (3): ReLU()\n", - " (4): Linear(in_features=512, out_features=5, bias=True)\n", - " )\n", - ")\n" - ] - } - ], - "source": [ - "### REPLACE THIS YOUR ANSWER ###\n", - "exercise_model.linear_relu_stack[0] = Adapted(exercise_model.linear_relu_stack[0], bottleneck_size=30)\n", - "exercise_model.linear_relu_stack[2] = Adapted(exercise_model.linear_relu_stack[2], bottleneck_size=30)\n", - "exercise_model.linear_relu_stack[4] = nn.Linear(512, 5)\n", - "print(exercise_model)" - ] - }, - { - "cell_type": "markdown", - "id": "f8244ee4-45b9-4713-8480-6aadf7a66b43", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "exercise_model.linear_relu_stack[0] = Adapted(exercise_model.linear_relu_stack[0], bottleneck_size=30)\n", - "exercise_model.linear_relu_stack[2] = Adapted(exercise_model.linear_relu_stack[2], bottleneck_size=30)\n", - "exercise_model.linear_relu_stack[4] = nn.Linear(512, 5)\n", - "print(exercise_model)\n", - "```\n", - "\n", - "
\n", - "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "151060fe-37c4-4ee1-8034-539b0e3a657a", - "metadata": {}, - "source": [ - "## Congratulations! You have completed the lab\n", - "\n", - "## Authors\n", - "\n", - "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo)\n", - "\n", - "Joseph has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n", - "\n", - "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) \n", - "\n", - "Wojciech \"Victor\" Fulmyk is a Data Scientist at IBM, and a PhD Candidate in economics at the University of Calgary.\n", - "\n", - "[Ashutosh Sagar](https://www.linkedin.com/in/ashutoshsagar/) is completing his MS in CS from Dalhousie University. He has previous experience working with Natural Language Processing and as a Data Scientist.\n", - "\n", - "## References\n", - "\n", - "\n", - "[TEXT CLASSIFICATION WITH THE TORCHTEXT LIBRARY](https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html)\n", - "\n", - "[Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751.pdf)\n", - "\n", - "[Simple, Scalable Adaptation for Neural Machine Translation](https://arxiv.org/pdf/1909.08478)\n", - "\n", - "© Copyright IBM Corporation. All rights reserved.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.20" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/notebooks/LLM_Specialization/Adapters_in_PyTorch.ipynb b/notebooks/LLM_Specialization/Adapters_in_PyTorch.ipynb index d63034b..7f236d0 100644 --- a/notebooks/LLM_Specialization/Adapters_in_PyTorch.ipynb +++ b/notebooks/LLM_Specialization/Adapters_in_PyTorch.ipynb @@ -1,12 +1,3368 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "b75e0832-1651-4d74-85dd-14fd49ef56e3", + "metadata": {}, + "source": [ + "

\n", + " \n", + " \"Skills\n", + " \n", + "

\n", + "\n", + "# **Adapters in PyTorch**\n", + "\n", + "Estimated time needed: **45** minutes\n", + "\n", + "**_Note to advanced users_: If you are already familiar with classical fine-tuning and you only want to see the section that relates to adapters, skip forward to Adapters and run all of the cells above that section by going to _Run --> Run All Above Selected Cell_**\n", + "\n", + "You can fine-tune a neural network in several ways. Common strategies include adjusting only the final layer or fine-tuning all layers. However, these methods have their drawbacks: fine-tuning just the final layer often leads to less than optimal results, while fine-tuning all layers can be very time-consuming.\n", + "\n", + "To address these issues, researchers have developed various parameter efficient fine-tuning (PEFT) techniques. One such technique involves the use of adapters. Adapters enable modular training, where small, task-specific modules are trained within the model without changing the pre-existing pretrained parameters. This approach efficiently tailors the model to new tasks with a reduced risk of overfitting. However, adapters are not a cure-all solution. While they are less likely to overfit and are computationally efficient, they might not always reach the same level of accuracy as full model fine-tuning, particularly if the task necessitates substantial changes from the pretrained model's original capabilities.\n", + "\n", + "In this hands-on lab, you learn how to apply an adapter to a transformer-based neural network that has been trained on the AG News data set, with the aim of using this model on the IMDB data set. You also evaluate and compare the performance of this method with that of a fully fine-tuned model and a model where only the last layer is fine-tuned.\n", + "\n", + "---\n", + "\n", + "# __Table of contents__\n", + "\n", + "
    \n", + "
  1. Objectives
    2. \n", + "
  2. \n", + " Setup\n", + "
      \n", + "
    1. Install required libraries
      2. \n", + "
    2. Import required libraries
      3. \n", + "
    3. Defining helper functions
      4. \n", + "
    \n", + "
    3. \n", + "
  3. Positional encodings
    4. \n", + "
  4. Import IMDB data set
    5. \n", + "
      \n", + "
    1. IMDB data set overview
      2. \n", + "
        \n", + "
      1. Data set composition
        2. \n", + "
      2. Applications
        3. \n", + "
      3. Challenges
        4. \n", + "
      4. Data set splits
        5. \n", + "
      5. Data loader
        6. \n", + "
      6. Neural network
        7. \n", + "
      \n", + "
    \n", + "
  5. \n", + " Training\n", + "
      \n", + "
    1. Train IMDB
      2. \n", + "
    2. Fine-tune a model pretrained on the AG News data set
      3. \n", + "
    3. Fine-tune the final layer only
      4. \n", + "
    \n", + "
    6. \n", + "
  6. \n", + " Adapters\n", + "
      \n", + "
    1. Benefits of using adapters in neural networks
      2. \n", + "
    \n", + "
    7. \n", + "
  7. \n", + " Exercise: Adapt linear layers in a different network\n", + "
    8. \n", + "
\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "9e4d071d-35d8-4d69-9168-a40d25df2205", + "metadata": {}, + "source": [ + "# Objectives\n", + "\n", + "After completing this lab, you are able to:\n", + "\n", + "- Define and pretrain a transformer-based neural network using PyTorch for a classification task [Optional]\n", + "- Fully fine-tune the pretrained model for a different classification task [Optional]\n", + "- Compare results by fine-tuning only the last layer of the pretrained model [Optional]\n", + "- Understand how adapters work\n", + "- Apply adapters to linear layers in a neural network\n", + "- Train a neural network in a parameter efficient way by training just the adapted layers\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "35a85867-b5fa-4780-b512-35bdf829b33c", + "metadata": {}, + "source": [ + "# Setup\n", + "\n", + "### Install required libraries\n", + "\n", + "For this lab, you use the following libraries, which are __not__ preinstalled in the Skills Network Labs environment. __You must run the code in the following cell__ to install them.\n", + "\n", + "```bash\n", + "!pip install --upgrade portalocker==2.8.2 torchtext==0.17.0 torchdata==0.7.1 pandas==2.2.2 matplotlib==3.9.0 scikit-learn==1.5.0 torch==2.2.0 numpy==1.26.4\n", + "```\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "a2d7c12f-2141-4af9-91a8-b0c1c8088d66", + "metadata": {}, + "source": [ + "### Import required libraries\n", + "\n", + "The following code imports the required libraries.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "81927908-a8c5-42d6-b24c-97b8b13a42e6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n", + "Tesla P40\n", + "Import Successfully!\n" + ] + } + ], + "source": [ + "# Environment setup\n", + "import os\n", + "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n", + "\n", + "# Suppress warnings\n", + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "def warn(*args, **kwargs):\n", + " pass\n", + "warnings.warn = warn\n", + "\n", + "# PyTorch and related libraries\n", + "import torch\n", + "from torch import nn\n", + "from torch.utils.data import DataLoader, Dataset\n", + "from torch.utils.data.dataset import random_split\n", + "from torch.nn.utils.rnn import pad_sequence\n", + "\n", + "# TorchText for NLP tasks\n", + "from torchtext.datasets import AG_NEWS, IMDB\n", + "from torchtext.data.utils import get_tokenizer\n", + "from torchtext.vocab import build_vocab_from_iterator, GloVe, Vectors\n", + "from torchtext.data.functional import to_map_style_dataset\n", + "\n", + "# Utility libraries\n", + "import time\n", + "from itertools import accumulate\n", + "import math\n", + "import pickle\n", + "import io\n", + "from urllib.request import urlopen\n", + "import tarfile\n", + "import tempfile\n", + "\n", + "# Data manipulation and visualization\n", + "import numpy as np\n", + "import pandas as pd\n", + "import matplotlib.pyplot as plt\n", + "from tqdm import tqdm\n", + "\n", + "# Jupyter Notebook utilities\n", + "from IPython.display import Markdown as md\n", + "\n", + "# PyTorch-specific configurations\n", + "torch.set_num_threads(1)\n", + "\n", + "# CUDA-related checks\n", + "print(torch.cuda.is_available())\n", + "print(torch.cuda.get_device_name())\n", + "\n", + "print(\"Import Successfully!\")" + ] + }, + { + "cell_type": "markdown", + "id": "5e710480-51b4-46c7-a3b4-5c80021007b2", + "metadata": {}, + "source": [ + "### Define helper functions\n", + "\n", + "The following code shows some helper functions to help with plotting, saving, and loading files. These functions are not the main focus of this lab, so you do not have to dwell on these too long. However, do run the cells in this section to define these helper functions.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "57d49c2a-d309-4608-8e45-b9fa8ec63cbd", + "metadata": {}, + "outputs": [], + "source": [ + "def plot(COST,ACC):\n", + "\n", + " fig, ax1 = plt.subplots()\n", + " color = 'tab:red'\n", + " ax1.plot(COST, color=color)\n", + " ax1.set_xlabel('epoch', color=color)\n", + " ax1.set_ylabel('total loss', color=color)\n", + " ax1.tick_params(axis='y', color=color)\n", + "\n", + " ax2 = ax1.twinx()\n", + " color = 'tab:blue'\n", + " ax2.set_ylabel('accuracy', color=color) # you already handled the x-label with ax1\n", + " ax2.plot(ACC, color=color)\n", + " ax2.tick_params(axis='y', color=color)\n", + " fig.tight_layout() # otherwise the right y-label is slightly clipped\n", + "\n", + " plt.show()\n", + "\n", + "\n", + "def save_list_to_file(lst, filename):\n", + " \"\"\"\n", + " Save a list to a file using pickle serialization.\n", + "\n", + " Parameters:\n", + " lst (list): The list to be saved.\n", + " filename (str): The name of the file to save the list to.\n", + "\n", + " Returns:\n", + " None\n", + " \"\"\"\n", + " with open(filename, 'wb') as file:\n", + " pickle.dump(lst, file)\n", + "\n", + "\n", + "def load_list_from_file(filename):\n", + " \"\"\"\n", + " Load a list from a file using pickle deserialization.\n", + "\n", + " Parameters:\n", + " filename (str): The name of the file to load the list from.\n", + "\n", + " Returns:\n", + " list: The loaded list.\n", + " \"\"\"\n", + " with open(filename, 'rb') as file:\n", + " loaded_list = pickle.load(file)\n", + " return loaded_list" + ] + }, + { + "cell_type": "markdown", + "id": "fb4827b6-33c9-4370-bfbf-983d89623c98", + "metadata": {}, + "source": [ + "---" + ] + }, + { + "cell_type": "markdown", + "id": "0d6f6a86-020b-4ed5-8c52-5fa69dceca97", + "metadata": {}, + "source": [ + "# Positional encodings\n", + "\n", + "Positional encodings play a pivotal role in transformers and various sequence-to-sequence models, aiding in conveying critical information regarding the positions or sequencing of elements within a given sequence. To illustrate, let's examine the sentences: \"He painted the car red\" and \"He painted the red car.\" Despite their distinct meanings, it's worth noting that the embeddings for these sentences remain identical in the absence of positional encodings. The following class defines positional encodings by inheriting from PyTorch's `Module` class.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "88c9cebf-7dbe-46d0-81d2-5a116120c374", + "metadata": {}, + "outputs": [], + "source": [ + "class PositionalEncoding(nn.Module):\n", + " \"\"\"\n", + " https://pytorch.org/tutorials/beginner/transformer_tutorial.html\n", + " \"\"\"\n", + "\n", + " def __init__(self, d_model, vocab_size=5000, dropout=0.1):\n", + " super().__init__()\n", + " self.dropout = nn.Dropout(p=dropout)\n", + "\n", + " pe = torch.zeros(vocab_size, d_model)\n", + " position = torch.arange(0, vocab_size, dtype=torch.float).unsqueeze(1)\n", + " div_term = torch.exp(\n", + " torch.arange(0, d_model, 2).float()\n", + " * (-math.log(10000.0) / d_model)\n", + " )\n", + " pe[:, 0::2] = torch.sin(position * div_term)\n", + " pe[:, 1::2] = torch.cos(position * div_term)\n", + " pe = pe.unsqueeze(0)\n", + " self.register_buffer(\"pe\", pe)\n", + "\n", + " def forward(self, x):\n", + " x = x + self.pe[:, : x.size(1), :]\n", + " return self.dropout(x)" + ] + }, + { + "cell_type": "markdown", + "id": "aadc814f-d060-47be-963f-c28cfd0618e4", + "metadata": {}, + "source": [ + "# Import IMDB data set\n", + "\n", + "The following code loads the IMDB data set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "c479b278-01b9-4863-9821-528b1607a74b", + "metadata": {}, + "outputs": [], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/35t-FeC-2uN1ozOwPs7wFg.gz')\n", + "tar = tarfile.open(fileobj=io.BytesIO(urlopened.read()))\n", + "tempdir = tempfile.TemporaryDirectory()\n", + "tar.extractall(tempdir.name)\n", + "tar.close()" + ] + }, + { + "cell_type": "markdown", + "id": "ec7531bd-8b33-45d9-8061-9de43bd188b8", + "metadata": {}, + "source": [ + "## IMDB data set overview\n", + "\n", + "The **IMDB data set** contains movie reviews from the Internet Movie Database (IMDB) and is commonly used for binary sentiment classification tasks. It's a popular data set for training and testing models in natural language processing (NLP), particularly in the context of sentiment analysis.\n", + "\n", + "### Data set composition\n", + "\n", + "- **Reviews**: The data set consists of 50,000 movie reviews, divided evenly into 25,000 training and 25,000 testing samples.\n", + "- **Sentiment labels**: Each review is labeled as either positive or negative, indicating the sentiment expressed in the review. The data set is balanced, with an equal number of positive and negative reviews in both the training and testing sets.\n", + "- **Text content**: Reviews are presented as plain text and have been preprocessed to some extent. For example, HTML tags are removed, but the text retains its original punctuation and capitalization.\n", + "- **Usage**: The data set is commonly used to train models for binary sentiment classification, where the goal is to predict whether a given review is positive or negative based on its text content.\n", + "\n", + "### Applications\n", + "\n", + "- **Sentiment analysis**: The primary application of the IMDB data set is in sentiment analysis, where it serves as a benchmark for various text classification algorithms.\n", + "- **Natural language processing**: The data set is widely used in NLP research and applications, providing a basis for testing the effectiveness of different models and approaches in understanding human language.\n", + "\n", + "### Challenges\n", + "\n", + "The data set is small, so it's hard to train a model from scratch.\n", + "\n", + "The following class is defined to traverse the IMDB data set. The need to define this class arises from the fact that the IMDB data set is split across a large number of files.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "51bc66a7-506d-4ac4-aa0b-1813c6a0e4c5", + "metadata": {}, + "outputs": [], + "source": [ + "class IMDBDataset(Dataset):\n", + " def __init__(self, root_dir, train=True):\n", + " \"\"\"\n", + " root_dir: The base directory of the IMDB dataset.\n", + " train: A boolean flag indicating whether to use training or test data.\n", + " \"\"\"\n", + " self.root_dir = os.path.join(root_dir, \"train\" if train else \"test\")\n", + " self.neg_files = [os.path.join(self.root_dir, \"neg\", f) for f in os.listdir(os.path.join(self.root_dir, \"neg\")) if f.endswith('.txt')]\n", + " self.pos_files = [os.path.join(self.root_dir, \"pos\", f) for f in os.listdir(os.path.join(self.root_dir, \"pos\")) if f.endswith('.txt')]\n", + " self.files = self.neg_files + self.pos_files\n", + " self.labels = [0] * len(self.neg_files) + [1] * len(self.pos_files)\n", + " self.pos_inx=len(self.pos_files)\n", + "\n", + " def __len__(self):\n", + " return len(self.files)\n", + "\n", + " def __getitem__(self, idx):\n", + " file_path = self.files[idx]\n", + " label = self.labels[idx]\n", + " with open(file_path, 'r', encoding='utf-8') as file:\n", + " content = file.read()\n", + " return label, content" + ] + }, + { + "cell_type": "markdown", + "id": "6bb31d42-b20e-413d-96f6-7d680eb83bb2", + "metadata": {}, + "source": [ + "The following code uses the `IMDBDataset` class previously defined to create iterators for the train and test data sets. In the latter part of the cell, you can return 20 examples from the train set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "01513044-f657-4203-b933-bad10ebeb4c8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(0, 'It is important not to be insulted by lack of logic or common sense and those who have any \"gray matters\" will agree that this movie just doesn\\'t work.

The problems lay in the direction, cast selections and lack of depth in the character building. The word comedy was very hard thing to say when i expect to laugh when these words are used. Let\\'s look at the problems in direction/script.

Brother and sister both in their mid 30\\'s seem to be well adjusted. They meet a complete stranger at a park and Heather Graham character walks up to her and asks the most intimate questions that even half sane person would be running the other way or at least scream for a police officer. He then awkwardly walks over and makes some stupid statements and she falls for him. Then after ONE date were they all go out together he falls in love with her and decides to get married in Vegas in a week\\'s time???? Hello does anyone feel stupid yet? He goes out with thousands of women and he meets this one person who says about 10 words that WE see on the screen and he wants to marry her. Not only was there no chemistry it just doesn\\'t make sense. Sure it\\'s a romantic comedy and I want to believe it could, but the direction made it completely flat.

Now Heather falls head over heels with her too and when Heather Graham and Bridget Moynahan (very shallow character) kiss or more to the point it was sloppiest kiss ever that chemistry MIGHT be there. I found it unromantic and unfunny and while many say Heather cannot act i think the reality is Heather was clearly the wrong person for this role.

This was Sue Kramer debut as a director and to me it was just too much for her to chew. It would take a lot of craft to make this movie work and IMHO it could be done with better writers and casting and direction.')\n", + "(0, \"THE SCREAMING SKULL (1 outta 5 stars) This movie boasts some pretty cool opening credits (an offscreen narrator warning that movie patrons will be offered a free burial if they die of fright watching this movie, a scary shot of a skull emerging from a placid pool and the ubiquitous scary music) but, sadly, the movie is all downhill from there. A widowed man takes his new bride to his secluded mansion... admonishing his servants and friends that the new Mrs. has a very fragile disposition due to a tragedy in her past. Well, in no time at all she begins to see and hear mysterious things that no one else can. Her husband assures her that it's all merely in her mind and... well, you can probably see where this all is going. You will have figured out what's going on long before our hapless heroine... because you have probably seen the exact same plot in hundreds of other movies and TV shows (and done better, too). To add to the movie's myriad transgressions, most cuts of this movie (on numerous cheap DVD compilations) seem to be missing a few key scenes. You see the heroine slowly walking towards the window... she goes to open it... you know she is going to see something scary... and then... suddenly the scene cuts to her sobbing in her husband's arms. So what did she see??? I guess we'll never know.\")\n", + "(0, \"

Whether any indictment was intended must be taken into consideration. If in the year 2000 there were still rifts of feeling between Caucasian and Afro-Americans in Georgia, such as shown in this film, obviously there remains a somewhat backward mentality among a lot of people out there. It is rather hypocritical, to say the least, if everyone adores Halle Berry, Whoopie Goldberg, Beyoncé, Noemi Campbell, Denzel Washington, Will Smith, et. al., whilst out in the backs there persist manifest racial divides.

White grandmother suddenly gets black grand-daughter thrust upon her, only to meet up with black grandfather in a very white social backwater. The story is sweet, not lacking tragic overtones, and eminently predictable as in most of these kinds of TV films, though the final scene has you guessing............ will he? won't he.......?

Gena Rowlands in her typical style offers a sincere rendering, and Louis Gossett is a good match for her; the little Penny Bae fortunately does not steal the show.

A `nice' way of relaxing after Sunday lunch without having to force your mind too much, though you might just find yourself having a little siesta in the middle of it.\")\n", + "(0, 'First, it takes a full half hour to get Hackman out of jail and to start doing the job. What a waste of time, we all know Hackman is getting out to do some job for his masters, why waste almost a third of the movie on these sequences. Then Hackman stays in a hotel and the story arc again goes nowhere, simply proving to us that Hackman is under close watch and anything he says or does is know by the masters. Again, another 20 minutes. Then more wasted time showing the reunion with his wife. All of this should have taken 10-15 minutes at most simply as a set-up for the real action, intrigue and plot twists. By the time the real action gets going, I was so bored that I just wanted the movie to end. Hackman is great as usual, and the other actors as well, but this is a dud of the first magnitude.')\n", + "(0, \"Banned as a 'Video Nasty' in the UK, Unhinged has naturally gained quite a bit of notoriety. However, the most shocking thing I found about the film was its amateurishness in all departments. The bloodletting I could handle: the terrible acting, shoddy editing, awful direction, lousy script and abysmal soundtrack were much harder to take.

Three girls on their way to a music festival crash into a ravine during a storm. They are rescued by a friendly stranger who takes them to a nearby house. The owner of the house, a batty old lady, and her spinster daughter, welcome the girls in, allowing them to stay for a few days in order to recuperate. However, someone doesn't want the girls to leave\\x97ever! One by one they fall victim to an unseen assailant.

Taking a long time to get going and featuring some of the worst performances ever in a horror film (and that takes some doing), Unhinged is a truly awful film. The music is a total mess (it sounds like a three year old has been let loose on a synthesiser) and as such, it complements the movie perfectly. Only a couple of bloody scenes towards the end and a bit of gratuitous nudity save Unhinged from getting the lowest possible score.

If you are a horror completist (and unfortunately, I am), you will want to see this in order to tick it off the Video Nasty watch-list. But be warned\\x97it is really, really bad.\")\n", + "(0, \"When great director/actor combinations are talked about the team of J. Lee Thompson and Charles Bronson is not usually mentioned. Probably because the output of nine joint ventures between the two of them runs the gamut from the really good action entertainment to the mediocre. Unfortunately Kinjite: Forbidden Subjects falls in the latter.

That's sad because Kinjite could have been a whole lot better. But for the life of me I don't understand why it was necessary to make the father of the missing Japanese girl, a guy used to getting some cheap jollies because the romance in his marriage has run out. That might have been good for another film altogether, but it served no purpose here.

A straightforward cop drama with Charles Bronson as a vice cop who's seen a bit too much in his line of work and has a strong prejudice against orientals. That part could also have used a little explaining as well. But he's going to have to overcome it if he and patient partner Perry Lopez are going to locate a captured Japanese school girl.

Bronson's time in the vice squad have told him exactly where to look for the kidnapper. A stylish, murderous pimp played by Jaime Fernandez is the guy and he and Bronson have some history. In fact in the film's best scene, Bronson made him eat an expensive rolex watch and set his car on fire.

At one point Fernandez happens to spot Bronson and Lopez in an all night delicatessen and this being after his rolex snack, he sprays the place with an Uzi killing everyone, but Bronson and Lopez. I really think that little incident would have had more than a couple vice cops from the LAPD after Fernandez. But that's another terribly big hole in the plot.

Still there is a very rough justice in the end for Fernandez. I wish the whole film had been better though. This was the last film of the Bronson-Thompson team and J. Lee Thompson's last as a director. He should have gone out with something better.\")\n", + "(0, 'First off, let me say that I am a great believer in Fanpro stuff. I see it as a way to continue a good show long after it has been cancelled. Star Trek Voyages and Star Wars Revelations are examples of decent efforts. So I have a soft-spot for fanpro stuff that means I\\'ll overlook things that I would ordinarily slate badly.

So on to ST: HF. Well, first off the good things. Enthusiasm is a major part of making any show believable and, for the most part, the crew of the various ships all seem to be having a good time with their roles. Next, the effects aren\\'t bad for a home-brew effort, with nothing to make you really wince. The stories aren\\'t too bad either. Nothing particularly innovative, but solid enough stuff and at least there are ongoing story-arcs.

But it has a lot of faults.

First off, although they quite obviously HAVE to rip-off Star Trek footage, set backdrops, music and effects, I see no reason why they proceeded to rip off virtually every other sci-fi musical score ever made. Everything from Aliens to Starship Troopers rears it orchestral head at one point or another. Likewise, much of the footage is from other movies, dutifully CGI\\'d over to make it look different. The Grey warships, for instance, though disguised, are quite obviously Star Destroyers from Star Wars. And the station is also rather obviously Fleet Battle Station Ticonderoga from Starship Troopers. Likewise, sound effects from various Star Wars movies appear in space battles between fighters, as does animated over footage. In one scene in either first or second season, I think, you even see two TIE fighters fly past during a battle, which hardly does your suspension of disbelief any favours.

Acting varies from the reasonable to the hideously painful to watch. Everyone does improve as the seasons progress, though, but expect to grimace at the screen a lot, especially in the early seasons. They\\'ve also made some interesting acting choices. Let\\'s just say that the food replicators on this show seem permanently set to \"cake\" and leave it at that.

Make-up effects are generally quite effective on the whole. But they really ought to mercilessly club to death the person who decided to use cheap Ferengi and Cardassian masks for anything other than background use or \"passing\" shots. They are just beyond unrealistic. Every time I saw one of these (apart from trying not to laugh too much) I kept expecting the unfortunate soul wearing it to pull out a gun and announce that \"This is a stick-up!\" In one scene a \"Cardassian\" actually talks whilst wearing one of these. Not only do the lips not move, but the mask doesn\\'t even have an opening where the mouth should be. Someone needs to be slapped hard for that. Couldn\\'t they have taken a craft knife to it, for goodness\\' sake! There are also some well-done, but unintentionally funny make-up jobs, such as the Herman Munster look alike.

The writing, though coherent, is nothing new. Instead the script runs like a continuation of DS9, with the ships heading out from DS12 on various missions. The new enemy, \"The Grey\" aren\\'t very menacing and the plot line involving them is effectively a reworking of the Borg threads. i.e. Starfleet meet the Grey, the Grey are hugely powerful, Starfleet barely escape with their lives, then through technology they begin to find ways to combat the enemy etc etc. All done before with the Borg.

Another bone of contention is the dialogue. Star Trek writers have long had the ability to write \"insert technobabble here\" into a script. It usually means an exposition of the latest plan to combat the enemy using \"quantum phase discriminators\" or \"isolytic charges\" etc. In other words, nonsense that tells you that they are on the case and a resolution is at hand.

The words are just gibberish really. I\\'ve no problem with this, but where ST:HF makes a mess of it is where they include real-world comments into this concept.

Tactical advice such as \"We need to regroup\" sounds good, but not when uttered by trio of characters already standing in a group. Likewise when asked what the situation is, a tactical officer is heard to reply \"We count three battleships\". He actually needed to count them? C\\'mon! I expected the questioner to ask him \"Are you sure?\" or \"Can you double check\". But my all-time favourite comment is this:

Captain: \"Can we establish two-way communication?\"

Comms officer: \"No, we can only send and receive..\"

Well, duh!.....

Having said all the above, the show does improve as it goes along. Seasons 1 and 2 are pretty bad, 3 shows an improvement but 4 & 5 are where it starts to get noticeably better. Season 6 so far looks quite reasonable.

I do have a problem with their choice of media for the shows though. Quicktime sucks, quite frankly and the sooner they move to divx/avi format the better. Some of us like to actually take our downloaded shows and watch them on decent size screen and not peer at a tiny QT window on a computer monitor. Not only does Quicktime make this difficult, but the 320x180 resolution the shows are in does not scale at all well. In fact, it makes the shows pretty unwatchable, like they were a tenth-generation VHS tape copy. The least they could do was to include a hi-res downloadable option.

Anyway, the show has promise, and I\\'m even beginning to like some of the characters. But that\\'s 40 episodes on, so I\\'m not sure this says that much about character development at all.

But what can you say, it\\'s free....

PS: Out of 28 votes, 19 people rated this show as a 9 or 10. Hmmmm... were we watching the same show? Or are you 19 all three year olds?')\n", + "(0, 'I read in the papers that W.Snipes was broke so no wonder he would take parts in low budget projects like The Contractor.He is just the next action star to join a growing club:the penniless action stars of the 90s (Van Damme,Segal,Lundgren,Snipes). Here he stars the lead in a cheap action flick which was shot in Bulgaria( we are supposed to believe that the location is London, like only a complete moron would buy that)The story is the one of 1000 other movies: retired special forces good guy gets hired by the government again to do a wet job- after that government wants to get rid of him- good guy gets away after killing bad guys (was that a spoiler? guess not!) The star of the movie: the little girl (Eliza Bennett) outperforms everybody else of the cast!!!One star is for her plus one star for eye candy Lena Headey, makes 2 stars. Only for die hard Snipes fans!Everybody else:avoid!')\n", + "(0, \"Did HeidiJean really see this movie? A great Christmas movie? Not even close. Dull, bland and completely lacking in imagination and heart. I kept watching this movie wondering who the hell thought that Carly Pope could play the lead in this movie! The woman has no detectable personality and gives a completely lackluster performance. Baransky was great as usual and provided the only modicum of interesting the whole thing. Probably her involvement was the only reason this project was green lighted to begin with. Maybe I'm expecting too much for a Lifetime movie played 15 days from Christmas but I sat through this thing thinking that with a different director and a recasting JJ with an actress that at least could elicit sympathy this could have been quite a cute little movie.\")\n", + "(0, 'Band Camp was awful, The Naked Mile was a little better, and this third straight to DVD in the American Pie franchise seems the same quality as the predecessor. Basically Erik Stifler (John White) split from his girlfriend after losing his virginity, and now him and Mike \\'Cooze\\' Coozeman (Jake Siegel) are joining Erik\\'s cousin Dwight (Steve Talley) at college. With the promise of many parties, plenty of booze, and enough hot chicks at the Beta House, they only have fifty listed tasks to carry out to become official privileged members. But a threat comes into sight with the rivals, GEK (\"Geek\") House, led by power-hungry nerd (and sheep shagger) Edgar (Tyrone Savage) offering bigger and better than what Beta have. To settle it once and for all, Beta and Gek go into battle with the banned, for forty years, Greek Games to beat each other in, with the loser moving out. The last champion of the games, Noah Levenstein aka Jim\\'s Dad (the only regular Eugene Levy) runs the show, which sees the people unhooking bras, a gladiator duel floating on water, catching a greased pig, Russian Roulette in the mouth with cartridges of aged horse spunk, wife carrying and drinking a full keg of alcohol (with puking not disqualifying). It all comes to the sudden death, with a guy getting stripper lap dancing, and they have to resist cumming, Beta House win when Edgar cums with a girl dressed as a sheep on his lap. Also starring Flubber\\'s Christopher McDonald as Mr. Stifler, Meghan Heffern as Ashley, Dan Petronijevic as Bull, Nic Nac as Bobby, Christine Barger as Margie, Italia Ricci as Laura Johnson, Moshana Halbert as Sara Coleman, Sarah Power as Denise, Andreja Punkris as Stacy and Jordan Prentice as Rock. The nudity amount is very slightly increased, as is the grossness of the jokes, and I could guess it being rated one star out of five, but I like it. Adequate!')\n", + "(1, \"I saw this recently on a cable channel. The movie is great; it's one of the few musicals I have seen that doesn't shy away from the light and dark. It portrays some of the splendour of the age along with a lot of the squalor. Some of the set piece dance sequences so much is going on, I didn't know where to look next. One day I shall go and see this on the big screen, just so that I see what's happening. But what really lifts this to another level is Oliver Reed's performance as Bill Sykes. Not only is a thoroughly mean and menacing man but there is something else, some inner demons. He gave me the impression that if you pushed him into a corner, he was capable of anything. It was almost as if the Sykes character was on the edge of madness, just awaiting the trigger. I have seen the Robert Newton's Bill Sykes from the 1948 movie, and I thought he was 'just' a bad egg, but Oliver Reed's performance intimidated me in my own living room.\")\n", + "(1, '\"The Gingerbread Man is the first thriller I\\'ve ever done!\" \\x96 Robert Altman

In 1955 Charles Laughton directed \"The Night of the Hunter\", a spooky slice of Southern Gothic in which Robert Mitchum plays a scary serial killer. One of the film\\'s more famous sequences consists of two kids escaping from Mitchum on a rowboat, the kids frantically paddling whilst Mitchum wades after them like a monster.

Seven years later Mitchum played an equally spooky killer in \"Cape Fear\", another film set in the American South. That film featured a local attorney trying to protect his family and likewise ended with Mitchum terrorising folks on a boat. In 1991 Martin Scorsese, trying to branch out and tackle something more mainstream, remade \"Cape Fear\", boat scene and all.

Now we have Robert Altman\\'s \"The Gingerbread Man\", another slice of small town Southern Gothic. Altman says he consulted \"The Night of the Hunter\" for inspiration and tackled such a mainstream film purely because he wanted to \"spread his wings and try a popcorn picture\", but what he\\'s secretly attempting to do here is deconstruct the canonical films of the Southern Gothic genre.

So instead of a showdown on small boat, we get a showdown on a giant ship. Instead of two kids being kidnapped, we get two kids being safely returned to the police. Instead of money being hidden, we have money being readily given via a last will and testament. Instead of the righteous attorney of the 1961 film and the deplorable attorney of the 1991 remake, we get a rather three-dimensional lawyer in Kenneth Branagh. Instead of the monster chasing the family we get the hero chasing the bad guys. Instead of the monster breaking into the family\\'s house boat, we have the hero hunting the monster on board the monster\\'s \"house ship\". Similarly, instead of a murderous serial killer we get an innocent weirdo played by Robert Duvall. . .etc etc etc.

Altman goes on and on, reversing everything just a little slightly, pulling at the edges and doing his own thing. His touch is most apparent during the film\\'s first half-hour, the film existing in an uneasy space between conventional plot-driven movie storytelling and Altman\\'s fondness for overlapping dialogue, casual narratives, prowling camera movement and the way that characters aren\\'t so much introduced as they are simply part of what\\'s going on.

Still, despite Altman\\'s best intentions, the film never rises above mediocrity. Altman\\'s too bound to the conventions of the \"thriller format\" to do much damage, his style is too lethargic to generate tension and the film is simply not radical enough to counterpoint other canonical films in the genre. \"Gingerbread Man\" is thus too mainstream to work as a more pure Altman film and too Altman to work as a mainstream thriller.

The film\\'s not a complete waste, though. Robert Downey Junior, Kenneth Branagh and the usually intolerable Daryl Hannah, all turn in juicy performances. The film also has a nice atmosphere, set against a approaching hurricane, and the final act contains some interesting twists and turns. While it\\'s not the complete disaster that Scorsese\\'s \"Cape Fear\" was, the film still never amounts to anything special.

7/10 \\x96 In the late 90s Altman made 3 successive films set in the American South: \"Kansas City\", \"Gingerbread Man\" and \"Cookie\\'s Fortune\". Unlike \"Gingerbread Man\", both \"Kansas City\" and \"Cookie\\'s Fortune\" tackle the genre on the broader, more looser canvases that Altman was most comfortable with.

\"Kansas City\" is the more important of these two films, its hierarchies of class, politics and crime, and its desire to break radically away from typical gangster genre frameworks, would prove influential on all serious 21st century film crime writers (see, for example, \"The Wire\"). That said, \"Cookie\\'s Fortune\", while a much slighter tale, is perhaps the better picture.

Note: Altman claims that this is his first thriller, but he directed \"Images\", an art house thriller, in 1972.

Worth one viewing.')\n", + "(1, '\"Read My Lips (Sur mes lèvres)\" (which probably has different idiomatic resonance in its French title) is a nifty, twisty contemporary tale of office politics that unexpectedly becomes a crime caper as the unusually matched characters slide up and down an ethical and sensual slippery slope.

The two leads are magnetic, Emmanuelle Devos (who I\\'ve never seen before despite her lengthy resume in French movies) and an even more disheveled than usual Vincent Cassel (who has brought a sexy and/or threatening look and voice to some US movies).

The first half of the movie is on her turf in a competitive real estate office and he\\'s the neophyte. The second half is on his turf as an ex-con and her wrenching adaptation to that milieu.

Writer/director Jacques Audiard very cleverly uses the woman\\'s isolating hearing disability as an entrée for us into her perceptions, turning the sound up and down for us to hear as she does (so it\\'s even more annoying than usual when audience members talk), using visuals as sensory reactors as well.

None of the characters act as anticipated (she is not like that pliable victim from \"In the Company of Men,\" not in individual interactions, not in scenes, and not in the overall arc of the unpredictable story line (well, until the last shot, but heck the audience was waiting for that fulfillment) as we move from a hectic modern office, to a hectic disco to romantic and criminal stake-outs.

There is a side story that\\'s thematically redundant and unnecessary, but that just gives us a few minutes to catch our breaths.

This is one of my favorites of the year!

(originally written 7/28/2002)')\n", + "(1, \"This was a pretty decent movie. This movie is good to just sit down and watch and be entertained. Just a typical Hollywood film. This movie will never win an Oscar or anything and definitely doesn't deserve one, but I thought it was pretty good. It's kind of like the show 24 but set into movie format. If you like the whole we've got to stop the terrorist from killing the president kind of movie then you will enjoy this flick. I personally think that storyline has been done WAY too much, but The Sentinel does add a little twist with the mole in the Secret Service. All in all, this movie won't leave your jaw to the floor or change your life, but who says every single movie has to be like that to be good?\")\n", + "(1, 'Actually they could not have chosen a better diversified actor to portray Little Richard than Leon. He captures Little Richard to a most believable essence. The outfits where wonderful and any person watching this movie will definitely keep a smile on their face through the entire movie. Although the movie is a little long, it keeps your attention with the personality and outfits of Little Richard in mind. The ending should have taken a direction of moving Little Richard more into the present where you could see him as he has aged into this new millennium. He will always be the King of Rock-N-Roll as far as I am concerned regardless of what the other media says.')\n", + "(1, \"If you are a fan of Altman's large ensemble casts, as evidenced in major films like M.A.S.H., Nashville, Gosford Park, and lesser seen films like A Wedding, then you will no doubt be entertained by HealtH. Centered around a Health Convention where two women are running for President, HealtH contains many of Altman's latter 70s regulars like Paul Dooley (who helped write the film), Carol Burnett, and Henry Gibson, while also including top star Altman newcomers like Lauren Bacall, James Garner, and Glenda Jackson. Like a lot of Altman ensemble films there are numerous subplots in this film, but it is not nearly as overwhelming as films like Nashville or A Wedding, rather it has a more centered feel, perhaps like M.A.S.H. or Gosford Park. The whole thing is an obvious satire on the Health movement, filled with over-top, outlandish, contradictive characters, with guest stars like Dick Cavett providing a wry commentary on the whole thing. Underlining the whole election process is Altman's characteristic pessimism about politics and public appeal but what is most appealing about this film is the sheer fun most people seem to be having. This would be one of Altman's last films like this for a while!\")\n", + "(1, 'I saw the movie with two grown children. Although it was not as clever as Shrek, I thought it was rather good. In a movie theatre surrounded by children who were on spring break, there was not a sound so I know the children all liked it. There parents also seemed engaged. The death and apparent death of characters brought about the appropriate gasps and comments. Hopefully people realize this movie was made for kids. As such, it was successful although I liked it too. Personally I liked the Scrat!!')\n", + "(1, \"This show has been my escape from reality for the past ten years. I will sadly miss it. Although Atlantis has filled the hole a small bit.

The last ever episode of SG1(on television anyway)was beautifully done. Robert wrote something that felt close to reality. As though he was trying to explain what it was like on the set of the show. (Everyone working closely together for such a long time there are bound to up's and downs. But over the years they've turned into a family). I thought this was a wonderful way to end despite anyone else's criticisms.

SG1 was something special and time and time again it took me across thresholds of disbelief and amazement. The wonderful characters, stories, directors, writers. From episode one I was hooked. The blend of action, science, drama and especially comedy worked so well that made me keep wanting more.

There are no real words in which to completely express what this show meant to me. I can only thank those who kept the show so fresh and entertaining for so many years. It has inspired me to do many things that I thought was impossible.

I look forward to the movies next year and I really hope there will be a number of them. I never want the show to die.

Stargate SG1 - 1997 - 2007?\")\n", + "(1, 'For a long time it seemed like all the good Canadian actors had headed south of the border and (I guessed) all the second rank ones filled the top slots and that left the dregs for the sex comedies.

This film was a real surprise: despite the outlandish plots that are typical of farces, the actors seemed to be trying to put something into their characters and what we, the viewer, got back was almost true suspension of belief. When the extras from the music video attacked the evicting police, you almost believed it was possible.

If you are a fan of some of the better sex farces (Canadian or not) you should definitely seek this one out. And the big surprise, this sex farce is also loaded with some very good nudity.')\n", + "(1, 'Thank God this wasn\\'t based on a true story, because what a story it is. Populated by despicable characters whose depravity knows no bounds, Before The Devil is a mesmerizing, jaw-dropping excursion into perversion which would be laughable (and sometimes is, even with - or perhaps because of - the sickeningly tragic undercurrent of human dysfunction throughout) if it weren\\'t carried out with such magnificent, overwhelming conviction by its stars. The excellent script by Kelly Masterson and superb direction by none other than Sidney Lumet doesn\\'t hurt either.

The main dysfunction here is of a family nature, with the two majorly screwed up brothers (brilliant portrayals from Philip Seymour Hoffman and Ethan Hawke) deciding to rob their own parents\\' jewelry store, an attempt that goes pathetically awry.

The story is told with time-shifts (which are noted on screen, such as: \"Charlie: Two Days Before The Robbery\", so no one should be confused); some people have said they didn\\'t like this device but I thought it worked perfectly, adding to the skeweredness of the whole affair, considering that the two brothers in question are hardly playing with full decks - between them you couldn\\'t make a decent poker hand to save your life. Throw in these cheesy extra tidbits: one of the brothers is a drug addict, married to Gina (Marisa Tomei, also excellent), who is having an affair with the other brother, toss in some monumental sibling rivalry, along with the fact that said drug addict brother hates his father (a wrenching performance from Albert Finney), who has apparently caused him serious past pain, and you\\'ve got a Shakespearean/Greek tragedy on your hands. Proceed with caution.')\n" + ] + } + ], + "source": [ + "root_dir = tempdir.name + '/' + 'imdb_dataset'\n", + "train_iter = IMDBDataset(root_dir=root_dir, train=True) # For training data\n", + "test_iter = IMDBDataset(root_dir=root_dir, train=False) # For test data\n", + "\n", + "start=train_iter.pos_inx\n", + "for i in range(-10,10):\n", + " print(train_iter[start+i])" + ] + }, + { + "cell_type": "markdown", + "id": "1a6c647b-b8ab-49df-8434-becaa0dea775", + "metadata": {}, + "source": [ + "The following code defines the mapping of numeric labels to positive and negative reviews.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "3fca543a-ffc0-4079-bc30-7e3e05576623", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'positive review'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "imdb_label = {0: \" negative review\", 1: \"positive review\"}\n", + "imdb_label[1]" + ] + }, + { + "cell_type": "markdown", + "id": "ac4da5eb-a679-45d2-90bd-cfde475e0f8e", + "metadata": {}, + "source": [ + "The following code checks to ensure that there are exactly two classes in the train data set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "e78c80a0-0dee-4236-a382-dfe9f75a8fea", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "num_class = len(set([label for (label, text) in train_iter]))\n", + "num_class" + ] + }, + { + "cell_type": "markdown", + "id": "9c35ca39-77a1-40d9-9f16-fddb3259fd64", + "metadata": {}, + "source": [ + "The following code loads a basic English tokenizer and defines a function called ```yield_tokens``` that uses the tokenizer to break down text data yielded by an iterator into tokens.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "6a41ea50-7e8b-423b-9de8-36a88e97e96b", + "metadata": {}, + "outputs": [], + "source": [ + "tokenizer = get_tokenizer(\"basic_english\")\n", + "\n", + "def yield_tokens(data_iter):\n", + " \"\"\"Yield tokens for each data sample.\"\"\"\n", + " for _, text in data_iter:\n", + " yield tokenizer(text)" + ] + }, + { + "cell_type": "markdown", + "id": "e1853ae6-c596-4fac-a2f4-185c0a354510", + "metadata": {}, + "source": [ + " The following code loads a pretrained word embedding model called GloVe into a variable called `glove_embedding`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "0cbe5e7e-13d0-4d64-8c0c-86b74954c9b2", + "metadata": {}, + "outputs": [], + "source": [ + "# Note that GloVe embeddings are typically downloaded using:\n", + "#glove_embedding = GloVe(name=\"6B\", dim=100)\n", + "# However, the GloVe server is frequently down. The code below offers a workaround\n", + "\n", + "\n", + "class GloVe_override(Vectors):\n", + " url = {\n", + " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", + " }\n", + "\n", + " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", + " url = self.url[name]\n", + " name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", + " #name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", + " super(GloVe_override, self).__init__(name, url=url, **kwargs)\n", + "\n", + "class GloVe_override2(Vectors):\n", + " url = {\n", + " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", + " }\n", + "\n", + " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", + " url = self.url[name]\n", + " #name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", + " name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", + " super(GloVe_override2, self).__init__(name, url=url, **kwargs)\n", + "\n", + "try:\n", + " glove_embedding = GloVe_override(name=\"6B\", dim=100)\n", + "except:\n", + " try:\n", + " glove_embedding = GloVe_override2(name=\"6B\", dim=100)\n", + " except:\n", + " glove_embedding = GloVe(name=\"6B\", dim=100)" + ] + }, + { + "cell_type": "markdown", + "id": "e81eda58-67d1-4414-a91a-c238a1434729", + "metadata": {}, + "source": [ + "The following code builds a vocabulary object from a pretrained GloVe word embedding model and sets the default index to the token.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "5a3dbfda-7d7f-4256-937c-af441df1ee1b", + "metadata": {}, + "outputs": [], + "source": [ + "from torchtext.vocab import GloVe,vocab\n", + "# Build vocab from glove_vectors\n", + "vocab = vocab(glove_embedding .stoi, 0,specials=('', ''))\n", + "vocab.set_default_index(vocab[\"\"])" + ] + }, + { + "cell_type": "markdown", + "id": "e1fc78f6-0d04-4b35-afdd-a6193c66166e", + "metadata": {}, + "source": [ + "Let's count the number of words in the vocab:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "bc7de385-2867-4836-a6dc-f87a9f0c38f5", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "400002" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vocab_size=len(vocab)\n", + "vocab_size" + ] + }, + { + "cell_type": "markdown", + "id": "aa4eca48-cfb4-4fb4-8ca3-180319852e0d", + "metadata": {}, + "source": [ + "Let's test the ```vocab``` function:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "59689af6-a6ed-49eb-a12e-51303c1eb9da", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[20]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vocab(['he'])" + ] + }, + { + "cell_type": "markdown", + "id": "6ca4130b-f548-4b92-a68f-b2d83b6b19a5", + "metadata": {}, + "source": [ + "### Data set splits\n", + "\n", + "The following converts the data set into map-style data sets and then performs a random split to create separate training and validation data sets. The training data set will contain 95% of the samples in the original training set, while the validation data set will contain the remaining 5%. These data sets can be used for training and evaluating a machine learning model for text classification on the IMDB data set. The final performance of the model will be evaluated on the hold-out test set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "be0403dd-b319-406e-96ea-d215c0a42f6c", + "metadata": {}, + "outputs": [], + "source": [ + "# Convert the training and testing iterators to map-style datasets.\n", + "train_dataset = to_map_style_dataset(train_iter)\n", + "test_dataset = to_map_style_dataset(test_iter)\n", + "\n", + "# Determine the number of samples to be used for training and validation (5% for validation).\n", + "num_train = int(len(train_dataset) * 0.95)\n", + "\n", + "# Randomly split the training dataset into training and validation datasets using `random_split`.\n", + "# The training dataset will contain 95% of the samples, and the validation dataset will contain the remaining 5%.\n", + "split_train_, split_valid_ = random_split(train_dataset, [num_train, len(train_dataset) - num_train])" + ] + }, + { + "cell_type": "markdown", + "id": "82b6143c-d493-49b1-a255-b5aae8f9280a", + "metadata": {}, + "source": [ + "Be aware that the Skills Network currently does not offer GPU access to learners. As a result, training on the full data set could be time-consuming. To address this, you further reduce the size of the training set. This approach helps you mimic the training process as if a GPU were available. However, if you want to train using the full IMDB data set, you must either comment out or remove the two lines in the following code block.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "0b86c021-06f7-463d-95d7-c83f45de8605", + "metadata": {}, + "outputs": [], + "source": [ + "num_train = int(len(train_dataset) * 0.05)\n", + "split_train_, _ = random_split(split_train_, [num_train, len(split_train_) - num_train])" + ] + }, + { + "cell_type": "markdown", + "id": "e2e0359b-b92d-4aae-9916-aaadbd206ddb", + "metadata": {}, + "source": [ + "The following code checks to see if a CUDA-compatible GPU is available in the system using PyTorch, a popular deep learning framework. If a GPU is available, it assigns the device variable to \"cuda\" (which stands for CUDA, the parallel computing platform and application programming interface model developed by NVIDIA). If a GPU is not available, it assigns the device variable to \"cpu\" (which means the code will run on the CPU instead).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "3e37cbbd-1c52-4b02-b26c-c7f4b694e944", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "device(type='cuda')" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "device" + ] + }, + { + "cell_type": "markdown", + "id": "9c0631e5-5eb9-4e91-a84f-315381189cb9", + "metadata": {}, + "source": [ + "### Data loader\n", + "\n", + "The following code prepares the text processing pipeline with the tokenizer and vocabulary. The text pipeline is used to process the raw data strings from the data set iterators.\n", + "\n", + "The function **```text_pipeline```** first tokenizes the input text, then **```vocab```** is applied to get the token indices.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "6b1807bd-4548-435d-b763-cc1641c95c45", + "metadata": {}, + "outputs": [], + "source": [ + "def text_pipeline(x):\n", + " return vocab(tokenizer(x))" + ] + }, + { + "cell_type": "markdown", + "id": "58976c04-a071-4f9f-8e55-b0207f39a7d1", + "metadata": {}, + "source": [ + "In PyTorch, the **`collate_fn`** function is used in conjunction with data loaders to customize the way batches are created from individual samples. The provided code defines a `collate_batch` function in PyTorch, which is used with data loaders to customize batch creation from individual samples. It processes a batch of data, including labels and text sequences. It applies the `text_pipeline` function to preprocess the text. The processed data is then converted into PyTorch tensors and returned as a tuple containing the label tensor, text tensor, and offsets tensor representing the starting positions of each text sequence in the combined tensor. The function also ensures that the returned tensors are moved to the specified device (for example, GPU) for efficient computation.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "442be80e-ae39-4001-98f7-0a0c5266bb65", + "metadata": {}, + "outputs": [], + "source": [ + "from torch.nn.utils.rnn import pad_sequence\n", + "\n", + "def collate_batch(batch):\n", + " label_list, text_list = [], []\n", + " for _label, _text in batch:\n", + "\n", + " label_list.append(_label)\n", + " text_list.append(torch.tensor(text_pipeline(_text), dtype=torch.int64))\n", + "\n", + " label_list = torch.tensor(label_list, dtype=torch.int64)\n", + " text_list = pad_sequence(text_list, batch_first=True)\n", + "\n", + " return label_list.to(device), text_list.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "fb6f5466-b4b7-4772-9f09-9836249d4279", + "metadata": {}, + "source": [ + "You can convert the data set objects to data loaders by applying the `collate` function.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "95d54253-4781-4607-9f60-f4727117a520", + "metadata": {}, + "outputs": [], + "source": [ + "BATCH_SIZE = 32\n", + "\n", + "train_dataloader = DataLoader(\n", + " split_train_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")\n", + "valid_dataloader = DataLoader(\n", + " split_valid_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")\n", + "test_dataloader = DataLoader(\n", + " test_dataset, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3cc82919-3aa8-4f02-9be6-696a71335672", + "metadata": {}, + "source": [ + "Let's check to see what these data loaders generate.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "0e6a5049-2b5e-4661-ab9a-30b6072da1bb", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(tensor([0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0,\n", + " 1, 0, 1, 1, 0, 1, 0, 0], device='cuda:0'),\n", + " tensor([[ 0, 3542, 0, ..., 0, 0, 0],\n", + " [ 40, 663, 838, ..., 0, 0, 0],\n", + " [ 39, 16, 2, ..., 0, 0, 0],\n", + " ...,\n", + " [16307, 0, 59, ..., 0, 0, 0],\n", + " [ 9, 12389, 1608, ..., 0, 0, 0],\n", + " [ 193, 44724, 144, ..., 0, 0, 0]], device='cuda:0'))" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "label,seqence=next(iter(valid_dataloader))\n", + "label,seqence" + ] + }, + { + "cell_type": "markdown", + "id": "1f0d739f-47a7-44e6-b61d-6ce1eecda895", + "metadata": {}, + "source": [ + "### Neural network\n", + "\n", + "This code defines a class called Net that represents a text classifier based on a PyTorch TransformerEncoder.\n", + "The constructor takes the following arguments:\n", + "\n", + "- `num_class`: The number of classes to classify\n", + "- `vocab_size`: The size of the vocabulary\n", + "- `freeze`: Whether to freeze the embedding layer\n", + "- `nhead`: The number of heads in the transformer encoder\n", + "- `dim_feedforward`: The dimension of the feedforward layer in the transformer encoder\n", + "- `num_layers`: The number of transformer encoder layers\n", + "- `dropout`: The dropout rate\n", + "- `activation`: The activation function to use in the transformer encoder\n", + "- `classifier_dropout`: The dropout rate for the classifier\n", + "\n", + "**Attributes:**\n", + "\n", + "- `emb`: An embedding layer that maps each word in the vocabulary to a dense vector representation\n", + "- `pos_encoder`: A positional encoding layer that adds positional information to the word vectors\n", + "- `transformer_encoder`: A transformer encoder layer that processes the sequence of word vectors and extracts high-level features\n", + "- `classifier`: A linear layer that maps the output of the transformer encoder to the desired number of classes\n", + "\n", + "---\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "7f290742-5806-47d8-8533-f3d92709eba9", + "metadata": {}, + "outputs": [], + "source": [ + "class Net(nn.Module):\n", + " \"\"\"\n", + " Text classifier based on a pytorch TransformerEncoder.\n", + " \"\"\"\n", + " def __init__(\n", + "\n", + " self,\n", + " num_class,vocab_size,\n", + " freeze=True,\n", + " nhead=2,\n", + " dim_feedforward=128,\n", + " num_layers=2,\n", + " dropout=0.1,\n", + " activation=\"relu\",\n", + " classifier_dropout=0.1):\n", + "\n", + " super().__init__()\n", + "\n", + " #self.emb = embedding=nn.Embedding.from_pretrained(glove_embedding.vectors,freeze=freeze)\n", + " self.emb = nn.Embedding.from_pretrained(glove_embedding.vectors,freeze=freeze)\n", + " embedding_dim = self.emb.embedding_dim\n", + "\n", + "\n", + " self.pos_encoder = PositionalEncoding(\n", + " d_model=embedding_dim,\n", + " dropout=dropout,\n", + " vocab_size=vocab_size,\n", + " )\n", + "\n", + " encoder_layer = nn.TransformerEncoderLayer(\n", + " d_model=embedding_dim,\n", + " nhead=nhead,\n", + " dim_feedforward=dim_feedforward,\n", + " dropout=dropout,\n", + " )\n", + " self.transformer_encoder = nn.TransformerEncoder(\n", + " encoder_layer,\n", + " num_layers=num_layers,\n", + " )\n", + " self.classifier = nn.Linear(embedding_dim, num_class)\n", + " self.d_model = embedding_dim\n", + "\n", + " def forward(self, x):\n", + " x = self.emb(x) * math.sqrt(self.d_model)\n", + " x = self.pos_encoder(x)\n", + " x = self.transformer_encoder(x)\n", + " x = x.mean(dim=1)\n", + " x = self.classifier(x)\n", + "\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "id": "68664b81-817c-4633-bc71-9b32f95ee7f3", + "metadata": {}, + "source": [ + "The model can then be trained on labeled data from the IMDB data set with two classes.\n", + "Let's create the model.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "545ebebe-2206-47ce-81ab-9f168a80e4bd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Net(\n", + " (emb): Embedding(400000, 100)\n", + " (pos_encoder): PositionalEncoding(\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (transformer_encoder): TransformerEncoder(\n", + " (layers): ModuleList(\n", + " (0-1): 2 x TransformerEncoderLayer(\n", + " (self_attn): MultiheadAttention(\n", + " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", + " )\n", + " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", + " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (dropout1): Dropout(p=0.1, inplace=False)\n", + " (dropout2): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "model = Net(num_class=2,vocab_size=vocab_size).to(device)\n", + "model" + ] + }, + { + "cell_type": "markdown", + "id": "18ece646-3872-41e1-bedb-be1c7d8a37cc", + "metadata": {}, + "source": [ + "The following **`predict`** function takes in a text, a text pipeline, and a model as inputs. It uses a pretrained model passed as a parameter to predict the label of the text for text classification on the IMDB data set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "4cde3566-d2df-40b8-8fd1-944018d4b0ab", + "metadata": {}, + "outputs": [], + "source": [ + "def predict(text, text_pipeline, model):\n", + " with torch.no_grad():\n", + " text = torch.unsqueeze(torch.tensor(text_pipeline(text)),0).to(device)\n", + " model.to(device)\n", + " output = model(text)\n", + " return imdb_label[output.argmax(1).item()]" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "c7388103-5814-402d-a7de-ede861dc0927", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "' negative review'" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "predict(\"I like sports and stuff\", text_pipeline, model)" + ] + }, + { + "cell_type": "markdown", + "id": "40449174-d493-4521-8417-7950dafb0072", + "metadata": {}, + "source": [ + "You can create a function to evaluate the model's accuracy on a data set. Here, you define two nearly identical evaluation functions, one that provides a `tqdm` progress bar, and one that does not.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "1171efb1-cf5c-4a25-a0e0-91feb88e8ab7", + "metadata": {}, + "outputs": [], + "source": [ + "def evaluate(dataloader, model_eval):\n", + " model_eval.eval()\n", + " total_acc, total_count= 0, 0\n", + "\n", + " with torch.no_grad():\n", + " for label, text in tqdm(dataloader):\n", + " label, text = label.to(device), text.to(device)\n", + " output = model_eval(text)\n", + " predicted = torch.max(output.data, 1)[1]\n", + " total_acc += (predicted == label).sum().item()\n", + " total_count += label.size(0)\n", + " return total_acc / total_count" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "571400ae-320b-4d1a-a103-91592812d1c1", + "metadata": {}, + "outputs": [], + "source": [ + "def evaluate_no_tqdm(dataloader, model_eval):\n", + " model_eval.eval()\n", + " total_acc, total_count= 0, 0\n", + "\n", + " with torch.no_grad():\n", + " for label, text in dataloader:\n", + " label, text = label.to(device), text.to(device)\n", + " output = model_eval(text)\n", + " predicted = torch.max(output.data, 1)[1]\n", + " total_acc += (predicted == label).sum().item()\n", + " total_count += label.size(0)\n", + " return total_acc / total_count" + ] + }, + { + "cell_type": "markdown", + "id": "0092f33b-ac39-41d2-8525-443d2759b55e", + "metadata": {}, + "source": [ + "The following code evaluates the performance of your model. Note that this can take approximately 4 minutes on a CPU. **For efficiency, let's not run this cell now, but trust us that the performance of the untrained model is no better than average. If you wish to confirm yourself of this fact, you are free to uncomment this cell**:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "6b48b713-72a9-4267-b32e-dfab588c659c", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████████████████████████████████████| 782/782 [00:09<00:00, 82.88it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.5" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader, model)" + ] + }, + { + "cell_type": "markdown", + "id": "b855a1d8-5e52-4fa2-a507-d7383b2ea73d", + "metadata": {}, + "source": [ + "Note that the current performance of the model is no better than average. This outcome is expected, considering that the model has not undergone any training yet.\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "8a28c3ce-3828-4b70-8442-ae9b7757805c", + "metadata": {}, + "source": [ + "# Training\n", + "\n", + "The following coe defines the training function used to train your model.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "376d4c30-7f50-4c97-9fa2-667f3c1a47eb", + "metadata": {}, + "outputs": [], + "source": [ + "def train_model(model, optimizer, criterion, train_dataloader, valid_dataloader, epochs=1000, save_dir=\"\", file_name=None):\n", + " cum_loss_list = []\n", + " acc_epoch = []\n", + " acc_old = 0\n", + " model_path = os.path.join(save_dir, file_name)\n", + " acc_dir = os.path.join(save_dir, os.path.splitext(file_name)[0] + \"_acc\")\n", + " loss_dir = os.path.join(save_dir, os.path.splitext(file_name)[0] + \"_loss\")\n", + " time_start = time.time()\n", + "\n", + " for epoch in tqdm(range(1, epochs + 1)):\n", + " model.train()\n", + " #print(model)\n", + " #for parm in model.parameters():\n", + " # print(parm.requires_grad)\n", + " \n", + " cum_loss = 0\n", + " for idx, (label, text) in enumerate(train_dataloader):\n", + " optimizer.zero_grad()\n", + " label, text = label.to(device), text.to(device)\n", + "\n", + " predicted_label = model(text)\n", + " loss = criterion(predicted_label, label)\n", + " loss.backward()\n", + " #print(loss)\n", + " torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)\n", + " optimizer.step()\n", + " cum_loss += loss.item()\n", + " print(f\"Epoch {epoch}/{epochs} - Loss: {cum_loss}\")\n", + "\n", + " cum_loss_list.append(cum_loss)\n", + " accu_val = evaluate_no_tqdm(valid_dataloader,model)\n", + " acc_epoch.append(accu_val)\n", + "\n", + " if model_path and accu_val > acc_old:\n", + " print(accu_val)\n", + " acc_old = accu_val\n", + " if save_dir is not None:\n", + " #pass\n", + " print(\"save model epoch\",epoch)\n", + " torch.save(model.state_dict(), model_path)\n", + " save_list_to_file(lst=acc_epoch, filename=acc_dir)\n", + " save_list_to_file(lst=cum_loss_list, filename=loss_dir)\n", + "\n", + " time_end = time.time()\n", + " print(f\"Training time: {time_end - time_start}\")" + ] + }, + { + "cell_type": "markdown", + "id": "353d2098-4d5d-4a06-aa6a-c8866a6ecb0f", + "metadata": {}, + "source": [ + "### Train IMDB\n", + "\n", + "The following code sets the learning rate (LR) to 1, which determines the step size at which the optimizer updates the model's parameters during training. The CrossEntropyLoss criterion is used to calculate the loss between the model's predicted outputs and the ground truth labels. This loss function is commonly employed for multiclass classification tasks.\n", + "\n", + "The chosen optimizer is Stochastic Gradient Descent (SGD), which optimizes the model's parameters based on the computed gradients with respect to the loss function. The SGD optimizer uses the specified learning rate to control the size of the weight updates.\n", + "\n", + "Additionally, a learning rate scheduler is defined using StepLR. This scheduler adjusts the learning rate during training, reducing it by a factor (gamma) of 0.1 after every epoch (step) to improve convergence and fine-tune the model's performance. These components together form the essential setup for training a neural network using the specified learning rate, loss criterion, optimizer, and learning rate scheduler.\n", + "\n", + "For the sake of time efficiency, **the following lines are commented out and the model is not actually trained**. If you would like to get a glimpse of what training would look like, uncomment the following code block to train the model for 2 epochs. If you were to train this model in a real-world scenario, you would likely increase the number of epochs to a larger figure, such as 100 or more. Given the reduced training set defined earlier, it takes approximately 2 minutes to complete 2 epochs of training.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "03ab8dc6-6dda-44fa-b86c-1ad67e651463", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/2 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/sybqacL5p1qeEO8d4xRZNg/model-IMDB%20dataset%20small2-acc')\n", + "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/eOt6woGoaOB565T0RLH5WA/model-IMDB%20dataset%20small2-loss')\n", + "acc_epoch = pickle.load(acc_urlopened)\n", + "cum_loss_list = pickle.load(loss_urlopened)\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "37fe982b-b1a7-4d31-a5fc-21481d97fa4e", + "metadata": {}, + "source": [ + "The following code loads your pretrained model and evaluates its performance on the test set. **For efficiency, let's not run the evaluation because it can take approximately 4 minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, you are free to uncomment the last line in the following code block.**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "ca827b34-bd27-413f-ba76-8fde3d73c570", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/q66IH6a7lglkZ4haM6hB1w/model-IMDB%20dataset%20small2.pth')\n", + "model_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", + "model_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "f7421582-5341-4554-bb43-93eaeec31de6", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████████████████████████████████████| 782/782 [00:10<00:00, 71.81it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.83208" + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader, model_)" + ] + }, + { + "cell_type": "markdown", + "id": "a91438ab-0a67-47b4-908c-2114cedb29e2", + "metadata": {}, + "source": [ + "As you can see, the pretrained model achieved an accuracy of approximately 83% on the test data.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d8f01fc5-871b-4978-a2dd-724efa504014", + "metadata": {}, + "source": [ + "### Fine-tune a model pretrained on the AG News data set\n", + "\n", + "Rather than training a model on the IMDB data set as you did earlier, you can fine-tune a model that has been pretrained on the AG News data set, which is a collection of news articles. The goal of the AG News data set is to categorize news articles into one of four categories: Sports, Business, Sci/tech, or World. You’ll start training a model from scratch on the AG News data set. To save time, you can do this in just one cell. Also, for efficiency, ** comment out the training bit**. If you want to train the model for 2 epochs on a smaller data set to demonstrate what the training process would look like, uncomment the part that says `### Uncomment to Train ###` before running the cell. Training for 2 epochs on the reduced data set can take approximately 3 minutes.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "4db8b115-13e1-4d42-99e0-fd09dc0527c6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Net(\n", + " (emb): Embedding(400000, 100)\n", + " (pos_encoder): PositionalEncoding(\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (transformer_encoder): TransformerEncoder(\n", + " (layers): ModuleList(\n", + " (0-1): 2 x TransformerEncoderLayer(\n", + " (self_attn): MultiheadAttention(\n", + " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", + " )\n", + " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", + " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (dropout1): Dropout(p=0.1, inplace=False)\n", + " (dropout2): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " (classifier): Linear(in_features=100, out_features=4, bias=True)\n", + ")" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_iter_ag_news = AG_NEWS(split=\"train\")\n", + "\n", + "num_class_ag_news = len(set([label for (label, text) in train_iter_ag_news ]))\n", + "num_class_ag_news\n", + "\n", + "# Split the dataset into training and testing iterators.\n", + "train_iter_ag_news, test_iter_ag_news = AG_NEWS()\n", + "\n", + "# Convert the training and testing iterators to map-style datasets.\n", + "train_dataset_ag_news = to_map_style_dataset(train_iter_ag_news)\n", + "test_dataset_ag_news = to_map_style_dataset(test_iter_ag_news)\n", + "\n", + "# Determine the number of samples to be used for training and validation (5% for validation).\n", + "num_train_ag_news = int(len(train_dataset_ag_news) * 0.95)\n", + "\n", + "# Randomly split the training dataset into training and validation datasets using `random_split`.\n", + "# The training dataset will contain 95% of the samples, and the validation dataset will contain the remaining 5%.\n", + "split_train_ag_news_, split_valid_ag_news_ = random_split(train_dataset_ag_news, [num_train_ag_news, len(train_dataset_ag_news) - num_train_ag_news])\n", + "\n", + "# Make the training set smaller to allow it to run fast as an example.\n", + "# IF YOU WANT TO TRAIN ON THE AG_NEWS DATASET, COMMENT OUT THE 2 LINEs BELOW.\n", + "# HOWEVER, NOTE THAT TRAINING WILL TAKE A LONG TIME\n", + "num_train_ag_news = int(len(train_dataset_ag_news) * 0.05)\n", + "split_train_ag_news_, _ = random_split(split_train_ag_news_, [num_train_ag_news, len(split_train_ag_news_) - num_train_ag_news])\n", + "\n", + "\n", + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "device\n", + "\n", + "def label_pipeline(x):\n", + " return int(x) - 1\n", + "\n", + "from torch.nn.utils.rnn import pad_sequence\n", + "\n", + "def collate_batch_ag_news(batch):\n", + " label_list, text_list = [], []\n", + " for _label, _text in batch:\n", + " label_list.append(label_pipeline(_label))\n", + " text_list.append(torch.tensor(text_pipeline(_text), dtype=torch.int64))\n", + "\n", + "\n", + " label_list = torch.tensor(label_list, dtype=torch.int64)\n", + " text_list = pad_sequence(text_list, batch_first=True)\n", + "\n", + "\n", + " return label_list.to(device), text_list.to(device)\n", + "\n", + "BATCH_SIZE = 32\n", + "\n", + "train_dataloader_ag_news = DataLoader(\n", + " split_train_ag_news_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", + ")\n", + "valid_dataloader_ag_news = DataLoader(\n", + " split_valid_ag_news_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", + ")\n", + "test_dataloader_ag_news = DataLoader(\n", + " test_dataset_ag_news, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch_ag_news\n", + ")\n", + "\n", + "\n", + "model_ag_news = Net(num_class=4,vocab_size=vocab_size).to(device)\n", + "model_ag_news.to(device)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "1ee6601d-69a0-4274-a237-38f981a6153b", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/2 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/bQk8mJu3Uct3I4JEsEtRnw/model-AG%20News%20small1-acc')\n", + "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/KNQkqJWWwY_XfbFBRFhZNA/model-AG%20News%20small1-loss')\n", + "acc_epoch = pickle.load(acc_urlopened)\n", + "cum_loss_list = pickle.load(loss_urlopened)\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "a17edde1-7268-468d-ad53-d1880848efc6", + "metadata": {}, + "source": [ + "The following code loads the pretrained model and evaluates its performance on the AG News test set. **For efficiency, let's not run the evaluation because it can take a few minutes. Instead, claim that the pretrained model works well on the AG News dataset. If you would like to confirm the result for yourself, feel free to uncomment the last line in the following code block.**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "e98831cf-bdb6-4e9c-9932-8ee7e6897450", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", + "model_ag_news_ = Net(vocab_size=vocab_size, num_class=4).to(device)\n", + "model_ag_news_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "346d1f7a-2331-4010-a979-ae71efb00fcc", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|████████████████████████████████████████| 238/238 [00:00<00:00, 323.29it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.9046052631578947" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader_ag_news, model_ag_news_)" + ] + }, + { + "cell_type": "markdown", + "id": "49ecfc70-585f-4fd5-a2d3-3eb558705103", + "metadata": {}, + "source": [ + "As you can see, the pretrained model worked extremely well on the AG News data set. However, can this model be fine-tuned to perform well on the IMDB data set as well? Let's find out! You can begin by loading the pretrained AG News model.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "92336349-f915-4471-a3d2-64c90ea31171", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", + "model_fine1 = Net(vocab_size=vocab_size, num_class=4).to(device)\n", + "model_fine1.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))\n" + ] + }, + { + "cell_type": "markdown", + "id": "80e739c4-0b6b-44fa-8d5d-456c52576dd6", + "metadata": {}, + "source": [ + "The IMDB dataset is a binary classification task with only two classes (positive and negative reviews). Therefore, the output layer of the AG NEWS model should be adjusted to have just two output neurons to reflect the binary nature of the IMDB dataset. This adjustment is essential for the model to accurately learn and predict the sentiment of movie reviews in the IMDB dataset.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "1190f1bf-a030-43b1-b146-5f27715ce6a4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Original final layer: Linear(in_features=100, out_features=4, bias=True)\n", + "Input dimention final layer: 100\n" + ] + } + ], + "source": [ + "model_fine1.classifier\n", + "in_features = model_fine1.classifier.in_features\n", + "print(\"Original final layer:\", model_fine1.classifier)\n", + "print(\"Input dimention final layer:\", in_features)" + ] + }, + { + "cell_type": "markdown", + "id": "682f6a52-d8d8-4ed7-9162-1bf74dee5542", + "metadata": {}, + "source": [ + "You can change the final layer into a two-class problem.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "46da3876-5b30-4174-bcf1-2702836f3d6d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Net(\n", + " (emb): Embedding(400000, 100)\n", + " (pos_encoder): PositionalEncoding(\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (transformer_encoder): TransformerEncoder(\n", + " (layers): ModuleList(\n", + " (0-1): 2 x TransformerEncoderLayer(\n", + " (self_attn): MultiheadAttention(\n", + " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", + " )\n", + " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", + " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (dropout1): Dropout(p=0.1, inplace=False)\n", + " (dropout2): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 48, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_fine1.classifier = nn.Linear(in_features, 2)\n", + "model_fine1.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "672f77ee-1134-4656-9e85-c158cb4a64ea", + "metadata": {}, + "source": [ + "The following code shows the layers that are frozen (`requires_grad == False`) and unfrozen (`requires_grad == True`) in the model. The unfrozen layers will have their weights updated during fine-tuning.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "9afe1e34-b1e3-42a8-b67d-5e99e761dbab", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "emb.weight requires_grad: False\n", + "transformer_encoder.layers.0.self_attn.in_proj_weight requires_grad: True\n", + "transformer_encoder.layers.0.self_attn.in_proj_bias requires_grad: True\n", + "transformer_encoder.layers.0.self_attn.out_proj.weight requires_grad: True\n", + "transformer_encoder.layers.0.self_attn.out_proj.bias requires_grad: True\n", + "transformer_encoder.layers.0.linear1.weight requires_grad: True\n", + "transformer_encoder.layers.0.linear1.bias requires_grad: True\n", + "transformer_encoder.layers.0.linear2.weight requires_grad: True\n", + "transformer_encoder.layers.0.linear2.bias requires_grad: True\n", + "transformer_encoder.layers.0.norm1.weight requires_grad: True\n", + "transformer_encoder.layers.0.norm1.bias requires_grad: True\n", + "transformer_encoder.layers.0.norm2.weight requires_grad: True\n", + "transformer_encoder.layers.0.norm2.bias requires_grad: True\n", + "transformer_encoder.layers.1.self_attn.in_proj_weight requires_grad: True\n", + "transformer_encoder.layers.1.self_attn.in_proj_bias requires_grad: True\n", + "transformer_encoder.layers.1.self_attn.out_proj.weight requires_grad: True\n", + "transformer_encoder.layers.1.self_attn.out_proj.bias requires_grad: True\n", + "transformer_encoder.layers.1.linear1.weight requires_grad: True\n", + "transformer_encoder.layers.1.linear1.bias requires_grad: True\n", + "transformer_encoder.layers.1.linear2.weight requires_grad: True\n", + "transformer_encoder.layers.1.linear2.bias requires_grad: True\n", + "transformer_encoder.layers.1.norm1.weight requires_grad: True\n", + "transformer_encoder.layers.1.norm1.bias requires_grad: True\n", + "transformer_encoder.layers.1.norm2.weight requires_grad: True\n", + "transformer_encoder.layers.1.norm2.bias requires_grad: True\n", + "classifier.weight requires_grad: True\n", + "classifier.bias requires_grad: True\n" + ] + } + ], + "source": [ + "for name, param in model_fine1.named_parameters():\n", + " print(f\"{name} requires_grad: {param.requires_grad}\")" + ] + }, + { + "cell_type": "markdown", + "id": "2f1597ed-a727-42d5-bee1-7aaabd6c7681", + "metadata": {}, + "source": [ + "The following code block simulates fine-tuning on the shortened training set for just 2 epochs. **For the sake of time efficiency, this code block has been commented out**. If you want to see what training looks like, uncomment the following code block, but remember that this code could take approximately 2 minutes to run.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "3d5c5154-4b9d-4360-83d5-fc21a6ba930b", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/2 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/3LEJw8BRgJJFGqlLxaETxA/model-fine1-acc')\n", + "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/-CT1h97vjv0TolY82Nw29g/model-fine1-loss')\n", + "acc_epoch = pickle.load(acc_urlopened)\n", + "cum_loss_list = pickle.load(loss_urlopened)\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "85866d1f-1a0d-487c-a426-9da326a06c1f", + "metadata": {}, + "source": [ + "The following line loads a prefine-tuned model that was trained for 100 epochs on the full IMDB training set and evaluates its performance on the IMDB test set. **For the sake of efficiency, let's not run the evaluation because it can take a few minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, feel free to uncomment the last line in the code block.**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "cda6a606-1a87-4846-9926-54edc577879a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/e0WOHKh5dnrbC2lGhpsMMw/model-fine1.pth')\n", + "model_fine1_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", + "model_fine1_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "id": "e75cfe66-a265-4e7a-8d5f-a7c68f17c6e3", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████████████████████████████████████| 782/782 [00:10<00:00, 77.06it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.8604" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader, model_fine1_)" + ] + }, + { + "cell_type": "markdown", + "id": "c9b127d6-b41d-4195-ac5a-3da2d8d70b33", + "metadata": {}, + "source": [ + "This model demonstrated notable improvement, exhibiting a remarkable achievement with an accuracy of 86% on the test data. This is higher than the 83% achieved by the model trained from scratch on the IMDB dataset. Although the training process was time-intensive (The fine-tuning was as time-intensive as training the model from scratch), the enhanced performance underscores the fine-tuned model's effectiveness and superiority over the model trained from scratch. Much of the computational effort was devoted to updating the transformer layers. To expedite the training process, one viable strategy is to focus on training the final layer only, which can significantly reduce the computational load but might compromise the model's accuracy.\n" + ] + }, + { + "cell_type": "markdown", + "id": "55331d63-1150-465b-b0bc-d13bbd24fb7c", + "metadata": {}, + "source": [ + "### Fine-tune the final layer only\n", + "\n", + "Fine-tuning the final output layer of a neural network is similar to fine-tuning the whole model. You can begin by loading the pretrained model that you would like to fine-tune. In this case, it is the same model pretrained on the AG News data set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "id": "c2ffcf34-695f-4b9d-b6c2-5529a74d568a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 59, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", + "model_fine2 = Net(vocab_size=vocab_size, num_class=4).to(device)\n", + "model_fine2.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "markdown", + "id": "5fa5ba41-8d73-4649-b459-cbc321ca26e2", + "metadata": {}, + "source": [ + "Now, the key difference. You iterate through all of the parameters in the `model_fine2` model and set the `requires_grad` attribute of each parameter to `False`. This effectively freezes all of the layers in the model, meaning that their weights are to be updated during training.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "000978e5-6244-4be2-8fd5-3e0c7ed1c619", + "metadata": {}, + "outputs": [], + "source": [ + "# Freeze all layers in the model\n", + "for param in model_fine2.parameters():\n", + " param.requires_grad = False" + ] + }, + { + "cell_type": "markdown", + "id": "54b17c25-96f4-43b9-b0a8-963085cf5638", + "metadata": {}, + "source": [ + "Replace the final layer to reflect the fact that you are solving a two-class problem. Note that the new layer will be unfrozen.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "f0c29db4-3051-4247-8459-b7adab12fa45", + "metadata": {}, + "outputs": [], + "source": [ + "dim=model_fine2.classifier.in_features" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "b79d7ed1-7a83-4140-ad93-b5e28cdab784", + "metadata": {}, + "outputs": [], + "source": [ + "model_fine2.classifier = nn.Linear(dim, 2)" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "9387fde1-0a71-45b5-89cf-a2c90ff663d1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Net(\n", + " (emb): Embedding(400000, 100)\n", + " (pos_encoder): PositionalEncoding(\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (transformer_encoder): TransformerEncoder(\n", + " (layers): ModuleList(\n", + " (0-1): 2 x TransformerEncoderLayer(\n", + " (self_attn): MultiheadAttention(\n", + " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", + " )\n", + " (linear1): Linear(in_features=100, out_features=128, bias=True)\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (linear2): Linear(in_features=128, out_features=100, bias=True)\n", + " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (dropout1): Dropout(p=0.1, inplace=False)\n", + " (dropout2): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_fine2.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "42123afb-6a78-4f98-87dd-e87c2cfa7c4f", + "metadata": {}, + "source": [ + "The following block simulates fine-tuning on the shortened training set for just 2 epochs. **For the sake of time efficiency, this code block has been commented out**. The following code should take a shorter amount of time to train than the full fine-tuning conducted previously because only the final layer is unfrozen.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "3114fa2b-90ee-4955-8459-e01c1db83f2d", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/2 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/UdR3ApQnxSeV2mrA0CbiLg/model-fine2-acc')\n", + "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/rWGDIF-uL2dEngWcIo9teQ/model-fine2-loss')\n", + "acc_epoch = pickle.load(acc_urlopened)\n", + "cum_loss_list = pickle.load(loss_urlopened)\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "3043392c-ee76-4103-a758-83e90f594604", + "metadata": {}, + "source": [ + "The following line loads the pretrained model and evaluates its performance on the test set. **For efficiency, let's not run the evaluation because it can take a few minutes to run. Instead, report the result underneath the cell. If you would like to confirm the result for yourself, feel free to uncomment the last line in the following code block.**\n" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "id": "14e51cf2-4e3f-4adf-8c58-f65da0e74f65", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 68, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/B-1H6lpDg-A0zRwpB6Ek2g/model-fine2.pth')\n", + "model_fine2_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", + "model_fine2_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "id": "d7cf2dfa-3705-4f3e-a114-b4c56368cc77", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████████████████████████████████████| 782/782 [00:09<00:00, 85.13it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.64144" + ] + }, + "execution_count": 69, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader, model_fine2_)" + ] + }, + { + "cell_type": "markdown", + "id": "dbd9b301-88ba-42d4-b8ca-6e00379fb3b4", + "metadata": {}, + "source": [ + "The previous code indicates that although fine-tuning the final layer takes a significantly smaller amount of time than fine-tuning the whole model, the performance of the model with just the last layer unfrozen is significantly worse (64% accuracy) than the fine-tuned model with all layers unfrozen (86% accuracy).\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "d23b9fe8-1655-4c4c-8ee2-f78db934d5f2", + "metadata": {}, + "source": [ + "# Adapters\n", + "FeatureAdapter is a neural network module that introduces a low-dimensional bottleneck in a transformer architecture to allow fine-tuning with fewer parameters. It compresses the original high-dimensional embeddings into a lower dimension, applies a non-linear transformation, and then expands it back to the original dimension. This process is followed by a residual\n", + "connection that adds the transformed output back to the original input to preserve information and\n", + "promote gradient flow.\n", + "\n", + "## Benefits of using adapters in neural networks\n", + "\n", + "- **Efficient fine-tuning**: Adapters allow for targeted updates to specific parts of the model, reducing the need to retrain large sections of the network.\n", + "\n", + "- **Parameter efficiency**: By adding only a few parameters, adapters make it feasible to modify large models without substantial computational overhead.\n", + "\n", + "- **Preservation of pretrained features**: Adapters enable the modification of a model while retaining the valuable features learned during extensive pretraining.\n", + "\n", + "- **Modularity and flexibility**: They enhance the modularity of models, allowing easy adaptation to various tasks without altering core architecture.\n", + "\n", + "- **Task-specific adaptation**: Adapters can be tailored to improve performance on particular tasks, optimizing the model’s effectiveness.\n", + "\n", + "- **Transfer learning and domain adaptation**: They facilitate the adaptation of models to new domains, bridging gaps between different data distributions.\n", + "\n", + "- **Continual learning**: Adapters support the model's ability to learn new information continuously without forgetting previous knowledge.\n", + "\n", + "- **Reduced risk of overfitting**: With fewer trainable parameters, adapters help prevent overfitting, especially on smaller data sets.\n", + "\n", + "The following code shows an adapter model.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "id": "c46c86b1-eb04-4976-9fa7-c3fc06339bae", + "metadata": {}, + "outputs": [], + "source": [ + "class FeatureAdapter(nn.Module):\n", + " \"\"\"\n", + " Attributes:\n", + " size (int): The bottleneck dimension to which the embeddings are temporarily reduced.\n", + " model_dim (int): The original dimension of the embeddings or features in the transformer model.\n", + " \"\"\"\n", + " def __init__(self, bottleneck_size=50, model_dim=100):\n", + " super().__init__()\n", + " self.bottleneck_transform = nn.Sequential(\n", + " nn.Linear(model_dim, bottleneck_size), # Down-project to a smaller dimension\n", + " nn.ReLU(), # Apply non-linearity\n", + " nn.Linear(bottleneck_size, model_dim) # Up-project back to the original dimension\n", + " )\n", + "\n", + " def forward(self, x):\n", + " \"\"\"\n", + " Forward pass of the FeatureAdapter. Applies the bottleneck transformation to the input\n", + " tensor and adds a skip connection.\n", + "\n", + " Args:\n", + " x (Tensor): Input tensor with shape (batch_size, seq_length, model_dim).\n", + "\n", + " Returns:\n", + " Tensor: Output tensor after applying the adapter transformation and skip connection,\n", + " maintaining the original input shape.\n", + " \"\"\"\n", + " transformed_features = self.bottleneck_transform(x) # Transform features through the bottleneck\n", + " output_with_residual = transformed_features + x # Add the residual connection\n", + " return output_with_residual" + ] + }, + { + "cell_type": "markdown", + "id": "a5c12e9a-5bf1-4a4f-b792-25a264253d28", + "metadata": {}, + "source": [ + "The adapted class wraps this adapter functionality around any specified linear layer, enhancing its output with the non-linearity of a ReLU activation function. This setup is particularly useful for experimenting with subtle architectural modifications in deep learning models, facilitating fine-tuning and potentially improving model performance on complex tasks.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "id": "97d59ae0-610e-49bb-a9c4-1020c2dc468d", + "metadata": {}, + "outputs": [], + "source": [ + "class Adapted(nn.Module):\n", + " def __init__(self, linear,bottleneck_size=None):\n", + " super(Adapted, self).__init__()\n", + " self.linear = linear\n", + " model_dim = linear.out_features\n", + " if bottleneck_size is None:\n", + " bottleneck_size = model_dim//2 # Define default bottleneck size as half the model_dim\n", + "\n", + " # Initialize FeatureAdapter with calculated bottleneck_size and model_dim\n", + " self.adaptor = FeatureAdapter(bottleneck_size=bottleneck_size, model_dim=model_dim)\n", + "\n", + " def forward(self, x):\n", + " # First, the input x is passed through the linear layer\n", + " x=self.linear(x)\n", + " # Then it's adapted using FeatureAdapter\n", + " x= self.adaptor(x)\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "id": "d8cfb80d-eb8c-4342-8066-33134c5b456d", + "metadata": {}, + "source": [ + "You load the pretrained transformer model that was trained on the AG News dataset.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "id": "01070ca9-5037-470e-b8f7-ceddfc2f279b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 74, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/9c3Dh2O_jsYBShBuchUNlg/model-AG%20News%20small1.pth')\n", + "model_adapters = Net(vocab_size=vocab_size, num_class=4).to(device)\n", + "model_adapters.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "markdown", + "id": "e1df412e-51c5-4e8c-9dbe-c6d38f11a0cf", + "metadata": {}, + "source": [ + "\n", + "First, freeze the parameters of a model named model_adapters to prevent them from being updated during training. Then, retrieve the number of input features for the classifier, and replace the classifier with a new linear layer that outputs to two classes.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "1820db89-b3b0-4cb2-b42f-1826ba1ec6d0", + "metadata": {}, + "outputs": [], + "source": [ + "for param in model_adapters.parameters():\n", + " param.requires_grad = False\n", + "\n", + "dim= model_adapters.classifier.in_features\n", + "\n", + "model_adapters.classifier = nn.Linear(dim, 2)" + ] + }, + { + "cell_type": "markdown", + "id": "905ad98d-2f62-475a-a7e0-a2f7119b5081", + "metadata": {}, + "source": [ + "Let's explore how to apply the adapted object to a linear layer to obtain the first output. You can obtain the unadapted linear layer for the first output by:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "cbdcd100-7fda-45ee-a823-9233b7e68ca6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Linear(in_features=100, out_features=128, bias=True)\n" + ] + } + ], + "source": [ + "my_example_layer=model_adapters.transformer_encoder.layers[0].linear1\n", + "print(my_example_layer)" + ] + }, + { + "cell_type": "markdown", + "id": "1cf2017f-bb26-4b9c-86ea-f32cc5ce4450", + "metadata": {}, + "source": [ + "In the following code, you copy the linear layer and add an adapter layer to it.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "83fd78c1-111d-45f6-9adf-3ab5c7264a8b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=64, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=64, out_features=128, bias=True)\n", + " )\n", + " )\n", + ")\n" + ] + } + ], + "source": [ + "my_adapeted_layer=Adapted(my_example_layer)\n", + "print(my_adapeted_layer)" + ] + }, + { + "cell_type": "markdown", + "id": "50e35b98-bcbd-4aec-b039-464e87ddd6fb", + "metadata": {}, + "source": [ + "You can print the adapted layer and show that the new layers have their requires_grad attribute set to True, indicating that these layers will be updated during training.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "d23c9515-4303-437d-8f02-99a3723e8b89", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "False\n", + "False\n", + "True\n", + "True\n", + "True\n", + "True\n" + ] + } + ], + "source": [ + "for parm in my_adapeted_layer.parameters():\n", + " print(parm.requires_grad)" + ] + }, + { + "cell_type": "markdown", + "id": "1a6bc1b1-f1bb-4c01-99ee-c2e16e5090c2", + "metadata": {}, + "source": [ + "You can set a layer in the model to the adapter layer, as shown in the following code in the commented-out line. However, because there are many layers, a more systematic approach would be to traverse the model and replace specific layers with the adapter layer. Note that you should set the bottleneck size to 24, ensuring that there are fewer parameters to train than during a full fine-tuning.\n" + ] + }, { "cell_type": "code", - "execution_count": null, - "id": "94133067-93b7-4652-b497-040866a7b78c", + "execution_count": 79, + "id": "f49d7836-5487-4cb4-a177-797f4eb75cee", "metadata": {}, "outputs": [], - "source": [] + "source": [ + "# Adapt a specific layer\n", + "#model_adapters.transformer_encoder.layers[0].linear1=Adapted(my_example_layer)" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "bc5a4706-ef6f-4d1d-9b44-4e6e3cfba925", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Find number of layers\n", + "N_layers=len(model_adapters.transformer_encoder.layers)\n", + "N_layers" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "6af0d8e7-f38c-47a6-9ba4-7a74994e7f33", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " before linear1\n", + "Linear(in_features=100, out_features=128, bias=True)\n", + " after linear1\n", + "Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=128, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear2\n", + "Linear(in_features=128, out_features=100, bias=True)\n", + " after linear2\n", + "Adapted(\n", + " (linear): Linear(in_features=128, out_features=100, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=100, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=100, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear1\n", + "Linear(in_features=100, out_features=128, bias=True)\n", + " after linear1\n", + "Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=128, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear2\n", + "Linear(in_features=128, out_features=100, bias=True)\n", + " after linear2\n", + "Adapted(\n", + " (linear): Linear(in_features=128, out_features=100, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=100, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=100, bias=True)\n", + " )\n", + " )\n", + ")\n" + ] + } + ], + "source": [ + "# Traverse model and adapt\n", + "for n in range(N_layers):\n", + " encoder=model_adapters.transformer_encoder.layers[n]\n", + " if encoder.linear1:\n", + " print(\" before linear1\")\n", + " print(encoder.linear1)\n", + " model_adapters.transformer_encoder.layers[n].linear1=Adapted(encoder.linear1, bottleneck_size=24)\n", + " print(\" after linear1\")\n", + " print(model_adapters.transformer_encoder.layers[n].linear1)\n", + "\n", + " if encoder.linear2:\n", + " print(\" before linear2\")\n", + " print(model_adapters.transformer_encoder.layers[n].linear2)\n", + " model_adapters.transformer_encoder.layers[n].linear2=Adapted(encoder.linear2, bottleneck_size=24)\n", + " print(\" after linear2\")\n", + " print(model_adapters.transformer_encoder.layers[n].linear2)" + ] + }, + { + "cell_type": "markdown", + "id": "1ea3e84d-649a-462d-8c21-ce1d4a915ce2", + "metadata": {}, + "source": [ + "The following code sends the model to the device.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "id": "67b7450d-b1f4-42c6-9995-b13da7a8a7d7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Net(\n", + " (emb): Embedding(400000, 100)\n", + " (pos_encoder): PositionalEncoding(\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " )\n", + " (transformer_encoder): TransformerEncoder(\n", + " (layers): ModuleList(\n", + " (0-1): 2 x TransformerEncoderLayer(\n", + " (self_attn): MultiheadAttention(\n", + " (out_proj): NonDynamicallyQuantizableLinear(in_features=100, out_features=100, bias=True)\n", + " )\n", + " (linear1): Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=128, bias=True)\n", + " )\n", + " )\n", + " )\n", + " (dropout): Dropout(p=0.1, inplace=False)\n", + " (linear2): Adapted(\n", + " (linear): Linear(in_features=128, out_features=100, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=100, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=100, bias=True)\n", + " )\n", + " )\n", + " )\n", + " (norm1): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (norm2): LayerNorm((100,), eps=1e-05, elementwise_affine=True)\n", + " (dropout1): Dropout(p=0.1, inplace=False)\n", + " (dropout2): Dropout(p=0.1, inplace=False)\n", + " )\n", + " )\n", + " )\n", + " (classifier): Linear(in_features=100, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 84, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Send model to device\n", + "model_adapters.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "d233586b-c7ef-49e6-b684-b8d24997db80", + "metadata": {}, + "source": [ + "Finally, the following code simulates training of the adapted model by training on a shortend IMDB train set for 2 epochs.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "f275d5f0-ee19-469b-a957-98bf72d3f3c3", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/2 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "acc_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/D49zrrMPWO_ktwQo7PSHIQ/model-adapters-acc')\n", + "loss_urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/RXWlmyaco695RiaoU7QsnA/model-adapters-loss')\n", + "acc_epoch = pickle.load(acc_urlopened)\n", + "cum_loss_list = pickle.load(loss_urlopened)\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "f7e5392a-549b-4b68-968a-8c8d70dbf05d", + "metadata": {}, + "source": [ + "The following code loads the adapted model fine-tuned for 100 epochs on the full IMDB train set and evaluates its performance on the IMDB test set.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "id": "5caaa3f7-5c89-4f27-8ef0-3d3d29f79b66", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " before linear1\n", + "Linear(in_features=100, out_features=128, bias=True)\n", + " after linear1\n", + "Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=128, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear2\n", + "Linear(in_features=128, out_features=100, bias=True)\n", + " after linear2\n", + "Adapted(\n", + " (linear): Linear(in_features=128, out_features=100, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=100, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=100, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear1\n", + "Linear(in_features=100, out_features=128, bias=True)\n", + " after linear1\n", + "Adapted(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=128, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=128, bias=True)\n", + " )\n", + " )\n", + ")\n", + " before linear2\n", + "Linear(in_features=128, out_features=100, bias=True)\n", + " after linear2\n", + "Adapted(\n", + " (linear): Linear(in_features=128, out_features=100, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=100, out_features=24, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=24, out_features=100, bias=True)\n", + " )\n", + " )\n", + ")\n" + ] + } + ], + "source": [ + "model_adapters_ = Net(vocab_size=vocab_size, num_class=2).to(device)\n", + "for n in range(N_layers):\n", + " encoder=model_adapters_.transformer_encoder.layers[n]\n", + " if encoder.linear1:\n", + " print(\" before linear1\")\n", + " print(encoder.linear1)\n", + " model_adapters_.transformer_encoder.layers[n].linear1=Adapted(encoder.linear1, bottleneck_size=24)\n", + " print(\" after linear1\")\n", + " print(model_adapters_.transformer_encoder.layers[n].linear1)\n", + "\n", + " if encoder.linear2:\n", + " print(\" before linear2\")\n", + " print(model_adapters_.transformer_encoder.layers[n].linear2)\n", + " model_adapters_.transformer_encoder.layers[n].linear2=Adapted(encoder.linear2, bottleneck_size=24)\n", + " print(\" after linear2\")\n", + " print(model_adapters_.transformer_encoder.layers[n].linear2)\n", + "\n", + "model_adapters_.to(device)\n", + "for param in model_adapters_.parameters():\n", + " param.requires_grad = False\n" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "id": "d582b1d3-0fcc-473e-82de-58f9ab1443ef", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/PGhd5G_NVrWNH-_jdjwNlw/model-adapters.pth')\n", + "model_adapters_.load_state_dict(torch.load(io.BytesIO(urlopened.read()), map_location=device))" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "df69e146-1a1f-4267-9372-9834fcf6616d", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████████████████████████████████████| 782/782 [00:11<00:00, 66.29it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "0.85608" + ] + }, + "execution_count": 91, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader, model_adapters_)" + ] + }, + { + "cell_type": "markdown", + "id": "08262362-924a-42d0-b928-0cf6ffe5331c", + "metadata": {}, + "source": [ + "As you can see, the performance of the fine-tuned adapted model is nearly identical to the fully fine-tuned model, with both models achieving a roughly 86% accuracy. This is an especially surprising result because a significantly smaller number of weights were updated for the adapted model than the fully fine-tuned model. Note that only the adapter layers with a bottleneck size of 24 and the final classifier layer are unfrozen.\n", + "\n", + "The above shows that adapters can be used for parameter efficient fine-tuning (PEFT) and that the performance of a model fine-tuned using adapters can be almost as good as a fully fine-tuned model with all of the layers unfrozen!\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "3b255d65-8f64-4c95-8fd6-ac69d1a67a5a", + "metadata": {}, + "source": [ + "## Exercise: Adapt linear layers in a different network\n", + "\n", + "The following code defines a neural network called `NeuralNetwork`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "id": "82730ce9-d9a9-483f-8d00-e0cbd78de764", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NeuralNetwork(\n", + " (flatten): Flatten(start_dim=1, end_dim=-1)\n", + " (linear_relu_stack): Sequential(\n", + " (0): Linear(in_features=784, out_features=512, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=512, out_features=512, bias=True)\n", + " (3): ReLU()\n", + " (4): Linear(in_features=512, out_features=10, bias=True)\n", + " )\n", + ")\n" + ] + } + ], + "source": [ + "class NeuralNetwork(nn.Module):\n", + " def __init__(self):\n", + " super().__init__()\n", + " self.flatten = nn.Flatten()\n", + " self.linear_relu_stack = nn.Sequential(\n", + " nn.Linear(28*28, 512),\n", + " nn.ReLU(),\n", + " nn.Linear(512, 512),\n", + " nn.ReLU(),\n", + " nn.Linear(512, 10),\n", + " )\n", + "\n", + " def forward(self, x):\n", + " x = self.flatten(x)\n", + " logits = self.linear_relu_stack(x)\n", + " return logits\n", + "\n", + "exercise_model = NeuralNetwork()\n", + "\n", + "exercise_model.to(device)\n", + "for param in exercise_model.parameters():\n", + " param.requires_grad = False\n", + "\n", + "print(exercise_model)" + ] + }, + { + "cell_type": "markdown", + "id": "ed4e3b4b-a5df-4543-8d35-adac4ff4b093", + "metadata": {}, + "source": [ + "`NeuralNetwork` is a neural network that uses the `Sequential` container from PyTorch. Adapt the first two linear layers in the `Sequential` container by using the bottleneck adapter with a bottleneck size of 30. Also, change the last linear layer to a layer that has 5 outputs.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "id": "2fde3cfa-d77d-4ad3-82e1-a3f7eb1d0cf0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NeuralNetwork(\n", + " (flatten): Flatten(start_dim=1, end_dim=-1)\n", + " (linear_relu_stack): Sequential(\n", + " (0): Adapted(\n", + " (linear): Linear(in_features=784, out_features=512, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=512, out_features=30, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=30, out_features=512, bias=True)\n", + " )\n", + " )\n", + " )\n", + " (1): ReLU()\n", + " (2): Adapted(\n", + " (linear): Linear(in_features=512, out_features=512, bias=True)\n", + " (adaptor): FeatureAdapter(\n", + " (bottleneck_transform): Sequential(\n", + " (0): Linear(in_features=512, out_features=30, bias=True)\n", + " (1): ReLU()\n", + " (2): Linear(in_features=30, out_features=512, bias=True)\n", + " )\n", + " )\n", + " )\n", + " (3): ReLU()\n", + " (4): Linear(in_features=512, out_features=5, bias=True)\n", + " )\n", + ")\n" + ] + } + ], + "source": [ + "### REPLACE THIS YOUR ANSWER ###\n", + "exercise_model.linear_relu_stack[0] = Adapted(exercise_model.linear_relu_stack[0], bottleneck_size=30)\n", + "exercise_model.linear_relu_stack[2] = Adapted(exercise_model.linear_relu_stack[2], bottleneck_size=30)\n", + "exercise_model.linear_relu_stack[4] = nn.Linear(512, 5)\n", + "print(exercise_model)" + ] + }, + { + "cell_type": "markdown", + "id": "f8244ee4-45b9-4713-8480-6aadf7a66b43", + "metadata": {}, + "source": [ + "
\n", + " Click here for the solution\n", + "\n", + "```python\n", + "exercise_model.linear_relu_stack[0] = Adapted(exercise_model.linear_relu_stack[0], bottleneck_size=30)\n", + "exercise_model.linear_relu_stack[2] = Adapted(exercise_model.linear_relu_stack[2], bottleneck_size=30)\n", + "exercise_model.linear_relu_stack[4] = nn.Linear(512, 5)\n", + "print(exercise_model)\n", + "```\n", + "\n", + "
\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "151060fe-37c4-4ee1-8034-539b0e3a657a", + "metadata": {}, + "source": [ + "## Congratulations! You have completed the lab\n", + "\n", + "## Authors\n", + "\n", + "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo)\n", + "\n", + "Joseph has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n", + "\n", + "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) \n", + "\n", + "Wojciech \"Victor\" Fulmyk is a Data Scientist at IBM, and a PhD Candidate in economics at the University of Calgary.\n", + "\n", + "[Ashutosh Sagar](https://www.linkedin.com/in/ashutoshsagar/) is completing his MS in CS from Dalhousie University. He has previous experience working with Natural Language Processing and as a Data Scientist.\n", + "\n", + "## References\n", + "\n", + "\n", + "[TEXT CLASSIFICATION WITH THE TORCHTEXT LIBRARY](https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html)\n", + "\n", + "[Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751.pdf)\n", + "\n", + "[Simple, Scalable Adaptation for Neural Machine Translation](https://arxiv.org/pdf/1909.08478)\n", + "\n", + "© Copyright IBM Corporation. All rights reserved.\n" + ] } ], "metadata": { diff --git a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/Instruction fine-tuning-v1.ipynb b/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/Instruction fine-tuning-v1.ipynb deleted file mode 100755 index 3c3120d..0000000 --- a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/Instruction fine-tuning-v1.ipynb +++ /dev/null @@ -1,1370 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "

\n", - " \n", - " \"Skills\n", - " \n", - "

\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Instruction-Tuning with LLMs\n", - "\n", - "\n", - "Instruction-based fine-tuning, referred to as instruction GPT. It trains the language models to follow specific instructions and generate appropriate responses. For instruction-tuning, the dataset plays an important role as it provides structured examples of instructions, contexts, and responses, allowing the model to learn how to handle various tasks effectively. Instruction GPT often uses human feedback to refine and improve model performance; however, this lab doesn't cover this aspect.\n", - "\n", - "The context and instruction are concatenated to form a single input sequence that the model can understand and use to generate the correct response.\n", - "\n", - "#### Context and instruction\n", - "\n", - "\t•\tInstruction: A command to specify what the model should do\n", - "\t•\tContext: Additional information or background required for performing the instruction\n", - "\t•\tCombined input: The instruction and context combine together into a single input sequence\n", - " \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's review certain examples for various templates:\n", - "\n", - "---\n", - "#### Response template\n", - "Template: `### Question: {question}\\n ### Answer: {answer}`\n", - "\n", - "Example:\n", - "```\n", - "### Question: What is the capital of France?\n", - "### Answer: Paris\n", - "```\n", - "\n", - "---\n", - "#### Conversation template\n", - "\n", - "Template: `### User: {user_input}\\n ### Bot: {bot_response}`\n", - "Example:\n", - "```\n", - "### User: How are you today?\n", - "### Bot: I'm doing great, thank you! How can I assist you today?\n", - "```\n", - "\n", - "---\n", - "#### Instruction and output template\n", - "\n", - "Template: `### Instruction: {instruction}\\n ### Output: {output}`\n", - "\n", - "Example:\n", - "```\n", - "### Instruction: Translate the following sentence to Spanish: \"Hello, how are you?\"\n", - "### Output: \"Hola, ¿cómo estás?\"\n", - "```\n", - "\n", - "---\n", - "#### Completion template\n", - "\n", - "Template: `{prompt} ### Completion: {completion}`\n", - "Example:\n", - "```\n", - "Once upon a time in a faraway land, ### Completion: there lived a wise old owl who knew all the secrets of the forest.\n", - "```\n", - "\n", - "#### Summarization template\n", - "\n", - "Template: `### Text: {text}\\n ### Summary: {summary}`\n", - "\n", - "Example:\n", - "```\n", - "### Text: The quick brown fox jumps over the lazy dog.\n", - "### Summary: A fox jumps over a dog.\n", - "```\n", - "\n", - "---\n", - "#### Dialogue template\n", - "\n", - "Template: `### Speaker 1: {utterance_1}\\n ### Speaker 2: {utterance_2}\\n ### Speaker 1: {utterance_3}`\n", - "\n", - "Example:\n", - "```\n", - "### Speaker 1: Hi, what are you doing today?\n", - "### Speaker 2: I'm going to the park.\n", - "### Speaker 1: That sounds fun!\n", - "```\n", - "\n", - "---\n", - "#### Code generation template\n", - "\n", - "Template: `### Task: {task_description}\\n ### Code: {code_output}`\n", - "\n", - "Example:\n", - "```\n", - "### Task: Write a function to add two numbers in Python.\n", - "### Code: def add(a, b):\\n return a + b\n", - "```\n", - "\n", - "---\n", - "#### Data analysis template\n", - "\n", - "Template: `### Analysis Task: {task_description}\\n ### Analysis: {analysis_output}`\n", - "\n", - "Example:\n", - "```\n", - "### Analysis Task: Provide insights from the sales data of Q1 2022.\n", - "### Analysis: The sales increased by 15% compared to Q4 2021, with the highest growth in the electronics category.\n", - "```\n", - "\n", - "---\n", - "#### Recipe template\n", - "\n", - "Template: `### Recipe Name: {recipe_name}\\n ### Ingredients: {ingredients}\\n ### Instructions: {instructions}`\n", - "\n", - "Example:\n", - "```\n", - "### Recipe Name: Chocolate Chip Cookies\n", - "### Ingredients: Flour, Sugar, Chocolate Chips, Butter, Eggs, Vanilla Extract\n", - "### Instructions: Mix the dry ingredients, add the wet ingredients, fold in the chocolate chips, and bake at 350°F for 10-12 minutes.\n", - "```\n", - "\n", - "---\n", - "#### Explanation template\n", - "\n", - "Template: `### Concept: {concept}\\n ### Explanation: {explanation}`\n", - "\n", - "Example:\n", - "```\n", - "### Concept: Photosynthesis\n", - "### Explanation: Photosynthesis is the process by which green plants use sunlight to synthesize nutrients from carbon dioxide and water.\n", - "```\n", - "\n", - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Objectives\n", - "\n", - "After completing this lab, you will be able to:\n", - "\n", - " - Understand the various types of templates including instruction-response, question-answering, summarization, code generation, dialogue, data analysis, and explanation and their applications for fine-tuning large language models (LLMs).\n", - " - Create and apply different templates to fine-tune LLMs for various tasks.\n", - " - Format datasets based on the created templates to prepare them for effective model training\n", - " - Perform instruction fine-tuning using Hugging Face libraries and tools\n", - " - Apply Low-Rank Adaptation (LoRA) techniques to fine-tune LLMs efficiently\n", - " - Configure and use the SFTTrainer for supervised fine-tuning of instruction-following models\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The concepts presented in this lab would apply to the other template formats as well.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# __Table of contents__\n", - "\n", - "
    \n", - "
  1. \n", - " Setup\n", - "
      \n", - "
    1. Install required libraries
      2. \n", - "
    2. Import required libraries
      3. \n", - "
    3. Define the device
      4. \n", - "
    \n", - "
    2. \n", - "
  2. Dataset description
    3. \n", - "
  3. Model and tokenizer
    4. \n", - "
  4. Preprocessing the data
    5. \n", - "
  5. Test the base model
    6. \n", - "
      \n", - "
    1. BLEU score
      2. \n", - "
    \n", - "
  6. Perform instruction fine-tuning with LoRA
    7. \n", - "
  7. Exercises
    8. \n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Setup\n", - "\n", - "### Install required libraries\n", - "\n", - "For this lab, use the following libraries, which are __not__ preinstalled in the Skills Network Labs environment. You can install libraries by running the code in the below cell. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install -qq datasets==2.20.0 trl==0.9.6 transformers==4.42.3 peft==0.11.1 tqdm==4.66.4 numpy==1.26.4 pandas==2.2.2 matplotlib==3.9.1 seaborn==0.13.2 scikit-learn==1.5.1 sacrebleu==2.4.2 evaluate==0.4.2" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Import required libraries\n", - "\n", - "The following code imports the required libraries.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import warnings\n", - "warnings.filterwarnings('ignore')\n", - "warnings.simplefilter('ignore')\n", - "\n", - "from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline\n", - "from datasets import load_dataset\n", - "import torch\n", - "from torch.utils.data import Dataset\n", - "from tqdm import tqdm\n", - "import evaluate\n", - "from trl import SFTConfig, SFTTrainer, DataCollatorForCompletionOnlyLM\n", - "\n", - "from peft import get_peft_model, LoraConfig, TaskType\n", - "\n", - "import pickle\n", - "import json\n", - "import matplotlib.pyplot as plt \n", - "\n", - "from urllib.request import urlopen\n", - "import io" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Define the device\n", - "\n", - "The below code will set your device to 'cuda' if your device is compatible with GPU, otherwise, you can use 'cpu'.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Dataset description\n", - "\n", - "Use the below sentences to download the CodeAlpaca 20k dataset, a programming code dataset. This code is available [here](https://github.com/sahil280114/codealpaca?tab=readme-ov-file#data-release). The CodeAlpaca dataset contains the following elements:\n", - "\n", - "\n", - "- `instruction`: **str**, describes the task the model should perform. Each of the 20K instructions is unique.\n", - "- `input`: **str**, optional context or input for the task. For example, when the instruction is \"Amend the following SQL query to select distinct elements\", the input is the SQL query. Around 40% of the examples have an input.\n", - "- `output`: **str**, the answer to the instruction as generated by text-davinci-003.\n", - "\n", - "The following code block downloads the training split from the CodeAlpaca-20k dataset:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset = load_dataset(\"lucasmccabe-lmi/CodeAlpaca-20k\", split=\"train\")\n", - "dataset" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's look at the example in the dataset:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset[1000]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To keep things simple let's just focus on the examples that do not have any `input`:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset = dataset.filter(lambda example: example[\"input\"] == '')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The original CodeAlpaca dataset may not have been shuffled. The following line indicates how to shuffle a `datasets.arrow_dataset.Dataset()` object with a random seed:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset = dataset.shuffle(seed=42)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The CodeAlpaca 20k dataset has a training and test set. You can split the original training data into a train and test set by assigning 80% of the data to the training set and 20% to the testing set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset_split = dataset.train_test_split(test_size=0.2, seed=42)\n", - "train_dataset = dataset_split['train']\n", - "test_dataset = dataset_split['test']\n", - "dataset_split" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Select a small set of data for the resource limitation\n", - "# This dataset will be only used for evaluation parts, not for the training\n", - "tiny_test_dataset=test_dataset.select(range(10))\n", - "tiny_train_dataset=train_dataset.select(range(10))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Model and tokenizer\n", - "\n", - "In this exercise, let's fine-tune the [`opt-350m`](https://huggingface.co/facebook/opt-350m) model from Facebook. A description of this OpenSource model was published [here](https://arxiv.org/abs/2205.01068), and the model was originally made available on [metaseq's Github repository](https://github.com/facebookresearch/metaseq).\n", - "\n", - "The below lines load the base model from Hugging Face:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Base model\n", - "model = AutoModelForCausalLM.from_pretrained(\"facebook/opt-350m\").to(device)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This model comes with its own tokenizer which you will be loading here:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "tokenizer = AutoTokenizer.from_pretrained(\"facebook/opt-350m\", padding_side='left')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's find the end of sentence (EOS) token. This is a special tokenizer token. Once this token is encountered, the model will stop generating further tokens:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "tokenizer.eos_token" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Preprocessing the data\n", - "\n", - "To perform the fine-tuning, first, preprocess the data by creating functions that generate the prompt.\n", - "\n", - "The `formatting_prompts_func` function takes a dataset as input. For every element in the dataset format, the instruction and the output into a template using the format:\n", - "\n", - "```\n", - "### Instruction:\n", - "Translate the following sentence to Spanish: \"Hello, how are you?\"\n", - "\n", - "### Response:\n", - "\"Hola, ¿cómo estás?\"\n", - "```\n", - "\n", - "_**Note:**_ \n", - "1. The template provided in this section may differ from the **Instruction and output template** presented in the introduction of this lab. You can replace the `### Response:` with `### Output:` to generate similar results.\n", - "\n", - "2. Introducing the `` end of sentence token at the end of the text informs the model to stop generating text beyond this point.\n", - "\n", - "Finally, the `formatting_prompts_func_no_response` function behaves similarly to the `formatting_prompts_func` except the response is not included.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def formatting_prompts_func(mydataset):\n", - " output_texts = []\n", - " for i in range(len(mydataset['instruction'])):\n", - " text = (\n", - " f\"### Instruction:\\n{mydataset['instruction'][i]}\"\n", - " f\"\\n\\n### Response:\\n{mydataset['output'][i]}\"\n", - " )\n", - " output_texts.append(text)\n", - " return output_texts\n", - "\n", - "def formatting_prompts_func_no_response(mydataset):\n", - " output_texts = []\n", - " for i in range(len(mydataset['instruction'])):\n", - " text = (\n", - " f\"### Instruction:\\n{mydataset['instruction'][i]}\"\n", - " f\"\\n\\n### Response:\\n\"\n", - " )\n", - " output_texts.append(text)\n", - " return output_texts" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The following code block generates the `instructions` (the part of the prompt that does not include the response), the `instructions_with_responses` (the full prompt with the response and `eos` token), and the `expected_outputs`, which are the parts of the `instructions_with_responses` that are between the `instructions` and the `eos` token.\n", - "\n", - "To find the `expected_outputs`, tokenize `instructions` and the `instructions_with_responses`. Then, count the number of tokens in `instructions`, and discard the equivalent amount of tokens from the beginning of the tokenized `instructions_with_responses` vector. Finally, discard the final token in `instructions_with_responses`, corresponding to the `eos` token. Decode the resulting vector using the tokenizer, resulting in the `expected_output`:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "expected_outputs = []\n", - "instructions_with_responses = formatting_prompts_func(test_dataset)\n", - "instructions = formatting_prompts_func_no_response(test_dataset)\n", - "for i in tqdm(range(len(instructions_with_responses))):\n", - " tokenized_instruction_with_response = tokenizer(instructions_with_responses[i], return_tensors=\"pt\", max_length=1024, truncation=True, padding=False)\n", - " tokenized_instruction = tokenizer(instructions[i], return_tensors=\"pt\")\n", - " expected_output = tokenizer.decode(tokenized_instruction_with_response['input_ids'][0][len(tokenized_instruction['input_ids'][0])-1:], skip_special_tokens=True)\n", - " expected_outputs.append(expected_output)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's look at the example to view what `instructions` include, `instructions_with_responses`, and `expected_outputs`:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print('############## instructions ##############\\n' + instructions[0])\n", - "print('############## instructions_with_responses ##############\\n' + instructions_with_responses[0])\n", - "print('\\n############## expected_outputs ##############' + expected_outputs[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Instead of keeping the instructions as-is, it's beneficial to convert the `instructions` list into a `torch` `Dataset`. The following code defines a class called `ListDataset` that inherits from `Dataset` and creates a `torch` `Dataset` from a list. This class is then used to generate a `Dataset` object from `instructions`: \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "class ListDataset(Dataset):\n", - " def __init__(self, original_list):\n", - " self.original_list = original_list\n", - " \n", - " def __len__(self):\n", - " return len(self.original_list)\n", - " \n", - " def __getitem__(self, i):\n", - " return self.original_list[i]\n", - "\n", - "instructions_torch = ListDataset(instructions)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "instructions_torch[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Test the base model\n", - "\n", - "Let's understand how the base model performs without performing fine-tuning in the model. This may involve response generation from the base, that is from the non-fine-tuned mode. \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code defines a text generation pipeline using the `pipeline` class from `transformers`. This pipeline is useful to generate text given by a model and a tokenizer:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "gen_pipeline = pipeline(\"text-generation\",\n", - " model=model,\n", - " tokenizer=tokenizer,\n", - " device=device,\n", - " batch_size=2,\n", - " max_length=50,\n", - " truncation=True,\n", - " padding=False,\n", - " return_full_text=False)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**_Note:_** The generation pipeline can generate tokens or text. If```return_tensors=True```, the pipeline returns token IDs; otherwise, it returns words. Additionally, the generation pipeline generates both the instructions *and* the responses by default. However, to assess the model's performance, exclude the generated instructions and focus on the responses. To do this, set ```return_full_text=False```.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code leverages the pre-defined generation pipeline to generate outputs using the model. \n", - "\n", - "**_Note:_** The code is commented out because it may take a long time for CPU. Instead of generating the raw tokens here, you can load output from this model later.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "tokenizer.padding_side = 'left'\n", - "\n", - "with torch.no_grad():\n", - " # Due to resource limitation, only apply the function on 3 records using \"instructions_torch[:10]\"\n", - " pipeline_iterator= gen_pipeline(instructions_torch[:3], \n", - " max_length=50, # this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice\n", - " num_beams=5,\n", - " early_stopping=True,)\n", - "\n", - "generated_outputs_base = []\n", - "for text in pipeline_iterator:\n", - " generated_outputs_base.append(text[0][\"generated_text\"])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code loads the generated responses for the whole dataset using machine that has access to a fast CUDA-enabled GPU:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/VvQRrSqS1P0_GobqtL-SKA/instruction-tuning-generated-outputs-base.pkl')\n", - "generated_outputs_base = pickle.load(io.BytesIO(urlopened.read()))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's look at the sample responses generated by the base model and the expected responses from the dataset.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "for i in range(3):\n", - " print('@@@@@@@@@@@@@@@@@@@@')\n", - " print('@@@@@ Instruction '+ str(i+1) +': ')\n", - " print(instructions[i])\n", - " print('\\n\\n')\n", - " print('@@@@@ Expected response '+ str(i+1) +': ')\n", - " print(expected_outputs[i])\n", - " print('\\n\\n')\n", - " print('@@@@@ Generated response '+ str(i+1) +': ')\n", - " print(generated_outputs_base[i])\n", - " print('\\n\\n')\n", - " print('@@@@@@@@@@@@@@@@@@@@')\n", - " " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see that the responses generated by the base model are not up to the mark. Also, the responses have the tendency to extend and repeat the answers until they generate the maximum number of tokens. Later on, you can see that the instruction-tuning can fix both of these issues. First, the instruction fine-tuned model will be able to provide more meaningful responses. Second, because, you appended the `eos` token `<\\s>` to the output, you will teach the model via instruction fine-tuning to not generate responses without bound.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## BLEU score\n", - "\n", - "Let's set up a metric that compares the generated responses and the expected responses in the test environment. In this lab, let's use the [BLEU score](https://en.wikipedia.org/wiki/BLEU), a metric originally intended to check the quality of translations made by translation models. You can calculate the BLEU scores for individual generated segments by comparing them with a set of expected outputs and average the scores for the individual segments. Depending on the implementation, BLEU scores range from 0 to 1 or from 0 to 100 (as in the implementation used herein), with higher scores indicating a better match between the model generated output and the expected output.\n", - "\n", - "_**Note:**_ \n", - "1. The BLEU score was originally implemented for assessing the quality of translations. However, it may not necessarily be the best metric for instruction fine-tuning in general, but it is nonetheless a useful metric that gives a sense of the alignment between the model generated output and the expected output.\n", - "2. BLEU scores are very challenging to compare from one study to the next because it is a parametrized metric. As a result, you can employ a variant of BLEU called [SacreBLEU](https://aclanthology.org/W18-6319/) invariant to the metric's parametrization.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sacrebleu = evaluate.load(\"sacrebleu\")\n", - "results_base = sacrebleu.compute(predictions=generated_outputs_base,\n", - " references=expected_outputs)\n", - "\n", - "print(list(results_base.keys()))\n", - "print(round(results_base[\"score\"], 1))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The SacreBLEU score of 0.4/100 indicates that there is very little alignment between the base model's generated responses and the expected responses for the examples in the test dataset.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Perform instruction fine-tuning with LoRA\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To save time, let's perform instruction fine-tuning using a parameter-efficient fine-tuning (PEFT) method called low-rank adaptation (LoRA).\n", - "First, convert the model into a PEFT model suitable for LoRA fine-tuning by defining a `LoraConfig` object from the `peft` library that outlines LoRA parameters, such as the LoRA rank and the target modules. Next, apply LoRA configuration on the model using `get_peft_model()`, which effectively converts `model` into a LoRA `model`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "lora_config = LoraConfig(\n", - " r=16, # Low-rank dimension\n", - " lora_alpha=32, # Scaling factor\n", - " target_modules=[\"q_proj\", \"v_proj\"], # Modules to apply LoRA\n", - " lora_dropout=0.1, # Dropout rate\n", - " task_type=TaskType.CAUSAL_LM # Task type should be causal language model\n", - ")\n", - "\n", - "model = get_peft_model(model, lora_config)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Instruction fine-tuning using the `SFTTrainer` has the effect of generating the instructions *and* the responses. However, for the purposes of assessing the quality of the generated text, consider only the quality of the response and not the quality of the instruction. For the purposes of calculating the BLEU score, eliminate the length of tokens corresponding to the instruction from the beginning of the tokenized model output. \n", - "\n", - "For example, suppose the tokenized instruction had a length of ten, but the generated text had a length of fourteen. Then the tokenized response that was kept for the purposes of calculating the BLEU score was just the four tokens at the end of the tokenized generated text because the first ten tokens represent the model's generation of the tokenized instruction.\n", - "\n", - "Although eliminating the first few tokens of the tokenized output worked well for the purposes of calculating BLEU. However, during fine-tuning, the first few tokens won't have an impact on the loss function. You can mask those tokens using -100 by ignoring the value of PyTorch loss functions such as [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html). By masking the tokens corresponding to the instruction with -100, only the tokens associated with the response can bear the loss.\n", - "\n", - "You can create such a masking manually by defining your own function. However, it is easier to instead use the `DataCollatorForCompletionOnlyLM` class from `trl`:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "response_template = \"### Response:\\n\"\n", - "collator = DataCollatorForCompletionOnlyLM(response_template, tokenizer=tokenizer)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, pass the `collator`, `DataCollatorForCompletionOnlyLM` object to the data collator into `SFTTrainer`, resulting in the generated instructions without bearing on the loss.\n", - "\n", - "To perform the training, first configure our `SFTTrainer`, and create the `SFTTrainer` object by passing to the `collator`:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "training_args = SFTConfig(\n", - " output_dir=\"/tmp\",\n", - " num_train_epochs=10,\n", - " save_strategy=\"epoch\",\n", - " fp16=True,\n", - " per_device_train_batch_size=2, # Reduce batch size\n", - " per_device_eval_batch_size=2, # Reduce batch size\n", - " max_seq_length=1024,\n", - " do_eval=True\n", - ")\n", - "\n", - "trainer = SFTTrainer(\n", - " model,\n", - " train_dataset=train_dataset,\n", - " eval_dataset=test_dataset,\n", - " formatting_func=formatting_prompts_func,\n", - " args=training_args,\n", - " packing=False,\n", - " data_collator=collator,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please ignore the above warning.\n", - "The below comments, runs the trainer, because this would take a long time on the CPU. Therefore, let's not run the trainer here.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#trainer.train()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to train the trainer, the `trainer` object would have a state history for every training step. You would be able to access this state history using the below commented out line:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#log_history_lora = trainer.state.log_history" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Instead of extracting the state history above, let's load the state history of a model that was instruction fine-tuned to the above specifications on a GPU.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/49I70jQD0-RNRg2v-eOoxg/instruction-tuning-log-history-lora.json')\n", - "log_history_lora = json.load(io.BytesIO(urlopened.read()))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can plot the training loss for each training step.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "train_loss = [log[\"loss\"] for log in log_history_lora if \"loss\" in log]\n", - "\n", - "# Plot the training loss\n", - "plt.figure(figsize=(10, 5))\n", - "plt.plot(train_loss, label='Training Loss')\n", - "\n", - "plt.xlabel('Steps')\n", - "plt.ylabel('Loss')\n", - "plt.title('Training Loss')\n", - "plt.legend()\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to fine-tune the model, the fine-tuned model could be saved using the below commented out line:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#trainer.save_model(\"./instruction_tuning_final_model_lora\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's redefine the text generation pipeline because the model has been changed to the LoRA model. Ignore the warning for the `PeftModelForCausalLM` not being supported for `text-generation`. However, if the PEFT model is supported, the warning is erroneous.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "gen_pipeline = pipeline(\"text-generation\", \n", - " model=model, \n", - " tokenizer=tokenizer, \n", - " device=device, \n", - " batch_size=2, \n", - " max_length=50, \n", - " truncation=True, \n", - " padding=False,\n", - " return_full_text=False)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code generates tokens with the pipeline using the instruction fine-tuned model. Only three records of data are used for demonstration because generating text is time consuming on CPU:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "with torch.no_grad():\n", - " # Due to resource limitation, only apply the function on 3 records using \"instructions_torch[:10]\"\n", - " pipeline_iterator= gen_pipeline(instructions_torch[:3],\n", - " max_length=50, # this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice\n", - " num_beams=5,\n", - " early_stopping=True,)\n", - "generated_outputs_lora = []\n", - "for text in pipeline_iterator:\n", - " generated_outputs_lora.append(text[0][\"generated_text\"])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "generated_outputs_lora[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can load the generated texts for the entire dataset from the fine-tuned LoRA model and run on GPU.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/o7uYxe15xvX4CN-6Lr10iA/instruction-tuning-generated-outputs-lora.pkl')\n", - "generated_outputs_lora = pickle.load(io.BytesIO(urlopened.read()))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's have a look at some of the responses from the instruction fine-tuned model and the expected responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "for i in range(3):\n", - " print('@@@@@@@@@@@@@@@@@@@@')\n", - " print('@@@@@ Instruction '+ str(i+1) +': ')\n", - " print(instructions[i])\n", - " print('\\n\\n')\n", - " print('@@@@@ Expected response '+ str(i+1) +': ')\n", - " print(expected_outputs[i])\n", - " print('\\n\\n')\n", - " print('@@@@@ Generated response '+ str(i+1) +': ')\n", - " print(generated_outputs_lora[i])\n", - " print('\\n\\n')\n", - " print('@@@@@@@@@@@@@@@@@@@@')\n", - " " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Compared to the base model, you can see that the responses are much better. Additionally, the responses don't extend until the maximum number of tokens are generated.\n", - "\n", - "To confirm the responses generated by the instruction fine-tuned model align better with the expected output, let's calculate the SacreBLEU score:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sacrebleu = evaluate.load(\"sacrebleu\")\n", - "results_lora = sacrebleu.compute(predictions=generated_outputs_lora,\n", - " references=expected_outputs)\n", - "print(list(results_lora.keys()))\n", - "print(round(results_lora[\"score\"], 1))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see that the fine-tuned model achieves a SacreBLEU score of 14.7/100, significantly better than the 0.4/100 achieved by the base model. \n", - "\n", - "Let's conclude. The instruction fine-tuned model generates responses that align much better with the expected responses in the dataset.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Exercises\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Exercise 1: Try with another response template (Question-Answering)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Create a `formatting_prompts_response_template` function to format the train_dataset in the Response Template. \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Template: `### Question: {question}\\n ### Answer: {answer}`\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "def formatting_prompts_response_template(mydataset):\n", - " output_texts = []\n", - " for i in range(len(mydataset['instruction'])):\n", - " text = (\n", - " f\"### Question:\\n{mydataset['instruction'][i]}\"\n", - " f\"\\n\\n### Answer:\\n{mydataset['output'][i]}\"\n", - " )\n", - " output_texts.append(text)\n", - " return output_texts\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Create a `formatting_prompts_response_template_no_response` function to format the `test_dataset` in the Response Template, excluding the response.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Template: `### Question: {question}\\n ### Answer: `\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "def formatting_prompts_response_template_no_response(mydataset):\n", - " output_texts = []\n", - " for i in range(len(mydataset['instruction'])):\n", - " text = (\n", - " f\"### Question:\\n{mydataset['instruction'][i]}\"\n", - " f\"\\n\\n### Answer:\\n\"\n", - " )\n", - " output_texts.append(text)\n", - " return output_texts\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Exercise 2: Try with another LLM (EleutherAI/gpt-neo-125m)\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The EleutherAI/gpt-neo-125m is a smaller variant of the GPT-Neo family of models developed by EleutherAI. With 125 million parameters, it is designed to be computationally efficient while still providing robust performance for various natural language processing tasks.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Download and load the `EleutherAI/gpt-neo-125m` model\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "model_name = \"EleutherAI/gpt-neo-125m\"\n", - "\n", - "# Load the model and tokenizer\n", - "model = AutoModelForCausalLM.from_pretrained(model_name)\n", - "tokenizer = AutoTokenizer.from_pretrained(model_name)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Initialize LoRA Configuration:\n", - "\n", - "- r: 8 (Low-rank dimension)\n", - "- lora_alpha: 16 (Scaling factor)\n", - "- target_modules: [\"q_proj\", \"v_proj\"] (Modules to apply LoRA)\n", - "- lora_dropout: 0.1 (Dropout rate)\n", - "- task_type: TaskType.CAUSAL_LM (Task type should be causal language model)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "\n", - "lora_config = LoraConfig(\n", - " r=8, # Low-rank dimension\n", - " lora_alpha=16, # Scaling factor\n", - " target_modules=[\"q_proj\", \"v_proj\"], # Modules to apply LoRA\n", - " lora_dropout=0.1, # Dropout rate\n", - " task_type=TaskType.CAUSAL_LM # Task type should be causal language model\n", - ")\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Apply LoRA Configuration to the model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for the solution\n", - "\n", - "```python\n", - "model = get_peft_model(model, lora_config)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Congratulations! You have completed the lab\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Authors\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) is a Data Scientist and a PhD Candidate in Economics at the University of Calgary.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Fateme Akbari](https://www.linkedin.com/in/fatemeakbari/) is a Ph.D. candidate in Information Systems at McMaster University with demonstrated research experience in Machine Learning and NLP.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## References\n", - "\n", - "[Supervised Fine-tuning Trainer](https://huggingface.co/docs/trl/main/en/sft_trainer)\n", - "\n", - "[Finetuning To Follow Instructions](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb)\n", - "\n", - "[Finetuning with LoRA -- A Hands-On Example](https://lightning.ai/lightning-ai/studios/code-lora-from-scratch)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```{## Change Log|Date (YYYY-MM-DD)|Version|Changed By|Change Description||-|-|-|-||2024-07-18|1.0|Wojciech \"Victor\" Fulmyk|Lab Written||2024-07-25|2.0|Fateme Akbari|Bugs Fixed||2024-07-31|3.0|Bhavika Chhatbar|ID reviewed|}\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "© Copyright IBM Corporation. All rights reserved.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.19" - }, - "prev_pub_hash": "280a2cf79e2287085899526a711a657e3abe91f52fd641be6356c1ef9f2bafbd" - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/RewardTrainer-v1.ipynb b/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/RewardTrainer-v1.ipynb deleted file mode 100755 index 73a5bb1..0000000 --- a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Instruction-Tuning and Reward Modeling/RewardTrainer-v1.ipynb +++ /dev/null @@ -1,1111 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "

\n", - " \n", - " \"Skills\n", - " \n", - "

\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Reward Modeling\n", - "Imagine that you're working as a machine learning engineer for a large technology company that wants to integrate advanced language models into its suite of AI-powered products. Your task is to evaluate and select the best large language model (LLM) that can understand and follow complex instructions, improve the quality of automated customer service, and generate high-quality responses.\n", - "\n", - "However, simply choosing a powerful LLM isn't enough. To truly excel in these tasks, the model should be fine-tuned to align with specific goals and criteria. This is where reward models come into the picture. By training a reward model, you can guide the LLM to prioritize certain behaviors, ensuring it generates responses that not only meet technical standards but also align with the company's values and objectives.\n", - "\n", - "In this hands-on lab, you dive into the process of creating and training a reward model by using the transformer reinforcement learner (trl) library from Hugging Face. You learn how to set up the environment, define the rewards that shape the model's behavior, and fine-tune the LLM to perform with precision. This project equips you with the skills to implement reward models in real-world applications, enhancing the effectiveness and quality of AI-powered products.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## __Table of Contents__\n", - "\n", - "
    \n", - "
  1. Objectives
    2. \n", - " \n", - "
  2. Setup
    3. \n", - " \n", - "
  3. Data set
    4. \n", - " \n", - "
  4. Model and tokenizer setup
    5. \n", - "
  5. Preprocessing
    6. \n", - " \n", - "
  6. Evaluating the model
    7. \n", - "
  7. Exercise
    8. \n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Objectives\n", - "After completing this lab, you are able to:\n", - "\n", - "- Understand the concept of reward modeling in machine learning\n", - "- Explore and preprocess a data set for reward modeling tasks\n", - "- Set up and configure a GPT-2 model for sequence classification\n", - "- Tokenize and prepare text data for model training\n", - "- Evaluate model performance using pairwise comparison of responses\n", - "- Apply preprocessing and evaluation techniques to different subsets of data\n", - "- Understand important concepts related to transformers and reward modeling\n", - "- Implement special tokens in the tokenizer and configure the model accordingly\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Setup\n", - "## Installing required libraries\n", - "Before you start, make sure that you have all of the necessary libraries installed. You can run the following commands to install them:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install datasets\n", - "!pip install trl\n", - "!pip install datasets huggingface_hub\n", - "!pip install transformers\n", - "!pip install peft\n", - "!pip install nltk rouge_score\n", - "!pip install --upgrade transformers\n", - "!pip install --upgrade peft\n", - "!pip install bitsandbytes==0.43.1\n", - "!pip install matplotlib" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Importing required libraries\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import json\n", - "from datasets import load_dataset, DatasetDict\n", - "import torch\n", - "from transformers import GPT2Tokenizer, GPT2ForSequenceClassification, TrainingArguments\n", - "from peft import LoraConfig, TaskType\n", - "from transformers import TrainingArguments\n", - "from trl import RewardTrainer\n", - "import matplotlib.pyplot as plt\n", - "import warnings\n", - "\n", - "warnings.filterwarnings('ignore')\n", - "# Disable warnings for a cleaner notebook or console experience\n", - "def warn(*args, **kwargs):\n", - " pass\n", - "warnings.warn = warn" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Defining helper functions\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def save_to_json(data, file_path):\n", - " \"\"\"\n", - " Save a dictionary to a JSON file.\n", - "\n", - " Args:\n", - " data (dict): The dictionary to save.\n", - " file_path (str): The path to the JSON file.\n", - " \"\"\"\n", - " with open(file_path, 'w') as json_file:\n", - " json.dump(data, json_file, indent=4)\n", - " print(f\"Data successfully saved to {file_path}\")\n", - " \n", - " \n", - "def load_from_json(file_path):\n", - " \"\"\"\n", - " Load data from a JSON file.\n", - "\n", - " Args:\n", - " file_path (str): The path to the JSON file.\n", - "\n", - " Returns:\n", - " dict: The data loaded from the JSON file.\n", - " \"\"\"\n", - " with open(file_path, 'r') as json_file:\n", - " data = json.load(json_file)\n", - " return data " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "## Data set\n", - "\n", - "In this section, you load a data set that is used for training the reward model. In this lab, you use the Dahoas/synthetic-instruct-gptj-pairwise data set from Hugging Face, a synthetic data set that is designed for training and evaluating instruction-following models. This data set includes pairs of prompts and responses, where one response is preferred over the other. The primary use case is to train models to distinguish between better and worse responses, essential for tasks like reinforcement learning with human feedback (RLHF).\n", - "\n", - "### Purpose\n", - "\n", - "This data set helps train models for better understanding and following instructions by learning from pairs of good and bad responses. This is particularly useful for improving the quality of generated responses in dialogue systems and other AI applications that require understanding and generating natural language instructions.\n", - "\n", - "### Applications\n", - "- **Reinforcement learning**: Enhancing models to prefer better responses based on feedback\n", - "- **Fine-tuning language models**: Improving the performance of models on instruction-following tasks\n", - "- **Evaluation**: Assessing the ability of models to distinguish between high- and low-quality responses\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Load the Dahoas/synthetic-instruct-gptj-pairwise dataset \n", - "dataset = load_dataset(\"Dahoas/synthetic-instruct-gptj-pairwise\")\n", - "# Display the dataset\n", - "print(dataset)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Data set features\n", - "\n", - "To get a better understanding of the data set, let's inspect a few samples. First, print out the `prompt`, `chosen`, and `rejected` responses for the first 10 examples in the training set. This gives an insight into the type of data on which you are working and how it is structured.\n", - "\n", - "```Prompt:``` A text prompt that the model should respond to\n", - "\n", - "```Chosen:``` The preferred response to the prompt\n", - "\n", - "```Rejected:``` The less preferred response to the prompt\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "for i in range(10): \n", - " print('prompt')\n", - " print(dataset[\"train\"][i]['prompt'],'\\n')\n", - " \n", - " print('chosen')\n", - " print(dataset[ 'train'][i]['chosen'],'\\n')\n", - "\n", - " print('rejected')\n", - " print(dataset[ 'train'][i]['rejected'],'\\n')\n", - " print('---------------------------\\n')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Model and tokenizer setup\n", - "In this section, you set up the tokenizer and the model for training. You can use the GPT-2 model for sequence classification, which helps in determining the quality of responses.\n", - "\n", - "Next, specify the model name or path as \"gpt2\". To initialize the tokenizer and model, use `GPT2Tokenizer.from_pretrained` and `GPT2ForSequenceClassification.from_pretrained`, respectively, with `num_labels` set to 1 for ranking (a numerical score value). To handle padding, set the `pad_token` of the tokenizer to be the same as the `eos_token` (end-of-sequence token). Similarly, configure the model to use the `eos_token_id` as the `pad_token_id`. This setup ensures that the tokenizer and model are correctly initialized and prepared for sequence classification tasks with GPT-2.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define the model name or path\n", - "model_name_or_path = \"gpt2\"\n", - "\n", - "# Initialize tokenizer and model\n", - "tokenizer = GPT2Tokenizer.from_pretrained(model_name_or_path, use_fast=True)\n", - "model = GPT2ForSequenceClassification.from_pretrained(model_name_or_path, num_labels=1)\n", - "\n", - "# Add special tokens if necessary\n", - "tokenizer.pad_token = tokenizer.eos_token\n", - "model.config.pad_token_id = model.config.eos_token_id\n", - "\n", - "# Define the maximum length\n", - "max_length = 1024" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, preprocess the data set for training. Then combine the prompt with the chosen and rejected responses into a format suitable for input into the model. This process helps create clear input-output pairs for the model to learn from.\n", - "\n", - "`Lambda Function`: Define a lambda function `get_res` that takes the data set and a response type (chosen or rejected) and combines the prompt with the respective response. Each entry is formatted as a dialogue between \"Human\" and \"Assistant\".\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "get_res=lambda dataset,res:[ \"\\n\\nHuman: \"+prompt + \"\\n\\nAssistant: \"+resp for prompt, resp in zip(dataset[\"train\"][\"prompt\"], dataset[\"train\"][res])]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Chosen Samples`: Apply the `get_res` function to create a list of chosen samples.\n", - "\n", - "`Rejected Samples`: Similarly, create a list of rejected samples using the same function.\n", - "\n", - "After applying the function, you get the following results.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "chosen_samples=get_res( dataset,'chosen')\n", - "rejected_samples=get_res( dataset,'rejected')\n", - "print('chosen',chosen_samples[0])\n", - "print('rejected',rejected_samples[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To facilitate the training process, create new columns in the data set that combine the prompt with chosen and rejected responses. This combination helps in evaluating the responses in a structured dialogue format.\n", - "\n", - "**Function definition**: Define a function `add_combined_columns` that takes an example (a single data point) and adds two new columns:\n", - "- `prompt_chosen`: Combines the `prompt` with the `chosen` response in the same labeled format.\n", - "- `prompt_rejected`: Combines the `prompt` with the `rejected` response in the same labeled format.\n", - "\n", - "**Apply function**: The `map` method is used to apply this function to each example in the training split of the data set. This method iterates over all the examples and modifies them in place.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define a function to combine 'prompt' with 'chosen' and 'rejected' responses\n", - "def add_combined_columns(example):\n", - " # Combine 'prompt' with 'chosen' response, formatting it with \"Human:\" and \"Assistant:\" labels\n", - " example['prompt_chosen'] = \"\\n\\nHuman: \" + example[\"prompt\"] + \"\\n\\nAssistant: \" + example[\"chosen\"]\n", - " \n", - " # Combine 'prompt' with 'rejected' response, formatting it with \"Human:\" and \"Assistant:\" labels\n", - " example['prompt_rejected'] = \"\\n\\nHuman: \" + example[\"prompt\"] + \"\\n\\nAssistant: \" + example[\"rejected\"]\n", - " \n", - " # Return the modified example\n", - " return example\n", - "\n", - "# Apply the function to each example in the 'train' split of the dataset\n", - "dataset['train'] = dataset['train'].map(add_combined_columns)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When using pretrained transformers for classification tasks, understanding the maximum sequence length supported by the model is crucial, as pretrained transformers have a fixed maximum token length, for example, GPT-2 has 1024 tokens. Inputs longer than this are truncated, potentially losing important information. So a function is written to determine the max length.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "get_max_len= lambda samples: max([len(sample) for sample in samples])\n", - "get_max_len" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(\"rejected samples length\",get_max_len(rejected_samples))\n", - "print(\"chosen samples length\",get_max_len(chosen_samples))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Sometimes, you might want to identify samples shorter than a specified maximum length. This can be useful for filtering or handling special cases during preprocessing.\n", - "\n", - "The lambda function `find_short` takes a data set and a maximum length (`max_length`) as input. It uses a list comprehension to iterate over each example in the data set, enumerating both the index and the (chosen, rejected) pair. It zips `prompt_chosen` and `prompt_rejected` to pair each chosen response with its corresponding rejected response. For each pair, it checks if the length of either `chosen` or `rejected` is less than the specified `max_length`. If the condition is met, the index of that pair is included in the resulting list. The resulting list contains the index of all examples where either `prompt_chosen` or `prompt_rejected` is shorter than the specified `max_length`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "find_short = lambda dataset, max_length: [\n", - " i for i, (chosen, rejected) in enumerate(zip(dataset['prompt_chosen'], dataset['prompt_rejected']))\n", - " if len(chosen) < max_length or len(rejected) < max_length\n", - "]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To ensure that your data set only includes samples that meet the required length criteria, filter out any samples that are shorter than the specified `max_length`. This step is important for maintaining consistency in the input data for the model.\n", - "\n", - "Now, use the GPT-2 model for classification with a max length of 1024. First, set the maximum length (`max_length`) to 1024. The `find_short` function is then called with the training data set (`dataset['train']`) and this maximum length as arguments to find indices of examples where either `prompt_chosen` or `prompt_rejected` is shorter than the specified `max_length`. The resulting index (`subset_indices`) is used to create a subset of the training data set by selecting only the examples at these indices. The training data set (`dataset['train']`) is updated to this subset, and the `subset_indices` are returned or printed.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "max_length=1024\n", - "subset_indices=find_short (dataset['train'], max_length)\n", - "dataset['train'] = dataset['train'].select(subset_indices)\n", - "subset_indices[0:10]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - " The ```preprocess_function``` tokenizes the ```prompt_chosen``` and ```prompt_rejected``` keys, which are crucial for the RewardTrainer. The ```chosen``` key represents the preferred responses, while the ```rejected``` key represents the less preferred responses.\n", - " Tokenizing these keys allows the model to process and understand the differences between high-quality and low-quality responses. By providing both ```chosen``` and ```rejected``` inputs, the RewardTrainer can learn to distinguish and prioritize better responses, which is essential for training models to follow instructions effectively.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define a preprocessing function to tokenize the 'prompt_chosen' and 'prompt_rejected' keys\n", - "def preprocess_function(examples):\n", - " # Tokenize the 'prompt_chosen' text with truncation and padding to the maximum length\n", - " tokenized_chosen = tokenizer(examples['prompt_chosen'], truncation=True, max_length=max_length, padding=\"max_length\")\n", - " \n", - " # Tokenize the 'prompt_rejected' text with truncation and padding to the maximum length\n", - " tokenized_rejected = tokenizer(examples['prompt_rejected'], truncation=True, max_length=max_length, padding=\"max_length\")\n", - " \n", - " # Return the tokenized inputs as a dictionary\n", - " return {\n", - " \"input_ids_chosen\": tokenized_chosen[\"input_ids\"], # Token IDs for 'chosen' responses\n", - " \"attention_mask_chosen\": tokenized_chosen[\"attention_mask\"], # Attention masks for 'chosen' responses\n", - " \"input_ids_rejected\": tokenized_rejected[\"input_ids\"], # Token IDs for 'rejected' responses\n", - " \"attention_mask_rejected\": tokenized_rejected[\"attention_mask\"], # Attention masks for 'rejected' responses\n", - " }" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "The `input_ids_chosen` and `input_ids_rejected` fields contain the token IDs for the `chosen` and `rejected` responses, respectively, which are the numerical representations of the text used by the model. The `attention_mask_chosen` and `attention_mask_rejected` fields contain the attention masks for the `chosen` and `rejected` responses, respectively, which indicates tokens that should be attended to (1) and should ignore (0). These fields are crucial for the `RewardTrainer` because they provide the necessary tokenized inputs and attention masks for both the preferred and less preferred responses. By comparing the token IDs and attention patterns of the `chosen` and `rejected` responses, the `RewardTrainer` can distinguish between high- and low-quality responses, thereby improving the model's ability to prioritize better responses in instruction-following tasks.\n", - "\n", - "You can apply the ```reprocess_function``` to one sample:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "example=preprocess_function(dataset['train'][0])\n", - "example.keys()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, create a dictionary with 'chosen' and 'rejected' samples from the training data set. This dictionary is created to make it easier to validate the model later.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "train_str={'chosen': [sample for sample in dataset['train'] ['prompt_chosen']], 'rejected':[sample for sample in dataset['train'] ['prompt_rejected']]}" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The code applies the preprocess_function to each example in the training data set using the map method, which tokenizes the ```prompt_chosen``` and ```prompt_rejected``` texts. The `batched = True` parameter allows the function to process multiple examples at once, improving efficiency. Additionally, the `remove_columns` parameter specifies a list of columns (```prompt```, ```chosen```, ```rejected```, ```prompt_chosen```, ```prompt_rejected```) to be removed from the data set after processing. This ensures that only the tokenized inputs and attention masks generated by `preprocess_function` are retained, simplifying the data set structure and making it more suitable for model training and validation.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset['train'] = dataset['train'].map(preprocess_function, batched=True, remove_columns=['prompt',\"chosen\", \"rejected\",'prompt_chosen', 'prompt_rejected'])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The only columns left are the tokens and masks indexes.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset.column_names" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, split the data set into training and testing data sets.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "split_dataset = dataset['train'].train_test_split(test_size=0.2)\n", - "\n", - "# Create a DatasetDict to hold train and test splits\n", - "dataset_dict = DatasetDict({\n", - " 'train': split_dataset['train'],\n", - " 'test': split_dataset['test'],\n", - "})" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "## LoRA configuration\n", - "Now that the training data set is ready, you use the pretrain transformer model to start training. However, it is advisable to use a more efficient LoRA configuration for the model. Now, define the LoRA configuration and training arguments.\n", - "\n", - "First, initialize a `LoraConfig` configuration for low-rank adaptation (LoRA) in a sequence classification task. The configuration is created by using the `LoraConfig` class from the `peft` library and specifies several parameters:\n", - "\n", - "- **task_type=TaskType.SEQ_CLS**: Specifies the type of task, that is, the sequence classification for this lab.\n", - "- **inference_mode=False**: Indicates that the configuration is for training mode rather than inference.\n", - "- **r=8**: Sets the rank of the LoRA matrices.\n", - "- **lora_alpha=32**: Sets the alpha value for scaling the LoRA matrices.\n", - "- **lora_dropout=0.1**: Specifies the dropout rate for the LoRA layers, helping to prevent overfitting.\n", - "- **target_modules=[\"attn.c_attn\", \"attn.c_proj\"]**: Lists the specific attention layers in the model that will be adapted using LoRA. This includes the \"attn.c_attn\" and \"attn.c_proj\" modules.\n", - "\n", - "This configuration is useful for applying LoRA to the specific parts of the model, enabling efficient fine-tuning by adapting only a subset of the model parameters.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "peft_config = LoraConfig(\n", - " task_type=TaskType.SEQ_CLS,\n", - " inference_mode=False,\n", - " r=8,\n", - " lora_alpha=32,\n", - " lora_dropout=0.1,\n", - " target_modules=[\"attn.c_attn\", \"attn.c_proj\"] # Target attention layers\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Training arguments\n", - "\n", - "Define the training arguments by using the `TrainingArguments` class from the `transformers` library. These arguments configure various aspects of the training process:\n", - "\n", - "- **per_device_train_batch_size=3**: Sets the batch size per device (GPU/CPU) to 3\n", - "- **num_train_epochs=3**: Specifies the number of training epochs and is set to 3.\n", - "- **gradient_accumulation_steps=8**: Accumulates gradients over 8 steps before performing a backward/update pass, effectively increasing the batch size\n", - "- **learning_rate=1.41e-5**: Sets the learning rate for the optimizer to 1.41e-5\n", - "- **output_dir=\"./model_output3\"**: Specifies the directory where the model checkpoints and other outputs are saved\n", - "- **logging_steps=10**: Logs training progress every 10 steps\n", - "- **evaluation_strategy=\"steps\"**: Sets the evaluation strategy to evaluate the model at regular steps\n", - "- **eval_steps=500**: Evaluates the model every 500 steps\n", - "- **save_steps=500**: Saves the model checkpoint every 500 steps\n", - "- **save_total_limit=2**: Limits the number of saved checkpoints to 2, deleting older checkpoints to save space\n", - "\n", - "These arguments configure the training loop, including batch size, learning rate, logging, evaluation, and checkpoint-saving strategies.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define training arguments\n", - "\n", - "training_args = TrainingArguments(\n", - " per_device_train_batch_size=3, # Set to 3\n", - " num_train_epochs=3, # Set to 3\n", - " gradient_accumulation_steps=8,\n", - " learning_rate=1.41e-5,\n", - " output_dir=\"./model_output3\",\n", - " logging_steps=10,\n", - " eval_strategy=\"steps\",\n", - " eval_steps=500,\n", - " save_steps=500,\n", - " save_total_limit=2,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### RewardTrainer\n", - "\n", - "The `RewardTrainer` is a specialized trainer that is designed to train models with a reward signal. This is often used in reinforcement learning scenarios where the model learns to optimize for better responses. It is initialized with several parameters:\n", - "\n", - "- **model**: The model to be trained\n", - "- **args**: The training arguments. Typically, an instance of `TrainingArguments`\n", - "- **tokenizer**: The tokenizer used to process the text inputs\n", - "- **train_dataset**: The training data set\n", - "- **eval_dataset**: The evaluation data set\n", - "- **peft_config**: The configuration for LoRA\n", - "\n", - "The `RewardTrainer` orchestrates the training process, handling tasks such as batching, optimization, evaluation, and saving model checkpoints. It is particularly useful for training models that need to learn from feedback signals, improving their ability to generate high-quality responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Initialize RewardTrainer\n", - "trainer = RewardTrainer(\n", - " model=model,\n", - " args=training_args,\n", - " tokenizer=tokenizer,\n", - " train_dataset=dataset_dict['train'],\n", - " eval_dataset=dataset_dict['test'],\n", - " peft_config=peft_config,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: You can safely ignore the above warning.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The next step is training, saving, and evaluating a model by using the `RewardTrainer`. The `trainer.train()` method initiates the training process, where the model learns from the training data set, optimizing its parameters to improve performance. After training, the `trainer.save_model(output_dir)` method saves the trained model to the specified output directory, allowing for future use or deployment. Finally, the `trainer.evaluate()` method evaluates the model's performance on the evaluation data set, returning metrics that provide insights into how well the model performs. These metrics are then printed to give a detailed view of the model's evaluation results. \n", - "\n", - "Note: The training takes a very long time. Therefore, the model has already been trained and saved for you. If you want to train the model yourself, go ahead and uncomment the following cell.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# output_dir=\"./model_output3\"\n", - "\n", - "# # Train the model\n", - "# trainer.train()\n", - "\n", - "# # Save the model\n", - "# trainer.save_model(output_dir)\n", - "\n", - "# # Evaluate the model\n", - "# metrics = trainer.evaluate()\n", - "# print(metrics)\n", - "\n", - "# model.config.save_pretrained(\"./backup\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, download the pretained model. If you have trained the model yourself, you can skip this step.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/VZcK8FJ-kQ3nEJoxWGNYTQ/RetriverTrainerModel.zip" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!unzip -o RetriverTrainerModel.zip -d extracted_model" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", - "model = GPT2ForSequenceClassification.from_pretrained(\"./extracted_model/model_output3\", num_labels=1).to(DEVICE)\n", - "model" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: You can safely ignore the above warning.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, plot the loss. You can see it converges nicely.\n", - "\n", - "Run the below code to unzip the file.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "log_file = f\"extracted_model/model_output3/checkpoint-2500/trainer_state.json\"\n", - "\n", - "# Read the log file\n", - "with open(log_file, 'r') as f:\n", - " logs = json.load(f)\n", - "\n", - "# Extract training loss values\n", - "steps = []\n", - "losses = []\n", - "for log in logs[\"log_history\"]:\n", - " if \"loss\" in log:\n", - " steps.append(log[\"step\"])\n", - " losses.append(log[\"loss\"])\n", - "\n", - "# Plot the training loss\n", - "plt.figure(figsize=(10, 5))\n", - "plt.plot(steps, losses, label=\"Training Loss\")\n", - "plt.xlabel(\"Steps\")\n", - "plt.ylabel(\"Loss\")\n", - "plt.title(\"Training Loss Over Time\")\n", - "plt.legend()\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "## Evaluating the model\n", - "The `RewardTrainer` uses pairwise comparison to measure the model's ability to distinguish between high and low-quality responses. In pairwise comparison, the model is presented with two responses: one chosen as the preferred response and one as the less preferred (rejected) response. The model evaluates each response in the pair and assigns a score or reward based on its learned criteria. To perform an evaluation, two separate responses are inputted, and the model generates a score that is logit for each response. The response with the higher score is selected as the preferred one, demonstrating the model's capability to accurately prioritize better responses.\n", - "\n", - "First, load the model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "model_=GPT2ForSequenceClassification.from_pretrained(\"./extracted_model/model_output3\", num_labels=1).to(DEVICE)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, define the device (CPU or GPU) for training. You'll check if a GPU is available for use; otherwise, use CPU.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n", - "DEVICE" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The code first tokenizes `text1` by using the tokenizer, converting it into the format required by the model. The `tokenizer` function processes the input text into tensors with padding and truncation to ensure uniform input length, up to a maximum of 512 tokens. The `inputs` are then moved to the GPU, if available, for faster computation. The `model` is also transferred to the GPU. The inputs dictionary is updated to move all of its items to the device (GPU or CPU).\n", - "\n", - "The model is then used to generate outputs without computing gradients (`torch.no_grad()`), making the inference process faster and more memory-efficient. The score, the logits from the model's output, are extracted, representing the raw predictions. These logits are then passed through a sigmoid function to convert them into probabilities, which can be interpreted as the model's confidence in the projections. The resulting probabilities are printed or returned for further use.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "text1=train_str['chosen'][0]\n", - "print(text1)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "inputs = tokenizer(text1, return_tensors=\"pt\", padding=True, truncation=True, max_length=512)\n", - "\n", - "# Move inputs to the GPU if available\n", - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "model.to(device)\n", - "inputs = {k: v.to(device) for k, v in inputs.items()}\n", - "with torch.no_grad():\n", - " outputs = model(**inputs)\n", - "logit_1 = outputs.logits\n", - "print(\"Score :\",logit_1 )" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Do the same for the rejected sample \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "text2=train_str['rejected'][0]\n", - "print(text2)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "inputs = tokenizer(text2, return_tensors=\"pt\", padding=True, truncation=True, max_length=512)\n", - "\n", - "# Move inputs to the GPU if available\n", - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "model.to(device)\n", - "inputs = {k: v.to(device) for k, v in inputs.items()}\n", - "with torch.no_grad():\n", - " outputs = model(**inputs)\n", - "logit_2 = outputs.logits\n", - "print(\"Score :\",logit_2 )" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To demonstrate how pairwise comparison is useful for evaluating and ranking responses based on the model's predictions, use the following code that performs a pairwise comparison. To determine which of the two responses, represented by `logit_1` and `logit_2`, is preferred, compare the logits, which are raw scores output by the model that indicate the quality of the responses. If `logit_1` is greater than `logit_2`, the first response (`text1`) is selected as the better response, and the second response (`text2`) is rejected, printing both the selected and rejected responses along with their respective scores. Conversely, if `logit_2` is greater or equal, the second response (`text2`) is selected, and the first response (`text1`) is rejected, again printing both responses and their scores. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "if logit_1 > logit_2:\n", - " print(\"--------selected---------\")\n", - " print(text1, logit_1.detach().item())\n", - " print(\"--------rejected---------\")\n", - " print(text2, logit_2.detach().item())\n", - "else:\n", - " print(\"selected \")\n", - " print(text2, logit_2.detach().item())\n", - " print(\"rejected\")\n", - " print(text2, logit_2.detach().item()) " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, convert the process of tokenizing, generating a score, and comparing the outputs into two separate functions. The first function handles tokenizing the text and generating the model's output scores, while the second function performs the pairwise comparison of these scores. Structuring the code this way ensures that the process is modular and easier to manage, facilitating a clear and efficient workflow for evaluating and selecting the better response based on the model's predictions.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Function to make a prediction and get the logits\n", - "def predict_and_get_logits(text):\n", - " # Tokenize the input text\n", - " inputs = tokenizer(text, return_tensors=\"pt\", padding=True, truncation=True, max_length=512)\n", - " inputs = {k: v.to(device) for k, v in inputs.items()}\n", - "\n", - " # Perform the forward pass\n", - " with torch.no_grad():\n", - " outputs = model_(**inputs)\n", - " \n", - " # Extract the logits from the outputs\n", - " logits = outputs.logits.squeeze().item() # Assuming binary classification and batch size of 1\n", - " \n", - " return logits" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Function to compare two texts\n", - "def compare_texts(text1, text2):\n", - " logit1 = predict_and_get_logits(text1)\n", - " logit2 = predict_and_get_logits(text2)\n", - "\n", - " if logit1 > logit2:\n", - " print(\"selected---------\")\n", - " print(text1, f\"score: {logit1}\")\n", - "\n", - " return text1\n", - " else:\n", - " print(\"selected---------\")\n", - " print(text2, f\"score: {logit2}\")\n", - "\n", - " return text2" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, evaluate the performance of a model by using a pairwise comparison approach over a subset of the data set. It begins by defining N, the number of samples to evaluate, and initializes a counter `correct_selections` to keep track of how many times the model correctly identifies the preferred response. The code then iterates over the first N pairs of chosen and rejected responses from the training data set (`train_str['chosen']` and `train_str['rejected']`).\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define the number of samples to evaluate\n", - "N = 10\n", - "\n", - "# Initialize a counter for correct selections\n", - "correct_selections = 0\n", - "\n", - "# Iterate over the first N pairs of chosen and rejected responses\n", - "for chosen, rejected in zip(train_str['chosen'][0:N], train_str['rejected'][0:N]):\n", - " # Print the chosen response for reference\n", - " print(\"Chosen Response:\\n\", chosen)\n", - " \n", - " # Use the compare_texts function to determine which response is better\n", - " selected_text = compare_texts(chosen, rejected)\n", - " \n", - " # Check if the selected text is the chosen response\n", - " if selected_text == chosen:\n", - " correct_selections += 1\n", - "\n", - "# Calculate the accuracy as the ratio of correct selections to the total number of samples\n", - "accuracy = correct_selections / N\n", - "\n", - "# Print the accuracy\n", - "print(\"Accuracy:\", accuracy) " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Exercise \n", - "\n", - "#### Evaluate model's preference accuracy on a different subset of data\n", - "\n", - "1. Define a new variable `K` to set the number of samples for evaluation from a different subset of the data.\n", - "2. Initialize a counter to track the number of correct selections made by the model.\n", - "3. Iterate over the `K` pairs of chosen and rejected responses from a different subset of the data set (for example, from the middle of the data set).\n", - "4. For each pair, use the `compare_texts` function to determine which response is better.\n", - "5. Count the number of times the model correctly identifies the chosen response.\n", - "6. Calculate and print the accuracy of the model's preferences on this different subset.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for Solution\n", - "\n", - "```python\n", - "# Define the number of samples to evaluate from a different subset\n", - "K = 50\n", - "\n", - "# Initialize a counter for correct selections\n", - "correct_selections = 0\n", - "\n", - "# Determine the starting index for the different subset (e.g., middle of the dataset)\n", - "start_index = len(train_str['chosen']) // 2\n", - "\n", - "# Iterate over K pairs of chosen and rejected responses from the different subset\n", - "for chosen, rejected in zip(train_str['chosen'][start_index:start_index + K], train_str['rejected'][start_index:start_index + K]):\n", - " # Use the compare_texts function to determine which response is better\n", - " selected_text = compare_texts(chosen, rejected)\n", - " \n", - " # Check if the selected text is the chosen response\n", - " if selected_text == chosen:\n", - " correct_selections += 1\n", - "\n", - "# Calculate the accuracy as the ratio of correct selections to the total number of samples\n", - "accuracy = correct_selections / K\n", - "\n", - "# Print the accuracy\n", - "print(\"Accuracy on different subset:\", accuracy)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Congratulations! You have completed the lab\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Authors\n", - "\n", - "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n", - "\n", - "[Ashutosh Sagar](https://www.linkedin.com/in/ashutoshsagar/) is completing his MS in CS from Dalhousie University. He has previous experience working with Natural Language Processing and as a Data Scientist.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Other Contributors\n", - "\n", - "[Hailey Quach](https://author.skills.network/instructors/hailey_quach) is a Data Scientist at IBM. She's completing her Bsc, Honors in Computer Science at Concordia University, Montreal.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "© Copyright IBM Corporation. All rights reserved.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.19" - }, - "prev_pub_hash": "e304da04016a4b41134ab217e0a1138cc3a8efdada740e1e4f9acc4b4e744ba9" - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/DPO Fine-Tuning-v1.ipynb b/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/DPO Fine-Tuning-v1.ipynb deleted file mode 100755 index 38960d3..0000000 --- a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/DPO Fine-Tuning-v1.ipynb +++ /dev/null @@ -1,1199 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "

\n", - " \n", - " \"Skills\n", - " \n", - "

\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# **Direct Preference Optimization (DPO) Using Hugging Face**\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Estimated time needed: **60** minutes\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Large language models (LLMs) have revolutionized the field of natural language processing (NLP) by achieving remarkable performance in various tasks. However, it is challenging to align these models with human preferences. Therefore, the direct preference optimization (DPO) method comes in place which directly optimizes LLMs based models on user preferences, enhancing their alignment with human expectations. In this hands-on lab, you'll use the transformer reinforcement learning (trl) library from Hugging Face to implement DPO and fine-tune LLMs.\n", - "\n", - "The objective of this lab is to provide a practical understanding of the DPO method and its implementation using the trl library. \n", - "\n", - "By the end of this lab, you'll have hands-on experience in creating a data set formatted for DPO, implementing the optimization process, and evaluating the enhanced performance of LLMs.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## __Table of Contents__\n", - "\n", - "
    \n", - "
  1. Objectives
    2. \n", - "
  2. \n", - " Setup\n", - "
      \n", - "
    1. Installing required libraries
      2. \n", - "
    2. Importing required libraries
      3. \n", - "
    \n", - "
    3. \n", - "
  3. \n", - " Create and configure the model and tokenizer\n", - "
      \n", - "
    1. Quantized model configuration (Optional)
      2. \n", - "
    \n", - "
    4. \n", - "
  4. Preprocess dataset
    5. \n", - "
  5. DPO configuration
    6. \n", - "
  6. DPO training
    7. \n", - "
  7. Exercise\n", - "
\n", - " \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Objectives\n", - "\n", - "After completing this lab, you'll be able to: \n", - "- Understand the fundamentals of DPO and how it is different from proximal policy optimization (PPO)\n", - "- Set up an environment by installing and configuring necessary tools and libraries, such as trl library from Hugging Face\n", - "- Prepare a suitable environment for running DPO experiments with LLMs\n", - "- Create a data set for DPO\n", - "- Understand the required format for data sets used in DPO\n", - "- Create and preprocess a data set that includes user preferences\n", - "- Implement DPO by following a step-by-step guideline using the trl library\n", - "- Set training arguments, create a base quantized LoRA model, and train it using a DPO trainer\n", - "- Evaluate the performance of the LLM before and after applying DPO\n", - "- Analyze the impact of DPO on aligning the model with user preferences\n", - "\n", - "By the end of this hands-on lab, you will be equipped with the knowledge and skills needed to apply DPO for fine-tuning LLMs using the trl library. This will enable you to enhance LLMs' performance and user alignment in various NLP applications.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "----\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Setup\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installing required libraries\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The following required libraries are __not__ pre-installed in the Skills Network Labs environment. You will need to run the following cell to install them.\n", - "\n", - "**Note:** In this lab, you don't have a pinned version to demonstrate the latest functionality, but you can always pin versions in your labs.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install torch\n", - "!pip install trl # for optimization training\n", - "!pip install peft # for creating LoRA architecture\n", - "!pip install matplotlib" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Importing required libraries\n", - "\n", - "_It's recommended to import all required libraries in one place (here):_\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "##imports\n", - "import multiprocessing\n", - "import os\n", - "import requests\n", - "import tarfile\n", - "import pandas as pd\n", - "import matplotlib.pyplot as plt\n", - "\n", - "import torch\n", - "from datasets import load_dataset\n", - "\n", - "from peft import LoraConfig\n", - "from transformers import AutoModelForCausalLM, AutoTokenizer,TrainingArguments, GPT2Tokenizer, set_seed, GenerationConfig\n", - "from trl import DPOConfig, DPOTrainer\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Create and configure the model and tokenizer\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "\n", - "# Load the GPT-2 model\n", - "model = AutoModelForCausalLM.from_pretrained(\"gpt2\")\n", - "\n", - "# Load a reference model \n", - "model_ref = AutoModelForCausalLM.from_pretrained(\"gpt2\")\n", - "\n", - "# Load the GPT-2 tokenizer\n", - "tokenizer = GPT2Tokenizer.from_pretrained(\"gpt2\")\n", - "\n", - "# Set the pad token to the end-of-sequence token\n", - "tokenizer.pad_token = tokenizer.eos_token\n", - "# Set the padding side to \"right\" to fix the overflow issue with FP16 training\n", - "tokenizer.padding_side = \"right\"\n", - "\n", - "# Disable the use of the cache during the model's forward pass\n", - "model.config.use_cache = False" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Here, you can check the model architecture.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "model" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Quantized model configuration (Optional)\n", - "If you want memory-efficient training and have access to a GPU-powered environment, you can download the complete lab, uncomment the following two code blocks to create a quantized model and proceed with training the model on GPU. This is because you will need GPUs for the bits and bytes package.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#!pip install -U bitsandbytes # this package is required for quantization" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**_Note:_** _You can run the installed package by restarting a Kernel._\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "'''## Quantized model --only available on GPU\n", - "from transformers import BitsAndBytesConfig\n", - "\n", - "# Configure the quantization parameters\n", - "quantization_config = BitsAndBytesConfig(\n", - " # Load the model in 4-bit quantized format\n", - " load_in_4bit=True,\n", - " # Enable double quantization for better accuracy\n", - " bnb_4bit_use_double_quant=True,\n", - " # Use non-uniform 4-bit quantization (nf4)\n", - " bnb_4bit_quant_type=\"nf4\",\n", - " # Use bfloat16 as the computation data type during quantization\n", - " bnb_4bit_compute_dtype=torch.bfloat16\n", - ")\n", - "\n", - "# Load GPT-2 model with the specified quantization configuration\n", - "model = AutoModelForCausalLM.from_pretrained(\"gpt2\", quantization_config=quantization_config)\n", - "\n", - "# Load a reference model with the same quantization configuration\n", - "model_ref = AutoModelForCausalLM.from_pretrained(\"gpt2\", quantization_config=quantization_config)\n", - "\n", - "# Load GPT-2 tokenizer\n", - "tokenizer = GPT2Tokenizer.from_pretrained(\"gpt2\")\n", - "\n", - "# Set the pad token to the end-of-sequence token\n", - "tokenizer.pad_token = tokenizer.eos_token\n", - "# Set the padding side to \"right\" to fix the overflow issue with FP16 training\n", - "tokenizer.padding_side = \"right\"\n", - "\n", - "# Disable the use of the cache during the model's forward pass\n", - "model.config.use_cache = False'''" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Preprocess data set\n", - "\n", - "The \"ultrafeedback_binarized\" data set on Hugging Face is a collection of prompts and responses. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Load the dataset from the specified location\n", - "ds = load_dataset(\"BarraHome/ultrafeedback_binarized\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This data set includes six splits. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds.keys()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Each record has different features among which you need to select from the three features, that is \"chosen,\" \"rejected,\" and \"prompt.\" This means that for each prompt, a prefered response and a rejected response are provided.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds[\"train_prefs\"][0].keys()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can check the sample record of data, where you can see three features along with other features that is the prompt, the rejected, and chosen responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds[\"train_prefs\"][0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, put the data set in the format that the DPO trainer accepts.\n", - "\n", - "| Chosen | Rejected | Prompt |\n", - "| --- | --- | --- |\n", - " | Developing a daily habit of drawing can be challenging
but with consistent practice, and a few tips. | One way to develop a habit of drawing daily is
to allocate a specific time interval for drawing. | How can I develop a habit of drawing daily?|\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# You can reduce the volume of data (due to resource limitations) by selecting the first 5% examples from each split of the dataset\n", - "for key in ds:\n", - " #cnt = round(ds[key].__len__()*0.05)\n", - " cnt=50\n", - " ds[key] = ds[key].select(range(cnt))\n", - "\n", - "# Define a function to process the data\n", - "def process(row):\n", - " # delete unwanted columns\n", - " del row[\"prompt_id\"]\n", - " del row[\"messages\"]\n", - " del row[\"score_chosen\"]\n", - " del row[\"score_rejected\"]\n", - " # retrieve the actual response text\n", - " row[\"chosen\"] = row[\"chosen\"][-1][\"content\"]\n", - " row[\"rejected\"] = row[\"rejected\"][-1][\"content\"]\n", - "\n", - " return row\n", - "\n", - "# Apply the data processing function to the dataset\n", - "ds = ds.map(\n", - " process,\n", - " num_proc=multiprocessing.cpu_count(),\n", - " load_from_cache_file=False,\n", - ")\n", - "\n", - "# Split the dataset into training and evaluation sets\n", - "train_dataset = ds['train_prefs']\n", - "eval_dataset = ds['test_prefs']\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's check the data record.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "train_dataset[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, define LoRAConfig for efficient fine-tuning.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# PEFT (Parameter-Efficient Finetuning) configuration\n", - "peft_config = LoraConfig(\n", - " # The rank of the low-rank adaptation weights\n", - " r=4,\n", - " # The target modules to apply the low-rank adaptation to\n", - " target_modules=['c_proj','c_attn'],\n", - " # The task type for the low-rank adaptation\n", - " task_type=\"CAUSAL_LM\",\n", - " # The scaling factor for the low-rank adaptation weights\n", - " lora_alpha=8,\n", - " # The dropout probability for the low-rank adaptation weights\n", - " lora_dropout=0.1,\n", - " # The bias mode for the low-rank adaptation\n", - " bias=\"none\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### DPO configuration\n", - "\n", - "First, define training arguments.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# DPO configuration\n", - "training_args = DPOConfig(\n", - " # The beta parameter for the DPO loss function\n", - " #beta is the temperature parameter for the DPO loss, typically something in the range of 0.1 to 0.5 . \n", - " beta=0.1,\n", - " # The output directory for the training\n", - " output_dir=\"dpo\",\n", - " # The number of training epochs\n", - " num_train_epochs=5,\n", - " # The batch size per device during training\n", - " per_device_train_batch_size=1,\n", - " # The batch size per device during evaluation\n", - " per_device_eval_batch_size=1,\n", - " # Whether to remove unused columns from the dataset\n", - " remove_unused_columns=False,\n", - " # The number of steps between logging training progress\n", - " logging_steps=10,\n", - " # The number of gradient accumulation steps\n", - " gradient_accumulation_steps=1,\n", - " # The learning rate for the optimization\n", - " learning_rate=1e-4,\n", - " # The evaluation strategy (e.g., after each step or epoch)\n", - " evaluation_strategy=\"epoch\",\n", - " # The number of warmup steps for the learning rate scheduler\n", - " warmup_steps=2,\n", - " # Whether to use 16-bit (float16) precision\n", - " fp16=False,\n", - " # The number of steps between saving checkpoints\n", - " save_steps=500,\n", - " # The maximum number of checkpoints to keep\n", - " #save_total_limit=2,\n", - " # The reporting backend to use (set to 'none' to disable, you can also report to wandb or tensorboard)\n", - " report_to='none'\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### DPO training\n", - "\n", - "Next step is creating the actual trainer using DPOTrainer class.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "tokenizer.pad_token = tokenizer.eos_token\n", - "\n", - "# Create a DPO trainer\n", - "# This trainer will handle the fine-tuning of the model using the DPO technique\n", - "trainer = DPOTrainer(\n", - " # The model to be fine-tuned\n", - " model,\n", - " # The reference model (not used in this case because LoRA has been used)\n", - " ref_model=None,\n", - " # The DPO training configuration\n", - " args=training_args,\n", - " # The beta parameter for the DPO loss function\n", - " beta=0.1,\n", - " # The training dataset\n", - " train_dataset=train_dataset,\n", - " # The evaluation dataset\n", - " eval_dataset=eval_dataset,\n", - " # The tokenizer for the model\n", - " tokenizer=tokenizer,\n", - " # The PEFT (Parallel Efficient Finetuning) configuration\n", - " peft_config=peft_config,\n", - " # The maximum prompt length\n", - " max_prompt_length=512,\n", - " # The maximum sequence length\n", - " max_length=512,\n", - " )\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please note that when using LoRA for the base model, it's efficient to leave the model_ref param null, in which case the DPOTrainer will unload the adapter for reference inference.\n", - "\n", - "\n", - "Now, you're all set for training the model.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Training model\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Keep in mind that training the model on a CPU can be time-consuming and may cause the kernel to crash due to memory issues. If this happens, you can bypass training by loading the pre-trained model provided in the next section and proceed from there.**\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Start the training process\n", - "trainer.train()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's retrieve and plot the training loss versus evaluation loss.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Retrieve log_history and save it to a dataframe\n", - "log = pd.DataFrame(trainer.state.log_history)\n", - "log_t = log[log['loss'].notna()]\n", - "log_e = log[log['eval_loss'].notna()]\n", - "\n", - "# Plot train and evaluation losses\n", - "plt.plot(log_t[\"epoch\"], log_t[\"loss\"], label = \"train_loss\") \n", - "plt.plot(log_e[\"epoch\"], log_e[\"eval_loss\"], label = \"eval_loss\") \n", - "plt.legend() \n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "![image](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/7KEnvtpUyNcJTINdArLf7A/loss%20dpo.png)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Load the trained DPO model you just trained\n", - "dpo_model = AutoModelForCausalLM.from_pretrained('./dpo/checkpoint-250')\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Loading trained model\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you encounter difficulty in running the training cell due to resource limitations, you can download the model to be fine-tuned: \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define the URL and the filename\n", - "url = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/YIDeT3qihEpWChdXN_RmTg/DPO-tar.gz'\n", - "filename = './DPO.tar'\n", - "\n", - "# Download the file\n", - "response = requests.get(url)\n", - "\n", - "# Save the file locally\n", - "with open(filename, 'wb') as f:\n", - " f.write(response.content)\n", - "\n", - "# Extract the tar file\n", - "if tarfile.is_tarfile(filename):\n", - " with tarfile.open(filename, 'r') as tar:\n", - " tar.extractall()\n", - " print(\"Files extracted:\", tar.getnames())\n", - "else:\n", - " print(\"The adownloaded file is not a tar file.\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Then, load it into the model for further inference:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Load the trained DPO model tiy just trained\n", - "dpo_model = AutoModelForCausalLM.from_pretrained('./DPO')\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Generation\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Load the GPT-2 tokenizer\n", - "tokenizer = GPT2Tokenizer.from_pretrained('gpt2')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Set a seed for reproducibility\n", - "set_seed(42)\n", - "\n", - "\n", - "# Define the generation configuration for the DPO model\n", - "# This sets the parameters for text generation\n", - "generation_config = GenerationConfig(\n", - " # Use sampling to generate diverse text\n", - " do_sample=True,\n", - " # Top-k sampling parameter\n", - " top_k=1,\n", - " # Temperature parameter to control the randomness of the generated text\n", - " temperature=0.1,\n", - " # Maximum number of new tokens to generate\n", - " max_new_tokens=25,\n", - " # Use the end-of-sequence token as the padding token\n", - " pad_token_id=tokenizer.eos_token_id\n", - " )\n", - "\n", - "# Define the input prompt for text generation\n", - "PROMPT = \"Is a higher octane gasoline better for your car?\"\n", - "# Encode the prompt using the tokenizer\n", - "inputs = tokenizer(PROMPT, return_tensors='pt')\n", - "\n", - "# Generate text using the DPO model\n", - "outputs = dpo_model.generate(**inputs, generation_config=generation_config)\n", - "# Decode the generated text and print it\n", - "print(\"DPO response:\\t\",tokenizer.decode(outputs[0], skip_special_tokens=True))\n", - "\n", - "# Load the pre-trained GPT-2 model\n", - "gpt2_model = AutoModelForCausalLM.from_pretrained('gpt2')\n", - "# Generate text using the GPT-2 model\n", - "outputs = gpt2_model.generate(**inputs, generation_config=generation_config)\n", - "# Decode the generated text and print it\n", - "print(\"\\nGPT2 response:\\t\",tokenizer.decode(outputs[0], skip_special_tokens=True))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Althought the model is trained on a small data for 5 epochs only, it can be seen that the response generated by the DPO-tuned model is more concise and straightforward.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Exercise\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Exercise 1: Preprocess the `argilla/ultrafeedback-binarized-preferences-cleaned` Dataset\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This data set comprises user-generated prompts along with corresponding responses categorized as either \"chosen\" or \"rejected.\" It provides a rich source of binary feedback, making it ideal for training models to align with user preferences.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Load the data set from the `argilla/ultrafeedback-binarized-preferences-cleaned`\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "dataset = load_dataset(\"argilla/ultrafeedback-binarized-preferences-cleaned\")\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset['train']" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Set the variable cnt to 50 and then select the first 50 (cnt) examples to reduce the volume of data for resource limitations.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "cnt = 50 # You can adjust this count based on your requirements\n", - "\n", - "# Select the first 5% of examples\n", - "dataset['train'] = dataset['train'].select(range(cnt))\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Create a function named `process` that takes a row of data as input. Within this function, remove unwanted columns such as `source, chosen-rating, chosen-model, rejected-rating, and rejected-model`. Then, use the map function to apply the process function to each row in the training data set.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "def process(row):\n", - " # Delete unwanted columns\n", - " del row[\"source\"]\n", - " del row[\"chosen-rating\"]\n", - " del row[\"chosen-model\"]\n", - " del row[\"rejected-rating\"]\n", - " del row[\"rejected-model\"]\n", - " \n", - " # Retrieve the actual response text\n", - " row[\"chosen\"] = row[\"chosen\"][-1][\"content\"]\n", - " row[\"rejected\"] = row[\"rejected\"][-1][\"content\"]\n", - " \n", - " return row\n", - "\n", - "# Apply the data processing function to the dataset\n", - "dataset['train'] = dataset['train'].map(\n", - " process,\n", - " num_proc=multiprocessing.cpu_count(),\n", - " load_from_cache_file=False,\n", - ")\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Split the data set into training and evaluation sets:\n", - "Calculate the size for the training set as 80% of the total data. The remaining 20% will be used for evaluation.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "train_size = int(0.8 * len(dataset['train'])) # 80% for training\n", - "eval_size = len(dataset['train']) - train_size # Remaining 20% for evaluation\n", - "\n", - "train_dataset = dataset['train'].select(range(train_size))\n", - "eval_dataset = dataset['train'].select(range(train_size, train_size + eval_size))\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "train_dataset" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "train_dataset[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Exercise 2: Prompt Inferencing and Comparison with GPT-2\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "PROMPT = input()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Initialize the GPT-2 Tokenizer\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "tokenizer = GPT2Tokenizer.from_pretrained('gpt2')\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Create a generation_config object to set the parameters for text generation.\n", - "- do_sample=True (It enables sampling, which allows for more diverse outputs.)\n", - "- top_k=1 (It specifies the number of highest probability vocabulary tokens to consider during generation.)\n", - "- temperature=0.1 (It controls the randomness of the output.)\n", - "- max_new_tokens=25 (It sets the maximum number of new tokens to generate during inference.)\n", - "- pad_token_id=tokenizer.eos_token_id (It specifies the token to use for padding.)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "generation_config = GenerationConfig(\n", - " # Use sampling to generate diverse text\n", - " do_sample=True,\n", - " # Top-k sampling parameter: controls the number of highest probability tokens to consider\n", - " top_k=1,\n", - " # Temperature parameter: controls the randomness of the generated text\n", - " temperature=0.1,\n", - " # Maximum number of new tokens to generate\n", - " max_new_tokens=25,\n", - " # Use the end-of-sequence token as the padding token\n", - " pad_token_id=tokenizer.eos_token_id\n", - ")\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Create a function named `generate_dpo_response` that takes a prompt as input and generates a response using the DPO model (`dpo_model`).\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "def generate_dpo_response(prompt):\n", - " # Tokenize the prompt\n", - " inputs = tokenizer(prompt, return_tensors='pt')\n", - "\n", - " # Generate text using the DPO model\n", - " outputs = dpo_model.generate(**inputs, generation_config=generation_config)\n", - " \n", - " # Decode and return the response\n", - " return tokenizer.decode(outputs[0], skip_special_tokens=True)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Create another function named `generate_gpt2_response` that takes a prompt as input and generates a response using the GPT-2 model (`gpt2_model`).\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "def generate_gpt2_response(prompt):\n", - " # Tokenize the prompt\n", - " inputs = tokenizer(prompt, return_tensors='pt')\n", - "\n", - " # Generate text using the GPT-2 model\n", - " outputs = gpt2_model.generate(**inputs, generation_config=generation_config)\n", - " \n", - " # Decode and return the response\n", - " return tokenizer.decode(outputs[0], skip_special_tokens=True)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Call both functions with a prompt and compare the responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#TODO" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "\n", - "
\n", - " Click here for hint\n", - "\n", - "```python\n", - "# Generate responses\n", - "dpo_response = generate_dpo_response(PROMPT)\n", - "gpt2_response = generate_gpt2_response(PROMPT)\n", - "\n", - "# Print the responses\n", - "print(\"DPO response:\\t\", dpo_response)\n", - "print(\"\\nGPT-2 response:\\t\", gpt2_response)\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Congratulations! You have completed the lab!\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Authors\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Fateme Akbari](https://www.linkedin.com/in/fatemeakbari/) is a Ph.D. candidate in Information Systems at McMaster University with demonstrated research experience in Machine Learning and NLP.\n", - "\n", - "[Kunal Makwana](https://author.skills.network/instructors/kunal_makwana) is a Data Scientist at IBM and is currently pursuing his Master's in Computer Science at Dalhousie University.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## References\n", - "[DPO Trainer](https://huggingface.co/docs/trl/main/en/dpo_trainer)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "© Copyright IBM Corporation. All rights reserved.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.19" - }, - "prev_pub_hash": "21ff78b44c97c4a9c4f0d7965c976d0f5a40a6c0de593f10a90787e44e4637df" - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/PPOTrainer-v1.ipynb b/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/PPOTrainer-v1.ipynb deleted file mode 100755 index 78c8832..0000000 --- a/notebooks/LLM_Specialization/Generative AI Advance Fine-Tuning for LLMs/Reinforcement Learning from Human Feedback/PPOTrainer-v1.ipynb +++ /dev/null @@ -1,2190 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "

\n", - " \n", - " \"Skills\n", - " \n", - "

\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Reinforcement Learning from Human Feedback Using PPO\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Estimated time needed: **30** minutes\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Imagine you are an AI engineer who wants to train a \"Happy LLM\" and a \"Pessimistic LLM\" to train customer service agents. You have a reward function trained on the sentiment classifier from the IMDb dataset, and you will now use Reinforcement Learning (RL). RL is a subfield of machine learning where an agent learns to make decisions by performing actions in an environment to maximize a cumulative reward. The agent, in this case, will be the LLM, and the decisions will be about what text to output. Unlike supervised learning, which requires labeled input/output pairs, RL relies on the agent exploring the environment and learning from the feedback it receives in the form of rewards or penalties. This trial-and-error approach enables the agent to improve its decision-making strategy over time.\n", - "\n", - "Proximal Policy Optimization (PPO) is one of the most effective and widely used RL algorithms. Introduced by OpenAI, PPO strikes a balance between simplicity and performance, making it a popular choice for training RL agents. PPO optimizes the policy directly and employs mechanisms to ensure the updates are not too drastic, thereby maintaining stability and reliability during training.\n", - "\n", - "In this lab, you will be guided through the process of training an RL agent using the PPO algorithm with a focus on sentiment analysis. You will use the IMDb dataset, a large collection of movie reviews, to train your model. By the end of this lab, you will have a solid understanding of how to implement and train an RL agent using PPO, and you will be equipped with practical skills to apply RL techniques to other problems and datasets.\n", - "This lab is based on [a HF example code titled `Tune GPT2 to generate positive reviews`](https://github.com/huggingface/trl/blob/main/examples/notebooks/gpt2-sentiment.ipynb).\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## __Table of Contents__\n", - "\n", - "
    \n", - "
  1. Objectives
    2. \n", - "
  2. \n", - " Setup\n", - "
      \n", - "
    1. Installing required libraries
      2. \n", - "
    2. Importing required libraries
      3. \n", - "
    3. Defining helper functions
      4. \n", - "
    \n", - "
    3. \n", - "
  3. Initializing the PPO configuration, model, and tokenizer
    4. \n", - "
  4. Dataset and dataset tokenization
    5. \n", - "
  5. Collator function
    6. \n", - "
  6. Initialize PPOTrainer
    7. \n", - "
  7. Reward function
    8. \n", - "
  8. \n", - " Generating responses using PPO\n", - "
      \n", - "
    1. Tokenizing and preparing the input batch
      2. \n", - "
    2. Scoring function
      3. \n", - "
    3. Proximal policy optimization
      4. \n", - "
    \n", - "
    9. \n", - "
  9. Plotting PPO training loss and mean
    10. \n", - "
  10. Generating and analyzing text with PPO and reference models
    11. \n", - "
  11. \n", - " Comparing PPO and reference models on\n", - "
      \n", - "
    \n", - "
    12. \n", - "
  12. Running the PPO model with negative sentiment
    13. \n", - "
  13. Comparing models with negative sentiment
    14. \n", - "
  14. Exercise: Comparing PPO models
    15. \n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Objectives\n", - "\n", - "After completing this lab you will be able to:\n", - "\n", - "- Apply the basics of reinforcement learning and proximal policy optimization (PPO).\n", - "- Set up the environment and load the IMDb dataset for training.\n", - "- Define and configure the PPO agent and tokenizer.\n", - "- Implement the PPO training loop.\n", - "- Generate and evaluate text responses from the trained model.\n", - "- Compare the performance of two models on the dataset.\n", - "- Save and load the trained model for future use.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "----\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Setup\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For this lab, you will use the following libraries:\n", - "\n", - "* [`pandas`](https://pandas.pydata.org/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for managing the data.\n", - "* [`torch`](https://pytorch.org/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for tensor operations and model training.\n", - "* [`tqdm`](https://tqdm.github.io/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for progress bars.\n", - "* [`transformers`](https://huggingface.co/transformers/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for pretrained language models.\n", - "* [`datasets`](https://huggingface.co/docs/datasets/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for loading and processing datasets.\n", - "* [`trl`](https://github.com/lvwerra/trl/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for Proximal Policy Optimization (PPO) training.\n", - "* [`matplotlib`](https://matplotlib.org/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for plotting tools.\n", - "* [`tarfile`](https://docs.python.org/3/library/tarfile.html/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for handling tar file operations.\n", - "* [`pickle`](https://docs.python.org/3/library/pickle.html/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for serializing and deserializing Python objects.\n", - "* [`json`](https://docs.python.org/3/library/json.html/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMML0187ENSkillsNetwork31430127-2021-01-01) for parsing and writing JSON data.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installing required libraries\n", - "\n", - "The following required libraries are __not__ preinstalled in the Skills Network Labs environment. __You must run the following cell__ to install them:\n", - "\n", - "**Note:** The version has been pinned to specify the version. It's recommended that you do this as well. Even if the library is updated in the future, the installed library could still support this lab work.\n", - "\n", - "This might take approximately 1 minute. \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install datasets trl==0.11.0\n", - "!pip install --upgrade typing_extensions\n", - "!pip install matplotlib" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Importing required libraries\n", - "\n", - "_It is recommended that you import all required libraries in one place (here):_\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%load_ext autoreload\n", - "%autoreload 2" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import torch\n", - "from tqdm import tqdm\n", - "import pandas as pd\n", - "\n", - "tqdm.pandas()\n", - "\n", - "from transformers import pipeline, AutoTokenizer,AutoModelForCausalLM\n", - "from datasets import load_dataset\n", - "\n", - "from trl import PPOTrainer, PPOConfig, AutoModelForCausalLMWithValueHead\n", - "from trl.core import LengthSampler\n", - "import os\n", - "\n", - "import tarfile\n", - "import pickle\n", - "import json\n", - "import matplotlib.pyplot as plt\n", - "import torch\n", - "import pandas as pd\n", - "import warnings\n", - "\n", - "warnings.filterwarnings('ignore')\n", - "# Disable warnings for a cleaner notebook or console experience\n", - "def warn(*args, **kwargs):\n", - " pass\n", - "warnings.warn = warn" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Defining helper functions\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def save_to_json(data, file_path):\n", - " \"\"\"\n", - " Save a dictionary to a JSON file.\n", - "\n", - " Args:\n", - " data (dict): The dictionary to save.\n", - " file_path (str): The path to the JSON file.\n", - " \"\"\"\n", - " with open(file_path, 'w') as json_file:\n", - " json.dump(data, json_file, indent=4)\n", - " print(f\"Data successfully saved to {file_path}\")\n", - " \n", - " \n", - "def load_from_json(file_path):\n", - " \"\"\"\n", - " Load data from a JSON file.\n", - "\n", - " Args:\n", - " file_path (str): The path to the JSON file.\n", - "\n", - " Returns:\n", - " dict: The data loaded from the JSON file.\n", - " \"\"\"\n", - " with open(file_path, 'r') as json_file:\n", - " data = json.load(json_file)\n", - " return data \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def pad_sequence_to_length(tensor, length, pad_token_id):\n", - " padding_length = length - tensor.size(0)\n", - " if padding_length > 0:\n", - " padding = torch.full((padding_length,), pad_token_id, dtype=torch.long, device=tensor.device)\n", - " return torch.cat((tensor, padding))\n", - " return tensor\n", - "\n", - "def pad_list_to_batch_size(tensors, batch_size, pad_token_id):\n", - " max_length = max(t.size(0) for t in tensors)\n", - " padded_tensors = [pad_sequence_to_length(t, max_length, pad_token_id) for t in tensors]\n", - "\n", - " # Add additional padding-only tensors if needed\n", - " while len(padded_tensors) < batch_size:\n", - " padded_tensors.append(torch.full((max_length,), pad_token_id, dtype=torch.long, device=tensors[0].device))\n", - "\n", - " return padded_tensors[:batch_size]" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def print_ppo_stats(stats, related_to_objective=False):\n", - " print(\"PPO Training Statistics\\n\")\n", - "\n", - " if related_to_objective:\n", - " print(\"Objective Statistics:\")\n", - " print(f\" KL Divergence (objective/kl): {stats['objective/kl']}\")\n", - " print(f\" KL Coefficient (objective/kl_coef): {stats['objective/kl_coef']}\")\n", - " print(f\" Entropy (objective/entropy): {stats['objective/entropy']}\\n\")\n", - " \n", - " print(\"PPO Losses (Related to Minimizing Objective Function):\")\n", - " print(f\" Policy Loss (ppo/loss/policy): {stats['ppo/loss/policy']}\")\n", - " print(f\" Value Loss (ppo/loss/value): {stats['ppo/loss/value']}\")\n", - " print(f\" Total Loss (ppo/loss/total): {stats['ppo/loss/total']}\\n\")\n", - " \n", - " print(\"PPO Policy Statistics:\")\n", - " print(f\" Policy Entropy (ppo/policy/entropy): {stats['ppo/policy/entropy']}\")\n", - " print(f\" Approx KL (ppo/policy/approxkl): {stats['ppo/policy/approxkl']}\")\n", - " print(f\" Clip Fraction (ppo/policy/clipfrac): {stats['ppo/policy/clipfrac']}\\n\")\n", - " else:\n", - " print(\"Reward and Value Function Estimation:\")\n", - " print(f\" Mean Non-Score Reward (ppo/mean_non_score_reward): {stats['ppo/mean_non_score_reward']}\")\n", - " print(f\" Mean Scores (ppo/mean_scores): {stats['ppo/mean_scores']}\")\n", - " print(f\" Std Scores (ppo/std_scores): {stats['ppo/std_scores']}\")\n", - " print(f\" Value Prediction (ppo/val/vpred): {stats['ppo/val/vpred']}\")\n", - " print(f\" Value Prediction Error (ppo/val/error): {stats['ppo/val/error']}\")\n", - " print(f\" Value Prediction Variance (ppo/val/var): {stats['ppo/val/var']}\")\n", - " print(f\" Value Prediction Mean (ppo/val/mean): {stats['ppo/val/mean']}\")\n", - " print(f\" Explained Variance (ppo/val/var_explained): {stats['ppo/val/var_explained']}\\n\")\n", - " \n", - " print(\"Token Lengths:\")\n", - " print(f\" Queries Length Mean (tokens/queries_len_mean): {stats['tokens/queries_len_mean']}\")\n", - " print(f\" Responses Length Mean (tokens/responses_len_mean): {stats['tokens/responses_len_mean']}\\n\")\n", - " \n", - " print(\"Time Statistics:\")\n", - " print(f\" Total Time (time/ppo/total): {stats['time/ppo/total']} seconds\\n\")\n", - "\n", - "# Example usage with the provided stats and the flag" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initializing the PPO configuration, model, and tokenizer\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `PPOConfig` class is used to specify the model and learning rate for the PPO training. In this case, the model is `\"lvwerra/gpt2-imdb\"` and the learning rate is set to `1.41e-5`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "config = PPOConfig(\n", - " model_name=\"lvwerra/gpt2-imdb\",\n", - " learning_rate=1.41e-5)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please ignore above warning as the trl version you installed supports this module.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`config.model_name` refers to the specific model identifier used in the configuration for loading the pretrained model. It specifies which model to load from the Hugging Face model repository. In this case, `config.model_name` is set to `\"lvwerra/gpt2-imdb\"`, indicating that the GPT-2 model fine-tuned on the IMDB dataset (by user lvwerra) should be used. This identifier is essential for loading the correct model architecture and weights during the fine-tuning or inference process.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "config.model_name" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `sent_kwargs` dictionary contains parameters for the sentiment analysis pipeline, specifying that all scores should be returned, the function to apply is `\"none\"`, and the batch size is `2`.\n", - "python\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sent_kwargs = {\"top_k\":None, \"function_to_apply\": \"none\", \"batch_size\": 2}" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `AutoModelForCausalLMWithValueHead` class is used to load the pretrained GPT-2 model with a value head for PPO training. The model is loaded from the specified model name in the configuration.\n", - "\n", - "The `AutoTokenizer` class is used to load the tokenizer corresponding to the pretrained model. The tokenizer's padding token is set to the end-of-sequence (EOS) token.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "model_1 = AutoModelForCausalLMWithValueHead.from_pretrained(config.model_name)\n", - "\n", - "tokenizer = AutoTokenizer.from_pretrained(config.model_name)\n", - "tokenizer.pad_token = tokenizer.eos_token" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please ignore above warning as the trl version you installed handles it automatically.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# first model\n", - "model = AutoModelForCausalLMWithValueHead.from_pretrained(config.model_name)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "During PPO training, update the model. In addition, the reference model is used to stabilize the model using the Kullback-Leibler (KL) divergence between the current policy and the reference policy.The KL divergence acts as a regularization term.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ref_model = AutoModelForCausalLMWithValueHead.from_pretrained(config.model_name)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Dataset and dataset tokenization\n", - "\n", - "**Dataset Name:** IMDB\n", - "\n", - "**Description:** The IMDB dataset is a collection of 50,000 movie reviews labeled as \"positive\" or \"negative,\" indicating the sentiment of each review. This dataset is commonly used for sentiment analysis tasks.\n", - "\n", - "**Loading the Dataset:**\n", - "The dataset is loaded using the `load_dataset` function from the `datasets` library, specifically loading the \"train\" split.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset_name = \"imdb\"\n", - "ds = load_dataset(dataset_name, split = \"train\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "N = 5\n", - "for sample in range(N):\n", - " print('text',ds[sample]['text'])\n", - " print('label',ds[sample]['label'])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " Rename the column \"text\" to \"review\"\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds = ds.rename_columns({\"text\": \"review\"})\n", - "ds" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The dataset is filtered to include only reviews that are longer than 200 characters.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds = ds.filter(lambda x: len(x[\"review\"]) > 200, batched=False)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Using a ```LengthSampler``` to sample different text lengths during data processing introduces variability, making the model more robust and capable of handling varying input lengths in real-world scenarios. This approach prevents overfitting by exposing the model to diverse input sizes, improving generalization to new data. It also ensures efficient training by managing the length of text inputs, maintaining practicality and performance. Overall, LengthSampler enhances model adaptability and effectiveness by simulating realistic, varied training conditions. Where sample length is between ```input_min_text_length``` and ```input_max_text_length```\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "input_min_text_length, input_max_text_length = 2, 8" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Create a ```LengthSampler``` object\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "input_size = LengthSampler(input_min_text_length, input_max_text_length)\n", - "input_size" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This code uses the input_size object, an instance of ```LengthSampler```, to sample and print a random text length between 2 and 8 for each of 10 iterations.\"\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "for i in range(10):\n", - " size=input_size()\n", - " print(f\"sample {i} has length {size}\\n\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, you will need to sample tokens and obtain tokenized indexes. Let's verify this process with one sample.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sample=ds[0]\n", - "sample" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, tokenize the ```review``` text into input IDs, truncate the tokenized sequence to the desired length, and assign it to ```input_ids```\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sample[\"input_ids\"] = tokenizer.encode(sample[\"review\"])[: input_size()]\n", - "sample[\"input_ids\"]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Decode the truncated input IDs back into text and assign it to 'query', this is a will need the raw text for the reward fuction.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sample[\"query\"] = tokenizer.decode(sample[\"input_ids\"])\n", - "sample[\"query\"] " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this function, combine the process of tokenizing the 'review' text, truncating it to the desired length, and decoding it back to text. This allows you to apply it to the dataset.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def tokenize(sample):\n", - " sample[\"input_ids\"] = tokenizer.encode(sample[\"review\"])[: input_size()]\n", - " sample[\"query\"] = tokenizer.decode(sample[\"input_ids\"])\n", - " return sample" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can apply ```tokenize``` function to the dataset\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds = ds.map(tokenize, batched=False)\n", - "ds.set_format(type=\"torch\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: you can safely ignore the above warning.\n", - "You can see the sample before and after:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ds[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can now iterate over the dataset, printing the first 5 samples with their 'review' and the added 'input_ids', and 'query' :\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "for i, sample in enumerate(ds):\n", - " if i >= 5:\n", - " break\n", - " print(f\"Sample {i+1}:\")\n", - " print(f\"Review: {sample['review']}\")\n", - " print(f\"Input IDs: {sample['input_ids']}\")\n", - " print(f\"Query: {sample['query']}\")\n", - " print(\"-\" * 50)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The ```build_dataset``` function incorporates the necessary steps to build a dataset object for use as an input to ```PPOTrainer```. You will then reinstantiate the dataset object.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "del(ds)\n", - "dataset_name=\"imdb\"\n", - "ds = load_dataset(dataset_name, split=\"train\")\n", - "ds = ds.rename_columns({\"text\": \"review\"})" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def build_dataset(config, dataset_name=\"imdb\", input_min_text_length=2, input_max_text_length=8,tokenizer=tokenizer):\n", - " \"\"\"\n", - " Build dataset for training. This builds the dataset from `load_dataset`, one should\n", - " customize this function to train the model on its own dataset.\n", - "\n", - " Args:\n", - " dataset_name (`str`):\n", - " The name of the dataset to be loaded.\n", - "\n", - " Returns:\n", - " dataloader (`torch.utils.data.DataLoader`):\n", - " The dataloader for the dataset.\n", - " \"\"\"\n", - " \n", - " tokenizer = AutoTokenizer.from_pretrained(config.model_name)\n", - " tokenizer.pad_token = tokenizer.eos_token\n", - " # load imdb with datasets\n", - " ds = load_dataset(dataset_name, split=\"train\")\n", - " ds = ds.rename_columns({\"text\": \"review\"})\n", - " ds = ds.filter(lambda x: len(x[\"review\"]) > 200, batched=False)\n", - "\n", - " input_size = LengthSampler(input_min_text_length, input_max_text_length)\n", - "\n", - " def tokenize(sample):\n", - " sample[\"input_ids\"] = tokenizer.encode(sample[\"review\"])[: input_size()]\n", - " sample[\"query\"] = tokenizer.decode(sample[\"input_ids\"])\n", - " return sample\n", - "\n", - " ds = ds.map(tokenize, batched=False)\n", - " ds.set_format(type=\"torch\")\n", - " return ds" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Create the dataset object \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset = build_dataset(config)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see each sample has ```input_ids``` and ```query```\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dataset[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Collator function \n", - "The collator function is crucial for preparing data batches in a format suitable for the PPOTrainer. It ensures that each feature from the data samples is grouped together,\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def collator(data):\n", - " return dict((key, [d[key] for d in data]) for key in data[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The collator function is best understood with an example. You can input two samples each with 'input_ids', 'query', and 'review'.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "data = [\n", - " {'input_ids': [1, 2, 3, 4], 'query': \"sample text\", 'review': \"This is a sample review.\"},\n", - " {'input_ids': [5, 6, 7, 8], 'query': \"another sample\", 'review': \"Another sample review.\"}\n", - "]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Apply the collator function to the above data\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch = collator(data)\n", - "batch" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, 'input_ids', 'query', and 'review' each have their corresponding samples.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialize PPOTrainer \n", - "\n", - "Proximal Policy Optimization (PPO) is a reinforcement learning algorithm that is particularly well-suited for training generative models, including those used for chatbots. It helps address specific challenges in training these models, such as maintaining coherent and contextually appropriate dialogues.\n", - "\n", - "Proximal Policy Optimization (PPO) improves policy gradient methods for chatbots by using a clipped objective function, which ensures gradual and stable policy updates. This helps maintain consistent dialogue quality. Traditional policy gradient methods can lead to high variance and instability, resulting in inconsistent chatbot behavior. PPO's trust region balances exploring new responses and exploiting known good ones, making it more reliable for training chatbots. \n", - "\n", - "The PPO Trainer collects dialogue samples, optimizes the chatbot's policy based on these samples, and manages the neural network models. This ensures stable and efficient training, leading to high-quality, coherent, and contextually appropriate chatbot responses. \n", - "\n", - "Lets Initialize PPOTrainer with specified configuration and components\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```config``` : Configuration settings for PPO training, such as learning rate and model name\n", - "\n", - "```model``` : The primary model to be fine-tuned using PPO\n", - "\n", - "```tokenizer```:Tokenizer corresponding to the model, used for processing input text\n", - "\n", - "```dataset```: Dataset to be used for training, providing the input data for the model\n", - "\n", - "```data_collator```: Data collator to handle batching and formatting of the input data\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ppo_trainer = PPOTrainer(config, model, ref_model, tokenizer, dataset=dataset, data_collator=collator)\n", - "print(\"ppo_trainer object \",ppo_trainer)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please ignore above warnings as the trl version you installed supports this module.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Determine the appropriate device (CPU or GPU) for training with the PPO Trainer.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "device = ppo_trainer.accelerator.device\n", - "if ppo_trainer.accelerator.num_processes == 1:\n", - " device = 0 if torch.cuda.is_available() else \"cpu\" \n", - "print(device)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Reward function\n", - "\n", - "In reinforcement learning with PPO (Proximal Policy Optimization), a reward function is used to provide feedback on the quality of the actions taken by the policy. For a generative model like a chatbot, the reward function can evaluate the quality of the generated responses. Here’s how the sentiment analysis pipeline can be used as a reward function:\n", - "\n", - "In reinforcement learning with PPO, the sentiment analysis pipeline serves as a reward function to evaluate a chatbot's responses. By analyzing the sentiment of each response and assigning a reward based on the sentiment score, the PPO Trainer can optimize the chatbot’s policy to generate more positively received and engaging responses. This approach leverages sentiment analysis to provide meaningful feedback, guiding the chatbot towards improved performance in dialogue generation. Although not a typical reward model, it allows you to train the chatbot in a simple and effective way.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "First, let's initialize a sentiment analysis pipeline using a pretrained model fine-tuned on IMDB reviews.\n", - "The model predicts the sentiment of text inputs, providing scores for positive and negative sentiments.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sentiment_pipe = pipeline(\"sentiment-analysis\", model=\"lvwerra/distilbert-imdb\", device=device)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You'll get the sentiment value as negative here.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "text = \"this movie was really bad!!\"\n", - "sentiment_pipe(text, **sent_kwargs)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `score` key represents the model's confidence in its prediction. Higher score values indicate greater confidence in the sentiment classification, such as \"POSITIVE\" or \"NEGATIVE\". Thus, the value for `POSITIVE` class can be used to determine the reward values. For example, a high score for \"POSITIVE\" means the model is confident, which can increase rewards. Conversely, if the model isn’t confident that a review is positive, it results in a negative reward, lowering the total reward. This means negative sentiment reviews decrease the overall reward, while positive ones increase it.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "text = \"this movie was really good!!\"\n", - "sentiment_pipe(text, **sent_kwargs)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Generating responses using PPO \n", - "\n", - "### Tokenizing and preparing the input batch\n", - "This section of code demonstrates how to generate responses using the PPO (Proximal Policy Optimization) Trainer. The process involves tokenizing the input, preparing the batch for training, generating responses, and decoding the generated tokens into readable text.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The code first retrieves a batch of data from the PPO Trainer's dataloader and selects the first two entries for processing.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch = next(iter(ppo_trainer.dataloader))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The batch contains ```label```, ```input_ids```, and ```query```\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch.keys()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now let's create a new batch containing only the first two samples from the original batch \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Let's take the first two sample in the batch\n", - "batch = {key: batch[key][0:2] for key in batch}\n", - "batch" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Initialize a list of ```response_tensors``` to store the responses for scoring\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "response_tensors = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code extracts the `input_ids` from the `batch` and assigns them to `query_tensors`. These tensors represent the tokenized input sequences that will be used in the subsequent steps. They are called \"query tensors\" because they represent the initial input queries that will be processed by the model to generate responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "query_tensors = batch[\"input_ids\"]\n", - "query_tensors" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The below code defines a lambda function `get_text` that takes a list of responses (`response`) and decodes each tensor in the list using the tokenizer, converting the tensor back to readable text. The `squeeze()` method is used to remove any dimensions of size 1 from the tensor.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "get_text = lambda response:''.join([tokenizer.decode(r.squeeze()) for r in response])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see the original input queries in their text form.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "get_text(query_tensors)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "\n", - "The dictionary `generation_kwargs` sets the parameters for generating a sequence from the LLM (Language Model). The parameters include:\n", - "- `\"min_length\": -1` - No minimum length for the generated text.\n", - "- `\"top_k\": 0.0` - No filtering of the top-k most probable tokens.\n", - "- `\"top_p\": 1.0` - No nucleus sampling, using the entire distribution.\n", - "- `\"do_sample\": True` - Enables sampling, allowing for varied responses.\n", - "- `\"pad_token_id\": 50256` - ID of the padding token, ensuring uniform length across sequences.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "generation_kwargs = {\n", - " \"min_length\": -1,\n", - " \"top_k\": 0.0,\n", - " \"top_p\": 1.0,\n", - " \"do_sample\": True,\n", - " \"pad_token_id\": 50256,\n", - "}\n", - "generation_kwargs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `output_length_sampler` is initialized with `LengthSampler(output_min_length, output_max_length)`. This object is used to sample output lengths for the generated sequences, ensuring they fall within the specified minimum and maximum length range. By varying the lengths, you can produce more diverse and natural outputs from the language model, preventing the generation of overly short or excessively long sequences and enhancing the overall quality of the responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "output_min_length = 4\n", - "output_max_length = 16\n", - "output_length_sampler = LengthSampler(output_min_length, output_max_length)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The code calls the `output_length_sampler` to determine a length for the generated sequences. The sampled length is then stored in the variable `gen_len`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "gen_len = output_length_sampler()\n", - "gen_len " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, set the `max_new_tokens` parameter in the `generation_kwargs` dictionary to the value of `gen_len`, which was sampled from `output_length_sampler`. This ensures that the maximum number of new tokens generated by the language model is within the desired length range, promoting more controlled and appropriately lengthened responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "generation_kwargs[\"max_new_tokens\"] = gen_len\n", - "generation_kwargs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, let's process one sample using PPO. Start by extracting the first query tensor.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "query=query_tensors[0]\n", - "query" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Lets generate a response for the extracted query using the PPO trainer with the specified generation parameters (generation_kwargs). The generated response tensor is stored in ```response```.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "response = ppo_trainer.generate(query, **generation_kwargs)\n", - "response " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: You can safely ignore the above warning\n", - "\n", - "You can print the decoded text of the query and response tensors using the get_text function, converting the generated response back into a human-readable format. This demonstrates how the model has appended some text to the original query.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(\"query:\",get_text(query))\n", - "print(\"response:\", get_text(response))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, append the tokens of the ```response_tensors``` list. The ```squeeze()``` method removes any single-dimensional entries from the shape of the tensor, and the slicing``` [-gen_len:]``` ensures only the newly generated tokens are included, ignoring any preceding tokens.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "response_tensors.append(response.squeeze()[-gen_len:])\n", - "print(\"newly generated tokens form response:\", get_text(response_tensors[-gen_len:]))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Repeat the process for the second sample. This section generates a response for a given query, decodes the relevant part, and appends it to the `response_tensors` list.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "query=query_tensors[1]\n", - "gen_len = output_length_sampler()\n", - "generation_kwargs[\"max_new_tokens\"] = gen_len\n", - "response = ppo_trainer.generate(query, **generation_kwargs)\n", - "tokenizer.decode(response.squeeze()[-gen_len:], skip_special_tokens=True)\n", - "print(\"query:\",get_text(query))\n", - "print(\"response ouput :\", get_text(response_tensors))\n", - "response_tensors.append(response.squeeze()[-gen_len:])\n", - "print(\"newly generated tokens form response:\", get_text(response_tensors[-gen_len:]))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Convert each tensor in `response_tensors` into human-readable text and store it in the `batch` dictionary under the key `response`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch[\"response\"] = [tokenizer.decode(r.squeeze()) for r in response_tensors]\n", - "batch[\"response\"]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The batch now contains both `response` and `query` keys.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Scoring function \n", - "\n", - "Next, prepare the text data for sentiment analysis, which can be a part of a reward function in a PPO setup where the sentiment analysis of interactions helps determine the reward.\n", - "\n", - "Now, extract the `query` and `response` tensors and add them to the batch.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "texts = [q + r for q, r in zip(batch[\"query\"], batch[\"response\"])]\n", - "texts" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The sentiment scores (`pipe_outputs`) can be used as feedback to update the policy\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "pipe_outputs = sentiment_pipe(texts, **sent_kwargs)\n", - "pipe_outputs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "These scores can be used to evaluate the quality or relevance of the generated responses, indicating the model's confidence in the likelihood of the responses being positive. The scores for the generated responses are extracted from the `pipe_outputs` list. Each element in `pipe_outputs` contains a list of scores corresponding to the model's output.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This line iterates over the `pipe_outputs` list, extracts the score from each output, converts it into a tensor, and stores it in the `rewards` list. The scores represent the model's confidence in the likelihood of the responses being positive sentences.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "positive_scores = [\n", - " item[\"score\"]\n", - " for output in pipe_outputs\n", - " for item in output\n", - " if item[\"label\"] == \"POSITIVE\"\n", - "]\n", - "rewards = [torch.tensor(score) for score in positive_scores]\n", - "rewards" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Proximal policy optimization \n", - "\n", - "The training loop is responsible for performing a single update step of the PPO algorithm. The inputs to this process are the query, response, and score tensors.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(\"query:\", get_text(query_tensors))\n", - "print(\"\\n\")\n", - "print(\"response:\", get_text(response_tensors))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To meet the PPO trainer's minimum batch size requirement of 128, you can pad the response tensors with additional sample.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "batch_size=128\n", - "pad_token_id = tokenizer.pad_token_id\n", - "\n", - "query_tensors = pad_list_to_batch_size(query_tensors, batch_size, pad_token_id)\n", - "\n", - "response_tensors = pad_list_to_batch_size(response_tensors, batch_size, pad_token_id)\n", - "rewards=rewards+[torch.tensor(0) for _ in range(batch_size-len(rewards))]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, call the PPO `step` method that updates the model using the PPO algorithm with `query_tensors`, `response_tensors`, and `rewards`.\n", - "\n", - "- It uses these inputs to calculate the policy and value function losses.\n", - "- It computes the gradients and updates the policy network parameters to improve the policy.\n", - "- It ensures that the policy update stays within a certain range to avoid large policy shifts, which is a core aspect of PPO.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "*Note: The following code is commented out to prevent the kernel from crashing due to the absence of a GPU in the current environment. To execute this code, please download the notebook and run it in an environment equipped with a GPU. Simply uncomment the code before running it.*\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# stats = ppo_trainer.step(query_tensors, response_tensors, rewards)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `stats` variable is a dictionary containing various statistics from the PPO training step. You can print out its keys using the function `print_ppo_stats`. These keys can be organized into two main categories:\n", - "\n", - "- **Minimizing the language model loss**: `related_to_objective=True`\n", - " - This includes statistics related to optimizing the model parameters, such as policy loss and value loss.\n", - "\n", - "- **Calculating the reward**:\n", - " - This involves metrics more relevant to reinforcement learning, such as advantage estimates and reward calculations.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# stats.keys()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# print_ppo_stats(stats, related_to_objective = True)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# print_ppo_stats(stats)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "all_stats = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `sentiment`should be set to NEGATIVE for bad responses and POSITIVE for good responses score .\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sentiment = \"POSITIVE\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "\n", - "This code snippet represents a training loop for the PPO (Proximal Policy Optimization) algorithm using sentiment analysis. The loop iterates over batches of data from the `ppo_trainer` dataloader and performs the following steps:\n", - "\n", - "1. **Extract query tensors**:\n", - " - The input IDs (query tensors) are extracted from the batch.\n", - "\n", - "2. **Generate responses**:\n", - " - For each query tensor, a response is generated using the `ppo_trainer.generate` method with the specified `generation_kwargs`.\n", - " - The responses are then decoded and added to the batch under the `response` key.\n", - "\n", - "3. **Compute sentiment scores**:\n", - " - Text data is prepared by concatenating queries and responses.\n", - " - Sentiment analysis is performed on the combined texts to compute the sentiment scores.\n", - " - The scores are converted into tensors and stored in the `rewards` list.\n", - "\n", - "4. **Run PPO step**:\n", - " - The `ppo_trainer.step` method is called to update the model using the PPO algorithm with the `query_tensors`, `response_tensors`, and `rewards`.\n", - " - This step calculates the policy and value function losses, computes gradients and updates the policy network parameters.\n", - " - The policy update ensures it stays within a certain range to avoid large policy shifts.\n", - "\n", - "5. **Logging statistics**:\n", - " - The statistics from the PPO training step are logged and stored in the `all_stats` list.\n", - " \n", - "**Note:** Training the model on a CPU will be very time-consuming. You have pretrained the model using a GPU and saved it for your convenience. You can skip the training part and proceed to the next block of code and load the saved model. You can uncomment the below block of code to train the model yourself.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# for epoch, batch in tqdm(enumerate(ppo_trainer.dataloader)):\n", - "# query_tensors = batch[\"input_ids\"]\n", - "# print(f\"epoch {epoch}\")\n", - "\n", - "# #### Get response from gpt2\n", - "# response_tensors = []\n", - "# for query in query_tensors:\n", - "# gen_len = output_length_sampler()\n", - "# generation_kwargs[\"max_new_tokens\"] = gen_len\n", - "# response = ppo_trainer.generate(query, **generation_kwargs)\n", - "# response_tensors.append(response.squeeze()[-gen_len:])\n", - "# batch[\"response\"] = [tokenizer.decode(r.squeeze()) for r in response_tensors]\n", - "\n", - "# #### Compute sentiment score\n", - "# texts = [q + r for q, r in zip(batch[\"query\"], batch[\"response\"])]\n", - "# pipe_outputs = sentiment_pipe(texts, **sent_kwargs)\n", - "# positive_scores = [\n", - "# item[\"score\"]\n", - "# for output in pipe_outputs\n", - "# for item in output\n", - "# if item[\"label\"] == sentiment\n", - "# ]\n", - "# rewards = [torch.tensor(score) for score in positive_scores]\n", - "\n", - "# #### Run PPO step\n", - "# stats = ppo_trainer.step(query_tensors, response_tensors, rewards)\n", - "# ppo_trainer.log_stats(stats, batch, rewards)\n", - " \n", - "# all_stats.append(stats)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# # Save the model\n", - "\n", - "# model_dir = \"ppo-good\"\n", - "# os.makedirs(model_dir, exist_ok=True)\n", - "\n", - "# # Save model configuration and weights\n", - "# model_1.save_pretrained(model_dir)\n", - "# tokenizer.save_pretrained(model_dir)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/gSWo8GeztngSmzHpqX_RaQ/ppo-good.pkl\n", - "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/we8t5N-45dVq3VhxGwYRAg/ppo-good-tar.gz" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# File name\n", - "file_name = \"ppo-good-tar.gz\"\n", - "\n", - "# Open the tar.gz file\n", - "with tarfile.open(file_name, \"r:gz\") as tar:\n", - " # Extract all the contents into the current directory\n", - " tar.extractall()\n", - "\n", - "print(\"Extraction completed.\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "model_dir = \"ppov3new1\"\n", - "model_1 = AutoModelForCausalLMWithValueHead.from_pretrained(model_dir)\n", - "tokenizer = AutoTokenizer.from_pretrained(model_dir)\n", - "\n", - "# Load training stats\n", - "file_name = \"ppo-good.pkl\"\n", - "with open(file_name, 'rb') as f:\n", - " all_stats = pickle.load(f)\n", - "\n", - "model_1.to(device)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: You can safely ignore the above warning.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Plotting PPO training loss and mean \n", - "\n", - "1. **Extracting values**:\n", - " - `loss_values`: Total loss values from `all_stats`.\n", - " - `reward_values`: Mean reward values from `all_stats`.\n", - "\n", - "2. **Plotting the loss**:\n", - " - Line plot of total loss over epochs.\n", - "\n", - "3. **Plotting the rewards**:\n", - " - Line plot of mean reward over epochs.\n", - "\n", - "4. **Displaying the plots**:\n", - " - Arrange and show the plots using `plt.tight_layout()` and `plt.show()`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "loss_values = [stat['ppo/loss/total'] for stat in all_stats]\n", - "reward_values = [stat['ppo/mean_scores'] for stat in all_stats]\n", - "\n", - "# Plotting the loss\n", - "plt.figure(figsize=(12, 6))\n", - "plt.subplot(2, 1, 1)\n", - "plt.plot(loss_values, label='Total Loss', color='b')\n", - "plt.xlabel('Epoch')\n", - "plt.ylabel('Loss')\n", - "plt.title('PPO Training Loss over Time')\n", - "plt.legend()\n", - "plt.grid(True)\n", - "\n", - "# Plotting the rewards\n", - "plt.subplot(2, 1, 2)\n", - "plt.plot(reward_values, label='Mean Reward', color='g')\n", - "plt.xlabel('Epoch')\n", - "plt.ylabel('Reward')\n", - "plt.title('PPO Mean Reward over Time')\n", - "plt.legend()\n", - "plt.grid(True)\n", - "\n", - "# Show the plots\n", - "plt.tight_layout()\n", - "plt.show() " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Generating and analyzing text with PPO and reference models\n", - "**Device Setup**:\n", - " - Determine if CUDA is available and set the device accordingly.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", - "# Set the pipeline device\n", - "pipeline_device = 0 if device.type == \"cuda\" else -1" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Text generation function**:\n", - " - `generate_some_text(input_text, my_model)`: Tokenizes input text, generates a response, and decodes it.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "gen_kwargs = {\"min_length\": -1, \"max_new_tokens\":20, \"top_k\": 0.0, \"top_p\": 1.0, \"do_sample\": True, \"pad_token_id\": tokenizer.eos_token_id}\n", - "def generate_some_text(input_text,my_model):\n", - "# Tokenize the input text\n", - " input_ids = tokenizer(input_text, return_tensors='pt').input_ids.to(device)\n", - " generated_ids = my_model.generate(input_ids,**gen_kwargs )\n", - "\n", - " # Decode the generated text\n", - " generated_text_ = tokenizer.decode(generated_ids[0], skip_special_tokens=True)\n", - "\n", - " return generated_text_" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Generate text with PPO model**:\n", - " - Generate text using the PPO-trained model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "input_text = \"Once upon a time in a land far\"\n", - "\n", - "generated_text=generate_some_text(input_text,model_1)\n", - "generated_text" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Sentiment Analysis**:\n", - " - Analyze the sentiment of the generated text using `sentiment_pipe`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "pipe_outputs = sentiment_pipe(generated_text, **sent_kwargs)\n", - "pipe_outputs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Generate text with reference model**:\n", - " - Generate text using the reference model.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "generated_text = generate_some_text(input_text,ref_model)\n", - "generated_text" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Comparing PPO and reference models on \n", - "\n", - "1. **Generation Parameters**:\n", - " - Define `gen_kwargs` for text generation.\n", - "\n", - "2. **Prepare Batch**:\n", - " - Sample a batch of size `bs` from the dataset and extract query tensors.\n", - "\n", - "3. **Generate Responses**:\n", - " - For each query tensor, generate responses using both the reference model and the PPO model.\n", - "\n", - "4. **Decode Responses**:\n", - " - Decode the generated response tensors into human-readable text.\n", - "\n", - "5. **Compute Sentiment Scores**:\n", - " - Prepare texts by concatenating queries and responses.\n", - " - Compute sentiment scores for the responses before and after training using `sentiment_pipe`.\n", - "\n", - "6. **Store Results**:\n", - " - Store queries, responses, and sentiment scores in `game_data`.\n", - " - Convert `game_data` into a DataFrame and return it.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def compare_models_on_dataset(model, ref_model, dataset, tokenizer, sentiment_pipe, sent_kwargs, device, output_length_sampler):\n", - " gen_kwargs = {\n", - " \"min_length\": -1, \n", - " \"top_k\": 0.0, \n", - " \"top_p\": 1.0, \n", - " \"do_sample\": True, \n", - " \"pad_token_id\": tokenizer.eos_token_id\n", - " }\n", - " \n", - " bs = 16\n", - " game_data = dict()\n", - " dataset.set_format(\"pandas\")\n", - " df_batch = dataset[:].sample(bs)\n", - " game_data[\"query\"] = df_batch[\"query\"].tolist()\n", - " query_tensors = df_batch[\"input_ids\"].tolist()\n", - "\n", - " response_tensors_ref, response_tensors = [], []\n", - "\n", - " # Get maximum position embeddings for both models\n", - " max_position_embeddings_ref = ref_model.config.max_position_embeddings\n", - " max_position_embeddings_model = model.config.max_position_embeddings\n", - "\n", - " for i in range(bs):\n", - " gen_len = output_length_sampler()\n", - "\n", - " # Convert query tensors to input IDs\n", - " input_ids = torch.tensor(query_tensors[i]).unsqueeze(dim=0).to(device)\n", - "\n", - " # ********** Process for ref_model **********\n", - " total_length_ref = input_ids.shape[-1] + gen_len\n", - " if total_length_ref > max_position_embeddings_ref:\n", - " # Truncate input_ids to fit within the max length\n", - " max_input_length_ref = max_position_embeddings_ref - gen_len\n", - " input_ids_ref = input_ids[:, -max_input_length_ref:]\n", - " total_length_ref = input_ids_ref.shape[-1] + gen_len\n", - " else:\n", - " input_ids_ref = input_ids\n", - " \n", - " output = ref_model.generate(\n", - " torch.tensor(query_tensors[i]).unsqueeze(dim=0).to(device), \n", - " max_new_tokens=gen_len, \n", - " **gen_kwargs\n", - " ).squeeze()[-gen_len:]\n", - " response_tensors_ref.append(output)\n", - "\n", - " # ********** Process for model **********\n", - " total_length_model = input_ids.shape[-1] + gen_len\n", - " if total_length_model > max_position_embeddings_model:\n", - " max_input_length_model = max_position_embeddings_model - gen_len\n", - " input_ids_model = input_ids[:, -max_input_length_model:]\n", - " total_length_model = input_ids_model.shape[-1] + gen_len\n", - " else:\n", - " input_ids_model = input_ids\n", - " \n", - " output = model.generate(\n", - " torch.tensor(query_tensors[i]).unsqueeze(dim=0).to(device), \n", - " max_new_tokens=gen_len, \n", - " **gen_kwargs\n", - " ).squeeze()[-gen_len:]\n", - " response_tensors.append(output)\n", - "\n", - " game_data[\"response (before)\"] = [tokenizer.decode(response_tensors_ref[i]) for i in range(bs)]\n", - " game_data[\"response (after)\"] = [tokenizer.decode(response_tensors[i]) for i in range(bs)]\n", - "\n", - " texts_before = [q + r for q, r in zip(game_data[\"query\"], game_data[\"response (before)\"])]\n", - " game_data[\"rewards (before)\"] = [output[1][\"score\"] for output in sentiment_pipe(texts_before, **sent_kwargs)]\n", - "\n", - " texts_after = [q + r for q, r in zip(game_data[\"query\"], game_data[\"response (after)\"])]\n", - " game_data[\"rewards (after)\"] = [output[1][\"score\"] for output in sentiment_pipe(texts_after, **sent_kwargs)]\n", - "\n", - " df_results = pd.DataFrame(game_data)\n", - " return df_results" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "df_results = compare_models_on_dataset(model_1, ref_model, dataset, tokenizer, sentiment_pipe, sent_kwargs, device, output_length_sampler)\n", - "df_results" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Running the PPO model with negative sentiment\n", - "\n", - "This code runs the PPO training loop with the sentiment set to NEGATIVE, which evaluates the model's performance when negative sentiment scores are prioritized. The training loop generates responses, computes sentiment scores, updates the model, and logs the statistics for each epoch.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sentiment = \"NEGATIVE\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# for epoch, batch in tqdm(enumerate(ppo_trainer.dataloader)):\n", - "# query_tensors = batch[\"input_ids\"]\n", - "# print(f\"epoch {epoch}\")\n", - "\n", - "# #### Get response from gpt2\n", - "# response_tensors = []\n", - "# for query in query_tensors:\n", - "# gen_len = output_length_sampler()\n", - "# generation_kwargs[\"max_new_tokens\"] = gen_len\n", - "# response = ppo_trainer.generate(query, **generation_kwargs)\n", - "# response_tensors.append(response.squeeze()[-gen_len:])\n", - "# batch[\"response\"] = [tokenizer.decode(r.squeeze()) for r in response_tensors]\n", - "\n", - "# #### Compute sentiment score\n", - "# texts = [q + r for q, r in zip(batch[\"query\"], batch[\"response\"])]\n", - "# pipe_outputs = sentiment_pipe(texts, **sent_kwargs)\n", - "# negative_scores = [\n", - "# item[\"score\"]\n", - "# for output in pipe_outputs\n", - "# for item in output\n", - "# if item[\"label\"] == sentiment\n", - "# ]\n", - "# rewards = [torch.tensor(score) for score in negative_scores]\n", - "\n", - "# #### Run PPO step\n", - "# stats = ppo_trainer.step(query_tensors, response_tensors, rewards)\n", - "# ppo_trainer.log_stats(stats, batch, rewards)\n", - " \n", - "# all_stats.append(stats)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# # Save the model\n", - "\n", - "# model_dir = \"ppo-bad\"\n", - "# os.makedirs(model_dir, exist_ok=True)\n", - "\n", - "# # Save model configuration and weights\n", - "# model_0.save_pretrained(model_dir)\n", - "# tokenizer.save_pretrained(model_dir)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Note:** Training the model on a CPU will be very time-consuming. The model has been pretrained using a GPU and saved for your convenience. You can skip the training part, proceed to the next block of code, and load the saved model. You can also uncomment the above training block of code to train the model yourself.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/8zCp__SHRSgGVlf5yP50Ag/ppo-bad-tar.gz\n", - "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/jMW99Z9mvxesgYR-H6y6Yw/ppo-bad.pkl" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import tarfile\n", - "# File name\n", - "file_name = \"ppo-bad-tar.gz\"\n", - "\n", - "# Open the tar.gz file\n", - "with tarfile.open(file_name, \"r:gz\") as tar:\n", - " # Extract all the contents into the current directory\n", - " tar.extractall()\n", - "\n", - "print(\"Extraction completed.\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import tarfile\n", - "model_dir = \"ppov3new_bad1\"\n", - "model_0 = AutoModelForCausalLMWithValueHead.from_pretrained(model_dir)\n", - "tokenizer = AutoTokenizer.from_pretrained(model_dir)\n", - "\n", - "# Load training stats\n", - "file_name = \"ppo-bad.pkl\"\n", - "with open(file_name, 'rb') as f:\n", - " all_stats = pickle.load(f)\n", - "\n", - "model_0.to(device)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - ">Note: You can safely ignore the above warning.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Comparing models with negative sentiment\n", - "\n", - "The below code compares the performance of the PPO-trained model (`model_0`) and the reference model on the given dataset. The `compare_models_on_dataset` function generates responses from both models, computes their sentiment scores, and returns the results in a DataFrame (`df_results`). This comparison helps evaluate how well the PPO-trained model performs in generating positive responses when the `sentiment` is set to NEGATIVE.\n", - "\n", - "Since the dataset is fairly large, we will only use a subset of the dataset for testing.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "df_results = compare_models_on_dataset(model_0, ref_model, dataset, tokenizer, sentiment_pipe, sent_kwargs, device, output_length_sampler)\n", - "df_results" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Exercise: Comparing PPO models\n", - "\n", - "In this exercise, you will compare the performance of two PPO-trained models (`model_0` and `model_1`) using the `compare_models_on_dataset` function and note the difference in performance of both.\n", - "\n", - "**Compare Models**:\n", - " - Use the `compare_models_on_dataset` function to compare `model_0` and `model_1`.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Write your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "
\n", - " Click here for Solution\n", - "\n", - "```python\n", - "df_results = compare_models_on_dataset(model_0, model_1, dataset, tokenizer, sentiment_pipe, sent_kwargs, device, output_length_sampler)\n", - "df_results\n", - "```\n", - "\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Authors\n", - "\n", - "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n", - "\n", - "[Ashutosh Sagar](https://www.linkedin.com/in/ashutoshsagar/) is completing his MS in CS from Dalhousie University. He has previous experience working with Natural Language Processing and as a Data Scientist.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Contributors\n", - "\n", - "[Hailey Quach](https://author.skills.network/instructors/hailey_quach) is a Data Scientist at IBM. She's completing her Bsc, Honors in Computer Science at Concordia University, Montreal.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## References\n", - "\n", - "\n", - "[TEXT CLASSIFICATION WITH THE TORCHTEXT LIBRARY](https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html)\n", - "\n", - "[Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751.pdf)\n", - "\n", - "[Simple, Scalable Adaptation for Neural Machine Translation](https://arxiv.org/pdf/1909.08478)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```{## Change Log}\n", - "```\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```{|Date (YYYY-MM-DD)|Version|Changed By|Change Description||-|-|-|-||2024-06-27|0.1|Kang Wang|Create the lab|}\n", - "```\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "© Copyright IBM Corporation. All rights reserved.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.19" - }, - "prev_pub_hash": "febcb0ff319ab930e46d30d4d1bc1329ad2f8aa613c9a5ec96659fa44d3daf95" - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/notebooks/LLM_Specialization/LoRA_with_Pytorch.ipynb b/notebooks/LLM_Specialization/LoRA_with_Pytorch.ipynb new file mode 100644 index 0000000..7aa576d --- /dev/null +++ b/notebooks/LLM_Specialization/LoRA_with_Pytorch.ipynb @@ -0,0 +1,3421 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f157571e-3e1b-477c-a61f-c1e6c062ca6f", + "metadata": {}, + "source": [ + "

\n", + " \n", + " \"Skills\n", + " \n", + "

\n", + "\n", + "# LoRA with PyTorch\n", + "\n", + "Estimated time needed: **60** minutes\n", + "\n", + "As an AI engineer, you are tasked with fine-tuning a model for sentiment analysis on the IMDB dataset, starting with a model that is pretrained on the AG News dataset. By leveraging Low-Rank Adaptation (LoRA), the model is initially trained on AG News, benefiting from its extensive labeled data and broad categorization capabilities. This robust foundation enhances the model’s language understanding.\n", + "\n", + "Subsequently, LoRA is used to fine-tune the model on the IMDB dataset, adapting its knowledge to the nuances of movie reviews for sentiment analysis. This two-phase process — starting with AG News and refining with IMDB data — ensures that the model is both well-rounded and specialized, achieving superior performance in sentiment analysis tasks.\n", + "\n", + "**Note: If you are already familiar with training a model on the IMDB dataset, you can run the cells and then jump to the Low-Rank Adaptation (LoRA) section**\n", + "\n", + "\n", + "![Documents Overload](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-GPXX0Y15EN/docs.png)\n", + "\n", + "```Efficiency in parameter updates:``` LoRA introduces only a small fraction of additional parameters compared to the total number of parameters in a large model. This makes the training process faster and less resource-intensive because fewer parameters need to be updated during backpropagation.\n", + "\n", + "```Preservation of pretrained knowledge:``` By keeping the majority of the model's weights fixed and only adjusting them through low-rank matrices, LoRA helps preserve the rich representations that the model learned during pretraining. This is particularly beneficial for tasks that do not require drastic deviations from the behavior learned during pretraining.\n", + "\n", + "```Customization to specific tasks:``` Despite the minimal updates, the changes introduced by LoRA are significant enough to adapt the model to specific tasks. This lets you fine-tune large models on specialized tasks without the need for extensive retraining.\n", + "\n", + "```Reduction in overfitting:``` Because only a limited number of parameters are adapted, the risk of overfitting is lower compared to full model fine-tuning, especially when adapting to smaller datasets.\n", + "\n", + "```Scalability:``` LoRA scales well with model size. As models become larger, the relative increase in the number of parameters introduced by LoRA becomes even smaller, making it a particularly attractive option for adapting very large models.\n", + "\n", + "```Compatibility and simplicity:``` The method can be easily applied to different types of neural networks, especially those based on the transformer architecture. It doesn't require major changes to the existing architecture, which simplifies integration into existing pipelines.\n" + ] + }, + { + "cell_type": "markdown", + "id": "29f9693c-5bf2-4e3b-9974-ed709cf69670", + "metadata": {}, + "source": [ + "# __Table of Contents__\n", + "\n", + "
    \n", + "
  1. Objectives
    2. \n", + "
  2. \n", + " Setup\n", + "
      \n", + "
    1. Install required libraries
      2. \n", + "
    2. Import required libraries
      3. \n", + "
    3. Defining helper functions
      4. \n", + "
    \n", + "
    3. \n", + "
  3. \n", + " Data pipeline\n", + "
      \n", + "
    1. Tokenizer
      2. \n", + "
    \n", + "
    4. \n", + "
  4. \n", + " IMDB dataset
    5. \n", + "
      \n", + "
    1. Dataset composition
      2. \n", + "
    2. Applications
      3. \n", + "
    3. Challenges
      4. \n", + "
    4. Train and validate
      5. \n", + "
    5. Data loader
      6. \n", + "
    6. Neural network
      7. \n", + "
    \n", + " \n", + "
  5. \n", + " Train the model on the full dataset
    6. \n", + "
      \n", + "
    1. Train the model
      2. \n", + "
    \n", + " \n", + "
  6. \n", + " Low-Rank Adaptation (LoRA)
    7. \n", + "
      \n", + "
    1. LoRA
      2. \n", + "
    2. Rank
      3. \n", + "
    3. Understanding LoRA in PyTorch
      4. \n", + "
    4. Applying LoRA
      5. \n", + "
    5. Loading the model
      6. \n", + "
    \n", + " \n", + "
  7. \n", + " Exercise: Apply LoRA to a different network\n", + "
    8. \n", + "
\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "80470861-da2d-41b1-a0ee-1048d18a0857", + "metadata": {}, + "source": [ + "## Objectives\n", + "\n", + "After completing this lab you are able to:\n", + "\n", + "- Construct and train a neural network from the ground up\n", + "- Fine-tune a neural network in the conventional manner by unfreezing specific layers\n", + "- Use LoRA to fine-tune a neural network\n", + "- Comprehend the functions of LoRA and the reasons behind its effectiveness\n", + "- Save and load models that employ LoRA efficiently\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "8b717c75-4b99-4895-b615-6ba25b33fa47", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "### Install required libraries\n", + "\n", + "The following required libraries are __not__ preinstalled in the Skills Network Labs environment. __You must run the following cell__ to install them. Note that it can take between __5 and 10 minutes__ to install the required libraries:\n", + "\n", + "```bash\n", + "!pip install numpy==1.24.1\n", + "!pip install -U portalocker==2.8.2\n", + "!pip install torch==2.0.1\n", + "!pip install torchtext==0.15.2\n", + "!pip install torchdata==0.6.1\n", + "!pip install -U plotly==5.22.0\n", + "!pip install pandas==2.2.2\n", + "!pip install matplotlib==3.9.0\n", + "!pip install scikit-learn==1.5.0\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3068bedb-32c8-45f4-92c7-bbd77fc5754e", + "metadata": {}, + "source": [ + "### Import required libraries\n", + "\n", + "The following imports the required libraries:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "7759af5f-1618-4edb-955f-e424b5357054", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n", + "Tesla P40\n", + "Import Successfully!\n" + ] + } + ], + "source": [ + "# Standard library imports\n", + "import io\n", + "import math\n", + "import os\n", + "import pickle\n", + "import tarfile\n", + "import tempfile\n", + "from itertools import accumulate\n", + "from urllib.request import urlopen\n", + "\n", + "# Third-party imports\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas as pd\n", + "import plotly.graph_objs as go\n", + "import torch\n", + "import torch.nn as nn\n", + "import torch.nn.functional as F\n", + "import torchtext # ; torchtext.disable_torchtext_deprecation_warning()\n", + "from IPython.display import Markdown as md\n", + "from sklearn.manifold import TSNE\n", + "from torch.utils.data import DataLoader\n", + "from torch.utils.data.dataset import Dataset, random_split\n", + "from torchtext.data.functional import to_map_style_dataset\n", + "from torchtext.datasets import AG_NEWS\n", + "from torchtext.vocab import GloVe, Vectors, build_vocab_from_iterator\n", + "from tqdm import tqdm\n", + "\n", + "# Suppress warnings\n", + "def warn(*args, **kwargs):\n", + " pass\n", + "\n", + "import warnings\n", + "warnings.warn = warn\n", + "warnings.filterwarnings('ignore')\n", + "\n", + "# CUDA-related checks\n", + "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n", + "print(torch.cuda.is_available())\n", + "print(torch.cuda.get_device_name())\n", + "\n", + "print(\"Import Successfully!\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "65025fce-7345-4630-b541-71b5703b71f8", + "metadata": {}, + "source": [ + "### Defining helper functions\n", + "\n", + "The following are some helper functions to help with plotting, saving, and loading files. These functions are not the main focus of this lab, you do not have to dwell on these too long. However, do run the cells in this section to define these helper functions:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "aa667f44-288d-463f-9c22-2401e3277c88", + "metadata": {}, + "outputs": [], + "source": [ + "def plot(COST,ACC):\n", + " fig, ax1 = plt.subplots()\n", + " color = 'tab:red'\n", + " ax1.plot(COST, color=color)\n", + " ax1.set_xlabel('epoch', color=color)\n", + " ax1.set_ylabel('total loss', color=color)\n", + " ax1.tick_params(axis='y', color=color)\n", + "\n", + " ax2 = ax1.twinx()\n", + " color = 'tab:blue'\n", + " ax2.set_ylabel('accuracy', color=color) # You already handled the x-label with ax1\n", + " ax2.plot(ACC, color=color)\n", + " ax2.tick_params(axis='y', color=color)\n", + " fig.tight_layout() # otherwise the right y-label is slightly clipped\n", + "\n", + " plt.show()\n", + "\n", + "\n", + "\n", + "def save_list_to_file(lst, filename):\n", + " \"\"\"\n", + " Save a list to a file using pickle serialization.\n", + "\n", + " Parameters:\n", + " lst (list): The list to be saved.\n", + " filename (str): The name of the file to save the list to.\n", + "\n", + " Returns:\n", + " None\n", + " \"\"\"\n", + " with open(filename, 'wb') as file:\n", + " pickle.dump(lst, file)\n", + "\n", + "\n", + "def load_list_from_file(filename):\n", + " \"\"\"\n", + " Load a list from a file using pickle deserialization.\n", + "\n", + " Parameters:\n", + " filename (str): The name of the file to load the list from.\n", + "\n", + " Returns:\n", + " list: The loaded list.\n", + " \"\"\"\n", + " with open(filename, 'rb') as file:\n", + " loaded_list = pickle.load(file)\n", + " return loaded_list" + ] + }, + { + "cell_type": "markdown", + "id": "2239f5cc-059f-44ee-9bd0-f26f8880ad69", + "metadata": {}, + "source": [ + "---" + ] + }, + { + "cell_type": "markdown", + "id": "ad8914c5-a862-4573-adbe-94efa43a10d2", + "metadata": {}, + "source": [ + "## Data pipeline\n" + ] + }, + { + "cell_type": "markdown", + "id": "dddf985a-c832-457f-bfd5-7ac48e1187ad", + "metadata": {}, + "source": [ + "### Tokenizer\n", + "\n", + "A tokenizer takes as input a document and breaks it up into individual tokens. Now, you might wonder, what's a token?\n", + "This example might help you understand it better.\n", + "\n", + "Imagine a token as a puzzle piece of a jigsaw puzzle. Each word, number, or small part of a word is a token. When you tokenize a document, you break it into these puzzle pieces so that a computer can understand and work with the text more easily, just like how you solve a puzzle by arranging its pieces.\n", + "\n", + "First, import the **```get_tokenizer```** function from **```torchtext.data.utils```**.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "8a5f684b-e84f-4768-a152-bc51ff205ca9", + "metadata": {}, + "outputs": [], + "source": [ + "from torchtext.data.utils import get_tokenizer" + ] + }, + { + "cell_type": "markdown", + "id": "9c2b2a53-ed87-4110-b631-8a38f5be539b", + "metadata": {}, + "source": [ + "Next, we'll create the tokenizer. We'll set it to the \"basic_english\" tokenizer that is provided by `torchtext`. The \"basic_english\" tokenizer is designed to handle basic English text and splits the text into individual tokens based on spaces and punctuation marks.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "60bc3e17-5998-4cc0-9a86-10001ce78f4f", + "metadata": {}, + "outputs": [], + "source": [ + "tokenizer = get_tokenizer(\"basic_english\")" + ] + }, + { + "cell_type": "markdown", + "id": "98b0055f-f35c-4d2e-a565-e52723a493b7", + "metadata": {}, + "source": [ + "Our dataset is going to be an iterable. Therefore, We'll use a generator function **```yield_tokens```** to apply **```tokenizer```**. The purpose of the generator function **```yield_tokens```** is to yield tokenized texts one at a time. Instead of processing the entire dataset and returning all of the tokenized texts in one go, the generator function processes and yields each tokenized text individually as it is requested. The tokenization process is performed lazily, which means the next tokenized text is generated only when needed, saving memory and computational resources.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e25b8369-e601-4cbd-8345-b2d9bde1247d", + "metadata": {}, + "outputs": [], + "source": [ + "def yield_tokens(data_iter):\n", + " for _,text in data_iter:\n", + " yield tokenizer(text)" + ] + }, + { + "cell_type": "markdown", + "id": "2bddee85-4f64-4911-aa9c-e6145681f0d6", + "metadata": {}, + "source": [ + "The following loads a pretrained word embedding model called GloVe into a variable called `glove_embedding`:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "25aabfc6-0ed7-4749-a2f3-3c4693b5d96b", + "metadata": {}, + "outputs": [], + "source": [ + "# Note that GloVe embeddings are typically downloaded using:\n", + "#glove_embedding = GloVe(name=\"6B\", dim=100)\n", + "# However, the GloVe server is frequently down. The code below offers a workaround\n", + "\n", + "\n", + "class GloVe_override(Vectors):\n", + " url = {\n", + " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", + " }\n", + "\n", + " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", + " url = self.url[name]\n", + " name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", + " #name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", + " super(GloVe_override, self).__init__(name, url=url, **kwargs)\n", + "\n", + "class GloVe_override2(Vectors):\n", + " url = {\n", + " \"6B\": \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/tQdezXocAJMBMPfUJx_iUg/glove-6B.zip\",\n", + " }\n", + "\n", + " def __init__(self, name=\"6B\", dim=100, **kwargs) -> None:\n", + " url = self.url[name]\n", + " #name = \"glove.{}.{}d.txt\".format(name, str(dim))\n", + " name = \"glove.{}/glove.{}.{}d.txt\".format(name, name, str(dim))\n", + " super(GloVe_override2, self).__init__(name, url=url, **kwargs)\n", + "\n", + "try:\n", + " glove_embedding = GloVe_override(name=\"6B\", dim=100)\n", + "except:\n", + " try:\n", + " glove_embedding = GloVe_override2(name=\"6B\", dim=100)\n", + " except:\n", + " glove_embedding = GloVe(name=\"6B\", dim=100)" + ] + }, + { + "cell_type": "markdown", + "id": "eaf6bdf2-2052-4b45-8e90-0ee9ffefe01f", + "metadata": {}, + "source": [ + "The following builds a vocabulary object from a pretrained GloVe word embedding model and sets the default index to the token:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "98e18d96-f913-4af2-8b9d-40787ff47a67", + "metadata": {}, + "outputs": [], + "source": [ + "from torchtext.vocab import vocab\n", + "\n", + "vocab = vocab(glove_embedding .stoi, 0,specials=('', ''))\n", + "vocab.set_default_index(vocab[\"\"])" + ] + }, + { + "cell_type": "markdown", + "id": "793e527e-a76b-4acf-bb08-0d027bf8d391", + "metadata": {}, + "source": [ + "The following prepares the text processing pipeline with the tokenizer and vocabulary. The text pipeline will be used to process the raw data strings from the dataset iterators.\n", + "\n", + "The function **```text_pipeline```** first tokenizes the input text, following which **```vocab```** is applied to get the token indices.\n", + "\n", + "The function **```label_pipeline```** simply converts labels into their integer values.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "c2494315-f59a-4904-84b8-d00beee9e7c2", + "metadata": {}, + "outputs": [], + "source": [ + "def text_pipeline(x):\n", + " return vocab(tokenizer(x))\n", + "\n", + "def label_pipeline(x):\n", + " return int(x) " + ] + }, + { + "cell_type": "markdown", + "id": "f115c03b-5da3-4142-9ee4-33accb9ab0ba", + "metadata": {}, + "source": [ + "---" + ] + }, + { + "cell_type": "markdown", + "id": "5766072a-eed9-4dea-8aac-857c091613d9", + "metadata": {}, + "source": [ + "## IMDB dataset \n", + "\n", + "The following loads the IMDB dataset into a temporary folder. This might take some time, so please be patient.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "7a571d7b-69a1-4bed-b66c-06f98357e59c", + "metadata": {}, + "outputs": [], + "source": [ + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/35t-FeC-2uN1ozOwPs7wFg.gz')\n", + "tar = tarfile.open(fileobj=io.BytesIO(urlopened.read()))\n", + "tempdir = tempfile.TemporaryDirectory()\n", + "tar.extractall(tempdir.name)\n", + "tar.close()" + ] + }, + { + "cell_type": "markdown", + "id": "c93501ce-1ea0-4503-a2b4-aa5335121706", + "metadata": {}, + "source": [ + "The **IMDB dataset** contains movie reviews from the Internet Movie Database (IMDB) and is commonly used for binary sentiment classification tasks. It's a popular dataset for training and testing models in natural language processing (NLP), particularly in the context of sentiment analysis.\n", + "\n", + "### Dataset composition\n", + "\n", + "- **Reviews**: The dataset consists of 50,000 movie reviews, divided evenly into 25,000 training and 25,000 testing samples.\n", + "- **Sentiment labels**: Each review is labeled as either positive or negative, indicating the sentiment expressed in the review. The dataset is balanced, with an equal number of positive and negative reviews in both the training and testing sets.\n", + "- **Text content**: Reviews are presented as plain text and have been preprocessed to some extent. For example, HTML tags are removed, but the text retains its original punctuation and capitalization.\n", + "- **Usage**: The dataset is commonly used to train models for binary sentiment classification, where the goal is to predict whether a given review is positive or negative based on its text content.\n", + "\n", + "### Applications\n", + "\n", + "- **Sentiment analysis**: The primary application of the IMDB dataset is in sentiment analysis, where it serves as a benchmark for various text classification algorithms.\n", + "- **Natural language processing (NLP)**: The dataset is widely used in NLP research and applications, providing a basis for testing the effectiveness of different models and approaches in understanding human language.\n", + "\n", + "### Challenges\n", + "\n", + "The dataset is small, so it's hard to train a model from scratch.\n", + "\n", + "The following class is defined to traverse the IMDB dataset. The need to define this class arises from the fact that the IMDB dataset is split across a large number of files:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "f0b1ac02-3b52-4e4d-ac61-a33e3e429c0f", + "metadata": {}, + "outputs": [], + "source": [ + "class IMDBDataset(Dataset):\n", + " def __init__(self, root_dir, train=True):\n", + " \"\"\"\n", + " root_dir: The base directory of the IMDB dataset.\n", + " train: A boolean flag indicating whether to use training or test data.\n", + " \"\"\"\n", + " self.root_dir = os.path.join(root_dir, \"train\" if train else \"test\")\n", + " self.neg_files = [os.path.join(self.root_dir, \"neg\", f) for f in os.listdir(os.path.join(self.root_dir, \"neg\")) if f.endswith('.txt')]\n", + " self.pos_files = [os.path.join(self.root_dir, \"pos\", f) for f in os.listdir(os.path.join(self.root_dir, \"pos\")) if f.endswith('.txt')]\n", + " self.files = self.neg_files + self.pos_files\n", + " self.labels = [0] * len(self.neg_files) + [1] * len(self.pos_files)\n", + " self.pos_inx=len(self.pos_files)\n", + "\n", + " def __len__(self):\n", + " return len(self.files)\n", + "\n", + " def __getitem__(self, idx):\n", + " file_path = self.files[idx]\n", + " label = self.labels[idx]\n", + " with open(file_path, 'r', encoding='utf-8') as file:\n", + " content = file.read()\n", + " \n", + " return label, content" + ] + }, + { + "cell_type": "markdown", + "id": "b43c4995-df1e-4dcf-b40d-5c4a42e1899d", + "metadata": {}, + "source": [ + "The following uses the `IMDBDataset` class defined above to create iterators for the train and test datasets:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "075f1957-46e9-4ad0-a273-600bfe5b8336", + "metadata": {}, + "outputs": [], + "source": [ + "root_dir = tempdir.name + '/' + 'imdb_dataset'\n", + "train_iter = IMDBDataset(root_dir=root_dir, train=True) # For training data\n", + "test_iter = IMDBDataset(root_dir=root_dir, train=False) # For test dataart=train_iter.pos_inx\n" + ] + }, + { + "cell_type": "markdown", + "id": "6242e5c7-b354-4888-a996-e81a6618a6f0", + "metadata": {}, + "source": [ + "The following prints 20 samples from the training set:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "fb923043-bde7-4289-bcb0-6c5c6ba051d2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(1, 'Let me start off by saying that after watching this episode for the first time on DVD at 10 o\\'clock P.M. one night, I could not fall asleep until about 3:00 A.M.

This brief review may contain spoilers.

I\\'m a long-time fan of The Sopranos and I can safely say this is the best episode I\\'ve seen. I\\'m not saying everyone should feel this way, but I do. This episode is identical to the weekend I spent with my family, watching over my own father, comatose in the ICU before he passed.

The episode begins with Tony in an alternate reality: he is a salesman who\\'s identity has been mistaken for that of a man named Kevin Finnerty.

By the time ten minutes had gone by, I knew either Tony was dreaming, or I was watching some other show. It wasn\\'t like the normal Sopranos and I loved it.

Option 1 is confirmed when Anthony (or \"Kevin\") looks into the sky at a \"helicopter spotlight\" and we see prodding through it, a doctor with a flashlight. We see this only for a moment and the sequence plays out until we go back to real life in a situation similar to the one I just stated.

Tony has come out of the coma for only a moment. His boys take A.J. home and Carmella, overcome by stress, breaks down in the hallway: a signature moment in the episode.

For the remainder of the episode, we cut in between the real world: the family dealing with the potential negative outcome of this coma, and Tony\\'s alternate reality, which parallels what\\'s going on both in his mind and in the real world around him.

Then comes the stellar point in the episode: after A.J. finishes telling his mother he\\'s flunked school, she walks in to see Meadow sitting at Anthony\\'s side.

She approaches Tony, and utters the best line of the episode: \"Anthony, can you hear us?\" In Tony\\'s world, he enters a dark hotel room and turns on a light. He takes off his shoes and goes to the phone. He tries to dial, but he cannot--as if he were trying to say something back to Carmella, but couldn\\'t physically bring himself to do so. Not yet.

He sits down and looks out his window. A shimmering light that has reoccurred throughout the episode now seems to call to him from the other side of the city.

\"When It\\'s Cold I\\'d Like To Die\" by Moby marries perfectly with these last images and helps in creating an emotional roller-coaster of an episode.

10 out of 10.

P.S.: Watch the next episode. You find out what the light is. It\\'s wonderful.')\n", + "(1, \"This is, in my opinion, a very good film, especially for Michael Jackson lovers. It contains a message on drugs, stunning special effects, and an awesome music video.

The main film is centered around the song and music video 'Smooth Criminal.' Unlike the four-minute music video, it is normal speed and, in my opinion, much easier to watch.

The plot is rather weird, however. Michael Jackson plays a magical 'gangster' that, when he sees a shooting star, he transforms into a piece of machinery. Throughout the film, he transforms into a race car, a giant robot, and a space ship.

The robot scene in particular is a bit drawn out and strange. I found it a little out-of-whack compared to the rest of the film.

A child is kidnapped, Michael tries to save her, is tortured and beaten, and suddenly turns into a giant robot that blows up all the bad guys. A little weird? Yeah.

But besides the bizarre robot scene, it's a very good movie, and any Michael Jackson fan will enjoy both the Smooth Criminal music video and the movie.\")\n", + "(1, \"This move actually had me jumping out of my chair in anticipation of what the actors were going to do! The acting was the best, Farrah should have gotten a Oscar for this she was fabulous. James Russo was so good I hated him he was the villain and played it wonderful. There aren't many movies that have riveted me as this one. The cast was great Alfie looking shocked with those big eyes Farrah looking like a victim and you re-lived her horror as she went through it. Farrah made you feel like you were there and feeling the same anger she felt you wanted her to hurt him, yet you also knew it was the wrong thing to do. The movie had you on a roller coaster ride and you went up and down with each scene.\")\n", + "(1, \"Surprisingly not terrible and well animated for one of Disney's straight to video throw away sequels. Like the previous sequel (The Lion King 2) I was glad that Disney brought back most of the original voice actors which makes a big difference and they kept a good level of traditional animation. The plot wanders around for a while but we are distracted by an unending string of jokes ranging from hilarious to dull. To break up the detached plot and jokes they gave us some silly musical sequences, which much like the jokes, range from entertaining to a quick trip to the fridge. For the most part the MST3K-like moments are bland and full of untapped potential and really don't add a whole lot to the movie other than to act as a vehicle for an hour-long flashback. The new characters are at least likable, and the old characters are out doing their thing so I can't fault them there. Overall this movie in not bad and it makes for a nice frivolous filler between the more serious Lion King titles.\")\n", + "(1, \"At one end of the Eighties Warren Beatty created and starred in the literate epic Reds about the founding of the Soviet Union as seen through the eyes of iconoclast radical John Reed. It was a profound film both entertaining and with a message presented by an all star cast. At the end of the decade Warren Beatty created another kind of epic in Dick Tracy that makes no pretense to being anything other than entertainment with a whole bunch of the best actors around just having a great old time hamming it up under tons of makeup.

That both Reds and Dick Tracy could come from the same individual speaks volumes about the range this man has as a player. In this film Beatty managed to get all the famous cartoon characters from the strip and put them in one original screenplay.

The city's top mobster Big Boy Caprice is making a move to really eliminate competition. The film opens with him rubbing out Lips Manlis's henchmen in a Valentine Massacre style shooting and then Lips himself being fitted for a cement overcoat. But Caprice's moves are making him a target for Tracy.

In the meantime a third mysterious and faceless individual is looking to topple Caprice himself. Will our hero sort out this thicket of crime?

The spirit of fun this film has is truly infectious. When people like Al Pacino, Dustin Hoffman, Paul Sorvino, William Forsythe, R.G. Armstrong get themselves outrageously made-up to look like the cartoon creations of strip author Chester Gould and then indulge in an exercise of carving the biggest slice of ham, you've got to love this film.

Al Pacino got a nomination for Best Supporting Actor, but any of these guys could have, it's only that Pacino as Big Boy Caprice gets the most screen time. Only Beatty plays it completely straight, the others all seem to play off of him. Dick Tracy won Oscars for Best Art&Set Design, Best Song written by Stephen Sondheim and introduced by Madonna, Sooner Or Later. The fact he was even able to get somebody like Sondheim to write a score for this film only shows Sondheim wanted to get in on the fun. As for Madonna, the Material Girl does more than hold her own with all these acting heavyweights as club torch singer Breathless Mahoney.

Before this film, Dick Tracy movies were consigned to the B pictures and worse as Saturday afternoon serials. The only thing that rivals this all star extravaganza is a radio broadcast done for Armed Forces Radio during World War II that got to vinyl. Can you believe a cast like Bing Crosby, Bob Hope, Frank Sinatra, Dinah Shore, Jimmy Durante, Judy Garland, Frank Morgan, and the Andrews Sisters? Try and find a recording of that gem.

Until then Warren Beatty's classic comic strip for the big screen will do nicely.\")\n", + "(1, \"This is one of the best Bollywood movies i have seen up until now. Family and friends feel the same way about it. This movie is really romantic and dramatic at the same time. In my opinion we need more films or movies like this to keep the south Asian culture alive. Shahid Kapoor and Amrita Rao acted extremely well in this, also their couple attracts a lot of people to the movies. This is a must see movie, it's a family and romantic movie. This movie is also from the makers of Hum Saath Saath Hain and Hum Apke Hain Kon. This movie is their best right now... the setting of the movie was beautiful which also is a huge attraction. This movie is must see... recommended to everyone!!\")\n", + "(1, 'I recently watched this, but when it started I had no idea what the concept was about, what the topic was.....in short - I had no idea what it was. Was it a documentary, was it a comedy routine.....Well, it was BOTH.

It started a little slow, but I think that\\'s because I had absolutely no idea what type of program I was viewing. But it quickly sucked me in. The episode I watched had Robert Wuhl discussing fact and fiction in history. Mainly how we (american\\'s) learn history that isn\\'t really true - and how we got to learn what we did. He did this in such a way as to keep the viewer completely entertained, and interested. I actually learned a few things and that is a true indicator of how effective this type of program can be.

I would love to see this picked up as a series for HBO. I believe it can be just as fun and effective with a variety of topics - especially if they are \"taught\" in the same type of manner as this episode.')\n", + "(1, \"Poor Basil Rathbone, an egotistical composer who's lost his muse. He's been faking it for some time, buying his lyrics and his music from various sources. Trouble is that two of the sources (Bing Crosby music) and (Mary Martin words) happen to meet and fall in love. And then they discover what they've been doing. Complications ensue, but all is righted at the end.

Crosby and Martin sing terrifically. Mary had signed a Paramount contract and also at the same time doubled as a regular on Crosby's Kraft Music Hall Radio Show. For reasons I don't understand, movie audiences didn't take to her, so she went back to Broadway and did One Touch of Venus in 1944 and stayed there.

Basil Rathbone in one of the few times he played comedy does it very well. His ego is constantly being deflated by sidekick Oscar Levant and again I'm surprised they didn't do more films together.

As in most of Crosby's Paramount vehicles, no big production numbers, but I agree with the previous reviewer about the title tune being done as an impromptu jam session in a pawn shop. Good job by all.

A surprisingly original plot and great entertainment.\")\n", + "(1, '\"Emma\" was a product of what might be called by the First Great Jane Austen Cycle of the mid-nineties, and it was recently shown on British television, doubtless because of the interest in the author created by the Second Great Jane Austen Cycle which started with \"Pride and Prejudice\" two years ago. We currently have in the cinemas the Austen biopic \"Becoming Jane\", and ITV have recently produced three TV movies based on Austen novels. These include \"Northanger Abbey\", the only one of the six major novels not to have been filmed previously, so the cycle should now be complete. No doubt, however, there will be more to come in the near future. (There is, after all, her juvenile \"Love and Freindship\" (sic), the short novella \"Lady Susan\", and someone, somewhere, has doubtless supplied endings to her two unfinished fragments \"The Watsons\" and \"Sanditon\". Then there are all those Austen sequels churned out by modern writers\\x85\\x85\\x85).

The main character is Emma Woodhouse, a young lady from an aristocratic family in Regency England. (Not, as some reviewers have assumed, Victorian England- Austen died before Queen Victoria was even born). Emma is, financially, considerably better off than most Austen heroines such as Elizabeth Bennett or Fanny Price, and has no need to find herself a wealthy husband. Instead, her main preoccupation seems to be finding husbands for her friends. She persuades her friend Harriet to turn down a proposal of marriage from a young farmer, Robert Martin, believing that Harriet should be setting her sights on the ambitious clergyman Mr Elton. This scheme goes disastrously wrong, however, as Elton has no interest in Harriet, but has fallen in love with Emma herself. The speed with which Emma rejects his proposal makes one wonder just why she was so keen to match her friend with a man she regards (with good reason) as an unsuitable marriage partner for herself. This being a Jane Austen plot, Emma turns out to be less of a committed spinster than she seems, and she too finds herself falling in love, leading to further complications.

Emma always insists that she will not marry without affection, and when she does find a partner, the handsome Mr Knightley, we feel that this will indeed be an affectionate marriage. It does not, however, seem likely to be a very passionate one (unlike, say, that of Elizabeth Bennett and Mr Darcy). Knightley, who is sixteen years older than Emma (she is 21, he 37), and related to her by marriage, is more like a father-figure than a lover. Much more of a father-figure, in fact, than her actual father, a querulous and selfish old hypochondriac who seems more like her grandfather. When Emma is rude to her unbearably garrulous and tedious friend Miss Bates, it is Knightley who chides her for her lack of manners. (His surname is probably meant to indicate his gentlemanly nature- nineteenth-century gentlemen liked to think of themselves as the modern equivalent of mediaeval knights with their elaborate codes of chivalry). Both Gwyneth Paltrow and Jeremy Northam play their parts very well, but this is not really one of the great screen romances.

Of the other characters, I liked Juliet Stephenson\\'s vulgar Mrs Elton and Toni Collette\\'s Harriet. I know that in the novel Harriet was a naïve young teenager, whereas here she is more like the character Collette played in \"Muriel\\'s Wedding\"- a gauche, slightly overweight twentysomething, fretting about her chances of finding a man. Nevertheless, I felt that this characterisation worked well in the context of the film and did not detract from Austen\\'s themes.

\"Emma\" is one of Austen\\'s more light-hearted works, without the darker overtones of \"Mansfield Park\" or even \"Pride and Prejudice\", and this is reflected on screen. We see a world of beauty and grace, full of stately homes and elegant costumes and fine manners. Apart from the ruffianly gypsies, who make a very brief appearance, the only \"poor\" people we see are Mrs Bates and her daughter, and, as they live in the sort of picturesque rose-strewn thatched cottage which today would change hands for over £500,000, we can be sure that their poverty is relative, not absolute. In Emma\\'s world, poverty is defined as not having your own stately home. This is, of course, not a comprehensive picture of early nineteenth-century life, but nobody has ever claimed Austen as the Regency equivalent of a kitchen-sink realist. Sophisticated romantic comedy, combined with a keen eye for analysing human character, was more in her line.

I would not rate this film quite as highly as the 1994 \"Sense and Sensibility\" or the recent \"Pride and Prejudice\"- it tends to drag a bit in the middle, although it has a strong beginning and strong ending- but it is, in the main, a highly enjoyable Austen adaptation. 7/10')\n", + "(1, \"A very strong movie. Bruce is good and Brad also.

As I think there are two cities missed in the receptionist list from the list Bruce remembered.

That means the woman was a real insurance and she did her job.

Well, Novikov property seems to me work in this movie. However, I do believe in Back to the future theory of worlds' multiplicity.

So Bruce could save the world, but not his world.

In the theory of parallel worlds the man can meet himself.

And I do believe there is no problem in that. Here I disagree with Dr. Brown from Back...

But the story pf 12 Monkeys has its own beauty. Inspite of all these theories of one world or many or continuum one can believe that he is really insane and the doctor - his girlfriend was just lost.

A sequence of events which may lead her to believe that he is from the future. The bullet - well it might be some mistake, some falsification.

Well I like this movie - has to buy a DVD.

Best.\")\n", + "(0, \"I have no clue as to what this was shot on but you can definitely tell that they had no budget. Bad acting, horrible cinematography, and lame plot and some decent special effects do not make a good movie. The WWF style cinemtography will make you cry...where's the tripod?! The filmakers aimed high, but sorely missed their mark.\")\n", + "(0, \"This film can be judged from three viewpoints: as history, as a profile of Amin, as a fictional thriller.

It fails as history, it mentions in passing the coup that threw out Obote, the expulsion of the Asians, and has the Entebbe hi-jack as background, but not in any chronologically consistent time frame.

As a profile of Amin it may have been interesting, because Forest Whitaker is incredibly good, and if this was a better film, he would get an Oscar. (He got it - which proves the Oscar voters don't watch the films they vote on.) It ignores relevant historical episodes in the novel, which observed Amin and the history of Uganda from the point of view of the doctor. It tells instead the fictitious story of the Scots doctor and his impossible love life from the point of view of Amin. But the story told is the one incident that Amin was probably innocent of.

As a fictional thriller, there is no plot to hold it together. The beginning is taut - it takes cinematic liberties with the novel, but sets up the story. The character of the doctor is well-defined, but becomes lost in the second half of the film which suffers as a result.

Why the doctor decides to stay in Kampala is badly explained - seduced by power? Why he befriends no-one is strange. The character of the friend in the novel has been lost because the Scotsman has the affair instead of the black doctor - a ludicrous entanglement which does not seem even faintly believable, but allows the writers of the film to show the ferocity of Amin close at hand. The Man called Horse bit at the end is risible.

Finally in 1971, Uganda drove on the left, not right, the number plates were three letters and two or three numbers - and where are the Equator tusks?!

In short - if you've never heard of Amin, you may want to spend two hours watching this film to appreciate Forest Whitaker's acting, but the last hour will bore you to confusion. If you know Uganda or have read the book - don't see the film - it will only depress you. And if you want to know why the doctor was so foolhardy - he wasn't.\")\n", + "(0, \"This show comes up with interesting locations as fast as the travel channel. It is billed as reality but in actuality it is pure prime time soap opera. It's tries to use exotic locales as a facade to bring people into a phony contest & then proceeds to hook viewers on the contestants soap opera style.

It also borrows from an early CBS game show pioneer- Beat The Clock- by inventing situations for its contestants to try & overcome. Then it rewards the winner money. If they can spice it up with a little interaction between the characters, even better. While the game format is in slow motion versus Beat The Clock- the real accomplishment of this series is to escape reality.

This show has elements of several types of successful past programs. Reality television, hardly, but if your hooked on the contestants, locale or contest, this is your cup of tea. If your not, this entire series is as I say, drivel dripping with gravy. It is another show hiding behind the reality label which is the trend it started in 2000.

It is slick & well produced, so it might last a while yet. After all, so do re-runs of Gilligan's Island, Green Acres, The Beverly Hillbillies & The Brady Bunch. This just doesn't employ professional actors. The intelligence level is about the same.\")\n", + "(0, \"Besides all of the technical mistakes ....

How about a female flight attendant who's able to kill, all by herself, 4 out of the 7 terrorists (including ex marines), 2 of whom without even using a gun. Then, she lands the plane perfectly. We're not talking about Sigourney Weaver or Linda Hamilton; we're talking about a regular, frightened, yet very well composed flight attendant. :D How about the leader in charge of the assault/rescue squad, having a full-proof (according to the logic of the script) plan of sleep-gassing everyone and having someone from his team fly the plane. Only he decides at the spur of the moment to change plans and instead lead an attack on the terrorists, guns blazing, not knowing where the terrorists are, or how many, and not securing a position of advantage, so that his whole team gets easily wiped out. Yeah, that's using the old noggin. Only later to decide to use the sleep gas anyway. And it turns out useless for all intensive purposes.

Bad as this movie was, though, I couldn't stop myself from watching and wondering, what next? :D I can't help but imagine all the excellent, unemployed script writers thinking to themselves, it's not fair. lol! :D\")\n", + "(0, 'We purchased this series on DVD because of all of the glowing reviews we had seen here. I gave it three stars because there can be little doubt that sometimes the acting, directing and writing are brilliant. In fact they are so brilliant we did not see the propaganda that was being transmitted so smoothly on the series. If one watches it with discernment, one will see the entire litany of the radical right wing beliefs being promulgated by the Fox (Faux) News Network. To avoid giving away any spoilers I will refrain from pointing out all of the dozens of specific instances. A brief look at the plots found here on IMDb will disclose that everything from torture to gun control to the right of a network to provide \"Infomercials\" and call them news is justified with cute plot twists and impassioned speeches given by some of the best actors in the world. We watched many shows and finally gave up in disgust when they justified torture using Attorney General Gonzales as a shining example of why all kinds of torture should be used in the name of protecting all of us. The series also manages to demean male and female gays in subtle ways by using them as plot devices depicting evil people. All in all the complete litany of the radical religious right wing.

No doubt the popularity of this program will be used by future historians as proof that America lost its way in the early part of the this century. As a student of history myself I would characterize this program as being in a league with the propaganda produced by Goebbels for Hitler and some of the propaganda produced by Hollywood for the American audience during WWII.

So if you want to use this as a teaching tool to help your students understand how subtle propaganda can be then by all means do so. Just be sure to purchase an inexpensive used copy so you can avoid enriching the ultra right wingers at Faux Network who produced this travesty.')\n", + "(0, 'It says that a girl named Susan Montford both wrote and directed this \"movie.\" No wonder she has no other credits to her name for writing or directing. She made a severe vocational error in choosing this as her career. This is one of the worst human creations of this millennium.

The fundamental thing wrong with this movie other than its ridiculous story of a woman running away from four weak thugs, is the blatant and complete lack of LOGIC.

**After she leaves the mall, she gets approached by four thugs as they surround her. Tell me, what woman would aggressively SHOVE a potential attacker while being surrounded, and insult them verbally? I don\\'t mean after an attack had already started, because then of course it\\'s completely normal for someone to fight back. But she shoved that guy and pretty much escalated it to the next level. No woman would do that unless she 1) had a weapon, 2) has the confidence of knowing that backup is very close, and so is relatively safe from harm, or 3) the attackers are so young, and weak looking that she\\'s pretty sure she can take them. None of that applied in this situation, so she was just acting like someone that\\'s asking to get raped or mugged. And by the way, when the security guard approached, as SOON as he came within viewing distance of Kim Basinger, why wouldn\\'t she immediately either run towards him for help, or scream??

**When she drives off after the security guard gets shot in the head, she drives into a deserted part of town, and crashes. She had a good three minute lead on the pursuers, instead of simply running off on foot in a diagonal direction behind houses and climbing fences and continuing, she gets out her Red Toolbox and starts messing around under her hood. I understand she was trying to fix her car, but she should\\'ve ran.

(I didn\\'t even mean this to be a chronological summary of the movie, because I loathe people who do that in their reviews, but it just so happens that every main sequence of this movie has something so blatantly stupid that I have to comment on it).

**Why would she carry a loud, Red Toolbox as she\\'s trying to sneak away in the dark? When she does get caught, one of the jokers demands for her to open the toolbox. First she resists, then eventually opens it. And takes out a wrench. This scene here is so rich in subtle overtones of the complete failure of dramatic effect I have to break it down, it\\'s one of the dumbest scenes in the entire movie. When asked to open the box, she\\'s resisting at first as if it were her plan to somehow get one of the thugs to open it themselves out of anger after she didn\\'t open it, in the same way that someone in some action movie might have some device that an enemy demands that person to touch/push/open/manipulate, and once that hero refuses to open it, the enemy grabs that device, only to have that device automatically dispense a chemical/shoot him in the face/render him unconscious, which was the hero\\'s plan all along. It feels like that\\'s what they tried to do with Kim Basinger here, as she opens the toolbox dramatically and quickly takes out a WRENCH and dispatches one of the thugs, and somehow GETS AWAY from him and the three other thugs.

**Throughout the rest of the movie, basically what you see is this suburban house wife, sneaking around the woods as she carries her Red Toolbox, taking out various tools used as weapons to KILL HER ATTACKERS.

**When she was running away, how did she end up moving BACK to where the thugs were? I think it was the scene where they had that radio playing loudly in tribute to the dead dude. She somehow crept up on them when I thought she was moving AWAY from them.

**Finally, this whole premise is so weak because the whole reason she\\'s being chased in the first place is because from the thugs\\' perspective, she was a witness to a murder they committed against the security officer earlier, and so they felt they had to kill her. How ridiculous. As one of the thugs even said, they could\\'ve just left town and returned back to whatever city they drove from, no one but her had seen them anyway, and she probably didn\\'t get the license plate. Even if these possibilities wouldn\\'t work in their favor, how is raising hell and hunting down someone to kill them improving your chances to get away with the original murder?')\n", + "(0, 'Well I just gave away 95 minutes and 47 seconds that I\\'ll never get back on this piece of trash. I heard someone online describe this movie\\'s villains as \"subhuman cannibals\", and I thought it was promising because I thought it would be like the Descent. WRONG! The Descent was a psychological thriller with dynamic characters and strong storyline. These villains are totally unrealistic and no part of their performance is enjoyable to watch. This movie isn\\'t so controversial, I\\'ve seen this level of gore in many films. This movie plain sucks. SYNOPSIS: A blonde who thinks she\\'s real hot (but she isn\\'t), her admirer, and her admirer\\'s friend (no, I don\\'t remember their names) go into the woods. Their car breaks down. They are warned to leave by a man named Mark. The blonde gets unreasonably hysterical and the next morning they can\\'t find the admirer\\'s friend. Admirer impales his foot (whoops!). Don\\'t worry, he is much more upset when his car won\\'t start than when he gets impaled by nails. After a nanosecond of coaxing, the blond leaves to find help. Events ensue that I cannot remember. During this and throughout the movie, we are shown grotesque torture scenes with no substance including one that made me gag. Blonde goes to save admirer from house of cannibals (even though all they are seen eating is intestines, which would logically be the last choice for real cannibals to eat since they contain actual food). Blonde finds admirer hurt and works very hard (unsuccessfully) to work up tears. Then you get a good laugh when the blonde is in the house and announces she can \"out think them\". Mark (the man who warned them to leave) has a remarkable change of character when he reveals the cannibals are his family. Then there is some shooting, they leave the house, the shooting continues, then a random guy shows up and says he\\'s been watching them. Before he is shot, we are shown an acid-trip inspired scene of more killing. The blonde or her admirer shoots him because he did not help them. There\\'s more killing, the admirer professes his love for the blonde. Then a mysterious hand covers the camera. What does that imply? I don\\'t know, hopefully not a sequel.')\n", + "(0, \"This was not enjoyable to watch. Frank puts all his dreams on the back burner and gets a normal (boring!) job just so his stepson can go to film school, but his stepson decides that he'll make a humiliating documentary about the man instead. A documentary filmmaker should point the camera and simply shoot, not manipulate and comment with snide captions. The bitterness and resentment of the filmmaker towards his stepfather is obvious. And sad. The goal seems to be to make Frank appear dumb and pathetic, instead he comes across as the most human of the 3 people featured.

Essentially a smear campaign all dressed up as something much smarter and edgier than it really is. It left me with an intense dislike for the filmmaker.\")\n", + "(0, 'The plot for Descent, if it actually can be called a plot, has two noteworthy events. One near the beginning - one at the end. Together these events make up maybe 5% of the total movie time. Everything (and I mean _everything_) in between is basically the director\\'s desperate effort to fill in the minutes. I like disturbing movies, I like dark movies and I don\\'t get troubled by gritty scenes - but if you expect me to sit through 60 minutes of hazy/dark (literally) scenes with NO storyline you have another thing coming. Rosario Dawson, one of my favorite actresses is completely wasted here. And no, she doesn\\'t get naked, not even in the NC-17 version, which I saw.

If you have a couple of hours to throw away and want to watch \"Descent\", take a nap instead - you\\'ll probably have more interesting dreams.')\n", + "(0, \"There's nothing new here. All the standard romantic-comedy scenes, even down to the taxi sprinting to the airport to stop the woman flying away. The only thing that saves this is the acting of Alison Eastwood & some of the minor characters (blink and you'll miss Gabrielle Anwar), who obviously had some fun.

Turn it off when the pair are in bliss, and you won't have to go through the inevitable plot pain.\")\n" + ] + } + ], + "source": [ + "start=train_iter.pos_inx\n", + "start=0\n", + "\n", + "for i in range(-10,10):\n", + " print(train_iter[start+i])" + ] + }, + { + "cell_type": "markdown", + "id": "50a6efb5-01fb-4783-9237-ec83b1f8a3b6", + "metadata": {}, + "source": [ + "The following defines the mapping of numeric labels to positive and negative reviews:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "8963e017-2696-42cd-a0ba-eb2fb001b8c8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'positive review'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "imdb_label = {0: \" negative review\", 1: \"positive review\"}\n", + "imdb_label[1]" + ] + }, + { + "cell_type": "markdown", + "id": "66729636-d9d6-45b6-a408-10a3636ec607", + "metadata": {}, + "source": [ + "The following checks to make sure that there are exactly 2 classes in the train dataset:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "fea7d361-1299-4e1c-a8e3-ba86a482edab", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "num_class = len(set([label for (label, text) in train_iter ]))\n", + "num_class" + ] + }, + { + "cell_type": "markdown", + "id": "804dfa23-be45-47d2-b420-a60ed40fb420", + "metadata": {}, + "source": [ + "The following are some token indices:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "2a502a76-b4af-4681-bb64-dcce35f0d0de", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[466, 13077]" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vocab([\"age\",\"hello\"])" + ] + }, + { + "cell_type": "markdown", + "id": "c7ae8bd3-a8d1-4ebe-b69b-f3da3d9f05a5", + "metadata": {}, + "source": [ + "### Train and validate\n", + "\n", + "The following converts the dataset into map-style datasets and then performs a random split to create separate training and validation datasets. The training dataset will contain 95% of the samples in the original training set, while the validation dataset will contain the remaining 5%. These datasets can be used for training and evaluating a machine learning model for text classification on the IMDB dataset. The final performance of the model will be evaluated on the hold-out test set:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "a9283486-de17-4b3c-ae75-96c01ce83bb6", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Convert the training and testing iterators to map-style datasets.\n", + "train_dataset = to_map_style_dataset(train_iter)\n", + "test_dataset = to_map_style_dataset(test_iter)\n", + "\n", + "# Determine the number of samples to be used for training and validation (5% for validation).\n", + "num_train = int(len(train_dataset) * 0.95)\n", + "\n", + "# Randomly split the training dataset into training and validation datasets using `random_split`.\n", + "# The training dataset will contain 95% of the samples, and the validation dataset will contain the remaining 5%.\n", + "split_train_, split_valid_ = random_split(train_dataset, [num_train, len(train_dataset) - num_train])" + ] + }, + { + "cell_type": "markdown", + "id": "8785376c-ec97-4bb9-94f1-8e3ccd0e3aea", + "metadata": {}, + "source": [ + "The following code checks if a CUDA-compatible GPU is available in the system using PyTorch, a popular deep learning framework. If a GPU is available, it assigns the device variable to \"cuda\" (which stands for CUDA, the parallel computing platform and application programming interface model developed by NVIDIA). If a GPU is not available, it assigns the device variable to \"cpu\" (which means the code will run on the CPU instead).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "52046e3f-5087-4b63-94cc-e8a7daf96407", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "device(type='cuda')" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "device" + ] + }, + { + "cell_type": "markdown", + "id": "d6ae581f-0cb5-4a81-8e84-e5244ce7eea7", + "metadata": {}, + "source": [ + "### Data loader\n", + "\n", + "In PyTorch, the **`collate_fn`** function is used in conjunction with data loaders to customize the way batches are created from individual samples. The provided code defines a `collate_batch` function in PyTorch, which is used with data loaders to customize batch creation from individual samples. It processes a batch of data, including labels and text sequences. It applies the `text_pipeline` function to preprocess the text. The processed data is then converted into PyTorch tensors and returned as a tuple containing the label tensor, text tensor, and offsets tensor representing the starting positions of each text sequence in the combined tensor. The function also ensures that the returned tensors are moved to the specified device (e.g., GPU) for efficient computation.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "9428567c-19c9-4bf6-9f55-11423d70c8bf", + "metadata": {}, + "outputs": [], + "source": [ + "from torch.nn.utils.rnn import pad_sequence\n", + "\n", + "def collate_batch(batch):\n", + " label_list, text_list = [], []\n", + " for _label, _text in batch:\n", + " label_list.append(label_pipeline(_label))\n", + " text_list.append(torch.tensor(text_pipeline(_text), dtype=torch.int64))\n", + "\n", + "\n", + " label_list = torch.tensor(label_list, dtype=torch.int64)\n", + " text_list = pad_sequence(text_list, batch_first=True)\n", + "\n", + "\n", + " return label_list.to(device), text_list.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "d59cb2e6-a0a2-43cd-8afd-55484bfed045", + "metadata": {}, + "source": [ + "You convert the dataset objects to a data loader by applying the collate function.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "a2020739-e68e-434e-a57a-e7db17162eab", + "metadata": {}, + "outputs": [], + "source": [ + "BATCH_SIZE = 64\n", + "\n", + "train_dataloader = DataLoader(\n", + " split_train_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")\n", + "valid_dataloader = DataLoader(\n", + " split_valid_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")\n", + "test_dataloader = DataLoader(\n", + " test_dataset, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "edb6cdd8-8f81-4530-9b3f-5abd5639cf62", + "metadata": {}, + "source": [ + "Let's check the what these data loaders generate:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "e08fc3af-9ffe-4271-9d2b-c9e5507d6d16", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(tensor([0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0,\n", + " 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0,\n", + " 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0], device='cuda:0'),\n", + " tensor([[ 39, 16, 31, ..., 0, 0, 0],\n", + " [ 43, 59, 1995, ..., 0, 0, 0],\n", + " [ 2, 139, 5, ..., 0, 0, 0],\n", + " ...,\n", + " [ 2, 4182, 17, ..., 0, 0, 0],\n", + " [ 38, 9, 0, ..., 0, 0, 0],\n", + " [ 194, 4510, 33, ..., 0, 0, 0]], device='cuda:0'))" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "label,seqence=next(iter(valid_dataloader ))\n", + "label,seqence" + ] + }, + { + "cell_type": "markdown", + "id": "a1369331-f011-4df4-9f59-8ef962d7f3dc", + "metadata": {}, + "source": [ + "### Neural network\n", + "\n", + "This code defines a class called `TextClassifier` that represents a simple text classifier that uses an embedding layer, a hidden linear layer with a ReLU avtivation, and an output linear layer. The constructor takes the following arguments:\n", + "\n", + "- `num_class`: The number of classes to classify.\n", + "- `freeze`: Whether to freeze the embedding layer.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "6e1f8a63-28ab-45dd-b1a4-c69664fe57d0", + "metadata": {}, + "outputs": [], + "source": [ + "from torch import nn\n", + "\n", + "class TextClassifier(nn.Module):\n", + " def __init__(self, num_classes,freeze=False):\n", + " super(TextClassifier, self).__init__()\n", + " self.embedding = nn.Embedding.from_pretrained(glove_embedding.vectors.to(device),freeze=freeze)\n", + " # An example of adding additional layers: A linear layer and a ReLU activation\n", + " self.fc1 = nn.Linear(in_features=100, out_features=128)\n", + " self.relu = nn.ReLU()\n", + " # The output layer that gives the final probabilities for the classes\n", + " self.fc2 = nn.Linear(in_features=128, out_features=num_classes)\n", + "\n", + " def forward(self, x):\n", + " # Pass the input through the embedding layer\n", + " x = self.embedding(x)\n", + " # Here you can use a simple mean pooling\n", + "\n", + " x = torch.mean(x, dim=1)\n", + " # Pass the pooled embeddings through the additional layers\n", + " x = self.fc1(x)\n", + " x = self.relu(x)\n", + " return self.fc2(x)\n" + ] + }, + { + "cell_type": "markdown", + "id": "67cd83d4-5789-48e6-929d-22287abee1d8", + "metadata": {}, + "source": [ + "## Train the model on the full dataset\n", + "\n", + "The model can then be trained on labeled data from the IMDB dataset with two classes.\n", + "\n", + "First, let's create the model.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "3acca5e8-1a69-4589-82c7-46c6c2081dc2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model=TextClassifier(num_classes=2,freeze=True)\n", + "model.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "05531fdc-68fb-4b6e-ad6d-edeadbb81dea", + "metadata": {}, + "source": [ + "The code line `predicted_label=model(text, offsets)` is used to obtain predicted labels from a model for a given input text and its corresponding offsets.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "f96b774b-4435-4864-a264-576a28fed21b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "torch.Size([64, 2])" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.eval()\n", + "predicted_label=model(seqence)" + ] + }, + { + "cell_type": "markdown", + "id": "9f9410bc-bd8c-465f-9fcb-48ba5493ce60", + "metadata": {}, + "source": [ + "The following returns the shape of `predicted_label`. Because your dataset iterators are batching 64 inputs, `predicted_label` should return 64 rows:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "065e2be0-d613-4287-8227-016209a8b87c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "torch.Size([64, 2])\n" + ] + } + ], + "source": [ + "print(predicted_label.shape)" + ] + }, + { + "cell_type": "markdown", + "id": "57ddc0ea-36d3-4f6f-9cb0-0e491a1844f1", + "metadata": {}, + "source": [ + "For each input, the model outputs two logits corresponding to the two classes in the classification task. If the value of the first logit is greater than the second, the predicted class is class 0, which maps to a negative review. If the second logit is greater than the first, the predicted class is class 1, which maps to a positive review:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "32e6eb56-18fa-4f03-a431-53180c744c57", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "tensor([[0.1659, 0.0446],\n", + " [0.1570, 0.0366],\n", + " [0.1655, 0.0466],\n", + " [0.1657, 0.0450],\n", + " [0.1656, 0.0384],\n", + " [0.1721, 0.0519],\n", + " [0.1721, 0.0525],\n", + " [0.1489, 0.0255],\n", + " [0.1621, 0.0394],\n", + " [0.1737, 0.0522]], device='cuda:0', grad_fn=)" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "predicted_label[:10,]" + ] + }, + { + "cell_type": "markdown", + "id": "6eb1d187-d606-4da5-93cb-d79a8a4dd889", + "metadata": {}, + "source": [ + "The following **`predict`** function takes in a text, a text pipeline, and a model as inputs. It uses a pretrained model passed as a parameter to predict the label of the text for text classification on the IMDB dataset:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "e8f1aefb-3058-4b46-9d9b-98ef5f392dbc", + "metadata": {}, + "outputs": [], + "source": [ + "def predict(text, model, text_pipeline):\n", + " with torch.no_grad():\n", + " text = torch.unsqueeze(torch.tensor(text_pipeline(text)),0).to(device)\n", + "\n", + " output = model(text)\n", + " return imdb_label[output.argmax(1).item()]" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "e5d08087-8aba-445d-86e1-5cdacabdcf92", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "' negative review'" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "predict(\"the is a good movie\",model,text_pipeline )" + ] + }, + { + "cell_type": "markdown", + "id": "b88a47f4-5326-487a-86cf-c74d1942fe5c", + "metadata": {}, + "source": [ + "You can create a function to evaluate the model's accuracy on a dataset:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "99a0391e-5770-4247-8475-3ee78fd8ee13", + "metadata": {}, + "outputs": [], + "source": [ + "def evaluate(dataloader, model, device):\n", + " model.eval()\n", + " correct = 0\n", + " total = 0\n", + " with torch.no_grad():\n", + " for label, text in dataloader:\n", + " label, text = label.to(device), text.to(device)\n", + " outputs = model(text)\n", + " _, predicted = torch.max(outputs.data, 1)\n", + " total += label.size(0)\n", + " correct += (predicted == label).sum().item()\n", + " accuracy = 100 * correct / total\n", + " return accuracy" + ] + }, + { + "cell_type": "markdown", + "id": "5838996b-d0fa-4f4d-ba39-e7dd7ea6fc26", + "metadata": {}, + "source": [ + "The following evaluates the performance of your model on the test set:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "eb37f6eb-84ab-4def-b607-391ae3fcb0c1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "50.0" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader , model, device)" + ] + }, + { + "cell_type": "markdown", + "id": "dac15043-ba2b-43ba-94bf-87528af585d5", + "metadata": {}, + "source": [ + "Note that the current performance of the model is no better than average. This outcome is expected, considering that the model has not undergone any training yet.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a9e263a8-6689-4546-ac57-9b19fc60f509", + "metadata": {}, + "source": [ + "## Train the model\n", + "\n", + "The following defines the training function used to train the model:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "31f35bc9-ee1d-439d-9ffd-905c821eda38", + "metadata": {}, + "outputs": [], + "source": [ + "def train_model(model, optimizer, criterion, train_dataloader, valid_dataloader, epochs=100, model_name=\"my_modeldrop\"):\n", + " cum_loss_list = []\n", + " acc_epoch = []\n", + " best_acc = 0\n", + " file_name = model_name\n", + " \n", + " for epoch in tqdm(range(1, epochs + 1)):\n", + " model.train()\n", + " cum_loss = 0\n", + " for _, (label, text) in enumerate(train_dataloader): \n", + " optimizer.zero_grad()\n", + " predicted_label = model(text)\n", + " loss = criterion(predicted_label, label)\n", + " loss.backward()\n", + " torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)\n", + " optimizer.step()\n", + " cum_loss += loss.item()\n", + " #print(\"Loss:\", cum_loss)\n", + " cum_loss_list.append(cum_loss)\n", + " acc_val = evaluate(valid_dataloader, model, device)\n", + " acc_epoch.append(acc_val)\n", + " \n", + " if acc_val > best_acc:\n", + " best_acc = acc_val\n", + " print(f\"New best accuracy: {acc_val:.4f}\")\n", + " torch.save(model.state_dict(), f\"{model_name}.pth\")\n", + " \n", + " save_list_to_file(cum_loss_list, f\"{model_name}_loss.pkl\")\n", + " save_list_to_file(acc_epoch, f\"{model_name}_acc.pkl\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "c9529fe4-f63d-4404-9563-f68d73b1b648", + "metadata": {}, + "source": [ + "---" + ] + }, + { + "cell_type": "markdown", + "id": "3d36ca3a-0c98-481a-9c61-8d3a931b8c1f", + "metadata": {}, + "source": [ + "The following sets the learning rate (LR) to 1, which determines the step size at which the optimizer updates the model's parameters during training. The CrossEntropyLoss criterion is used to calculate the loss between the model's predicted outputs and the ground truth labels. This loss function is commonly employed for multi-class classification tasks.\n", + "\n", + "The chosen optimizer is Stochastic Gradient Descent (SGD), which optimizes the model's parameters based on the computed gradients with respect to the loss function. The SGD optimizer uses the specified learning rate to control the size of the weight updates.\n", + "\n", + "Additionally, a learning rate scheduler is defined using StepLR. This scheduler adjusts the learning rate during training, reducing it by a factor (gamma) of 0.1 after every epoch (step) to improve convergence and fine-tune the model's performance. These components together form the essential setup for training a neural network using the specified learning rate, loss criterion, optimizer, and learning rate scheduler.\n", + "\n", + "For the sake of time efficiency, the number of epochs has been set to 2. This is to give you a practical demonstration of what the training process looks like. However, if you were to train this model in a real-world scenario, you would likely increase the number of epochs to a larger figure, such as 100 or more. Given the reduced training set defined earlier, it takes approximately 2 minutes to complete 2 epochs of training:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "ac97cc88-2c4c-4d81-926b-1a8b81e01cc6", + "metadata": {}, + "outputs": [], + "source": [ + "LR=1\n", + "\n", + "criterion = torch.nn.CrossEntropyLoss()\n", + "optimizer = torch.optim.SGD(model.parameters(), lr=LR)\n", + "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "c51c4267-910b-4cb1-86f9-448922630c65", + "metadata": {}, + "source": [ + "You have pretrained the model for 300 epochs using a GPU and saved this model for your convenience. However, to demonstrate the training process, the following code has been included that trains the model for just two epochs. Please note that you have limited the number of epochs to two because training on a CPU can be time-consuming. Even with just two epochs, you can expect the following code to run for approximately one minute.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "a9120faf-0f9a-4c23-83d6-e67ac7474887", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/10 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "cum_loss_list=load_list_from_file(\"model_imdb_freeze_true2_loss.pkl\")\n", + "acc_epoch=load_list_from_file(\"model_imdb_freeze_true2_acc.pkl\")\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "9cc31ff0-ee7c-4f88-8005-01de11c81ff0", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "%%capture \n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/ZvhVWJU0flC7BmU1jjYxjg/model-imdb-freeze-true2.pth\n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/2RdN-JG4Rm5Gx3UNtOP4NA/model-imdb-freeze-true2-acc.pkl\n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/8qoGvWk0BdXRGoFAOT-dAw/model-imdb-freeze-true2-loss.pkl\n" + ] + }, + { + "cell_type": "markdown", + "id": "8495d949-6aec-4330-bf71-0ce599265cbd", + "metadata": {}, + "source": [ + "Let's plot the cost and accuracy for each epoch for the pretrained model that was trained for 300 epochs. From the plot, it becomes evident that with just a few epochs, the accuracy exhibits significant volatility.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "8c51d7c9-8125-47d4-a43d-371b9ea9d9b4", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHVCAYAAAB8NLYkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOx9d7gdVb32O7PL6TknvRFC6C30EBFFMDQVKzaKgCJYgkhQPwQbRgW8KpcrIogFVAhFBaVcUGqESyCEagikAQnp5bSctsvMfH/MrJnfKtP22acl632e85w9M2vWrJk9e9Y7768ZjuM40NDQ0NDQ0NDQGPEwh3oAGhoaGhoaGhoa1YEmdhoaGhoaGhoaOwk0sdPQ0NDQ0NDQ2EmgiZ2GhoaGhoaGxk4CTew0NDQ0NDQ0NHYSaGKnoaGhoaGhobGTQBM7DQ0NDQ0NDY2dBJrYaWhoaGhoaGjsJNDETkNDQ0NDQ0NjJ4EmdhoaGhoaGhoaOwk0sdPQ0NDQ0NDY6bF+/XqcffbZGDt2LOrq6jBz5kwsWbLE324YhvLvZz/7WWifV155pdR+//33H4zTCUV2SI8+glEul/HSSy9h4sSJME3NjzU0NDQ0NAYLtm1j8+bNOPzww5HNxlOZtrY2HHvssTjhhBPw0EMPYfz48Vi5ciVGjx7tt9m4cSO3z0MPPYTzzz8fp59+emTfBx10EB599FF/Ocl4BhKa2FWIl156CUcfffRQD0NDQ0NDQ2OXxeLFizFr1qzYdj/96U8xbdo03HLLLf66GTNmcG0mTZrELf/jH//ACSecgD333DOy72w2K+07lNDErkJMnDgRgHtTTZ48eYhHo6GhoaGhsetg48aNOProo9E4ehx29JX89fmsiZpsRmp/33334ZRTTsGnPvUpLFy4EFOnTsVXv/pVXHDBBcr+N2/ejAcffBB//OMfY8eycuVKTJkyBbW1tTjmmGNw9dVXY/fdd6/85PoJTewqBDO/Tp48GbvtttsQj0ZDQ0NDQ2PXwyk3LIFZs8xf/vqcfTDvpH2ldm+++SZuvPFGXHrppbjiiivw/PPP4+KLL0Y+n8e5554rtf/jH/+IpqYmfOITn4g8/uzZs3Hrrbdiv/32w8aNG/HDH/4Q733ve7F06VI0NTX1/wQrgOE4jjMkRx7hWLduHaZNm4Z33nlHEzsNDQ0NDY1BBJuD31j9FqZMmeqvD1Ps8vk8jjrqKDzzzDP+uosvvhjPP/88Fi1aJLXff//9cdJJJ+H6669PNa729nZMnz4d1157Lc4///xU+1YL2utfQ0NDQ0NDY0SiIZ9FU23O/1OROsC1rh144IHcugMOOABr166V2j711FNYvnw5vvjFL6YeT0tLC/bdd1+sWrUq9b7VgiZ2GhoaGhoaGjs1jj32WCxfvpxbt2LFCkyfPl1q+/vf/x5HHnkkDj300NTH6erqwurVq4fU914TOw0NDQ0NDY2dGvPmzcOzzz6Lq666CqtWrcKCBQtw8803Y+7cuVy7zs5O/OUvfwlV6+bMmYNf/epX/vI3v/lNLFy4EG+//TaeeeYZfPzjH0cmk8EZZ5wxoOcTBR08oaGhoaGhobFTY9asWbj33ntx+eWXY/78+ZgxYwauu+46nHXWWVy7O++8E47jhBKz1atXY9u2bf7yunXrcMYZZ2D79u0YP3483vOe9+DZZ5/F+PHjB/R8oqCDJyqEDp7Q0NDQ0NAYGug5OBzaFKuhoaGhoaGhsZNAEzsNDQ0NDQ0NjZ0EmthpaGhoaGhoaOwk0MROQ0NDQ0NDQ2MngSZ2GhoaGhoaGho7CTSx09DQ0NDQ0NDYSaCJnYaGhoaGhobGTgJN7DQ0NDQ0NDQ0dhJoYqdRNTiOg/Z7/47epa8N9VA0NDQ0lHjg1Q14/u3WoR6GhsaAQZcU06gaCitXYuPll6Nmv/2w5z/+PtTD0dDQ0OCwfNMOXLTgJQDA29d8aIhHo6ExMNCK3TBCeds2bP/d79C3fPlQD6UiWK3uW3B5+/YhHomGhoaGjLe3dw/1EDQ0Bhya2A0jbL7mp9jy81+gbcEdQz0UDuWtW7Fp/nz0rVgR2c7u7QUAON5/DY1qwrZ1WWuN/qFs6XtIY+eHJnbDCC2fPB0A0PnAA7B7eqrWb7mtDdt//3tYnZ0V7d/2l7+gbcEd2P6730W2cwoFAC7Bcxz9ANWoHv5v1TYc8sN/4e8vrR/qoWiMYJRte6iHoKEx4NDEbhih/uijkZs2DXZ3NzZeeSXe+fJXUFi5Ej0vvID1l16K0uYtoft2/vNf2PrL6+FYlrRt89VXY8vPfo6t1/1PReMqrn4TAFBaszaynd3b532w4ZRKFR1LQ0OFz/3+OXQVyrjkrpeHeigaIxglrdhp7ALQwRPDCIZpouWTn8TW//5vdN53PwDA7upCafNmlN55B5mW0Zj0/e9J+5U2b8GGb30LTrGImn32xqgPfMDfZnV2Ysc//wUA6Hz4YUy84nIYWfdrL23ahO7/+z80f+xjMDKZ0HEV33rL/b9uXeT4nUJf8LmnB8jnE565hkY0tBVWoxooW1qx09j5oRW7YYbmj38MRk2Nu2Ca6FmyBKV33gEAdDzwAGzP3Emx/Xe/g1MsAgDa7r4bANC3YgXWfvECbPzOd3wTqdXaitZbb8Xmq69GubUVG/7fZdj4ne+i83//N3Q8juOg+Pbb7v7bt8PuDnc+9hU7BP52GhoaGsMFJfKGoN1FNHZWaMVumCE3YQL2uPsuwLLQevvt6PjbPf42u7MTm344H7As5GfMwKjTToORy6H9rrv8Nj2LnkVxzRpsnv8j9CxZ4q/PNDfD6ujAlp//AgBQfHsNehYvBgD0vvQymj/8YeV4ylu2cP5+xXXrUbvfvsq2VLGjJE9DQ0NjOIAqdrYDZIwhHIyGxgBBK3bDELX77YfaAw/E2C9+EUYuh8yYMRh9zucAAB333IOOf/wDW6+7Duu//nXs+OfDcIpF1B16KBqOey8AYP23/p9L6nI5IJuFOWoUJv3wh9wxuhYu9D/3vf566FiYGZahtO6d0La8Yle94A8NDQ2NaoBGxdpasdPYSaEVu2GMmhkzMOPee2DW1QEAOv5xH4xMBs0f/xha/3AL+l57DfAeTk0nn4Tagw5C99P/h75XXwUAjP7UJzH2wgsBy0J28mQ0f/J0mPk8el9+BX3LlvnH6Vu+HI5lKf3sZGIX7mfn9PWSz1qx09DQGF4oE1OsZTvIhbsWa2iMWGhiN8xRs/fe/ue9H3sMZj4HI59Hz5Il6HvlVZ+gNbz73ag94ADs/rvfYv03vwU4DsZecAFykyb5+0/58Y8BAB0PPIgN3/wmjFwOME04PT0orlmLmj1ncMcuvvMOCitXCuvCiZ3dF/j/2T3ax06j+siY2namUTmoKVYLdho7KzSxG0HINDb4nxvfexz6XnGVuczo0ajZbz8ALsHb+8kn4PT2ItPcrOxn1KmnoLBqJWpmzEDbgjvQ+8or6Ht9GUfs2u64w/Xn81B78MHoW7rUD+RQgSp22hSrMRCoyWrvEY3KQYMnLM3sNHZS6KfkCEWj508HAA3HvAuGGXyVZj4fSuoAwMhmMeGSS9D80Y+i5oD9AQAF4mfXt3wFNl99jfJ4USlPqGKnTbEaAwFN7DT6Az54QhM7jZ0T+ik5QlF78MHIjBkDwFXpKu7ngAMBAN3PLUa5tRXbbrwRaz73OTjFIuqPeRfqjzoK+T32QNMppwJwfexomoDy9u146/RPYuP3vgebKnbEFGv39WHbb26OLUmmoRGHWu0UpdEPUB87XaJOY2eFNsWOUBimiYnfvgxdTy7kEhKnRcPso4FMBn3/+Q9Wvue9gFdyp2a//TD15z9HduxYl8iVy0A2C6dQQGn9BuR3mwrHcbD2/C+i8MYb6HvtNdTPnu33a3V0YNOPf4LG44+H1dGOrf/93+h95RVM+/UN/T53jV0XWrHT6A9KQroTDY2dEfopOYLR/JGPYOq1v4DZ0BDfOAT5PfbAtBt/7ZpubRv5PffElJ//HDPu+RuyY8cCAAzDgJHLodbz4+t95WUAQPtf/4rCG2/4fVk7glq0XY8/jrbbbsPW665DectWd3tra8Xj1NAAgJqsVuw0KgcldpZmdho7KbRip4HG447Dng8+gMKKFaifPTu0vFjd4Yej77XX0PvyK2j+0Iew4+F/ctutbdv9z8X1brH28rZtsNraACCyagVDua0NZk0NzPr6Sk9HYydGTU6/i2pUjmKZRsVqYjfQaO0uYnR9Doaho9kHE/opqQEAyI4bh4Z3vzuyZmzdYYcBAHpffhmO46D3P//htpe3B8TO2rbN/d/WBqu93f3c3RU5Bru7G6tPORVvf/aMCs5AY2cFnYBrtWKn0Q9QYqejYgcWz6zehiN+9Aj++9GV8Y01qgpN7DQSgxG7vtdfR2HFCtidnTBqapAZPdptYFnSPk6hgNLGjQAAuzs6BUppwwbYnZ0orFyp36Y1fJRItQCt2A0tCmUL/3h5PVq7i0M9lIpQKGsfu8HCqi3ui/zyTZ0xLTWqDf2U1EiM3NQpyIwfB5TLaFtwBwCg9sADA2IXguKbbwJwFbkowmbt2OF+cBw4vTrBsYaLvnLwwqCDJ4YWV//vG/j6nS/jogUvDvVQKgJV7HRU7MCCXV/tyzj40E9JjcQwDAP1nmrXftddAIC6Q2bCbGyM3K+0YYP7oVyGUwx/07c6OvzPSfzxNIY33mntwWV/fRWrtuzoVz99pYDYZU39yBpK3PrM2wCAZ1Zvj244TMErdppwDCQYnytrYjfo0E9JjVRonDOHW6495BBkYogdrd0TRdjsHQEBsHt05YqRjnteXI+7lryDOxaHVytJgr4icXiHniQ0KgfnY6cJx4CCEWd9nQcfmthppELzRz+KxhMDcld3yCEwm5oS7x9F7KzOHYnaaYwMFD2fSzqZVgJqih3pIsut//cWjvuvJ7B2+8h7caFuFPtMiHmZG6Yo9LPyhOM4OO+WxTj/1ue1H3AM2OXRxG7woYmdRioYhoEpV12F2kMPQcP7jkNut91gNibPoxdN7LQpdmcCe56HTaCbOvrQUyzH9tNbJMSuKiMbOlx5/zKsbe3Br54YeZGCGzuCMoHTx1aeO3MoUSBm/Ur4RlehjCeXb8Vjb2xBb0kOFtMIwH732hQ7+NDETiM1MqNGYcZdd2H33/wGhmEg05hcsStv247WBQtQ2rRJ2mZ3alPszgTHJ3bytm1dBbz3vx7HeX94PrYf6mO3s4gk9fmRl0J0xebg95nLjMy8ZMV+Jiimu+ws9+JAgV0erdgNPjSx0+g34oInKNruuAOb5/8IW3/1K2mb1RmExWvFbuSD+cOpTFYb2/tQshysaY3/nvs4U+7ATRJbdxRw27Nr0FWIVxErAZ3gJoyqqaiPzZ19uP25NYmUzmqDEruRSmoKpf6bYv3PVRnRzgut2A0dRt5ro8awg9mUnNgVVqwAAJQ3KhQ7UpJMK3Y7ASJ8bNhDP8ncSk2xAzlH/Gbhavzu6bdg2Q7OffceVe9/W1fB/zy2IV9RHzc8sQp/WrQGpmHgjKN3r9bQEmHF5iDB+EgNYqGKnV2B6ye9/3RUbTR8xV4Tu0GHVuw0+o0MCZ4waqKVCJb6hJUZo7A6tGK3M4FNfKrnuiP8j0KBC54YuEmis68EAOjoLQ1I/5uIj1qlJZY6vbF1DtAYo7CSKHYjda7mfezSnwR9SdG8LhqM0GnFbvAxpMTu6quvxqxZs9DU1IQJEybgYx/7GJYvX861Of74490i9OTvy1/+sr/91ltvlbazvy1btoQee4899pDaX3PNNQN2rjszzIZAscuMGRPd2IuULLcriJ1Od7JTgU18KjJWqWI3kFOEHaEwVgObOgNiV+mJBAEp/R9PWmwgxHSkkhrOx66fptgRKloOGoLfU/+i4jXSY0hNsQsXLsTcuXMxa9YslMtlXHHFFTj55JOxbNkyNDQEUVcXXHAB5s+f7y/XkwLxn/nMZ3Dqqady/Z533nno6+vDhAkTIo8/f/58XHDBBf5yU4q0HRoBqCk2M7oFZa+EmFFbC6evT7mP1dYurbO1j91OBTbvqZSRYIKMnx0HK3giCPYYmINsJsSu0mP4hHgIWAVvUht5rMZxHC71TiXqrzbFJof2sRs6DCmxe/jhh7nlW2+9FRMmTMALL7yA4447zl9fX1+PSZMmKfuoq6tDXV2dv7x161Y8/vjj+P3vfx97/KamptB+NZKDmmIzjU1ANguUy8hPn46CoMAyOL29sPv6YNbW+uto8ISlid2IR6QpNoXyRIMnBnKKYGRpwBQ7qnhV2Eeggsa1c/Dahk7sPaERtblMhUfjQYnMSOQ0Zdvh7jerAiGJqnwj8BJUhPXtvTAATGmpi21LwYizjoodfAwrH7sOr6TUGMGcd/vtt2PcuHE4+OCDcfnll6Mnwkz3pz/9CfX19fjkJz8Ze7xrrrkGY8eOxeGHH46f/exnKJfDI83sYhFWV1fwp4mHDxoVa9TWwPSIdn6PPYI2zc3SflZ7u//ZsW3YXcQ52/uOi2vXou3uu+GUBt+nSKN/iFLAbJ+gxD/0OVPsADIKP6HqAB1jUxUUu6hIY4p/r9yG065/Gj9+cFlFx1GBTtAjUa0SE2VXcg4252M38q5BWpQsG6f98il8+PqnUU7JhAfatUEjHMMmKta2bVxyySU49thjcfDBB/vrzzzzTEyfPh1TpkzBq6++issuuwzLly/HPffco+zn97//Pc4880xOxVPh4osvxhFHHIExY8bgmWeeweWXX46NGzfi2muvVbbf/pubse2GG/zlTaXwmqe7GiixM2vrYNbWwt6xA9nx42E2NsLu6kJu6hQUSC1YwA2gyE6YAMM03XJi5EHJiPPma36KrscfR3bMGDSdeGKi8RRWr8b23/4O477yZeSnT6/CGWr0B6r5LzApxmOwKk+wiXqgovioKbbS82DuSnH7b2jvBeCmlakWRrp7WUEkdhV8z/Qa7Ap8pa9koa3HfakulG1kM8m1IF1SbOgwbIjd3LlzsXTpUjz99NPc+gsvvND/PHPmTEyePBlz5szB6tWrsddee3FtFy1ahNdffx1//vOfY4936aWX+p8POeQQ5PN5fOlLX8LVV1+NGkVk59gvXYgxnz/PX65fvx7Yf/+kp7dTIyModka9S6ozzc3IjB4Nu6sL+alTUVj2Ordfx9//jra//BWTfzQfdYceym1jil3J89crbdiYeDztd9+Njr//HdlJEzHhkksqOSWNKiDKFJPUpAgAfVzwxAAqdt7/Skx0SVANU2yUeZvCv74VHifq2LT/kQRZsUvfB3cNRiS9TQfOdJ3yS2f7ah+7wcewMMVedNFFeOCBB/DEE09gt912i2w7e/ZsAMCqVaukbb/73e9w2GGH4cgjj0w9htmzZ6NcLuPtt99WbjfzeWQaG4O/hpFZUmcgYNTVARnXj8dV7AJi1/KJj6PmwAPQ8J73Svu13/t3OD096H7qaS5wAggUO5YWhZpt48Bqzjq9vanPRaN6iCopxkhfEnNWX4k6vFdnbCrElUDrLzZ3BnnsKjXj+SbsGFJhJ7y+tu3g4jtewvWPxZc4G+n+ZTRtDlCZyd0a4eQ2NahCmZKgDUcfu/Xr1+Pss8/G2LFjUVdXh5kzZ2LJkiX+9vPOO0/KliEGZ6pwww03YI899kBtbS1mz56NxYsXD+RpxGJIiZ3jOLjoootw77334vHHH8eMGTNi93n55ZcBAJMnT+bWd3V14e6778b5559f0VhefvllmKYZG0mrIcMtK+aqdtTHLtPSjHFf+Qr2vOce5CZNlPZjZK68fTsXOAG4ip3jOBURO5YqxS5qc/lQgpEP1XM9ICjxiDPFihN2pRjIiairUOYqWlROChIqdt7/uHYvrG3Dfa9swC8eWRF7ZL6c1vCZrJNCUuwqMsXuWsSOvuSk/V34UbEDJYGnRFtbG4499ljkcjk89NBDWLZsGX7xi19g9OjRXLtTTz0VGzdu9P/uuOOOyH7vuusuXHrppfjBD36AF198EYceeihOOeWUyHRrA40hNcXOnTsXCxYswD/+8Q80NTVhk1c/tLm5GXV1dVi9ejUWLFiAD37wgxg7dixeffVVzJs3D8cddxwOOeQQrq+77roL5XIZZ599tnScxYsX45xzzsFjjz2GqVOnYtGiRXjuuedwwgknoKmpCYsWLcK8efNw9tlnS1+yRjKYjY2wOjpg1tah+RMfh1MqoeGYY4LtEQpnefs2n9gZuRycUglWdzec3l44BVflqITYOQVN7IYSUXnsgiCA+H56I0yxf1r0Nn78wOu49fOz8O69x1U+WBBT7ADM2NQMC/RfsYu7cL4iGtcfDYiwHZhmeOJkPnAgpuNhCMnHrp/pTnYFUyw9w7Q8eChzLqrw05/+FNOmTcMtt9zir1OJSTU1NamyZVx77bW44IIL8PnPfx4AcNNNN+HBBx/EH/7wB3z729/u/8ArwJAqdjfeeCM6Ojpw/PHHY/Lkyf7fXXfdBQDI5/N49NFHcfLJJ2P//ffHN77xDZx++um4//77pb5+//vf4xOf+ARaWlqkbT09PVi+fDlKXmRlTU0N7rzzTrzvfe/DQQcdhJ/85CeYN28ebr755gE9350ZppfyxKyrxehPfxoz/vZXZMePD7ZH1JO1tm7z1bus94Nyunu46hQVETut2A0pokybaaJiaboTcZJ4cU0bipaN/6zvQH8xkMETlJwClU92qX3sYq5vfT54t+8tRSufI92/TCR2lSiz9BoMF8IykODPt0LFboATFHcXy9jRV/L/whT8++67D0cddRQ+9alPYcKECTj88MPx29/+Vmr35JNPYsKECdhvv/3wla98Bdu3bw89drFYxAsvvIATSWCfaZo48cQTsWjRov6fXIUYUsUu7qEzbdo0LFy4MFFfzzzzTOi2448/njvWEUccgWeffTbZIDUSwWx0FTmjpla9nSp2Xp47hnJrKywvYjY3aRJK77wDq6cHZZLEWBO7kYgoU2yKqFhKioQdrCqqAn66kwGYscVJsfLgCbZ/Mh+7ONTkgnf77mIZDTXqKcFxHMEUm6j7YYVqBE/wJcVG4EVIif6YYgfy90Rx0vXPw6x5zV/++px9MO+kfaV2b775Jm688UZceumluOKKK/D888/j4osvRj6fx7nnngvANcN+4hOfwIwZM7B69WpcccUV+MAHPoBFixYhk5HzQW7btg2WZWHiRN7VaOLEiXjjjTeqfKbJMWyiYjVGNjKjmgEAJqkKQkGJXX76dBRXrw42WhaKa98BAGQnexJ4qYTyls1Bk1TEzg28YGZcjaFBVB67IHgivh/Ox04gNIGC1f/JYyDTM0jErtI8dgmvWyWBID0FCwgpviN2MxLz2IlKTpJzaO8p4qaFb+L0I6Zin4lNfMqXkXcJ0oOcY8U+dgNM7B752ixMmTLVX85n1YZI27Zx1FFH4aqrrgIAHH744Vi6dCluuukmn9h99rOf9dvPnDkThxxyCPbaay88+eSTmDNnzgCeRXUxLKJiNUY+xpxzDppOOQVNc96v3E6JXc2ee0rbC6vdKOfcpCAoprRuvf85FbHr9hQ7nWtwSBFN7Lz//Swpxsym1VBPBjJBsTi3VXqIpBU7khPAoEF3MSJBu0RMo/sdjqgkQfH9r2zATQtX4+Z/vyntMxKvQVr0p4QarQc9ULkhAaAhn0VTbc7/q8mqK61MnjwZBx54ILfugAMOwNq1a0P73nPPPTFu3DhlFg4AGDduHDKZDDZv3syt37x585BWtdLETqMqaHjXbOz2P9dxfnUURj7vmmAB5PeSiV3ff5a626ZPh+HlESytX+dvt3t6Eke5+lGxOnhiSOGraQoXm8DHLr4f6vslNk/qc5YEfvBElSYh23bwxT8uwfz7l0nEs1IftaS1YpPmCaTbe4rhPnYi2R2JnKYSH7uugntNerx7kDPFjsirANz1/Fp89Ib/w9Yd8RYNeo7pFbvg80BVc0mDY489FsuFEpcrVqzA9Igk9uvWrcP27dulLBwM+XweRx55JB577DF/nW3beOyxx3AMCR4cbGhipzEoMAzDV+1q9tpb2s7Mpvk9Z/jm3OL69Vwbi/jchcGxbT+5sfaxG1qwR7k6eCKFKbYUXridZVKohmnQD56o0iS0oaMXj76+Gbc/t0ZBSF3i19GTrlRe0usWpJpJRgABoLsQrthJ3Qz9PJ0aomKX5GtmqTqY4sQrWFUb2qDiby+uxyvvtGPxW62xbfuj2NHf6nDIZTdv3jw8++yzuOqqq7Bq1SosWLAAN998M+bOnQvATZn2rW99C88++yzefvttPPbYY/joRz+KvffeG6eccorfz5w5c/CrX/3KX7700kvx29/+Fn/84x/x+uuv4ytf+Qq6u7v9KNmhgCZ2GoOGUaeeipp99kbDu2b760SFr2bGDJ8AUlMskMwcS5MSax+7oUWUalSpKVacH6qq2FXZ2ZuW/xJNUY4DfPFPS3Do/H9hxeYdqccYm3jYv74x/ZEWkYqdMP6R6WOXXrErecTOUpj8R2rwBDuXJNGqXDqclKdLL89wqD4xa9Ys3Hvvvbjjjjtw8MEH40c/+hGuu+46nHXWWQCATCaDV199FR/5yEew77774vzzz8eRRx6Jp556iqtGtXr1amzbts1f/sxnPoOf//zn+P73v4/DDjsML7/8Mh5++GEpoGIwoYMnNAYNk394JQD3gZhpaYHV1YX62bPR+cADAIDMmDHItLT4il1JVOwSEDtmhgW0YjfUiFKNwnzAbNvBv5ZtwuG7j8bEUW6Eddkik6nQT9IKC8nG66Ja+VTp+asI6eNvuAlMb3t2DeZ/9GBxd3WfiX3s/EEka4doxa5aUb1DiWIFwRNFi7+H+Tx2IxOVBglVGjwBAJY1PK7WaaedhtNOO025ra6uDv/85z9j+1BVp7roootw0UUX9Xd4VYMmdhqDDsMwMO23N8PesQO9S4Mw9fyebrJIptjZO3glQxO7kYWo4IkwRenO59/BFff+B6Prc3jp+ycDEPxzJFNs9cyn1YywBfjzjyKepRRMMqkpNq0vHhCt2InizkhUq4pW+uCJsqDY7QzpTtgpJCFq/as8EXwe6Fx2Gjy0KVZjSFA3cyYa3v1uZMeO9dfVzHCDKsQqFYZXoiwRsfNqzALQJcUi0N5TxHfu/Q9eXNs2YMdwFCqHv82vPMFv/OdrbvWZNuJ7xpm/pGMg9BhpUXVTLDl/OSo2WCGaCJP1mWyM8aXHggapomITHX14oVBKn8fON8UqTOAjlNel8iWlTSqNigWGR/DErgRN7DSGFNnxQRmovJcGRaxSkd9jDwDhxM7u7fW3acUuGX784Ou4/bm1+MSvwxN79xe+NTCFYtenqH4QlRh3WCt25LM4sdFzEp36k/YZhaRpYDjFrhCh2EWMf6RAVOwS+dix+0sRPDECLwEA6mM3sIodvWWGQ/DErgRN7DSGFBmq2Hmm2PpZR3FtambsASAgdoWVKzk1bs1552HV++egtGULT+x08EQoVqZw2K8UvsknhY+dSr2ik4JMMJKZJtOgWpMQP7GFR2SmIXZJS7EldLFLnMdO+g5HoAJTSa3YUpk3xfanxNZwgZ+8OhGxo58rV+zKw8THbleBJnYaQ4rsOFmxa/7IR7g2VLHrffllvPnhj+Ctj38CTrEIx7bR98qrsHt60HHPPZJiN1L9YAYag/ECHdReVW2T2wGyYiflfxPGXc1qEdU2xVJaJU5s1AQqKkmRPSaMAk6eFiWAqNhtaO/1SafE62JHOvwgJShO8D0zVYsRW0pwR+qjxUn1m6FENt1xuDx2WrEbVGhipzGkyI4bh5p99kHNvvsiN2UKACDT1IT6o48O2ngZvK32dvQtXwEAKK5ejc0/+zkXYNGz5AW/6gQA98lbDlchdmUMhtrAjhCVxw6IVq8k3zSpn/BjpEW1TbFRE1ulptj0CYqTm2KpYvfS2ja8+5rH8Zmb3ULm4vhHIqkRS4olEZGKQh47ej1HrmKXxhQbfO5XVOwIvVYjFZrYaQwpjEwGM+69BzPuvQcGKbI8/mtu6HjN/vsj09ICwCV2VJFr+/Of0bt0qb/c8/zzKG/fzvWvq0+oMShv0D65kDeF+SrF5RqTExRXzxRbbcUuMo8X2ZgmKjZ5uhNGAGN79D/RqNg7F7u1m19a2+4dT20CH0kolqPvJRX8qFiF+jwCLwGAdH6pnOk5tY9d5f55Gv2DTneiMeQwsvJtWD9rFvb421+RHTMGxbXuJGO1t0spUPpeW+Z/dgoF7HjsUW67Wy+Wj7LtLxzHgWEYVe1zsDEYk1KUAiYnenWvp2iKjZt8qqmyMRWsWu5AUeWYKlfsvL4Tmljj06IEn2keO1HdktOdRPc7HCGm3EiWoJgRuuqquUOJ4AUmeVsgvepGL7f2sRtcaMVOY9ii7qCDkJs8GZnmUQAAa8cOWF08sSsKySL7XnmVW652AIVdLOKtj3wU67/xzar2O9gYFFNshJmU87Ej62Ufu/D9aN/VMcV6/ytQF3731Jv4yYPLuHXcxCYqj+Ss06Q7CYJOHDy1civm3v4iWrtlVTrpdeGiYoliFxdoMBKnaZFcpEt3Il/PkUryAr/UBJUn+qG69Wdfjf5BK3Yawx6ZUS6xszs6YHcKxG7tmsh9q53ypPjW2yisXIniunVV7XewMTg+duFm0rAJsk80xcaYAINasf0ZqYd+mGJ//ODrAIBPHTUN+05s8rqjE1tEVGyFCYo/9/vFAICeYhm3fP5orl3gYxfdH1VOqY9dnHI6EoOSKklQHJQUY/sE20bgJQBAExTHt6XnmD4qNvisExQPLrRipzHsYXrEzimVUCY1+gCguMYldpnRo5X7Vl2x29Hp9tvXNyInN4bBiYp1/6tMOGETZFxNUrEnqmD1F1El0KJAFT5qVo3ysas4j51CBX1i+VZ5TIzYxfVHPtOo2D4pma9I7GKHOuzA/OVyGdfsnygqNsIUOwIvAQBSRSPBl8gRu5TcbGcINBmp0MROY9jDbGgAvMAKVj82t/vuAABrq0v0ameqa21Wu/qExXz8HAdOqRTdeBhjMB60URGrcVGdfruYUla+I3gVBIE0pZYoqBKUzQS+l1GElUt3UlFULJA1w/08wyp7hPUH8Iqd5GMnmsRHIK1h/nI1WfdZkoTYiKbYqJyKIwVBupOUptjUil3QXvvYDS40sdMY9jAMA5km17xV2rABAJCfNo1rUzNjTxj19dK+1TbF2l1dQd99fVXtezAxOJNSOOlSKXZlRWUAcTIRR13V4Ak24aXsi0a1Zs3gkUrJj5THjixWEhXrOMBBU5v99du7CqHtojsMPvYULf8aiIrdzpDuhF3nmqz7HSXzseMVO0dx3440pDLF0v36UStW+9gNLjSx0xgR8M2xnmk1tztP7MzmUch7Kh6FU+V0J1Znp//Z7h3BxG4QXF6icqnx9V/dz91CgtySZcuETVgMVMH+jZXrK61iRxS3jJlQsSMbK1LsHAeNNUF6oFfXdSj7j813Rz5btuMHTciKXTTBHgmQiF2iqFgvj53iBSKJ+b9YtvG7p97EikGo9JIUaV6GqqbYaWI3qNDETmNEgAVQMOR3myZsb1YTu1KVFbsdRLErjGBiNyimWDaByNtUykdnH2/aLiqIXXhJsWr42LlIr9ip29O1UlQs9bFLo9iR/7SPl99p59olTosibGeRsVJUrDDEkWiGZN9BTc4lxEnOQaw8kbZW7FMrt+LHD76Onz70RrrBDiDSVGuJejkZyH01+gdN7DRGBERiJyp2meZRyE8nxM7LM1ft4AlrB1HsBtgU+9cX1uGLf3yeyy9WLQynyhPsc5dwnqWyLedPE/pJk2w1FmzyTqlmUlNqmMO46M9E57kwYqgCVVto/yKx89XShP0xsHstLnhiJEp27DrnM+60l4TAMzWVfX1U5Utyy7F7Wry3hxJpfEn7EwCh050MHTSx0xgRMJsFxW736dxyprnZD6gAAiJYdR+7HYPnY3frM2/h0de34IU1bVXvezCjYlXHUikfErGzHGkyEeeWIFrRXU5j1gwbU9rUDFTdCov2jcpjlwZsaI7D9/9Oaw/fLqG5TdzqK3Zx6U4Sjnc4wTfF5tKbYlUvEEmITjUro1QLaRQ72iTtC482xQ4dNLHTGBHINBFiZ5rITZ3Kbx81iiN7LP2J3duLwqpVVUtNMpiKHXO4H4i33cFI1eKbDeN87LyPOwRTrMrHTiRENPL2V4+vxCE//CeWruf9zZKPl13vdPuFKXZIGDyRBg4hbFHEUTGEyP4YWGRsX5yP3XBiKglRriB4ItIUm2B/n9gNIypsC+cUBa4sWGrFLvisFbvBhSZ2GiMCGaLYmY2NMBvqYeRywbpRzchP2y1Y9tpv+cW1ePO0D2PHQw9VZRy8j111zbxhGIhJYTAetFFRpir+s6NPVOwUplihK9a34wBL1rShr2RXTuwISUwDSuw4hSNKsauQGDnkf5Spi6ZFSdIfA8tlJ5qHxdtlJM7TUrqTJIqdb4qVFbskv8vE0cmDCN8Um8AFgFfsUp4Ed//rBMWDCU3sNEYETOJjl2lsdFOgkKTEmeZRyE6ahNqDDkLNPvsgN2kyAMBqbQUA9Cx5oSrjoLVq7d7eqvQZhoGcFAbVFKs4mGqClIldvCmWKlhsoq7U7FNxHjualJjMZrzzuFB5Iv3wAJDr5vB9iClTkpLUMMVORFQevpECMSo2Cbku2eElxRIpdikiUAcLvik2iWJHq6doH7sRA03sNEYEqCnW9HLaccRu1CgYpok9/nI3Ztx7D8zaWm7/wurVVRmHRYjdQCt2UVGl1ep7IBFVUowT7LwF2cdOYYoVlv0s+nZgmhTz4SUerx88kZLYUcWOHJqOVa480T/yKQZPhCp2saZYfrknhNjtFJUn/KjY5METJcEdQhU8sWrLDvz8n8vR0SMnLE+qnA4m7BT3Ob1E6fPYaWI3VNC1YjVGBDhTbFOju84jdmZ9YJY1TBMwTRj5PLd/tYgdp9gNsI+db3YbgFl0MCZmP5IwaVSsoNgVLRu1Dv/uKfZE89ixfipV7NiQ0ioTRS54gp5X0CYuwW/ZspHNxL9nO4SwRQdnsP9xih2/zIInRMSZxEcCmFmVmWLjbhPbdiRCxwf9uAsnXvtvAMCG9l5c+5nDpD6A4XW9giChJKbYysmZ9rEbOmjFTmNEgDfFeopdS4u7rblZam/U1HDL1rZtsNrb+z0OTrHrG7mK3aD42CF8/A43QbqQgifKtlzKSlgOJs5gEk6TPkQ13rTXhh7PUUz8gJp40WTGYt640DESxc6JmHQpAYzsT1gOO/WdIXiCmVWTJiguETarMqmKu7+8rl3qI8gnOHyuV7rgCbJfylOIUqw1Bhaa2GmMCGRGBeQtMMW2eNtGSe1FxQ4ACm++2a8xOMUil+LE7qu+j53jOEGiXv9ZWP2H4qCYYgkJkbfJZGiHYIot2wofOykqNphw+2+KDR9vFLioWDo+qlhIUbGOn08NAPpKaqVMBDXt8bnw1D52caci56dT77BzpDvxTLF+VGz0WdBIZpX6LJK1jCHX7g2iYocP0phi06Z34fcNPmvFbnChiZ3GiEBmVFPw2TPFZj1TrJLY1SiI3apV/RqDRerEAgOj2H3zL6/ikCv/hRfWtBJTbNUPMyimoajxq0xaYvBE0bJjTZg0DQX7XHnwRGWKXTEkj110VCzfR1LFjipAnGN7qI9dOlOs7fB91eZ4EsRExmEkQCUCVXRZ5Yk4/k/JcpJ0J1SBZUjq6ziYSGOKpS3Sm2K1YjdU0MROY0SAmmJNL5AiM3as+3/MGLm9QrErrlYrdn1vvIGO++6LHQP1rwMwICXF/vbiOgDADU+sHvnBExHRd9zxvY+ij12pbCuJB7dMlJTAx65CxY4cI43pjAueoIoOR7zkSg50e3JiFxAF2mXZ5k2zPgGM7VEmhFQ99P3RvGNlTS+idFhpUPGg5vKkUbF0H1WCYvEaRBG74RIVy90jKStP9CePXepUKRr9gg6e0BgRyDTJit2oD34QhZUr0XL66VJ7pSk2JIBiw/+7DIUVK5DfYw/UHXJI6BisHbxiZ/cOXPCEaRBz2kDksRtiU6yq8oSYakOV7kRFRNix2OQhJgNOPmB+fBl5nlaCT1AM5WeVYkdXJTXFglxT8SzpmJOalVWW2F4ylnyWjyA1TQDW4KTLqSbod1STTRYVK5q3bduJLCmmInasi2HC6zjVLckzgL48pCVn2sdu6KAVO40RASObhdnQAAAwveCJ7OjRmHzllaibOVNunyfBE57vS+FNmdg5joPimjUAgL5lyyLHYJOqEwBgD4Bix2AYxoAqdoNtipXUEYXvjkjISqlMsY5/nSoNnqg0ApDLY8cpOmScqqhVsiq1Ygf5mlKlMmnwhJx42EGvIjKW9ecrdiNsni5zil2yqFjxfrTIPaba31T42A23dCdp/d74ezjtscIVa42BhSZ2GiMGrJoES3cSBarY5XafBgAob90mTYZ2R4efj66wYmVkn5Zoih3AqFhOsRtps6gHVYAEg8pXia3KebJT0VJExQr9U2UqSFDcP1Ms6y8pwitPhCsWYh46sTZrGAIfO3mMlIg4wv8wiLqf4/DqoW9O904xEKVG1j1JzeXs/opToIoCk7GEYB4peEJliiVR28MBaV9eqhU8oRW7wYUmdhojBtlx493/nm9dFGjwRM0eM9wPpRJsIQCitHmL/7mwMprYiT52AxEVy6B6+x9pCCM54nKg7LmfWLRoybKlCVHlR8Y+VyuPHZBOsaMKocp3UNmfw1OjvpSKnVgrFuDPu9IExQ4czhQrmnRZrr1hwlMSg5H9XMbwCVjcdyy+IFBVGJCprdrHzms7TK5X2nvc6Rexq0wB1+g/NLHTGDGY+O3LMP7rF6P+qKNi29LgiezEiTDr6wEA1vbtXLvyls3+58LKlZFv1oOr2BncJD7QWN/eW3VVgVfAwreJZkPm11W2HLUJ0++TV0/YYqXpTriJKMW1KJRDfOwiolYd4XhJFbsoVZEeI6naKxNucKZYR2jHyMtwCQZICqZmZk0TZsJzKJXl68v72AmKnSrdyTALnrBSkq1KX3b6u69G/6CJncaIQf0RR2DcV74CIxsf80NNsZnmZj9yttzaxrUrbw6IndXeLhE/CrvTJXas4oUzgJUnDM4UO2CHAQA8vHQTjr3mcfz6yepU5/AR8bavqrnJiBDzgSoRU6xPKGz1xMSlO6k0QTFVAFMpdmofOyr4qNKR0OMlT1AcpdjJ0blpr4TtOJx6KL5cZL3vYaRN08ysmssYvhoeS+xExc5W37cMkabY1CMeGKR9eeF88rRiN2KgiZ3GTglaeSLT0uITO6utlWtX2rKFW44yx1pdLrHLjhsHALAHsFYsVewGmti9uc01T6/e2hXTMh2iTLGqc2LtmWJXJKZYpobwSh/fPztGqWJTbGUTUVgeO9qDbNbj+0ieoNjr25HJgtLHjpzT82+3YuGKrdw+qpgWTrFjpljfx85Q7jfcwa5NLmP691LcV1wSyHZc8ER0HrvhccGclFGuVHXuT61Y7WM3uNDETmOnhJEjil1LC7K+YscTu/JmkdiFJzG2OzoAANkJEwAAzoD62MlmsIHCQCmD3KQgEQhZzfN97DxiVyo7vkrgT5ohCoLt0HQn/Q+eSKNOlMLy2EUQRXGSTBsVKwZfiMfwzdtk+Qu3Po8v/vF5rnSbHDzhqIMnfB87g1s/UlDyFbvAFBtH3sXoail4IkEeO8u/t9OPeSCQlmzxUbQpjxWhWGsMLDSx09gpQYMnMi2BKdbaLhI71xSb8VS4wsoVoX2Wt7lm2txuuwEA7AH2sRMn54FCEPlY3SOJihqFMiqWKXYkeEI0xXLkSyAyaTLqx443xSTGlxQjfZAFNiY294vXOm2tWMeRyUKZux7eGPxIYQc7+sooWQ46egNiJ56n7fB57ETlz/8eRtg8zb6jbMbwv4PUplgnOo+dKuApcDMYHkjvYye/gCUFn8dOpzsZTGhip7FTgvOxa2lBdozrFyebYl1i1zB7NgCguPad0D7L27YBAHK7TQUQ72NnF4sot7WlHLkLwzAGLd0J677aSYu5QAKJQCjMht5/X7GzbH8iDUpZhStiVj8Vu2oET/CTnzyJ+hGZQv+FcrwpVkwfIyt24T521EzbV1ITUdavyhTLLqnKJD4SwEhvPmMmDgARTbFiVKy4f0Yxm/q1Yqv02+pvP2nTl0S9nMUfK/hc4U9So0JoYqexU8IUiF1mdFjwhGuKrTvsMABAaePG0D4Zscv7il00sXvn/C9i1Qnvr4jcUVPsQKsjvsm3yopd1CTCkT7BFFvjE7vA9MXSbIQlWKVlwCpW7OjYK0x3Epa7jxErVRAIwJOtMPD+e3LwBD8Ov6G7jZA+lak1OIaY7sTx19PxjzhTbDlQ7AzmYxdzycX7SDTFioguKZZmtGpsaO/FMVc/jl89Hp2WKQpOyO8nDP0JgNAJiocOmthp7JwgkbM0KtYiPnZOsehHwTJiV960CY6Qwb+4di2cUsnfN7ebm/A4TrErrFgBp68PpXXrUg+fC55IvXc6DEZaFZnYyYqdGDyhNsXyZI72x1QwsRRUUqSd9BhKoelO5P6Y4lWJYidGZIrkSiS6dB86RnosWbFzuNQroinW97GLHe3wAguoyZpm6Hcg7SOVFIuLipWn0yAqNv6KbWjvxbaucPeOV9d1YFNnHx5/Y0tomzikJWr9Uexoax08MbjQxE5j5wR5CGWam31TbNkzxTrlsl871sjlULvfvoBhuGSPkL+2O+7A6pNPwdbrf+X2aZrITZ7k9lEqwbHCJ2Sm6Nm96YMsTJP4SQ2WKbbKL9X8pMBvU1eeEBW7oKRYVuHbJSqCtn8eFSp2KR3LGYqJKk+4bcKS4xYSKXbq81WPWTDFkm2cKVYk3OB97NgxA1Oyya0fKWDENpc1lWZ95T7KkmLhKrSqtnDSBMV9JQun/Pe/8bEb/i+0TTV8blPXiu2HYldplLlG/zGkxO7qq6/GrFmz0NTUhAkTJuBjH/sYli9fzrU5/vjjYRgG9/flL3+ZayNuNwwDd955Z+SxW1tbcdZZZ2HUqFFoaWnB+eefj66u6qZ70Bg65GfMQOOJc9Dy2c/AyOWk4ImNP/gB3vr4JwAA2fHjYeTzyI53K1tQc2zPc4sBADv+9S8AQGbsGD/ZMQC/HJkIx3H8bZXkuxvMdCeBYlbdA1GVQmXyE7exVSyPHU13Yip8u/hJNlBHKq0VG9Z3HMKjYoM2oo+d2H0yHzt+rKIKxPvY8fvQMfYpKkvQ8asrT7j/M4Z6v+EOv/KEaaSIilWVFAuWpeCJfkTFtveUsKNQxrq28EThgXoa3VcUuBejBC9y/D2c7li6pNjQIT7T6wBi4cKFmDt3LmbNmoVyuYwrrrgCJ598MpYtW4YGr+A7AFxwwQWYP3++v1xPJlaGW265Baeeeqq/3NLSEnnss846Cxs3bsQjjzyCUqmEz3/+87jwwguxYMGC/p+YxpDDMAxM+9Wv/OUsMcU6pRI6H3rY35bfZ28AQG7yZJS3bEFp40bUzZwJACisctOfFN9+2+1n3HgYtbX+vnZfH0f0GCjhi/PFU8E0Aie7gQ+eGBhTbFRyU1lbIlGxnI+du06Z7kQooRWUFKtC8ESFeexUJmZ3TDyxq0SxExVQcYgqXz9G/koJgyfcyhPydsnHLna0wwtFksfOTJjHTgzCcZNCO6HLqsoTSWvF0t9HyXKQzw5MTjyVihwF3vUh3XG5Fy9N7AYVQ0rsHn74YW751ltvxYQJE/DCCy/guOOO89fX19dj0qRJkX21tLTEtmF4/fXX8fDDD+P555/HUV55quuvvx4f/OAH8fOf/xxTpkxJeSYawx1MsXNKJXQvXgynpwcAMOVn/4X6I48EAGQnTwZeeQVlT7Gzi0Wf0DFkx42DYZow8nk4xWKoGkfNr5UpdnK06EAhMMVWWbETFLWwbb4p1mGTrzuplcp2kD9N4WPHB2AEx6hG5YlUxM4K8bFTEMXwqNh0plgIpEIcc+Bj5/4vp1DsVMEVIrEbacyuTNKdsOjVOKJSVOSx41LsgFeiIhMUx4yPEp+ybSOvMKb5KmxMX5HHSanARUUBx+7LndMIu2FGOIaVj12HlwB2jDcJM9x+++0YN24cDj74YFx++eXo8SZlirlz52LcuHE4+uij8Yc//CHyrWbRokVoaWnxSR0AnHjiiTBNE88995xyH7tYhNXVFfx1d1dyihpDBLOuDkZdHQBgx8P/BAA0njgHzR/+MHIekc9NngwAKG3cBAAovvU2IPjQsaoTTLULy2XXX8XOGERT7EAFT3CmTSlBL23HT360pJiYPy08KtYhpthKgycqUydKIT52tAtfsfMjMvn+k1SeEE3PbJER4bJAOugHTrEryz50dEcV8WOnmB2hPna08gSLio0j76JiJ5pi4fAvEcoExV4XcdeLbhdr1PqHq8LvVPzNxCHqZadMfp9p99UYWAwbYmfbNi655BIce+yxOPjgg/31Z555Jm677TY88cQTuPzyy/HnP/8ZZ599Nrfv/Pnzcffdd+ORRx7B6aefjq9+9au4/vrrQ4+1adMmTPCqBzBks1mMGTMGmzZtUu6z/Tc3Y8VRs/y/Nz/4wX6crcZQIOvVeO38p0vsGmbN4razoAjmY1dYJacVYMTO9EqWhVWf6L9iF+SxG/DgCe9/tTMSiIoaf0xZeWPn6ZtiSXoJVZoNSyBioqN/f8abLipWPVnSc2T9mSGKXV8SHzvuc3C+OU+CsmzZJMzGUObSnUSZYh1lguKRb4oNasUmLimmMMWKwRP0uqqIneg/GgZ6v4mJkYO++P+VgAsQSvACFBYssqOvhHdf8zguvvPlRPsOF8Vu/fr1OPvsszF27FjU1dVh5syZWLJkCQCgVCrhsssuw8yZM9HQ0IApU6bgnHPOwYYNGyL7vPLKKyUf//33338wTicUQ2qKpZg7dy6WLl2Kp59+mlt/4YUX+p9nzpyJyZMnY86cOVi9ejX22msvAMD3vvc9v83hhx+O7u5u/OxnP8PFF19ctfGN/dKFGPP58/zl+vXrgSH+8jTSITNmDEobNsDu7AQA1B99NLc96yt2HrFT1I3Njq9EsUtWoYI+dE0Dg6bY+apMtRW7CAUssvKEX1LMDhLjKggF178Nku6kQmLHjS95H8WQ4Ak6P/umwJA8dknGTJM8U9OzS+wsZR67uOAJ8eayHXUwiFx5YnhM1EkRmGLN0FyCIlQlxbiXFfCKnaryBLsn43gNR4JC7gVG0vvDkXjTanz7sMjWt7f1YMuOAha/tT3ZsYYBsWtra8Oxxx6LE044AQ899BDGjx+PlStXYrT3wt/T04MXX3wR3/ve93DooYeira0NX//61/GRj3zEJ39hOOigg/Doo4/6y9ns0FKrYUHsLrroIjzwwAP497//jd285K9hmO1VCFi1apVP7FRtfvSjH6FQKKCGFINnmDRpErYIxd/L5TJaW1tD/fTMfB6gSW9JcIfGyEBuyhT0LV0KADBHjULNvvvy2ye5xK7sK3Zu4ATzpwOIYucRO6egVuOoSpe0pix9cJqmMYi1Yr0Jo9o+duSzTOxk0scOX8PlsRMJRdCHxREdWlKs/6bYNNbcYoh/HD3jOMUuybWXTbGiYqe6pjLZVeWp85cdNemOItgjAez88xkTjH/FVp5QKHZ8GTteiVIRu2Bz9LHoNQ9zJWC3dbWCJxLlsSOfud8bc5+I6CJtoMZA46c//SmmTZuGW265xV83Y8YM/3NzczMeeeQRbp9f/epXOProo7F27VrsvvvuoX1ns9nEPv6DgSE1xTqOg4suugj33nsvHn/8ce4ih+Hll18GAEz21JWwNqNHj1aSOgA45phj0N7ejhdeeMFf9/jjj8O2bZ84aux8mPD//h/GnHce6t/1Lkz4xjdgZDLc9twUj9ht2wanWERxpUvsGt79br9NRvSx6w0JniDELqliRyd7w8CgzZ7sMANZUkyaQ5SH8iZfQuzEdCduvzxpYf37dVErVOzC/PfiEKbYqfLiZf2oWL6PtDnFHCdQj5Q+dkyxY8fnfOzIeEXfR0VQhkNMkKp8giMBzLyZNY3ECYrlyhPyCwklLKokxEFUbPT4OFNsCLFjLfpz7bk8dkleJkJ88sTgHBUGy8euu1jGjr6S/xeWOui+++7DUUcdhU996lOYMGECDj/8cPz2t7+N7LujowOGYcRm2Vi5ciWmTJmCPffcE2eddRbWrl1b6elUBUOq2M2dOxcLFizAP/7xDzQ1Nfn+bc3Nzairq8Pq1auxYMECfPCDH8TYsWPx6quvYt68eTjuuONwyCGHAADuv/9+bN68Ge9617tQW1uLRx55BFdddRW++c1v+sdZvHgxzjnnHDz22GOYOnUqDjjgAJx66qm44IILcNNNN6FUKuGiiy7CZz/7WR0RuxMjv9tUTPz2ZaHbM2PGwKipgVMoYOuvf43imjWAYaDppJPQ9eSTANx0JwDxsQtR7OwKFDv6UjuQeezow9owKFGq8nEiTbHhZsO8p0AVLYc47Rtce8MIT3dScfBEhakdONMlp2oE8BU7Q23KTKLY0RaOEyyrfOzEeyc0j514DEdOfOyqeLziyNoaCpVqOIIR22zG9M8h7isWldg4U6yqP/a9x91P9F4O80cLImwr/6FyZCvRy4R8fLcfn2ZG7Bt/TtXASdc/D7PmNX/563P2wbyT9pXavfnmm7jxxhtx6aWX4oorrsDzzz+Piy++GPl8Hueee67Uvq+vD5dddhnOOOMMjBo1KvT4s2fPxq233or99tsPGzduxA9/+EO8973vxdKlS9HU1FSdk0yJISV2N954IwA3CTHFLbfcgvPOOw/5fB6PPvoorrvuOnR3d2PatGk4/fTT8d3vftdvm8vlcMMNN2DevHlwHAd77703rr32WlxwwQV+m56eHixfvhylUslfd/vtt+Oiiy7CnDlzYJomTj/9dPzyl78c2BPWGNYwDAOjPvQhdNxzD7bf9BsAQMtnPo26Iw732/g+dl6EbaiPHVnPVL24iZC+/dN0J9U2xdIHuksgveNUPd1J8DnSxw785BdlinXbq/tnfQ5+8IQ6GEGl2IXlsUut2JHgCaZwciRDGEfSBMUOZNLpkGOrCPZIADv/fMbwyXVsVKwdZ4p1BJVUodj5ZEyN7/79P+gr2TjnmOnSWCUkUMniIJpi455JqpcTOoaosfSnakUaPPK1WZgyZaq/zH4P0nhsG0cddRSuuuoqAK4//tKlS3HTTTdJxK5UKuHTn/40HMfxeUoYPvCBD/ifDznkEMyePRvTp0/H3XffjfPPP7/S0+oXhpTYxfkKTJs2DQsXLoxsc+qpp3KJiVU4/vjjpWONGTNGJyPWkDDp+99DYfUq9L3yKvJ77omJl10GI59H3VFHwmxogNnYCCDwsbN75dQ7AGATlc4p9GHd1y5GaeNG7HHHAhi5nHqfMMWuGidGwPnyGdSPauAevqKLjag+0XUBUZHTnbjtHQAGR4b4qMIqELsUZDrcFCu3zYb42CURGUXTtki2wkiG6AsWFxWryjko+tip9o0fv4OeooWGmsGfdkpUsUvqY1cWvyNH+n7L3Hcv9xGluhfKFm571jXZfeTQwFIUFkhT7QTF7rK6FBpDeABU/FgqdW1Ii4Z8Fk216mcqxeTJk3HggQdy6w444AD87W9/49YxUrdmzRo8/vjjkWqdCi0tLdh3332xyvPRHgoMi+AJDY3hArO2FtNuugkd9/4doz5wKkxPmdvjttv4dl7wjB2Sz5BT7Lp70OW9oBTXrEHN3nsr96EKgTGA6U54YhcQyOpXngibFET1yfvvfQjy2AUKiYpQhEUSJknjoAJnik04EblqWAihUrQ3QyIy0wZPUB87nwgrfOzYOMIVO3miF78rquJRxc52HGSQXLL76u0v4qGlm/Dope/D3hMaE+9XDbDzz6WKihVMsY5YUkxQ7FQ+dhG/YZqUmh4r7P5VKdVpIY7Dsh1lmhb/mCHkLE6xi0qePVQ49thjpZKlK1aswPTpRC31SN3KlSvxxBNPYOzYsamP09XVhdWrV+Nzn/tcv8dcKYZNHjsNjeGC7OjRGPuFz/sJi1Vgyp3dpSZ2VLErt7b6n0ubN4f2yak4McpPfyCaYhmqnqBYfTreskz6RNNi0bL9iSMjmAABfmIuCXVSKzErV6IwiOpKmE8SQ5jjfhKFULyebNlXASOCOEohwRPy9yJP1tR/UfU9LF3fgaXrO2LH/9BS14f6z4vejm1bbZR9YmckLikmKr+2LZYUS+5jp2L5pRBiF6vYRQ87EqrvNro9fdkJ1vsqe8j+4nGGQx67efPm4dlnn8VVV12FVatWYcGCBbj55psxd+5cAC6p++QnP4klS5bg9ttvh2VZ2LRpEzZt2oSilxUBAObMmYNfkXKV3/zmN7Fw4UK8/fbbeOaZZ/Dxj38cmUwGZ5xxxqCfI4NW7DQ0KoDZ6Cl2XV3K7VSxK2/bFnzesjW0T9Ub8UDA4nJvBQ/nqpcUiwhGUJI+pkBlFD52NCoW8njFsZdsGzUmH/UcO94IhTEMRUFdiSKzAFXs+PWpFTsE14EFTygrT4ARkLDgCfF7UUXFyulO2L7Fso3P3vwsDAAvfv8kfyxREEt1DQYYScuaQa3YODJdUgRPWMJ3ULYTmmIV/dN7h16TsNQg1VDwxfssjnBxLzuK30fYUFTK4FBj1qxZuPfee3H55Zdj/vz5mDFjBq677jqcddZZANzkxffddx8A4LDDDuP2feKJJ/xYgNWrV2MbeaavW7cOZ5xxBrZv347x48fjPe95D5599lmMHz9+UM5LBU3sNHZZlCwbv/jXCsw5YAJm7RGUsduyow+/fmI1zpy9O/adqI5qyviKnZrY2SRaliN2UYodJSrcm3J1H4p8WhUSPFHlZy/vEyYqW3SZn/xoguIopYjP/dX/iYTukdSaK07+KhMzRSbEcT+9YheYBVWmWDE4hapPhZhasSriEPjzBcTNcYCiZaGrUAbg/p6SELuSZeNfr23CK+va8c2T9xuUyFr2PeWyBtgpqNSmhSu24pnV2/Ctk/dTBk9EmWJV33iUmwP14QtT7yiSVrGIgviziPudhL2cxamH4vkOhzx2AHDaaafhtNNOU27bY489Evkvvi3UD7/zzjurMbSqQhM7jV0W/3xtE25auBo3LVyNlT/5gD8p3f/KRtz6zNsolG1c/YmZyn2ZKdbqDlHsaH47Eo1d3pKM2Nkh6ks1IEb2iVGp1UKUWZKfIPk2QR67YCLNZlTELny8lVSf4PyJEl4LcRLmgxbkPth5yMET6RQ76gunSlAsmvJ5xS5cZaImXnpcP92JwX8PlZmvbVz4ZzeH6OHTRuPEAycm2q8/YAQsZ5qh5BoAzv3DYgDAHmMbJGXRsuWgFPr9R5liVduKpA41JT5h967j/++HYif+DuOIHf0N828Lyv5U+7n7Jh6iRhWgfew0dllQ/5h/vRYQLmaqCkt0CQBmQ4yPXUh+u5JQ8YQizLRYdR87gTT2N01IONQKEqAOMhCDAUq2LeV/c9vHE9G0ARSV5JUDeAd4d0y0T7l9v/LYCSocW/YTFFvh15sLniiHm2KVwRNkHUew4XBvHUlvHzrO1u5iRMvqgdaKNRKkO1nb2iPdQ5agZjpwBPcJlWIXtJXGRBQ7zhQb5mPnddYfkiQraXHEjjyTHPn+SmqKHS6K3a4CTew0dllQ894fiUO3/zCLeOal8bGjKG9OSOwq8PdKCnEyUgUjVAOhb/vCNkeY/FgeO6qIiPnTgOiJOa2ztniJ06hPfD8xil2V8tjReTIqQTH7zAVPROWxcxx5HQLTtKzYRZ+vCtS3LCznWLWhqhUbNVzLdqTv1rZFIhcfPOGTMaViR1S6JKbY8OEmRvrgieCzqqRYUsVuOPjY7UrQxE5jlwV9KC1+qxVvbev21svbRVAfu8Kbb6HtzjvhlMtB3yHVJiJ97EL86qr9SJTVwPDJpz+g3UVNKGG1YoFAETNjEhSLSFt9QnL2TmyKlRUv1WcGU5FzDkjm0xdm2vaDTcLSnQgEhJpiRTiQrwUtM5YVvoeo7zgM9LtJ4pNXDbDzz2WSlRRziZ34HYk+duF+jcG68JfEYmhUbEit2ATPpTikDWqwQ55JEcG+yuMMh6jYXQnax05jl4X4UNvc2YcZ4xokoqEC9bHbfNVV6H76aWQnTUKTFzkVqtht2wbHsqQ6tQA/+VoVKCFJIUZPMqGn2rVio9QcpcnS+58n16bgkZCMIi1L1KSUViEQWydVL8WyU0mDJ1Q1WmPHSNrQ71DlYyeOIyyPnWocKr87dRCLoyTocaDHHyzFLjDFmn61jKjxUsXOMIJrwLsQOJy5VmVuZT9pZfAEzV1nq79bimoET6TNLxfmdxqMRb2/FKQxBJHQuzK0Yqexy0J8+LBJOu5tFODz2JU2bAAA/z+A0BqysG2Ut20PGY/67b/aPnbcJOMQs8pAmmJFssB95s06uWxAHpg/2EAHT1Sq2EWnO5H7yPQjeIJTx0j7rMLHjsJ2HC7PX6EcVPSQCK2juK5OoCjyCYrF7zjZNesuDD6xo7ViMyEpZ7j2tu3vU+slzJZLigm57hSnH3adAf6lgH4OTVBcBcVO7Dqe2IW8LDj8/6j9gOq/NGpEQxM7jV0WIpFhb9DsoRT1AGXEzuntRXmrm5vOIoTN7g0hdgiPjOXMOpyqVmXFzuIf1tWYMFSIygvHPfgFIm0ahh8Q4Ct2NDt+gvGmddau1CdIVOzivqtMSHLctHns6EQZ52MnmmKBwMQtnje9H/h17kpT+B647zjhJd/RF0SJ5yKqHlQTqlqx0YpdsE9tzvTXielOwq550E+4slXkFDt1TjsKdm/151ea9gUm7GUizsdOvJ21KXZwoYmdxi4L8aEUKHbRZgYgKCkGAPaOHQCAcishdmGKHYBySGQsfXAOZLJi0Uw3UFGxUWqOuqSY+8kwArLCIpPVCYrDjx2mXiVFpcETUSlegEDxEolcouAJcig6vpps4GOnNNc5sgrEzKESEVUodg4ZX1ZIUMxN/AlfDHb0Bb6oKSqS9QtcgmJv1ou65rYdqJysxJ0lmmIdXhlW9ca+J2XwBOdjFzQIU+yCSNTK7+200dhhLxPsXgzbezgmKN6VoH3sNHZZiA929gbtOylHEAczn4eRz8MhpWas7UHpMCdCsQsrK0bfai1hAqkmxGLxwdt3dY8TNemLzv30+AYMj9hZvrKUERLjqvqkSKsQVDoRycQu+ntjile/89gpTLHr2npx7DWP46x3TZfScogqEAugUAW1qNRV1k5MFF2Jjx1LaOwNblAQRMUaJOWMe26qBMll2/ETCDPFzhaCJ2wnPi1R1CUJC5gI97GL7zMOaZU07nxtul7xAgF5O4PlvXQMRjJqDa3YaezCEN9WJcUuYtYpWTbWjZ/OtaA1YVWKndnc7LYLSXnCRZ0J5KuaEPPYse6rr9jJb/j+cgQh4BU7RuxIvyH7UYSpHh29JaxvlyOWVebHJJBMsZSwKtpnCKmo5HgMKlPsv1dsxYaOPvzsn8sjS4oBJIDB68f3O1OYYh0E92PG5INYVAQ9DjT332AJOYEp1hQCcdTtLdv2zaO+YiekOxGDJ5SmWCf8d5w+Ktbh/lcCKc1OmuCJkEh91fNJNUSt2g0eNLHT2GUhBU/4Pnbq7RRX/+8b+OKsL+GFCfv566ztgSlWFRWbnzbNbdfRruyTV+yC9dV+HIoqQxKfwkoQpdiJPud0cjANA3lPhWIExBQIRdx4w4Inzvztszjh50+ivYdPjCu2TpotRQ6eiFawMhlesWAcI71iF6xXpQzhCZdY+ioISmFrKeGMUvHEdCeVKHZi34MBGjxhKiKsRVhOQLx8xc4Ro4DjTbFhAVGAQOZoebGYm68/VyzKJUJ5rBBTrBNxXmH96gCKwYMmdhq7LKTC8b5DucP9V2HNdjfn3ab6oMZsmRA7u89V7Ix83l+XHTfO7TckFUpoHrsBVOwAYnqu8nGiJjU6O1FTH+C6XTECxCa5jJAYF4jxsQuxo7/T2oNi2cbWHfx3kHbCY4jKY6fqIiOYonyfOyf+e6Z903slryR2POEQyUJginXbMUu3chwObcd/D7wfZeTwlRisqZ5WniBW/VBCbdm2/93W5ALFjvddFKNk5b6ifseFEMUuvvJE5VdNHGJs5QluX/6F0B+X6ry1Yjek0MROY5eFFDwh+NhF+sewh6wR/ITsHTtgF4uu35pH7DItLf72zLixbruQwAqa62kggydkYuco1/cXomqkOiYgK3aGERSbL/qmWMNXt5KVFAuZHNl3GzFWoPKo2DhCmDFFYkfun5hDco7sXB472W9JJJgysbP8bQBV7BQ+duArT9DvIcxUlxSDrdjlBMUu7PBlyyFRsUHwhJQfkLxAqPqKyobCVZ4gDeNrxVYO6QUmzscuQUCX8rwVpnsdGTt40MROY5eF+JBjD9REpj7mjycoMFZrK1Aq+bYySuxSKXYhb8fVgGgSSZKQuRLQ7lTmPdqQbjYMw58QgiSxRhBAyRTGiAGHTSKh322FxE4uKUaOpehDInaElKXxd+L7UCh2oPePIxHdICrWhR9QAPV3xYh3hnwPjiPep+lvoGqr0WEokeAJ+h2EmQfLdmC+rs0GwRPib5S+jMX5lUnPG2p+TVBSLC5gIQnS+thxKnFICibVc5Ktoi8dOknx4EETO41dFuLzU8ztFfXMYw9fqtgBrjmWmWEBgdiNdYldWLmxsEmg2qqGJZgpueNWkd1F+ZvxSgavhLiKnTsh+IqdERRvdxR9iAhPGeERu4hgDtVyGCJrxSray4pdvL+Xqm8KlSmWnp+rLInEjr/XmelbHRUbECDDQPA9ONHkPQkGqzZ8yTfF8opdGLGh1TF8xc6Wo7lLEcRNXCduLlrBMfio2OgExf1Ld8Ivx75McAQOys8qBD6Zwb2pFbvBgyZ2GrssZMWO9zuKVOwUpljADaDwiZ1pwmxq8rdlPVNsqGIXYvaoNkTOE+a71V9E+eFwhECYMA2AKHbuBtMIUp75puME34+IIP9W+HiA5NdfHEKsj52k2AX3Txr1hO8jOoWE4/CKEBDkB6RKHDuGSsxkxzYNAyYxxfb3BWSwpnpGKnKmCS7HcsiYezli5yUoVphi6UuSqqekAT6cWTbMFFsFZT11guIwxS7me/d9Mg0+4lpjcKCJncYui7h0J1EIM8WWt7f6/nVGbS3M2lr3c11dUIYsxMeOTwYarK9+8ITgFxajOlQKXs0RVSB+gqSbTcPwlSymoprUx44pqlGm2JTmrLQTXth+Yv44EWHBE0mOqfpuDENtihUJl6gCSaZYM9zHzrYdzmeKGWMdJ5q8J8FgTfa+YpcVTLEh91BvMSB2LN2JLaQ7sQUTt+p3Sm/DsIToAO8TGlpSzP9f+TUTTzf+ZYI+k9TqY5RvoWkG11srdoMHTew0dlmEpR9IEiXKJkpG7LITJwIArNZAsTNramDUucTObGyAUVMDIFyx40p9DWjwhLBMSUAVTWNJ89jRJMmAS1ZEHztTaYqNInYhil0IsZOIXsKLLjZzIrYB0T52ccdUnS4lwWFtHQQqEKvNGphiA2WF7aeeqN2VhgFfOqW+d2Hji8Ng+Ng5juOff9Y0uSS5YZe8QIJ2sqS+LzdcR0h3ougr6vrwKl28YlcVHzvxBSbFPRem0ql97Nh9Fdyf2sdu8KCJncYuC6nyhJDuJIrk+IEWnim2Zu+9AXiKXcElbkZdLcwal9hlGpt89Y762PUsWYK+5cul8cQpP/2BqN4MlCmW98kRFTvyWVh2JwOvTJZP7ECc9t3GUZNSePBEyHggTnihXQv9hSt2qhk4Kio2TUF2Bmq2DhuHQxS7phq32JAYFcvGEeZjx9a5il2wPkqVTYLBEOzovcCc+ePMg0yxy2WMwExty6bnOFNs2G8aSJ+gOPCxU25OBPEeSqXYCfkv/c/K/dz/BoBZe4zBMXuORS6rq04MFnRJMY1dFqGm2BA/LAoxeKJm773R/X//B2v7Nti9LnEza2ph+opdIwyP2DHFrmfJEqw5+3PINDdj3+eeDfWx4whSsQiT5MarBFEpDwbKty/OF00cU0YInuDSbCSY4FQO6HwR87jxJbsO4kQZN+GJJIwuxpti5XWhip0wJhaB2VibxfbuolRSjPFLlWJnO47/m3B97NQBH5WlO0m9S2pQ9ZYlczYNwEIEsStZfnv2nVkCsXOcBMETEfccJXC05FvcS0m/TLEhqY7CICrQrCwYdx0UPDRQeA388QtHVzxejcqgFTuNXRaceQk0j52nxkU88wJi5+6c32tPAIJiV1sLoyYwxZo+sXNNtVt+cS0AwOrogGNZoVGx7GPPkiVYfsSR2P6731VwtgFEM2V/U1aoEJW3Tt7OT1Wu3xjvl2MS3y6GyOAJhdknynwkdpWUpEim2BgTlUzsAh+kODN4Gh87qVg9U+xqPcXOrzwRmMzYMVSBLtQZnhJs3scuevxJz6naoLnm2H3FzjcuKjafMbn6vlRMc4DYdCdRwTS8j10Cxa4KNZ3FfWMTFEu/YzYWeVx8u+B+0Rh8aGKnscuCEYNazzk6CJ5wt0eRHEaObJgwm5uRnTDB7bO1NfCxqw0Uu0xjk0/y7EIBva+9ht6XXvL7s3t6IqJi3c8bLr8CKJex5ee/qOh8/WNFmGOqpdhFKXTidjESk5KdYF1AKJJELasc0KMmWckUm5BwRKZxUXQhBk8YRrAuNkJRsY6mhuHacsph4ORfn3eJXUFMd8KVFJNJuOVP1AYXndzvPHap90iPPs+sahpuVKz7WV2zl4H52OVIbVnbdiTiXkoRFdvfWrGBUt0PxS6tKVb0x/Xax5UUY5tNQzO7oYAmdhq7LNjDh6UzYA/UJGkFmLpnZTKo3W8/ZEaNcpe7ukhUbA3qjjgCZn09Gt59DMxaN3gCloWOv/2N608idpT4eA9Xu6sr/UkqIL6lh0W+9QdxeeFERZKrPAGZrFATIGuaNkFx1CQrJeWtULET02GIUCl2fjmvFP5OdP94H7uA6Nbn3ZcY0VTtR8XCUZwTMdkKQSz9VewGI3iis68EAGiqzfnnSc2rUchlDU6xk69rNLEN80sDwlOchAX+DES6k3hTrLo99+Kg6CMgdhUMUqPf0D52Grss2EPXTUBakkyxSRS75k99CtM+fDBK69e7+3Z2+gQs09iI+iOOwL7PL4aRycAuBoXnSxs3cf3Z3d3ofOZZAM3usiLLe7WInexnE3yu1jwrdhOXFoRuNlSKHXXa9/sIP75qcuRNscJ4UyoZYfvxREfuQ0XCMjFmwbBjAWoSDIimssCHq8FT7MScjdFjENKd+KZYJ/Z84zAYptiOXpfYNdfl/HWi+hsGqthZtnzfRBE3sX9xc6hiF+Nj5x7L4aJ7k0IyxcZEqoadb1yyYupjpzH40Iqdxi4L9pCq8zLLMwfzIHIyfF//QdzQCLO+3k9EbHV1weroBACYTa6KZ2Tc/o1czp9Ryq3buf56X3oJnf9+ShobQEwwpVKq8wuDqGYNiilW6NcR2tJl0+CjRdk6EEIRN1Zl8ESEyiD52CUkHJJJl7eBSjCVil0yU6zKB880DHUeOzHfmrdzXZ65HfDKixmhYFHzrEGjkxHtt8j3EaZChe5SNaiIXdKkufmMiQxRVMXrSgmZOlEvQrfz1SaCbWIyadX+lV631IpdiCLNlayL8rHTDGNIoC+7xi4LNtnUeMSukFCxc9NH8OQiwypMlMsob9nirvPMswBwz4vr8Pun3/IjY61tPLErvvMOLK4gfLi61F+Ik3d/IxtViPI9A+ToUdpeqdgZKsUufKzxwRPh4wHSmGIj1E9Fe1Fdozn7YvPYKdaF+tjRzw6Ij533EsPcDsCUOLctJZcZ3zwbnJer2AXJjOlxooI/wk5tMKJiO3vLAIBRdYGBilbaiEJOCJ4QyZX4Avbi2jb85MFl6C64x0xsiqWBFDElxYDKfRPFe0x052vtLmL+/cvwxqZO75jC/c3cQriXJMVx2AuDVuyGBJrYaeyy8IMnmI+dUCs27OFJSYOvZNTXA54yV1y/DgBgjgrKiV169yv48YOvY93oKQDcmrIU5U2bufJkYQW3q4EoYjdQprFoHztHmhxEssLVivV9jSIUO1XwRMQ1lYMnQrvm+wxRNOg4KcTgCdMwEpMMdVSs2seOb+r4JKJOJHaMsBFzoz9WomqxayemnUl674RtGzpTrKxQqr4vLo+dw/sfusET/D31q8dX4bdPvYWFK7bKqrBwjzELAcCTvDgfO3bsSiCZVoV+vv23V/GH/3sLH/ifp5THYe3jElPTBMUagw9N7DR2WbCHnB8Vm7BWLDWhsInBMAxkvJJhzN8uM6pZ2re9cQwABAEWOXeyKW3eBMtQK3aOAzj0Ld4jkJVCzj4ffK6WgiKbKMOXXVMsmwjcdSJZMYQ0G0B0EmF18ET4eCoNnogKwlDmnVOcV5QZNOpYADNbq4gdr076wRM50ceOHxdNuKuKlDUMElGKeJ/CuG2DEjzhEbtRtdQU6/6P85GjeezEBMUO+BcI2wnSpPSVLOn7jFTsOB+76JJiYWNNAomoCT+iF9e2cf2LhwmiYsP7dNe5/zWvGxpoYqexy4JN3kyxE2vFhpmWygrFDoDvZ1fasBEAkCGKHYOVr+WWWZoUUbET/cGsjo7gOPX1EWcVD7G0D51cq+ZjFxJNp1qmwRNMSREL23NRsWy/KMVO6WMXPomLfYWZw6L6BDc69eQrusMZBq8IRR9LXmeE+diRtm5iXfdzYIr1r6I7Lub7SXb0TbFO8DKQMYTKEwn9vsK2iet7imV0eWbMakGl2KnSnaiufz5rBjnvHPke4l4gnKA/GklM21PQ4Analip5FNVQ1mXFjl9miavD2qteesWRbN1R8H8/WrEbGuioWI1dFn7whGCeCoIn1A/PokKxAwLTq9PT4y2PggirRiZ2pfXrUdq8GdYeByj7dQBY1HRrWegPoupFVss0pkqZQSEpdkw5ClHsaCmrcmsbtv/v31AadUjo8ZVRsRHnKSsToV3zfUpKH+1TpbAJih2Sp95QfTUqxc4w+GNTNSjMFOsnKFYQO5uYyk0SFWuLPnYVmWKDz47j4AP/8xQKJRtPX3aCkrBWAkbsRimIHV/yS95XVuz48VLFjqaKUSd65peLITdZEh+7SiFFxAvLTHEMjql+VnCEmPSx+K1WfPo3izBxlJvaSac7GRpoxU5jl4VkihVqxYaBPng5n6RGXqFjwRO0v3K+hmvDFDunpyfUx852gDIJtrCLRZRbW7Hlf/4HxXXrIseqgkggoupZVoq4yhOi8uGb+jz6JkbFUlNs+/0PYMtPf4re5StCj68OnggfT9x4wyBWL4mLXFRF+5qKwIWoY1GofOzyGZM7V6oMNdSE+NiZ4YodENwzLiEiJlrhPg0fe/w5FS0ba7b3YFNnH7oL/Xt5oWB57DhixyJdBeVYRC5jcKbyqOAJm7ygOI6j8GXl+w5LRKy6d8X9K1fsRGU6ell2WZAVO4r/ecz9TW7udKvvaMVuaKCJncYuC/ZwYlGxxYSKHTWVcKbYUWHELlhXVih2DOE+dg4smh6lXEb73X/B9htvQuutf1SfXASifH+qZ4rlIaU74S1YQfsQxY4SCqvTNUtbfUWEIdYUK45XNFEl9rFz/2dJBGmwTe4j0hQbc0wlsYM6mTMdCFWG6jwfO5bXzs9jp1ANWbeUeJsGWS9QoahrFupjRz9XgbiooEx3orjmoT52xEzN37cOR8IcotLZTvzLQzEkrYkq8IcdMWqsSRAV7KNuLyp80W3auvmUTDqP3dBAEzuNXRbsgRTqYxfyzKPOzfShJip2LI8dl04kRxS7bBaZ0S1BX4askACAXS6jvL2V67u8ZbPbjvjeJYWk2NFjJTRBxkF+0xeX+QnRIcQBUETFEhNguVD0xhqu6qhMsVERkBIRTanYqfKiqXoQFQwzVfCEuj+RBItmQEo+pAhw1o9iDCpTLJ+gWP4eQ8cecl+F+ehVk9j56U5qA8+joDZu9DFpHjvxZcFWKHZU0RJ/S1E+dhShih0X5BR/fV7b0IFv3P0K1rf3kjGE//ZVkF54FIod7aKth3/Z0qbYoYH2sdPYZcFXnpB97MImKlVULKBQ7JoZsSP7EmJnNjQg09AQ9GUE0a7cxLy9FWWTT49itbe7Y+zrRVpEErtBM8XStsEEwkyxqlqxvlJUdM08aRMUi359UeNLqtix/VwTqx1LTsSgEANGilqx8nbTgOSLJhI7RiDyGRP5jFg+L+gHEIgdCVahpliDmGLBnW+6sQPhfo/VKm0HhARPKIhsmI8dI4Hiy4LjQKoVy1qoFbvgs207yshtICoqNvqlQcSHfvk0AGBjRy8WXPAuAPJ9HTYGf5wJfOzob10mdprZDQW0Yqexy8JX7DwfO5YWgj2owuaWsKhYqtgZuRyMGpfE0Qey5aU3WTTpIJx57Dy8ZAf7hCl2TrEoJTRmxM7u7Ys5S8X4Jd+f6k+oYjcySaaTFI2Kdf/LAQEBobD6PGIXEeGQvqQYv5yc2Ln/2XDjfOyk4AkuQXHMsRTbDcOQrpXt8BM/I3bZjIFc1n3ks3uA3ZusD/r9+/VjCVHk0s5AzusWOvaQTQ7XJvraVYpOZfAEO0604pjLBoqo6BPnBk/wbwv0pVD8LdHnQFjghNeN8v6LStcThbWtPco+gCTmf3HZkdbTz2JUrVbshgaa2GnssmATJTNPAa4ZJM5BOC4qFgDM5maSUDfYt5R1yd78d30e2/ONuGB5oODxlSeCfSwvWIKi3NbutqtAsYsKZEiavy0OYi9RUbKOE0x6jPhkhCCDDEmMa/um2PDJkSWOXbO9G4Wy5R0zfBKPuiZRYP0w1Uz0HRShCgpRRWiqoA6ekNVNajoFAkKSNQ3kMrzbQaDY8QoWHRdVnzImSTsjqFL9jopN0D4tLNvBDi99iirdSZqoWCmwAKrgCcc/bpRKHRY4EbU9TCWLQ0t9cN5hCYdFBLeUWrHj+wkfi/axGxpoYqexy4I91FgKCMCd8FRlcyho+R8uKrapSfmZPgTL2eAhCwD08c3lsaMTRrHEpzsBYHW0AwCcqih2pN8qEbs4oiT6orHDsmlAymNnkhqlhQSmWMvG0vUdeN/PnsRlf31VOmbVFDvvC1T62CkmTbF2JvWRiy0pptisKikW5suVy5jIZXj1SawQwJlcyeQeKJPBsRyEBz+ISJKg2OF8yML7SoMdfYEzP5+gWPaxU31feWKKVSl2JSndSaBoyT52RLEL8a9jUBO7yhRNSmiT+tgx95QkwRZRritasRsaaGKnscuCPYjyGdOfxAqW5T+8wvyCytxbOlHsKLEjOezo804kdhQWl+6EkMdSSSpBZrW7QRN2X3piF5XLqloTapjTtWq7Q5SOsKhYg5QUs4ruZB2l2JVtxzdBsf/8uUWfaPKSYp5iR5L5+kdQ9CGWFEtVeSIkL56qpBgFq4GcU/nYsXEJqpRpgEtErCJ8oi9flMoWtinMJ7FayjELnKjLZZDPBr8vVd4+1RipYicGNYjBE9RXVG2KDRAWIMGgciWoVNFsqcv7n6NyWFIwYhdaK1Zx3Tr75MTSWrEbGmhip7HLwp+siImqZAVmrDDeEGaKpSodTU7MKXaZcGIXmseuWEJ5yxZ3wfPRY0mQnd54UyyNOgVkxW5A8tiJtVgj3vxp2gw24apqxTJYXvCEavJn+5Ut2z9PNkdGKXZyWoeEip2gZMXmsVNU1GCr4k2x8jrTI7yqsmIMJepjR+5zOkZTUA0NkNq85LxM6mMnSHaVpDsJI4bVug9VgRMAOd+YY/LBE4JiB15Zc03WQV9yip8Uil3KdD0i6FibOVMs3y6U2HkkWGrvv/TK49q6oyD1oxW7oYEmdhq7LCwyKdcQ3yNfsQuZXMJLigVkLsMRu2DfUiY8EN0OyWNn9/b65sf8tGn8PjGKndXejlXvOx4bLrvMP58oxa5aplhx5onKY0d9k4wQxc40DN+M6fg+dvJY8yQ4gKme1O8pOGY08Uyex85T7DIKxS5B5QkAiU2xYXnsaB8qlIhix8Yp1kVm5JKRYRokYdt8upPgHOSEveFjj1/vhKzvD4KqE/zvThUFrPSxI8ETcvJePrKVvkDFRcVGBU8AakUvTN1Uob03MEE3kTQvYlqhsBRAgSlWrfCpvncVsdOK3dBAEzuNXRbsQZYx4UcLlixC7EL249/SqWLX6H/mUp+Qjsqm+5A1FYm9rJBasZbnR5dpaeFUQSCe2PW+9hrKW7ag8777seOf/3LHEOFjVy2lJCpYAhBIs0OiYr1VctLdIBWKVXSJnYp85bOBIsW+JnWKBmE80viTKnbsHlIpQHJ7ufKEkSJ4Ql4XFkVM4UfFmoZvii0LplhTQaQD+iamO/HWO8nJRhhppeR3IBQ7VnVCVOxU9XnDfOxYNhlRZXMcCMQu6M81U/N90cU4xU6VpJi7JjGXp607SDsSfGOBFYIpt/SeK5AxUbO1agyq731rl1bshguGlNhdffXVmDVrFpqamjBhwgR87GMfw/Lly7k2xx9/vO9fw/6+/OUv+9tfeeUVnHHGGZg2bRrq6upwwAEH4H/+539ij73HHntI/V5zzTVVP0eN4Qs6WeU5xc7dHlp5IjQqlip2zf5nzhTrEbu8xWdoB8CnO1E8w7MTJ/opVPxtvb3REXLlwO9l89VXw+7uHpw8dsLME5nHjiwzktP1wANce1qj1PGIXVhCWcCdGJlip/o+44I7Eit23v+s0hlfbi8GT9Co1thjqhQ7ZrqOqKtaJIodm9B9/zCmxCkSJ9Oobj7diRwty5bTIoxsp4n6jEKoKVaRoFh1xFzG9M9XzI3oILxWLPVLZOByC1ai2HF9Re6Otp7g+UKvJSNy7HdCCXdPMUj4XROj2Km+H7UpVjO7ocCQJiheuHAh5s6di1mzZqFcLuOKK67AySefjGXLlqGBJG694IILMH/+fH+5vr7e//zCCy9gwoQJuO222zBt2jQ888wzuPDCC5HJZHDRRRdFHn/+/Pm44IIL/OUmQQ3RGBnY0tmHHYUy9hrfGN+YIPAbMpDLBiYqak5RgT506bM+0xgcPzNKHRVbNN0HZt4qoc9LfQLTBGybS1BM4bDJe+IEefZxHDiFAozaWnlHAHYxeHMvb96M7ucWo+uVNYA53l9P39pj5pvEkBSxCCLlOAER9KtLLH8D2HuG34ZXkMJNl0xpsOxAsWPtoiZDyRSbWLELxuf2Q85LQRVUJmaV2hd1LH5/93+kYucTuyCPHeC+oLAxKk3fJNcbVSYDxc4RyEaEYhfmYxcSvFCt+9DPYVcr+tix40S/1HAlxQSy5aY0CZZdBdO712xHuufpYly6E3WC7eh7i6KVKHaqFxpmki9zxC54CaSqLIXKmhHtY6eJ3VBgSIndww8/zC3feuutmDBhAl544QUcd9xx/vr6+npMmjRJ2ccXvvAFbnnPPffEokWLcM8998QSu6amptB+RdjFoq8UAIDV3Z1oP42Bx9FXPQYAWPydOZjQpCY4KtBUFSrFLtzHjih2pI2Ry8Goq4PT28upd7SXkkfeaohiZ9bXw+7qgiOoccH+7sMxN3Gin5iYO4/eXpghxI75o/njbWtDcfNmYHJA7BzFg7+/iFLo3GPSzzRBsXuuZkkuTeQrSBGmS1+xIz52bCzRPnbCeBMHT/DEiO6lupRS4mXQVCPJjkUR5P2LCp5gk3mQ7gRwCR/7DYimWIOYvkVFNUhQLJCNSGIXv34gTLHtiuTEgDrYReljlwmIt6iyiUSP5g+0nehI8NjgibI8GN78Gbk7VwFCpSLnFIpdX8ki7djLkPi7gLQ+2scuepwaA4Nh5WPX4dW9HDNmDLf+9ttvx7hx43DwwQfj8ssvR09Pj2p3rh+xDxWuueYajB07Focffjh+9rOfoVyWw7UZtv/mZqw4apb/9+YHP5jgjDQGE6u2dKVqzx68pgESLUiDJ9T7hZligSAyNkMCKTjFzvvJ5QViBwB2TZ3yeA5TZSZMhJGXyZ8T4WdHX0YAwNrRyfnyueMjn/vhtV7etg19K1a4xw150/fHJZRH8k193jqJ2JkKxU4xVKrYlW1+corKAyZ2lVSx84Mnkuaxk9KdJM9jF7U5WrFzJ+xcxkCO2IJLZaLYiaZYoSYsn+4kMNHyKlt6xW6gfezWtblR41Nb+N9W0ryD+Wx4STGR6NH72Hac/pli46JiY64PT+zk68qed8+vacXN/14N23Y4U2zgvsD3G9SKlcel9rEbXsxu/fr1OPvsszF27FjU1dVh5syZWLJkib/dcRx8//vfx+TJk1FXV4cTTzwRK1eujO33hhtuwB577IHa2lrMnj0bixcvHsjTiMWwqRVr2zYuueQSHHvssTj44IP99WeeeSamT5+OKVOm4NVXX8Vll12G5cuX45577lH288wzz+Cuu+7Cgw8+GHm8iy++GEcccQTGjBmDZ555Bpdffjk2btyIa6+9Vtl+7JcuxJjPn+cv169fD+y/f/oT1RgwiJNTHKgptiYbKHbBW7f64VkKiYoFgMzYsShv2YLsuLH+Ou5NnRE7O3iJMD23gzjFLjtpIkobN8rn4QVXlNvaYORynElYInbbtnHRtwA/IfenpNjaCy9EYfkK7P3440CON4tHKXg2IQjsK5QVO14pUvUJEKdw2yERfOpjUkhENKEpUFS84ny2lJUnWFBIzLWPIoqZTASxIwmKTdMlkpbtcKl9VKZYQ6GM8vntxJJi6cYu7hPmb9cfrNnuWlamj63n1vt+guR7Vh2TrzzB3xSS6uYE50NVaBXigyeiFbu460ODJ1Tkmym377T24qr/fQN7jG3gVE3/tyX9LuSXXvaRHpNhOAVPtLW14dhjj8UJJ5yAhx56COPHj8fKlSsxevRov81//dd/4Ze//CX++Mc/YsaMGfje976HU045BcuWLUNtiFXkrrvuwqWXXoqbbroJs2fPxnXXXYdTTjkFy5cvx4QJEwbr9DgMG2I3d+5cLF26FE8//TS3/sILL/Q/z5w5E5MnT8acOXOwevVq7LXXXlzbpUuX4qMf/Sh+8IMf4OSTT4483qWXXup/PuSQQ5DP5/GlL30JV199NWoUE6yZzwP5INEjLd6uMXSgKoeYIyzpvqZhkGhKm7x1q/eLUuwmfvvb6FnyPOoOP9xfxxM7d4x5i/izePeSncsBiuc9Mz3mJk5EoVal2PWitHkL3vroR+EUixj/zW9g9BlnwDAMidgV162HZUyT+gg7nzQovvkWYFkobVgPZ/f9uG0iUQpTH0zDgGNZyJT54JIMIRowZBLFwKc74RU70RTb88ILsHt60fje91QcPCEnKKbHkNvLlSfgM6XYBMUR/eXEjgnYiwgbYy7DiF3gdiApicGwuN9YxhRMsSGKmwjWRXNdDt/50AF46D8b8cTyraEqajUUO8dx8NY2l9jtMY5/XqtyB4b62HmXVgxoUNVcps+OKMWukpJicYE5FK3dwe9HZTbNCcE2bT1Fbl1YAJkqeIJ9Fq8HMDiKXXexzFUYyWdN1GRlf+Wf/vSnmDZtGm655RZ/3YwZgR+v4zi47rrr8N3vfhcf/ehHAQB/+tOfMHHiRPz973/HZz/7WeXxr732WlxwwQX4/Oc/DwC46aab8OCDD+IPf/gDvv3tb1flHNNiWJhiL7roIjzwwAN44oknsNtuu0W2nT17NgBg1apV3Pply5Zhzpw5uPDCC/Hd73439Rhmz56NcrmMt99+O/W+GkMHarJI+xBhz6EMSVBcIIpdqI8dVbiEh1nD7KMxfu5cGJngwUIfjgXPrpq3gweRVe+qW7YZ9p7FgifUpli7rw/bb74ZVns77J4ebJ7/I3Q+4CrWTpE3j5TWrYMdEqQBVK6U2MWibxK2u7pio07DTLUGXJUxI6SDMahS5O8jj4MpEZZNEhSrJiPbxjtf+jLe+epXYXV19Tt4wvex4yZfuQ+pogbSBE/I25mBOsrHzk934t3j1O0AfvCE0C9RSOn9bhhirdhgH8cBCmUrpIA9U4pMfPqoaZg+tsHfR3V+1cin2N5Twg6vGsLuY3jFThXsEnY/hT1XpBJjCM7HcuRasal87JTELxmJBoD2GFOsGEU9oamWM8Wy6yIeRZWg2LdwKC7gYOSxO+n65zHzyn/5f79+YrWy3X333YejjjoKn/rUpzBhwgQcfvjh+O1vf+tvf+utt7Bp0yaceOKJ/rrm5mbMnj0bixYtUvZZLBbxwgsvcPuYpokTTzwxdJ/BwJAqdo7j4Gtf+xruvfdePPnkkxx7DsPLL78MAJg8ebK/7rXXXsP73/9+nHvuufjJT35S0VhefvllmKY5ZNKpRmWgb9FxpZVE0MoT+SwNnpDNDRT0oZxk/ud8a7xdafBEsXEUTABWJgMo3DxZGpTshAlSuhMAKL71FtrvvhsAkJ8+HcU1a1DasMHdlyl2XuRt6Z13YLWEX6fet97Cm9//Mib94AeoP+qo+JNjY/T8YwGX2ImXhScA8oQXmGINOIUCMoLEx5WyYj52inHkSPCEaDbizIalMuyuLn/sDvjrmtTXMPCxY2lECHlUtJdMnqZcpzUMUVGxUfc+y0/GAkuCsmKBuVDOY0fHZXPrGRzwkZ99JQsn/OxJ7Da6Hnd/+Rh+7F4XvqldGbwQTbLS4m3PDDu5udZPuMsQlHEL1oXnsVNfW5GcucETgUos+5UGiI2KVZhi48z8FK0hwRNsTHnBuuHAQW/JktqFBRmpyhCqg3tiBloFPPK1WZgyZaq/HJaD780338SNN96ISy+9FFdccQWef/55XHzxxcjn8zj33HOxadMmAMDEiRO5/SZOnOhvE7Ft2zZYlqXc54033ujPafULQ0rs5s6diwULFuAf//gHmpqa/IvX3NyMuro6rF69GgsWLMAHP/hBjB07Fq+++irmzZuH4447DocccggA1/z6/ve/H6eccgouvfRSv49MJoPx493Iv8WLF+Occ87BY489hqlTp2LRokV47rnncMIJJ6CpqQmLFi3CvHnzcPbZZ3P2do3hDxqhmpbYBT524EothT3UGEohUbFhoE0KzCeLKFL2XvvAfPxfQE1t0IDuD8DI55FpaYFRk5e2t/75NjilEupnzULNPvuguGaNX6mCmWKz48ahvGULrI4OyceOYstNN6OwdhU2XPEd7P2vf8aeG4PV2Rl87upSkLfwSZuWFDMMl4xmHItr46bZEEyxivPIEx87UbHj0roQE7VdKMBxavzjWLZc5zMM/j2kUt0UXUi1YoliV0mtWHYtotwQ2P3K2lDFjvUo57ELiDTlIK4pNjCF09Pd2NmHDR192KbwtWJjN/mvMJSsVCOP3ZrtbpCd6F8HAI017tTXVaAmS7mPHAmeECFXoqA+dnJ/3AtenI9dTLqTOMWO97GjSqj7XzTFWjbQS9KdhAZPePurvquoqO2BREM+i6ba8FKNDLZt46ijjsJVV10FADj88MOxdOlS3HTTTTj33HMHepiDiiE1xd54443o6OjA8ccfj8mTJ/t/d911FwAgn8/j0Ucfxcknn4z9998f3/jGN3D66afj/vvv9/v461//iq1bt+K2227j+pg1a5bfpqenB8uXL0ep5P6Ia2pqcOedd+J973sfDjroIPzkJz/BvHnzcPPNNw/uBdDoN6hiRzOsJ0FQTokqdpZS4aGgD/Qkyg5ngrEdIJuFbQYKQt3Z52DvJx6HXa/223RguGZYw4BZIzvwFt98EwDQ+L7jfEWPmWBZupPsuHF+ezEqljsWM1EV5Ai3KFgdAbGzu7ojKzmoFDuaAFdliuVqlLL9FN93jkt3wk84nCJE0sA4fX1SEIEVU6Q9OC/3f1ZhilVNdKrgCVUVBBWUPnbeJajLhZvXfWLnHZvmbBTTtdB+2Zqy4O7AB08Eg/KrWSgG6iuDLJ2N/12q74tqKnZ7jJV/V2Ma3Bek7V1qAsSQi1DsRCJO72PblqNiuRe8GGJXjFPsYq4PTVBsK1RJ8UXAEqNiKUOl7VS/pYjnZYTr56Bj8uTJOPDAA7l1BxxwANauXQsAfuqzzZs3c202b94cmhZt3LhxyGQyqfYZDAy5KTYK06ZNw8KFCyPbXHnllbjyyisj2xx//PHcsY444gg8++yzicepMXxBJ524pJ0i2IPXIJUnqGIn9rd6axfueG4tNnUG6UWSKDucj53lYMo11yD/GoA2d11f2UFu6mTYzgrl/o5hIDfVNTWoTLFMlcu0tMBi5kWPuPjbSJRulGJnedO2mTI4yOpoD/pXKHb0oW+VytI23xQLI8THjhalZ4qdPOFyCYqJIzsgEIcSmfj6CnByAcl3j5EMUh67mMlXrjxhKM2CymNFOKdfPGcfPLJsM257do00wbIXESbSsECLMjXFKtKwMCbNVVcxSM1e8OfIXrJUEzyNQKf/eSIst+8PAsVOvpfHNnrEjipbij5oHjsRUiUKJ3hi2I7q+wqW2bUyDPV9oiopllTRLFu2X3HDHYus9ImKne2I6U7U36U6KjZcsRtOtWKPPfZYqbLVihUrMH36dABuIMWkSZPw2GOP4bDDDgMAdHZ24rnnnsNXvvIVZZ/5fB5HHnkkHnvsMXzsYx8D4CqDjz32WGwe3YHEMOLTGhrpQX1R0s4FVKXxExRbdugb6O+eegu/e/otPPBqkHIkiZM3R+zKNppP+xAyREFjD1RVVBkAZCdNxoRvfAMAlKZYhszo0TCZYieZYkmliQhix0hfadOmVOYwu5MqdtHBCJt/+Utum+PFVwKukqPysaMVD/woUsV5qNOdMBWFjIeYYp1Cn6S8JSUWTsR+ySpPJFfsVLcH6+29+4zH/I8ejPq8/K5Oo78BtSlWJHau75/7ucwRO6KMO/yYmTKoDp7wxssOw0yxQqSyOOb+IFDsZFPsuAb3d7Kd5F5TXf98JtwUK6q6LtEN7jnZFBt8ZtdKVFpZ2iV1uhN6b4WjqyC/OImfZVOs6GPHjuNI7aSxOPw2iuGUx27evHl49tlncdVVV2HVqlVYsGABbr75ZsydOxeAS0IvueQS/PjHP8Z9992H//znPzjnnHMwZcoUn7QBwJw5c/CrX/3KX7700kvx29/+Fn/84x/x+uuv4ytf+Qq6u7v9KNmhwLBJd6KhUQmov1taYsfIRsYk5ikueILvsLVbNk8mMsWSz4USXw0BCEr5hPWVmTwJdTPd3I5hFSYAl9ixqFmVj50/5gTEzunpQd+yZehauBCjzzgD2RjfU2qKtbrl4Al6LYtvvgW0HEC20YnfVezMSFOsp/Z4K6jqQR2nmR+Tn+6E87Ejikah4I9P6SsXgSgfO3Wwg+zLZib0sUviw6SaRxkxY+qJqnyeaJqjJtewdCe2w0/7/EuWw6k1oYod2T9MvUuLZRs6cdGCF31ip1LsfFMsLb2lUExzGTOURKnTnbifHUf20+TVTfdgtbkMX6M1a6JQtpUJjOPU4MvveRXbu4q46hMzhf1k8iy+YNiOg16VYkeq86hUcNo2ylVgOGDWrFm49957cfnll2P+/PmYMWMGrrvuOpx11ll+m//3//4furu7ceGFF6K9vR3vec978PDDD3M57FavXo1t27b5y5/5zGewdetWfP/738emTZtw2GGH4eGHH5YCKgYTmthpjGiUQ974kyDw6zKQ99KTuJOdu13sjqVOoEgWPEEVO8v1TSLP7d4YxY4eQpXuhCHTMto31dqejx37nx0bVGKJ87Ez6uvh9PTg7dM/6Y5r8xZM/uGVfpvWP/0JbXfcid1v+QNynh8JDZ6wdyjSnZDzLff0AC3qczQMl2iJwRNcVKwQPJEzTX8ipNF+zI9JqTJQxa6vIPnYJSUWUh47xTkxGIaCiCEwkcYHTyggTJwqhYT165timWJHUvso89ixigtCupPAx46/ptQ0aTtBrji3baDI0v/hUbGVM7vHXt+MN738dROaarDn+AhTbJc6LQhDLmuG+ltG+tg5cn/0WjFTbF5QzmpyGaCvrDTFRl2fQtnCHYvfAQB86X18flfaku0nJhMWTbGse3p/W3YQ9cv7RqrHBAwvxQ4ATjvtNJx22mmh2w3DwPz587na9CJUKdEuuuiiITW9iui3Kdbq6sKORx9FYbU6d4yGxkCCRpelJnbsbZQmKC7TBMV8f519fNJcIFmxctEUUrZ5p3NmAglT7OiEEG2KDaJmHd/Hzh2zUVsH0yt3VjbDHe3tbA41e+7Jret98QVuuePBB1F86y10Ez9Vq7Mj6ENhiuVylPX0ctuoE77hjVkKnqBRsX66EzkilJqYCmXL65+NIejPIj52TqGPRG0KERoxEPPY8V8h34lpGJKCYRhGiuAJ1cTJL6v8wWg5MICPABejVVVjDfYPxgyE+9ipzoVdFzYGvw5tiArVH2LX5SngnzhiKp745vFSqhMAGMtMsd0F5XVlyGWM0AAAKSoWwW/Ydhzp90wXGXGryQnEjiTYFhGl2HWQYAmRLPImbve/GLxh2UBvqazcBwheXNjzTqXYqV5Mhhmv22WQmtitu2QeWm+7HQBg9/Xh7dM/iXXzLsWbH/0YOv/5r6oPUEMjCrxil25fOuExpYdGCor9qRS7JBOQ2KRAyCMQ72NH14aaYg0DmVGjQn3sjHwemaYmFMwsttS7ZtWMbcn91NcjN2UKtyo3bXdu2e521RCrtS1YJ+axk4hd8NkSaj27yg87DQNOUZXHjkZSen2yclqEleQUplg/3Qm9V2hqB06xY+NNdjMxUuAHXUSYYl1zsmw6VZlie4sW7ntlAzdhJ1FEVKYv0Qyapz52AjGl42LkK1D8AtM3O1eOrJDvTKq64Nva+XGqqiKIn9Oi2/Mx262lDg01aqMUU+z6Srb/+wvzsUseFSsETwjdUZWLmWJrhJxrNF2PiCjFrj0kWIKNRdz2mVnTcMTuLcF6MSpWeLn1o8UVZld6ziKGm2I3HPHM6m3xjVIiNbHrWbIE9UcdCQDY8cijcOBgv8XPYdJ3rsC2m26q+gA1NKLAmyxSKna+fxSEBMWkR/IE6+yVFbskBEAym5Qs7hi9EROLO4bgMzXFGnVBYfNMczOMTCbUx87I52A2N+PN5qmwzAzGZh1M6AmImY+6euRI8m8AEluwu11iZrW1+utkH7twM5So2NkOOOVIne4kcLLzU7IwUyxRKHjFjvex4ybGUljwhJxoOAr+fhmVAsX3QTXH4LwCUkjN+n954R1cfMdLuOnfgSVEGTyhUABFBD527jKrzkGJncr3Tyx1ZvhqWzAenqyoSRodu6/YKaNiyT3SDye7noL7ewojdQBQn8+g1lPLWj2zpOqQuYwZWoNajooNzsdx5HQn9CdR8raJaiK7j1Rjiboi7eQFQOX7J34e25DHPV89Fice4PqBWY5I7Pj/7HcVRMXS71pex6BpXTzO+8PzOO6/nsD1j63Ehvbe+B0SIDWxs3fsQKa5GQDQ/fRTGHXyyTDr6tD4vvehuGZNVQaloZEUvPkn3b70bTRHomJVEV+O46h97FJGxQLpFTvalppiaUBDpqWF284qTjCCxxS75aPdOrEzx2RhKKYKp64euSk8sbMJaQMCxa68nRA7Liq2O9IUWxZNsQjSbhjwKk8oiJ04SSgVO/KZETuVz6SU7oSQfHe8SARR0YiqPEG4abCOlhQjB93m+X6FFXNnSKTYCabYrCK1jxytG5hi/XQpAikDeMWOBjLJVRcC4k7HMhA+diwqtD6C2BmG4Ztjt3mRsWF57MSqHAySYgf+JSIq5U85RLHLRLxYRCUopiXEVEqiOAZ2TtS/s1dVUkxU7FTpTrzPluJe0opdPJ69Yg7OOWY6/nfpJhz3X0/gc79/Dg+8uiE2iXUUUhO73KRJ6H35Zdg9Peh66mk0HHssAPfBbubD/X80NAYCnMN2SmZHVYQwxY49QPtKtpJ4pU1QDDBiFyz3eL4tYSSR7k9NsZkxQUBExiN5YaZYs6YGmeZRWD7aNaseOqUJhmryqKtDVjDFWsTM6jgOMcVSYpfMFOvYNqw+0cdOCJ5QVJ6gUbF+8IRH9SiZM03D9wcqej52yqSqxJ/I6etTVmBIku6FNQmUPnlbcA6GpKiZprq8FSvjFZXYWQVl8IRvinWX86p0J5GmWJvb3zeJO0LwRISPnagMqipPhH1Oi27PzN5YE+5LCgTm2GjFzghV7CRzs0Or1siBVVTdZNdKLFbPXEJUz5UoUzU1xYpKIl0UzfL0hYSmOxEJnK/YqX5LTN3zPlAfv+GUoHi4YkxDHl9875546Ovvxd/nHos9xzXge39fitlXPYor73sNyzZ0xnciIHVU7Ohzz8H6b/0/mJ4vTv3RRwMAep5fgpp99009AA2N/qDcD8WO+tjR3F6qh5YqcAJIX1IMCCJjGfq8N+UwYscrdoEpNjNWJnaGSOxKgY+d2TQKKxxXsTt0zwn421OKhMh1dchNFogdUeOcQgGwPIWREDuq6tnd3bDFyYVNCD29UsUIB8Gk59aKLaprxfrtvcnIM8VmSPBExnQVsLLtSKZYzseOKnaFPindCeB+b3GCgyNMlFGqCiVGwVDUwRNlRbLfZD52ClOsxY+RmmIZ10hSUoxdGz/wAWLwBB8VS0Ej0N3+2RbaMB2JDUM3M8UqcvpRjBWqT4jHzGUMLoG0CMnS6vAkJ6ryRDFEsctm+BeEzr4SPvHrZ3DKQRMjFU3qi6kinOKYReVUVOz8FzF/XIJix50Xf6/ms6ZPEodTguKRgIOnNmN8Uw1a6vO4ceFq3L3kHfz52TU4YvcW/OTjM7HvxKZE/aQmdmPOPBN1Mw9BadNGNL773TBYmZppu2H8JV9P252GRr/A57FLqdiRaD822ZUtPpqNEY4dIcROlftKaiP52KlNsUnMukaemmIpsWvxtovpTgJi19U4GhtsN1Hx4ftPVZpiUVOL/G5T3Vdt7+SoYsfUOkBQ7EgbOA7sPtGPjhG7bt9HjmvPFDtAncfONCTSQtOd+O0MN4KvgCB4gilLYaZYLt2JkHvNjPESivKxE8H5lxEVTRU8wdTh+Lx4wrJCIRFJFed2AGY+k/sJJn2m2HkHI4EPHBmlxDkkIlSMrOXVJPXntGDBE40RplgAGMNMsV5+SvGY7DrF1aBmRN12HJ/x2ML9xtYxMLIt+dgJJv3X1ndi1ZYu2LaD5vqgHqp4edp7A1Os7GNHPgtmeWpiLSgyDMiuBvK5iOtoLsnhlMduOKNk2Xhk2WbcveQdPL1yG2bu1oz5HzkIHzlsCrZ3FfGLfy3HV29/EY9e+r5E/VWUx65u5sF+wlTHslBYsQL1hx/u+95paAwWOIftlPvSh5Y/iQkPZPa5o1f2r2Pt4yC2KJRtbkLrKSUPnuBMsRE+dn66E++/kctjRe0YoBOY0rMdo0fJ2fgBwKmtRaalBVN++lM4xSI2fuc7cHp7YReLMPN5JbGz+/q4vHAAUO7cwS2z87W7u9WKnU8+4EbFRiUoZrn2FD52phH4rPGpcPjra9Es+yR4gvaVhFyIpq2wKE92bgDvUG4Y6uCJoO5q0FZ1e0im3Yg8duzUcn5qH4cQLsEUi0CxKwv7+8qpw4+pHOFjJ14nmuTYb0MueJLfVRi6E/jYAcA4ZooNVew8YhejOmVZHkWHEiK1Dx4DM5fKih2v/NLqKWFBXQAfPCFVxFAofb6PHblvLVqeUSBrIuHkD8+TQM4UqxW7WPzgH0tx3ysb4AD4+OFTcfkHDsB+kwJlrn5MFld86ADMvuqxxH2mJnabrroKtfvui5ZPfhKOZWHN585B70svwairw7Qbb0TD7KPTdqmhUTH4pKjpJgOLTDb0zVVl8ghT7CoLnrC4dXEJikNNsWMCYpeN8bEzavLorWkEAIwtu2qa6m3aqXGJY/OHT4Nj29j43e+6ilpHB8zx4zliZ/f0wO7rCyJiMxlkGhthdXSgtI0P4fcnvO4eSbGzncD7yDTUtWJdUywzAbqpXSwhGIC1Y9+lqEBQsiApdr5yxSt2cQiiaaODAYBgMnUnO3rvee0Vih1nPlaaYsXlcGLHzo1NvGU7CBRSlToL259WjaBjigpkkhMUB334bRTtK0G393uK87ETq0+EKXZxfmIZ0wAsOXgiKkFxkfnYCXnsRF9NWulBFdTFwPvYqUk17df/Hoh/p+peY7uycamSfQfmZ/d/TVYTuzRYuaULV37kIJx68CTJ55JhTH0ed1zwrsR9pnZt3PHPf6Fmv/0BAF1PPIHSunXY838fxJhzz8HW665L252GRr/Qv6hY979pGpwTscok1KmIiPXbxBxYnKQKJZt7MPcUy7Bt2XTj708+c6ZYRfCE5GPHgifyeaDOJW2m9/CQ40wB1ASKoGGaMEeNAhD42VFiB7iqne0FTmSamoIkyAKx8xWA7m4/6IFuo4qdXShIOfYybpHSYHnUKF/5yxEfO9MIIgupYucS9mB/u0yCJwp9ynxuSbiF6JsXpbCJpkz2UWmK9X3s6GQuD0iVF0+EWFLMDy4hwRNyHrsg0EPcn+axoyhGuEWwSV/0sat2VKzjOL5iF5XuBADGNkZHxbJAhnjFTn52OI58Dip1U5zI/VrHjNARxY5zJRAuT7SPHR0Dr5xSxU7lz+kTO8HHThUkpDLFal4XjwUXvAsfPWxqKKkD3JfXd+05NnGfqYmd1daG7Hi37mTXwn+j6dRTUDNjBlpOPx2FFQpnbA2NAUSU+ScO7KGZMYgpVlDs2IMwTLFLclxxs5jupLdkR5qeOMXOMHzylhkdQexKJTi2zaU7yXqJhrONDV5firHW8CXLMozYdaiJXXl7q0/6zOZRMBtdVbC4NUyxk33sHJCoWEBZeYKkVYPtJWNmPnZZQkoyJCqWVZ4A5AhOGhVr9xUkU6E7ruRqrLKkmFR5gv/vnpehNMWWbNkUq3p/EL9ClUJCfUkB3hTL+lelTRErT2QkYif6jREiHWKKDbhtNBFO4ruqQqEcRK/XxwVPCFGx4uVlBCVOdcoo/CtFoiRuL4codrlMQLQAPjo1iuTzPnbqwCXaHzslk6jFYkk4ehyq8gL8tRL98fJasUuFG55Yhbuff0daf/fz7+DGJyur6JWa2GXGjUVh1Wo4loWup59Gw7vfDQBwenuBTLT0raFRbZTs8IddHGgaCOocrJpMO0N87Gg/YRAf8IWyxe3TWyxHm3SFTaM+fBrqDj+ci0JnPnY05ZBTKMDxTI5GPg9zvBs4UTtjhrtO8dB1hFq0zG/W6mh3/3d1cduttlY/cCIzqhmmRxrL27Zz7Thip1Ds/OAJw81jZ4pmTBKl6cCA2dzspz2hplgjxBRrOXyUIqfY9fUF55vax47fLyrYIUjwa5B1ZF9bnoD5vHjygMSJU6UuUZcDAFwEeLgpNhilVFLMN4k7SrLijpsfA1sUfezCoogrVexokt2GfPR81FznBiN0eGZM8ZiBKTaZYmcJ44+KTi3ZasUuIyi/1CQfRfLbIyqUiIQTCO4T6ltM+aAq0hUILCRivkZahSTHuUZAIwYLnluLvSbI9Yz3mdiI259bU1GfqX3sWj7+CayfNw/Z8eMBAz6x6331VdR4E4aGxmChbMkOv0lAoyRpxKVdiWIXoy6oEhTzplgrktiJW6b8+McAAKu93V+XaeEVOwCwCQkzampgOy6BCXy9FMcSiZ2n2Nkhptjy9lZfNso0Nfmm4tL27UBL0I7N+XaP7GPnEoRASXCKRZhwYDp2kNLEMGCQC5FpaoJdDFHsWHk4wceOMx8RYmcXgqhYoxxdwktEVIJi8YsLIkLpOhq4E6xXmWKVip0RvQzIxCxPK0+QcfBjlU2xfroTTrEL9ilF5JQM87FTmfTc9ZURO2aGrc2ZHOFXIUN+8+JYAJ6gZEwj9DcqJu9lfclJmgOElRQTKzyE+eyJJJ+aYstC8IQqoCdIUBxcA0pMxReLmmzwMiCejBhxroMn0mFrVwETmuRSkWMbarBlR6GiPlMTu/Ffuwg1++yD0qaNGHXqqYFCYGYw9sILKhqEhkalqDSPHW3rmmLdz5b4APU+qqpOMMQpdpIptiQHTyQ1xVIY9fV+WpKsF0hhZLOucm5ZsHYQYpfPS35OKhZgZ3PccqaFKXauKsfKiTFYra0wmO9eY6NP7MrbtnHEzmEThcLHDg6v6DheqhbTtmF7k4RhAIaXtNgxDJijmuBslxU7qr6K6Te4smaCYseucenll4Hxe3tjli6PBN8HSeVjJ5lieZUEcDmxSrFjJi9LoaKo+gxbdvtgpJlX7IpcSTF+H4OYYsUUGbQcmCrpLttGQRVZ1r+7Pkyxk04jEboSpjoB5PqnkmJHSFd9LoMdBfUzQAwsADwFSyK3wecgQbEYPMETXpVPm9hXybK5sUXnsVMTbEnR9j6yNXmB2InPSPr84n3sNLGLw5TmWixZ04ppY/hMBUvWtGLiqJqQvaJRUbqTUaeeIq1r+fjHKhqAhkZ/UKzQx462pZGUtuhk77ULS1AMxEfGSsETipJiYooCfn/1ejOfx8TLL4dTKvmmWMBV55yeHtg7gqTBRi5HzDDeOhWxE02gMT52VlsrMo57bLO+Hkat+yAqbtsO7E36ZZUUurslxzDXMdwjH4Cfey/j2Ch76wzD8GcbxzBh1tfDZsROSHeSVUiRooJil/l0Jz7BcdLdT6JiF2Uu84kRXWcakcETnKlSQTTFr1AVwRlmii1bjk/MshkVQRQUO98Uy85PMMVGRPCGEQraqpqKXZx/HR2DP2zhkHlyTZrrc6HETpXnznZ4BRbgv0tGkORaseoKD07ICycg17COymPH7gX22/dfghTPH4f8LvOeyTi4L2n//NhqdB67VPjs0btj/v3LULIcvHsvN0DimVXbcfVDr+OL792zoj4rInbdixej9Q+3oPDmmwCAmr32wtjzv4D6o46qaBAaGpWCVwmSTwZ0EqVlncIeikyxy2dMjkwCstlJhOxjx+exsxwnUrGLcuIf87mzpXVmPg+rpydQ7HI5GKYpOcqrXqbZA/ofL69HbS6Dw0d5il1bG8pbtwbEzjAAx0F5e6uv0pn19TDr69wxl/lJkBI7dVQs6bbAiJ1LvpggZzDH7XzeVSAVwRPUtCmeF/3aKLFzCsWAeBBil+RuCsvz5Z4X34M/LDI8AyF57BL62CXJYyeqcjmivrD7UMpjZwTj9RMUi6ZYxTkyyMET/HGCsmTq86tUsWOpTuIiYgFZKQ3zsQOAlvoc1rWpC7SrXyTkWrF0iX2/simWv4+oST5MDW4XiF1krVjv+84IxK6gqEtK/Y3zROWl43P754meDp5Ihy8dtyfaeor43t+XEhN9Bl9+316Ye8LeMXurkZrYddx3HzZc8R00nXQixpztTio9L72INZ//AqZcdRWaP3xaRQPR0KgEqkiuJJAUO0P95soeiuyteFJzLda2CubIOFOsMCGLJcUcR3a05saaMkKQ+dnZXW6SYOYuEUQmepOrIt2Jbbv+Ol+/82UAwNN7ucSubcECtC1YgPye7htkdvIklDdshNXa6vvhmQ31MBvcqFgxQIISO2VULBu7l8cOgB8Z67vxs+Vczk2W7AdPEGJnyuqTe3whP6GlrhVL6+cmUuyEcluiUzmF0hQbmseOTaDkWIrhyCbU8InUV+xM2cdORRBZ34FiJ4zfCf/NSaRGUA2VlSdizM5JEFSdiA/kY9ed5oqjaCGVHlrqSJohr2Qdgyq4wnZUBCv4XPKIVI1UeYJX7NjY5PyawT40cAKIVuzE74H9L1kqYhcck5G1sFJ39FzzOngiFQzDwOUfOAAXv38frNrShdpcBnuMq49MfxKH1MRu202/wYRvfgNjzzvPXzfmnM9h+y23YtuNN2pipzGoKEX49USB87EjeexKAotizZhiN6GpRiJ2qRW7ki1N2EkSHScFI3bWDpfYMUVNLAZvKJ66luOgj6hZm2qbQeO1ip5Kn99tGsobNqLc1obshAluv/X1/mcpCTGbELq7faWNwXEI6QT8FC2sXqw/Xvbd5HOCYicmKJbtka76QO+VYHx2oeBHyZpwYDgOHMNIROxYn0ofO2F3VfCE68vmrljb2oOzfvcsvvjePUOCJxSKnbCs4LTB8U3Rx443gbPSWGxcjFCLkZTsEA7kJLzBWMVl9lIB7n/Y+VX6e+iqwBQrJt2dObUZZ87eHcfuNc5vS8t5mfRCIUSxs+V0J3xUrPu5NlSxC/pxxyaarYOljl6+6osVke6Efr9AQG7DiZ37WQqeoHV9hWNoH7vK0FCTxaHTWqrSV2piV3rnHTSdcIK0vun9J2Drf/93VQaloZEUleax40yxhiE95MU+mY/d+CbZmTVuDhLHVbRsaUKLVOxSqhemV1bM7uSJnWgOU/rY2Q4XTbo+04h9pVZAbvIkt/2OHbB7XKJr1tej4d3HAJDVKttT21hbCpo2g0XFAoFiJxG7rGuKZeSxvPwNAKP8c1NNtJZgiqXE0+nrQ7m9wz2G48CEAwtGoheFIN0JXzEAkFUnlWJHgz1WbunCyi1dGFWbU5tilYpdvCmWHgugeexsss39DdB8fqwrRjJlUhb+MiX52EkJimUfuzh1Mgl6KgieEKNia7Imzjh6d65tS11A7LKmAUql1D52isoT5LOfoDgXlu7EbU3vg7CXhjSKnehvych6UWGKdcjLkBw8QdvxJFabYtPj1XXtePDVjVjf3iuR7N98Lr2LW+o8dtnJk9G96FlpffeiRch6D3sNjcFClMN2FOiky0VSSqkC3P9UsaP7AQnqWire3MU0D2HlxNj2NDC8lCVWF0/sgioJXrsQXzTqb7POVkdlZSdMdNt3dflkzaivR27SJNTss49kirU4xU7hY+d9Ng0DthcV6xM7v7K6N65cDsgGil15+et+X1R95c5LTGMjDKDs1b01yMjTBE9khQlZOgZocl6yDvJ4iyTJbpx5MomPnbiNpjuhShqfhkVWtIJxBupkqI9dyAuSmKSZ35+S2MqYXeBjF2/GohGh9Jiqa0jNsmKuwDAfuyhfN2ZpyGdExY6PsA2CKML9EUViF1UrVnwGMBW3pAieoGQy7+c+5K+V+5m3WmhTbDrc98oGnH7jM1i1pQv/em0zypaDlZu78Mzq7WiqzcV3oEBqxW7s58/D5p/8BH1vvI76ww8HAPS8+BI67r0XE6+4oqJBaGjEYfFbrdhjXL2U76dUYR47+tB1yYDcn9un+4BmJh6q2OWzJvpKdgJTLL/dsuWJq6wwhfhjSOTGH8D3sfOCJwym4Em+UvK+lsNfg3eK6ne/7CSX2Fnd3ZxiBwCN7zsOuO8prr1dLqP1j39EafNmRYJiMY+dO1GZXvAEG4FBfOxAS6vZVHlK5szuGIYbWeuNvbR9O4B6NwIXgfkrDr4Kqgye4NuqglYogWIoWrZ/P1Qjjx09FkASFNtEKQUjieQeEV5cpMCHiDtTPHdK3N3j8SZH+XNlxC6VKdZX7ITjK64h9bHLCPbusKjYqETBvpN8aOUJd5kSPO76kMcFdZ0AktWKzYiKndIUG3xvoilW/N7CTLFxyZ01gF8/sQrfO+1AnHPMHjjo+w/jBx8+CNPG1OGKe/+D8Yr8dkmQmtiNPuMMZMaNQ+stt2LHQw8DAPJ77YWp/30tmubMqWgQGhpRWLR6O8747bPImAZWX/VBblupUlMsp1TIyVgZHAfY3h0kiaTEzvXtsmP9gSRfG1uOglW9MYftHwefyHnpTkJNsaYhq4mCKfadcg61M2fCME30vvKKvz430SV2Tk8PLO84jNg1vPc42Pf9H9dvadt2bF5wjbtP8xRumwMSFQsj8LHztvvThGUBMODkcrBzOQDuJE7Lj9HUNdx5Obyy6lev6O0FHAel7a0A6mE4ro8djKT3k6DYcccU7iUvFyCnsinGW7Yc4vcV3h8gk/MoxU7MY1cq88ETYqkztihVnvDVtvBrJJOasD7UivugmGJFxQ78GCmoj5344pBV+HQq89jRnH++j110uhNK7Oj+nM4pXCtJLaUBKv5zzyPpJJBGBA3YEIMnxPucPk/5SjBStxoC1mzvwQn7ub7JuayJnlIZhmHg/PfMwBm/fQ6XnqRyholGZXnsTjoJo046qZJdNTSUWLO9G7c/txZffM8MTBjFv6UsetMtT6UiUFFJUaPA2oq+JuIDznYcLFrtHn//SU2cNJ7UFCsVRFc89FUPVnGsSWEyU6yn2Jk5Pio2mFxNpZmYvr2vaevFjL/cDadcxhuHHuaRKyDrETsAKG/d6vZb74ZZ1B9xOBzhgW4LpEo8PzZpUR+7LDPbsW22DSADZHNAnhA7O1AswomdYP42DJh1dTBqa+H09roVNBp2gwHHL2eWzseO99dS7W97Zdno6FSKXcmy/fuBI6OK8UglxSIUEtY2SytP0BccMjLTCCZ/9hvz0534plgn9BrJgQPwjsP72IkuCQxpgyfauov4zb/fxHNvuSb1JOlO/DqpxNxJx0ZBfeySXHNXxRLWMWXQDoi7qNix5xC7FjRBsYqgsf4oIhU7gaSLUbE04pfmzmPEjj0bRJcD+jzNkethqORPDQ7NdTl0F91n2aRRtVi+aQf2nzQKHb1l9BWtmL3VqIjYaWhUGx++/ml09pWxrq0Hvz7rSG5bfUTNR1ortpLgCTGfk8rHbuEKl7i8b7/x2G9Sk79NNZmrIG62FBOiWLibIq2/UWCKFX3s3O1RwROWoNhtaO9FsWwjn80iN3EiShs2AHBr0xo1NXAKBVheXVim2Bm5HCZfdRVw/5t+P9SvTvKxA6lTWSwGip1P7LzzYsQul4OdzQNw84plHULszKgoRXKvGAbM2lqYNTWwentdH7sGAA5gQDaDhkH2saPnxcM05aLyKh+7Epn44/PYRS9zx/e25UmtWJo0mVfsyIuLYMIn2U5ClbW4BMVxUbFp7vlC2cIFf1qCJWva/HWJ0p0QEiW6A4hoqefTnVAo0+s4sirPlmjkvZjMVzTph1XF4JVc/thiVKyqrf9Cy1xQyo5/LmVCJn0fO6bYsVySHCEPflsZw+BM1doSG4+jZ4zB0yu3Yf9Jo/DBmZMx//5lWLR6O55auQ3v3ntsRX0mInbLj56dWFPd7zk5sEJDIw6dXnDCis1d0ra6XPhDutJasWL6hbCoWMt28O8V2wAA79t3PPYa34g/n380xjXW4HO/f85tk1axs+WIORp5GOaflBR+upOuLn5ZZYYRXghFxc52gHVtPdhzfCOyUyb7xM5saIDZ1ASrUPAHbDYEJXEyY8YAeBMZx4ZlmJxKp1bsXPQ+95zv95b1FSJvq20ByLk+djnizJ5IsROvqwGj3lXsAKDU2gZMC9KdsH3i4CsgZEKed9fL3vfIdyAm+HXHG0yuDKWyTYqt02PJx68keCJHHOHz2YDk89G6gdbCJnOxcDxVWkWILzsioYhKpEzbJ8HV//sGR+qAZD529D6xiL9hbPBExT527jJ9eaS5yjKmIZFpmu4kjPhKz5JIHzuHGzM714Kv2LnuJaytT+x8873ihcOhZeuAnJB+SCMa8z96kB+wdtEJeyObMfDimjZ84OBJ+Nr796moz0TEbuLll1fUuYZGEtBJYO/xjdJ2Suws2+EepNQ3LY1i52dgNwXFTpg9X9vQgW1dBdTnMzhq+hgAwHv3GQ8gmFRT+9gpzDRUQSyHTAhJ4fvYdYo+dqJqEq/YAa4PyJ7jG5EdPdpfZzY0INPQgOK27ejN1aCx1OcrduwcAXipQ3iVTpWgmOaxY8h4k4nJETvAyWbh5Igzu+Bjp/J5smyhFiYMmHX1MD3Sy0zHhuPAtMnEZlmAaYbm42JfDSOh3UUL9760HgCwu1D70WR1b8k6Q2GKLds28bGLvr/FUUUpJGIeu5Jl+876psF3ZhK/Uzpp04OmMcXKCYrlc4rL2ReGJ5dvkdYlM8USYkdUJ9V3HWWKVSnEUbViKbGrzfEkSCxNRz00uFq0wrEoktSKZafg5+/0fvOMXDJiytqztCwlW1azKQE0DYNTMDWvi0bZsvHY61tw3L7unGKaBr56fGXVJigSETtdB1ZjILG+PSjVs/vYeml7HTHFdhfLGEX83CqtPMHUq8AUq+7jqZWuWvfuvcZx0V5037g5SFQ16CTCwN6yM0JWe9WY4uD72DHFTjDFsoe5EZLIVyR2b293y4iZzc3+OiOfh9nYiP866kw8M3kmfv/oNdiXEDt/ImfLkYpdINkZ5FoxJ2zm82awWS6Xh50NHl1ZotiFpjsRrrljAGZtra/YBWMK2pQ7OrDy4x9A43vfiylXXyX1yfoF5BQYgDzBMlMsJQ6GQmEsWQ6pPEEnUPn4leSxYyqdm+4k441DMBEbwaTs/1Z8BdWFg/CXDjmPnf8Fe/0zN4agTaWKHRvf4bu34KW17QDSBU+wcQSqotx2FCF2vYLfU9j9JrrNsucAVcTFnG/+dWGmWHKBuBcTcrFEi4EqACzYLzgWHTvzsXOJneGTNbYrNd+LfVI1MWMaUok/jXBkMya+8/f/4NFL31fVflPnsdPQqDZWbtnhf1Y9B3LkDbCnIIT20+CJFEbLMFOsiK07XH+vPcc3SNvYQ7GSqFhxPpRzhQXov49dzj8uEEyqqsoTtiMrdqxMU4YSO8OA2diIt5qnoJzJYl3jeJh1dcGYvf9MbWM55zItLag/9ljh/EjwBDlXRuz8dYzAZbNwqCmWKHaGEZXuhBwTBsz6Ohi1TLHzlCsnCJ5o+/s/YG3bho5775X6C/r1zlNxTFH9VaU7MQyZFJasII8dTw7k+0A8bKKSYsQUG/xm+KhY1xTL7m9x/Ix8pPGx4/ugKVNU+8T5rXJ9e+NjkYUAUJ+opFiIYqdoW0usBiynJYNKIXaJYohiZweBCvS7p6ZY0cfO3Y83f/rHEp8lkp+wfF1FkzgjmzRZO6fYiVGx3PEdTtXN6jx2qXDobi1YtqGzqn3q4AmNIQf1q1NxGLqO5aliEP3BksIWiFRYNKHoOE7BnucVRcVGKHby/pHdS2CmWDbjMXOjqEioyIhlO4rIYPd/fvfp3HqzqRGW925Yzte6vm/+mN2dWGQrM8Xuef996OjOADctIm2DY/iKnWG4plgbMD2fH59h5HJwssQ0RohdJsLHjjNlGQYyLS0wa/gIbMM0/TGUNm6W+pH79c4z5FpyfTvB5OmP3TCk76Fk2WQCjVbsRCIn+uvxbd3/OVLQ3QkuN9eXSRU77z4KXoLgjy3UFCupVfy+cVGxqdwqvLbH7DUWeMRd15DAx45eOssO5Kk4lam3lEyxCyO37LvNZUxuXxohzdqGmWKjzNZBbV+ZfPvuGCY/dmaKzZoGMZPLwRO+Ygf1uYk+rjqPXTw+d8x0/PjB17Gxow8HT22WggUPmDwqdZ+a2GkMOVZsDhQ71Zs6XdUtEDs+eCLNZOD+j3LmBqiJQt7G3rbj1AVZnZMfxmzyVBK7yN4D3PDEKoyuz+PkGr5ahCGlO2HKi8oU60jJStl+LR//GHqeew71Rx/tjrWhEXaPlym/jjeh+2ZfluyXXeemJtg7uoXzk9USo6bGV0KYgmbaZbdBNgvba2naFkzigG6aRmiUophyJTNmrK/YMUXRzJg+USxuVhO7zr4S/uvhN/DRw6ZK5m0KyRTr8ASJna8qj53aFCvfCeJtmyR4gk93EmwTFTu2LL500COkz2MnmHO57ySaxIaBHasul8GvzzoCb23rxr4TZV9dEbwpNtrHLrKfkJcx6RqwFwYWqJAxvByabnuTKnaKqOgwy4D8fAkCIeTyhe5/qaSYFXzHpuK55qc7KasSFJOoWNPgLCy6Vmw8vnbHSwCAK+9/zV9nwH3uGwDevPpDqfvUxE5jyLGSKHaqZxd9MEnEjr7FppgNfCWORYfFKHYqHyq2T7wpVqXY8W3YeYQ5YsdhXVsPfvbP5QCAE/cWiF1Y8IQyLYjsY+erafk8pl77C3+92dgIq9cjdrV13D6cQuN4Pmy5nJsiBQKxcwB42fOZ2dXI5fzoQ6Z0wbLdJ1Y2C3PiRACvwzQN5BqDSVyVF849LyFzv2EgO26sr9ixTUYm44+h6OXnE/Hk8q247dm12NjeRxQ7mSSLvk6s1i0dHTV9MRSJYsdPoPJYxDxhiaJivbE6TmAWNCD7/gWmWPGFwPD3T+xjJ7g+UHOu6vwqfUn74MzJifejhIz+JtNyESWpV/nYef2XiGIHBAFTGcEMCvCuJqGKnXBjcETcCo7Ll1Hkn3tF7/dnmpAIPRD42NEcd/S86LMlY2pTbBo89f9OqHqfmthpDCls2+F87FQKAEfsBMdlPio2xXEFp3cVcQOCPHmqN08xc334sfhlVV3YKJOv6rxeXNuGS+58Gd877UCcdOBE7q29OyuYF31TLH8uquAJy+FrxQLhhNlsaoS1LYTYMSXLMADHJVKZxkYYhiF9xw4Aq+SWEWNmULuryydLBiuZ5PvY5eAwn71sFmZdcL5uVKz6GvJkwUBm7FgYuay/DHjEzhtDub1ded59nimur2xJpi0KifA78ndsGDIxoD52cbndZB875ZC5tlw0qB3sR3elwRN+gmJBbXOrIaghDjXUxy5EkUyToDhIOZN4FwDg1DLLCfwN05KRJCXsgOBcaTJgd9yuzdSkapkT9MNAfTYjfeyEl0SxigUQnGNQSjF4HgYvrHKQR2hJMf8+4hMU6+CJeOw2Wg4Y7C8SEbt1X/ta4g53u/76igejseuhu1hGXynanMqRlghTbCV+OcFkp24XZSINEhRHHytJXVj2YFUqdorp86kV27C2tQePv7EZJx04kRtfZyaPPGnrB0+IkyvJWcVgOyofO/V1zTQ2wjJdM2i5hid2AXFm52DAbHKTO8uTvgPbqw/rn6rjIN88Cujqg9Hdhe2//71f9cLJ5jgn8EwtT+wySdKdGAayY8ehuG69vwwAmWwGBlM4aCRvuQzDi8R1yETJukyi2Jn+7BesMwwjSOfigRLrqEoWgKw0JykpxudvY4qdmMdOTucjpslxEH5vlPsKePPDH0H90Udj0ve+K//eYhS7/rykpQFTy/io2OqYYkVyypbYfUEVO/ZfrIbBq3R8/8H6EMUuwxM7ur9vqTACszw7F7aO3r9BrViXsNIjUleKjGHo4ImU+NsL6yK3n37kbqn7TETszMam+EYaGhVAJEUq9Ys+uMTgCZX6lQSiKTZsUigT3xMR/sQXGzyh7pMfj/dgVfqHKcbltVdVKNhh5EDzlYumWPbcVZti5ajYsEtsNjb6hMgSghDYLux0bMNExjOZSoqd7cD2FDtKcvJjRwPrN8KAgy2/+G84x34BqAOQzXBRxFkSjdv5t78is9e7pLE6CvN3dtxYFFet4tYZmYwUyQsATrHoEzvLtwwHHSbxsVMFTxgAzDJ/T4epMf3OY2fIbdjvxzD49aYRONFL6U58tS08sKewcRMKK1fCam/HpO99V1H1hPWhZivpXtLg9ZmeRTC1zCIKWxixG1Wb9ROpU4QpdlLlCW+RvdgxX7QgWXBwbL9tyI8vSsmVFTt5H9G3mJpv2enQ+5emZSnbvBopmmLp9dA+dvH4IfGtA9zr21uykMuYqMtlBo7YheVw0tDoL8SHn+o5Rh8wPUX+wVqqWLFz/4v5nEQEEWYqxS7ZceU3almxC3zsFNKhktjxb+G0y04jzxE7M6SkmKvY8aZty5GJXZi/k9nYBMtwcxBaeYHY+SZKb3IxIhS7vgKsGiG1CYipKpNxT7CnGxgLIJvjfLYydbWAm5UGHXfcAevMCQD4yDLRFGsbrim2+fTT0fbnPyO7t5sU1MhmYHjZ9WkiZadYBLw8fezY1A0gUVSsZ0qmLU0DMMslad9g3NFkR6o8EcHsxGoDdIxiVKzrY8e3kQIfIJNlf6xMXfX+i6l2fAJD9+E4XorfsqAopgENgBJz7Ym448J34Yf3L8PpR0zFZX/7j79edc3F9DpsHRBE8vt5GomJXFRJw9wgwq4bIAe7sGtJxxOYYmXFV1WFhyN2liMRS/95KgQvaVNsPF698hRp3VvbuvHdv/8HFx63V0V96jx2GkMKkeTEmWK7CqKPXYXpTgQVImxCLPv5neRtiaNixT4jfexU+4e3V0XPdQrva76PnTC5Kn3sbIRGxYowGxtgGZ4pNs8HbLBdsmwCNwyYTWrFzi70wSkV2Z7+ej8VjZjPLpvlvj+q2JmODXS0yeclKii5HMx8HhPmXYKp1/03Rp1yqnesjF8rlppi7WIx+Mwc28m9G/ZiQGEIaUPczwYMS1aBGEIELR/iYZMETyiJHQxuXDTdCfsNBIEPdDwhihKrKcqIna+q8f/DK0+EnoZ8LOG3nAY0FyU7ZNg1PGhKM+7+0jE40qtA4/dh8NcOcN9DJFOsr9jxihpNueQ/U5gpNuS3R5+TYhsaFev2JbcLywbAInUBwRSbCV6UaJoc1j8l/2IKF430mDGuAZedur+k5iVFRcETnQ//E50PP4zSxg1wSvzb5p733FPRQDR2TYjilcpfLTIqtsKSYiKRCjXF2uGTRqVRsVTpYbmmoky+SlOsxT/4OWLn8GoVK/XF2voKgSKHi6P0sZOPD7g+drbR7vadE4gd+POxYSDTGKHYeY8Rk2zkFDuKbNYfU8YwkK2vA9rJ/p0dQHaccA6CuuSZjs36eow69VTgsZUAACObhdnHFDtyPsXgOccIMr330hA7yZetlEyxU93e4m0bNQzWVqyRyrZJtWJ98sXvF5gLnVD/Ukbs4JmZxcCEuMoTluPg6v99HWMa8vjS+6JVi0p949xxBMcL+oneR9yeMQ0/PQWDq2KJ1gj+hYCpYD6xMwIzKNs1VLGLUDfFZ4mteEaEWSroGOixc9mgXdmyJX8/hzxbcpyPnWZ2lSJjGtjSWaho39TErvVPf8bW665D88c/jq7HHkPzJz6B0jtr0fufpRh95pkVDUJj14VsipUfZPQhEmWKTcHriE+I+s3VH1+UKVZ4uw4/Frw+2NttMGaWayrwGVOTLXlcoo9dsK3TDvow6uvReNxxXj9sHFGKXRAVm8sYKFlOKHE1m5pgeX5o5Vye28aO5ZuDqClWLLFW6INT8lS5bBbmqFEYc965/rUIaqx6hCuTJYTEQKZeVOw6wdmiAS5PGQBAyPXnk5dsJoj6pMETpUCxY3yOKhoqU6wIw7YUK41IU2xYiguGdCXF2L0erKOuBryPHW3DE1LOxy5EsbMstWInV54IQM9v644CFjy3FrmMEUvs2DOkEg4RBEAFfmNx3Yjmb8MIynAx2E74c6FY5hU7el3FkmLhPnbks+inLPrYecskj7cUFeuvDwmeYNHmZdvxAyiCsQTPCNM0BB875fA1CB5ZxufLdBwHW3YU8KdFb+PI6aMr6jM1sWu74w5Mmj8fzad9CB333ouxXzwf+WnTsPWXv4TV3lHRIDR2XYhvpPHBE0JJMS5qMDmzYw/DwBSrblfyUz3I2wLFLuZgDnvQmlyOMv/4FlUGFbsrugzySbFDBK06rKCT0Z/9LDItLQB4B2dAHTxBfexqsxmUrHLodTXqG2CzqNismti518iBY5gwWfCEOBH1FWGX3P2zo5qw77OLYJgmsn9fCsAlWwD8aFXeFAtk6+sB9HltHNcUKxI7R7jXRNMxK2mWzcJwXKLlcMETAfli14NGNyfJsG+GKHYmIY0i0gZPJCkpRtN8BPvJPnYM8suNR9aF8XHjFkyxYmACVf0Y6HhYShlGIqLOy/HvhQpMsSQ6V3zxCYO4NWPKBMZWqJlBQATvY0dNsb5aFmeKDSnF5vbPXw9f/UtiijXVip1pGMhlTJRti0ts7Y6FJ+5ZUyt2aXDhn5dwywaAMQ01ePdeY/HdDx1QUZ+piV1p40bUH36YO4DaWtjdbrLR5o98BG9/5rOY9P3vVTQQjV0TYX4o/LpgZbQpNvlxxZxu4abYiHQnCaNi2biyGQNFCyhxih0jh+w4KsVOMS5mio1R7Macey4ZR/BWDQCGKdfTdJwgQXFNLoMdhXL4dW0I6ueWSYkveix38vKCBjxTrOxjV4BVqgGQgZnJ+EqiT7r9yhL+hfQnTdMwYGaDx5gJB0YPnwAZYKlJwhU7tslNUOwSLSebRXbCBJS3bHGDJ0hfQHzwhAg/eIJTxgwYEYodX5lB0adw30aVFBOjXul9awjjop9piSq6TRUgwGCXeVOslKCYtQvxsROV+Ch+0B9TLHWnSFp5QqWSisZY2wm3RrDfbt4PnghIlh+4wMhYEsVOaCLWnVaRRJVZ3j8Xb2PJ5ttnMwZQ8iqWkHN1S8sF94gOnkiHtyqoLBGH1MET2XHjYHW4ylxu8mT0vvwKAKC4bn2KEuwaGi7SmmJFYkdJUiofO6L4uP9DTLFMsVMRO8HUEQbRuZsW6WbpTfy37JDnoORHY/MPa0qQd2RqMPE738FuN/4auYlBYXTJHBZiimWTam2OOV+HqAb14cSO7cERVS94QuzNLhTgeATAIMzEdy73iJvvf5cJFDvTMFCz29Rgn1FNyDiyhCqlO8mLCqPjH4ulOzHHjYPhJT+mpljWDyUfFQdPADAiFTv5xUUkaBRJ8tgB8ouM0seOKXPsspO0HGx9eOUJ22/k2LZ079HqFQy0pzS+s1HuEnGg7hRJfexkv0Y5eIKSHX+d95+WFAN430UxkjXUf5dTOtXPhqyfxw7e/4B8hb3QZsxgDNRlxDQMn4iWLP635BASKyYI18ETQ4PUxK7+XbOx4/EnAADNn/g4Nl9zDdZ+4QtYf+mlaDpxTqq+rr76asyaNQtNTU2YMGECPvaxj2H58uVcm+OPP94zHQR/X/7yl7k2a9euxYc+9CHU19djwoQJ+Na3voVyOTzSDABaW1tx1llnYdSoUWhpacH555+Prq6uyH00qg+RFKmeY3zlieB7tWxeMUil2PlEijdPiSgL7Sh8x+u4qFgmNJnym7AYWatMdwLVW7ln6lI4Rrf3FDHmc2ej6YQThHEIyotC3rHsoFZsbS6jPLY/JjLWckYQ/xkZoI7UnilWnIjsYjHIY0cCJRjpZcTO8BW7bDCRmOB87PKTJiGj8Oi3HaDcRZQ8yRTrwshl/Xxz2fETglQxyqhYqtjFP0qZYicSKLMYFTxBxqgwOapIRhjoJCs2o+lNWFtVGwAB4UNYTCz/8oJyWXIDEE2O4mdKmqMUcb5MVmizUGSUil26PlTEjvqdBWN1/wfJyPl0JxnT4NRQNi4Vokz0vvov+OupTM3iCyufxy5Yb0CsMcx/b5wplvzmdR67eHz5zy/gxidXS+tvWrgaX739hYr6TG2KnTx/vu8kM+ass5BpaUHvSy+j8YT3Y/RnPp2qr4ULF2Lu3LmYNWsWyuUyrrjiCpx88slYtmwZGoiZ54ILLsD8+fP95fr6oASHZVn40Ic+hEmTJuGZZ57Bxo0bcc455yCXy+Gqq8Lz75111lnYuHEjHnnkEZRKJXz+85/HhRdeiAULFqQ6B43+Ib1iF/jYidGbldSXFFWMsvT2K/tF+e0FU0f4sdgbNF+WJ2PKqQXC+IF7bsEYfMVO8LUDgPZeNVGgZX/cY8kHs6mPXY7VFFWfHyU2ZZN/lPhRseQh7zQwU6xwzEIRdokpdgGx853Lc0LfZoYj5vS7qZ08CZk23hkZALr/sxRdi18BJnk+K5Ji5/43qWI3fjyMLQpi55tiqY+ddEgJhiXnsYMBGFYEsVMET2RMI9T3M2oepZO4qDAakAmnSNtEUywcOfKTwaHlr8plRYJihWJHPhcJMYz6edF7qTJTrPtflQokfB/Z/C3W7LVsxzensqADMSqWJSim9ar9ihwsCXbY9VUouQxxPnb0/KSoWJPmsQu+Q8MIol1FHzsaKJLRwROpsfjtVlxy0j7S+uP3G4/fPfVmRX2mVuzKmzYB5OHb/KEPYdJ3v4PRZ5+F8rZtqfp6+OGHcd555+Gggw7CoYceiltvvRVr167FCy/wLLW+vh6TJk3y/0aNGuVv+9e//oVly5bhtttuw2GHHYYPfOAD+NGPfoQbbrgBxaLaxPH666/j4Ycfxu9+9zvMnj0b73nPe3D99dfjzjvvxIYNG1Kdg0b/IL/VOnhq5VZ87vfP4Z3WHn8dAzXFysQu/XG5nEuKV/5SlClWeCMOg6jYWWRSDt6OoxU78QisvSU8tAGgo0dNFCzhwa5S7Dhi5/m2hakGVJURFTu2S44kNmU+eeLlsosF2J7CHvjTBWbcjEfsWB47x+HNynSiyk+ahIwjR592Pb+YSziMnNon0Mxm/YttjhsHw2vHK3beOXM+dskVO0MgUGYxPKWBKt0JVY+lBMUJgidU7UTFmllHKPzKE/7YwtVces84liUl/1UpdvR3TgNTaF99JQsX/GkJbn9ujbQtSQCLCFWC4jiVSdzqRsXy66iZWq4F7K6XS4oF30saxU7OY8c/S8S+RB9PCpqTj1YlMYgplpJUwH2Jo0mih6uP3ZVXXilZ//bff38AwNtvvy1tY39/+ctfQvs877zzpPannnpqqnF1F8pcihiGrGlih6LSSRKkJnarTjwJVmurtN5qb8eqE0+qaBAMHZ7v3pgxfALI22+/HePGjcPBBx+Myy+/HD09Pf62RYsWYebMmZg4caK/7pRTTkFnZydee02d3G/RokVoaWnBUUcd5a878cQTYZomnnvuOeU+drEIq6sr+OuWHbQ10kMVqn/3knV4auU2PP7GFn8dAyV2YmmuND52olkSCA+gANRO6UmjYsWcbiXf1yp4iEYRSCA88k1piu0tKVU2uT5uWIJit12N72OnPi86odijWrhtPpmlOehCSorZlg3L+02bCsUu49W69YUi8OkV6CWrmTyZy4XH0PPGCi59iVHPF95me5g56mM33i/HRvN1svMucz5I0iElmLbt1pylBAoAQl5AAbWTvBmhiESNIyydCRsHTzhlAiOpbZD9yBhYNCzgKnahPnZkH3quYRVlXl3XgUeWbcYfnn5L2laJKZZWWQhU/GT7MFDlnY5ZTDsiRlOrfOykqNiQHx9dmzSPHUtjRIlmtGLnETtvm2+KLdvC8XkLCB8Vqxz+kOGggw7Cxo0b/b+nn34aADBt2jRu/caNG/HDH/4QjY2N+MAHPhDZ56mnnsrtd8cdd6Qa0/6TmvDAKxul9fe/sgH7TGxM1RdD+gTFjqO8852eHj/DfSWwbRuXXHIJjj32WBx88MH++jPPPBPTp0/HlClT8Oqrr+Kyyy7D8uXLcY+XCHnTpk0cqQPgL2/atEl5rE2bNmHChAncumw2izFjxoTus/03N2PbDTcEfUQ4PGskh6qkGHvwiWWzAKC76L79m6bBBU6I7ZIeV0w9EYaoPHZJo2LZW1mZU+x4swfzbRK7FJd9xU5xjSzbQVehjKZaIaCBKT4mP7nyY3VQLLuTcp3nYxc2edNJp+Twfflklihwdl2DfwwOhoFyRyfQxJti/TQQvroWKHa+GiIoSzW7T1MqdrZl+yXBAMCRFEbvO8nlfGXQHDcuIHaEfLFjMzLOFA3DcXhVUIABB06pxPuymfDSneSU+6gULT6zv6DYRdzEUb5Vch47A5bhSG0AIY9dyK3P+c5aliJBsdcu1MeOrCc/c0b4VC4I/YqKJWbl9METqnQnctCUGHST833siClWaBtK7Bz19aH70OCJmxauxjUPvcEdj42dQkXs2LJvihUVO0JiRVNsGitKpegulrGjL3jxymdN1GTliH/AnecnTZokrc9kMtL6e++9F5/+9KfR2BhNrmpqapR9JsXX3r8PvnzbC1jT2o137+UmVn9m1Tbc98oG3HDWERX1mZjYbb76GveDYWDr//wSZm1QG9KxbfS++gpqPVmzEsydOxdLly71GTTDhRde6H+eOXMmJk+ejDlz5mD16tXYa6/K6qhVgrFfuhBjPn+ev1y/fj3Qj/PVcGHZojk1eHNWqVEA0FOy0FiTlRS7ND52SU2x/rYoH7uUUbF8Ulh5nQHZ9CqemjjBideovacEB8Br6zsxe8YYmGaQQNUQJmhxrCx4osYPnlCfH/WxC6tWkaX+cV66ExEOAMsjdirFzvSIHVPiHPBJaenX1nDkERj9oQ8BwvuZbRgwx44Divz4uEHA8zVjPpGE2NmKdCcMQQ1VhytDJsJ0HDjFIncvGTBgFIsAGpT7WIqob84UKx4joSk2PipWQRoZKeOCJ8JIP/GxsyzJx070JQOSKXZi6g76UtWfPHaOQ1TblKZY9puloHnsmG+tFDwhKHbUX1RVLYKC93ETFDtFiiZG6oBoU6xJAjhEYuf7CJdlH7uwdCdxL73VwEnXPw+zJrDOfX3OPph30r7KtitXrsSUKVNQW1uLY445BldffTV23313qd0LL7yAl19+GTcQMScMTz75JCZMmIDRo0fj/e9/P3784x9j7NixsfsxnHjgRNx8zpG44YnVeOg/S1GbM7H/pFG47Yuz8a49k/dDkZjY9b3+uvvBcVBYscL3PQEAI5dD7X77Y+wXPl/RIC666CI88MAD+Pe//43ddtstsu3s2bMBAKtWrcJee+2FSZMmYfHixVybzZtd5+kwFj1p0iRs2bLl/7P35fGWFOXZT3Wf5e7LLHf2FZgZtgFEHIdNlmE1CIKJIIooQjCDRtyixIVoIkajoglB/SSgUdwF1CQYFoEgAwgKyOIIAwjMvs/c9Sxd3x/dVfVWdVV3nzt37r3M7Wd+87vndFdXVS+n++nn3bRltVoN27Ztc27jlUqaw7Xfar8R52gMphkz4ACE47BhbhToH6pZiV1jptjwb5IjMcWelBQzfezk9sx+EzWz2APxB6hJ6ExyuXOgin+9+1n86JFXcOO7jsKJi3ti6Rxse1sP4j52zqjYBGIndtov+PAQndLonhHfN4ZaFJGu+9hFDzzfg9feruZLyD9NzyDadp10InDz7/Qx/AK8yZOB9ZHfprEv4rvf2iJrxaK5xarYmcdDHc/k64AhJHa6LxvgVd0+dpz6qhmKazi2nXxZx9fGtVzPTG9s7o9SesPvtiS8Atr1SKJiZeCObZ7cfj3R60XVSI6vG447F/0Nm0mUXbD5NZrLuEWxE79hFTzhaXOgEalit9w+dpTs6utkuhPHxZB0zyt4lsoTUZOSTM1kJCjmblNsIy/bw8Ud7zsKM2eqlEelgt3DbNmyZbjpppuwePFiaWo97rjj8OSTT6K9XX/pvOGGG3DggQfi6KOPThz79NNPx7nnnosFCxZgzZo1uOqqq3DGGWdg1apV8M1SiAk4ack0nLRkWnrDjMhM7OZ959sAgHUfvwrT/v4q+CnyZBZwzvG+970Pt9xyC+655x4sWLAgdZvHHnsMADBjxgwAwPLly/FP//RP2LRpkzSv3nHHHejo6MBBBx1k7WP58uXYsWMHHn30URx55JEAgLvvvhtBEEjimGN0YN64As4152z6V+Af/+sZvOe4BWgtG9GSDdxDqI+WQJKPnd0Uq+acBHFzKxhJ6qhKQpPA2qZhDmGaqc1jtKO/ig1RncFNuwa1eSaVUQsIsSun5LFLUuzEGsbCtCVBnau6tiYHZIpCMJJsWJVcQlQ9Q+1zYBBhAY/ZSXjnX/0VMNQEIB6QI/YbCPPgNe2/P7AtJCOK2JFasca2kqxwoJ5ErHhkiqWKGxOKnR30UJlEIdxeb59ZsTOefYzpnSWlO5HDc7diF3BdsYunO9GVqfCz2p6aYjVfTkPJ0stkDUOxI1GxWV8MzWEoIaPzFPOWpb0MxU5ExYrbgjDpi+2B4fnY1evx68Scr+1zOBcmI4XNxNSCsFVsJcWImqwpdmlVeUYAraVCzO3EBuort3TpUixbtgzz5s3Dj370I1xyySVy3cDAAG6++WZ88pPpxRbOP/98+fnQQw/F0qVLsd9+++Gee+7BySdnS//2+Ms7EHCOI+Z2a8t//9J2+B7D0tldmfqhaDh4YuY1n5OkrrphA6oOn7QsWLlyJb773e/i5ptvRnt7OzZs2IANGzZgYGAAALBmzRp89rOfxaOPPooXX3wRP//5z3HRRRfh+OOPx9KlSwEAp556Kg466CC84x3vwOOPP45f/epX+MQnPoGVK1eiHPn8Pfzww1iyZAnWrl0LADjwwANx+umn49JLL8XDDz+M3/zmN7jiiitw/vnnY+bMmcPenxyNI+ZIz9WN3GWO+Pnj6/DPt//Rotg1Pq7mV9SoYmf4orjHEn3oPzfq10QVu7hhJ34MTB878+a+Y6AiyYokf8aN2ibZBVw9eIRi53reUXNbJWYWF0PEfXZsD1CpmBHFrrs1JFWTWkvwu7pUVCyIP5SnR3OaPj4CxQMP1AhCzIcRar4FUdOWc7CSJSo2Zoo1e9HhC/+yyBRLZ+cxAEMJih2YPLc2ldlWt9QFevnZEhubPnZOUywJfHCquTRdSa0eC56Q7Zw+dvYa0HVDzddMscMhdtIkzK3H1wa7+TK+X/TlA4DcEZWg2F1SzLZ/FDZTrLjulWKnTvjCKcrCVK2pY2vbFzO3prgfiQj3Wj2InXepdnq6UtiIFWW00dXVhUWLFuG5557Tlv/kJz9Bf38/Lrrooob7XLhwIaZMmRLrMwmfuu1JrN85GFu+cdcgPnmbPQA0DQ0TOx4E2HzddVj92qPw3Ekn47mTTsbqo16Hzf/+71ruoiy4/vrrsXPnTpxwwgmYMWOG/P/DH/4QAFAqlXDnnXfi1FNPxZIlS/ChD30I5513Hn7xi1/IPnzfxy9/+Uv4vo/ly5fj7W9/Oy666CIt711/fz9Wr16NKols+973voclS5bg5JNPxplnnoljjz0W3/zmNxs9HDn2EDHFjkSnmcSE4jfPbbX4dWW/idj8lZIVO8sy8ia+s7+Kt1z/AP5z1Yuxdm5TrEWxs9SdBOKUoWaQJJtiFw+wiG7U0icsPpBWKzZS7FzEVVPsavZzwRgtRm7flwBMBh0wEtRw+iHT8dXzD8cHT1kcErtoOeeUCJu+YUwmNqbghiITMwcLIkr6Czis6U7Mh63wybNF44o5AiR4gppEwRKjYuncbIpdzPk9o2IXT3cS97EzezLTnYTVFexjaT52tSrMKHQxlmbSI9tXHelOzGChkTTFmq4KLpirw2tQXxYGT4SfzQoQ4oW0SKJhgfDcmbnnXIqXTemkyZbpdwDoalGK1m6SWcBmio3l1oyaiPmaJcUCTgNPdJI7noldb28v1qxZI61/AjfccAPe9KY3YerUqQ33+corr2Dr1q2xPpPw7KZeHDKzM7b84JmdeG7j7obnAAwjKnbzV67Fjp/+FD0f+iCaXxNGbPQ/+ii2/Nt14EMV9Fz5gcx9pdnf58yZg3vvvTe1n3nz5uG///u/netPOOGE2FiTJk3KkxGPA1gTFBvJOW3XyfSOpj1KUCw2TUraSpGWx+63L27DI3/ejt6hGt6xfL7Wznyjln1qPnbxFCgUZpUs0d6lavZXarHjZ6omtnHqgYpwa0oJntCiYh05BRlTkX7Ctyh2nhiTQQfUx65c8HH24aHvzNrOTrDtQrHTiXmM2Fl2jD646fwE9HJLapk9KtaYfvSQY5ZSZuEcgSoPiV9gBE94DOAJih0Q/g484vGmJYA12iZHdpPPpinWoPmMsdgFIh/Y8vi4f3PUNxD1uiU5tupDwGWWtRFy8y81YzYC+hvmpK8kxEyxoR071s70dXP52NEo9VjwRAYzBL2/DMEePOF6OTN/KzR9kKk4ivlWbCXFSO1mijRrxmjiwx/+MM466yzMmzcP69atw6c//Wn4vo8LLrhAtnnuuedw3333OfnEkiVLcM011+DNb34zent78Q//8A8477zzMH36dKxZswYf/ehHsf/+++O0007LPK9SwcPm3iHMnaynYNq0e3BYAUHAMBS7nbfeihn/+Fl0X3ABmhYvRtPixZj0trdhxmc/g5233DKsSeSYuIiXFOMwVSjx/Z3L5+G2lccACE2Ntdi2DYxrNcW629vIAn3bFzmidlqqPrh87GimeZUCJW4CA+K+TKJ93SDBAvVAHVvTXCv8imz3DHojbkrLY5fFxw7xfFoxMw5p7xXs75qhYif2QyepugnR7jRO/fLoXExQ0zHnUCXFaB47YwcYD+uhMkefsmwdtwdP8JTUSSaJoS8ZsbQlGSO7s9SKdamB9Pi4fnJaOoxaXSNgAImsTTCP25abvmeCMA7HDAtQd4q4D6oL9uCJeDuZ1sSIiq3UxP3ANMXG08DUHFYwV0WS8Du07+FcHGqycc+jL0pm8IS4f9WsJcXi91M6l/GAV155BRdccAEWL16Mv/qrv8LkyZPx4IMPasrcf/zHf2D27Nk49dRTrX2sXr1a5tv1fR9PPPEE3vSmN2HRokW45JJLcOSRR+L//u//pBtYFhx3wFR84fY/YhdJ2bJzoIov3L4axx3QuGoIDEOxq+/ciZIlyKG0YCHq0Q7nyJEV8eAJojAZZpeC72Hh1NBXZLAaoHdIz8rtcuS2wW7WyvZQFJCO1wHHUJT7bYel6oPbxy7up2czgQEJwRMOHzvq0GwGWCSZYimaLHns7vvTZsyb3IJ5k1uN4Al9fGp688nD07YvnJhiPVf+qWk9YNgZtacJis3Eusz6lhua+AmJMNeT+So1ya7YmaTQ4xx8aMgZFSv4vBeZYk0fuWBoKPEVW5piLSQm7mPn7ifNFEsXhdehSWCitmJecAcc6HnsakqRleRQ9OHYhsBuijXI7nCJHQme4MbvwwVztZkkW8BMRCxNsWZJMWGK9ahiB+2vCf24IRpHv4joC46LIGYpKSa+lxwlxTjUuTP7y6I4jhZ+8IMfpLb53Oc+l1iKlN4Lm5ub8atf/WqP5/X3Zx6Iv/rGKhzz+btx8MywqtbT63ZhSnsZX3nr4cPqs2FiV16yBNu/dzOmf+LvteXbv/c9lJcsHtYkckxcxH2dOMSjI37zBtrKBfgeQz3g2LJ7yOirgXGFY3BWHzvLg5eacYRiN1CtY7Bal6QIUISzaKor5E3flttOm29MkYuUC0mC9fa1OifE2P4QpMOIY0qh0p2Eyx9+YRsu+o8wtdCLn3+jdtOuZDDFuoInOIN88DOHYtf1lregZfPtwK54KgkzAMZURsWYuvnIvO4gJkzUJJDKE25ix3iAYHDQ7WPnMQAcjAfY+dOfYnBwf6A5DNJiAPhQBWi2bqqNpxzUCbEzx8rqY2dciwzMIHZxFUpsI5ZT5TQ2ZyOPXcD9qN9oPMPkCGRT/0wTpa1MViNQPm175mNne0kSvqoiiEGaYkUeOxI0EfZDa8Xq+2fCdkxsJE3ADDSTbVh8G+UeEv4V31Wt2HiCYvOl0TbPHHZM72zC7R84Drf+fh2eWb8LTUUPf3nkHLzp8JnWUmNZ0DCx6/nwh/Dy5e9F36pVaD78MADAwGOPo7Z+PeZ88xvDmkSOiQvTObgecHVjMW7iwjG3q7mIrX0VbOnVTVg2f59n1u/C9x76M95/8gHoaVdJtcV9zk942FGkmWIrJHhg10BVI3Y204jYn1j5HgarZGfuWS3Fx66uKXa6cqcerqp9wSB2BY/FTDv3/WmzMQeq2Jmm2Gh/oBNgINnHzi/a0xYUurvRcvjhwH3Pa4EQHiGOYr9MZVTsg26+0tfT+Wo+dkWLYmdcsywIkhU7zwNQBwPHztt+jmD5eySx8xgL+04kdmKOoj+1brh57Mx2pik2/K63UYmYFRlyVyVRn3mtDi6JnU4Os5jH9Tx20V95LYXfh+2LRH5/prk4bRsBW7oTQEWKC6VLnEBpoi0YlScYk+c2Nd0Jj382XxypYidevA6d1YkPrFAF5817XpJiJ0uK1W0lxdTLN8VoJCjeF9BSKuCo+d2Y2dUkrR/3rA7vt6cc1Hh+u4aJXevrXof9/ud/sP3mm1F5/nkAQPspK9B9wdtQnNaTsnWOHDpsJcWoL5VYBqibUKckdoZiZ7E23PibF/CjR17BflPb8K5jFpC28bfc5Fqx8XW0pJgwxQJhrdaeDkUipXOzLY+dzBmlbqKZFLu6TnrN+38QcJIaAlpb9XBV4wjHa4Gi72kmSQDY2qcTac3HrmZX7MDiFTfMWz0vllRUrMMUG3Ul+xbnmvqCiXHsPnZxlUFfT/tThNZaecI0xQYB+OCg28dOmNwEcSDtGMsQPGEonZS4xlS1RNWZOduZ1x0Dt/iS6X853H5xNI8d6jUEPKr3a/rpkW3cfZHPjqjY4ZpizUjSLH2Zq0XxdxPiN61MseEYZkkxQeZozVkxnUwlxcR1Ydxf6HUiFLuPnLYYxy9SflvW68CRx46WRDQJuUs1PGwYOdgmGl7a2o/L/vMRrN64W1Ydokfx+Wve2HCfDRO76rp1KMyYYY1+ra5bh2KeBy5HA7CVFOPGTc18G+yMQvc3x0yx8ZvgYDXQ/pptNRUj4a3fmqlfpvHgGCL9x/zsomkVrHnsItJD6sdanyvGrombrityuM7dpljlwK5g3pBLBS8WobfdIHZJPnaUKEliVw9w/jdX4cHnt+k7Uyyp4AmHYhfOW5GBOqfHSyeqWXzsYoqdNCUR4sIzJigO6giGhpwei+IBKwgdbecxhqCSTOzMqGb6/HapajboQSbGSwb0627LtV9F8zveqbWRx1WcB+72a9UUpTrNY6f/pe0yKXamKTajyuYCVZPNdEAuxBQ7R3vxsmOmMJHRskZJMf2lQt8/E3SpiorV7y9U2bVFytrmTitPKFIZfi8mKXaGa8v9f3ci1m4fwCGzOq3zz6HwD794CnMmteDmS1+P4/75bty68hjsGKjiH//rGfz9mQcOq8+GDbjPrTgF9W3bYstr27fjuRWnDGsSOSYubCXFzLdxLh8K4U2jqzl8+JuKne0WmGSqBIw8dklRsRayQJWdoRoldjoBcqc7UTfCNB87c99U8IQYQ18fBDz2EDTfqjVTrLHzpYKn1INojG0GsTN97LQIR2qKjcb7/Us7NFInCWapKL+4gidoe85phK86XuJc2hQ76hwfzs+Emi+N+rTlsTMdwllQBx8chOdKdyJKR8kxyPZMJ41eUIcJaYq1qCIuVc2GpFJSjBl99fejbiSfV0E30XyQVFKM5rGrwVTWhDlXV1HtfdUNdYj+tR2TRqDcKdzmxNQ+iPJOUTVMsVwu1xMUq+CJOOF1BR8k5bFT81JqdrWuH3/Zxpi3FqnvUOyqdW4Qcv1FDgBmd7dg2TDrnE40/O6l7fjgKYswqbUkXUuOmj8Jf3faYlz981FKUBze7SwPnv5+sAZCfHPkAOIm0YCriEeTkIgHS1dLqKLETLGWJ4PLV8Xmm5NoirWso1GxNHhgh5HyJMuNt07eeG2zMPdNRNy53uzrAbeYrdS4gO7wHVPsfC/mG7et363Ymd9tCpgJeUyLJQQieCLhHIg14TUC2V6l4Yj2z5HuRK88YSiMJK+h5mOXoVYsq9cRDAw4TbHiAS4VO9LOY0wzxXYN9Vrmbn/BASyO/Imqs7tdmOaFrI+CPSho6atwXm7Fjv7eeK0WS1BMj7GAS7HTSAS5pvWAh+ERO+pOIQMAUqLFY+Zvz14tphKLig0HUITPrdiJ42f+xgR0Hzv7iyPNiyfuF0kBFoBIdwJtDsrHjkbF6i9xKq+kdbo5ElAPONqi8pjdrSVsjEpAzupuxvNb4veDLMhsit14zefDD4xh81e/Bq9J+RDxIMDAE4+jacmSYU0ix8QFracoksiapqeYKVYqdmbwRLz/QPqZxYkPYKTKSHgopkbFVpXSstMwxYp5maoYdVivSYJrJzfmvqWVFKsRxc4015rmMCDueB2aYvWxTcXONKNX60EsiouaYmPE1mPhSS4WwcUDH27Qw0IVVyb3x63YmZUnzONJCQr1c5IlxWgeO9MUC47q+g3O4AkZ3CFNsWQshD52lzz5C/xh9sFYsPEF/HCxXmPSVKmor2Y8bUm2l5NY8ISnH3vGeSSnq/PpEQICpETF0mNEEhRLZdYwT4bb2PuypTsR7U3y0SjkC2VAS4olb2MzxSYNL6Nio/5lGhThY0eOiapmo7/cmggsxy3+4ijmylEl95ekfSn4joAuKCJaC/SSYuF1sGfnYSJj8fR2PL1+F+ZMasHhc7rwjXufR8n3cPPDL2HupJb0DizITOwGn3km/MA5hv70J2miAEJzRdPiJZj87ncNaxI5Ji7EzaPkexiqBeENFjohMd/KRXmcmGnQwuzMyFBz3KQIQwrbOhoVSx84OwaymmLp27EKnrBNw9wzVVJMH4OOaZI+MyWBSWoZUw+fkk9MsUKxixE7fU7VGgdK0fjkoSB8zMw5KsWuGMtzZoMtGtPzSFShg0CGc9XNhvE0O5Dji81dip1Joj3OUV27Fox3WeetCJVdsQsqQ3jLi/fi7aXNuKk+Pba9aW43o1e1uSQ8VxMTFJv98gAsqIM+IkxTLMCdOUo0RalWl7/pJB87V2e6j52+fLjmUwEmf3/0mkpR7Izv9Bq0oWjIWEKxEwRd+oaSa0/sstvHLk52kyrbCDJpi+iNb6NUTLqNeDGt1OKBSLZKPjmy4YqTDsBAJczJ+sFTFuHd3/4t/vIbq9DdUsK/XXDEsPrMTOzmfefbAIB1H78K0/7+KvhtbcMaMEcOCnGDKBY8YCj8Lm4Zqvi5fvMWPnYCzUUfA9W69Y1fbGuaNLhx0wKGHxVL89gB8eAJlx8QjYKjPna2WZhv7rFErZa0MaYpO0nd8BjDzM5mrN0xAEBX7OrcntbCTHpKzdFUkxLkeciInJXHo1BUUbGJxET1TfdFqKmyO1e6E4tfkvmdqqgcUJUnaFSs6WMXETsPndZ5S1Mb8eOT2zLlY+d1dqCwOe60lpSMN6sjPwDw/j6gtWTdzozs9MDBAtMUq58jzu0vU3SuAKKoWH2+Nh87p78eJXOGemczTzcC6gKSNeeatfJEQntBiMxqEsL3jlaCiQVPZDDFuiJS9f5gbRPPY6d+R3UjIKpITbFkOf1t5byucbyBRCnPn9KKuz90Anb0V9DZXBxWmTxgGD52M6/5XE7qcowYzLfNgCtCYpYNMn3sBBZPbwdgz2Nn+uvJ5baHZMKvIU2xo3nsqI8dJ0TVfHNnVh+79IcU51xFxRrkTSDgxBQbqO3CfRHj07kA80itQj0qFrEqH3RsAZrLjvrYiYeHmetOzqO5WU4m8UZmMQH6HsOUtjJ8j2F6Z7NcZiIt3QkN9lCmRoePXU0PcPAgFDv7Q1gGq9hMsUylO/E7u1AI4sdZzjv6o9WKjSl27uNXe+kl9D3wAJ496STwXTu1deF+699hBHKo60b9Vl1USPexqzvdAHSCkEWx49pyW8H7RqDXihVzS+7LFomctIn43Yv+lWKnlxQz/TuBbOlOpKuHZ7p6xPPrxYMnTGKnfvfyZdPT96MWmD529hflHMNHV0tp2KQOGE7wRI4cIwhx46L1FM2EuqZSIdKdCCyRxC7ev+jDVroMMIInEh4OVsWORNTRPHbCx+5jP30CJ3/pXvQN1aP27pJitFas7ffsUptUSTG9fWgeRrROP56yggDRGTzGMG9yq/xeInnsOOex1DKUXApQ4kbf4MV4FUOxkw+Vchnt55wTzckNsS6MxlTHa0pbGf/1/mPx7XcfBcCRxy6IFy7X9weyP1nkPrATu9qQbpIWip07QXH04BbEjgzOwAix60TBEllrKq9a5YmY8madQjjvP7+Ily69DLV161FZ/UdtnXndMc4jUyxpY0RTZ1XsOFHsTDeAJBVVLrdExYr2WZMKu6BHxUbLUvoyjzlV3m0oGK4IVSOogkZ105cpTl7OTOg+doIoxpXENNOrucxnisiZ990iMcWaimFuih1faDiPXY4cIwnzpkSVJjNXlc0U6zFg/542rS9b/7FEyEYof/jZfVOy3a+cptjIx+7OZzZhS+8QOqL52n3slOonxrGmOyHTtxGoeFRs/K3fvFGbCs18otgVjTx28ZyBcRW0ajHFMjCnYieWcwCsKVTbMgh2huknXLhkeofq1xKaZ5qTTa2JE4JgTVBMSorVKwaxE4rdgSmKndMUG/bnd3TYFbvopUG+jGimWL1t0jVcffFFoB72ZaZmMZXikNiZCqv+QhAqNfaxtOX1eux8UXIot3FF2GrmWqLeBSMXFRuQ6yNLX4zpqnTSFuLeJmYuXuJUgmIm/9JzG5I7e59ZTbHmrrhcSupQfTDj9yq2EPevMHhCV79zU+z4Qq7Y5RhTiGe9uGmEiXXDZWaaB3FToqbYeZNb0VzSa5pSuKLLVBH5bIqd7WZPzSY2HzuTWMXfqFUf1MfOBu05SR9uhv+VgJY2Riqfalw6//AL4oqdeOgFwGYjtUw9iCt2lVpcEmMZFDvO1f5lC56Ik30Km58Z5/GISm09nRc5ryqPHYmKreg+lEKB81yKnSAzUrEjx6GmiJzf3Q3f4mhWq9bkfAD9Ok3LS0YRbNwo98dW11ZzS0AAGCZn4QtGSZmrpJj24K/ZEhSrsbjxOzdBl2ulyri74kFWUHeKRtQ/2oQqbTaUiDUCUD52xYKuhNGIVCD+IkRBSbAyxZrXAospaHaXErIvnkpzZPrklgp2Hzt6HeSm2PGBXLHLMaaQwRPCwZgqTVxvI33siGK3cEorccSO9183VD81bvh3j4InyEOBEjthihVjS9OL0QfNMyWiYhlj1oczfVBSQuVKUFyzBE+YN1/NWZ4xLJiiiF05VbHjyT52Yh/JfseCJ4gvm1TMzB0nUNPlMbKv9ev0sVPfs5QUA5Rip6U7GaoAUImUbSZWini6E7JPNdVvYepUFG2KXTS26F1LUGyOlXAN88ogWEsL+M6dMWLHmN4Z4xzcqdhF/RnHVJszPdZaHru4Whzw0AToMuvaXmTEchp9PRzQXJTyms2k2CnJjvrG2SAVu6i9eMERJto3HzELL27pw5sOmwVGfvuJxM6q2LnTKQkkBYGF60nwhLSfh3/cCYq51Wc5x9ghJ3Y5xhTSLCHfatWbs+lrJ244HYTY9XQ0OVInQNs27mMX3QwpuWlQsaMRdTSP3e6hGqr1IBYEwpieUiQ0u4afaZZ3W4ydrlpQ5UknbXJ5wGNqnpmXyhDstJxJ/ZWadlxNYlcLkokdJePiOFVMU6xQ7Mj+JT0X6HxseQgFrJUngpQ8dkStoUqs1cducAiAOlbKxGonJmI+sh0ZnBHFrjB1CnyLj11dEDuLOmWL0HSBDVXgtbQg2LkzNleT0HqcIzB87Gx1Xl3m01hULMxrj2ltfTCnYqdHzuokbyRNsVl97ESbuvyc7GPnm3nspF9xuM0hszpxw8WhfygNUqrVHQcE+u9dzDspnZKci02xM17w4u4hTOvfVN45t78o5xg75KbYHGMKmaBY+tgpc0vMaVw4GZO7VU97WYtiBIAnXtmBd/7Hw/jjhl0xciVgc0RPuqHbVDR6AzRvdrsGqnLfqJnVd9xEafCEbR70Rk7TjJj+iHJ54I6ctSl2jEGatAFg3Y5BLZ+VSezM3H2Ake6EEDUxXtwUq9qq9gnERJ5nex5CAbtipx8j85FJFUabjx1qNalg1Yf0Y2GrKGGbjy1BMSLSxkol+J2dDsVOmGLD78l57JKI3SC85ihy2PSxg+lzyWNRsfF0J0klxdRnXqvHItupMiUOW5bKE3r1EPsLWiNg5DcsX0aybAf9vpG0TUn62On3AzNhuehLoOo6uNAVUflbsLp6GIqdxf+U3gMLvi2PXbiuWBDBE2bdbfXbSirLmGP0kCt2OcYU4oZQJJFjKopTV6PoTW/RtDb8aWMv3nT4TPzuz9u19m/+9wdQDzieWb8Ls7vDB5lp1aDBCgKJUbEJil0YFasPsGOgGjPFSn8zQvTog0XMx1p5wjJ3QPm3mCax0FdRVzy5QQzoMCYhWLdzQPM122mWSbP42FWJwqCIElHsXKZYNObflOboLlTCesBR9BmqdR6Lio0nKFZEn5E2rKj8OXmlAtbUFJpim9W2wrfO5rdG52gLnkAUlMHKZfgdHVYfO6HYKZNbvG/1XX1mMAjs0JAkdiYJNUvZeZyjXrenOxHgZv8E1AuL12vgnv57o/M2fWlN2BRqsXxPo2KV6g65M5kiO437RtL4KkF3+N2VUBjQj0s1SbHTfOzs/WVV7EyfTdMUK64McY82lXeAx14ac4wtcmKXY0xhVewME6z5tg8AP33v0djWV8G8ya147KUdWntxQ9q0ewgzupqjdXbFzlTQXGgkQTEA7OivxEyg1PQa7g81xSofO9s06PRNEw2NEpXLgnh0sUkMkm7B1XqgTKU8rniGplijpFgtrtjRKN94HjvVPyWCLkilCPY8hBSK2HmoRlGZeqoMk9iJ8fV5eSVl9q9t2YJiT08YFUuJXfTAS093Ep1jzRQbkidWLsPr7LRGxYpgDZtfoXlZ0t+IzwARz8J4EBLTqJ53LCoW8WPJTGIn052o36oreEJbXK8jKIj5CpIbb5sldcqIR8USdwrTlzcJZjR90nWoFM4ohYlUt1KIXS2bYhdYrgtA9+GV/SdYHkQfphVBrBamY6til0FxzzF6yIXTHGMKqdiR7OySiDhMiADQ3lSUUZzUJ4oi9GfTyZUaN/ybNSrWdsOiEXUij52IgBusBrFC3qYplt5E6c3ZNgtX8IQY39z3WsBjD0zz5mtT7G5811GY0lbG9RceqeUaiwef8JgKqicoVg8F6ZtjEjsaFUuIoAs0Ktb1MBMQY4pIvjpP9rGjD3Wavw+kdOKaFafglSs/GFeyioVofsnEzpagGLVIsSsV4bW2omDZnToJPqD9hfPV22qEgyZC5hx8aEgGgdiCJ7RgGllSjPbNtDHCoJf4fAG99Jc9QXF2xU5X6Ui/nJoAh0coqDtFIyk7zByQLj4T/saj+UJXue2KnfpsVnahoMdKBqAl5MmU87EqdupzwVPXv6uk2JARLa2TVeeUc4wi8tOQY0yhEhTTN2fIz+Hf8Lvrhut6Wy76noqKNcmQhSw2HhWr5ineYoWfGlUTanWlxuk+ffEIWNvNGDCDJ8w3Zh4jOpRkmUEU6gEdJwgnLu7BI59YgROX9Gim2Njxsyh2tpJiNHhiqGo3xVLTatILPyMPSFseQlvfeuJrtd4kEdQnkCpSjDGtLnbvXXeBG7dNP1rv8rErR+SyEBElRpoxQbRKZTDGUG4ux7YPqjVt7r523owHt0PN8xARu8g/0FTsqi+/jMqfVqt+wWP+C6bSy5FNZbMnKFZtuWUbvS97v5SsD5PXaVGxKgi0McXO9TIGhERJS9NDiZ3Vx44odg0GT8R97OKEMy0q1uYeIlaXfLePXZ7uZHwhJ3Y5xhSCdKh0J+pmLe5raTcNl2JX9Fh6VCz1V2rQx46aLIQptiUidvRtW7ylm+YuUyUJl6VXnkhS7AoWYse5/lfsJh3GNibNY2c+c2157DQfO+KI7lnmBOjHW2yZKUk0twe/UIiHXIkqwTw+P3N8BqYRWoAEUEQITPNWyZ0bDgDedcx8vGmmj6PXPxmNQc4N8bEDgFJLc2z7oFbT5j70+OPys7n32osKWc44R1AZktG9ZlmzNaeehqGnnlT9cA5W183CZtUIcLePnXYoau4ExUD8Bc6E6VdHl++pCdBeUix9Oz1VkHt8n1FTbLpiR7vJmu7E7WNn8cG0qYSG1SIeqS8Uu0h5N6NikZtixxtyYpdjTCGUlwJRVsTNW5pRU9QcSkAoigUvVr3CHDerYpfkm0JNE83F8HFKk/WqiFebo7LlLTtFMYj72KmHkiz7U48/DM30BWD6XFz751bsTGJH052IIRiEkBDLY0eiYl1qDQWteCBLGDnOmXjICSWYFoyn8xOgpkIzL2IasfOLeok7E6+Z243PHdaMSUO7w/48RblYFPEqxii1tsS2r1d1Yld5QhG7mF+codLJz5yDD1UQVIbkd7kNh2UZl1UqzLHoi1QWlY3XlGJnM8Uq8dDelyuPXRDsea1Y6k7RUOUJow+nKZZEmXIAdfK7tM2Z1ndNInY2f1G7j50xH6spltwHPEYSFKugL4DeW8x0J8pVY7jRyTlGFjmxyzGmkIodcWIW9yyzcoLr5k0JCEXR95QaEDPF6tsCyYpdUh67AWJiFKZYelMW6p3nGbnCWFwdCN/+4+P/+z3P4R03PIShWt3qL6gIcqSO1SjJ4tpfyetIH7Y9pw9wkxin5bGjpk1ZecJZUoyT9u5zYCMUrgeJL4ldeIszyXC8pBiZL1FYgDix4yaxK4frXYodYwzBoEqR4jcRc6tIdxL1UW6zEzvatUfeYGwvBvIzVeWkKbYSmyuzBXUgXitWpTuJzhuHU7KjZ5rXbQmKqVqbrNjZ1ClALxM3bFMsuXfI+0cmxU59TgqeKGhR1noKE9e1q4KN3C87dI04bkklC+WYKabYgqeIqAqe0F+SzHlxbs9ckGPskEfF5hhTyJJi1qhYXblLM8Waz9WS78m+bM7/YZ9qWdJNKckUO1BRJqumYpzYiRshi43nUOwsY/33HzYAAH72u7VYNK1NW1cnxEtlh7cRu2hfvPjD1cYmaWCHSYwDbispRn3s1D67asXaS4rFphEDVfhc7UVWf0nsYn6JRp/RXwamExcA9R079G2N92EvIn4MCaazoUHVnip8lXC5V24CYFfsgppO7GgOOvO0OYkdD4md2ID62EliR/uxKnbQ2tE0NbE508WkVqzYmJ4306fWhKbYUVNssOcmQE2xE8uyKHaG0u/aQo+K5UQ1d79IhuPzxATFdsUuHjyRdH3IZZ6+L+aLslhddERGUEU/U6qYHHsduWKXY0xhRsVSh2izZqzbFBv+NVWYos9i6p85Ln2DTawVaw2eCJf1V6KI2IJnDV4QoIEE4ru5T1QxsmHz7iGrKVbsXslG7KKP8ZJiZP8sY6p0J3HFzm6KVd+pAubyzaEVDLIkh9VMWikPEqnYFQSxM1UGO9EPj7/+YOMkIXF50aK4KTZBsRPTK++/v9oPn3i/CQUt8q0rd6iybnJuhinWQwKxoz6jZBufB+DVKvhgSCS1yFxxrgxTLK2KEfatXzeUkMfmTD6HUbFRH6afHhr1sdOX76kJUHt5kXPMsp36zBwqu5gXTZ+TlJyY9gckm2LpgZdq/Qgodp7Fx06sdd0fuXbscmI3HpATuxxjippB7AJbmo4UfyoaxUiRFBVrrzyRTBLiy8O/AxGxKxdUAe2K5W07VraJxcdMSp0AhCWHbCRVOlBbSFQ8KjZcToexDamCCCym7CBO9mzpTug+x33sCIHKoLxQE2nag0SmO/F1s5KaH6zfqV+SeT15ra2Y/a9fgzd9hr4f5Xgkq4DYn6aDDsKcb3wdC3/5C/iU2AmiFSUOLrW3x/qoU8ULOmGT6Tp27sTAU08lmmIBkHQnQWxdrL0j3Qn1QXRXiyAkvx73sQPiSnuWnHhmIIVKLG3dNBUysr1RHzuq2Hl2lR0QPnbhZw4ufexsgRMCrryPFLpip+ahz9F2f7GMRxaG6U6i82v45Lrug9SMnQt24wM5scsxpjCjOamy4krTYSKLj13c+V/fFkjy4bPPXZpiq4LY+UqxsyQXDf3N9O1tPnZJD5bdg7V4VCyPR8VWLBGqWfLY6fNVx9Wq2BlkqWpLdwI4j4mvnnhSbU16MIhVHHbFlUI8nGymabFPFMoMR03ExvU0by5K8+aBtejmUs/30PnmN6PQrpvI6ZwBoO0Nb0B5//3BCorY8aGBsI/msM+mDguxq9U0ZUwnZSHWXvlBvHjeW1Bds4a00xU4bc5Womi0N0yxggSpQ55QUox+0fLYxV+kxDoHr9OuvXhUbHYyZoMkxryxKha0iZdgii14nmbaFy4BiVVupOrv0kNNHzu3Ymeq8tZ8nLQNMR1LxY6JfXEodkj/PeYYXeTELseYop5gmhD3cGXGsffhUlhoVGyWyhNpwRkmfIOMlgueM7WHmKdu9rC8UXvJMbE2xY7m4LIRGRmEYryBm7VibfMV28dKstl87CzBEyCKXTxBcdQWJKI54anKCBFIe6DHEhSbplijPc2jJ3qUD8yeHgBA17nnWfvyGMPMaz6H5sWLY/Ow+jQVqCk2ilJtCn3syp1xYhdUa6Bp52jZMc9jqG7YgL4HHgAAVJ79k1pHo2KNPTbz2IXL9PZm5QnzhcBFxACgzhm+c+Bp+M2MQ8CJ4qi5dYr9MxR6E2buOro8LTo6DaosYLYAHgGW8juW64zrSZpiEwiQIlbZKk+IeZv3UHNeBYesqbmjkN+rymOXrNhRBT1PdzI+kAdP5BhTKKf/+A3BJGWumwZ1TqYo+el57NJUKyA74QtNseFnl4+dafq157Fz3xx7B6sWfzFCQmTkWjz1SMwUqz1kbW/ySmkwjy1NUFzyPVTqAaokxQtV4FwKBFVsaLCFC5RQmAXKTZhRsVWHj6Wcr3gwkeAJ0WT+929G/+9+h443vjFabu6Hvj/6pOOLvIKvJK2ByBQrfOw62wFEkbI8AGce6jXTFKsrdrv/93/l9/r69QC6VTum2unTiptpTR87GHnsTFMsh5uMPV/uxqOLT8H0vq04rf6w1XQuggTEuXcRRXqutKjYID2IJg22kmJZ+qKnmuaqMxEqduFnzpVLQJKPnXwRSigpxg2CG44VV+w0C4FjSNNqYfrYie8uYpjl95hjdJErdhMcvUM1/MW//h++dtezYzK+cvy1KXamKdbeBzV1aJndPRoVq29jy3/lutemmYAFwuCJeB45OU/EH2xZ050I7B6syUoWAvWAx4JQkqJizQc04FLs1EPPmscuWtRUtI0JOYaLGIuHHAdVZROUjOhvSDTDz6klxWS6k3hSVQo9eEKNAwDFWbPQedZZYJ7yA6WwBaOYc9aWFdT7tAhmEKbYcmenXFeMfNzMBMWaLxxj2HX7r+T3+tq1qh0hgKZi51vMtMxQ+FjNnu5E7FQQBAginz0TA14Y+TvkF60JisO5R/2kKnbqs2mKFQRnZEqKRfPKkO+Etkjyi6VlAzmIhSLRxy78a77AUdhMsWm1YtNSA5nzlYpdtLcuYjgS5yHHyCIndhMcT67diSfX7sKtj61Nb7wXIEuKWYpkmnnssvjYDVbVw6hY8EitWDOpZvjXNEPY+7fP3byJlYu+JId2UywzyvfYnJvdObGAkIjbfezCz6JeJFXHYgTZiG40P8u5kJJp9qhYvYyaK4+d62Y/GBGHUHF0z8OcJAePmYlMHDKrE6WCh8XTQ9NmY8ET6noCwn3dtEulKzGft2IKdj/F+Nw8nxhKpI9dqNgVurvgR4SuyAWxq+s+dsSUG+zYjoHf/U5+r69Tv2OmkUFjXpTYORU70xQb9SUUpfUbUN26Lb6DACosKq3HfHCSrsUWPCGIslOxM1Q6uZy4CAy78oQ0xUKypWxRseR37CXksfMZeSHhmXzssgRP2EqKFWIlxUyTsYPYxRS78Lt4GRL3AadiB+VKkZtixwdyYjfB4YoaHS24ClgD6kavCIm9D+pjRyMvSz4j+6dvYyMGzsi2lKhYgbJP0p1YzCieh5gp1uw6KScWECp2MX8xYkZKIsjmw1V7yCaYYl1RsYIsidx9ug+dMq26CLOoHRvOPd0US5U05SNpb/uP5xyC33/yFCycGqYPMf2V4iXF1Pg0+hYA3n3Tb/G6z92F3720HUA8J6ItGEXN2XJci1Sxi4hdZIr1OzpQDEITaDFS3Or1QCmKPECBRM7WIiIn/AD59u1yHfXF0xSm9nZrAIZZqcIsKSaubdE+GByMJWsWCFjk2+h5UVSsUkTlGMa2WfLYmdGge2qKVfcOGjyRQbHT9iOpf6abYrModiIAKtEUK/6q42Hep8z7izvQSG9Df/ein/CvfS5Bboodd8iJ3QTHWBM7qdhZntCNRsVyQ7Gjptl45QlBDPS3VRtcyzub9YoE5aKX+LZN65ACdn86ZlHxKKyKHanlajdpi7/6caRkLqlkmjWPHXkQNsukzOShK5zaPeZUCgQRpIpdsilWmbTSasUyxtBaLsjzax4z83LXxjcebPf+aTMA4PsPvRTN1zTFuudum51PFDcWVaQQip3f0YGz19yPY9Y9gVn1PgBAfWAQux/8bdiOc/htKvpWmEv9KZNR6OnRzanUFEuJSGurPWKW7BZDXLEzTc68HoCnmC3rzNNMsXqNVV0Zdd2B6OE2y4uNlCm2UR870y/NRQZpJQeOrD524d8kU6wt2j+tVmwmUyyjPnb6PYwxZiWknJ6HXLEbF8iJ3QRH3XKDGN3xw7/2qFhBysLvzjx2pD0lduGNX32msJXAcT0cXDft6Z1N2vcySVBs9bEzzJIuU2zSszKMirX42AnFzkKQXTV304MnSP+Bvk29riL8ZLUNS+UJIEGxq1ESLmyh1qba2HoG/+QHiVkeSSCmDhFTMFVxKNqbQr8xW1QsnZ9tfG0Z9bEbCBU7kceOtbTg4ufuxCce/g786FzuvPNOvPLRjwIIEw37bSSJcS3KS1csobRwoU7YSB46+uvyymV7MXia+NgSPCF+osqEGjgVOzk95oPX4wmKgexRsWYkrPwcpJvk00CjYocb2WlT3mn/9P6UzcdOXLPpih29FM3KE2at2ORKF2q+Ko9dfL2tj6wvZjlGDzmxm+AQSlbS2+FojJ8lKjYtQTE3TLE0ia7NlAjETaM2uIhJW7mAtrJ6SJcLvrN8lug/NXjCSzZH1gMeM9FQtcGm2IlErkkBCkmkJOBUWVUVQsSyZksZNd3Hzr4vwhQbRsW65ybnQ/rO+iBRKWmSgydoFC91dqfoaA7PtflTUaaqbMeVlhSTptgoeIIxhq63nIeWZctQiFKgcLBQ+UKowhXaSEqUSPFjxSJKC+brfnIRaRT7JZeXSvDISfE4ByuX47VijcoTzNhPHijFzpY+BQCCyBSb6GNnBPfE+nARO56eqDoNNCpWXYPp25kmZdcmBV8vUdeYj12SYif+Jil2THthyxo8IZqJudLNbIQ0Lyk2/pATuwkOl6lytMe31SFsPI9d3BQrbnzxgIPwr26KtfefdBOe1qGqDpRS8thRNQiI+9yF+5IcPAEA2/r0SESaz8t0oA7X6+YsGxFJS1wqjl+JVAgRKlh7U0h4dg8pIkCJmqkkCAgSzkGjaN2Qih1AEqImbED2IR484TLNKrLNOZdVRQCgI1LszN+KCiqwzNk2J5rHbkD3sQOAGZ/+NOZ9+yZ4IgqXMemz5vMAPk2EHBFDViqhPH++YYolnymhKpfhaweOw2tri9WKNX3sYj5xQQAeLfK5/f4RMA+BIyqW1grWP+jgnKjO1BRLlGqXr2UaaARoI5Un9Be0ZDcRj1y3tQyVJ0RXiSXFoL+sATYfOyMtS4PBEzbSbOuD3mdzXjc+kBO7CQ5xoxwrxU4oajZTrIqKTTa30AeEqdgJocZZK9anN2hX/+75U3OsVlLMFjzBstSKTS4pBgBbeoe07/ShVHKYtOmbvZijboqNg85VmH+Fslojaujs7lBtWrdDKUS09quLfAkSTh/cSfveSIJiARfRjlliyfhSqQyAbf0V2aYcpXWJ57ETc7ApdpZlBaXYSWLX3BxrJ01izFDsaNmxAUXsWl7/evgOHzstSKhclqRRzNpra41HypqVJ+R1E1fsfK63pdCCPwylC1DH3nUL6q/UcNq19+HvfvKEVuliJCpPSMWO5MTLkO0kRphcwxc8GjzBM9WKTao3LfCnjb14/TV34TurXpTLTKtHzELgCj5LCeii312K3Z6ehxwji5zYTXC4TJWjNr5Q7BxOuUD626BLsaMO/ubDWNw06Y3KNEmo/pMUO53YpVWeSHvTT3r7F9japxM7zRRrVey49tBkwlfKGNcEJQNCaZA1fYn5Zd7kkNht2DmoCDQxxbr2R5BwLUdbFlMslOKaSuyEv1IseMKu2NGHYcA5tvcpYid+I6a/pi3K2FxHUWhSKq8wxbLmllg7SXyIYudxjgLxseMD/eH2xSKaFi/Ggu/frLZ3BE+wUlFT7DwehAEVZlRszHk++ht9DwKOQKi/DrUNCF8KbOqP7MdiVqRYs7kPf9rYi/99eoPhb0fT5GQnFINPP41qlO9PKnYprgomaItwbIca5nkq6Ie4NOypKfalbf3YuGsIdz2zSRtLmyPTyVyWdE4Fa+oWel+0UwYaLJVj7JETuwmOMVfsEm504iYub7jOm4YyHQxWSZJcoirZEuya4zo/J9zopxNiFyYoDj/bgidCM5/eb/ztOItiV9G+0weczaRNM/SLMYD0VC90blVRZaIgEv4q9WFGZxMKHkMt4NgY5XsToyUlKO5pL8u2Wep0Up+/rNGQrghD8+yo+VLfL2A7UezMvIpqDKb9tc1Za0+IHUSC4pYExQ6eJFB+UIffSkggUewAoNjdpcZG/JwDgFcqw/eJORiA39oWy2Nnzlwca6uPXeBW7Gq1wGrmVNGiyYpdpRKahGsBj5li1e/YObyG6tq1eOHc8/DcySu0faLBE41ExapjYm/ne9SFQOWx29OSYgJV0sZaecJi+o7PUW9je9lUbePbNxpRnGPvIyd2Exy2sPmxGJ+WslHr9DZpih0H1yIt6Vu4qQbYfPvoQ7jgpd8QAdMUS4InrKZYxEyxtptomvqw1WKKVVGxtnyA3PCx0/+an9WyuGJXIsETMvCl4GFGV3gc1kbmWErUTPJ1/KKpOHHxVFxz7qHR/NzJaSmkYsdpNGTyNjLdicUUq/nZkfnS62kbUewEOYz72EUPd8vd1DY9r9xE1kfXdlNTvJ1wMdBMsRx+iyJ2UrGLiJ324uBQQlm5rAVPMC587AxTrFlf1jDh84DLqFhX8AQQHjelrNH+wr+Cm5h+jwLV6Dddq+suBbTiQVYT4MBTT2nfqdrfUF9RE5tbA0VYUiwi6EGDil0t/UdBLQO2PHaue5ptPNGH2Yzumy04S/exy5ndeEBO7CY4xH3BNC+N3vjRDcHyphgYConTx47cOIequo+d7TOgHtIuZS6rYmeaYn1Z+cERFWuYeK21Yp2jhdjaZyh2gSJutps3jQ4W40aD0ZGt8xWQwRMFEjxBUjfM6goVp7XbQ2LHyTkzz+vxB0zBje96HWZ1C5UqmxmMrnKVUYpvo1Q+sw96ydtMsZwDO/pVoIrp8ykgpmA7c7Zr1ieKnTBhMiuxC4+1ZopFAK+FKnbKFGvun9PHrlTSFDsGwGvVfew8HsTMq55hwuec+tglEbvA+nKmlFFdmTch/FXpC4zYrlFTLB/SfztSFSWqU5aeTJLrjKj3mPaiYLo02PsO/yYHT4SgQUH2PHbxOdvmSPuwBXTZ2gqMRHRyjpFFTuwmOKTfEHHAH03IN1gLATDVxEwJioliRwMY4kl9FSkRoPdaXVlzz5+aYstFZYrNFBVrfBf7kmbOoGQD0NORuPIB0gei6StFl5nzNSEeSKFZTKVumCmIXaTYUdOm+cBRFQwUgcpmioVqn5Lb0BxLgM6FXhEy2QWjZDCu2HHDX5HOIesxjJH55mZZh1brV5gJwVCP1hcKPvwSSZfSryt2mlmNUzWHjFcuWRQ7I2kxdFOubT8Dotj5CWbDep1rRN/sT8DlYyd+S9Ug0CrI1DUFzDm8Bj6kSsNxzuVLG03ynanyRPRXXstOxU6lHAm4/YXShFTsMlhRaLUXk5DFasVmiIq1vWzSPqwJipFHxY435MRugsN0Rh5taKZYz1wX/k3Lak59oqhiR8mcaT6z5ZNyZWlPuglTU6zv0eCJ+MH0mFEr1uHPS9sypQAAl0pJREFU0miCVBokUnKlOyEPRJv5KM0UKyAi7wIS4eczhtkmsaOmTQe5Ur5Huk+eCyq/HCcljJKPlblf9FxSIkEVQ49cT6aPnY17iOvW6mNnmZOuqnFrRGzYb7S/zEMQRdKWZ8zQ+pQJjiPFTjO5aiotCZYol/XqF+ChYkdNsZzHHBFN30yaf1AodkVLdGwtsEdN0iAVwF15okpqClOT+nBMscEQcWOo10lUbGN+YqZS57puPRoxy1V0+Z4mKBYQpNcdiKW+u0uK6fe6WDNmbysQvmhlU9BzjA7GlNhdc801OOqoo9De3o6enh6cc845WL16tbUt5xxnnHEGGGO49dZb5fKbbroJojST+X/Tpk3WvgBg/vz5sfaf//znR3oXxz2oiTKLs+7eGt+Wvy1uirX3QU06NCqW3hhtRewB3WckSyCFiSltyqy2e7CamKDYJDmmggcIsucczgrNFJtBsbPmsbM8mGz7LRMUB8rHruAzaVY1TbEmmQ37jcyK5Lw1UoBd9+lJbhsnler42EyxDDRak2M7UUdpihcKlbjXMr7lotWXJRE7ZYpFSxgJW/A9fXtDsdNIIw2eoNddsQTPMMX6bbbgCX1f5QuBmLnFx85mkqWE2HaM0qJiqfJOFaogaNwESE2xvFq1RsVmebHyjHOepNhpptgsil2C6m9C+OFZ05R4uvrmNMVqbWwEMUWxe5WYYq+++urYM3/JkiVy/QknnBBbf/nllyf2yTnHpz71KcyYMQPNzc1YsWIFnn322b29K6kopDfZe7j33nuxcuVKHHXUUajVarjqqqtw6qmn4umnn0Zra6vW9tprr7X+4N761rfi9NNP15ZdfPHFGBwcRE9UGNuFz3zmM7j00kvl93aaH2qCgN5MxyKAIpCExG2KzVwrFtBMsbbapQJWHzvH56QbPW23o78qk/XaiZ1R4sdp9tBvtGmnJS3diemblNUUa3v2CB+7ekAfUh5mdYV+X8oUqxQ78yGmuKdQfhqNiuWZFQLzmnErdhaFMabYBVbyoR7uFsXOMj26yOPcGhEbztUDECAAQxDVhzVzpvG+sJ4sK8UVO0qyNGJXLsMv6IZoU7HzwDWiB0ClySG/N9PHruAgdvYExdHoKT529Lek+dDy7MqtAK8oxY7XappiJ4Zv5MVKHFfXPSLsX1y38bRB1j7ly2H6/Vi8jJskDtGojSh2ouZt3D0kuQ/qCjKOeR0A4OCDD8add94pvxcKOgW69NJL8ZnPfEZ+b2mJpyGi+MIXvoCvfe1r+Pa3v40FCxbgk5/8JE477TQ8/fTTaLL4zY4WxpTY3X777dr3m266CT09PXj00Udx/PHHy+WPPfYYvvSlL+GRRx7BjBkztG2am5vRTN54N2/ejLvvvhs33HBD6vjt7e2YPn36Hu7FqxtJAQajOb4tzYfMsSfeBl0JNolpiN74q0mKnSUDvMsfJWtW+85m5ftkN8Wapt9s5pMg5QavRcU6o9bCz4yph5BuirWRkmTFjvpHUsWORuEyxFUzodhJUywx6SWRaKkUcXpNpBE7/XuSCSxszzQCqRO7+AuC2Cacu3vOWntKsgBrDjvaLlTsWuRYumIXEbtiPCpWC56gY5ZK8Ao1pZJyQezIvLkl3Ymp2DGmfOwSiF3VQdyVKRbRX5ePnVquKXYNKLdym0HiY1eryZeMuvbik96ZaCOOiWt8V4LipBcS0XcWxU6omTZ/3ZiPnZN86uttBFHA7mNHKoCMc1NsoVBIfOa3tLRk5gScc1x77bX4xCc+gbPPPhsA8J3vfAfTpk3DrbfeivPPP39E5jwcjCsfu507dwIAJk2aJJf19/fjbW97G6677rpMB/w73/kOWlpa8Ja3vCW17ec//3lMnjwZRxxxBL74xS+iZtRGpAgqFdR7e9X/6E351Y7xQuxoKRsB08fOrdip9lSxqxn7RoND0hQ7V+JiG268+Cice8QsvOe4BbLtkKPyhOljZAueSCNcJmhUmq3mrksx0cyvjmHM+dF0J/QYzoh8DQeqdWzvr2pmLfNhIH3sou8c5MHv3EvdJy8toEbNP5tiR8enJmKaoNit2DHtrz7n5PkxHrhNsSIZNGNARP7C2qOqjQqeEIod2Z6a32mliXJJ87EDOFhTk74tAmfwhFTaSAtReaJg8bHTc8TppBZoTLGjZlnqgpA1MS4n9XM1U2yDPnamSuvahPrRhtdtuo+d+AlniopNcGUxXxxdL8a+cV6TTLFWxY4EnoyFKbavUsPuwar8T1NemXj22Wcxc+ZMLFy4EBdeeCFeeuklbf33vvc9TJkyBYcccgg+/vGPoz/6fdnwwgsvYMOGDVixYoVc1tnZiWXLlmHVqlV7vmN7gDFV7CiCIMAHPvABHHPMMTjkkEPk8iuvvBJHH320ZMRpuOGGG/C2t71NU/FseP/734/XvOY1mDRpEh544AF8/OMfx/r16/HlL3/Z2n7rN76JLdddJ79vqFas7V5tCCxkZyzG91k8ka3ysUu+4VKFhSYoNnPJBVzdNFUUqa6gCWStPAEAJy7pwYlLerTtrKZYo19bKhDGdMKV5Q04jOgT+5PsY0e7y0IgPca0a6QoTLF13ceuqehjcmsJW/sq2LhrUFNoYg8K03zFs0UkyuPC9esmCUnEjhIJ+ZnMN+B6STG3j53c1LnOtYwBqT52AfPAIx+72HmqhPOTeezodeswxYbBE3Ugch9kPFLxKPnjcVOsSndCjqGoYSsIiyVRsasOa2bFjtyXKkbJwKwEX25DX8hrNXgsPG5BwImSnUWxC//6hnnaRBgVG6LhqNgMplgVPGEjdvpLQKoploVVOXyvQ2+QoY86+b2PNk7519/CK6v8hH978gG48pRFsXbLli3DTTfdhMWLF2P9+vX4h3/4Bxx33HF48skn0d7ejre97W2YN28eZs6ciSeeeAJ/93d/h9WrV+NnP/uZddwNGzYAAKZNm6YtnzZtmlw3Vhg3xG7lypV48skncf/998tlP//5z3H33Xfj97//faY+Vq1ahWeeeQb/+Z//mdr2gx/8oPy8dOlSlEol/PVf/zWuueYalMvlWPvJf30ZJr3rYvm9Ze1agDhevlpB+cdYlBWTb5xenLgpH7vwu+vmKRYHAdeUsqphN6sHXN6YUqNiGyB2FOqm7PKx000jZtdhrrt4f0CollUs/dLyXu5asWoOdD7yc+L+qOvCpdgBYR4/INx3xZPihN2m2GXxb6JO6FnVlbh/X7JiR1XUgUpde1GoBxy2dG3Jip27PRASKJePnQyeAANvbgGCuLItAiRSo2IpaSuV4BcUYWXg8EolMBJQ4VlMsSr/YfgnoPMQLxY2YgdmDYCiJsrwb2xTAEmm2MYrHgS9itjxWk3WiqYvfdkUO/2cu24RvueRqFiVxy5LVOzQlq0AfGc7QB2b0MUi3o+NSMfmKPZhYAAvvOUvEVz/E+t8wv2xKHbkPKS9aO0N3PG+ozBz5iz5XfgBmzjjjDPk56VLl2LZsmWYN28efvSjH+GSSy7BZZddJtcfeuihmDFjBk4++WSsWbMG++23397bgb2AcWGKveKKK/DLX/4Sv/71rzF79my5/O6778aaNWvQ1dWFQqEgHR3PO+88nHDCCbF+vvWtb+Hwww/HkUce2fAcli1bhlqthhdffNG63iuV4Le1qf9GcMerFWOu2BFyYBK3gIuIqzRTrDJ16FGx+v5IokhMQ+6oWPvyNCTlsYuZRmxqFmOaIkKH7mopwgZK3FzOzfI4O8ic635smm9KhbCh6WMHKLWwquUti+cYM0tT0bQVSc8FSeBHyMeOXu5EsJPH30wE7VTsxFjWyhMWsqd95mAOxc4nplgemWLN4AlB3jxZeYJsryl2lNiV4RXVOz3jHKxY1HLbiblRqNQeiOZFfiNJpljPs55fM3m0s/KE474UEF/OrL/RoE8ndjQqtpHqCfKcy2PiIE2efn+SSb0TgifE8IO7s7v72BS7UC2nc0lW7LygDgQB+O5dRt/qs6vyRKMm8ZFEa6mA9qai/F8uJJNhga6uLixatAjPPfecdf2yZcsAwLleuIZt3LhRW75x48Yx990fU2LHOccVV1yBW265BXfffTcWLFigrf/Yxz6GJ554Ao899pj8DwBf+cpXcOONN2pte3t7JfMeDh577DF4npcaSbuvYcx97Mibnu2eoN00XOSDqbYasTP2x1YX16yTKJC1pFh8Lm4zCmNMIzm21Cae8eZN59fdUrKOqfkHefEbuCvdRFZTLIU1eMITxC5SQ+u0NqjFHGqoHPQcJ9fdiB6QJCo21ccuQbEzMhRHc1LEKVa6zShpZY6RVm9XLTMUO1fwRKSghcQuJH9mIm+WoNhRHzufJiQ2fOwYIlOsqdiZplipaEWEjObGE79jm2LHfKuPnfrd6sq8iarD1UyPxsxI7Hp75Wde1aNiXaZgK6Lx0hIU+56nFE7OM/nYyTx20YYFlj4vj8VfLEyy51LTRBthTjfr/qa5hlCLwXhOd2Kit7cXa9asiQVkCgi+4Vq/YMECTJ8+HXfddZdctmvXLjz00ENYvnz5iM+3EYypKXblypW4+eabcdttt6G9vV3apTs7O9Hc3Izp06dbme/cuXNjJPCHP/wharUa3v72t8faP/zww7joootw1113YdasWVi1ahUeeughnHjiiWhvb8eqVatw5ZVX4u1vfzu6u7v3zs6OU4w5sZOm2PgbJxClNEi5aVDlxxa0ICAIHd1PLUjCYXLIGhVr9hefp/FQZ3HiIfIn2ebRVPJRLnixfQw08hvmjasT1hI41AibI3t8zg5iR86LIHQiIrdGUkfYTLG+cUB59C/cf8dEyDoO8kKQ8mpqzp8qDnZTrNpmoKI/4Gokd59tXrapp6VAYQC8ZntaBKGgcTDwckjsCr5OfcUz35rHjip2vmGKpYodeLiMqnpG8AQD9UELl1FTrIyKtRC7GiWAlutPjOIiVq6SqY0EPNR37AAKBUOxq2pRsT5nsTm6oEiu/tcE9bHjGX3spLuIIHYA3GF9YnyLj52nX3+uF1RxDMT1wgwXFkoYX82VJz784Q/jrLPOwrx587Bu3Tp8+tOfhu/7uOCCC7BmzRrcfPPNOPPMMzF58mQ88cQTuPLKK3H88cdj6dKlso8lS5bgmmuuwZvf/GYwxvCBD3wA//iP/4gDDjhApjuZOXMmzjnnnLHbUYwxsbv++usBIGZWvfHGG3HxxRc31NcNN9yAc889F11dXbF1/f39WL16NarV0Fu4XC7jBz/4Aa6++moMDQ1hwYIFuPLKKzW/u4mCcWOKtdyYAIOUOO4aykTHMVR1R0QFUrFTNy4aPOGKkG3kLTQtjUFardiQWEBrI+fqMXQ0F7F5t64kUbWBCR+9ur7e5tysDe2Yttle3Njrda78FA31okoUO7D4MYlVnuAqjUj2dCfZlBrzdGjBE2S5JKJE+TD9GanJWx9DJwSMUQUyGUmmWBUV64FHObHMa4YhehiLkmIWNQ/QTbFeuQy/SMz6PFT8WMFU7FQTatySJbKoy4AwxdbjFKTmqceMOPzbf/xj1DdwAK0xX9qsoC4VSb5dQaWCP70+VFAKM4n6Qk2xAUcgCE4GO5YYTaq1TlOsXnvYVsow1ne0SlPsePKVZM8/py9zjSmjYiWxM++hyeSwEQV9LPHKK6/gggsuwNatWzF16lQce+yxePDBBzF16lQMDg7izjvvxLXXXou+vj7MmTMH5513Hj7xiU9ofaxevVpm7wCAj370o+jr68Nll12GHTt24Nhjj8Xtt98+pjnsgDEmdsOpTera5oEHHnBuc8IJJ2jbveY1r8GDDz7Y8Nj7Iqh61ZApYqTGl8pLvKQYYIbS2/ugvjpJit1jL+/A/c9twcVHz5fLqILjDJ5oxBSbotiZWd7N1qGPnf5dzokxtDcVYsRO9w8S+6Tn++Ip5DiLKdZjSn2rBXG/vqI0xdJkr5bKE5IAKcWG+ri5QBUeQQT3JCrWVlKMBriYfpIuHztTtSn5SlW1Tc9UStNMsZwx8KZmAJWYj50wgdpNsUGsL0AETxCyFSl2ejUKvfKE7YVA87ETJN+m2Hmk32jjDZ/8FOonXAl0tRJC19j9h/qOJv3u6tu2qblsUP5QsQTFDfjYmWTemUqEnK+Ac+mikeRjp0yxYZuicextt2nfAyp//KPRj3E9pEXFRueOGdd9GjkMX7TEPMYvsfvBD37gXDdnzhzce++9qX2Y/IMxhs985jNaUuPxgHETFZtjbEAfVGawwWhAFnN3mGJdZkQKeS8xfOxMfPWuZ/HYyztkzjVtWyTksWtEsUtSnFi88oQteMJFMH2PoaMpHkARVmKg2+vr69yRRyyTKVZ9pilpKOkR5Fg8rGpBoKIgEb/ZS78ksYASz8TjJ9pnT4hqPszod3qPtuXRMxWkehBYTbFKqQv/6sQuPj+xSBAvV7oTGTwBBl4uQxA7a/StxRTr06hYzRSrK3aMR8SO+t0ZPnYeOIJKBUFfn5XYiehcP7ApdkQJpNd/tE2aj50L9IUm8SeqnWh13ZolxbLkUhQQ46nfu32rgnG+6pZofBNilTBhF6nyypjj5YJhwyc/BSy/gsxRf1l2JiiOKXb6OWwkKnYcC3YTCuMiKjbH2CEYL4qdwxSb5eZNfewGXZ7WAHYNVqO/4Y0rzAqvK2gCWWvFxuaS5mNH1tvMJ4xBe0Zo/i0+kyXLKPQC5nGfNldkscvkS6Erl+o7VUaFz1xBkj49yjVWr9XX1Q4a3Zj0YFDKR/YSRubDjJq66du3TbEzUSPRvvq8BLELv9N0C7aeZP+CbLvSnZCo2HqhKPeH9ikVO0nslELUftyxsb6AMHjCKxoJio10J5qpFoDHgZcveQ+eO+lkGYCg+dhFak/RYoqtasSOXP9cELvwe6P3n6ymWO5IPM+rNXltBgE1x6f/3sVZEE1dm/gezWOnUgRlCZ6oRsSOBk+49pMB4Lt3x/px+etq7TxB7KJryfCxo0PqlXrCv3oFkJzZjQfkxG6Cg6ruY+FjRyMrbfcEHsCqNlFQU0dS1nFRbkz44Zk3Os9B5hq5VyU9YEyzpDVi1HNHsvmRj505R9OJPJbomfiw0VWUIrimzYy5iPnQRLFiGY2Ypf3GyZX+UAxNselqiZhvmMcuGjtNsWPmd1oyTC1PiuIVqKeaYiPFrmA378v9EO3FPjt97IgpVhA743diRsXSMZvmz4/1BQBeqYRCiSh20fY0UtZrKseCJwZXrwYfGEB9/XoA9jx2dlMsCdRgimiJ/gOD4GVFFjUfCJU56/JaVV6btP5wlvc4qdgJUsTi64CICMkXCcQiyW0Q96F69HguaL6Sjm2Yup60ZQ28vIkAGK9et66ncwOUUs959t9jjtFBTuwmOOhbct1WCHNvjm2kHbHdeGigQ3pULBIVO2GmFX/Nt2ZKQBopKab10YBp0GNxMhvzsTPMwx1EsRM+bXWuk1/zODlLijkeRvp89LmI+WjELlomlLhqPUhUEMUDQSylD4ZEtYQ8ILMqBLF0J0Ql5eRBqHz87Gl3AOFX6DbFSh87qthZ+mLGutKcOfa5R0QrgAfe2hbO3/idMEOxo/Ogblw+9Z8rl+HZTLFUsWtq1tOlgMuIUqXY2fLYxX9/NUOxE/VaxdwD4y+zZYG2IKsp1k3sapIo1TXVOINiZ5B5+qs1UyVpeewid5dioo9d+FeYYguIv0TFtgEs6WlMtw77eGnBEy0ldf70fQv/6r9H+xg5Rhc5sZvgoMQpQ2nCEQVVP1x57Kj64wyeIJ8HkxS7iIwI8pdUlUBPd5L9bpVsijUywVvIrMf0h5QZvEB97EQiTs6J47cljx1gv/HSVq6HmelfI27sFc3HLiJ2NN0JecY4jzNV7DI8oBURbIDYGesZg0Oxi683UXekO5G7YyiXLoh2XrGI+T/5CZocFWwKEdHqOPtN4MUo6pUoQECyYqe9RGim2DJ8TbGLTLFEsfOby6DBDIwHgFByBMHTiJ07eKKqBU+oeq2CONa2bQegzoGfkdhlddrnFTuxQ60mf9uNkhNB5KQZk2xj+uqKb6EpNouPXTQncT1pip19OwbAQ9yEmuYfp+2DI91JMzHba4nbCWFtNJ9gjr2LnNhNcOim2NFldhpp8+wP6apm1ksnH0nmHKnYReTPjExz3QQbuVklmmI9/a059LnT29jKjgmYPnZCGaJF1l0+Yqr0kH2/XLM25yKOC1XsxIOhqCUoVmO4gyfUQ1WctiSi5lkeJGkPYaspNvpMfez02rb2vuhxpoj52FESlWCKZQxoPuRg59wlAZw8RY8eT1XsmPYXCBW74syZ8Do64Hd16cETQFR5gip2ZV05JvcGq4+dyGNnqTyhpzthCCJiJ0jpkDDtSnNuRsUuyJZmI0mxo9dmNUNfAuJ3a5J6wKxm46l1xBSbxcdO9kdfgF3kjNkVO9dLIoXI+CTOITP8JJuKLsVOvCCRCiA5sRsXyKNiJzi04InRVuwymGLrJFLXdS/M6rArfAgHHT527rqxmbpPbRua+fQxUhU7TUX0NB+7kiVZsMeYlptPoCaKhZP5ZTHFmnMR8xXEjt7otZJixGfOlXKELm4keILmzWo8KpYQRKrYiTEQzy0oQKttaH0aRKqo+dhZ5mQhXta5R6vp+TWDJ6RiZzPFGurw/J/+BLxSjfLY0QTF0QtBwZdZckJT7HbVhihxvLcXQDdoNVlBxuwJinVTbHVAN8VW14XETqY1ymqKzZig2EnsSPAEoH4jWW4n4izYiIyu3qnvHNkSFJtd+hlMsQxcM52H83BH2Gv9G4qdWXmiSVPs4v1xrqwvefDE+EBO7CY4tHQno63Y0XQKzJ7HrprBx441qDsPVO1lfSgpG66PXbLiZJpW44pOPJJNn5NNsaNmJBrgQFEjxEBAU2QyqKEeU350Q/W4SakYfabpTuw+dkKxU8iSaoIqfGkBNbb5i++KIKrlklR4bpIQ7ldS8ET4t6wpdrb90P+mzZ063ZvmezOPHd1Oj8AGCqSqjk+IoOjOLxSAqDxuobkJ1BRLHeqDvjD6sq6VFEsgdka6Ez7QH44rFbuw4pC8hjObYtOTlwPZgifC8cO/w/Gxo+eEWgJCxU70zzP52Jm/lyKPq+MmPKjjKZcx/Vp2B58JYhddS4Y7S7OL2EmXhvTgjhyji/w0THBQxW60S4plCp6o621saPQt0aXY6W+3yZGNLtjUMgHTLOnKFk8XmVGx7WWLYmfUgrXd/GsWM1MWU6x5I09W7MLP1bqyrTIWVxlUJKFansVHR1m0VHRqWvJoW0SuZ3kgSR8/xK9DW/Sx2afYFgCKBXpc4/MzSYELYtdoLc4wRQ/pS/y1ETVDHabQomLFg91XLw1+S7NWeYL6XfHdevAE46r8mG8xxVaNBMUyeCJaVo9Mu4362IVKpn3/KHgtmylWIJOPnSTP4rtaF8+HqQh6Iz52so9MwRPx2r4xt44UxU6ZYg1i5wyeiPYLeknDHGOPnNhNcFDVbLSJnWaKZXYTGE2E6zQXNjiuKyrWZX5t5GaV1JZZ3qBtzv0u8uUzPd2J7mOniJFVsbOYmbRACufbPGlPgyeit3rTVByOpZtizdqwwzbFkracKJRJMFfT2AN6tXPS3tymXFBBIXYfu/DvEXO7UCp4OGr+JG28eHum75AD1IdJSwtE+7IpdhaH/tgLRJkSwah9pMx4PABratZLktEI4t5QsRM+dgzAQdteRLFexYHb/hzbD+FjJ4lqvx48Ua+EMqG4ZmwBGDbUAz1NjQsuxQ61mvV6k8S/vx/VjZvk8uqGDdjwuc+h8uc/y3MQS7aNuNqvTLE81ceOcw5UK9oyGmnsfLkFLKZY4/7hInZSsYvGCRpT7DjnuSl2nCEndhMc9TFU7DRTrMdgE7v0qNiRVezM4AnfQlLM5WlIexPX051YHriGIqPNyRE8oefzakSxg/WzOWc6F5nupB5X7GTwRGAET5gKhBE8IfYhXOYGLUGWPXgiTt6paUwgKY9dmRDoJB+7kw+chievPg3nHjGbTjq+H465ufoNiA9TmMeOKoIcKBZjLwAAjGvNVOziCp8XlRljnMNratIrTxByYaY7YQDOfc1s3Lrqy1i24enYfghTrDx/g1HwRNRnMFSR+xmOle0+5Eq8HUOCj53txUAseek9l2LNihWoRSXJdvz0p9j+nf/E9u//gJjfk481fWENgnQfu81f/jJ6/+u/tWU0IMVl6mRALCrWMywCrjGFwlyQJcX04+UKnqA+dsokbp9fjtFFfhomOLQ8do1mCN3TsYWjtsU0J5Alj525OCniDFDpTmKKXcyEkjyuDckJiuPmsbiPXfzBQOdkC56gUWmex6wBHMKkrRE72D+b86FzMRMUUwIsogGrdT2/mHmzl8eZ9F23EE8T0hRL8t6lmmKN9R3NRWVC0qJi5Six66lEFDtbrWo6RKng6QEqljlJ020qKY3mFhiKHSXkgJaTLuyXybbmMgGa7kSSE0HswOG16IodJXmBodh5BR8z//mfUfI9a+CDMMXK/YmCJ7zoQNWjdCSN+tiFCaP1fbAha1SsgDhWlRdeAK9WUV27Npxfb5Tmpb9fqZxEtRSg7hi+r/+yxHl0+dht/X/fAjMIWiEYnik2Fjzh2PbY/afiuOklvPGFVWG7uhk8ofsMyv6jj9r9J1fsxgVyYjfBMR4UO9+4SVLUhhEVSxPE2uCMinWYLRqJik0iGqb/lml2BSzfaZJZI3hCmK7qAY1Ks9/ABUHWVpHPWY6t56mI2yFJ7FRbmTA5CEiUaXw+UrEji7MkmmWyrX1+9vnr3ye1ljSTLuRnQkQd11PgUuwSTPp2U2y2uat8e3rUsyRiwifKLP8VdWtWOaHQ0p2YxI5zsOZmp2KHXSGx40KxE+TG961qW53pip1MdxKRS2mKlT522UyxATHJJylF7qjYKmgJNgFpOo2245FPIB8aCv/WakR1Df+6TJ60VmwQBBhYvzHWxoR5DHXFzvFya4mKZUYgkGvbqe1lfP317Vi+4alwgVGCjfrYaUmviaIsx8iJ3bhATuwmOLQ8dvXRJXYyB5XFAVmAljlz57HTv2cldnHFTn12pT5JQ1oaA9NkYzY3iYX5kGgrKWInlEeztJLtBl61KHYus6w+H/2NPx48QRQ7EjyRLY+dgoxITDDGirnUMigYdBvaZHJbSTNxClDFwexSJIJ2+9jZ98+2Llwmxkqcum6KtQVPWHLY0e3o0PEXhvgcRaULj3N4Tc161DT1sYtMqXJ78aFYiKlGgDLFSsIUbe9HRDKoVjUlNKtiZxJeFwRBKy9ahMnvuQTtZ5weLo+CKlwl78R2wWBI6ILKkNzOVEVpF/S+0nvbbdJnrT4wgMr2HbE2JkyCViLELlGxi0XFstjv1wVeVWQuFjzhSFBsu8/keezGB3JiN8ExlqZY9bAKL0O7KTbdl8p8eKZl/ndWnjDMngJp5j6KxHQnhh+hLRI4KakoLekFAAf0hGWmgoCTWrDM+tCoW6LxNPHOMW8tiEQLnoj3pypPBJrPmovY2aNirdPQ1tGsPFlS3dBxprSWtehaAaowJvnYJaU7Mecp+nPPJ/m68ompS093EvVjyWFH5+OqpGJ+V8ETxBTb3KQHT5D9NmctiWShGKtXClBTbERURfBEpNgFlaoeSNVIguIMPnaS2O2/P3o+/GEUe6aFKyJlyvx9y+sjWs8FoYt8AWnQhVQryfb0ZWf3Lbdg6OnQ75AHXKaIsb0A8ohQmeS4KVCky/XiyLgtj53x8jY0YG6mxiaRw2aC4maXj53tpSVnFOMC+WmY4BgPplj5oLJcjYKQpOWHoyh6cZWIQjn+JwVPZHvTNZGo2MFU7OJExiyt5lnI5qOfWIH7PnIipraXAYg6l3FTHYVS7Mh8HJ8pzEhhMf8aUZAEVOUJrkWZOhU7srgRU2wjip3ZZlJrSXNmF0g6fioq1p7HzlQZk9KMUGRV7GgC2LDv6PhZImIBdY508uYeW0bRElMsikUtbJjWb42n1Yj+FgpWqqqiYqPjHpk2vYiQBkEgzbHhPma7DwUkX2JiSbFIjRLHSZiAxXLzGmKMhSQrIlqBaYqtVmPmbpfK7vM6uCjBBpX7z5YWqb59e9iXQY7LJJjBGUCG+HnZ/u1va+d5+9e/jr6HHrZuz2tUsTMqT5SSExRT5Ird+EBO7CY4NMVujNKdJAVP2EyIJmKmMN+e8sNELI+d46bViGKXHDxh+ti5FDvHQyIiopPbypg7uYX47tCSYvYbri1PXJbgCWaQlJjZiip2tPIEMa3GIjI9T64TyFIaSszFrFiSBtMUK75SxY4eP7NLYYp1R8Wa35Mlu6zBE3R/xfEp+ErRlbqfwxTrMrub38VHUY3CAwc41wgG/Wwz+QEhsQMA3yAY1YjYScU1SlAs/fwYQz0iPwCs0fE2VLbvINeNu51Q7AShQzRPQWbi+Sx1osMHhWIniF1NXruVZ/+Ebd/9njuSPQiAoYgYco56pF6aL5UAUNuyJZynYYouB4rYuRW7IEYIt3z1q1r+QY9zDK15zro9EoidK92J7bea+9iND+TEboJjTBW7DMSuJmucJvdF73dFku09CeZbs+utuwFel+jEbZolrXnsPJ0L2BQ7NVZE7Gi6AWZXK0U+QF0NtH8250znYuak0xIUSzUvUHnsLERTOZyrZZRYuSDW6X6X7vZ03gJT2sqaEiZASZ7ZZ7lIo2KT+wcM/zXLfETzRhIU03Q1kkg5fOyEckpr1iaRTxapSCK6liEkM05TrHEMqGIXbq83qBmmWC6iYiPCHIAh6Ou3zi0JlU2btfyNLkhiJ/aPELvq2rWx+TLGtICLICJmysdOmWJrz6/Bpi98wZ3HjgfAkAoOSTLF1jZvBhD3saPELrHyhGmKrde11CUeDxJSvxBTbE03/Ra1ShrJil3O68YHcmI3wUHJXG2MiJ1SGOJtsmRqp32ItlmUnKw+diNligWLkynbA9dVUsw137pWWinNV9GuJmUzxcYVO62kmK8SFAuhwMxjZ+ZhExCmxsTgCdGWViPJcG4GSYmkSa0ljTAJ0HQx5vxkhY86t/oCJpo5bX5IwpSaMm9KQANi+pZKnSDPhin2fScdgL967WwsndPpnIdmphUF7QtKsQuGBg1TrFuxkz1RxY8gnu4k8rGLjitnDLV+RezS0hXJfrdsUabYhohd+Lfy4ot47uQVQJSXT2JwUCM6wrdO/q3V1D2Lc/BKRUvqq7288QC8QhQ7Qexspdc2RcQuwRTrUjM9Uv1DgHGuETmPB5oSSaEFT5DfC1XrgHTf40ZyfubYe8iJ3QQHfbgFo0zsZN6qDIpdemoI9bnge5ke+MlRscnRX1n71Po3FDpXHjvmIAZxX7XwLzXFuhS7Wj3uqzisPHZG37aEpdW6nu7EZeLWFLsGgmTMGsNpoEJGS8lXiXItUbHM0me5SKNiw4bFhJJzSb5tYXt9f1yglSckMffSFbtzjpiFL7zlME1piV1nmnIckY0SUeyGKlrCW5lapVx2zlMQJpdiJw5yMCiIXaScgaFGFLtC+ikFAFT7B9D/zB+jObjbSZImFMXob/9DD4XbGiTr+ZNPRvWll9T2Q3EfO/FzkSSsYjeXhopdtB2AunBDGFT7KyAUO1N504hdQvAEg27G9cDBapTY8cScfrIvohA2GcRO9z2O95ObYscHcmI3wTGeFDvbg86af80Cum3Bs6f8MJGcx04tb+Rmlewsb9aKtSt2Nsd2IE4aNR+slOAJRQzo9nTc9P3xvPgxK5Mbv6o8wbW8cHTursCCxoInCLFrUCGg6U80xY6Yjs0upWJHlNGirx8XirQ0MtJ0mTJ1qiyq4AliynVExcpxLH1Zv0tipkypfGhIU+yEedBrbo4lIZYKolD8DGISNLWE66NgBN4fV+xEpCygXCTSHk4BGIKIkDRkihXRv9FxM/eH9/eh74EH1DgieIKmO4nWScJLyoA5iR0HalFOP69X+RQKOIldTfXtDJ4Qvx8jepmZil3VodhRAkhIXnNJPwvUfcXmJ5gLduMDObGb4KCp6+oZ0wyMFEzFzloxIYNTfbhefS748fxwNiRHxVK/kvS+bH2YMNUgW845xnT1zJU0mX4PuK542VRDm/Kpm2Vdip0+vjkHPRWCCJ6IVwsR+0nnRnuilR9ckIQsYzkxEyKDvuiHPj6p4hlX7Gi6k3BZwXcrdi5VVC6Trgdp17Qg7mqffV/5j7qiYs1xbGPZCHZ5TlgKzS+WUF6y2PCxC89pfceOhHQndlNsXRK7kDAIosSIjx01xYqKFMWU311h3nwEiL8wmDCJnRk8ESOqHGBNzWp7kcdOpDupKlOsOAecRPVq5kpC7AKoMmxe3+7YPKWPnVF5oly3k0Z9zpEiHx17uU8Vk9i56+baPpumWPNFT5sDS1ehc4wOcmI3wUHNr/XR5XVyvGym2OS+zCCD4fjYUTVnuCXF0hS7tATF5jIzQTGFyvyul/SxmmJTasU6FTtDuUwkdrLyBCkpZsyVBl/YHgJJp03MPa3epguTWkpaPzQpLjfmS2GrFZto5kzhyyxhna0fTkyxPmPERy9ZsXMpv+acBZEqtIQErNDRhs6zzkLPlR8gcw7HmvI3fwMzgkQSzSJJl0JQKzeFy0XCXxEV64fXDmcM9QGq2GUjdt6kSaqsWRKxqwliFx4nQUCFI6gZxeuBIyBRulKpE8odCZ6QyZQ1xY68FPJAmnDBlSkWfYZfH9zBE6VaBlOsuBYM5U5TEpN87Kgplnw2TbEF7YXX/UKTY2yRE7sJDj0qdnSZnVnIPbPTvwU6sfMy3WRsuaRsBdQbIRCJip1nKiU2UsC0J76ZoFjrL/oaM8Um+NhpIh3ty+ljZyh2xnxpuSEaPCGfTUxtC+j7bxuxkTf+RtWBrojYia30qFjVZzyPndpHoUaWfP08UqRXnmhMsQu4SncS5hKEth9OxQ4Z5yhMsWRezPfRfe6b1fbRweo852zM/db/s+4PCnZiVxeEqloB55xExQofO+g+dpLYJR+fwPdlWbOkpq7gCbVvpmLHERAFMYilO6nG/RxpHj76MhQEYCJ4Aip4gpkBG6DpTgxiVx1Uc9m61bqP0lRummSz+thV3cRu6w03YOt/3BjtWzYVOMfYIid2ExxjWXnCNMXa7guKkCTfNOjagp9NsbOZLD0LAWnkTTTdFEu+W0iEGSmrmZidpljlg8WY3ak5TbFz7aJZksjcvyZLVvqqVitWkAV9zq4xk4602b7RB0l3azGaizhuap0qgRY3MZVJiTpRcaNYcJti9VyBccjgiZT50nkqldKTPn/FyOlf5mczoCuySaqiTo4kaaHEUJj6WlvRNHeOPo74GxEmkyjVRFBFvY6hZ59VtWKdil24PE2x454nTbFm7rXKK2vBoxfVuI9dCrEzFbtBke5ERcUyua3NFGsodhExrIOp+rq9u/R94ZyYYvX7cFN1SH6uPv0UbBD56oRyJ/0vK6Zilx48Qclgkwds+uK/YNMXv4igv98aLCXnkPO6cYOc2E1wjKfgiWTFLrkvuiktvJ0EP8H5d6+UFDPUtLCkmN6GEVObaOOaLyV20kfMEThiy2On+2C55qy3TzLFaulOjKoiQoXRfOysptgkYmwntlnRLRS7aDNhfqUmWWaZA609XJVVS5KPnXIviK+TpsuU6YvTTWsB+x6wYEorVp64Hy594e6wH1fwhOMFIeybnAdhQhWEjsW3kYShpSV2HiQR9EWtWZ0oiSTjjHP0/eYBVXkiGjdgHuokeMIvCGKXfIDqzJOmWOHHBgA7b7sNa1aswMZ//EcA7uAJuW+maRnQiF0wNBQqjUMqj108KlaNH0t3EkXV1ki9LbZ7pzZm0NsrCWRMsaPBE446ujKII8EUm+Rj5wqeaBL+fpGKmVSRJzfFjh/kxG6CQzPF1keZ2MXSncTbZE134hkEKCn9gYBNsZO+YAk3sCQkkQ2zckOozsUJwnCiYgU58Zm96oaZDBowTLGOfdRNsfH9ayKyijBtV+uBtG2aZkeXOdn1fbhtbVDETlfs6HPU5veoKXb1dB+7sB/3OkWgUq5pYoqtE8WOMYaPnLYEJ+8MqwhkCZ6wmvwjlGfMABDm+KN/NbWWc8DzwJqanOdB+djp5EOU8GMI0Hf//eCROid97ACp2DEeyMTFNlcJCg4GLlKpDCjT6aavXAsA2H7z98N2gtgZ6U4EqE+bIEiaYjc0FAYUCAWwpkyxKkiBKHY+PW4BMCiInXoJYrt0xa62cWNsDgBQqNeMJMP2e7TnCJ6gSqLHOeCKiqWEjyp2UKlggsHBxIo8eQ678YNCepMc+zLqY2iKFURSEBarb1gQV5psoA+q4jBLitE56EpZaleqbZLixGCkG2HaQ9Jmkk6MiiUERUV12vdLKJ9JD3sbTB87k2DbomJrgV4rNtxWtDGIHYCsV11Svrgs6G4RptjwuyDD1B0hdHHU+y1bFLtiQlSsWsbtfoSyTfJ8aU1bvVZstD4idF6mdCf2awdQRGfBlFbc9K6jMH9ya2x7xnmo1iWprII4Gcq/OGaMA32/+Y3arqjy2Clix4li5wGIJ/IVqAccXJwHQuz8ri7UNmwgE7CXFJPtLXVwqY8dHxxUEbEAUFWmWKmuabVudWJXJ2qihEHsqhsosVPHrxRUtYTBLsWOmYqdNMWqsT1ki4qlPnZlTohdfz8KXpP8Hq+xa+86x+gjV+wmOMa0pJislqCrOhSyVmzKU9CMJM1iPrUpdgfN6EBHUwEzu1S6g0ac9JOUQmZR6DwL0XLXijWJTfg3oFGoLDl4QjevkXk7FTu9jXnM9OAJJscSpEn61lmUUDFfbbwkH0VjVVbTz3EHTAEA/OVrQ98wYUoUVzu96lnM0BiSODGU9LFLCJ6gc0tU7FK87GjUMw2ekP1EhG44pliXf+UJi3swf0prfHsewIuiZl3nyOVjJ3/DxhuSZ/GxY4BU7Eopb1QB5wikYqcCDPyuLq0dr6QET5D0IpJMm6ZYQpB4tapM02JfCXnTFLugDjaozMxyHMMUW9sYEVHP09LMFIO65j/oJHbCZcUIokDFMMVmqjyhyF+RqIV8YMBQ7PQ+clPs+EGu2E1wBGNI7OTDSprr4m3qUrFLu2lQxS5b5Qmz7ikAfPc9yzBUC/DKdqIAjKQp1ogqs5liXQ9dFymqa+lOHHnshhk8kZbHTgue8EUeO07a6WpsbB+M8ZKOtLku64Pkpne9Dn2VGjqaogd7tJkgw1SoZpaSbL4XEtpqnSsfuxTFLikgyGfudRRKWaRuC8QE7AgGsM0rsTqGY3zNBxNcErvYeRBRuo4ExdKM3KTUHlYuyzQrHAy1gUGgNVIGC0UAAYopJSgCzqUpllHFrrNTfuacSzJj1oqV89cqmUR9G4odJ8SN12pSBdR87KQ6TY4b5zJ4Qu47D8B37tCWVdeHxK44e7au2NWrmoLmMsVKxU4odRbfv8yVJ4hPHyV2wcAgCl30ugndFmRZt9wUO26QK3YTHJopdjwrdhkfgoDdZGiDjQAVfQ9t5UKM0GRFwwmKDUUsbEfGJu1deexqdfpgclSeSElQ7AyeoGXWWFqC4mg+QaBVcgBIuhPjNb8RFS7e1tlUg+8xSerCfiLFLjpsminWMgcaZS0Uu5LmYxcfMyny9bA5nXjtvG781WvnWNYqUOJeH45ip/Vlzs9N+mzbMM7htcaVPLq9qlxhV5X8JlWOzGtqUsEhjCEQKVDAUYhMtMUUxa4eQEaZYkApbFSxC3bvTg2eoCqY2DVNsasMxYldRJisil0sj51SE8WyYKfpYxcSu9Ls2YZiV9OCGfwUU2wsiMJU7LIQu4r6XCTBF8FAf+zFMEsAVo7RR67YTXDQpMRjFRWrFIz4ncGMnHXB9LHLVlLM/eCgWzcSFZuk7pmkK35jjCuXSfn0aG1W2ofdxy65NJvLLKgnsrUQu1I8QXGtziH4nmgttov72DFQY2jyaXarTo1AqjIW9cMz/B7DcbzoYR3IQICCHz9vWj8JLyvtTUX85L1HNzRP2+9A+Iy5FDt6uOLJZEkz5zVB2hNTrHmtSG22aFfsBArNyr0BpaLcMjTFhgpZqNgVAFRSTbGccwTiN0yiahnZrrZliyWPnZvYyWOupTsZ0n3s6nVJ5GS6k6EhIBIkpVrLg1DRG9BNsX4QoO7wsSvOmQNv4zq5vFSvAfV0xU7Uu2WmKXZoCIh4f3KCYkX4qCmWRuTygQGNtIoXU+GFl1edGD/IFbsJDvpwC0bbFJslKjZjrVhTscsS8GBT7ASG+yaaqNixeIJP2wOW/qX7bZZAE+RBr51qn0PVotg1bIq1qIFaVCwNnuD69rbEz7Zxkw71cH3sTMjKE9F3M3jCnGOYPif8rHzsUkyxGc2tWeYZcPWCU7CZYp2KnZt80sCdLOfeA9C09NBouaOdI0GxbEcUu/rmLcrUDIZ6pNgxcBlUUSz4sT4o6gFX6U6IKZaqUrXNcWJnBk+UZ82Sn6ViZ5piK7o5VSRZZpRARZD3M0n6TMWuHiN2tQ3rAQDF2bM08hYqdnpZMBvMPHaNmmLhyGNXrJDkyAN6VCxjbutCjrFFTuwmOMY2j1341+V/BWRPd0KJWMHL6GOX6A9H2jVwwzIjXc11uhpnN4kpk6zxcI45K4d/a4ZiZztW9nQn9CZtn7RvzNckw03F5OAJ0xQbT9kC43uy4ql9H+bdS3Sj8tiRdRbt0vcY8R/MFjxh5oQbDsQx41Sxo6bx9o5wWXu7dXvdzG/p32L6p6Bz7zx1BXo+/OFohdkw+iOCJxxxzj5jgK+uF0mwGUOd5HDzIwJWLKYQO86lKZbv2I6tN92Eyp//rAUC1LZsjqc7MRTOUlkRY7lrpApPUKloplgAmFQLFb3JLCJBhLxJtwNhFq3rkb0+D8AHBmTCY0ApdqXZszViXAxMxS6Z2MWCJ4YosUtIUOwKntCI3UBMqWYp11iOsUFuip3g0IInRrvyhDTFht/3pKSYrmyxTGaBzIpdg3csnzHUzPqTYh8zBE/Q9UmKnfSxo4ods6uVVUtJMS31imNfYj6BST52gvzQdCey8oSdvMfMegmHOka4hsmaaBoRwIiKZZZAA2Lerlny2NkOnli9J+YpsSmtLEKvganvuwJNBx2I9pNOcmyfTNxlShbXiwhVY4pFZzSv6WPnMhe2lHy0HHkk+h9+WN8/MJmg2AOHFxEvmhiaouAxqQoHjAEc2P3zn2NT72Zs/X/fQuvy5bJtnZhi4YiKbS/7AGrRvsTH44ODsqyYwNvqL2Heb+7Da3vCTG9c87GLrnXDPCrgRxdesHMnvKlTUe/tQ7B7dzjF2bM1YlwMavDqNN1Jch47Ga0rdoQQzszBE4TkFcj2fFCPig127Ry2y0qOvYtcsZvg0IInoofWQKWOtTviIfp7a+zEkmIBd66j0GrF+l62kmIJ9lpbUENW2G5wpklSLGOWcVSqjLiJ2TYODZ5gzE54rD5ammJn3xcz9UqSj11RzifQSnTRuaflvko60iNlihVbOU2xRre+r9K8DMnKE1ny2CXvTxpEH/UA1uCJ8gEHYMrll8OjvmsE2kPX6gcYtXMcR/0lwH2tiK9SETNIhkBLuYAZn/snFOfNxbSrrlJzYkA9qp3KOEch8h10mWKFalQPuCwpJghPfetW3RRr87Ezgic6m9R366HgHIFR27WwdROO2PwsWuaEZlyWpNgZx0G8yIrasLVNoVrntbfD7+rSFbt6TY/adQSmCFVQ3kOiC0WbF6+7ExTX7IpdYYjUzO0f0H6/fb++Rzf15qbYcYOc2E1w2Eyxl/3nIzjun+/e6+QuU0kxmX8t+aZBH8YFL1uC4iTFzqy40AhsY9v20WPmA1MnuGZ5MXO+oq9qYAZPxCdctZRms5HK+Lz1NrE8dhbFjtY2Fd2K/TSrCZijNlJSbLgKgRk8ETPFGnMoEEJbjXzsfI+osEnEbg8edtJU6QieSEOamSyNfLquj5hJXPQjgiciIlNmBrEr+ijNno39f/UrTLroHZpiJ2u+gqMQEbpy0W5QKkbXdz3gMgU0JUOUpNQ2b0lNd9LRrBQ81/Gt79qpf98Zfi9OmxYuIGRS+pPK8l56X+I3VHnllXDT9ZF/3fRpYIWCVrmjFNQ0YpiWoNhvbor+RmR/MGO6E7qcfC6QaOPAyGPHOJcmYCA3xY4n5MRugkMLnog+P7+5DwEHXt7W79psRMdOCp5QCYqT+9IVO5bJ/yot0MHWdxZY+5UER+/X5gdFH7hJip0gnHq6E52IishCG0Gmvbl2Ma3yhJ7HTq2TQS/RKNI8FfOxS1bwktYN90FipjuhgopnU+w8RWgr5DgmXbdSDRveFLU+aIJi0xyfBFvENUVagIcefEGXm+NEH4zgiRixK+kKnEwUzRi4IEMAzjp8Fo7ZfzLeeNhM67yKkYmWc466UOxAiZ1LsYvSwxjErrPF4mNnIDCCHYLekPD4kydHg8bJl8/rYKVSTLGTLguvrA3nGPnXFaZNBysU4JumWNK3nxIVKyKP/SiCWQ+eyBYVK/oCgGI/IXaDuo8dQ6CR0NwUO36QE7sJDptiNxSVsKnt5dqxNaOqRFJJsTQFzvSxy2SKzehj12haDbs6Eu8r7mOnP2g9xrQdi6esCL/T4Amz6gZNQRL2HVcIzc/aGNRZ34srdlrwBGkslC1pirWYoul6+T2BCo2UKVZGYwrznWaKtSl2yrQvfBU9QnJtxy4p6Xb2eYYbUwW0kYAROi3bodKuM+v46rMZDan3Ex0HX/exazKJXVkndlI59XylvAE4fE4Xvvee12Pp7C7rvETgSp0rX07pY1Yua4qTzRRrRsV2tJL8ei7Fzsg7J9KheK0tYE1NGrH0I5Lk8QD+lMlxYhf9ZqpCsYty2BVnTAcKBU19LNVrOnlyKXbSh1a/l3qBvq0zKtZhoi0QYsf7TcUu3dyfY2yQE7sJDM45aCCsUAWGquHNgOZH2xsQip2sFZvgG5a1YDoQvhFnuclkjYodCcXOZoo1eJsck5pkaU9uU6xOTCh5Ek7+kiBrtWrJZ8e+NBY8odZVDIVQEJLUyhOJJdns+98opGIXfRcvMsUoEbFNsTMTFIfKaIJiN4Km2IBz8ltpQLGz9EUhfVtd27teAjLmsSsbU20pGfnjxPFraZGKXRalXByDgEN6nAkyVOjp0aNiN22SqTxcef+62khFDMfBMNOTSGJXLsNra9OOiCB2Pjj81ta4KTY6ToLYDT37LACgOHsOWLEYi4qlip0rlQyLVDZxTFl0jCip9EkVDhOu5YX+3fJzMDAQfzGzVO3IMfbIid0EhpndRDz8B6MH3d4idoJAmgmK7abYeI1TG0zFLstD3/T3osiSmd8FG7ETS7R+jXl6xoOWGe1dCYrNOrC0nUxBYg2eIPNz7GKaryFN+0GJZ1UqhPqcYj525oPCPg3ruuEmKBZbCbI0UAmvd6E+2n3swp2v1NU1q65bC5FPIU1ZIBWtgMvz14i/Z9K1o613TFIjK/S6MdPuGCXFROWJktFvs5G+RF4b02fIfHT678E+LxEtG9ZIjtqK/G3FokZS6tu2qfEcPnZd7Sr4JKuPnSB2rFyG39qqmWJZZP70AbCm5rhiFxHMytpXwDnHwCOPAgBaXnMEWKGgBUgUg6qmBqamOxEqrMfAmpv1wIuMlScoZm9+Se3zoJ6gmDOdODby0pFj7yJPdzKBYZYQqwehk7Z4KFf3gin2dy9txzv/42FcdeaBmUqKDafyRFZTbGLlCUbbjZxip5ti7QSS/tUJq5GgWJhiE4JQpGJnTVCcTl41c5yljanq+B7TriuT0MYT5ZrjuY/1SPnYqaCE8PtgpFALYpfkY6dMzCzxujXV1+FApmXh2X8H+vb2z3KOknza+7SpyWF7o51YYqQ7afL0+0dr2e5j58+YoUyqdH5OxS5cXicWB0F4eLXqJCmSeJo+dh0takzHRRXs2q1/F1G8pVCx83YTAhWV4fIZ4DU1xVS2YpQ3r7p2Haovv4za5s1AsYimQw+NK3b1uq6KpSl25HfmtbQY2wbg9WyK3ar3vgZDTS0YuFURY94/oNXXDpin9W/6UOYYO+QUewLDLKlUD7g0NQFKwRtJPPzCNuwerOH+57aQPHbCDBO/qWbPY0fUCcMU69o00ceO5vAagahY5c+klrmCJ+TmzCBWhtol1pmkjZJH6WNnqeCR9LBW87YTQRdcCYjF8nhJMXM8d98jFRUrxhDXv1CoRRWNWB47GhVbD2LLrKTJYlpsFKL/YQdPJFSeCJfpf2PbW9RkczndnpnBE8b12myaYsW1MX068bGjpj0HsfNpVCy0MXm16lSlpAnW98mFWdAUO9cJM02xPEouzMolsOYmPQBHKHYMYM1NMcWuWC4DjIEPDGD3HXcAAJoPOSQkgb6vKXSloGqYU12KXZQzj7gHeC0tmvqXnMdOXz6lycPskj5WMDio/X658YtszonduEFO7CYwbIrdYFVFRFGSN1IQ/Q9Vg1jlCdsDpmaY9Fyg2xYNPylXMfGsPnaNqi42wsHIm7RaZlfNqMqlEVYL4QDipE03xSYodhny2KWZ80yYx5oZZNNUSeOm2OyK3XATFMcUu8gU2ywVO73fgjUqVkUc266vkcljBznPPQ2eGE5ULF3nMuGHbaJ+jMoTza16fr2Wohk8EfXU2oZA2HcJ8XCaYkkeO9mWEjtbdQXPA/OVqV0mUy6X0UnSnbgOhWmKld2Wy2GiYUKgCpFiV/QAr6k5ptgVCh4KUZqUnbfeBgBoOfI1cj39jRSDWlSBRuyn/Z7smXnsHIodqlUZNKTBCJ7g1ZpWLxeIpzsJGNPm01oa3wbAq6++WgZHif9LliwBAGzbtg3ve9/7sHjxYjQ3N2Pu3Ll4//vfj5077edd4OKLL471efrpp4/G7iRifJ+JHHsVZqWJOucYImRub5hiBwSxq9VJguJwnW+QiDrxLWrEFGsm0i35npWkZs5j1yCBsJti4+tMxc58iMaIjBk8IVUkPW2MRuw8vRSWbbzws30fXZGR5joBV546pSa6+zfnZKIRs20SxGZC71GKna+tF9AUOxE84TG8/+QD8Ie1O3BAT1tsDBVYMXxqZwueaMQtwKXOqvWCfCaQaYRiVFIwkfhqBk80t7cCm3bIdjFTrCSuHFyQrgyKnSDS1AdYECterVojPM2ACRSLQLUK1tys5bHrd9zzAiMqVvZbLqP7wgvBvvULuewwtgsnvfwsjpvswWtusvqGFmfPQm3DBhk40XzkkapP8iMpRaZTnwE1nmCKjdp5ImhCELutyoQst63VZBUOgZj5ulZFMKDnMQ0G+rV7IQd71ZliDz74YNx5553yeyEi+OvWrcO6devwL//yLzjooIPw5z//GZdffjnWrVuHn/zkJ4l9nn766bjxxhvl93K5nNB6dDCmit0111yDo446Cu3t7ejp6cE555yD1atXW9tyznHGGWeAMYZbb71VW2cyZsYYfvCDHySOvW3bNlx44YXo6OhAV1cXLrnkEvQa2cX3dQSGYlercxkhCOwdU6yIuA0VOz2Lv5amwwwMSLlS6TOg6HvWlB8msuaxa9jHzmr2ij/ofWZPdwL9j4RJRM1xJHkiy4sFXd1wmS9de2iWQKOwlXwyTYVpip05cmOmWHfbJJglxUwfO3MOBc+T1xCN9n3bsrm45tyljkojYqzhzZFuK3xfgQaJnabIWhS7BFOygIpqdqu78kXEqDzR3KYrdqYplkYnT//HzwIAiiIvHNwvVOJc0BdPmynW6+xU6w0iQxU7mrKnr2q/59V377YuZ6Uyus8/HzOv+Zya99Yt+Mij38epTbvC4AnTx873UJo1S/VRLqPliCPkd89Q7ADy0pcWPBEzxcYDL2w+iKaJltdqsWobfGBQuw4C5r3qTLGFQgHTp0+X/6dMmQIAOOSQQ/DTn/4UZ511Fvbbbz+cdNJJ+Kd/+if84he/QM3hsylQLpe1Pru7u0djVxIxpsTu3nvvxcqVK/Hggw/ijjvuQLVaxamnnoo+QwIGgGuvvTbRJHbjjTdi/fr18v8555yTOPaFF16Ip556CnfccQd++ctf4r777sNll122p7v0qoLdFKtuHHvVFFurS5JXLoi3TNWuSPxogAZ97Dx7yg8TSf5Kum9Z4tAx2E2xYh1p5yB29C8l33EfOzsp0gmyqt9qbpNE2mzLzcNYshzXosMPUFaecPjgye+jmccu+m5GxcZ87HwSFSsqT6QpyBlIU/o8qWKHTONSpCl2WQI8bKbYWBvxV0bFRsEThunVVHRk1C/nKC1cCEBVTjDnT2H7PXf9xV8ACH3fBHEpEJLoInaMjAcAzlseqdeq7UMUCOGRKhnB5k3hstbW0G/OLCnmMZQW7ie/z/rSv8Dv6lJ9+nFiJ1/anHns0k2xPgkwMWGSPV6rob47JHZ+RFRiCh7TFbvW8tgYAPsqNewerMr/VJww8eyzz2LmzJlYuHAhLrzwQrz00kvOtjt37kRHR4dU9Vy455570NPTg8WLF+O9730vtm7dOux9GSmMqSn29ttv177fdNNN6OnpwaOPPorjjz9eLn/sscfwpS99CY888ghmzJhh7aurqwvTp0/PNO4zzzyD22+/Hb/97W/x2te+FgDwr//6rzjzzDPxL//yL5g5057xfF+D3RRLFbu9Z4odrAbSBFa2PFDlW7n0H0t5kJLVBdPHzqFyZPWxGwnFTipXlDB6pjk0+ku+01MQy2MXM2sKVUwtK8UIMhlPm599X5JMcFbFLmaK1fc7LY9d0qE2V41UuhNpii3YgyfCEnXh50rG9DsjkcdOBWwML6UEHTkxJUuqSsr169ZFsCMfO1EdIY3YiWsj4Oo6N6O2Gecyx52AjdhN/+AH8PKtPwXqdfChMHihMHkyKs8/H/blVOx0YhcDY3rNOXN1ZHaj57m+eRN8AH5bG+D7VuW9+/y3ghV8tK9YgdK8edp6qmoLU6xY4k5QLEyxVLFrzq7YmcSuWkPQFxK7wtSpqG/fDj44CE6sOKFip/o309mMFk7519/CKz8lv//tyQfgylMWxdotW7YMN910ExYvXoz169fjH/7hH3DcccfhySefRHt7u9Z2y5Yt+OxnP5sq9px++uk499xzsWDBAqxZswZXXXUVzjjjDKxatQq+P3YK5rjysROOipMmTZLL+vv78ba3vQ3XXXddInFbuXIl3vOe92DhwoW4/PLL8a53vct5U121ahW6urokqQOAFStWwPM8PPTQQ3jzm98c2yaoVGQkFADULariqw02xU7zsdvLip0ygYkHqmonHmA1mTcsud9YgmLqZ2YhIEBcXXL116iPXZJpLpbw16bYkeLsNHLZFq2ZNkbBcDR3RQu7iZ2uhFJYFbuYKTaah8X/zzZusnpkbjtcxU6X7IRiJ0xJ5ukLfeyEX1eySVtuI/3Xhg8xhOZLthfy2CXNUSl29n5pB8LHTkbFGr+7WIJicRq44k4x0ghVNkzA5hvrlVRZMKEs+VOIYmeoLi7FLtZvWxsChxkWUMSOTinYvDlc1hr6XpqmWN9j8Ds7MfmSS+xjJppiXT52IkGx8rFred0ysP97QhsXSFHsIt9DXqtK83Nh6lQM/elPYbvBQbUNYxqxM30oRwt3vO8ozJypTNu2F04AOOOMM+TnpUuXYtmyZZg3bx5+9KMf4RJyLnbt2oU3vvGNOOigg3D11Vcnjn3++efLz4ceeiiWLl2K/fbbD/fccw9OPvnkYe7RnmPcELsgCPCBD3wAxxxzDA455BC5/Morr8TRRx+Ns88+27ntZz7zGZx00kloaWnB//7v/+Jv/uZv0Nvbi/e///3W9hs2bEBPT4+2rFAoYNKkSdiwYYN1m63f+Ca2XHed6qNasbZ7NcEkdrW6HhVb3SuKXeRjVwvkWE2FuGJXNPxo0oMn1OdCRlNs5lqxjSp2mrk1VCOEOqGbP40Hpqwvqh64lNi5Kk+Y321RsQKuCgKZ8tgZ49sIs8uf0WmKdVQxsCGm2A2TNYldFcdWvMyI69AkjD5TUbHZXQP0v8ObZ7gxLe033OAJ2zwaiYrVrxsd0qTbqClWEA0SHBK7psFRR6g2ichZ23Xnl5QiJ17AC5OnqDnHgid0xa656EtrgjZ+ezKx84RiR45KfZMyxSKox0yxacqWzRQrrnUP9nuyZ1Se8BjQee6b0XrPo7KNH13f3IyA5VxGI3tNTQiqVaBWk/VwC1PUcaTmWA5oVTVMH8rRQmupgPamYnpDA11dXVi0aBGee+45uWz37t04/fTT0d7ejltuuQVF87pJwcKFCzFlyhQ899xzObEDQsXtySefxP333y+X/fznP8fdd9+N3//+94nbfvKTn5SfjzjiCPT19eGLX/yik9gNB5P/+jJMetfF8nvL2rVAFCr9aoUZGxFwLv3egL1TeWJQmmLrlsSwNqUpmymW3lgLPnOSm3LBkw/yJLPWHlWeMNTDSi2QD7+S74VpToCofFV8HEoKKLFz1YqV34UqlkBqdeUlfV+S8tjZCHMsibIxJ5MkNxIV6yKyjYIm/gXUNSmUYzEPceh9n8X8G9MIlirXNXxmJ/avYtQCzoq0BNTZ/OfiLwvOFDUiCIrbf1+uqEka9RsjjRGRKdZrGCqEqpzNtaJgeQAXqGJXMk2xURWKSLFrbypYiZ3f2oYk13lliiULRbmxtjbwSiW2TzQK1waN2EUmVtGHs6QYhJKs3AkYY5j2gQ8A/+/hcF8ksTNECeI/6DU1Idi9G7xeR7A7jAT2OjvAmprABwc1YheaYtW1aaazGe/o7e3FmjVr8I53vANAqNSddtppKJfL+PnPf46mphQzvQWvvPIKtm7d6nQZGy2Mizx2V1xxBX75y1/i17/+NWbPni2X33333VizZg26urpQKBSkE+N5552HE044wdnfsmXL8Morr2Ao8rUwMX36dGyK3qoEarUatm3b5jT3eqUS/LY29b+1tcG9HH8wfexqhim2theI3ZA0xQbSn89mii0apti05xm9sfqeXtOUmlzpwyVzVOwemGKFuVI8DJtLPj5+xhJcdeaBKBd8a8oRmvOOnoKYkuQgSVqql0ICGWKO5Y725nGwmWKd6U6cwRPJ362dGX02CrGVSHcigyfItWFWMkkL+jChnNiHNUXnto1ei0nkLa1WrL49WWa0kUpzRK6EudC8FuKmWEWwxa0oRt6j5SKfG+DIG1jww8TDBD4JnjDTe8i5NoWRu21NZG6EkHqG7xVradG/RyZg/fhE96zWFngRcWTEN669KVlP8Ty1H6VA97Ez1T+5jaOKD1Uypc+XJVBCzr0lPB68WkN9x45wu64ueM3RckrswDTFbqxMsVnx4Q9/GPfeey9efPFFPPDAA3jzm98M3/dxwQUXYNeuXTJw84YbbsCuXbuwYcMGbNiwAXVCfJcsWYJbbrkFQEgMP/KRj+DBBx/Eiy++iLvuugtnn3029t9/f5x22mljtZsAxlix45zjfe97H2655Rbcc889WLBggbb+Yx/7GN7znvdoyw499FB85StfwVlnneXs97HHHkN3d7czn8zy5cuxY8cOPProozgyyh909913IwgCLFu2bA/36tUD0xQbGAmK924eO2KKFYqdJUXJcPLYFX1PJ4nkQdBSKmB7f1UbI62/RtNq6IpZ3Nx12fEqIi5MzxM+2DzjIcoAezJR6O3MOWuVJ4zJa8pLhgTF9BlqPk+tplhjUuL7lLbwATiptRTbJm0OrvV7mqBYKnY13SUgbAOIX4KZFzHL2CowYc8VO23cBpkiQzwPndl/0hyzpDsR11Fp9mz0fOQjaNnWA2y1pOdxvIgk+dgJQuTxAAWfoVrnseuOseh3VCyCk4dwkilWmo2bwmcENeWxUkn6knnteo5Cr7UF9f7+8EuxKJMeU7orSJbf1gYWmTHFeQCAjhSzoV+Im2K9iEG5FDtF7PQMA9qLmVTs4qlNZD8R0eW1KmrbtwMACt3dobK5HTHFjpqGx8oUmxWvvPIKLrjgAmzduhVTp07FscceiwcffBBTp07FPffcg4ceeggAsP/++2vbvfDCC5g/fz4AYPXq1TIWwPd9PPHEE/j2t7+NHTt2YObMmTj11FPx2c9+dsxz2Y3pmVi5ciVuvvlm3HbbbWhvb5f+bZ2dnWhubpZ5YUzMnTtXksBf/OIX2LhxI17/+tejqakJd9xxBz73uc/hwx/+sGz/8MMP46KLLsJdd92FWbNm4cADD8Tpp5+OSy+9FF//+tdRrVZxxRVX4Pzzz58wEbFAvKSYqdjtHVNs2Gc94Ogd0hU7+oCRwRMyP1Nyv55GQHQfO/qAac6q2JHPe1IrVpDKJGLqMYY658ThXj1wzXNkbmf77lnGF3D5XbnrhZIHljFe2arY6cu6WkIi957jFuKAae1YcaDu26r5Mqaa23UMN4+d3I4LxS7Ku0aujXC/I+XJ81L9G2Nj7IFSp+YZJ0KNEkUWvTXYtssWFSvGpsTOrV5OvuTdKH/vd8DW9bFrIT638C/1sYuVK6vXAT9M1VHwPFTr9ZgpVv5uCCEDDFOsKyo2IjIdREWj/fjtHfpYLa2oY0u4LQnY0KctFLtWNB18MLyOjvBai9qk+YN5WlRsSMJE9y7FTixnnn6/0SwPhQICAL333ott3/sepl91FbzWVo3oeZH5kddMxS5UKoMBPXiCRgyP9wTFSbltTzjhhMSXaAHaprm5Gb/61a9GZG4jjTE1xV5//fXYuXMnTjjhBMyYMUP+/+EPf5i5j2KxiOuuuw7Lly/H4Ycfjm984xv48pe/jE9/+tOyTX9/P1avXo0quYC/973vYcmSJTj55JNx5pln4thjj8U3v/nNEd2/8Q57VCxV7Eae2FE/lp39oa+HCp5Q7YqGYpee7oSQGU+PiqVRUtRxOWvliYZ97KzEzt3ejGaVih0Dkk6BK8LUphiqsez75ZpfYlSsRbGj43U0FWSbzuYi3nTYzJg5zmEZtiJeo3R47Imm2QDi6U7CvqF9dlX9cEH6FO6RYmf0OQy2qIhZfJ04VYl+gJZrCjBfEPR1i6eH5svF03QzpglbZY2YzyWIYifM+TG/0YjEGORNy2Pn26NiPanYEWJH+mk77lh9rDal4DGiymi/JbEvbW1ghQLajj1GU7Y6mpP1FJomQwZPiDGdil0AFArKFOvF5yUUu83/fj12/vRn6P2/0J9dEjvGpGkZtRrq23eE23V3S1NsMNAv+wsY0/arduf/4sUL3x6rq5tj9DHmptg93eb0009Prc1mY+OTJk3CzTff3PD4+xLSEhTvDVMsNfXuHAhvKCKPnRl0ALh9b5KQlKA4s2LXgJJkG1+gVIirkfGxQnVINKF/h6PYaaZYR8JgQCdSruklRsVaTNnU9Du5Ld0ckSUy1zaXLO2dYxKlCFB+nzSKU8yr4IVO6FnKqeljpKthaUhLb5O9D+5IwZM+R1cbpWfGj8X7Ttofb3/9vESze9hnuCHX+jL3OYqEbSoj8AWxM0l21J9B7KiPHU1VFXYoTLEhYelpJ4mRiRLXdMghaD3uOPT93/+FYxHfakrstN8S51rb1uOPB36j1qcqduQFQ/jYhX26KTjjHKxQiL8kSoIHeMLfLiJysv5tZIplhYJMWRNT7ISSp0XFMlABse/66zDQtwVbvv4NTPvoRxL3McfexbgInsgxNjBJw2godjTqtq9iBk8QQtLgg9RMlZIleCKt8sQFr5uD0w6eJv3DssKWtiX54Wlup9SeRGJnTN+sT+oxd9mx+JzsE0xSLm1O7PRYT055sIs5pkyBrN5zogMoQiHea4SK3FzSfewAkKAPt6+iDeLQ7AGvi5Oc4bBEeW1Z+m8oeCKb8htuw2Kkzu7jF/4Ngyfsil2hI1T9St3dkuSZQTtWxa5QkGQEAIL+fm0bERUrFLsPrDgAh83pwtVnHaRF0LJiEZOJnzdV7KgpVqsoA661bTvuOM2EmhY8waxRsck+dgyC2In7jX4v8D0mE0gLBH3hMdFy2EVKJq9UNWIngiqoKTaWoLgWBitW/vznxP3Lsfcxvr0dc+xVCMWu5Huo1IOo8sTe87GrB1xL3SBgq9EZT9ORXc0x04hQ0w01xaY9nK85d2nieheG42MH6G/XAGJ57GLjOMxjqi4rS4w61dUy+xjMOK4UB87ogAlKgNIUm/h80tomf88K6bQffR+Upe3iUbHuxMpp16M4p8OndjGFco9MsTbFLr1fscaeaseeosQGm9lebBdwrlIvmQTS9wHUUCj4KES52lxpfyixMxMSx4mdrth1tZRw28pjAADPU9+5YhEtrzsK3W+7APUdO1GYSgIyNMWOXMeCpEYRtIXJkzW/ubTgCUrAZPAEDwDoRIrC4xysWIwFYNGXRlPRFLVgOVXsorHrO3bINCi6j10/gPD4BIyhRiJ4y/VQFa0bJbVevPDtQL2OmV/4Z5Tmzk3e9xwjglyxm8AQpKFIqhNQU2ltD0yxg9U6PnXbk7j3T5u1ZTYk5bETSHuIm1GxtrJagK7KJPnY7QlspDJpJM94u6Zv20k5otNMsZ7FhEgPKyPHyEVU6Pai/5++92i894T98N4T9ou1p+ctmyk2Pv+sGG5UrNiKy+CJuGJnkuRYhGcascughqXBVtqsUbgUNyBbuhNX2hbNJy3DebClxvEIw3b52Mlr2WPy2vKN61q+CBV1pY3CJHatr18Gr6UFzYcdFpsXK+rEjjGG6Z/6FGZ9+Uv6GDTykf6uwMM6rYTMURWwI0Wxq/txYifJIjmOBULyhCnWVOoYuSfEyG6Ub08kLGbFomxTi6pnsOZmeE1NTlPskE/M1rWQ2NW2bVNtOMfgE09g4LHHYuPn2HvIj/QEhhDPSgVPmkXFQw6AVV3Ligef34rvrPozHn9lJ96waGrYt4vYiRqd5N4fM32lPDzowyUpKraVOO+bSWdHCjqp1G+0NiifOuMhysIUNC64kv1SxS5J9dFJlWMMS4DFkfO6ceS8bmt7qrRmMcVqD8SU0xESdpa5+oML1GkfcARPGISuUR87Obc9uMRGzsfOpdilz1ESQ2PszpYitvZVtDZJKFuS1wqy11epOX3sqHJaICTPYyodjSTRSYqdUcB+0jvfie63v52kK1GgPnYxMqIRO2KKNZRnarIFoPkupyUorpFAjyJJ9wIAjChkRcZR49ExAQeK1Mcu/NvdUkLBY5jW0RQnu31CsavKfZXEbksY+et3d4X9tYaKHS2lGTCGIUJC/egsUsUu6OuTwRl+t/2ekWPkkSt2ExjSFEseaH2VkVHs+qJUJrsHVCRyI4pdUjSnDXR1wfSxI/tHfezMuqYjBVseuSwO6jQaVixPNMU6TFI++Ws+kF2VJIYTPGEDJdGTM/gmMsdnG3yPYd5klSB22AmKhdN+dGiHjAoodC7Cryvm85kaFRu124PoCWZcnsPpS74jWDaVqk6GQm7m2F2EnGQ5DTbFbs6k8Fy+tK3f6WMnfqK+x6T67azYkkDsuKHYAbCSulg/jjQpAOCV4sETgoB5CQnszRq6JuqE2BUi8ujZFDtOKkbwAKxQjCl1k1pL+Ol7j8a33/U6p2JnC54Qip3f1RX+jQJR6lsUaeOMYdCP/8aDvj55PutRLjzW3Cwja3PsfeTEbgJDkAZK7PqHVLLKPfGxEySunxBF+tZKYTfFmqWpso9d8DytL/pQaWrAx2640IIXPP1Ga2+vbycjKpGS7sThPyd99YwHIB1L9K8+2+en5bHLQuyoYpfFFJvghG/Doh6VQiMlTVrCmOHf5OAJXbGLq6PJc/XIORwuzPM7PFNskmKn/7VBqsAmsWshqlaGvbSRGUHSX97WLyPwzb6oe4FUT2Om2OhYG75xw0VSP4z4vzESnCHnQFKdOPtPuXaoYidr0QrFTkuFou6tTPrY6S+JAHDYnC7MndwS25e6xccOvq7YFbpCla0wNbS6CMIHhMETNd9u9AuiJL71yCxbyNW6UUVO7CYwaPCEQF+FELskB68UCPNWP+nPptgVSV1XPRedW2mygZos47Vi1efR8LGj+0OLcrtgmmCpypKk2JmHRGwvzKEFwyQt5mT77Fbs1IosPm2aYjfCUbEAsGgaiUoctik2/CtKiskKKAXqY6cIBWCropAyhiT0w5piNM/GVEIblCk1vs7PMEflJqAv1xS7DE8RW/DEjM5mlHwP1TrHuh0D1nE0YkcizG0uAhpxiZSn4rzQWb+0cGH6JCNQYhcrRUZUL2qKVVZtPdXJcEDvx7IWbRRd4lPFrk4qRkD42EXfLSdVqHECQa/uY4eixRQbKXaS2JFSnEHCD7a6bl3YPlLscjPs6CL3sZvAqMvgCaLYEYWtWhu+YifMW7piFyd2ZhkngaT8azZQAlRwRMUWffXWz9jwTXlpoAXvs5jklAlWJ7hpptiS78lyZIB6UO83tQ1nHDIdh87uxGBFP+aaYqd9ts+vYVOsFjyRxRRLyGVqa2DRdKXY7WmCYh6l2RCKXVMpnqBY+SsOL0p7j0yxxqbDUuzkfCwP+gymWOYwxXa2FGNtkmBT7HyPYc6kZqzZ3IcXtvRZxxG7XPAYAk5NsbRNtB8WE+rcb30LW2+4AZPf9a7UOcptBbGLAie0dcVkU6xS7OzErjVDdYbXlwdwzItP4IAdryhiZ1HsCrUKUFTj0nQndmJn+tgJYid87FTwhFDcBCErTA0rxtQ2bwYWhdvzhGu7snYtmg46CPVtEbGbNCl1v3OMHHJiN4EhVC560+0jpthasAem2EixqwUclVqAUsGzBk9Qp2pmIWMCaQ9I6g5Y8PSo2KIkdp6mZu0tSDJATEbJqoiu2Ili2i0lPzkq1mNoLvqSPFOV6fq3hzWQr73zT/o2jom4ptdoBQ7aJlu6E7JthnOyiFQzGC5nEhyN8zD9jiDG1ExvmmIbLSmWJeI0DY1Wu7ChtVzArsGatdyTLMWVQbEz59LV7CqnZQdNJUMxf3Ir1mzuw/MRsXMpdh5jKBJ/Ozofeaw1H7vwc2nOHMy4+ur0CRIIYmcz57qiYuVvWNSJbbWbYttSImIBoFAs4BO/vTHs74ADwr9BWFrNo8SuWgEitzXPSFBsPSemj12vJXjCUPWkYtcTKXYkMIInXN21SLGrbw9NsSIII8foICd2ExgyujB6Aw64odjtUboTRQoHKnWUCp7Vx04kJwZ0dcg0xaY9SDVTrKHYicjUou+plAmjQezIPLL42IntjpjTjb8/80C8dn43vnX/C4ljtZQUsbPtUlJiWY1UuRQ7ywM0CfTFYFJLhqhYgixnZP5kpYRs2jXUUP/mSAHXr1ObekzPJUV6ME/SEzYbir6HUsFDJVLOh5Pe5QtvWYqXtvVjdndLbJ2s2JB4bdrJXxdV7DLMy2aKBYB50fl0KXbSFE6UYI8x63WcFDzRCEQ/Vj89zRRLiV30N8UU21ZOnxezjOEFFsWO+tiBIxgaGp5iR33sjOMmFbvJk8MLhrzs183oHoLK2rVhm8gUW+jOFbvRRE7sJgi4pRC4MPP5jKHghUmK6YO5siemWFLBoq9SQ2dL0W6KdQQzmIpd2rODlkczgwamdTSh4DHM6momit3ecy+lfnU++exsb5A/z2O49PjQJygInk8cK6y9WtH60fs25xYfNxzbNTf1OUuwwk4SBZ1WBD4ctzFFkBKENZt70ydkgdingKu8jb7HNF/MmI+dbxKO5DFmdYVSyozOpuSGKehoKmJL75A2l0Zw3AFTnesaCfCIKXaE2CVN69BZnfjD2p34q6PmWNfPnxISzs27w300LwF1Hjwgqhvre3q0d5IpdjgQOeesih0JnvCoj52cb7TOETzRlpacGLq5lzW5fezM4In6rp3xlEmOuQMkdYklQbGAUOyY78OfPAn1zVvkOmqKpXMBgOrayMduW+5jNxbIid0EwO1PbsBHf/I4vnrBEThxcY9cLiIuPWHaqOu55vbEFEtLhwlFyWaK1RU7SuwaVOwMXzT6IJraXsbtHzgOXS0l/N+zm2PrRxo+IQVZHp5Jb9lJPnaAnr7FqtglKE10TZbgiSzEixK7LMgyBxe29FbSG1kg9oNzGjjhWdO/uBTeNJXqsuMX4uj9JmPp7K5hzVGgo6mwR8QuCS41jsLljN9JgieSZvXDv349ntvUi0NndVrXz5usK1suHzufQSb89RkzXjhsptg9UOyEKdbSh6amER87+TInvjsUu7TkxAA01Uz48bGIOLECVexI8ATnCHbsTPTtjBHVahVBpaIUu1Ixts+C2AFhAAUldgEZoynQf/eiHJlQ7PxJObEbTeRRsRMA9z27GbsGa1i1Ri/1UieKnbg5UvNrmim2Ugvwh1d2WpPoUnVOJD0eSgmeoPciM8dcI8ETAGJv9Pv3tGNKW1kqdaPiY+fRqFj3eOrhGV/3sTMORFdLER88ZZF125aS3UdRwBzXmcfOme7E3ZcNuwYbJHZal9nOyb+97Qi0lnz883mHNjSWOSbnXJpimw0fNBWNqcgERZpZtOh7OGJu9x6TsXZCoEaa2Ck/wHRTbCyPXYs9Oa+JllIBS2d3OYnw/Mm6idhsRoNXjt1/CjqaClg6p1M7/lZTbHH4xM5L9LFzmGKNv67giWym2HhKlddsfwEdQ31YFOyW62geOwaOoL8/0fXDRlSD3l4Eg2H9V1Yqx33siG+ciIy9anovWqsD+OgjN8t15ZruFiEqVIh0J7liN7rIFbsJgF2RijJgREgKQmY6Iwuk5bH7t18/h6/d9Sy+9JeH4bwjZ2vraM1ZkULF7mMXd1gHLIpdykPN5JauaM7iKPjY0YdhtqhYN/lbMKUVv/vEKc79b6GVNCzbx9OdqM+NKnZZjllPexOAXantbP1nPSV/sXQmzjxkxrCjmqViB6Uim879NBoTsAUyDGvohkEVnpG+ZpNeKNLa0HQnWXzsXJjV1YyCx1BzVBMRfRci94RLjl0QulqkmGLNNCWNIHvwBA0g0U2gfns7bMhE7CxjvPPZO3HhIz9D+7HHyHXU/CkTIxv1pl39CgR9fQh2hb9Xv7MjRv4EmQOAYk9o7Tl74AUc81/fCatdRCjVdPVckMXajsjHLo+KHVXkxG4CYPdgSKz6DWJXI8ETwyF2wuH5OYuvk02xa8QUW2w0KtZgdrY3egCjpNiJvyxTPrPESDYkk1otqa6FbLiiDM11zuCJBk2xnzn7YBQ8hkuOXZDa1pxDI/xgJFLVUB87eh0CVLGLiEWsdvHeu34oaMH44dbGdSFTVGz0N67YqXnV98Blo+B76GkvY93OwWgcc47RX6F8WxRwuR9aKbA9IHYJwRNa5QlL8IRfLqP9tNPQetxx1r5bGwyeEKZYPjAQ5qojczJNseE83C+JNhUz6O1FfVeoAnrtHWBl5RM65W/eK8kcoEheZe0rIakjN5xyPSob1tmJ+s6dCAaFYpf72I0FcmI3ASDMY2bwglTsmF0NSCsptjvqd0d/3PxGxxKE0hY8US468tjFfMMSp5JoitXqxgrFbi/ViaVjh4pdFmKXbq51odVSLYEiKZozS0kxyq+zKEazu1vwzYtem9pOjqt9Hh2yRH3sbFUn6MSUv2RjtYtHCu17UbETNUvbExz6TYKr5qW2ES+Ow8VUQuxMc7xrfPrdaoodCR+7tKhYW0mxUgmzv3qts+/2DD52evBESLR4pRKbkxkVC6j7pPUlz6bY9faivmsnAMDv6ED7qadi8Kmn0H7qKWg/6SStrSB2IjCC9leuh/Pzu7tR37kTfGAQvFJBsHu3XJ5j9JATuwkAceM1FTPpY+cxq4JVC7g1mlagN+p350Dcid1mihXjF30m/fe0FBMJqTX2RLHTTbF7PyqWBk/YqmqYyGISc6GZmGKz+NiZYzAWEhzX0Ew7jo3PLxXDMMWO1JABV36fTTFTrK7UNZrHbqSwN4ndX79hIeZNbsGbj5jlbuRQk+lcGg2YMTG1nQQhOBTm+P0g3kZEswJ7GBVbTDDFav5vdN7iBS75HGUido7qFuacikSxK/VMxay//xq8/oR5WMhuva8PgVDsOtpRnNaDmZ+/xjovSeyiHHVUIZWKXRRsEQwOohYFUMDz4Hfag2dy7B3kwRMTAEJZo+W9AJLHjsVrigokBVAIwpim2A1IxS4ke50kuWlWU2xqSTFjmvobvfo8d1ILGAv/7i3QFCdZgif2RLFrNCq2aOQTE2vdlScaM8U2Ck2xGyWyJA4JR1LwRPjXlcduZteepTHJCs0UO8LErqe9CRctn5+o2IkRk8ZuNGDGhE7sDAJH3Bq05ZYXt9FQ7OgymynWdZhE2psVB05Ln4DD3GuOTxW7mZ/9DDpOOYXcS+Ld2hW7PtR3Rz52HcnkqxCZZUVVClYs4ujp4fzOWfN/YR8RseMDAypwoqtLRjTnGB3kit0EwK4BoZjpvjABVewcpslqPXAmF+0dSiJ28XQnQh3pbFYpHJpcptgGS4pxwxRLnw/0oTBnUgvu+8iJ2sNkpCFvrl5jptjhEJs0U6x53Ka0lo31Ydkyd/CE+rw3Ak5GictpsJli48ETulJHFeAPnrIIC6e6i7yPJPamYpcFWV469lixa6NpQ+zjx2v1xpXekYqKlWW8GoiKTUsdc9eH3oDt/VWZ3zBxfLofJQuxiw63UMls41t97Cx+h0FfH4KdgtjZAz4EaCBFOLcSvnH6XDx84bsxsy/MuEBNrtUNG2LLcowOchq9j6NaD+TDy6wbSitP+Cx+owSS/ezEm7rtxk4TFPcbpliaKoEqdpTYxNOdNGiKtdz4BeZMatEI5UiDkgFVUmzvm2KtPnbGMrN+a9KDwFy+NxQ7rf/RuhsJU2zgDp4wExQfPrcL8ya34OKj5+N9J+0/ShNVfnDA6Pn1UZx+yHQsmNKKg2d2ONuIF8fhgr5kmX6WZtCEbKddNxbFbg9Msa3HHI2W170OXX/5ltg6Vx67/aa2YfG0dpxxyAxrny2lQiZSF45hN/cC4X5d/ob9MLtvC/7ihQfkcjN/XVoeO1G7NejtRX23Cp5IgknQWKmEclurJHWAnvdOmGwLObEbdeSK3T4O6tgc87GTwRN6VGxrqYDdkRpXcUTGcs6JYhf3sbMpduIhSlMl2Mo4AY0rdnUzQfFeJiRJEA8a32OZSNseBU+UU0yxRp9m/dbwQequ+qhFzu4NxS42l70PW7qT5qIr3UlI+Ka0lXHvR04clflRtO9FU2wWfOjUxfjQqYsT29ii3RuBRuyMXaTpTihstaA1MrcHpthiTw/mfefb1nV6VKz6LTWXfPzqyuOHPaY2RjHZFPuxM5bgvC/+DTZV+lU7wwRru5XQuRd6elDftg1BXx/qJN1JErymJrCWFvD+fjkXr0Unq15rK1ixCF6torZeKHZdif3mGHnkit0+jt3E/8VMd0JNsfShUS76Mt+bK+VJf6Uui6f3Veqx8mO6YqenO6FZ67OXFEt+qJlJkm15rkYLNGBCmmIT2iflnkpDsyMPoOpbLWsrF+JKZcKDwOxzbyhGw013skdjRn9v/f1a/OLx9QAQOy4yfcUYkCmKvZnHbrwg0cfOcR6eXKtyJb71tWG5Mt3HbviKXSIctWJHEi5VMPwe7pfX0iJz1wFx5d2q2JHgkkJPaFYN+npVHjtH7j0Kqr6xUgles0Hsmspg0bLalrBKhdeW3m+OkUVO7PZxUDOJmW5ElhQzFLtywZNvwS5TrJniwDTH6iXF9ATFnS2U2DmCJxqMQownKCaEZJQfiI1GxVKfvEZBExTbTJm0S1Oto+td89NM2nvhbkFVutEi4GKcrX0VPLM+fKj1tMd9D4GxJ1NUsdsbiume4C+WhmbHk5f0pLRMxtQ2FYgSi74V58FYceLiqXIOIjm6lsduD0yxSRARs8AoEDvPi+2H+O61tMInxE4cH+Hm0t2SHPghAiFqW7fJVCpehshVnyQaZsVimI6Fpk1qaoYXpWiRxM5RXi3H3kNuit3HQRW7gWpdS1+iFDvd1FEuCmJXd5pie4d0IrdzoIL2pgIuvvFhHHfAVAxaFDtliiVh8kV7STFTsUtLtRH3sVOfR/t5KCNhtQTFST52wzfFtpSTS4pRYmL61wHp5k8tCGVvEC9m/bhXQXejvamAj56+BOccPtNoM16IHTGfjTNi98/nLcWKA6fhxD0kdlPa1XVpWhXkeTBcMz57ziF4cu1OnHLQdNV2hKJik6AFT5T2UgBWNHfm+1pt2HB8u2In7jPvWD4PMzqbcPKB8XNCj4lIPFxdv150AK8lPVMArfnKSkUwxuA1NyOIzLOhYhcRu61CscuJ3WgjJ3avAgh/tqS0BC7sIspaPeCo1AMZAVgnJcXmT2nF46/sBBAGWQhTbM2RVX6Xodjt6K9ia+8OPPj8NqzZ3KelSVGKnYqKFchcUixVsTNMsRbn6tGCzJbPsmX3V6pZ42O1pJhiKdmbbFHsxOpMpti97WM3WqZYMtBBMzrwjtfPi7VRPnZjbIolbgvmy8tYo7VcwDlJOfAygqrOW3v1mqPiBc08D7O7WzC726gzO0JRsUlw+diN6BhiPwoFwC9Y13ktLZpiJw5PW8I50RW7MO1KdX0Y4OC1t2dKSVLoVoqdrKnb0gIIv7tyE7ym0BRb35wrdmOF3BT7KsC/37MGh3/mDtz9x40Nb2vmmBqsqJsBzWP3iTcepNrUAmmKrdbsD5NeC7ETZM+8Ocd87KgptuAyxTbmY5ccFTs2Pna0pNjeC56gtWItc9GIXVxhEGszRcXuBZLjqoSxN0F3Y9E0u//PeDHF0tqippq1L2Jrnx6IJc9DhmtjpKJiE8cYFR+7ohwrq2KX6bcj5u77UnmrrQsVO78jOXBCgJpiRT1e6mfnNTcpU6zIY9c2OqmBcijkxO5VgC/+ajXqAce7b3qk4W1NXzgawUaDJ6a2l/GrDxyPpbM7ceUpi6RiVnUodiIiVmDHQFWafU1hYaBSx2C1jk27Q8I3b7J6gxupPHaxBMV72ek/CbaSYsk+duHf4RCb5gby2NlMsWn5t7Q8dns7eGLEe3eMSUZaNN1O7ESLsVbsKLHsG9qztCKvBmzZrb8UKoKd/qgaDVOs19oKf9Ik+JMnZzJdDgdCbWS+D/hGsBNV7AKq2GUnvl5LC4rT9ETJWQInAKBATLFCsaPEjpWbZPAE6uGzJlfsRh+5KfZVgCltJWzpDd9k//DKThw6O3t5ll1GUAOtPkEVOwBYPL0dP7/iWADAjb95AQBQrdmJ3W5DCdzRX4lVixDoq9Tw3KZecB469c4m+Zw0YkejYhsOnjATFFMlKHHTEYcePJE+B+Vj1/hYtPKEjRnRY2oLnhDbuI6vXlJs5A9k2aHY7k1oil2PXU1ohFCMFiaCYtdn7KO4fl0J1ClGKt1J4hiFAhbcegsYY3uNPBZnz4E/eTKaDjkYLMEU60Hd87L8NOW2ra0oLVyorfNSUp0I+N168ISYi+ynuSkeKZsTu1HH+Llr5ciEb6960bluW18Ff3/LH/D4yzvksiTFjtaKNSFMoTWHX48tKtYkkXLMSh3PbgqTYB4wrV0LmLBFxdJSXGqdtWsJk9jRfRptc5pnMcUmqXFpkalJoP5JNh8s2ueUNrcp1jXy3jZpv534t63euHvE+7ehn/wGXKZYsatZCMVooa+y7yt2Jhr6bYyCKRYIAw/MKgwjCb+tFfvffRfmXH99oimWAdIcm+X4NB14IFqOOgrd578Vfns7CkS181OSE8t2NHgiihCmuexYuQles15uz2vNTbGjjZzYjXPU6oHmd3LfnzY72/7XE+vwvYdewr/9+jm5LOZjR02xQQKxK4TLXFGxJrHb0V+VSY1N9FfqWL2hFwCwaFqbRuZsplhb7drGfezU59H2sROpM6a0lTP5zyXVd0wDVezMdDaAfhxsit3U9jI8plcD0edm/zxSOPnAabj0uAUAgOMOmDLyA1iwZlOv/NxtUzGxZ36Pewv9Q/uuYvfO5SHBP+OQ6dpyUW4sSwnAUcljN0rwyuUwmMF3ELvWUCXzI9Uuy2XqNTVh3n9+B1MuvxwAUN5PqXZeSjkxgQJNdxLlxWPNRLFrKoM15YrdWCM3xY5zbOurgIpR4XduJTqbI/+Ul7epjOSmyXRAC54I/9oeXiLjvpnH7v5nt+DjtzwhK0aUCx6GagF2DFRjyYzFuv5KHX+K1JhF09pRImxDrzwRPUyNEmeuOVLEEhRrTv+Jm444jj9gKm5611FYOrsLP330FQApCYqln1vjJIKavwercRKuRcVafOz+4+KjsKV3yPngHI18gFedeSBOWjINC6eOzgPg5e0DqW08RzTmWGJfVuyueuOBeMPiqVi2YLK2/G9XHIBj9p+CYzOQfk/LY7dvPNqSTLFASOyqGN5vs7RwP/Q9sCrspyObe4+ex87iY9es8tgJ5OlORh+5YjfOIQIORD6rWsBjgQsC2/tDErd2h3pwmcoa9bGjeexMCPJlkrVbH1uLl7cN4NlI9ZjdHf6od/RXYmPR9BpPRKlUFk1rB2MMr5nbhZmdTZjWqQjF1PYypnc04bDZnTES0mhJsb1d4zQJnsdwwuIeTGot4eBZHfA9hqUJfpFm1vjhwqbY0R5tpth5k1tx5LxJseXm3Bjbe1GrjDEs328ypnU0pTceAXzsjCVgDPjIae5SWeMlKpZiX/axKxd8nLRkmhblDYQJmk9c0uP036UYjajY0UZ58SLN7FuaPx8A0Lp8OUr77YdC9GI8nHtHef/95Gc/o2JHfey4CI6gUbFllcdOLssVu1HHvvFasw9jc5Q6ZE53C17Y0oeBah3b+6rWnHbbopqtuwdr2DVYRUdTMWaK1XzsSK1YE8IUaxK7Pxl+ULO7W7Bmc1+s8gQQ5uBat3MQALAl2g/h0/Sjv16OWsBlTj0gNMve85ETUPQ9+B7DflNbsWZzH4D0G9fU9jJe3qYIreZjN4bmtKP3m4I/XH2q5gtnYk9MsRSDlkAXSra7HebWJDSSbuLVgjcsmoqn/uG0xHMiMJ4Uu+GcvwmFUYiKHW0Uurux/913ofLKK2CFAkpzwvJppfnzsd9//RLFz/wv0F8dFrGjARRexnQnwgQMAEFv+HJPgydCxU43xebpTkYfuWI3ziHMq1Pby9JHShA4E9uJL97ayNwkHuyiPuugJXjClp9MmGKrdY5KLcCPHnkZ2/oqeHZjr9ZOKXbVWNLi5pKv1TKd0laS+1DwvXjdUoTkTpCyNyxS2dPTcqj9v4tei9fNn4TvX/r6sP0YKnYm0giE9C3cQxIxZFHsdpBrpVRo/Oe+J+XOxjPSz0lEaMdB8MRP33s0jprfjRsvPmqspzKusS8qdkC4L+UFCySpo7j46AU4/eDpWDClcVWsvP/+aoyMRJiq9vXe8CWfteiKnRY84Xkq/UmOUcO+8VqzD4MSuy29Q1i7YwDbXcSuX6lma7cP4MAZHTJStae9jJ0DVc2cI/zSbKpEkZhiv3X/8/jC7avxugWTNMUPAOZMCt/WtvdVYv00FXy0lHy5zZLp2d4KBd6weCr+I0q7kvZ4XTK9Az+6fLn8vrdrnI4kRspR3zw3APbYvCksYPuSYpcF46XyBAAcOa8bP7786LGexrgHreO6t9KdjDf87YoDhr0tDYTgA+l+pyaC3siaIoInfD+qH0uIXkvLqCUez6EwMa7+VzEosdu4K7xxbe9LV+zW7RwA51wqdtM6mvDspl6rKdamxsiSYvUAdz2zCQDw8AvbYu32mxrK7LuHajCTbTQVPS2i929O3A+NYNkCdeN5bnNvQss49GjO8X1j2ZM8dhQ2H7sVB07DVWcuwRFzuy1bpGO81EwdbYzHPHY5kiGiNIFXf1TsaGHSOy/Crtt/hY6/+IuGt5Wm2EiRE0ETVLHzcjPsmCC/a41zCB+7qW1l6WOzzULsOOeaiXbt9gHsHqrJPHTTO8Mf22Alo49dJNVs7q3g9y9td85vRmcTWqOUG2ZQR7ng49j9w2i2vz/zQBy9X2PpLJqKPvaLIiWPaXBbSlbHOynZkzx2FLaoWM9juOz4/XDUfHeARBImt5ZQ8j15/UwUSEI7vi+dHARaua99yBS7NzHt4x/H/vf8GoXJk9MbR2hbcTIAYNI73g5A+dixiNgxEhWbB06MDXLFbpyDKnbdUY3VNZt7cdwX7sapB03HJ/8irPE6UK2jQpznX9kxIP3suluKMkKVmmKTEhSLxKz3rt4UK9fV2VyUwRJt5QKmdTbh+SjIgaKp6OGac4/Aup0DOHhm9moZFLesPAbPberFEXO6GtqOktVxLtillvXKCptit6foainhv95/rPTRnCgQ58LPEI2ZY3yAeV5ogq3V9pl0J6OBRk2ls778ZVSefx7lxWFUuUhQrBQ7YorNU52MCfK71jjHFkrsInL240dewcvbBnDD/S/gle1hzjpTxVu7XRG7mV3NMlDBaoq1/LBFupPHozQlNM/ZKQepjOVtTQVMd/hxNRV9dLeWhk3qAKCjqYjXzO1u+OYzXqJis0BY+/ZUsRtylH/bUxwwrR09o5SKZLzg9Qsno61cwOGzu8Z6KjkagFDq9pWo2PEIr1RC05Il8p4sgiOEUkfz2Pm5YjcmyIndOIctKpaW+freQy8BALb36elG1u4YkPnsZnU1ywoFlNgFSSXFDKXiI6eqnF+nEmLXnkDsysOIwhwpUFPsePexExHIwy1fJRz8Z3Xl0WcjhUuOXYDHP31qQ3WZc4w9vHL4AspKeWqY0ULzIYfAnzIFbccdBwB68EReTmxMkL/WjCPc/+wW3PnMRhx3wBScfOA0DFTqskzX1PaytezT9x9+CUHApaI2raOMjbuGsHn3EF7YEppHZ3U3ozkidoPVOjjn+Onv1uLB58NgCGu6E4NknLl0BjbuGkRfpY5TDpqGdx+zAC0lH+WCj2kO/ytbOpPRghY8Mc597C543Vz0V+pYceC09MYW3LryGFx755/wkdOWjPDMJjbGu29mjjgmX3opBlf/EaUFC8Z6KhMGhSlTcMB994amcBjBE7liNybIFbtxhP97bjNueuBF3BlFoQrFranoob1cwCQLsdvRX8U37nse//hfzwAAFk5pk+rcA2u2AAiVHEGy+it1XHvns/jwjx+X5lubqZKW/ZrV1Yy2cgHvO/mAKGs/w6fOOggfjjL3U8WO1iMtjyGxaykV4HtMVuwYz1i+32R8652vxcxhKm6HzOrEt955FBZPz5Y9PkeOfRWTL3k3Zn3hC5Jk5Bgd0OM9XoMnrr76ajDGtP9LlqiX4cHBQaxcuRKTJ09GW1sbzjvvPGzcuDGxT845PvWpT2HGjBlobm7GihUr8Oyzz+7tXUlFfvWPIwh/nsdf3gEA+MPa8O9BMzrAGENXi+7A/s/nHYqLj56vLZvUWsLBM8N8cX/aqMp+CbJ3z+rN+Opd6RceVewWTUuW02muNJGwGBhbU2xncxHXve0IfP3tR47ZHHLkyJFjokEPnhhfptiDDz4Y69evl//vv/9+ue7KK6/EL37xC/z4xz/Gvffei3Xr1uHcc89N7O8LX/gCvva1r+HrX/86HnroIbS2tuK0007D4ODg3t6VRIx/OWMC4fC5XQCA1Rt3Y6BSx+Mvh4ELh0URoVQNA4AzDp2BvzxyDn76u1dkvrru1iJmdDbhty+qFCWzulqwabd+ob3j9fPw8AvbsHrjbsyf0gIT1MdOlAFzgabCmNpWRqngoVILxtQUCwCnHzJjTMfPkSNHjokGb5wqdgBQKBQwffr02PKdO3fihhtuwM0334yTTjoJAHDjjTfiwAMPxIMPPojXv/71sW0457j22mvxiU98AmeffTYA4Dvf+Q6mTZuGW2+9Feeff/7e3ZkE5IrdOML0jib0tJdRDzieWrcTj0XK3eERsaO1Iie3ltDRVITnMRxGIvcmtZQkERSY1d2slfYCgEuPW4hfvv9Y3PWhN2D5wngOI0rsDkgjdkSxa28qyNQYTcX88sqRI0eOiQQ2yulO+io17B6syv9DNXfap2effRYzZ87EwoULceGFF+Kll8Lgw0cffRTVahUrVqyQbZcsWYK5c+di1apV1r5eeOEFbNiwQdums7MTy5Ytc24zWhjTJ+8111yDo446Cu3t7ejp6cE555yD1atXW9tyznHGGWeAMYZbb71VLn/88cdxwQUXYM6cOWhubsaBBx6Ir371q6ljz58/P2Zv//znPz9SuzYsMMYkKfvti9vx9LpdABSxay75kizNm6xUtsPmqMi9rpaSbA8AzUUf3S1FGTwBAAumtGLu5BYUfQ/7TW2zphIpNmCKndJWksEKHc1FdER+bU2FsVXscuTIkSPH6IKVSjIR5GikOznlX3+LQ6/+X/n/33+9xtpu2bJluOmmm3D77bfj+uuvxwsvvIDjjjsOu3fvxoYNG1AqldDV1aVtM23aNGzYsMHan1g+bZoe9Ja0zWhhTE2x9957L1auXImjjjoKtVoNV111FU499VQ8/fTTaDUuiGuvvdZKQB599FH09PTgu9/9LubMmYMHHngAl112GXzfxxVXXJE4/mc+8xlceuml8nt7+9g7nx8+pwt3PL0RP/ztS6jUA3S1FDF3kiJxk1pKWLdzEPMnq+OjKXatJczubsak1hK29VUwq7sZjDGN2L1h0dTUeewaUFUk9u9JJnYF34tKng2hvamASa0lrNnch9ZyTuxy5MiRYyKBMQbW3Aze3z8qPnZ3vO8ozJw5S34vOXy7zzjjDPl56dKlWLZsGebNm4cf/ehHaG7et1JFjSmxu/3227XvN910E3p6evDoo4/i+OOPl8sfe+wxfOlLX8IjjzyCGTN0v6l3v/vd2veFCxdi1apV+NnPfpZK7Nrb2632dhuCSgW8opIA1/vilRZGAoKkvbi1X36nhLa7NSR28wixowpdueCFyt/sTvx69WaZ24yaYrMQuxe2qv1rKaVfJtM7mrBx1xA6moq44qQD8MvH1+HYA9LHyZEjR44c+xa8pibU+/tHxceutVRAe1PjlXG6urqwaNEiPPfcczjllFNQqVSwY8cOTbXbuHGjkyOI5Rs3btR4ycaNG3H44Yc3PJ+RxLhygtq5MwwWmDRJ1bXs7+/H2972Nlx33XWZSdjOnTu1Plz4/Oc/j8mTJ+OII47AF7/4RdRqNWfbrd/4Jv702qPk/+fPPDPTXBrFobM7tTeOo+brxdsFUaOpLWhVgDmRunfUgnD/95savjF1NSv/vGUL04/NUfPCcc1IXBcWRuNM72zCGxZNxRf/8jC0lfPYnBw5cuSYaChMCWt7F3qGl5tzNNDb24s1a9ZgxowZOPLII1EsFnHXXXfJ9atXr8ZLL72E5cuXW7dfsGABpk+frm2za9cuPPTQQ85tRg18nKBer/M3vvGN/JhjjtGWX3bZZfySSy6R3wHwW265xdnPb37zG14oFPivfvWrxPG+9KUv8V//+tf88ccf59dffz3v6uriV155pXt+Q0O8tnu3/P/iH//IAfCXX3452w42gAee28Kv+/Wz/NsPvMB7B6vaule29/OfPvoyr9UDbfkz63fy//nDOvm9b6jKf/jbl/i23iG57N7Vm/ijf96WaQ5D1Tr/2e9e5ut3DGRqv2HnAP/poy/zoWo9U/scOXLkyLFvYmD1ar7zf/93r47x8ssvN/QM/tCHPsTvuece/sILL/Df/OY3fMWKFXzKlCl806ZNnHPOL7/8cj537lx+991380ceeYQvX76cL1++XOtj8eLF/Gc/+5n8/vnPf553dXXx2267jT/xxBP87LPP5gsWLOADA9mem3sL44bYXX755XzevHnaSbrtttv4/vvvz3fv3i2XJRG7P/zhD3zKlCn8s5/9bMPj33DDDbxQKPDBwcFM7Ru9qHLkyJEjR44cI4NGn8Fvfetb+YwZM3ipVOKzZs3ib33rW/lzzz0n1w8MDPC/+Zu/4d3d3bylpYW/+c1v5uvXr9f6AMBvvPFG+T0IAv7JT36ST5s2jZfLZX7yySfz1atXj8j+7QkY55wnCHqjgiuuuAK33XYb7rvvPiwgpWA+8IEP4Gtf+xo8ktW6Xq/D8zwcd9xxuOeee+Typ59+GieeeCLe85734J/+6Z8ansNTTz2FQw45BH/84x+xePHi1PavvPIK5syZg5dffhmzZ89ueLwcOXLkyJEjx/CQP4PdGFMnKM453ve+9+GWW27BPffco5E6APjYxz6G97znPdqyQw89FF/5yldw1llnyWVPPfUUTjrpJLzzne8cFqkDwgANz/PQ09MzrO1z5MiRI0eOHDnGGmNK7FauXImbb74Zt912G9rb22Xul87OTjQ3N2P69OnWgIm5c+dKEvjkk0/ipJNOwmmnnYYPfvCDsg/f9zF1ahiV+fDDD+Oiiy7CXXfdhVmzZmHVqlV46KGHcOKJJ6K9vR2rVq3ClVdeibe//e3o7u6OjZcjR44cOXLkyPFqwJgSu+uvvx4AcMIJJ2jLb7zxRlx88cWZ+vjJT36CzZs347vf/S6++93vyuXz5s3Diy++CCCMrF29ejWq1SoAoFwu4wc/+AGuvvpqDA0NYcGCBbjyyivxwQ9+cI/3KUeOHDly5MiRY6wwLnzsXo3I7fs5cuTIkSPH2CB/BrsxrvLY5ciRI0eOHDly5Bg+cmKXI0eOHDly5MixjyAndjly5MiRI0eOHPsIcmKXI0eOHDly5MixjyAndjly5MiRI0eOHPsIcmKXI0eOHDly5MixjyAndjly5MiRI0eOHPsIcmKXI0eOHDly5MixjyAndjly5MiRI0eOHPsIcmKXI0eOHDly5Mixj2BMa8W+mhEEAQBg/fr1YzyTHDly5MiRY2JBPHvFsziHQk7shomNGzcCAF73uteN8Uxy5MiRI0eOiYmNGzdi7ty5Yz2NcQXGOedjPYlXI2q1Gn7/+99j2rRp8LyRs2jv3r0bBx10EJ5++mm0t7ePWL/7EvJjlI78GKUjP0bpyI9ROvJjlA0jfZyCIMDGjRtxxBFHoFDINSqKnNiNM+zatQudnZ3YuXMnOjo6xno64xL5MUpHfozSkR+jdOTHKB35McqG/DiNHvLgiRw5cuTIkSNHjn0EObHLkSNHjhw5cuTYR5ATu3GGcrmMT3/60yiXy2M9lXGL/BilIz9G6ciPUTryY5SO/BhlQ36cRg+5j12OHDly5MiRI8c+glyxy5EjR44cOXLk2EeQE7scOXLkyJEjR459BDmxy5EjR44cOXLk2EeQE7scOXLkyJEjR459BDmxG0e47rrrMH/+fDQ1NWHZsmV4+OGHx3pKY4arr74ajDHt/5IlS+T6wcFBrFy5EpMnT0ZbWxvOO+88WeZtX8V9992Hs846CzNnzgRjDLfeequ2nnOOT33qU5gxYwaam5uxYsUKPPvss1qbbdu24cILL0RHRwe6urpwySWXoLe3dxT3Yu8i7RhdfPHFsevq9NNP19rs68fommuuwVFHHYX29nb09PTgnHPOwerVq7U2WX5fL730Et74xjeipaUFPT09+MhHPoJarTaau7LXkOUYnXDCCbFr6fLLL9fa7MvH6Prrr8fSpUvR0dGBjo4OLF++HP/zP/8j10/0a2gskRO7cYIf/vCH+OAHP4hPf/rT+N3vfofDDjsMp512GjZt2jTWUxszHHzwwVi/fr38f//998t1V155JX7xi1/gxz/+Me69916sW7cO55577hjOdu+jr68Phx12GK677jrr+i984Qv42te+hq9//et46KGH0NraitNOOw2Dg4OyzYUXXoinnnoKd9xxB375y1/ivvvuw2WXXTZau7DXkXaMAOD000/Xrqvvf//72vp9/Rjde++9WLlyJR588EHccccdqFarOPXUU9HX1yfbpP2+6vU63vjGN6JSqeCBBx7At7/9bdx000341Kc+NRa7NOLIcowA4NJLL9WupS984Qty3b5+jGbPno3Pf/7zePTRR/HII4/gpJNOwtlnn42nnnoKQH4NjSl4jnGB173udXzlypXye71e5zNnzuTXXHPNGM5q7PDpT3+aH3bYYdZ1O3bs4MVikf/4xz+Wy5555hkOgK9atWqUZji2AMBvueUW+T0IAj59+nT+xS9+US7bsWMHL5fL/Pvf/z7nnPOnn36aA+C//e1vZZv/+Z//4Ywxvnbt2lGb+2jBPEacc/7Od76Tn3322c5tJtox4pzzTZs2cQD83nvv5Zxn+33993//N/c8j2/YsEG2uf7663lHRwcfGhoa3R0YBZjHiHPO3/CGN/C//du/dW4z0Y4R55x3d3fzb33rW/k1NMbIFbtxgEqlgkcffRQrVqyQyzzPw4oVK7Bq1aoxnNnY4tlnn8XMmTPx/9u519ioyjwM4E+Z9gyFUoZxpjPDpeO0QEtty6VKHYlkdQgwiVmE3bWgH7qwC3IzoqjcVhf4ICRrQJYElERodoPLEiJh10REKCXR1GqhQ0G31SlTaqXDZbAXmd6Y+e8HlrOOVOiutIeePr9kktN535553yfvaf89l6alpeGZZ55BXV0dAODkyZPo7OyMySszMxOpqan9Nq9AIIBgMBiTydChQ5Gfn69mUlpaCpPJhAcffFDtM23aNAwYMABlZWW9PmatlJSUICUlBRkZGViyZAlCoZDa1h8zampqAgCYzWYA3Tu+SktLkZOTA5vNpvaZMWMGmpub1TM2evLjjG7au3cvLBYLsrOzsWbNGoTDYbWtP2UUiUSwb98+XLt2DW63m2tIY/FaD4CAK1euIBKJxCxwALDZbKiqqtJoVNrKz89HUVERMjIy0NDQgA0bNuDRRx/F2bNnEQwGoSgKTCZTzPfYbDYEg0FtBqyxm/Puag3dbAsGg0hJSYlpj4+Ph9ls7je5zZw5E3PmzIHL5UJNTQ3Wrl0Lr9eL0tJSGAyGfpdRNBrFihUrMGXKFGRnZwNAt46vYDDY5Vq72aYnXWUEAE8//TScTieGDx+OyspKrFq1CtXV1XjvvfcA9I+Mzpw5A7fbjba2NiQlJeHgwYPIysqCz+fjGtIQCzu6J3m9XnU7NzcX+fn5cDqd2L9/PxITEzUcGfVlc+fOVbdzcnKQm5uL9PR0lJSUwOPxaDgybSxbtgxnz56NuX+VYv1URj+87zInJwcOhwMejwc1NTVIT0/v7WFqIiMjAz6fD01NTThw4AAKCwtx4sQJrYfV7/FS7D3AYrHAYDDc8sTQxYsXYbfbNRrVvcVkMmHs2LHw+/2w2+3o6OhAY2NjTJ/+nNfNed9uDdnt9lsexrl+/TquXr3ab3NLS0uDxWKB3+8H0L8yWr58Od5//30cP34cI0eOVN/vzvFlt9u7XGs32/TipzLqSn5+PgDErCW9Z6QoCkaPHo28vDxs2rQJ48ePx7Zt27iGNMbC7h6gKAry8vJw7Ngx9b1oNIpjx47B7XZrOLJ7x/fff4+amho4HA7k5eUhISEhJq/q6mrU1dX127xcLhfsdntMJs3NzSgrK1MzcbvdaGxsxMmTJ9U+xcXFiEaj6i+l/qa+vh6hUAgOhwNA/8hIRLB8+XIcPHgQxcXFcLlcMe3dOb7cbjfOnDkTUwR/9NFHSE5ORlZWVu9MpAfdKaOu+Hw+AIhZS3rOqCvRaBTt7e1cQ1rT+ukNumHfvn1iNBqlqKhIvvzyS1m0aJGYTKaYJ4b6k5UrV0pJSYkEAgH55JNPZNq0aWKxWOTSpUsiIrJ48WJJTU2V4uJiKS8vF7fbLW63W+NR96yWlhapqKiQiooKASBbtmyRiooKOX/+vIiIbN68WUwmkxw6dEgqKytl1qxZ4nK5pLW1Vd3HzJkzZeLEiVJWViYff/yxjBkzRubNm6fVlO6622XU0tIiL730kpSWlkogEJCjR4/KpEmTZMyYMdLW1qbuQ+8ZLVmyRIYOHSolJSXS0NCgvsLhsNrnTsfX9evXJTs7W6ZPny4+n08OHz4sVqtV1qxZo8WU7ro7ZeT3+2Xjxo1SXl4ugUBADh06JGlpaTJ16lR1H3rPaPXq1XLixAkJBAJSWVkpq1evlri4ODly5IiIcA1piYXdPWT79u2SmpoqiqLI5MmT5dNPP9V6SJopKCgQh8MhiqLIiBEjpKCgQPx+v9re2toqS5culWHDhsmgQYNk9uzZ0tDQoOGIe97x48cFwC2vwsJCEbnxL09effVVsdlsYjQaxePxSHV1dcw+QqGQzJs3T5KSkiQ5OVnmz58vLS0tGsymZ9wuo3A4LNOnTxer1SoJCQnidDpl4cKFt/zxpPeMusoHgOzZs0ft053jq7a2VrxeryQmJorFYpGVK1dKZ2dnL8+mZ9wpo7q6Opk6daqYzWYxGo0yevRoefnll6WpqSlmP3rOaMGCBeJ0OkVRFLFareLxeNSiToRrSEtxIiK9d36QiIiIiHoK77EjIiIi0gkWdkREREQ6wcKOiIiISCdY2BERERHpBAs7IiIiIp1gYUdERESkEyzsiIiIiHSChR0RERGRTrCwIyLqpmtln+FfmeMQaW7WeihERF1iYUdERESkEyzsiIiIiHSChR0R9RkSjeLK27vg90xD1fgJODfrSTQf/hDAfy+TtpSU4NwvZ6EqdzwCBQVo++qrmH00f3gENU88gaqcXPgf9yC0e09Me7SjA5feeANf/+KxG32mz0DjgQMxfdq++AKBX/0aVRMmonbuPLSfC/TsxImIuile6wEQEXVXaNcuNP3jn7CvXw/lfifCn5fjwiuvwGAepva59Kc3YFu7BvEWKy5v3Yr6JUuRfvgDxCUkoPXsF/j2hRdgWb4MyV4vWit8CG7cCIPJBNOc2QCAC6tWodV3GrZ1azEwMxOd9fWIfPddzDguvfkmUla9gnizGQ3r16Nh3Trc/7d3ezULIqKusLAjoj4h2tGBK2/vQurudzBo4kQAgDJqFMKnTqLx7/theuopAIB12VIkTZkCABi+eRO+/sVjaDl6FMleL64WFWHwww/DunQpAMDocqG9xo/Q7ndgmjMb7YEAWj44jNTd72DwI4+on/FjKStWYPDkyQAAy8KF+ObZxYi2t2OA0djjORAR3Q4LOyLqEzrPn4e0tqLud7+PeV86OzFw3Dj168QJE9Rtg8kExeVCe805AED7uRoMedwT8/2DJk3C1b/8FRKJoL2qCjAYMOihh247FmNGhrodb7UCACKhEAYMH/5/zY2I6G5hYUdEfUI0HAYAjHprJxJstpi2OEVBR903P/sz4owDu9cv/gc/OuPiAAASlZ/9+UREPxcfniCiPkFJH404RcH1hgYoTmfMK8HhUPu1nj6tbkeamtBRWwtjehoAwJiWjtZTp2L2Gz51Csb7nYgzGGAcOxaIRhH+/PPemRQR0V3GM3ZE1CcYkgbDvGA+Lm7aDIkKBuVNQqSlBa2nKjAgKQkJ/7kMemXHDhhMJhjuuw+X39wGwzAThnhuXH41z/8tan/zFC7v2HHj4QnfaXy3913YX3sNAKCMHIGhTz6JC+v+APu6tTBmZqLz2wuIXA0h2evVbO5ERN3Fwo6I+gzr888j3mxGaNcuNNTXwzBkCAZmZcHy7CL1Uqj1xRdx8fXX0VF7HsZx4zBq507EKQoAIPGBBzBi61Zc3v5nXNn5FuKtFlife059IhYA7Ov/iMtbtiK4YSMijY2IH+6AZdGzmsyXiOh/FScivDGEiPq8a2Wfoa6wEGM/K4MhOVnr4RARaYL32BERERHpBAs7IiIiIp3gpVgiIiIineAZOyIiIiKdYGFHREREpBMs7IiIiIh0goUdERERkU6wsCMiIiLSCRZ2RERERDrBwo6IiIhIJ1jYEREREenEvwG0dxN5molUUgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "cum_loss_list=load_list_from_file(model_name.replace('_','-') + \"-loss.pkl\")\n", + "acc_epoch=load_list_from_file(model_name.replace('_','-') + \"-acc.pkl\")\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "8779d43e-2ae8-4f27-b90a-d436904f4d9e", + "metadata": {}, + "source": [ + "Here, you load the model that has been trained for you. Please comment out these lines if you want to train the model yourself.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "cafb242d-1968-48f4-973f-b1894c138e8e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.load_state_dict(torch.load(model_name.replace('_','-') + \".pth\", map_location=device))\n", + "model.eval()" + ] + }, + { + "cell_type": "markdown", + "id": "0f52df3f-cac4-48ef-abec-7317fcc02dec", + "metadata": {}, + "source": [ + "The following evaluates the model on the test data. The pretrained model achieves an accuracy of 66%.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "a80d646f-83df-4a88-b498-b3726935559c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "66.216" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader , model, device)" + ] + }, + { + "cell_type": "markdown", + "id": "c8cc8cd8-04f9-4089-be92-91927d136ce4", + "metadata": {}, + "source": [ + "## Low-Rank Adaptation (LoRA)\n", + "\n", + "PyTorch and the Hugging Face library provide robust tools for model manipulation with LoRA, but they are not intuitive. In this section, you delve into building a LoRA (Low-Rank Adaptation) implementation from scratch using PyTorch. LoRA is a general method, but it's commonly applied to the Attention layer. For the sake of simplicity, in this lab, you apply it to a Vanilla neural network. This decision is made because accessing the Attention Parameters in the PyTorch Encoder module can be challenging.\n", + "\n", + "### LoRA\n", + "1) For any arbitrary layer of a network, you have the model with pretrained parameters $ W_0 $, which are the parameters of the model. If you only consider the attention parameters for each layer, at a minimum $ 4 \\times m \\times n$ for each layer. For many models, this can reach in the trillions of learnable parameters. Each time you fine-tune a new dataset, you have to store trillions of parameters.\n", + "\n", + "2) $ \\Delta W $ represents two matrices $ B $ and $ A $, where $ B $ and $ A $ are constrained such that $ B \\in \\mathbb{R}^{m \\times r} $, $ A \\in \\mathbb{R}^{r \\times n} $, and $ r \\leq \\min(m, n) $. The total number of parameters is $ A $ and $ B $ is much smaller than $ W_1$ and much easier to store.\n", + "\n", + "$ W_1\\approx W_0 + \\Delta W = W_0 + BA $\n", + "\n", + "\n", + "\n", + "3) To train and predict, the forward pass holds $W_0$ constant.\n", + "\n", + "$h = W_0 + \\Delta W = W_0x + BAx $\n", + "\n", + "\n", + "To scale $\\Delta W \\times \\dfrac{\\alpha'}{r}$, where $\\alpha$ is a constant in $ r $. Adjusting $\\alpha'$ is similar to tuning the learning rate if the initialization is properly scaled. Therefore, you set $\\alpha'$ to the first $ r $ you try and do not tune it further; just use $\\alpha$. This scaling reduces the need to retune hyperparameters. The final form is:\n", + "\n", + "$h = W_0x + \\dfrac{\\alpha'}{r} BAx= W_0x + \\alpha BAx $\n", + "\n", + "The following example illustrates the process.\n", + "\n", + "\n", + "$\n", + "W_0 + BA = \n", + "\\begin{bmatrix}\n", + "w_{11} & w_{12} & w_{13} & w_{14} \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n", + "w_{21} & w_{22} & w_{23} & w_{24} \\\\\\\\\\\\\n", + "w_{31} & w_{32} & w_{33} & w_{34} \\\\\\\\\\\\\n", + "w_{41} & w_{42} & w_{43} & w_{44} \\\\\\\\\\\\\n", + "\\end{bmatrix} +\n", + "\\begin{bmatrix}\n", + "a_1 \\\\\\\\\\\\\n", + "a_2 \\\\\\\\\\\\\n", + "a_3 \\\\\\\\\\\\\n", + "a_4 \\\\\\\\\\\\\n", + "\\end{bmatrix}\n", + "\\begin{bmatrix}\n", + "b_1 & b_2 & b_3 & b_4 \\\\\\\\\n", + "\\end{bmatrix}\n", + "$\n", + "\n", + "This illustrates the product of matrices $ A $ and $ B $, denoted as $ AB $, which can be added to $ W_0 $. However, the resulting matrix $ W_0 + AB $ is limited depending on the dimensions of $ A $ and $ B $. This limitation is due to the concept of rank.\n", + "\n", + "### Rank\n", + "The rank of a matrix is the number of dimensions the rows of the matrix \"live in.\" A square matrix is said to be **full rank** if its rank is equal to the number of its rows or columns. Let's make this idea more intuitive with an example.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "f5054a80-14ad-4f3a-86f6-4b2475dfd125", + "metadata": {}, + "outputs": [], + "source": [ + "from sympy import Matrix, init_printing,Symbol\n", + "from numpy.linalg import qr,eig,inv,matrix_rank,inv, norm\n", + "from scipy.linalg import null_space\n", + "from sympy import Matrix, init_printing,Symbol\n", + "init_printing()" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "0fa127f4-d7bd-44f2-a22b-5e4723f1ec42", + "metadata": {}, + "outputs": [], + "source": [ + "def plot_matrix_and_subspace(F):\n", + " assert F.shape[0] == 3, \"Matrix F must have rows equal to 3 for 3D visualization.\"\n", + " \n", + " ax = plt.figure().add_subplot(projection='3d')\n", + " \n", + " # Plot each column vector of F as a point and line from the origin\n", + " for i in range(F.shape[1]):\n", + " ax.quiver(0, 0, 0, F[0, i], F[1, i], F[2, i], color='blue', arrow_length_ratio=0.1, label=f'Column {i+1}')\n", + "\n", + " if F.shape[1] == 2:\n", + " # Calculate the normal to the plane spanned by the columns of F if they are exactly two\n", + " normal_vector = np.cross(F[:, 0], F[:, 1])\n", + " # Plot the plane\n", + " xx, yy = np.meshgrid(np.linspace(-3, 3, 10), np.linspace(-3, 3, 10))\n", + " zz = (-normal_vector[0] * xx - normal_vector[1] * yy) / normal_vector[2] if normal_vector[2] != 0 else 0\n", + " ax.plot_surface(xx, yy, zz, alpha=0.5, color='green', label='Spanned Plane')\n", + "\n", + " # Set plot limits and labels\n", + " ax.set_xlim([-3, 3])\n", + " ax.set_ylim([-3, 3])\n", + " ax.set_zlim([-3, 3])\n", + " ax.set_xlabel('$x_{1}$')\n", + " ax.set_ylabel('$x_{2}$')\n", + " ax.set_zlabel('$x_{3}$')\n", + " #ax.legend()\n", + "\n", + " plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "0718ea57-d8af-4bec-9608-d425656912dd", + "metadata": {}, + "outputs": [], + "source": [ + "def plot_matrix_and_subspace(F):\n", + " assert F.shape[0] == 3, \"Matrix F must have 3 rows to represent 3D space.\"\n", + "\n", + " ax = plt.figure().add_subplot(projection='3d')\n", + " \n", + " # Plot each column vector of F\n", + " for i in range(F.shape[1]):\n", + " ax.quiver(0, 0, 0, F[0, i], F[1, i], F[2, i], color='blue', arrow_length_ratio=0.1, label=f'Column {i+1}')\n", + "\n", + " # Calculate the null space of the transpose of F\n", + " normal_vector = null_space(F.T)\n", + " \n", + " # Check that the null space is 1-dimensional\n", + " if normal_vector.shape[1] == 1:\n", + " normal_vector = normal_vector[:, 0] # Simplify the array to 1D\n", + " # Create a meshgrid for the plane\n", + " xx, yy = np.meshgrid(np.linspace(-3, 3, 10), np.linspace(-3, 3, 10))\n", + " # Calculate corresponding z coordinates based on the plane equation ax + by + cz = 0\n", + " zz = (-normal_vector[0] * xx - normal_vector[1] * yy) / normal_vector[2] if normal_vector[2] != 0 else 0\n", + " ax.plot_surface(xx, yy, zz, alpha=0.5, color='green', label='Spanned Plane')\n", + " else:\n", + " print(\"The null space is not 1-dimensional, so a unique plane cannot be determined.\")\n", + "\n", + " # Set plot limits and labels\n", + " ax.set_xlim([-3, 3])\n", + " ax.set_ylim([-3, 3])\n", + " ax.set_zlim([-3, 3])\n", + " ax.set_xlabel('X axis')\n", + " ax.set_ylabel('Y axis')\n", + " ax.set_zlabel('Z axis')\n", + " #ax.legend()\n", + "\n", + " plt.show()\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "82792cf9-222a-4d02-8b3b-cf4b124b0dc5", + "metadata": {}, + "source": [ + "In the context of Low-Rank Adaptation (LoRA), where $B \\in \\mathbb{R}^{d \\times r}$, the matrix $B$:\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "10fb63a2-c601-4911-a5ad-d495cb935e48", + "metadata": {}, + "outputs": [ + { + "data": { + "text/latex": [ + "$\\displaystyle \\left[\\begin{matrix}1 & 0\\\\0 & 1\\\\0 & 0\\end{matrix}\\right]$" + ], + "text/plain": [ + "⎡1 0⎤\n", + "⎢ ⎥\n", + "⎢0 1⎥\n", + "⎢ ⎥\n", + "⎣0 0⎦" + ] + }, + "execution_count": 48, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "B=torch.tensor([[1,0],[0,1],[0,0]]).numpy()\n", + "\n", + "Matrix(B)\n" + ] + }, + { + "cell_type": "markdown", + "id": "6d7e2826-6a0d-4244-9e48-f1803873cc8c", + "metadata": {}, + "source": [ + "This $3 \\times 2$ matrix has columns that span a 2-dimensional subspace in $\\mathbb{R}^3$. Specifically, the columns of $B$ are:\n", + "\n", + "- $\\mathbf{b}_1 = \\begin{bmatrix} 1 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ 0 \\\\ 0 \\end{bmatrix}$\n", + "- $\\mathbf{b}_2 = \\begin{bmatrix} 0 \\\\\\\\ 1 \\\\ 0 \\end{bmatrix}$\n", + "\n", + "These columns are standard basis vectors for the $xy$-plane in $\\mathbb{R}^3$, and, thus, they span the $xy$-plane shown in green in the following image. Muliplying each column vector in blue by a scaler always falls in the plane.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "3c3e3709-0681-44c8-9301-36f3b52401f7", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAGRCAYAAABCCEUTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAD3V0lEQVR4nOz9e5Bk510ejj/n3vfr3C87O3vVaiWtdrVa7Y4MWMRXCMEGG2OKgJMqkiKQ/CjgD0iqXEUIIS5TCQlFGVIh2Alxkq8FwsY2GDDGsi1ZkVdz39ndmdnduU/PpXtm+t7n9vvj7Pvu6dOn72dmeqXzVKkk7c68fbr7nPd5P5/P83k+jK7rOly4cOHChYs2wR73Bbhw4cKFi7cHXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnHhwoULF47AJRQXLly4cOEIXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnHhwoULF47AJRQXLly4cOEIXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnHhwoULF47AJRQXLly4cOEIXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnHhwoULF47AJRQXLly4cOEIXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnHhwoULF47AJRQXLly4cOEIXEJx4cKFCxeOwCUUFy5cuHDhCFxCceHChQsXjsAlFBcuXLhw4QhcQnFxLNB1/bgvwYULFw6DP+4LcPHOgq7rkGUZhUIBHMeB53lwHAeO48AwzHFfngsXLtoAo7tHRRdHBE3TIMsyVFVFsVgEYBBMPp9HsVhEb2+vSzAuXDzGcCMUF4cOXdehqioWFxchiiL6+vrAsixYloWu60in09ja2kI0GkWxWATDMGBZFjzPuwTjwsVjBJdQXBwqSIpLVVUcHBzA4/FgeXkZa2trCIVCiEajUFUVAMDzPHRdp/8Ui0WUSiUAcAnGhYvHAG7Ky8WhQVVVyLIMTdPAsiympqZwcHAARVEwMjKCXC6HVCqFbDYLlmUxMDCAaDSKSCQCQRAAoIxgNE0DABrBCIJA6zAsy7oE48LFMcONUFw4Dl3XoSgKFEWBrutgWRZ7e3vY2tqCJEm4ceMGAIMYGIbB6uoqVldXoes6FhcXkcvlEAwGEYlEKMGQyMRMMIVCga5DCIb8nEswLlwcPVxCceEoNE2Doig0jcUwDBYXF3H//n1KEqIo0lQWYKS6BEHA+fPnAQDFYhGpVAp7e3uYn59HoVCoIBiS9qpGMCRycQnGhYujg0soLhwBSUnJsgxd18EwDIrFIqamplAoFPDCCy9gdXXVdlNnGKasL0WSJPT19aGvrw8AUCgUkEqlkEqlcOfOHRSLRVp/iUQiCIfDFQSjaZpLMC5cHDFcQnHRNswpLsDYwLe3tzE9PY2enh5cuXIFPM9XEAdBvY3d4/Ggv78f/f39AIB8Pk8JZmNjA6VSCeFwuC7BFItFFAoFqjBzCcaFC2fhEoqLtmDuLSEb8u3bt7G6uoqLFy9iYGCg7OeraUCa0YZ4vV54vV4MDAzQPhZCMGtra1AUhRJMNBpFMBgEx3H0dYiMmfTDrK+vo6+vDz6fj0YypL7jwoWLxuESiouWQDZlRVGoiiuXy2FychIAMDY2Br/fX/Y7DMNA07SKjbpa5NIIGIaBz+eDz+fD4OAgdF2n6rFUKoWVlRVomlZGMIFAADzP0/exuLiIaDRKScQuReYSjAsX9eESioumYe4tAYwekfX1ddy6dQvDw8M4d+4cWLbSJs5MHObN2cmNmmEY+P1++P1+DA0NQdd1ZLNZSjBLS0vQdZ0W+KPRKHRdB8dxEASBRjCKokCW5aoEY/f+XLh4p8MlFBdNQdM0lEolGpWoqopbt25he3sbzz77LLq7u6v+bq1I5LDaoRiGQSAQQCAQwPDwMHRdRyaToQRz//59AMCdO3fQ1dWFaDQKv99fFsFUIxjSB+MSjAsXBlxCcdEQSIqLqLhYlsXBwQEmJyfh8Xjw4osvwuPx1FyjVlH+qPprGYZBMBhEMBjEiRMnoGkavvnNbyIQCGB3dxeLi4vgOI4W+KPRKHw+X1WCAey7+F2CcfFOhEsoLurCmuJiGAZLS0uYn5/HqVOncOrUqYbSVq2qvA4TRN01ODgIn88HTdNwcHCAVCqF7e1tLCwsgOf5MoLxer0VBCPLck2bGJdgXLwT4BKKi5owRyUMw0CWZUxPTyOdTuPq1auIRqMNr1WLODrFAYhlWUQiEUQiEQCgHmSpVAqJRAJ3796FKIoVBEMIw9yPQyIYhmHKCIaoyFy4eLvBJRQXtiBpnbm5OQQCAQwMDCCZTGJqagqRSAQvvvgi9dtqFJ2Q8qqGaq9P0l+EOFVVxf7+PlKpFNbX13Hnzh1IkkR/JhqNQpKksnUJwZRKJVqDIQRjVpG5cPG4wyUUFxUgG6CmacjlchAEAQsLC3jw4AHOnz+P4eHhljbATiaURsFxHGKxGGKxGABAURRKMCsrK7h16xa8Xm8ZwYiiSH/fSjAAsLu7i97eXkiS5Dopu3is4RKKCwrzZkdUXLquY3l5GRzH4fr16wgGg22/hhWP8+bJ8zzi8Tji8TgAg2D29vaoRHl2dhZ+v7/Mh8xMMLIs49atW4hEIlAUxZ0F4+KxhksoLgDY95ZsbW1he3sbwWAQL7zwAu02bxXHIRs+avA8j66uLnR1dQEwCIMQzP3795HNZhEIBCjBBAIB+nvk8yXSbHfYmIvHDS6huCizT2FZFpqm4fbt21hfX6dpm3bJBHh7pLyahSAI6O7upv05pVKJEgyx6geAe/fuIRaLUat+4BHJugTj4nGBSyjvYNjZp2SzWUxOToJlWYyNjeH+/fuObfadKBs+6tcXRRE9PT3o6ekBAGSzWbzxxhtQFKXMqp+kx4hVP1BOMO40SxedCJdQ3qGw6y1ZW1vD3NwcTpw4gbNnz9IeDTIpsV1YrVfM5PJ2jVDqgSjlzp8/D47jWrLqtxuX7E6zdHEccAnlHQjraF5FUTA7O4tkMonLly/T/D/gbDqq2qb2dk55NQry2dSz6pdluSGCcadZujgOuITyDoJ1bgnLstjf38fk5CR8Ph9efPHFsh4K8jMkimkXnZryOk7UI9JaVv2rq6tQVdXWqt+dZuniOOASyjsEZjkwwYMHD7CwsIDTp09jdHS0oWmK7aCTVV7H/fqNWte0YtVfb5qlpmngeR5er9clGBdtwSWUtznsRvOWSiVMT08jm83i+eefpzYjdnCyhkKux+41jntDPy60875bseoPBAJgWbaMYIidzMjIiDvN0kVbcAnlbQy7wvvu7i6mpqYQi8UwNjZW1z6FNDc6AUIcmqZhY2MDHo8H4XDY3azgTNrPzqo/nU6X9cEwDFPmQ0aGoJE0mHWapZ1M2SUYF9XgEsrbFNbeEnISXV5exoULFzA4ONiWQ3ArYBgGqqrizTffRKFQoBtXMBiErus4ODhAMBh8R21WhxmZMQyDUCiEUChErfrT6TRSqVSZVT8RZuRyOVurfiItd6dZuqgHl1DeZrDrLcnn85icnISqqrhx4wbtzm4EThLKwcEBstksBgcHcenSJTAMg1wuh62tLezv72N8fJyeoMk/Pp/vbb9ZHdX7Y1kW4XAY4XAYAKhV/507d5DNZvHmm282ZNVfb5qlSzDvXLiE8jYCOeVvbm7ixIkTYFkWiUQCMzMzGBgYoL0OzcCJGoqmaTQ6EkURFy9epDWdQCAAQRDw4MEDvPjii7QGYJ1FEovFEI1G6w7xagXHufkdZ+2IWPX7fD5EIhEMDAxQq/7Nzc0yq35CMlarfneapQszXEJ5m4D0lmQyGTx48ADDw8O4desWEokEnnrqKfT19bW0brs1FHN0dOHCBSwtLdG/IyIBsqGbT9AnT56ks0iSySTW1tZw+/ZteDyeqk6+jyuO+zRPvodaVv3k87ez6q9HMO6wsXcOXEJ5zGHuLdF1HRzHQVVVvP766+B5HmNjY/B6vS2v307Ka2trC9PT0+jt7cWFCxeQSqVqrmX9O+sGZ+fkGwgEyk7QJD3zuKAT1G2EUKxo1arfSjDuNMt3Dh6vp89FGTRNg6IoZSqu7e1tyLKMEydO4PTp020/rK0QCklxrays4OLFixgYGKi5FtnM6r2O1cm3VCpRiazZB4ukx0KhkCOmloeN445QNE1r6Bpateq3m2ZpN2zMJZjHHy6hPIaw6y1RFAUzMzNIJpNgWRZnz5515LWaraHUEwA42SkviiJ6e3vR29tLX5sQzPr6OhRFoU1+sVispoKsEyKF40K1CKUemrXqj0QiZTJ18328sLAAj8eDvr4+d1zyYwyXUB4zWO1TGIbB3t4eJicnEQwGceXKFbz55puOvV4zNRRrissaHdSLdtrd1K02JeYu8uXlZQAoa/Lz+/3Hvlm1upl34jU0YtUfDAbLCIakKPP5PP1vcwRDmjDNRf7j/rxcVIdLKI8RzL0lJF1w79493Lt3D2fPnsXIyAgKhQI0TXNsk2gk5VUtxdXoWo2mvJqBXRe5tQeDKMiIHXy70ygfVxwWqVmt+ovFom2KMhqN0v82H0LsUmQuwXQ2XEJ5DGDXW1IsFjE9PY18Po9r167R3gLz5nwUhNJMj0stt+HDhrnJb2RkBJqm0QKzpmmYnp4uUzDFYrEjUZC9nSKUepAkCX19fVRxaLbqz2QyODg4wM7OToWTMrlGALYE486C6Ry4hNLhsBvNu7Ozg+npaXR1deHy5ctlyiZzAdQJ1Kqh1Etx2a11mCmvZsCyLCWP1dVVPPPMM1BVFclkEsvLy7h16xb8fn+ZgulxU5A1iuMiNbNVfz6fR1dXFwRBKKuBEat+q8jCnWbZmXh7PiFvE5CHhUQluq7jzp07WFlZwZNPPonBwcGK3yGEomnaoY3tbTTFVW0tayf1YaS8mgHpwYhEIlTBJMsyPT0vLi4in8/T9Ew0Gi07PbeDThADdEKUBDyKYJq16gfcaZadApdQOhAkxUVUXCzLIpfLYXJyErquY2xsjJr6WUEeFqccgq1F+cOwcTluQrGDIAhl+X9zemZubo4OuiIS5WAw2LLc9bg3uE4gFHJoImjFqr/aLBh3muXRwSWUDoNdimtjYwO3bt3C4OAgzp8/X3PjOsyUV7MpLru1ql1Xpz/U5vQMOT0nk0mqIDPbxMdisYYVZJ1Aop1AKOTgVA1OWfW70ywPFy6hdBCso3lVVcXc3By2trbwzDPP0NNyLZB0kpNz4DVNw+3bt5tOcdmhU2oo7cB8eiabWyaToZvb/fv3y2o0xGTxOEUJtdAJhNJocyVBq1b9tQhG0zRkMhn09PS4BNMiXELpAFh7S1iWRTqdxuTkJERRxIsvvtiUKaKThCLLMmRZxu7ubtMpLrvrIqRhfUiPe8hWu4OugsEggsEgtYknJouJRIKaLJL0GPHAavd1nUK96OAoYE15NYtGrfrNBOPz+coIJpvNYmpqCu9617vomu645ObgEsoxwzqal2EYLC8v4+7duxgdHcXp06ebvoGdGoq1tbWFqakpMAyD69evt12EfpxTXs2AuPhGIhGMjo5CVVV6ciYeWD6f78ikyfXQKRGKk6RWzao/lUpha2urzMmakAwRsgiCQKMX87hkd5plfbiEckwgN+va2hq2t7eppfvMzAwODg7w3HPPUVO+ZsGybFsRilnFdfr0aXq6axfm4rsdsXTCaf0wwHFcmQeW2aJkfX0dxWIRb775ZtnmdpQeZJ1AKIcdJdmRPCGYjY0N3LlzBzzP02mixKofQFmKjAyFKxQKLsHYwCWUY4C58C7LMi0uTk1NIRQKYWxsrK2Tazspr3w+j4mJCWiahhs3bgAAFhYWWr4W63XV+ru3K6FYYbYo6e7uxtzcHIaHh5FKpXD79m2USqUy9VIoFDrUzbYTCKXZGkq7sLPqX11dxYMHD5qy6reOSyYpMrMP2XF/tkcJl1COGNbRvBzHIZfL4ebNmzh37hxOnDjR9g3YasrLTsWVy+UcVYwBzhpEPu4gmznpILfrv9A0rUK95OTn1QmEctx1HI7j4Pf74fF4cPXq1Zat+t/p0yxdQjkiVLNPWVxchCzLuH79OkKhkCOv1WyEUqtRkUQOTmw69fpNjrOx8ThhbfK09l9ks1kqUSYKMrNEuZaCrBEcN6GQ9O9xfw/mOo7Vql+WZUowjVr1VyOYt/M0S5dQjgB2vSXb29uYnp5GKBSCpmmOkQlZv1FCsaa4rCouJ73B6kUo75SUlxn13rNZHmtVL5HisiAIZR5kREHWKI57MyefwXFvrrWEAYIgtGTV/06bZukSyiHD2lui6zpu376NtbU1PPnkk/B4PJiennb0NRtNeTXSqOhko6Sb8rJHM+/dbkyyeUzv3NwcfD5fWWrGPIPEimoy7qMEOfwc92ZK0tCNwM6qP5VKYW9vr6ZV/9t9mqVLKIcEu96SbDaLyclJMAyDsbEx+Hw+7O3tOdYzQlAv5dWMF9dh2KKQRkBZlhEOhx3v7n+c0O57thvTa26wnJmZQSAQoD0w4XC4zOSyEwilkyKUVtV11mFvtaz6idrMTBhmgsnn81hYWMD58+chiiJ4nkcqlSpTnnUqXEI5BFh7SwBgfX0dt27dwvDwMM6dO0dvpHYlvnaoFaGYU1y1PMEInCQUstb9+/extLQEjuNowVlVVeRyOYTD4WPZ3I6TzJx8vzzPV5ycSf3lzp07KBaLZQ6+JMXZCRHKcUepTvbC1LLqJ0o+8j2YrfqJQ8bW1haeeOIJ2lj84z/+4/hn/+yf4ROf+IQj13dYcAnFQZgHApGag6qqmJ2dxe7uLp599ln6oBMcFqHYrdmKF5eTZpOyLAMANjY28Pzzz8Pj8ZQZ/t25cwf37t2jp+lW6gEuyiGKYtnGZjcmGQBWV1fR1dVVc0zyYeHtSChWWL3gzARjteonIgtzJElqNJ0Ol1AcgrXwzjAMDg4OMDk5Ca/Xi7GxMVv7lKMglFbt5slaQPsn+L29PUxMTAAArl69Cq/XC1mWqWXJ+vo6zpw5A47jkEwmaT2AzCSJxWJlI2PfLjhqhZV1TPL+/j7eeustpNNprK6uAkBZ/cXn8x369RHJ8HETiqqqR9JQyjBMxfdgJvqVlRXouo7x8XEsLy8jEAggn8/XzSa0g8985jP4zGc+gwcPHgAALl68iE9+8pP44Ac/2NQ6b6+n85hg7S0BgAcPHmBhYQGnTp3CqVOnqj4sJD3l5OnIXENpNsVVDa0Siq7r1ErmzJkzuHPnTtWHlpj5kWYzoqRJJpM0D01OcbFY7NAb/t7uIBJlAHjqqafAMAwymQySySS2t7cr7ElisVhTnnKN4rhVZubrOI77ySoVT6VSmJmZQXd3Nz7/+c/jf//v/41CoYDf+I3fwPT0NF566SVcuXLFUfIbGhrCf/gP/wFnz56Fruv43Oc+hx/90R/F+Pg4Ll682PA6LqG0AbveElmWMT09jXQ6jatXr9LNsRrMA7GcupkJSbVrNw88ci9uhVAURcHs7CySySS1krl7927DKi+rksZ8ipuenqb1F5Iia9QyvpNw3EIEc1GeZVlqsEgUZMSeZH19HXfu3IHH46lo7msXx7WR211HLUXcUV/H0NAQPv3pT+O3f/u3cfbsWXzf930fvvOd7+Df//t/jx/6oR/C5z//ecde80d+5EfK/v+3fuu38JnPfAbf/e53XUI5Ctj1liSTSUxNTSESieDFF19s6OY0E4pTYBgGiUQC6XS6bbt5sl6z15fJZDA+Pg5JkjA2NkZrIWZyMm+mjZCWNU1ALOOJmyw5TRPVU6P1l+MmoU5QWNldg9me5NSpU1AUhfZekOa+QCBQ5kHWSkryuLvkCVRVPZQIrJXrMB/+WJbF3t4e/vk//+c4c+YMNRs9zNf/whe+gGw2S+2XGoVLKC3Arrdkfn4eS0tLeOKJJzA0NNTwJuE0oeTzeezs7FBpshN512atXNbX1zE7O4uRkRGcOXOmYhKfE+aQdpbx+/v7ZfUX4ujbyTPhOyFCadQOhOf5suY+0nthJ40lKclGouJ3esrLCiuhFAoFqKpKi/LEbNRpTE9P48aNGygUCggEAnjllVfw5JNPNrVG5z1hHQxzbwk5VRUKBUxOTkJRFFy/fh3BYLCpNZ0kFJLiEkUR3d3djhXxGk15kUFcGxsbuHTpUtWBYIfR2GgeaAWU11/ITPhOrb8cd4TS6utbey+IcimZTGJ2dhaKolCTy1gsRicoWtEpG3mnXIeVULLZLAAcusrr/PnzmJiYwP7+Pl5++WX87M/+LL75zW82RSouoTQITdOQz+cxNTWFS5cugWVZJBIJzMzMoK+vr+UaBQDaj9HOtZlVXE6Hw40QCin+67qOGzdu0GJvo2s5bb1irb8UCgXaj2Gtv2iadmyRQqdEKE7AKo01S8KXl5cBoMzkktS8OinldZRjAxq9jkwmA5ZlD72pURRFnDlzBgDw3HPP4c0338R//s//GX/4h3/Y8BouodSBubdEURRsbW1BURQsLCxgY2MDTz31FNX4t4p2pMN2Kq6DgwNHN6p6NZTt7W1MTU2hr68PTzzxRM2HshZxHObm6vF4MDAwQOsvxHAxmUxClmVMTU0hHo/TFNlR5tIf1wilFhimcga8dYIiqXmReSPHjU6NUHK53LEITjRNQ7FYbOp3XEKpAat9CsnB/7//9//A8zy1T2kXrRIKSXFZN3KiNnMK1Woouq5jYWEBDx48wJNPPonBwcG6a9Waq35Um4rVcPHb3/42RkZGIMsynYXh9XqPpP5y3BvpUdUvzCN6R0ZGaM2LjEnO5/N47bXXyiTKRz3NslMJJZPJHDqh/Pqv/zo++MEP4sSJE0in0/j85z+Pv//7v8fXvva1ptZxCaUKzL0lpGi5sbEBAIjH43jiiScclfk6ZTcPODtTnqxn3fhKpRImJyeRz+ebqh3VSnkdF0iBPxKJ4NSpU7b1l2AwWOaH5eTG83aMUOrBXPPyeDzY3NzEyMgITY/dunWLNrUelaiikwjFrBDNZrOH2tQIGIfTn/mZn8HGxgbC4TCeeeYZfO1rX8N73/veptZxCcUCu94S0k+RSqXAMAxGRkYcn3/dit18NRWXUzPlCawElUqlMDExgWg0isuXLzf1oBNCsdvEjvu0TmBXfyHF5vX1daiqWjaPpJ3T43G/5+OehUKugWXZivkjpP5iJnVCLsT7ykl0Ug3FnHIlhHKY39Mf/dEfObKOSygm2PWW7O/vY3JyEn6/H2NjY/jWt751ZN5bVlRLcbW6XqMwS32XlpYwPz+Ps2fPYmRkpOmb/KiK8k7CWmw2D7y6d+9eWf9LK/WXd2KEYoZdZCAIAnp6eqhS0Ox9NTc3B1mWy1R7wWCw7UNeJ0UoVpXXYUcoTsEllIfQNA2lUqnsprp//z4WFxdx5swZnDx5kk5cI4TjFOoRQLNeXE6nvEiURiSFjTgA1Lq2Tkx5NfOz1oFX5nkkzdZfjptEO4FQGlF5WUmduCYkk8mKMcmtRo2dSiikhvI44B1PKCTFRRyCWZZFqVTC1NQUcrkcnn/+eUQiEfrzxF7aSdQilFa8uJxOeWmahjt37iAQCGBsbKztYmmnp7yagbkWQLrJ7VI1teovboTSnDDAbkwycU0wj0k2118aGZPczjwUJ2EXoTwOTsPAO5xQ7FJcu7u7VEJqVx9ot2fEDvXs5huR4zayXitYW1tDJpNBT08PLl++7MgY4E5MeTn12tZ5JNb6i6IoZUomp++lZtEphNJOZGDnmkA8yBKJBO7evQtRFMuiRqstD+lF6sQIJZfLuYTS6bCzT7l79y6Wl5dx4cIFDA4O2j5ohxWhmNdsx24ecCblpaoq5ubmkEgkEAgE0Nvb68jGYyYO83rHvakdFuzqL4Rg7t+/D8AgofX19UNz862FtwOhWMGyLJ2KODo6SsckJ5NJrKys4NatW3RMMhmLQD6DTiQUN+XVwbAbzZvP5zE5OQlN03Djxo2ap4HDjlCcsJtvN+WVy+UwMTFB/cBmZmYcO8HX2rwex5RXMzDXX4aHh6FpGhYWFrC7u0vdfL1eb9lGd9jut51wKj/sa7COSSaycCKqMKeU9vf3abPlccGaestmsy3XLI8a7yhCsY7mZVkWGxsbmJ2dxcDAAM6fP1/3RjrMgVitpricvMatrS1MTU1hYGCA9to4mY7q1JTXcYDYaQQCATz99NMt1V/aRadEKEe5gVtl4cViEYlEAgsLC7h79y6KxSL1IItGo0fu+2aX8hoeHj6y128H7whCMdunkPBa0zTMzMwgkUjg6aefpgZ39XAYKi+GYbC1tYV79+4dm928pmmYn5/H8vIynnrqKfT397e1Xq1r6zSVV6egkfoL8R9rt/+FoFMI5TjnkEiShHg8jnv37uHGjRtlc3eIgiwcDlNiDwQCh/aZEZGQmcCy2awjjhxHgbc9odgV3jOZDCYmJiCKIsbGxpoyXXM6QiF28wCOzW6+WCxiYmICsizbpvycVI0dl5dXp6LWhl6v/sKyLN3kWq2/dAKhdMI1kCjJTkFGPneiIDNPFnV6TDLZW1yVVwfCbjTvysoK7ty5g5MnT+L06dNNh7JORigkxSVJEnVfdQLNkF4ymcTk5CRisRiee+45254JN+V1/LCrvxwcHCCZTGJjY6Pl+kunbObHXcexRgUEdp87MbkkY5IFQSgjmHaEFWRvcQmlg6DrOorFIorFIgRBoGaJs7Oz2Nvbw5UrV1oeUOOEysuq4kqn01Qk4AQaSVHpuk4bN8+fP4/h4eEjM27UdR07OzvY2dmhk/6Ok1COuw+kFZiVTADoNEWr/xghmGr1l04YbtUJhNLoNbAsi3A4jHA4TMck2zW2mgmmmXSeOZNC4MqGjxEkxbW0tISdnR0899xz2Nvbw+TkJILBIF588cW2GvPaVXnZqbgWFhZQKpVaXtOKeikqMvf+4OAA165dQzgcrrme0533m5ubSCaTiMfjuH37NmRZBs/z8Pv9SKfTh5qj7kQ48V6t0xSLxSK1hyHDrkj9xVwH6IQIpROUZq0KA6wKMjOx379/HzMzM3RMMiH2Ws4JpCBPvhOScnMJ5Rhg7i3heR6qquLevXu4d+9ey95TVpBO+lZQy27eaauUausdHBxgfHy8qa53p2oosiwjk8mAYRhcu3YNkiSBYRjk83ncunULhUIBb731Fp1l3uxs+McRhxWVSZLUUP2FCFaOE50QJVVLeTWLamOSk8kk7ty5g2KxSD3I7JR7dgaVLqEcMex6S3Rdx8HBAYrFYkOn8EbRSoRSr1HxMMwcrevpuk5nrZ86dQqnTp1q+CF2Ih1FiIxhGIyOjiIYDKJUKiFbysIjeRAIBCAIAkZHR2ltgFyv3++n5BKJRA5FYnqc9ZvD3kxr1V82NzdRKBTw+uuvl33GR6m6epxSXs3COibZrCAzK/cIwSiK4hLKccLaW8IwDLa3t3Hnzh3amOfkHIVmayiN2s07HaGYN0hVVXHr1i1sb2+3VD9ql1DW19cxOzuLU6dOIZVK0QfmTuIOvjrzVWi6Br7A40TsBII9QfREespmk5ATHkmPEQknmVN+3KfbdnAcRGauv7Asi3Q6jf7+fiSTSdroZ3byPYz+FzM6JeV1FNfg9Xrh9Xrp5NBcLkdTk0tLSzRaW11dBQD09fW5bsNHAXNvCckD67qO27dvY3V1FSdOnMDm5qbjQ3maiVCOy26eEAq5YScmJsBxHMbGxlpSoLRaQ9E0Dbdv38bGxgaeffZZdHd346233oKqqfj63Nfx7flvUzLYPNjEWnoNd9J3EPQEcTJ+EqNdoxiNj1Ibc+IyS0b3PnjwgKZuHuf02HGLAjiOa6n+4hQ6JeV11N3x5jHJw8PD0HUdDx48wPr6Ora3t/HzP//zSCQS6Ovrw5e//GV8+MMfxokTJxy9ht/+7d/Gn/3Zn1ExwdjYGD71qU/h/PnzLa33WBKKtbeEYRjkcjlMTk4CMPo5FEXB2tqa46/dyOZvTnFZmwRbXbMZ0E16cxOzs7MYGhrCuXPnWj6BtaJsKxQKmJiYgKqquHHjBm3MKqpFfHHmizjQD6r+brqQxvTaNKbXpsEwDHqCPZRchqPDGBoawtDQUFnq5ijTY07iuKXSdkV5a/3FfIo2O/kSgmmmj8sOb+eUVzNgGAaiKMLv9+PZZ5/FN77xDXzpS1/Cr/7qr+L//J//g1/5lV/ByZMn8e/+3b/DT/7kTzrymt/85jfxC7/wC3j++eehKAr+9b/+13jf+95HJ2Y2i8eOUKy9JQzD0JTK0NAQzp8/T8N4pzvagfp9KK3azR/Gtc7MzODpp59GX19fW+s0m/JKJpOYmJhAV1cXLl68SDf19b11/Pncn0NhlYqaVrUTqq7rSBwkkDhI4Lv3vguBEzAcG8ZofBSnuk6hO9Jtmx4jBVBzZ3m1k/Vxn46PO0Kp9frWU7TZyZf0v3g8njIn32brL++klFc9mCMlr9eL559/HtlsFt/+9reRy+Xw6quvOmrD8ld/9Vdl///Zz34WPT09uHnzJr7/+7+/6fUeG0KxG81rrg1cunSJTncDHm38Tssia0UTnWA3TyIDABWzXFpFo4Rinuho7W2ZWJnAX878JTKlTNW0VCOvIasy7m3fw73te/g6vv7Yp8c6IUJpZiO1OvnayWSJ/1ij9Zd3asqrkesgtissyyIYDOKHf/iHD/X19/f3AYDKoJvFY0EodvYp6XQaExMT8Hg8ePHFFytqA+RLcZpQ7CKUZlNcVjhFKLu7u5icnERXVxf29vYcs0JvpIaiKApmZ2eRTCbLJjqqmoqvzX4NN5du0rWchDU9NhQZwlB0CCfjJxtOj6mqeigRYqPo5AilHuz6X0iU2Gj9pROig04drpXJZI5M4aVpGn7pl34JL774Ip566qmW1uh4QrHOLQFAT8G15K/kS1EUpe0Jg2ZY01NO2c23Qyi6rtN+mwsXLmBgYADr6+uOnX7r9aFks1mMj49DEASMjY3Rk/9B/gAvv/Uy1lLltazDsl4JiAGs769jJbWC1++93nB6rFQqYWZmpqH0mNN43CKUepAkCX19fejr62u4/tIpKa/jNKg0X8dxzZP/hV/4BczMzODb3/52y2t0LKGYe0vIDUc6vNPpdN255uRLcbppy6zy6gS7+VKphOnpaWQyGbzwwgsIhUJtr2lFrc2e2N1bC/8Pdh/glbdeQaaYqVirGtrZXMPeMA7yB9DxaA279NhofBSjXaM4GT9J02OpVIraaBxHeuxxjlBqwa7+kk6ny/zHPB4Pte2XJOnYNvVOiJIA4wBsFjnkcjlHXKXr4Rd/8Rfx5S9/Ga+++iqGhoZaXqcjCUXTNCiKUpbiSqVSmJycRDgcbqjDm2EYsCzrqEcWuRZFUXD79u2WU1xWtGrnsr+/j/HxcYRCIYyNjZU9jIdtOa/rOhYWFvDgwYOKz+C7976Lr9/++pF0YHMsB6/oxX5+v+7PpgtpTK1NYWptCgzDoDfYi9GuUexl93CKP4X+/n7b9Njt27fh8/kORT3WCRHKURGa2QfLXH+ZmprC6uoq5ufnK+a/HFUaqlNqKNYI5bCnNeq6jn/5L/8lXnnlFfz93/89RkdH21qvowilWm8J2bjqmRhacRjTFUn6bXd31zG7+WY3f13XqWvy6dOnMTo6WvGZOG05b76+UqmEqakp5HI5XL9+HcFgEIAREXxt9mu4vXm76vtx0m3YI3igQ0emkKn/wxbouo7Ng01sHmxidXUVU5kpnB8637B6jDRXxuPxttNjxxmhHGdBnOd5Wvy9cuUKANDP+datW2Vd5IedhuyUCMWuKH+YhPILv/AL+PznP48vfvGLCAaD2NzcBACEw+GW5OAdQyhW+xSGYVAoFDA1NYVSqVS2cTUKp4dhkfQOAFy7ds2x8Jykpxo5LZqL388991xVNYaTKS8zORELlWAwiBs3btDPIJlN4uWbLyNxkAAAiLwIr+CFpmvIl/JQNGcjxaAURE7OQdWc+X4VTSlLj4U8IaoeM6fHANC6QDKZxNLSUlld4DjmwreD4zaHJPcVy7IQBKFq/YWkIZ3sfzGjkwjFfB2HXZT/zGc+AwB497vfXfbnf/zHf4xPfOITTa/XEYRi7i0hqapEIoGZmRn09vZWndNRD04RilnF9cQTT2B2dtZx5RhQ/+Emg8GsxW87HMYMk7W1Ndy6datCDDGfmMefT/w5CnKB/k5JKaGkGCaaDBj4RB8ETkBWyKKgFui6rSDkDSGdT5fVS5zGQeEAU2tTmF2fhU/ywS/6DWlyV2VzZTqdrpgLT8il3nzy4y5IHzehmC2TzGi0/mImmHYOeJ2S8lJVtWyvIzWUw4LTKddjJRRd11EqlVAsFsHzPD1V37p1C+vr67h48WJb9QknCMWq4vJ4PJidnXXcKgWofUra2NjAzMwMTpw4gbNnz9bdhJyMUHRdRyaTwe3bt3H58mUqEdV1Ha/Ov4pvzX+r5o2pQ0eulAMA5JU8WIZF0BOErukoyAWU1FJDBMgxHHySDwf56l32raDahirxhhtyupBGupDG5sEmXr/3OkRexHB0GKNdRnqsK9yFcDhM02OkL8M8n5wQTDAYPPaeCzOOm1DMEUotVKu/EPWYuf+llfpLp0Yoj5MxJHCMhEJ6S1ZWVrC+vo5r164hm81icnISLMtibGys7TnK7RKKnYqLPABOptLMhGIF8cNaX1+vaN6st6YThFIoFHD//n0oioIXX3yRficFuYA/n/hzzCfmm15T1VSkC2kakUq8hIAUAKdz4BgOql752Zo396OAX/KjKBdtU3UlpYTF7UUsbi8CAEKeEI1eTsZPls2Fz+VytC5glx57JxXl7VAtQqmHWv0vc3NzTZuIdhKhWIvy5D0+DjgWQiGRCQnvVFWlzWaNnsAbQauEUqtRkaTkjoJQSHSk63rTBOtEyotYqAQCAfA8T18/cZDAyzdfRjKbbHpNu+sqKkVkihkUS0WEEIJf8oNneciqjFwph4AUQEEuOF6HqYZmU2oHhQNMrk5icnWyTD1G0mODg4MYHBy0TdtwHAev14udnR1EIhHHzUzroRMIhWGYtq/Brv+FEIy5/kLI3Fp/6aSUl/k6crlc07Xj48SxEArZlEn+OJvN4u7du9SR1im0QiiNNCo6rR4jD5R5ze3tbUxNTaG3txcXLlxo+mZvJ0KxWqiIooh79+4BAGbWZvDlqS9DVuWW1ibrW0GIRtM1ZItZ+udRXxSqpsIn+mh67LDAMAyCUrCtlJpZPVYrPTY6OgpZljEzMwNZljE/P49CoXDk6bFOIBSnIwNz/cVc50omk0gkErh79y4kSSqrc3VqhJLL5drO1Bwlji3lxTAMDg4OcOvWLei6jhdffNHx5rFmCaUZu3mnrTrMSi8ik37yyScxODjY1nrNQlEUzMzMIJVKUS+wra0tKKqCv579a7xx/42WroegYck3y8Ev+pHKpcr+3Ct6IXIiFE1BvpSHpjtD7BzLwSf6cFBwtj5TkR7zhgxpcvcpjMRG4PV6af3FfKpeXl4GwzCHrh47bkI5ClGCXf1lf3+/zH8MAFZXV9Hb23uk/S9mkOf/uKxXnMCxEcrS0hLm5uYwNDSEtbW1Q+lEbpRQmvXiOoz+FpZlUSgUMDc3h3w+35JM2oxWGhuJhYooimUqslwph6/d+xo8+0cjhxV5ERzL2W7u+VIeeeQBGJ+ZXzTSY0WlWKYya+r1OBEMmLLI6LBwkD/A4tYiZtdnoeoq9KyO0a5RcGEOQ9Ghmukxs3rMqfTYcRPKcfTB8DyPeDxOB80VCgW89tprUBSlpfqLUyB7lbUPxSWUBuD3+/H8889DFEUsLy8fyo3dSCTRSXbzU1NTiEajjkyZbLaxMZFIYHp6usJCZTW1is+/9XkkMgmMxEfauibgUWrLTiaq6zqtlxDJcS1oWnl6TORFeHij2bHR3pegJ4gNdQOyKkPA4dt+hL1hpAtpGlnt5Hawt76H+7n7EHkRJ2InaHNlPBwvO1WT6MXJ9FgnEMpxp5qI3Pj8+fMQBKHp+otTsCOUw5YNO41jI5Tu7m4oioJisUinCzp9Y/M8j2KxWPXvW/XicjJCIfUKWZYxPDyMJ5980pHPodGUl67rmJ+fx9LSUsXslLeW3sJri681tLk3CzKOgOM4+n4DYgDZYrbl/hK73hee5aFqKnKlXMW6IU/I2Nxx+BYxgJHuslrEmEm/pJSwsLWAha0F+vOnuk5hNF6pHjNb8y8vLwNAmfdYo+mxTuiDOW5CIc8Jma/UbP3FqQZn84wnwPhsstmsW5Rv6gIensSddgUGqkcSnWI3b65XeDwe9PT0OEaqjai8SqUSJicnUSgUcOPGDRpaK6qCv5z5S0ysTAAAZE2GwAoIe8N0c261dkGua2trC+l0GhzHwef1ISAFkCqmEIAz4b259wUAeJaHV/SCZVgU5AJEXnS8XlINHMPBK3mbLvYf5A8wsTKBiZUJMAyD/nA/NbccjAw6kh7rhAjluPtyzIRihbX+oqoq7TN68OAB7X8h0Us79Rc7pZmb8moS5Es8qumKTtjNO9EwaZ7nMjY2hu9973uON0vWWo8YS4bDYdy4cYNuOHu5Pbz81svY2NugP8swDIpKkZ6uOZZDUAqCZdimaxeqqiKXy0EURQwNDYHRGZQKJWzvb0NRFKyvr8Pr9cLr9UKSJMc2G0VTkC6kIXIiOI6DrMoIeULQdA0sc3gnZJEz6kGt+I2Zoes61vfWsb63ju8sfsex9FgnEMpxRyhmh4564DiurP5SKpWoPYy1/hKNRptKRdoRSi6XcwmlEZAPmWEYcBznuCswULnxO2k33w6hkJHFJ0+exJkzZ+jN7LQUudp6q6urmJubqzCWvLdzD6+89UrZyd4OpDGRQOIleAQPNF1DrpizbUwEDDLf398Hx3EYHByEl/eiqBTBizw01nD4DYVCyOfz2Nragq7rlFy8Xm/bdSW/5EdJKSFfMor6hAh1XYdf9CPkDaGkPvr7duEX/cZ6cu31WtnQremxsDeMc73nMBQZaio9RjbT40KnpLxavQZRFMv6X8hnnUqlsLS0BABlSr1a9RcroWiadqTzUJzAsUcoAGhzo9MghNJuistu3VY2f1VVcfv2bWxublb03DhNKHZFeU3TMDc3h83NzTILFQD4zsJ38Pd3/972Guqlz4pKEUXFqFWxDAuf5IPACmWb8/7+PnZ3d+Hz+aDrOsLeMPaye2V1DYZhEAgE6ImsWCwin88jk8lgd3cXPM/D6/XC5/PB4/E0tRFai+Fm6NCRLWWh541rETgBXsF48PNyvqWem5AnhEwxUzc16FSnvKZruLl8E28+eLMiPTYUeaQe03WdWvNvbGwgn8/j9u3b6O7udlQ91vB1d0jKywlSYxgGPp8PPp+vrP6SSqUq6i+kyG9O89sN1wLg1lCahdOuwOZ1FUXBG2+80VaKy4pWIpRcLoeJCSMXPjY2VnFSOQxCMa9XKBQwPj5Ou+7J6xflIr40+SXc3rxdc71GNz4SpRDwLI/93X0UMgWcGDqBfCEPvaTjoHBQQSbW15AkCZIkIRKJQNM0FAoF5HI57OzsQFVVeDweGr1Uq78xYBD0Bhual0IgqzIlEQYMfIIPAi9AVVVk5WzdzyLsDTf1eu3C2tlvTY9JvGSkxx5275vTY9/5zncwODiIYrFYlh6LRqOIx+OH3lzZKSmvw+g7MddfyAA3Un9ZWlrC7OxsWf2lVCrZEooboTQA803K8/yhpLzS6TQymQyGh4fbSnFZ0WyEQlJt/f39eOKJJ2wfoMPovifrkVnzPT09ZV33O5kdfOF7X8BOZqfuWq1AURSsbq4CAPr6++Dz+MBqLDKlDPyivyl7E5Zl6ekPMObS5PN55PN5pFIpamFC/mFZFjzLQxKk9jrfoSMn54CHQQppgGRYBkX5UWQGGNGZX/IfGZkwYBD01O/sLypFzG/NY37L8F0Le8OGeqxrFCW1hEgkgkgkAqA8PbaysgIA9ER9GJLZxz3l1Qzq1V9KpRJ4nseDBw+QTqchiiJEUTzUaaGvvvoqPv3pT+PmzZvY2NjAK6+8gg996EMtr/e2jFBIimt5eRmCIODixYuOrQ00HqFomoaFhQUsLS3h4sWLGBgYqLmm0xGKoii4f/8+FhYW8MQTT2B4eJj+/e2N2/jS5JfKNsRqaMUXrFAoYGNjA36/H93d3Qh4AigpJWRKGWRKGWRLWbAsC6/gBcdykIvNpZUEQYAgCAiFQtB1HYVCAfl8Hnt7e9je3kbAG4DX5wUv8o4+kKqmIl0srx9JggRoRnTWinllK4TNsRy8grclpdp+fh/jK+MYXxnH6soq7ih38OTwk4Z6LDxomx7b3NzE3bt3qXqMpGzaTY+9nVJezcJaf1lYWMDe3h4ODg7wkY98BJlMBuFwGH/4h3+I973vfWUjI5xCNpvFpUuX8E//6T/Fj/3Yj7W9XkcQipMRilnFdenSJWqr4CQaEREUi0VMTk6iWCyWSXKrwWlCAYCdnR3ouk4tVADjRPh3d/4Ory285uhrmUHqJfG4kVqx1i+oY7Om0pnzsmpIk0OeEFRdbcpWhWEYGpkAgJfzInmQRDqbRj5p1HCcLO6bUVSKBiGqMhRVqTC2PAyInAiWZeln1w40XcPGwQaSC0l8e+HbkHgJI/ER6pxspx5LpVJYXFxEPp9HKBSiBedQKNT0hvd2Tnk1AyJOCgQCuHDhAmZnZ/G5z30Ov/3bv40vfOEL+KVf+iUMDAzgc5/7HL7/+7/fsdf94Ac/iA9+8IOOrdcRKS+nIhSriqtQKBxKbabe5p9MJjE5OYlYLIYrV640tIE5SSiZTAZra2sAgHe96120vpAr5fDK+Cu4t32vqfUalZjquo7t7W1ks1n09/fD5/Mh5Clv5qv1+yW1RE/cLMsiIAbAsRwKSgFFuX4kBTxqVvT6vfD6H9aJLMV9QRAoubTrjxX0BJEtZel3Z+7c5zkePuGR3b+dsWWzkZ9P9EFW5ZZtZuxe3/ydFJUi7ibu4m7iLgCgO9iN4egwTsZP2qrHiDyZpMcaVTSZX/+4CaUTSA0oH64lCAJOnDiBWCyGv/u7v0M2m8W3v/1tnD9//pivsjY6IkJpVzZcTcVF6hJOa+2rEaCu63jw4AEWFhZw/vx5DA8PN/y6ThEKsVAJBoOQJImSycb+Bv5y+i+xld5qes1GCEVRFGxubkLXdQwPDxtKL06wrSdUcxs2Q9O0shO4yIvGDHnNaFi0SpNZhkVACtimgKzFfVJ7IcV9hmGQy+VoGq1RhL1hHOQPqtaBFFXBgfroeryCFwIvQNO0lppDreR12PAKXuzl9rCd3sZby2+BZVhDPfawuD8YHsTAwAAGBgag6zqdXEkUTR6Pp6yj3O5g9U5OeVmhqmrZ/Wd2Gvb7/Xj/+99/XJfWMI6VUEhuvh3ZcK1GRRLGWsdqtgu7zV+WZUxPT+Pg4KAsxdTOms1A13VaN3r66adRKBSQShlOvZMrk/jqzFehqErZOF5FUwxLkjalq4VCAZubm/B6vejp6YHf44eiKsiWKs0W60U41WC1VfFLfnAsB0VVICsyBF5oqJ7Asiy11gCM721zcxPFYhFra2u2xf2KNR6SV7PF97ycpz0pxNjSL/oh8vUdIuqRV6uodkgISAHk5TxU7dFzqeka1vbWsLa3VpEeG42PIhYy0l7miYrJZLJmeqwTNnOrXPe4QJSLBI+b0zDQQRGKLDev9a/XqHhYhGKNUA4ODjAxMQGfz4exsbGWLGRYlkWp1Jpnlp2FytLSEmRFxlenv4qbSzfpz9pZkvgkHxiGQb6Ut03LmCMUKw4ODrCzs0N7GGr1ezgFHTpNLflEH3iOB8/xCHlCTfeNCIIAjuMQiUTg9XorivuSJJV17gucAIFrjLxqgRhbZktZ6LyOAB8wIjC93NiSKLmOUobcKHlZ02MRX8SY+xI/hZH4SNlExWrpMVmWj33eh3Xs7nFeh1U2/DhJhoEOIRSe55HPN96d3GijIjFaO4zZJWRN0nV+6tSptlQYrUYo1SxUcnIOX53/KrhQ7ZOXoillm6NH8EDiJdvoxfrfOzs7yGQytF4SlOpvfNUUY618bubmQXLyZ8DAK3gh8sbMlFyx0hSy1rWZi/uKotD02P7+PiROguSVIEhGDcaJUy35LOyMLQVOAMuw2Mvttf06tV7f/Nm300Ozl9vD+PI4xpfHG0qPJZNJrK6u0tnw9dJjh4VOiJIAe0JxI5QmQDaXZoryzXpxHUbTJFlzenoaW1tbFV3nraAVQiFkdubMGZw8eZJuDEu7S/i/k/8Xm+lNDIaaG9BVkAu04GvuuRDYR7ldVVWxsbFB6yVeyeuI2WIzqbdqG58OvSy1xLEcfIIRgTU78ZHneQSDQQSDQQTEAFKZFDLZDI3KrMV9p2oBOnQoqgJN11CQC9TYkmEYFErOT60kE0MbORA0Cmt6zCN4HjVXmtJjuVwOkiQhHA5XTY8Fg8FD3fA7mVAOO0LJZDJYWFig/3///n1MTEwgFovhxIkTTa/XERFKo5t+K15ch0EopVIJ2WwWPM/jxRdfdGSSXjOEomkabt26hUQigStXrtBGKQB44/4b+Nu5v3WkNmLuuVB0BRIngQOH5fVliJKI7u5u+CU/FE1peDhVu7PuOZaDV/Q2vPFZ+0Y8ggciL9KO/kZScyQFJIgComKUjoy1FvfNtZd2LM2tSi5ibEkgCRIkXjIis1J5naMZkO+BZ/lDmVZpRkEulKXHBiOD6An1QE2qODdwriw9VigUKporm1WPNQNrMfy4cByE8r3vfQ8vvfQS/f9f/uVfBgD87M/+LD772c82vV5HEEq9PpR2vLicJpREIoG5uTmwLItr1645drJplFBIhAagzEJFVmV8eerLmFkz+m4OQzmzndrG3t6e8WBHY3Tee6sbmhmNXK/EG+7D7Tj3miMwUlz3C36IXGXdq9ap3VrcL5VKyOfzyOVySCaTtLhPfMdq3Sfm9x70BJEtZmsSXVEuUhk16c7nWd4wvqxjRGkFz/IQOOFIplUCj6xwSPSytbWF72x8Bxe2L1DvsYHwgG16rBn1WDPo5Ailt7f3UF/z3e9+t2N+ckAHpLyA2pt+u3bzThEKIbXV1VWcPn0a9+/fd/QmbIRQdnd3MTExgd7eXjz55JP09ZPZJF6++TISBwn6s+1GAmaQdfb29h7VSzxB7GZ36c/Qk38Dkthq11Xreskkx0amMDYKTTekyVk5C17h4fP76MTHolxsWDkGgNpkhMNh2rlPyEVRlIriPoH5PZMemmaUXJpePrVS4AR4RS+g1ze29PLGYaSgFI5kQyXkZ7aKIcP1VlOrWE2t4lsL3ypLj53qOoVoKIpQKISTJ0/WVY+1kh7rZEJxaygtoFqE4oTdvBPjeguFAiYnJyHLMm7cuAFd17G4uNjWmlbUIhRd13H//n0sLi7iwoULGBoaon+3sLWAV8ZfqWh0c4pQVFWl/SV9fX0I+oOQBKni1G6tvQRF48EuykUUlEfX1krkFPaEK8wkDwOkMO4RPHRGStgbbvrkX6+4b/57UhQPeUKOpJxkVYacf2Rs6RW9EDmxQmQRkALIFDJQNOVI+kB4lofESxXWNHayZWt6zKweO9l1smp6bHV1FbquN50e6yTZ8OM8/hfoEEKxRhHmaODixYtt2c23a41PjBW7urpw9epVcByHXC7neHNZNXNIRVEwPT2N/f19XLt2DeFwGIDxIL46/yq+Nf+tqqqpdgmlWCxiY2MDHo8HHMch6A2CY7m66ZEKvytBgof3QNVVlAqlhlVetN+jcHSS2YAn8KguYTrjtGNpby7u67pOO/cPDg6glBTkD/J4UHwAn8/n6FAxHYYEOQ9T74vkh8iKFT0mhwmRF8EyrG1fUiNNx1b12EBkwNH0WCfIhnVdryA2tw+lSZhTXiRCMae4bty40TZDtxqh6LqOe/fu4d69e3jiiSeM6YKm6yU3wGHWUDKZDMbHx+lUR9LfUpAL+Iupv8DtjeqW8+0SSjqdxvb2Nu0v2VrdQkEuQGObJ1Jzzl+HDi/vRdgbRraQRUEu2Pa5CJwAkTu6Mb1A7ZSTnaU9zz2cVy83LoBgGAYejwcejwe93b1YX1sHwxvSdjJUzGzL72SxWNM0cAyHZC4JwPAE8wlG+vKwCMYreKFoSlmUakazLhaarlWkx0ZiI1Se3Eh6zGzNT5674yYUskdZI5THaRYK0CERCs/z0HUdm5ubmJ2dbXuiohmt1FBkWcbU1BQymUxZVEBAbr7DJJTNzU1MT09jZGQEZ8+epQ9d4iCBl2++jFQ2RZv6FFVxzIhQ13Xs7u4inU6jr68Pfr8fQU8QD5QHCGrt39w6dOSVPPbz+1BVFSIvwit6UWALYGF8lj7RZyjHbE60hwFqA98gedla2j9sDm1U1kuUXCW1hHAgXLW4T4aKEWlyq/cbw1Q2SBaVInJyDplixnAfEP3gOcPYMl/Kt51iDIgBFJTada92bZEKcgF3EndwJ3GH9r70hnpxqquyudKcHltbW4OmaYjFYigUCocyPqMZkGffWkM57qbPZtERhEJuqOnpaUcmKprRLKHs7+9jYmICgUAAY2NjtifEw+jAJ4SiaRrm5+exsrKCZ555pkzlMbM2gy9PfZmelM0kQouxAPKlPIpMsekIhdRLVFXF0NAQvB4vPIIHB/kDx9Iw1sippJSM078sQ4OGmC8GDRp05XDrJQQ8y8PDe5AupFuOhu1GIhNZr51AoZaSy1rcJ7UXa3Hf5/M17MjAsRz9Hs0wb+ZkaiUBz/LwCob1TF7O06bLRtGIWo3AiXuLYzh4RA9VjxHvscHIII1e+kP9tumx3d1dzM/PY3l5GfF4nFrzH6WUWFGUsrn2uq4jm826EUozYBimTAZr7alwAo0OrtJ1HSsrK7hz507FrHUryJ87KUcmqbnvfe97KJVKuH79Os2fapqGv537W7xx/42qv28txvolP7ycFz7JVzZBsRrM9ZL+/n74PX5DBfVQouukaqwavLyXpmOARySp6w8NIR1OyUi8BDBAXslDRPN2OdVgHYkckAJgGRYltQSJlxr25DKPlAUeFfdzuVxFcb9a577Ii+CY+nUvKxRNse/faUDFZ50gWQtO3FMCJ4Dn+Ir3qOkaVlIrWEmt4NX5V+EVvGXeY5FQBKFQCOvr6zh37hwAIJVK4f79+3Saotl77LCbK63fn6vyahK7u7t488030dfXh3Q6fSiTyRqJUBRFwa1bt7Czs9MQqZHZBU4W5rPZLGRZhiiKZZb32WIWf/rWn2Jpd6nhtUi3eFbOIlfMlRWUc6VcRQqC1EvIySzkDSFbzFZs4E48/ISYstksEomEMQPCF0DYH65IcdmRpFOzRvySH0W56KgM2Q5EmkxSTrlSDkFPsMxTrdETeq3ivp3vGEkd5hV7hVoz6Sa7/h2WYY30mEkB16x1S7spL5EXwcDwoauHvJzH7c3bdNx1zB/DyfhJpPZSOMueRXesuyI9lkqlMD09DU3TytRjTqei7GayuCqvJhEIBHDx4kX09fVha2vrUPKYHMfVNF3MZDKYmJiAIAgYGxtruOvdCTkywcrKCubm5gAAly5dog/YamoVf/rWn7Y0wtYcUVQUlB/WXmRFxsr6Cg7SB+jr64PP56u6ITimPHrYd5BIJBCLxSAJEoqFIlYSRkd0IpGgKR1zOtFsCAmgLknWwmE591aDNeVk/i4kTkJACsAjepoiSXNxPxqNQlVVmh7b2tqCxEpgRRYer8fxoWKEJAnId9GKaWY7hEIK/q1a0SSzSSSzSSwtL2FemcdozyhNj1nVY2SWzvb2Nubn5yFJUpl6rN30mJVQyPfpRihNQJIk9PX1ATgci5R6625sbGBmZgYnTpzA2bNnmwppnYhQVFXF3Nwctra28NRTT2Fqaor+3c2lm/ja7NdaTvNUm2FCTsakXsLoDJ48+yREwUj5VDtdOpHy0nUdyWQSuq5jcHAQUX/USDd5RHgDXqysrMDj8VQUpEm3ufl9WEmS9FvUil5Il/ZROvdSyaxNykmHjoJSMAwuOY0O5NKhoyAXmpImk2l/gUAAIU8Iuwe7yOaydCM0F/e9Xi/9Pp04KGi6Bk3XsJvdLR+PoCp1FXCtXoNf9KOoOBNh6roOHXrV9NiprlMIB8MIBoM4efIkVFWl6jGn0mNWQslkDMJ2ayhN4DCmNlpht66mabhz5w7W1tYqCt+Not35Jfl8HuPj42AYBmNjY/SzkBUZX7v1NUysTLS8thl2D6y5XtLT0wOGZVBUiygpJfhFvzFO4KHSxymYB3ABQE+0B3uZPWh6+YClcDiMcDgMTdNotznxyvJ4PPD5fBVyWmu/hXlSIukZ4RgOXsnbUrTXKvyiHyW1VFUya4V5IJeZJFVVRVbONkToJMIURAERMUKHihFb/t3dXfpZCoLQ9iGB2PmTiMU6HsFsMFqUH9WWCFohlIAnYNRxHEo5202NNKfHGDAYjA4a6rGH1vzxeJymxovFIlWP2aXHCIHXgrUXJps1DiBuhNIinJwrb4aVUKxWLq3mQtshwJ2dHUxOTqKvrw8XLlwAy7KQZRlZOYs/fu2PW5qqaEW1G9haLyF5fRIJmesYAifAJ/qg6Ro4lmt58yEE5vV6EYvFsLO+Y2zsDECyTtaIimXZsoK0LMvI5XK2clrrA2vdmIOeIPW6YsAcSarLbK3fCqwkSUwxWZY1XJMtqqt6vmPks4zH45BlGfl8HtmsMf1xeXmZRoLVhorZQeKNmmetVF1FkysvQRIkQ2hRbK6GBLRmT1MLhJRqCXACUoD2vtxcugmO5R6px+Kj6A/3o7/f+Iekx5LJZFPpMbsueUmSjtTG3wl0zNUeRYRCNvLe3l5cuHChrT6XViKUWhYqS8klfO3B1zBwYsCR/htrsyDpLzk4KK+X1KolyKpMNyjSkBjyhpqKXswNkvFYHDx4LMmNCwwIBEGg0YtZTms+cZNN0fzA+kQf8nIeimocVogc1mxn77T1SDMqJ6CxDVXV1LK6hSRIkDjJcB9QSk2NDyCjjkVRxPb2Nrq7u5HL5bC3t2fUXyzFfbvrI300zaTlgHIFHMMYNaSQJwSwtYkJOJzaF3k+qjk1+ERfhV2MqqlYTi5jObmMb+Kb8ApenOw6Sbv3SXpsZGSk4fSYXcrL7/cf+3jkZtExKa/DjFAURcHCwgLu379fsZG3s24zBFjNQgUAvrPwHfz9nb9HUW2+d6QazISiqioSiQQURcHw8DA8ogc+ydd0LSEn58DnjVtG5EV4BS/ttbCr9ezu7mJ/fx99fX2IR+KGkWG+/WZFs5zWfOLO5XJIpVLgOA4+nw/dkW6jdmF6Jq1yWK/ohV/wU5Jp5/MnUUIzabWWo76H7gMiL0JgBTBgEPaGUVSKFb5utV7bXNwHUFHc13W9LBLkeb7cnqYNaJqGglJAupgGx3FlqUrr7Jp2Bn/VQjVC4VgOHt5TRuLVkJfzmNuYw9yGIayJB+KUXEZijaXHyHdB/k0I5XHD2z5CIXMr1tfXcf36dceKXM2ovKpZqBTlIr4y/RWsplbpACUnez0YhkGxWMTOzg5EUcTQ0BB8kvHAWk9djaxlbUikEwaZh5Jejocsy8gUM0gkEiiVShgaGkI8FKekU22ccDURQSMgJ+5QKGQ4/eYL4DQOyxvLUBSlzMrE2gyYLxnyak7mEPKEjI53MCgolWmlWuBZHpIgHalVjFf0GoVvpfxkL/Ki7ThhO1RspKbiPgAqTSbF/bA3jKSQdGSomHUzN6cqyfsTOaOPZi+/1/Lr1LsG63ugfS0tOjXsZnaxm9nF95a+13B6bHV1FcViEa+99hr+5//8n+jv70c4HD70COX3f//38elPfxqbm5u4dOkSfu/3fg/Xrl1reb1jJxTz1EanI5S9vT3Mzs5C13XcuHHD0c7XRlVe1SxUdjI7ePnmy9hObz9ak+XgF/3wSJ6mDQirIZFIIBKJGNbelnpJs6hlO09UTLIsYzuxjYAngDMnzoDn+CNxCibgOR49sR6kC2kMBYZo9EJmmlebU6LrekXHO2nmU1UVeTlftR5SS8l1WKgVJVjHCROhhdWip5HDiyRJkCQJkUgEQSmIRCpRIZSoRtb1UCvdBBgHLp7lsZ/fB8uyCIgBcCzXVBTWyDWYX1/kRaPh2iFBijU95hN9hnrMkh4rFAxPO5Lm+sIXvoCVlRW88MILeN/73of3ve99uH79uqN72P/9v/8Xv/zLv4w/+IM/wAsvvIDf/d3fxfvf/37cuXMHPT09La157IRCwPPG6dYJ6LqO5eVl3L17FydOnMDS0pLjNgr1IhSzY7JVSXZn8w6+OPHFSsULdKQLaZT0Upn8Utbkhrrdzdjd3YWu6+jq6jJqD74wDnKtb+yNnJRyuRwSiQSCwSDCXWFwHId0MQ2f5DOK4moJmfxDNdAhdN2LvAiO5cpIoSJ6eah2MluZKIpie5ixWvKTZj5zUdwv+qlCrlU0ewptpjBdYaliSiuVCo3Xj0LeEA7yBxVCCTuyJv/UK+7XIhRipUK+S00r730ReZHOrqkXhdW7BvL6Ei/RWTiHhVwpV5Yeu3byGt5z4T104ufg4CA+85nP4H/8j/+BP/mTP8HP//zP46//+q/xkY98BK+99hpOnz7t2LX8x//4H/FzP/dz+Cf/5J8AAP7gD/4AX/nKV/Df//t/x6/92q+1tGbHEArHcSgU2j91KIqCmZkZpFIpXL16FR6PB/fv33dMc09QK0IpFouYnJxEqVQqc0zWdR3fuPMNvLb4Wl3Leav80jw4KSfnaJHZCnO9hOM4+Lw+hDwh7Ofayz/XS8eRwmN3dzfi0Th4lqfpH/PJXeQNh9uA9FD6iXKVTatEQ4rEtU6WZquSWCwGRVFo3WVvbw/pdLrqhqhqatmkSEmQEBADKCmltuSrzb5fsrG3CnNaSVZlSJyEsDdcMS+FwG4oFkE1st7b28P29jZEUaTRYDUXDLtnspqVihnWKKys96WUa/jgpGmGbJ2kD53ICjSK873n8dJ5Y/yu3XCtWCyGn/mZn8HP/MzPOO6IXCqVcPPmTfz6r/86/TOWZfGe97wHr7/+esvrHjuhOJnySqfTmJiYgCRJGBsbgyRJtEve6SE61SKUvb09jI+PIxqNllmo5Eo5vDL+Cu5t36u6Zq1Nu8yGhDF1u5s20VKphI2NDVov2VjdgMA2373cDHRdx/b2NnK5HAYGBhAPx1FSSlUVO7IqIyfnkC6k6XwOgRNQKLV+mGhVosvzPEKhEPVM4nm+bEOsZcTo4T1IZpPQoVMrEqfTMWa0UvCvB13XUdJKtNhtliYTWxqP4Gmo3mYdKqaqKnK5HPL5PDY3NwGgorhvd8hrxkqFvg/L4YtneeN9MCzypXzNTnpd1+ETfJAV+dBteMy4OHARP/L0j1CSqDdcy2kfMZKytPbg9fb24vbt6mMx6uHYCYWg3UFY6+vrmJ2dxcmTJ3HmzJmy2SWAvVdOO7CKCMzmkmfPnsXIyAi9ho39Dbx882Xs5fZqrtloUZ6YJRKIvAglr2BnYwfhUBiRaAQBKYB1rCMv5+mo13Zgd22qqmJjYwO6rhvF92Ac6UK65sZe1rX/sPZC1hZYASGPIaPMyY3VepxS/1Sbsmg1YvT5fOiJ9JS9ptWKhPZaaIbNfWJLQyCgo0FXnwpwLAev4D30gr9ZmixyhoqPkGWz81I4jqO+Y0BlcZ/IlokVD8MwbVupECiaYu/+DK2i7uQVvDjQDo6UTC4NXcIPPfVDZc+CnWz4cWtqBDqIUFqNUDRNw9zcHDY3N/Hss8+iu7u77O/NJwAnQZoRydq3bt3C9vY2nnvuOcRiMfpzU6tT+Mr0V6qmqMxoVeW1kdjA/v4+ent7EQwEEfFFoGoqeI53VIZsXsvcrNjb04uwP9xWWo1hGMia0ffC8zxtKOMYDgW1UJHXptMcD8lGxc6IsZQvoZApYHZ7tuqMeKC81+I73+Hxrb+LIhLi8P/7lTSKNl3ztVKxImfUhRqRrzaLamlgj+CpIEmS+iINos2MRAbKi/tEeZnJZKDrOpaWlhANRpEVsxA9ouP1Tqv7M3kfALCeWYfOHI1gBACeG3kO77vwvorP3S7ldZiy4a6uLnAch0QiUfbniUSC2mG1gmMnFPLBthKh5HI5an1/48YN26534gzsNKGQNck1EAsVqufXVPz17F/je0vfa3jNZglF0zRsbm5ClmUMDQ3BI3kQ8ASQzBoW8LImQ+AEhLyhhmzHG0Umk8HW1hai0Si64l3wit6GyaRRebCu65VF2IcbnSzL4DjuyCS6DMMgHAiDCRrNkGElXHVGvM/no4eY9XUG33mNBYQs9vI6ltdkDA8+LCY/jDJrfd9e0QtVU5vevNtBNRdmTdccM+dkWRZ+v6E8KxaLODtyFjv7O0hn0yjsFpou7jcD8j6IsIFlDPVYyBNyTFlZDddPXccPnv9B++uypORJCvmwIIoinnvuOXz961/Hhz70IXoNX//61/GLv/iLLa977IRC0Oymv729jampqTL7kmpw0hnYvGY+n8frr7+O/v5+PPHEE/Qa0oU0Xr75MlZTq02tyTBMwwVec71keHiY5r6tOfaSUqJ/xrEcglLQyC03OTSJXBtpVuzt7UU8YjRrmYvV7aAWwZAirFfwAoxx0gx7wyjIhQq1nNMgSi4SQVezkd/f36c9P5ubUXz962FoquEv88EPKujt1SuKyR7eY6T5BLasbtBKmqlZWEk9KAWRLTU2FMvWwZrlDbv8Bqc96rqOoBhESS8hGAoiGAraFvdrRYOtwOxkIKsy8koeB4UDOtpZ4IWqIoVW8X1nvg/fd/b7qv49EdEQHHaEAgC//Mu/jJ/92Z/F1atXce3aNfzu7/4ustksVX21go4hlEY75XVdx/z8PJaWlnDx4sWGWNzp2SXENXdvbw9PP/00BgcH6d8t7S7hz976s5ZSFI1GKGSWCOkvCUgBI6wvlW+s1vWsfRakcayRh0fXdeRyRt1maGgIsVCs5W7pWu+z1jUEpSCtrZhJhNQsnIzCCOoV/K028oqi4vXXdXznOz4ADPoHsrj6XAYjIzxUtXwIFnEbPigcQNIleurnWR7pQvpQycSKZu1izLArivtE+453M4JSEEk1iSii9M+q1bIIYQMo8x1rti5qVclZp1ZaRzt7JaOOZGds2Sh+8PwP4vqp6zV/xhqhHMVwrY997GPY3t7GJz/5SVoy+Ku/+quWzHIJjp1QzMXzelEEkeMWi8Wmut6dbJqUZRnT09NIpVIIBoNlZPLm/TfxN3N/05blfD1CIUTW09ODQCCAiDeC/cJ+XRmyHcocelmedtFbQ39ZlpHJGEOihoeHEQvEjnSeCFDbx8l2QiLb3iZQ7zXtoOvA3/2dhLfeMjaGy5cVfN/3KSgW9YohWGYpLXWaVmX4RB+SuWTFjPd2B4rZX6+xmTpta6JoSlk60iN4DNdkXUWuaEh6w94wNnY36kr5rdFgqVRCLpejnyeRJjfSuW8nua4lx7VKxWnvi643JBphGAbvvfBeXB25WvPniD3ScRTlf/EXf7GtFJcVx04oBCSKqPYFp1IpTExMVMhxm1m7XaTTaYyPj8Pn8+H8+fNYWTGGQsmqjK9MfQWz67PwST6jkNxCKqYWAWiaVmZn4pE8CEiBmpYUzdRkzJsAAwY+yQeBFbCf3sfqyioEUYAkSYgGoo5sPtUI0O7Pqrno2sFWcfVQ4ZMrNha9NPuaAFAqAV/8koDFBePe/cEfVHDtmgrAA5+vfAhWLpejUloS+fE8j4gvUmbGaXV+JuOQ22nks8Iv+g99PkxZgyjDIeQNQdVV8Exz2w/DMLS4H41GaXE/n89je3sbmqaV1V5IcZ/MwbGTXDfTn1bhQFBjgijDMPjgxQ/i2eFn665LlG7WGoqr8moDhCCscwF0XceDBw+wsLCAc+fO4cSJE003KDpRlCfDuIgseXt7G6qqIpVL4eXvvYzNA2ODMJ9oPIIHEi81nI+tRgCkXiIIAjV3NDcONrtePegwrMX39/exu7uLvp4+iIwIRVEcsRch35/d92i+XoETmnLRtUNZ9MKytt3uZrTiyZXJMPjCyzwSmyw4XseP/EMFTzxRSVx2PlkbGxvIZ/MopAvY5DaruvxaxyFTF4UWoxeO5eATfNjL7sGPozEhZBkWPsmHVC4FACipJUi8hJA3RCXWzUT3pLhPag2lUokSdjKZfGQSGu7GXnbP9qDaasOzdYIoSfMRF+v3X3w/nhp4qqG1yN501DWUw8CxE4pdvwg5WciyjJmZGezv7+P5559HJBJp6TXaIRTzMK5Lly5RjxuO47Cyv4Lvfvu7VZuwzCczmlJijFSTnZrEjgBIvSQcDiMej8Mv+Ws2DtZbrxGQZsVsNouBgQHEQjFsbW8hW8jCq3upEWRRPpwmPuCRdNVJfyxN08q73R/WXvZ5Q6kl8cYm3sxrbm8z+MIXBBwcMPD6dHzkx2UMDjb2mUuSUTcZGhhCSStVdfn1+XwVtRe7mkWjkx7JUKzd0u6R2aOTMcjmGp6u65A1mUYOZdJktdS0n5YoihBFkY44KBaK0BUdy5vL1GbHWty3G67VCkiEz7EcPnTpQzjfd77h3yV7k9lXLpvNPnbTGoEOIBQCIu8ltQ5zesns0NsKWiWUYrGIiYkJyLJcYaHy3aXv4usPvo6RkZGG1qpIKYmGmqSkPHpwrARgrZeEvWHDaPEQfLAIyGhgTdNovSRTzEDRFRqaW61UiI19tpRtKLVYjejI5uaUPXo9kOglp+TQy/dSY0BN0xpqrnvwgMWfvcKjVGQQjen4iY/KiEYb/26I7LagFCAIgq3LbzqdpsoxUoi2Ri8V99ZDpRJxUTDXgMh7dFLBVA/VrFSs0UE1aXIrfl08xyMQCSBbzGIoOFRR3CfFf1mWqdS/XfAcjw8/+2Gc7Tnb1O+R+on5s3AjFAdANv7V1VXMzc1hdHQUp0+fbvsU1QqhEAuVWCyG5557jqbkCnIBX5z4IqaXp1uuy9AT5sM9i2zKeSlPR5ta6yVBT/Oz0JuNUMyjgQf6BxD2h+tafZjzyizDIuAxmhEbsSBRVRWqqpbVw4JSENlC9kgL/n7Bj7ySB1t4dFKlTsOaarv5Tk2x+Mu/4qFrDIaGNfz4j8nwNmFIQGTB1TZJayMgsTGpNqOEwKpUMtuQKKrRhW5O9R12hCLyxkHQLtqol26ykyY34tclcAIEVihPSVWReudyOfrfZrFEs5+LyIv48cs/jtGu0aZ+D6hM8wNuDaVlmL84juMwPz+P/f19XL58GV1dXY68RjOEUstCZSu9hZe/9zJ2s7uOzi4hm3JeyUNRFOxu7sIv+jE0MEQHD7VSOG3mGsuaFWNd8Em+MjJpZC1N1yprSIJkzEQvPZqJTua0kGmLkiQZLra8MfTLqRNjIwh7w9hUNhHQyx9eq9MwmYteKBXwt99Q8dprxqPz5JMqfviHFTSjXjVLkRv5fliWrTmjRBCEqpshsSEh0nKBN8Y6K5qCFFKNX3QLaMRwsdGN29avSzB6r8x+XSInUuueWq9JpN6lUgkejwccx5URttmWv17nvsRL+ImrP4Hh6HBD78UKq8JLlmUUi0WXUNoBOSmwLIuxsTGqQ3cCjRKKqqqYnZ3Fzs5OhYXK7Posvjz15Uen8RZGANeDLMvIZDOIRCIIxUPwiB5ANx4ev+RvOk3RKKGQ1Jq5WdHOELBZAq3YlB8Or+LAYXt7G/F4HF6vF6ViCVpJQ6qYQjaRpRbpTndJm9GMkovMRVdV4Ctf4XFrKgCoEn7gpQLGfiCNZj4WJyS6djYmuVyubDMknx/P82UEppQUKhUn0uSQN1RVpNAqqnXcm9HOgaxi8qbgpWq+ZhptSf+HmbCtxX2e58ukyeZ70it48bGrH8NApPWudjvJMAC3htIqEokEpqenwfM8RkdHHSUTANTioRZyuRzGx8fBcVyZhYqmafj67a/ju/e+W/bz7UwYtEMqlUImk4EkSYjH4wh7wxVGi+bCfq5Y3+6iHqFomoatrS0Ui0WjWTEYQ0Eu2K7b7nskTZW7u7soKkUM9g2iJ96DQrEATTJqFsl0EpFIhM7hruf42ypaUXLl88Cf/ZmAlRUWjJjHBz+QxjPPaGAZgyjrOdu2IkVuBHZKp1wuR6OXiC+CJG8/YVHVVOTkHI1ESZqv3QbRRjvunR4pkS1lIatymZqvXi+SreOxqbivaZrtDB2v14uuSBc+/vzH0Rdu3fsKsHcaBuDWUFpBPp/HzMwMLl68iLW1tUMpFNaLUIiNi9VCJVvM4s/G/wwPdh5U/I5ThGLe1CORCGRZRsgbst14qhX2ZVm2DfFrWbnIsoyNjQ1wHGeQSSBWs+DfborPXBfieR46a9heFBRjUp1P8NFplYIo0L6NXC5X5vhLohfrSbFRSLwEMLAtEFfD3h6D/+//45FMshAlHT/2YQUnTxqfq92UR4k3pfmgg2MMW/hqBObkpko2w0gkgoAYQCKVoH0a5lSOz+erUDiZI8pmNmUzmhr+5RCh+EQfSkqJHoSqqfl03ZDDq3q5S3ita2BZtmyoGJmhwygMnuCewML0ApKxJGKxGGKxWEuHHjtjyFZcADoBx04oPp8PP/ADPwCe57G5uXkoc+WrEYqu61hcXMT9+/crbFzW9tbw8s2XqxalyU3YzuAbsqnzPI+hoSEU80UwKtPQzItqhf1G1FZkRkUgEEB3d3fDaZhWCUVRFGxsbIBlWQwNDWF1dRWCLiBbyIJhDdJb315HpphBEEFInCGnVXgFLMvSQqr1pGjeHBtxqLV6cjWCtTUGL/+pgHyOQTBkKLm6u6t/DtY0H/FOq2bFcxgHKOIWcFA4sO3TyGazSCaTYBgGgiDQekKZ4sqyKZMucU2vHr00m85zglD8kh8FuVBTEWjnNswxHEpqqelr4Hkew73D+PjzHzdcKvb3kUwmsbKyglu3biEYDFJyCYfDDe0NdikvMgr4ccOxEwpgTH5zasiWHewIRZZlTE1NIZPJ4IUXXkAoFKJ/N748jr+a+auaKSWGYdo6tZNu6VAoZPSXePwoFZq3BSewqq2CUhCyR67IJ5Nmxa6uLsSjcYi82NAm0OrNXSgUsLGxgUAgYIgsdGCgawDrO+uUFDRNg6Io6O/vhyiKKKpFFJSCMSPF5GrLMRw8Hg9isZjt+Flz9GK93laGcN25w+JLX+Khqgx6+zR89CMKAoHGv2+RE5GTc/R7KWt0LTY+VbAZkH4Pu2jImspJJBLQdZ0OW7LrMiew3l+kX4Rs1q2k89rtAQl6gsgWGzOzJLBKk1mGRcgbgtfjbcg1OeqL4qeu/RTC3rDx/9EootEoTp8+jVKphGQyid3dXczMzEDTNESjUcTjccRisaqp/KO2rj9MdAShELQ7ZKsarIRCelz8fj/Gxsbow6OoCv5q9q8wvjxObd91zYgEzGEyQauEkkqlkEql0N3dbcxff1gvUXXVkROrpmtIF9PIlDIoKSXqpbS+sY7UXgr9/f2Ih+JQ9MY731t5r+l0mhbfQ6EQWIaFR/AgiyxOnDiBQqGAra0tqKrxvre2tsoK8sBDP6WSMTcDOh5NquRkOnFR13VKLtbN0efzIR6Mt1S7WFtjoKoMJEnHqVEN6TTg9wONcKvdydna6OoVvPDxhhTWCXIROREcxzX0nbIsC57nIQgCIpFI1UK0HUGbN2UGjDF/R1fLzDsbRasHlWZSa7WgqAoyxQxkyGBgDPkSeMFIt1qkyV0Bo2YS9NgXy0VRRF9fH/r6+ozxCw/rWIlEAnfv3qWjp2OxGKLRaNXhf4RQ3AilTRzG3BLrumSyo7XHZT+/j5dvvoz1vXUA5VYX5v4Ks0dXK/NLtra2UCgUMDg4CI/HsC4nm52TUmTzetlCFoubi9A0DSMjI+gKdkHWZJSKzql6rCDKsb6+Pni9XqMfQTcKpwzDoFQqYWtrC5Ik0aFoZEMz+zIRguE4ziAOJQ9dNhosJV6iI2NJo1o8HjdqSg9rL4WDAjY3N+k6zfQY9PUZ30WxyOD113m8/jrg9ek4fUrD6dMaRkc12ymMjURDRKWUU4xTccATMCZvNjkTnYA4CzQ1OteU7qlWiCYEbVaOkQMYy7Dwi35qpUL/zBS91OpFavVeb8cZ2e4aaIc6dOTlPM0SkB4eMuzto1c+Cr/UWOTAMAztezl58iQURUEqlUIymcTdu3dpzTQWiyGfz5fVXtwIpU2QjY/n+bpqrFZAUmlzc3NYW1urmOx4f+c+Xhl/perJzq6/QuRFY7Rug/e0uQhO/Lis6abDIBTSKCpJEvr7+xH2h+ksdOpoy/NVC/vNXptVOSYIRsqKFE0ZhkEul6N2MtFolG5qJN9vdpW1doqbXXrNpEhECizDQmZleEQPPD0epPPphi1NrHjySQ0nTpRw7x6DxUUW9++zyOcYzMxwmJnhAEbH0KCO06cNgunp0W0dbavB/HlaNzJi/97I0Ce/6EdRqS3RbQbmQrSVoIlHVsAfQDQYxYF2ULPbvcyht9RcQdwOTjsja5pW9RpID89AZAAfe+5j8Iqtq095nkd3dze6u7tpRE3SY8lkEizLolQq4c0330Q+nz/yHpTf+q3fwle+8hVMTExAFEXs7e21tE5HEArBYUUoiqKgUCggmUxibGysbLLja4uv4Rt3vtFUTwlJXZR0YzMLeYz6S7aUtQ33yQYaDAYRj8fhk3xQNbWCwJwmlGKxiEKhQCcrBqRA+SwI4mhrV9i35KYbuTZr8Z0ohTLFDP1dUsTs6uqqqrO3usoStRcREwAoi15YlqWW4rquQ+RESJwERmeolX01Wa0gCNA0jY5ztiIQ0PHMMzqeeUaDqgJraywWF1ksLjLY2WGxuspgdZXFN/8eCHqDGD2/h9OnWJw8qaHVWVBWNZ9X8NLoxexADDyU6MqNWd5Y0eiGLggCwuEw9chSigqKxSJWNldo9FJNHGHn0MsxHGRNbppQnCYTc6NtNZyIncBHn/uooQ50CGa14tDQEG2Z2Nvbw3/5L/8F9+/fRyQSwSc/+Um8//3vxwsvvNCUu3orKJVK+OhHP4obN27gj/7oj1pep6MIpdEhW80glUphZmYGAHD9+nV6Ii0pJXxp8kuY25hreW2GYaCopoefeWRpTVJj1npJyBtCpmCfDnGSUJLJJNLpNARBQH9PvzHNsU7fhV1hn2UNZ17SDFcNxLbF5/PRU1hADCBdSNMHdmdnB9lsFv39/U11w3McV2GbQaTEpFeFPKCRQAQlpYSs/Kgzn2VY2lXNs4/qBsTShHxW6XSapnXsZJscB5w4oeHECQ0vvQTs7zO4d4/F4oKApUU/0oU0piY5TE1yYFgdw8OP0mNdXa19r3ZpGBK9MGAMqXcbqZ9mIwSP4AEEQPAICIQDVByRy+XKxBF2TYB2Dr1BKYigJ1jXq6ualL4d1COUk/GT+OhzH4XAOTvj3u46gsEgLly4gPHxcfzbf/tv8eqrr2JhYQGf+cxnoKoqVlZWDjUN9hu/8RsAgM9+9rNtrdNRhOJkhKLrOpaXl3H37l2cPHkSi4uLdIPYzeziCze/gO30dluvYSUAs3GipmnY392HUlJwbvQcVFZF0GM/k6Haeq3AnHKKxWLQiho0XauY5lh3nYeFfQKP4DF6RWw69oltC5VKPtzA08U0TbslEgmoqorBwcGGJL7VYLbNiMVitC8gl8uhlCmV1Uu8Xi/9TEn0ous6nVQpa7Ih6X0o0xRFkQ5vsqbZ7PoLwmEdL1zlMPYCg3RuFysrD6OXeyxSSQbLSwyWl1h84xtAKPyIXEZGNJg/gmY2dRK9kOFfJP3aio19s/eanZWKIAgQBIGKIwqFAiVps7Tb6/VWfIaE+JkCUzZQrKQ8UjuShtBG04jNoBahnOk5gx979sfAc4e/RVrH/2qahqeffhr/7b/9N2iahrm5ucemptIRhGK2sHciQiEWKru7u7h69Sq8Xi8WF42i9PzWPL448UVH5pBXaxyUZRmbm5tgWRZ9fX0ABwTEAKAb6bFq8sR2CcWacuI1HqvZ1bo5+EZQVIrIFDPIFrPGKdljNMatba5hJ7mD3t5e+P1+CKwAhmGQKRkTHslnwfM8BgcHHbdSIUqv4d5h7OX2UCgUkM1msbu7Szc0QgokbWAuFnMsBy/nhV/wg/fwEEWxIs1mbqokmyPLsmWFcJ4HRkeNQv17AKRSDCWX5SUGB/sMxsc5jI9z4Dgdwyd0nBpV4fE01wjHgCkzCi2LXjgePsGIXhqRwAKNk1kjVipEGEEUenbSbrNyzPz6dgPFfKIxrK6deTi1UI1Qnuh7Aj966UfBsUfTWGgd/2ue1siyLC5evHgk1+EEOoJQCJyQDRMLFZ7ncePGDXg8Hsiyka/9+tzX8caDNxxLK5HcvfX1E4kE7bvwi36oenk3NU2NcTxKcvlprNVrs2tWXN9Zd2y+utVwcD+3j0QigWKxiLOjZ+H3+o2JhA8ddBmGQT6fp59FPB4/FBkkUeDs5/dtNzRzMZnn+bJeFQDY2d1BNp+FJ+QBFOMUzrM8VE4ta6okaTZiCRMJRMCLPESPaBu9RKM6rl5VcfWqClkGlpZYIz22yGJ/n8GD+wwe3GcBnMJrr2s4fcoo7p84oYHnAU0DrNxLBlRV22AVVcGBaqq9kEisSvTS6L3WSr8HYB+9mBtTiUBDEISKz1DTNOM+K+47MlDMDnZ9ME8NPIV/+PQ/PDQPOTvYWa/E4/G21/21X/s1fOpTn6r5M3Nzc3jiiSfafi2CjiKUdlNe29vbmJycxODgIM6fP09vipJawjdXvwlPyeOonYGVAPb29mjBORQKIeQJ2RbqrTNFJMEYU1viSi0RCknRxONxxGNxeHgP3WCdIk/zWoqi0OL44OAgFCjQVKNDn+d4hD1h7O3tIbGZQCweK2sadRJ2Q5vMMBeTzSaKZNomuT/6+vogSYY1B2motDZVsgwLSZKMBjXei63UFrK5LHaSO2V1A5JmK78O4MwZDWfOGBvy7i5DC/vLKyz2Uixu3gRu3uTA8zqGhnSsbzB4+ikN73qXAq/XqDeIvNiw8SGZIUJqX2SEMPTy6KUeyTvV72Eme5KqXF9fp+pHcwQY9AchCRJ9RqoNFAMaU8FVg1XhdXn4Mj5w8QNH3v9hta/PZrOOqLx+5Vd+BZ/4xCdq/sypU6fafh0zOoJQyBfYalG+loXKxv4G/vTmn2Izt4kT2olDIRTSlJfP5zE4OAhJkpqSjxblIopy0bixHtpmcCyHXClX82EhXc6ZTAb9/f2IhWKG26rJ5sNpaw9SfPd6vejp6TEKilIQBwVDPlpSStjc2sTBwQH6+/rRFel6NHK3gaFVjYKobhptzDSbKJLUIHFnWF9fhyAIFdELcRkGHp34PbwHOdmYVWG2hMnlctSO365nw4x4XEc8ruLqVQ2LiyvQtJO4d5/D4iKLTJrBgwfG83DzJoebN1n80AdYXHsebZ3M7UYI+3gfJUw7OK2qMoPnebAsi2g0Co/HQ6OX9H4a+7v7YASmqilotYFizToQmFVmV0eu4r0X3nsszYTWmUBOTWskMuWjREcQCgHHcdA0rSk5IbFQyWazuH79etkXMbU6ha9OfxWyKjsuyQUeqrwUBaurq2BZFsPDw5BECV7e21IRkWEYaLqGg/wBWJZ9lLbgxYo+ETJZUVVV6hScK5V3KTv5cJB60draGqLRKCKRCJWBEjIhgoBSqYTBwUGIolhGbsR6xG6SYDOwmgE2g1KpRAmxu7ubigZI9ELsSAi5EFkydOOUn8wlaVOlxEvQoRsOAB6Pbc8GSbPZuf0CAM9rOHlSw/nzxmextcXgezcNpRgAQPbh3LksClVkza2AnPgzpQwERUDYG4ZH8IABQ6OXwyQTeh0Pn3MSvUSCEQBAtpCl34d5uqLdSINaA8VquT+bX//GqRt46fxLh/lWa8IaoeRyuSMvwi8vLyOZTGJ5eRmqqmJiYgIAcObMmaaipY4iFMLSVsauhnQ6jbfeeguBQAA3btygp0FVU/E3t/4Gbz5401AnicZJzOn5JaRwGwqF0NXVBZ/og4ZydVQzMDsYA6a0xcPuZ9InksvnsLy6DFES0d/fj4g/Yts57CSJ7u8bmwspvvMMD47jkClmKLGSxs3BwUHbSLDCekT0ggFTtX/HDiFPCOliuqX3RepM1oZK8zwMO1myV/IiFokhKSfpabmsqfKhWzLHciixhpuyueM8l8thZ2cHmqaVRS92hN/To+M9/0DBpWdU5NNBLD7IweNzjkzsYJ6MyDIsor4oNF0zpoi26C3XCMwHR4/ggaqpkFXZdrqiuX5lNxuegDQjEpT5p1nUibqu43Lf5WMlE13Xaxbljwqf/OQn8bnPfY7+/+XLlwEA3/jGN/Dud7+74XU6glDMDzZg5OjrEUo1C5V0IY2Xb76M1dQqAFMTolYCx3AIe8NNzT+vhr29PeRyOXg8HqPHxBOsiBCahZVQrCgpJaT2U0gkEohGozjRfwIiJ1bNcTtBKOZ0HmC4Q0u8BEVVkC/lwTAMCgXD3sTv96Orq6uhyMj64BOPrpJcQkGxt+sIeUItK36IrxjpB6oGqyyZ0RkUC0Uk95PI5/NlTWk+n89WlkyiF03XaNc5UNlUScdKFwpl0YsoAhdOB5EpZnDmCWcPQWZYMwF2VioCJxi9J0Db97cdGIapGXGavw8AVWfDV+sdsro/e0UjwimUCrh88jK69aNNCVlBasbHbQ752c9+tu0eFKBDCIWAYRiwLFuzjqJpGm7fvo2NjY0KC5Xl5DL+9Oaf2lqFsyyLklKiYby5cS8v5xueVmfeYIPBIKCD9gQ44S1UiwRIk2RPTw9ikRhUTcVuYRcAqAqmpJZoRNMuoaiqio2NDQBAf38/VlZW4BN8yMt5aLpR0MxkMtje3kY0GkU4HG45zWauD4icCI9onFjzpTygA36PvyUy0XUdqVQK+/v76OvrK3NJqAdi1657dfR5+8rqJalUCltbW7ayZHP0wjIsfIJBPNamSkIsZFYJSev0RfscKYQ3AypwsETX1ea6m++zVqHruiFHVooNE1W12fDWJle76EXVVBpRv+/J92FAGKD393HBSii6rjtWQzkOdBShALWlw4VCARMTE1BVFTdu3CjbHP7f/f+Hv53726o3pnVztTbuEYllrQeFpHUYhsHw8DCymSwYhXE012xHArquI5FIUFPJaChKTQQJzP9NNPwFtroxXz2Yi++k8z3sDePW3Vvw+oyNr1Qq4eDgAD09PY6eqEpqCaW8sSELrAC/1w9N0+j30yh0Xcf29jYVSzQz/MjOLdh8Gq5VLzE79Oq6/qhrXzdSMAIvQNVUaF4Nu9jF8PAwjV7UvIrp7emaG6NTIBGKwAngOb6uwMGqtiqLXor2jty14OE8KMgFcHxrQhlz9FKtd8gavTAMgx966odwaegS1tbWjn2Ilaqq9CBNkMvlmjr4dBI6glDMp9pq0uFUKoWJiQnE43FcvHiR3giyKuMrU1/B9Np0zdeoNwPeLLEktQpVV5ErGsOEzH0eXV1d8IpelPIl7OX2EIRzpwkroZibFYeHhxENRKtatxDIqoz9/L7hT6Ubfk8Mw6CgNDY3PJvN0rRaJBIBCxY+jw9MP4NAMYBsNktrApIk0SmMoig6KgQgSq693B79M2LFr2hKzcI+6c7XNA2Dg4NNeSE1WqepJ0s2+43xPG+cqB/OegFg2PHzPtohHg/FkSlmEIgFGi5KtwuRE2kBu1lYay9+yQ+O5SArct3aS9ATxAPlgaNRmJ1FTz6fx8HBAba3t+HxePAjT/8ITgZPQtf1imL4ccDagwI4p/I6DnQEoZhh7ZbXdR1LS0uYn5/H+fPnMTw8TDetVC6Fl7/3MjYPNuuu20z6x+ppJedk7O/so7e7F76Aj9ZLiMGdkzB335PhVH6/H93d3Yj4Ik1FQ0Q1VhGJ8SJkxb5BzJxWCwQMy36BFagnF8/zKBQKEEURXV1dtGBq9nHy+/0tj+glqJZXtxb2PaIHLNiywr65O39gYKCp6wh7wtgvNB9xmmXJuq7T6IV07VtlyaVSCRubhvdZrpSDV/BCVmSjEfbhRmcWCeTz+YqidDVLmEYhsMYcFidcI5pxGQ55QzjIHbTkNtworNELdOD7T3w/IohgcnISACBJEjiOQ7FYPLQosB6shFIqlSDLsksoTsGc8lIUBbOzs0gmk7h69apxYzzE4tYiXpl4peGTVb0IxQ66rmMjsYF8Po++vj54JA9ifqN2IXHSoUmRdV0va1aMRqPwi/6mU2vmIj/5b7NqjDTu6TBmppD3StJDEvdQHSPnwDAMisUiNjc3qRCBZY1mv1AoVKZoIid0c/H6MCIERVNoox/pR1BkBRubGzQt1eiGRS1NWiCTirUYpmy2u50smUiTe7p64BE9VIFEZr0InAAP7wHYR02V0WiUFqVJ9MKybJmdSaPkGRADUDQFiqaAg/NpH7sJjxzDgWd5JHNJGmEfRd+HwAn4scs/htPdpwGAPl8LCwvI5XL4zne+Q90c6DC4I4pc7Mb/AjhylZdT6AhCsUt5ZbNZjI+PQxAEjI2N0ROEruv41vy38Or8q01t5s1u/uZ6ydDQECRBgk/yIZlN0p+ReAlezouAFEC2lHWEXBiGwf7+PgqFAvr7+xENRgEGVbvB660FVLcpJykLVVWxubEJD+/BxTMXoUABx3B0djjDMMhms9ja2kIkEjF6UCzr2c3QyGazVeeZVNtIWo0QdOjY3tvG1tYWotEouuPd8IqGVLzaDHR67QwLn1jd0qRdmGXJxEjT6/WC0RksLy+D5dmyzwZ4NKkSeNRUybM8ilyRpnbs7EzqNVUCj6xUVE09kg2dRC9hbxjJXBIiL0JgjMMMx3KHKj4QeREfufIRnIyfpH/GMAzC4TBCoRAdgEVmk0xPT0PXdTq6Nx6PH2r0YjetEYBbQ3EKZC7A7du3KyxUCnIBX5z4Iu4m7ja9bjUjRzuQeglJNXkFr+2mrugKMqUMMsUMOJaDT/LR9EsrTXeqqkJRFGiahqGhIUQDURTkQsuDk+rJkIFHjX6SJCHaE0W6lEbYG0ZBLiDoDUJRFWxsbSCZSqK7u7uhk5P5hG4uluZyuTKbDXPjIAsWfo+/5QiBzFkhAoGKiZuSkb7Ly+XNbgJrFKTtlIFOg1xjb28vuqPdKCklFOVi1c+G1Et03bCwN8uSPbwHGjTaVBmLxcqs5KuN8bVaqRxVZ7jZfr6klJBXH5paMka0xDIsSmqp5oTHZuERPPiJ534CQ9Eh278n/R/W0b3pdBq7u7tYX1/H7du3EQgEEIvFEI/HDTdtB6OXauN/j7u20yo6hlDIhp/NZpHNZvHMM8+gv7+f/v1Wegsvf+9l7GZ3W1q/0ZTX/v4+dnd36c0T8ARQKNlv6uaoR9UeGUBSeSUvlJk/1gLZ2AEgFouhO9SN/cJ+W1FPPUIhxfdIJEIb/UhqjWEYFHNF7OzsoJAv4NzJc/B4PS31IliLpVbprd/nRzQURVEuNm1tr+s6dnd3qf2M3ZwVTddsO/Z1TUdBLRxq8x65RjJzpb+/H13hLho11ftszLJk8tlYmyq9wqPohbgv67pO+zXIGN/ucDfSfBper5eKBA4bDBgEvZX287R5V9fLvhuRFw0X5wYiy1rwCl785PM/if5wf9Wf0TSt4n5jGMbw4QuFMDo6ClmW6VTFmZkZaJpG58LH4/Gm5vrYwS7l9bjOkwc6iFBKpRImJydRKBQwMDBQRia31m/hL6b+ouFeETvUi1CIxDSbzWJgYMCY915ndnW1NBqVVz68XEmQ4BE8j+aFW36HbOzhcBjFYhFBKYi9/F7L79XuvVlBjCxJ8Z0FC5EXafHdPMOkf6AfMiNDLsiULMnM8GYLulbpLauzhmpsbweFQqGieF3rwSJWL7IsNzVnpSAXwLEcCnKB1k6Aw2ncM0uXBwYG0BXuqnpP1ZMlE9GDddaLOXrxCB6j9qUrNNohhqGJZAK5TI6KBEjBn+f5Q9nASKOknQ1RtTSstfYSkJqPXvySHx9//uPoCfbU/Dk7hZUVgiBURC/JZBKbm5u4e/cuTfHGYjFDEdlkZGHtkj8O2xUn0RGEomka3njjDfh8PgwODlZsgH7JjysnruBu4m5ZDaMZ2FnNE5jdc4eHhyGKIgJioK4fV6N1GWL+CIDOEgEMY8Od3R2qqoqGo9ja3DIGKEnhZt5ezWs0g2xwuVyOFt9FTqQnQoZhUCoZg6pIKqDCO8mkDiPpF1mT6cbWKIiSyxvwwhvw0gmK1Ty1zA8e+c5YlsXAwEBT/QTWor85svSLD6Wvan3paz1omlY2WCwejDclrLDKks0mlIqi2MqSS2qJkjypDYmciHQxjVAoVCZx3traorUDQmTWz7lVVGuUJGhE4WWNLCVegiRItOHVLnoJeUL4+PMfRzxQ3/5d07SmCMAcvZw8eRKyLCOZTCKZTOLWrVtQVZXWXmKxGB2jUAvW4VqZTIY6MDyO6AhCYVkWzz77LPx+PxYXF5HLlctZR+IjGImP4L1Pvhfb6W3cSdzB3cRdrO+tN7yBVdv8rdJcj+BpaFwuue5mlWOKphhd9Q877nVZxxOnnoAkSSiqRRTVIjx6e2G0GWVpuYeGkqRGw3Gc0QD5sIGPYRhaPwqFQob1SJ0b2xylcAxn9Ks04M9lp+QiUtlqnlqk2U8URezs7JQZPDaKWvYt1iFPIifSIVrNpl/MhDc4MIiwvz2zRbPooRFZ8sM3BIZhkMwlHzVVckKZhHdgYICq0EjtQBRFSi6tFKSJ1X6tRslWJMPme80cvZA/D3vD+KlrP4WoL1pnJQPNEooVgiCgt7cXvb29RuruofNBIpHA3bt3aaQZj8erRi/WCMUp6/rjQkcQCgAqPa03ZKs72I3uYDfedeZdyBQymN+ax93EXdzfuV/T6t1u86+ol0gBOgu+EbQqGzYryEjeP1vKQuAEBKUgOJ5zTJJM1jEX30k6MSgFaYGWYRgqVSbzXJqFdZAYSY2VlHJ/rkaUXFZPLTLql6QcWJalBNiIXJaktppRcpXUEi3g1yrsV/zewwhPkiT09fYhINWPdpuBVZZsF9n5/X7EgjGktTQ4jqvZVCnrcoXEmRwszGk46rxcAyIn0tELtdBuD4o1eukL9eHHL/84wr7GI/tGUl6NgmEYWgs7efIkFEWh0d/c3BxkWS5TjpHoxWqES2oojys6hlAImhmyFfAEcPnEZVw+cRmyKuPe9j3cTdzF/NZ8xenIvEGb6yX9/f3wer2GH1fhoGkpMlmv0YeDGCn6fD5jsqIvTDcbVVORlbPgdR5BNmgMRIJh592srYX5GomslBTfoRufHbGdh46yuSqNhOqNwOrP5RUM+4u97F7Ta5E8f7FYRHd3NziOoz0vmqZVpH/McEIWXK2wb7XiJ99vMBhET1cPJEFq2X26UVgjO13VUcgXkEgaEzWtkm0SPfEib/QYmepiMieXNVUSS5j9/X3s7OzUbKoUeREMmIZThU6ldbqD3fiJqz+BgNTcyb7dCKUWeJ5HT08PnRlEIsmtrS3Mz8/TQWPErZzAraE4BHJztTpkS+AEnO87j/N956HrOlZTq7i7dRd3N+9iJ7NDIxTyMOm6btRLBBEBT6CldAS5ZmvYWg3E8TYWiyEWjRmzRCwnV0J8FbJXj3E6biaCAh6poHp6ehAMBsGAoVMOiVAhkUhAUZSmCtvNQtVVY6RrYd8w5hSDAGNM3FPU2t93NYNH0pVONj27npegL3gosmA7K/5sNovEpmFZ0x3rBsuwjo2rbRQewQNd0I3elqCvQrJNIIoiJWZd15FXHhX2RU6EJBipLpZhK+TfZq8sIhCIhqK0htMInFKY9YX68JPP/ySd4NgMDpNQzGAYhhL+yMgIFEVBKpXC7u4u9vf3jemmiQReffXVI095PXjwAL/5m7+Jv/u7v8Pm5iYGBgbw0z/90/g3/+bftOTC0DGEQtDuGGDA+AKHY8MYjg3jHzzxD7Cb2cX3Fr6Hb2S+gbW1NXg8HvT09MAjeMCxXMvpCHIz1ns4yKaeTqfR19eHaCgKlmVtmxXtUl2arpWNfvWKXgjcowl11V6TykUf9o8InADoQLaUBcMw1KKE47imC9vNgE5XfFifsNrBUAdbpVJiXc/gkWEYSJJEO8nNG+ju1i52sQvBI8Dv9zvug0WgaArWttaws7ODnp4eDHUPGZFhm268zcIreKFoSlnq1yxLLhaLdDqlpmlYXl6mdSlzxKFoCuTiI4dh0lRZUktgWbaiqTJ3kENyJwlBEmj0Uu9g4oTtymBkEB+7+jFqUNksjsvLi+d5Ok0xn88jHA5jfX0dX/va1zA5OQm/349/9a/+FT74wQ/i3e9+t2MZAzvcvn0bmqbhD//wD3HmzBnMzMzg537u55DNZvE7v/M7Ta/3tiQUK+KBOEZ9o9AHdIycHkFRKuL+zn3c27nX9gmyXq1D0zRsbm5CURQMDQ0hEogYDW0l+yijkQZM66xwn+ij3ciarpUV3wVBMFxwBR8KyqPieyszTFpBI9MVrQ62xJgzk89gY3OjKYNHsoH2xntRkAvI5rI03WBVRjkRjem6jr29Pezt7aG/vx/xcBzpYpoKEkiqjxqN4nDmm/hEnzFGukpqlIhPzE4HpC5l9WKrJUuWeIkKFViGRXe4G3klj2KpSKMX81qk/mKnNmznnhuJjeCjz30UIt+6l1mjmYXDhKZp8Pl8eNe73oVvfOMb+KVf+iXs7OxAURT8i3/xL6AoCpaXlw/t+fzABz6AD3zgA/T/T506hTt37uAzn/nM24NQWk15VYOmaZibm6OOvRfOXgAAPHviWSiqgge7D3A3cRd3E3dbtjepRiikEC6KotH57o/WrdM0W4wnzsLAw/QEI2I9sQ6JlxDrjhkW3RqHTCEDhjXGrZLUG/EtOqybtZXpisQORpZlJBIJBD1BDA4MoqgWa4ouzAh6jOFUZMYIOeGR1Fg1ZVSznwOJAnO5HO0xIaROUFHYFwN0wFMzVvy1EJACNVVoxDYnFoshHH5UtCZNkFYvtmrkq+u60VRZMK47KAWNKZScB6zIguO4iqbK3d1d6rxM1iMS51bvu9GuUXzkykeMiLsNHFXKqxaswoBCoYBnnnkGv/mbv0nHVhy1hHh/fx+xWKyl3+0YQiEfmpMRSrFYxPj4ODRNw3PPPYc33nij7EbmOR5nes7gTM8ZfPCpD2J9f90o6ifmkThINHzddhtmLpej42a7uroQ8oQaqtO0o+7KZDP0NWOxGLyCFyd6TiCxk0B6K003zUKh0PSwqaagA2Ff6zJZEj0FAgGE4iHqaeUVjFQfKYTD5jkLeUNVU5j1DBur9bzYwdxUOTAwgHgwXrfob1vY5ySjh6eGFX8t1Gu+JYcH0sBaDWZZMlBJvuZZL16vF0GPoRDUNK28qZKXoOqqUeg3+bpZ58bwPE9/t5kN82zPWXz42Q+D59rfujqRUMzTGhmGQV9f35Fez8LCAn7v936vpegE6CBCISCy4XZD4r29PYyPj9P5KYSkqoW5DMNgMDKIwcggXjr/ElK5FI1clpPLVdNQdnJk0oXe3d2NaDgKSZAa3lxbJRQigSYjbhndiEZUQUX3QDe61C7s7eyhUCwAOrC7u4t8Pg+/31/TrLFZsGDh9zbvjExATtN2EyDzcp7WWAT+oVOyrlNjTjuLj2qoNkfebBHv9/vp6dx8HSSlCBh9HLFArKX3a1fYBxrv2A95qpMn8Mg7rJXDg1WWbJ714mW92BK26KA1Uti3NlV6+YfjdpkCeJ4va9Dc3zfm9SwvLzfcVHmh/wL+0TP/CBzbfppK1ytnuR8H7AjFiaL8r/3ar+FTn/pUzZ+Zm5vDE088Qf9/bW0NH/jAB/DRj34UP/dzP9fS63YcoZAP16rPbgarq6uYm5vD2bNnMTIyUrEZNHITRX1RvDD6Al4YfQEFuYCFrQXcTdzFwtZCmcrKKkcm44EHBwcRCUYq5kTUQ7OEQtIumUyGWsYIjACGY5AtZmmunPQU9A32GQ+kYty824ltqHq51XyrpzaO5SDxUkupQ+DRBtiICaV5uBPHcgh6glSh1Gwqya7nJZvN2k5i5HkeiUSCNrWZTQ/bgaIp5R37D+3erT08AOpGgGZVXDV/s2ZgnvUS8oSwvb9dVVVHGiE1TUNOfmgzZGqqVDSF3pMsyyIajSKXy9GmQJKGJJMqybP79ODT+OGnftixiIIcAjstQnFKNvwrv/Ir+MQnPlHzZ06dOkX/e319HS+99BLGxsbwX//rf235dTuGUMwpL8Bo/muWUMzz5q9cuYJ4/JH9ArlxWkmneQQPnhp8Ck8NPgVVU/Fg9wHmE0ZDJSEAs30LcQpuxRuqGUKxFvx5noeH86CklqDIStkMk4qucgHwR/zwhX1gVAbFfBF7B3vUWp0Mymr0OyBKrlZEDo0YPFYDz/EQObFsc63WI9Lwmg9P09ZJjFtbW/SgE/AH4OW9LZNnLejQy4dVWQr7tSJA8llms9mmxx7XA2lIraaqI/e/ufZibarUdR08Z9ynnMjBI3kgCEJZgyaxhSFpyGunruF959/n6ObfCYSi6/qhRShERdYI1tbW8NJLL+G5557DH//xH7f1mXQMoRCwLAuWZZve+IvFIiYmJqAoSsW8ecDYqJ2oz3Ash9Pdp3G6+zQ+8NQH8BX2K9jHPsYXjdktPT09iPgjxkS6FnLijRKKLMvY2NiAIAgYGjJkqgEpQIvCjcwwIa8HHpCCEnqDvYAGqEUV+5l9LO8uQxAEmvqplhrzS34U5WJLNvukFlEqlZrugzG70ppRkUoSvNAZnY5zbgbkdM6yLDKZDEKhEDyiB4VsAbe2b9HTud/vd3wEMgEp7JNBVbquI+QNIV/KlwkVSIRcLBYxMDDgXE+RDoS99u4GdmN37exyzLLkfD6PxI7hcq0oCpUlE0NLsqEWi0WcCZ3BGekMXnvtNQSDQdppHgwG2+u0f0gox5nysl4DaYA8yj6UtbU1vPvd78bIyAh+53d+B9vb2/TvWqnfdByhAM0X5vf39zE+Po5IJIKrV69WvUkOQ5IcFsLgszz+xT/4F4j1xrC4s4jbG7eRKWRacq5thFCILQZ5wHRdN2xF8gf0Idvb20MqlWp4hgkFC3BeDjFvDLGuGHRZRzaTxdbmFjRolFxIT0crSi6CMr+rwcGmHu5G5MjAw1TSw54XYs/PcZxh2NlggygZitXd3Y2uaBcAw1cqqkZp4ZpMT7TOMnEKLMPCK3gruu5JYb+klHBv+R5UTcXAwEDL6WIrGDAIeAMNzampZpdDalMsazRJ5vN5RKNRRCIRo5/lYeRCUpYewQMdOm6M3sAPnPsBsCyLYrFIrUyWl5fBcRxisRi6uroQi8Wafr+qasjnj9OEkexFVnPIoxz/+zd/8zdYWFjAwsIChobK58a08kx3DKGYv9hmpMNra2u4desWzpw5g5MnT9a8QVqJfKpB13XcvXsX2WwWQ0NDGB0dBQBcOXEFV05cQVEuYnF70ai7bC803ORWj1Aq/LZ00I578rtmu/S2ps0xACMyCMQCCEQDYDQGhVwByb0kEokEusPdOBAOmh7xCzySVJMm02Ye7KAniEwh03QESAr4BCJvpJJqpcYIMff19aEr3GWkEx+SmPV0TlJjTve8CKwAjuPKrp2gIBeQLWQNKxWWx5nRM2A51hErfhZGRNRqWs8sSyb9OqlUCizLIpVKIZ/Pl0Uvuq7TA8D3nfk+3Bi9AUUxohae59Hb24v+/n5omkZFKPfv38fs7CzC4TCNXhqZJ9IpCi+gPO121NYrn/jEJ+rWWppBxxCKGY1EEpqm4c6dO1hfX8fly5fR1dVVd916xpONQpZlTE1NIZfLIR6P2ypoJEHCkwNP4smBJ42u5NQy7m4aqrFULlV17VqEsrOzQ4c0eb1e8AwPjuWQKWbKZpg00wjYMBhA53RIQQkDwQF4WA/SmTQO0gdY3lmGKIo0eqmX+iERVjgcpoO9GkUtt+BmYZ69wbM8PKIHDBjkSjkoqlI2FCsejle1TAdAbUiqyW5JsblZVZ3ES9ChV50FQoxGeZ5Hd283snIWkE2Ffba5aIyAZVh4xcqIqFWQ5kmiQrSTEpPP7x9e/od4YfQFWmPQNK3suWVZlt47Z86coV51hGAEQUBXVxfi8Tii0aht5NtJCi+zJ6DrNnwI4DiuZoRSKpUwPj4OWZZt6yXV4ESEks1m8dZbb8Hr9eL69euYm5ur29nOsixOxk/iZPwk3nfxfdhKb1FJstWC39Z6xa74znuMDVEt1Z1h4iSINXmulAPv4xHzxRDTjdRYJpPB5vomwIJuntYuadIX0ayjMQNjfG2rI4LrQdGUR/Y2OpBOpYEScGrkFALeQNnY3EZQreeFFK7NqbFqG5udlYoZpI7m8XgqbPyrFfYVXUG+mK/ZsU/Ues2oE2uBpAx7e3vp6ds66yWfzyOfy2OUH0V+KY+p9BS6urrQ1dVFay+EXIgIBnjkvNzf34/BwUGoqoq9vT3s7u5ifn4ehUKhzOWX7BXHZbtihpXUCoUCVFU90pSX0+gYQrGmvKpt/OZ6yXPPPdfUKbzdGsrOzg4mJiYwPDyMc+fOtVzo7wn2oCfYQy34727dpRb8VkIhmwbP8xgcHDScYMUAsqVHxXfSnNfoDJNWQU7LFUquh6mxYCyIYCwIRjGuaW93DwklQVM/sizj4OCg6b4I4hZ8WGRiBiFvTdPQ19eHgC+AolxE0BOEoilNDxED7Htestls1TG/DMPUtVIpFovY2NhAIBBAPB6v+53bduwzLPJyeWGf53jwLO+YqeXBwQF2d3drfucsyyIYCOInb/wknh58GplMBjs7O9jY2MDt27epPVBXVxft9Nc0jf5jjV4IgQCgKcidnR0sLCzA4/EgHo8fmoCiGViHa5E5UG6E4jCqRSjr6+uYnZ3F6dOnMTo62vQN0Sqh6LqOpaUlzM/P48knn8Tg4CD9u3ajnoAnQOsusipj6v4U/vL1v4RP8mFnb6ei+E467klB0drQeFjwS346iKsedF6HN+SFJ+gBq7Mo5ovYTe4iV8pBFEU6dtbaMGgHIgt22i3YDkQkwHEcBvoHEPVHqTy3bIiYZEzUy5VyTSvbzIXral3k3ZFuFIQCJI99aoykDGup92rB2rHvFbwQeRGqqkJW5YZH7daDubGylsEhy7D40Wd/FE/2PwkAtC5lnum+s7ODyclJ6LqOeDyO7u5uSgwkNUYK++boRZIkDA4OYnh4uMzld2VlhaauSfTSbr9Os7BGKJlMhs6feVzRUYRCTufWCMVcL3n22Wcb1ldb0QqhaJqG2dlZ7Ozs4Pnnn0ckEin7e5ZlHfMeIxb8yb4knnjiCbz61qt4+pmnkdJS2M3sIiAFKJlYezcO8yZsVcnFMAwUTcHO/g40RsPJEyfBqAwy2YzhMWZjRmiGxEvQdZuI6BBARAJerxe93b3GSAObiEjV1QqnZLshYo3CmvrhNR6JVAK5VI7OeSGpQ57nqRSc+LA5gbxs1IY0XaOKQQAVvmTNIJVKUcPMWhs1z/L48OUP41zvOdu/t850J3NZlpaWMDMzQ62Nurq6qDiCkItd9ELIKBwOY2VlBeFwmE5YJFYxZODeYafEqvWgHHfk1A46ilAIzBt/qVTCxMQESqUSrl+/3pYCguO4pkb2Ei8wXddx48YN2weD4zgUi80VPGuBFNfv3r2LD7zrA4jFYlBVFTuZHaNbf+suVnZXsJnYPPQZJrX6DxoBscfneR4DAwPGAyoYBBWKhQAFyOfySO4mkVASZd36QW+wIVmwEzAPxeru6m6qGG03RKxVZ+GIL4KDwgG6u7srhlttb29T9WM0GnU0GiVKN/JZl3XsP5RZl5RSQ5EL6dI/ODioqzIUOAEfufIRnOo+VfVnzGAYhkZlZ86cQaFQoNHLgwcPwHEcJZd4PE5NKEnthfwDPGqcHh4exsjICJ0Pv7u7i5mZGei6jlgsRgnGyQZRAmsdh0xrdAnFYZAH5+DgAG+99RbC4TCuXLnStmqpXrHfDFKricViuHjxYs3elmbnyleDoii4desWAODatWvw+/30YegKdKE72I1nB57F62++jm1pGwgDy6llx1xrzaCeXC2Sidng0TbHzwAQAG/Yi4HQAFiNRalQwt7BHtLJNBJCokxSelgwO/F2xbrAsVzLxeiqdYpSHrJWwynZxkrFOucllUohlUpBkiTs7e3h4ODAkZ4Xn+hDUSnapjJ12Mis+eqFfXOX/sDAQM3vTeRE/MTVn8BIfKSl6wYAj8eDwcFBDA4OQtM07O3tYWdnB4uLi5ienkY0GqUE4/P5KLkoikItY4gsmWVZdHd30/nw6XQau7u7WFtbw9zcHE07k0jIiU3fai+Vy+UOz7D1iNBRhEJSORzHYW9vD2+88QZOnTqFU6dOOfIFNrr5b2xsYGZm5kh7W3K5HN566y0abUiSVKZTZxgGe3t7mJiYwEDfAN597t1Guk1VcH/3PuYT85jfmnek1kCUXK32H9QyeLQDwzDQOR2CX8DprtNIZ9NQSyrS6TTW19bBciyVJLdiM18NpGBMGhY1TXOsfmCtU5AhYrIqG0PRHr4F2jhYx5eLnPg9Ho9jPS9+yV9TCm2FWWZtLeyXlBJ2dnZo/1Ota5B4CR9//uMYjA5W/ZlmwbKsMQk1FsO5c+eQy+Wws7NDi/GSJNEmyNXVVaiqiqeeeoqau1pTY0REMTo6ilKphN3dXezu7mJiYgIMw9DIJRaLtZwhcCOUI4Cmadjd3cXe3h6uXLnScr3EDvXSU7quY35+HsvLyw3XapyIUFKpFMbHx9HX14fTp0/jG9/4Bu2YJTfc+vo65ubmcO7cOQwPD9Pf5TkeZ3vO4mzPWei6jo39DSpJ3s5sV3vJqiDDk1qtWzRj8GgGlQXn9wEWYD0swp4wwl1hQAFy2Rx2tnag6mpZXaGVXgLrSOF4OH7o6bWyIWIPnZI1TQPDMlWJu9qJ39zzous6Leybe17q2eWYZ8a0AjNh6rqOg+QBWIXFmZEzKGnVI2av6MXHn/84+sP9Lb1uo/D5fDhx4gROnDgBVVWRTCaxvb2N6elpaJqGeDxO5eter5eSip0smeO4sqZKchBZWlrCrVu3EAqFKME0UwM5LB+v40RHEQrpL8nlcohEIo6SCVC7KK8oCiYnJ5HNZnH9+vWGv9h2IxTS6X/u3Dl683d3d+PNN9+Ez+dDV1cXSqUStra28Oyzz5YZXlrBMAwGIgMYiAzg3effjVQuZZhYbt3FSnKl7knUL/rpVMdm0Y7BY01Z8MPUmC/iM4wstYdGluk9Krk128w3cp1kKNbg4CDiwTiVYB8VZFWGpmvw8B7kijkEpAA4hjNO+g9TZo36cpE+DGvPSzabrdrzUmtmTLMgQ6DIXJiSXgLPGVb8Osr90/ySHz917afQE+xx5LUbBbFpWV5eRiAQwPnz57G3t4fNzU3cuXOnaVlyKBRCJBLB6dOnaR2HEAzP85RcotFozTS9SyiHjOnpaXAchzNnzmB9fd3x9asRSjabxfj4OCRJwo0bN5oKYVuNUMzR0OXLlxGPx+kJ6dKlS1BVFdvb21hYWEChUIAgCLQ/IhaLNWzBf230Gq6NXkO+lMfC9gLmt+axuL1IUxcE7XhytWPw2JQs+GG3vhgQ0RPoQa/WC7Wk4iB9gJXdlbrd6JqmIZFIUDFDPBA/kt4WK6xWKtahWzzLY2V1BXJJbtqXq17PSzwYx56w50htinyeqmr4h1GncKsVv+hHyBvCjz77o4j7qx+IDguqqmJiYoIO2uN5HpFIBCdPnqwqSzY3VRLFmF30IggCbaokdZzd3V0sLi4in88jEomUNVVaR2mYRQvZbNatoTiJZ555BizLYnt729ExwAR2hELyooODgzh37lzTxU27AVv1oCgKpqamkMlkcP36dfh8PkompF6iKAqWlpbg8Xhw7do1Y3bJ9jbu3LmDYrFI5Y/d3d0NbQxe0YunB5/G04NPQ9VULO0uGQ2Vm4YFf6t2JuZZK80aPFZtlGwQOquD9bCIeCKIdEUA2UiNmWe8kNSYruuODMVqF0QKXa1WQ3y5AODE8An4PYaqsRUJr7Xnxct5sZncNJR1JrsTv9/fdG1K0zRsbGwAwCMFnw106BB4AR++/GFEfdGmrt8JKIqCiYkJAMDly5cryNkqSyZeeSsrK5idnUUoFKLkQjzJakUvkUgEsVgMZ8+epTWu3d1d3Lt3D6IoUgUaiSbdCOUQIYoi/ZCddgUGyglF13UsLy/j7t27uHDhQoXTZitrNoJ8Pk+L7y+88AIEQagovh8cHGBiYgLxeBwXLlwAy7KQJIkWHEnRmwwSC4fDlFwakVVzLIdT3adwqvsUPnDxA9jY38D81jzmE/PYPNhs+L20Y/DYqFtww2AAiMa63ogXrMaikCtgb28PiYQxzlkQBPT19pU1LB4lrPJcK4gvFxmDoEF7dNJ/6JTMsiyKSrEiwqwJk4rMbs4LGYFMalP1JieqqoqNjQ1aW6h1CIv74/ipaz+FkNeZnplmoCgKxsfHwTAMLl++XPewwzAM/XxOnz6NYrGI3d1dbG9vY3l5GSzLUnKJxWJNNVWqqkqbKu/cuYNSqUSvJ5/Pw+v1IpPJPPaEwuitVuUOAUTSR8b3vvTSS46uv7W1hfn5edy4cQO3bt3C1tYWLl++jGi09ZNTM9e6t7eHt956Cz09Pbhw4QK9CYFHjqOJRAKzs7M4depUxbRJOxQKBezs7GBrawvJZBI+n4+SSyMKKysO8geY35rHncQdLCeXq9ZT2jF4DHqCbTXNNYNisYidxA78HmOOSKFQgMo9mlDp5PjjWqhnpVLLl8sOHt4DSag/RIwBg4AnUFOxR3peyJTKUqlUMceEXI+Z9Hp7e2teZ3egGz/1wk8hIB39JknIhGVZPPvss20bQZplyaQGF4lEKMH4/f6Kpkrz1krmPLEsazTr5nKYnJwEy7IYHx/H7//+72N0dBQ9PT34kz/5k0OVypvxj/7RP8LExARVZb7nPe/Bpz71KQwMDLS0XkcSSiaTweuvv473vve9jq5PmpYkSYKmabhy5UrbdgvpdBpvvPEG3vOe99T8OWIbc/bsWZw4cYLefCQq0XUdDx48wP379/HUU0+hp6f5wqWiKPREtbOzA4Zh0NXVhZ6enobrLmYU5SIWdxYxn5jHwvYCTdO0avAIOOsWXA/k9B2NRhGPxiEJEvLFvGFkmc0guZ+0nfHiNIJSsGbhv1lfLit4lodHMFJWZqJmwcLvad5+3jzHJJ/P0zkvHo+H9sLUi0j7Qn34+LWPwycefU1AlmWMj4+D53lcunTpUFyF8/k8JZdkMknTWSR6Ic+0WTlGQJ75t956CyMjIxAEAX/xF3+Bz372s7h9+zYYhsF73/tefPjDH8ZP//RPO37tZvyn//SfcOPGDfT392NtbQ2/+qu/CgB47bXXWlqvIwkln8/jm9/8Jt7//vc7enpcW1vD9PQ0+vr68PTTTztyo2WzWXz729/G+9//ftu/13UdCwsLWFpawqVLl9DV1VVRL9E0Dbdu3UIymcSzzz7riKUGOVFtb29je3u7pbqLdb3l5DK+OflNjN8bhz/qb6qAeNhuwVYQ0uvu7kY8GgcDptLCXQegAoVcAamDFPKlR/M5WpnxYoeQN4R0vrpTcbu+XFYwMIwlOY4DA6bt1B7peclkMshkDPGAOTVm9xkNRYfwsasfg0c4Wm8swCATklI+LDKxgsiSyWGuVCrR4V/VZMmqquLmzZs4c+YMurq6wDAM/vE//scYGxvD+973Pnz1q1/F/v4+Pv3pTx/69ZvxpS99CR/60IdQLBZb6q/pqBoKAblJrZ2k7WBzcxOzs7NgWRaXLl1yjKjIzGxd1yvWVFUVU1NTODg4wAsvvFDW+U7IpFQqYXJyEpqm4YUXXmhvIJYJ1kYvUtRvte4CAJnNDE6Jp/DjH/9x5PQcbaZc31uvae1+lG7BxO+JDMWKh+NQNMXeTYABwAOekAd9wT6wOgu5IGM/vd/0jBc71JPnHoYvlw4deTkPDzzIlXKQeAkewQNFU5Ar5VryY+N5Hvl8HqFQCMFgEPl8Hul0Gjs7OxU9LyPxEXzs6scg8keTsjFDlmXcvHkTkiTh0qVLR2ZPz3EcfZbOnz+PbDaLnZ0dJBIJ3Llzh8r/u7q6EIlEqDchGUBGaqiLi4u4evUqrly5gitXrhzJtZuRTCbxv/7X/8LY2FjLzZodRSjkgSWnCicIxRwhXLhwAbdu3XI06iE3rfVaC4UC3nrrLXAch+vXr9sW3zOZDCYmJhAKhWrau7QLMqebdP6Susv29jYWFxdpUb1a3YW4ssry/7+9M4+L8jrb/zUM+77OsIgICoLKjhKNMRpN3AFjTJtEo83SVs3WX5uYpEmatE3SxDS1jYlN3xi1afU1ERTjvgFqXCKrLMqOyDYzMDAww+zz/P7gPSczyM7APKPP9/PxD2FgzgzPPPc593JdWsycOROOjo5whSsEbgLcP+V+yNVyOu9S11pnUnQmU/fjoRZsPAsTGBgIbzfvHoXkfuoWxvB4PDA8BrbOtvBx9oEv4wuDxgC54iePF+PU2IDXUB9SKr0hJyiBQGDWQqwt3xZ2Nna0c06t+8lci2/To5QMBujWDs3RkaTj3N3daa3MwcGBdimR1FhzczMC3QIRPSEa7W3tVEtrvNBoNMjPz4ejoyPtFrUExp810pYslUrR2tqK4uJi6PV62NnZ0ZS7q6srDAYDvvnmG1RVVd0hPjsebNmyBdu3b0d3dzfuu+8+HDlyZMS/i1UpL4PBAK22R/Po5MmTmDt37qjEII3bc4kWWHZ2Nh555BGzXXB6vR6nT5/GQw89RNNIMpkM+fn58PX1xbRp00yK70R2nlxgEydONJu0zEjor+5C5MHJsKmTkxOio6MHvUlo9VrUSGpQKa5EvbQeCo1ieB1JI8R4EDAgIAA+bj4jnqu585cDPH2Px0u7rB1qnbrfjqihFMGJmoBQKDTr3IE9v+cUNRR3Rpoas+H3q5SsUqnQ3NxM/d8HIlwQjodCH0K7tB0SiQTd3d1US8vPz29M5ys0Gg3y8vLg7OyM6Ohoixtn9YfBYEBRURFkMhkcHR1x6NAhnDp1CtOmTcPx48dx6NChflPnw+H111/HRx99NOBjbty4gcjISACgdaBbt27hvffeg4eHB44cOTKiexJrA8rZs2cxc+bMEacCiDYWOf4SIbgzZ85g4cKFZlPoZRgGJ0+exPz58+Ho6GiiAxYSEkL71cmpBADq6+upt0pAwNhKUAyHvuouDMPA09MTM2bMGHYDA8MwaGhvQIW4ApWiSrQp2sZk3cbWxwEBAfBy8Rqzwj/DMOAzfKhVanTIOtDV3QUHx56OKFdXV3i5evV7GjPW5fL39zer/waZbxmpUKi9rT0cbR17lJI13dRd0tvbm06P98e0gGlIjU29wxvduGjt5OREg4unp6fZbvrWEkwYhsGNGzcglUqRlJRE7xWffvopvv/+e9qhuXTpUjz55JOjCiwSiQRtbQN/1sLCwvqsozY0NCA4OBiXLl3C7Nmzh/3crEx5AaNzVyTDioGBgZg6dSq9yIzTU+YKKESpVKfToaqqCrW1tYiNjYWfn1+fxffy8nKIxWIkJiZa5Hg7EMZ1F09PT5SUlMDb2xtarRYXL14cdt2Fx+Mh2DsYwd7BWBi5EG2KNqoz1tjeOCxL3f4w9lQPDAyEl7PXmNZqeDweDDwD7Jzt4OfsBwEjgF7dI2TZ2tyKZjT3OSxoLPkymBLvcBlsvmUoGAs/KpVKyFplCA0Kha2j7YC/NyYoBitiVtyxmzXW0tLpdH1qafW2+B0uarUaeXl5cHV1pUKPbIRhGNy8edMkmADAtWvXsHv3bvz3v//FihUrcPnyZRw9ehSlpaWjCijkMzoSyJD2SC05WHVCIf3wAHDx4kVMnTp12G9MfX09ysvL+x1WNEcqrTenT5+Gl5cX5HI5EhMT4eLicsfJRKvVori4GGq1GnFxcax2Zauvr0dVVRWmT58OoVAIACZ1F6lUOmjdZTC6Nd10mLK2tXZEO2tjUyyBQAA3R7cRKySPBiKlotKowNPzoFAoIJVJodVraceYQqGARqNBQECAWf1rBpKfHwnE/924tuNk5wR7vj00eg2UWiV9bOLERCyePrxOTONpdIlEArlcDnd39x7FZ1/fIYsrkmDi5uaG6dOnszqYVFRUQCwWIykpiX7ujx07hvXr12P37t1Ys2aNRdZ29epVXLt2DXPnzoWXlxeqq6vx9ttv01m4kTQIsTagXL58GaGhofD39x/SzxoMBty4cQMikWjAYcUzZ84gOTnZbAZFKpUKOTk5cHFxQVJSEp32B36ql3R3d6OwsHDIdQhLQS7+5uZmxMXF9XuCGqzuMtzmgpFI8BsPVvp6+8LJ3snEu2O86DfVxKBHyLJbDUm7BCqtykTI0hwnlOHKzw8G6dwSCAT9brjs+HZwsnfC9MDpeGjqQ6N+TrJRaW1tRVtb2x3zHH1dSyqVCnl5efDw8MD06dNZK/dO9PpaWlqQlJRE60hnzpzBk08+iX/961948sknLba+4uJivPzyy1QUNyAgAEuWLMFbb71lYnM+HFgVUICfjlrXrl1DQEDAkCRRSOFYp9MhISFhwN1/VlYW4uLiRjUdTyAGYFqtFnFxcfD29qZHRhJM2tvbUVRUhICAAERERLD24tfr9SguLoZCoUB8fPyQi6jmnnchEvzlonJUiir7lOA3brf18ewJYObyMRkOg6Wa9Ho91TkLEAaA0TLolHeivbMdfFv+qDxeRis/35vhNArcP/l+zJ863yzPawyRJyGnFzLPQU4vjo6ONJh4enpi2rRprP08MQyD6upqNDY2IikpiQbonJwcrFmzBp9//jmefvpp1q5/pLA2oOTn58PHxwchIQM7upGbOikcD7b7P3/+PKZPnz6gDPxQaGlpQXFxMSZPnoyGhgZERETQCdneHiZTp04dsVbYeKBWq1FYWEhlKkaakmEYhs67SCQSdHZ2wt3dnabGRpJm7C3B397RDqlU2jP97+E9qkL0aHBxcIFK039Lcm9dLpOUDAMwOgZKhRJtsjYqZEn+DZa+cXd0R5eq/2HJ4dLR0YH29vYh2Q7Mj5iP+6fcb5bnHQjja6m1tRUymQzOzs5Qq9Xw9PQc1zmTkVBdXY2GhgaTYHLx4kWsXr0an376KZ577rm7LpgALAwoGo0GDMOgqKgIbm5uCAvr32+a3NSH4+r4ww8/IDw8fETSJkDPhV5TU4OamhrExMRAIBDg8uXL4PP5mDBhAnx9fcHn81FVVYWGhgbExsbC29t7RM81HsjlchQUFMDT09PsuWi1Wk2DC6m7+Pn5QSAQDLvuwjAMSstLca38Gmx9bNGp64RcJR8Xz/neDCalotVq0dTUBCcnp8F1uRgABkCj1KC9sx0KlWJAjxdzStcYd50FBAQMmjN/OOphzAqdZZbnHi5k42hrawutVmsi1DjeMy+DUVtbi1u3biEpKYnWoa5evYq0tDS8//772Lx5810ZTAAWB5TS0lLY2dkhIiLijseQ42RtbS1iYmJo4XgoXLlyBSEhISNq19Xr9SgpKUF7ezsSEhLg5uYGvb6nw0csFkMsFqO7u5veBGJiYsySWhsrpFIpioqKEBwcjMmTJ4/pRd677gL81I0yWN2F1MekUini4+Ph6upqIsFfKaocN32wwaRUyCCgm5sbPbEOB56BB51a15Ma62o3mUQXeArMGkykUim6uroG7TrjgYelM5YifmK8WZ57uCiVSuTm5sLX1xeRkZFgGOYOoUYvLy+aGrOkp0hdXR3q6uqQmJhI67R5eXlISUnBO++8g1deeeWuDSYAiwPKzZs3wTAMoqKiTL6v0+lQXFyMzs5OelMfDsOpzRijVqtRUFAAhmEQHx8Pe3v7Ozq5yHS8wWCAnZ0durq6Rp3yGStIOi4yMnLEBbiRYjAYIJPJIBaLB627kOFUtVqN+Pj4flMyI5XgHw6DSamYW5cLDABdT7Fc2a1Et67bxONlpKdJ4xbmgICAAYOJDc8Gy6OXI2ZCzAhfxOjo7u5GXl4elTXp6z0lMy8SiQTt7e13SJ2MV2rs1q1bqKmpQWJiIp2fKyoqwrJly/D666/jtddeu6uDCcDCgKLVamEwGFBZWQmVSoXo6Gj6ve7ubhQUFMDOzg5xcXEj6pQZam3GmK6uLuTl5cHLy4t2lfQuvstkMhQWFsLPzw+RkZE9vhVGKZ+2traeXeb/BRd3d3eLXFwkZVdfX4+YmJhR15LMsZ7+6i6enp4oLy8Hn89HbGzskGs7MqWM1l0GkuAf+iIBDyePAedbxkKXC+g5Hbg5uqFT2QmegQeVUgWpTAqlusdDgwSYoaZ8GIaBRCKBSqUatIWZb8NHamwqogKi+n3MWNLd3Y3c3FwIhcIhN7SQkzA5vZCZF3ISHitZ+Nu3b6OqqgoJCQl0ELS0tBRLly7Fyy+/jLfeeuuuDyYAiwNKbW0tZDIZ4uLiAPSkZwoKChAQEEBv2CNhKLUZY8RiMYqKihAWFobQ0FATKereHiaTJ0/GxIkT+7xwdDod3UW1traCz+fT4OLl5TUuuyjj1FFcXJzZWqfNCQnCLS0taG9vB5/PR1BQEIRC4YjmXfqT4B8qQ5FSGStdLh6PB1d7V3Sp73xunoEHnUoHWZcMHYoO2Nvb0+DSn5Clsf97QEDAgEHI1sYWq+JXIUJ4Z8p5PFAoFMjLy4O/vz/Cw8NHdDMmMy/kMyeXy3vazP+vxd3FxcUsN/mGhgZUVFQgISGBttrfvHkTS5cuxfPPP48//elP90QwAVgcUOrr6yGRSJCYmIjbt2/j5s2biIyMRHBw8Kh+P/FDCQ8PH/BxDMOgtrYW1dXViI6OhlAovGPynTymrq4O0dHRQx7CNBgM1ONbLBbDYDDQdA8p6psbY4HHuLg4s8p+mBty2iM+LiOpu/QFkeCvEFegUlyJju6OAR9PVJIHmosZK12uoTw3hQGgBbrkXZB2SgEeaMcYSY0Z+78HBAQM+N7Z8e3wWMJjCPMb2qbL3CgUCuTm5iIwMBBTpkwx282493Cuvb09/cx5eXmN6HPX2NiI8vJyk9m3yspKLF26FGvXrsVf/vIXVnejmRvWBRSdTge9Xo+mpibU19fD3d0dzc3NiI+PN0u31I0bN8Dj8agwWl8YDAaUlpaitbUVCQkJVGKaSNTzeDzo9XqUlZWho6NjVLt9sosiwUWlUsHb25ueXsxxRFcqlcMSeLQkra2tuH79OsLCwjBp0iT69b7qLuR98vX1HdFUr7hLjApRT1G/WdZsUmjn2/DhaOvY77DkWOpy2fBs4GQ38kFNnq5nmLajswNKTU9qjHRGDRZM7G3t8bOkn2Gi98SRLn9UyOVy5OXlISgoaEwbRcjMC9msaDQaEzmYofw9m5ubcePGDTqDBvR0eC1ZsgSPPvoo/va3v91TwQRgcUBpbGxEWVkZnJ2dhzVoNxgVFRXQarWYPn16n98nQ5J6vR7x8fHU3dH4ZKJWq1FUVAQAiI2NNZuHCfBTLp7UEzw8PCAQCCAQCEYk12K82zfWNWMjpFFgMNHM/uoupCV5JM0PXaouVImrUCGuQEN7AwD0mx4bTlF7uNja2MKOb2cicTJSGIYB9ECbuI1aBds79O/x4mjniJ8n/RxBXuPbpEEgwWTChAnjqsDNMAzkcjk9vXR2dsLV1ZWeXvqqd7a0tKCsrAyxsbG0DllfX4/Fixdj2bJl+Pzzz1n9WRsrWBlQOjo6kJubC51OhwULFph1R11dXQ2FQoGYmDu7Vrq6upCfnw8PDw/MmDGjz+J7V1cXCgsL6aTuWDrCqVQqkzkOFxcXGlyGonlExPiG6k9vKYj9cV1d3YgaBfqbdyHKtsN93RqdBjWtNT1SMJJKKDU/3dx7y+SbU5fLjm8HG57NkOTnh4Jer0dzczP4fD6EQiH4PD70aj0UCgXautrA4/FocPH28MZTyU/B331oUkfmhjS+kBZ2S6LRaEzkYMjMi5+fH7y9vamVOHFgBXo2Q4sXL8aCBQvw5ZdfjotTJBthXUBpampCfn4+AgICIBaL8dBDo9cLMqaurg7t7e2IjzftqReLxbh+/TpCQkIwefLkPovvEokEJSUlCAkJQWho6LjeoLVarUlRn0xg9ycH3pfAIxthGAbl5eUQiUQjagPvjV6vR1tbG8RisVnqLsYS/Debb+JGzY0h1SGGy2jl53tDJvXt7e379n//P48XpUIJhUKBB4IeQFhgGN2Vj1U3VF+QYEK8gdgEkRYinz2lUgmGYRAUFARvb2/4+/ujpaUFS5cuRXJyMnbt2nXPBhOAhQGlubkZGo0GLi4uuHLlCh5++GGz/v76+nqq/An8tDuuqqrCjBkz4O/vT/2fjYvv9fX1qK6uxrRp04YsWDlWEA9rsitnGIbeNL29vVFdXT2owCMbIIOixADN3ArMpO4ikUjoqWKkdRetVouCggJ0ajrhKHBEdVu12ST4neycoDPooNVrR/27gJ61Njc305PaQBsfd0d3PDHrCTgwDjQId3V1UasCX19fs3VD9UVnZyfy8vIwadIkhIaGjslzmAuJREJ1+VQqFZ588knqYBkcHIxjx46xutllPGBdQNHr9dDpdFAqlcjJycHixcOTxx6MpqYm3L59G8nJyTAYDCgrK4NEIkF8fDw8PDz69DC5efMmJBIJ4uLiBjUbGm+Ihzop6iuVSvD5fISFhSEoKMisKRlzotVqUVhYCIZhRjxTNBxGU3chA6vExInsQBVqRc8wpbhHgn8kAcHZ3hkancZsEjJE0t/FxQU+Pj4DfnY8nT3x1Kyn4OnsafL13t1QDg4OJilEc9UGiLNpaGioSQMGG2lra0NRUZHJib+6uhobN26ESCSCTCaDTqfDkiVL8M477wzY9HM3w7p2n96+8gaDwaxHSBsbG+j1epPi++zZs6nsfG8Pk+vXr0Oj0SA5OZmVuw8ejwdPT084OTlBKpXCzc0NPj4+aGlpoR7VJDXGFg+W/m7QY4mx13doaKhJ3aWmpqbfuotCoUB+fj68vb0RFRVlcjN1cXBBXHAc4oLjTCT4K8QVUKgH79BysXeBUms++fnhyL74uPjgqeSn4OZ4Z4rR0dEREyZMwIQJE2gKkVhWGw8K+vr6jnjDQoIJqe+xGSJRFBUVRYNJe3s71q9fj6CgIJw7dw58Ph/Xrl3D0aNHzdqkY22w7oRCbIANBgNOnTqFBQsWmPUPJJFIUFZWBh6PBzc3N8yYMcPEHZIU3xUKBQoLC+Hi4jIkFWNL0p/Ao1KppDfN9vZ2uLq60qL+WKYxBltrfn4+1WViQydMf3UXV1dX1NbWIjAwcFjDdQzDoKmjieqM9SXB7+rgim5Nt9mCiUqlov4wgzUiCNwEeHLWk3BxGF43nPGgoEQigUKhgKenJw3EQ+3E7OjoQEFBAR0EZjPt7e0oKChAZGQkAgMDAfSk6VJSUuDt7Y1Dhw5ZbKO5Y8cO7NixA3V1dQCA6dOn45133sHSpUstsh6AxQEF6HFXfOCBB8w6MFZTU4OKigqEhYVhypQpfRbfyY4kKChoxFO648VQBR61Wi29EbS2tsLBwcGkqD8er5Gs1RJNDUOFCA/evn0bIpEIPB7PRGdsJJub9u52an3c0N4AVwdXs8rPEw0xLy+vQWtmAR4BeGLmE3CyH/1pValUmqTGnJ2d6cmlv2uqvb0dhYWFmDJlyqiHlMeajo4O5OfnY+rUqVTvTi6XIy0tDU5OTjhy5IhFT/3ff/89+Hw+wsPDwTAM9uzZg61bt6KgoKDfsYixhtUB5ezZs5g1a5ZZJEJIYb28vBw8Hg+LFi26o/gO9MgolJeXW0Q0cbg0NzejrKxs2GslO3ISYHg8nklRfyxSUESeJiIigtXeMMBP7dbh4eHw8vLqs+4yUukOpUaJKkkVKsWVqJZUUx/3kdLd3Q2RSDQkDbEJXhPw86Sfw8HO/CmZvtSkjV08bW1t6W7fGq4BkpIzDnwKhQKrV68Gj8fD0aNHzSqzYy68vb2xdetWPPvssxZ5ftYFFGMb4OzsbMTGxo5aAt7YHjgyMhKlpaVYsGDBHTIqlZWVaGpqQkxMDKs9TIjky61bt0Yt8GjsuCgWi6HVaumNYDQ5cmNIC/Nw5GksBRmunDFjxh3t1uaed9Eb9Khrq6N1l4H0wvpCoVBAJBLBz89v0E1XiE8IHk98HPa2Y98OTBpFyHvV3d0NV1dXyOVyTJ48mfXdXCSYGKfklEolHn/8cahUKhw/ftysAqDmQK/X47vvvsP69etRUFCAadOmWWQdrA4oFy5cQFRUFB0eGgkajQaFhYXQarVISEgA0GPDGRISAqFQSGVViouL0d3dbdap/LFgLAUeycQw6RhTKBTw8vKiqbHh5ooZhkFVVRUaGxtpFx2buXXrFqqrq02mn/uj9ykPGN28C9AjwV8h6tEZE3WKBnws+TsJhcJBlQEm+03GYwmPwZZvmTpgU1MTysrK4OTkBKVSCRcXF/peWUp1uz+6urqQm5tr0iygVqvxxBNPQCqV4tSpU6xqxS8uLsbs2bOhUqng6uqKvXv3YtmyZRZbD6sDyuXLlxEaGjriuQ+iWOrq6kq7ifR6PVpbW9HS0oLW1lbY2trCYDDA0dERCQkJ4zrQNVzGW+BRqVTS4CKTyeDm5mZS1B8I0pJNzMjY5AXTG2LY1tDQMKLAR+ouJLgQPbbR1F0GkuDv7OxEW1vbkAQppwqnYlX8KvBtLDNsR9ptSVFbq9WapMaMp9BHGojNBRmwJDU+oGdDum7dOjQ2NuLMmTOsy1xoNBrU19dDJpPhwIED+Oqrr5CTk8OdUIwhvvI//vgjgoKCRlTLaGtrQ2FhISZMmIDw8HCT4jvp5CJFYnt7e2i1WlpLICq3bOhAIlha4FGj0Zh4uzg6OtKTS29ZeWKKpdFoqB4aWyFmbhKJBAkJCaPOiw807zLSuotaq6Z1l7zKPDSLm+Hv7z9oQXh64HSkxKRY7DomQp9RUVF9arMZp1uNBT9JunU8u6fkcjlyc3NNpvW1Wi1+8YtfoKqqCufOnRtVpmS8WLRoESZPnowvv/zSIs/PyoBCXBvz8vLg6+s77D51UnyPiopCUFBQn8V3Iu5Gim5kl0l25Hq9Hr6+vnSq2pI7J7YJPJJTHrkR2NjYmLhSFhUVwc7ODrGxsaxutzYYDCgpKUFXV9eYTOoD5q271NbWoqa2BoJQAZqVPekxmbJv06+44Dgsm7HMYukk0tgwVGUJhmHQ3d1N3yuZTEYFGkmNaKxeC5HLDwoKwpQpUwD0bIqef/55FBcXIysri9XyRcY89NBDmDhxInbv3m2R52d1QBmuGRaZam9ubqZmN315mBDHwujo6D53HX1JypNjuZ+f37hOn7Nd4JHsMsViMUQiETQaDRwdHTFlyhT4+fmxNqDo9XoUFRVBo9GMW6pzpHUX45ScsVc5AIg6RdT6mEjwJ4YkYvE08ypMDAeJRILr16/32dgwVIwFGklqmpxczNmJSFwhAwICqPeKXq/Hpk2bcPXqVWRnZ9P5E7bxxhtvYOnSpZg4cSK6urqwd+9efPTRRzh58qTZJauGCqsDylDNsICfpDzUajUSEhLg6Oh4h+y8Xq9HaWkpZDIZ4uPjh5TeICkMElzkcrmJX8lYpnOsReAR+KkzhhTvxWIxuru7x+29Gg5El8vGxgZxcXEWCXpDrbsQ8UyxWIzExMQBa1Fdqi40y5ot5rII9IisFhcXjyqY9IYY0pH3iniXkAAz0utKqVQiNzcXAoGAWgwbDAa89NJLyMnJQVZWFqsHL5999lmcPXsWzc3N8PDwQExMDLZs2WKxYAKwPKDcvHkTDMMgKmpgT2sij+Hs7IyYmJg+J9/VajUKCwthY2OD2NjYEe9IexeqR+tX0hcMw6CiosIqBB6Bn05RU6ZMMfkAGtcSZDIZ9YonqTFLYAnZl8Eg6R5jHxxSd5HJZOjq6kJSUhJrpHP6QyQSoaSkBNHR0RAIBGPyHH15l5D3ytfXd0i2DsBPwcTPzw9Tp06lweR3v/sdTpw4gaysLNa3N7MRVgYUIr1SWVkJtVqNGTNm9PtYUnwPDAzE1KlTwTBMvx4mXl5emDZtmtlqECQ/LhaLIZVKzSJtYqzAy/YWZqDHAvXmzZuYPn36gLly41pCW1sbnJ2daXAZr9bRgXS52AR5r2pqaqBWq+Ho6AihUDiuqgbDhQSTmJiYcZ01UqvVNLi0tbVRW18/Pz94eXn1+TdWqVTIzc2Fj48PIiMjaTB54403cOjQIWRlZdFaCsfwYHVAqa2thUwmQ1xcXJ+PM/aaJ0J2BoMBPB6PXkhisRglJSVU0XSsPozEr4ToQZEuKIFAMOQbprlOUeOB8XBlbGzssNopyVQ1ea/4fD4NLv3dBEZLZ2cnCgoKEBAQwHo5HYPBgOvXr0OpVCIuLg5dXV0mdRfSLGLpNlsCaXCx9OAqsXUgAUan05mkxuzt7aFWq5Gbm0sN8kgweffdd/Hf//4X2dnZmDp1qsVeg7XD6oBSX18PiUSCxMREk++TdFhTUxP1c+6r+H7r1i3U1NSMew3CWGxQIpHQG6ZAIOhX/luhUKCgoAAeHh5j7gQ5WoxbbePj40c1XEny4ySNaDAY6A7TXN11RD/KGmTSSbMAGcQ1bgAZi3mX0UJ81WNiYljVVsswjEkglsvlcHNzg1KphKenJ2JiYmBjYwOGYfDBBx/gq6++wrlz5yymgXW3wMqAYuwr39DQgOTkZJPvFRYWQqlUIiEhAc7Ozn16mNy4cQNtbW2Ii4uzqEyCwWCAVCqlwYWYYZEdpo2NzZAFHtlAb1UBc+b1++quMy7qj+TERuo71qAfpdPpUFBQAACIj48ftFmANIuYa95luDQ1NeHmzZtDUhawNF1dXfS91Wq12LNnDxwdHeHi4oIDBw7g3LlziI2NtfAqrR9WBxSRSITq6mrMmTMHQE+LX35+PhwcHOiMQ+/iu0ajwfXr16HT6cZlmnw49J510Wq1cHV1RWdnJyIiIljdUQL81B3F4/EQFxc35u3TvW+Yw22AGEiXi21otVrk5+fT+Z3hnsxILYHU84zVpD08PMyeRmxsbER5eTnNELAZrVaL3NxcakXBMAzS09Px9ddf48cff4S9vT1WrFiBlJQULFmyhPVNMGyGnUMC/4dxt5ZUKqU5cJLj7MvDpKCgAG5uboiPj2dd2ojH48HLywteXl4IDw9HeXk5Ghsb4eDggMrKSiqnMd6e3kOBTOqTD+V4vLcuLi4IDQ1FaGgoVCoVTV9UVlbCxcWFBpe+OnuILldcXBzrd89qtdqk82wkN38HBweqKmE871JUVATAvHWXhoYGVFRUWE0wycvLg5OTE2bMmEHTXGSKPysrCzY2Nvj+++/x4YcforOzE7/85S8tvWyrhZUnFGIDTIx4wsPDcePGDUydOhXBwcHQ6/V3eJi0tbXh+vXrmDBhAh1QYivGKTlSgzCedenq6hqVKKO56erqQn5+PgQCAe2KsSSkAYLoQdnZ2Znsxmtra0esyzXeqFQq5OXlwd3d3cQczVwYW0Sbo+5y+/ZtVFZWIj4+ftQq4GMNOfXZ29sjNjaWBpOdO3fi7bffxtGjRzF37lyTn2EYZtyu7w8//BAZGRm4efMmnJycMGfOHHz00UdW3RTA6oDS2dmJy5cvw9bWtt/iO9BzkVdUVCAqKoq1U62EoQg8EqdFsViMjo4OOr8hEAjGvY2Y1HcmTZo0pl1yI4V09pDTi07X480eERGBwMBA1p1Sjenu7kZeXh58fHwQFRU1bq3T5Noabt3l9u3bqKqqQnx8POvTQjqdDvn5+fTeQYLJN998g1dffRXff/895s+fb9E1LlmyBD//+c8xc+ZM6HQ6vPnmmygpKUFZWRmrxVQHgrUBhQygSaVSzJ07Fy4uLn0W3ysqKtDS0mIW35SxZiQCj0SUUSwWo62tbdBUjzlpaWlBaWmpVQRqg8GA4uJiyGQy+Pr6QiqVQq1WW0wyZzCIFbJQKKRT2uNN7xkOBwcH2jDSu+5SX1+P6upqqwgmer0e+fn5VAmBz+eDYRjs27cPr7zyCg4dOoRFixZZepl3IJFIIBAIkJOTg3nz5ll6OSOClQFFLpfj6tWrsLOzQ3t7OxYuXEgDCAkmRNFWpVIhLi6O9QOAZA5iNAKPvVM99vb2NLj0VvwdLaQGwbZ20L4g14JWq0V8fDzs7e37lMzx9PSkqTFLTp0TmfQJEyawpquvL50xEoy7u7tRV1eHhIQE1qcQ9Xq9SaccOaEeOHAAmzZtwrfffmtRv5CBqKqqQnh4OJWusUZYGVAkEgkaGhowZcoUnDt3DnPmzIGTkxMtvpOdvqOjI6Kjo1m18+yLsRB47D3rQhR/BQLBqIYDjZ0rraEGQQzU+Hz+gOrGJI0okUjQ3t5uFlWDkUDqgpMmTWKttIdx3aWpqQlarRYeHh4IDAxklSZbb/R6PQoLC2EwGEzarjMzM/Hcc89h3759SElJsfAq+8ZgMCAlJQUdHR24ePGipZczYlgZUAwGAzQaDc2DdnR0wMfHBwKBAA4ODigpKYG/vz8iIiJYK59BIEXMsRyu7G84cLhdPQaDgYpnkhkfNkPSosPtPNNqtTS4tLa2mrTYjqW0iVQqRWFhIcLDw6lPOZshSghRUVFUw84S8y5DwWAwoLCwEDqdDgkJCTSYHD16FBs2bMCePXvw2GOPWXiV/bNx40YcP34cFy9eZP281ECwMqCIRCI4OjrCxsYGNjY2UCqVEIlEaGxshFKphLOzM0JCQli9W7KUwKPx7lIsFkOj0Zj4uvS3g9fpdCgqKoJOp6NpIzZjLl2u3qkeYrLm5+dnVpl0ckolzoVsh1g89JbLH07dZbwwGAwmVgQkY3H69Gk89dRT+J//+R888cQT476uofLCCy8gMzMT58+fZ+2pdaiwMqA8+eSTOHPmDJYvX45Vq1bh/vvvxxtvvAEfHx88++yz0Gq1EIlE6OzspHlxgUBg8fZaAlsEHvvyiCcnPePJczIH4eDggJiYGNZ6mBA6OzuRn59PDZHMtUM2GAwmwVir1dI6gq+v74hTq0Q4cTABTbZQXV2N27dv3xFMejNQ3cXHx2dcriPSjKFUKpGYmEj/RtnZ2Xj88cfxxRdfYN26daw4RfWGYRi8+OKLOHjwILKzs4dk08F2WBlQdDodsrOzceDAARw6dAg6nQ58Ph9vvvkm1q1bRwMHGXYTiUQm7bVCodBiRVeNRkO9Ntgm8Egk0knqwtPTE56enmhqaoK3t7dZlZjHCtLGPNa6XH0F45HMBhF5EksLJw4FYuTV2NiIxMTEYdkhk5Mx6UgcD50x4ripUCiQmJhIP2sXL17E6tWr8be//Q3PPvssK4MJAGzatAl79+5FZmamyeyJh4cH660K+oOVAYXQ1NSElJQUaLVa3HfffTh27Bg6OzuxdOlSpKWlYdGiRXT3r9Fo6IefSMkLhUJadB0PyKQ+GVJj8wyESqXCrVu3cPv2bTAMYzLrwtYeeKIcPXXqVAQFBY3rc5MagkQiQUdHB9zc3Ghw6e/GS+pn1qB1NZpg0he9vXCM3y9z1F2IAR/xiiHB5MqVK1i1ahU++OADbNq0ibXBBEC/a9u1axc2bNgwvosxE6wNKAzDICEhAXFxcfjnP/8JBwcHGAwGXLlyBenp6Th48CDEYjEWL16MtLQ0LF68mH4ISNFVJBKZzG4IhcIxKyJak8Aj8NPNOTw8HEKhEK2trRCJRJBKpXBycqLBZSy9vIcD2enPmDFjzMybhgqZDSJ1BGJVQCb1eTwe6urqUFtbaxVzGwzDoKqqCk1NTUhKSjL7hqL3+0XqLqQJYrinYoZhUFZWho6ODiQlJdHTT15eHlauXIl3330XL7/8Miuu23sN1gYUoGeHN2HChD4vDIPBgPz8fBw4cAAZGRloaGjAokWLkJqaimXLllEPEp1OR4/hra2tY3KzbG5uRllZGaZOnWoVHRpEi6mvzjOdTmfi60JkTYj0viU+pMSGYLi+K+NBX+3bDg4O6O7uRkJCglUEk8rKSrS0tAxqMWwORlt3YRgGN27cgFQqRVJSEk09FhUVYfny5Xj99dfx6quvcsHEQrA6oAwVUpgjwaWqqgoLFy5ESkoKVqxYAS8vL+opT3bixoOBQqFwRK6BxiZT0dHRrB8AZBiGdu/ExcUNqixAZE3IzZJ0QAkEAnh7e495vYWkYaxFl4s0Y7S2tsLW1hYGg4E2QQzUYWcpSCeiSCRCUlLSuDePDLfuQnx42traTIJJSUkJli1bhldeeQW///3vuWBiQe6KgGIM2cEcOHAABw8eRGlpKebNm4e0tDSsXLkSvr6+NLgY7yxtbW2HtRPvS+CRzRgMBty8eROtra1ISEgYdo7cYDCYSO/r9foRzboMFfJ3bGtrQ0JCAmvrOgRysyPvr7OzM7q6uuj1pVAoTLxdLN3uzjAMysvLqYEdG2aOBqq7ODs7o7KyEmKxGElJSbRofePGDSxbtgy//OUv8cc//pELJhbmrgsoxpDcMAkuBQUFmDNnDtLS0pCSkgJ/f38q6SKVSiESiehOnJxc+srxDkXgkU3o9XoqUxMfHz/q9fZlhGU86zJa5QLSvSOXy5GQkMD699dgMNCcfmJiYp8dOr1vlqQJghSpxxPj4Gd8c2YTvesuRNwxKiqKXmOVlZVYsmQJnn76aXz44Yes71C8F7irA4oxxBI4PT0dGRkZuHr1KpKTk5GamorU1FRaq+k9dc4wDD25eHt7Q61WU9kXa5jZINIkpI3Z3DI1fWlmjWYnbm0DliTdSlpXh/J61Wq1yc3S2dmZvl8jSb0OB+MaRH/Bj02QtBxpba+qqsILL7yAWbNmoby8HCtWrMD27du5YMIS7pmAYgzDMGhsbERGRgbS09Nx6dIlxMfHIy0tDampqVSm3dhhUSQSQafTgWEYeHl5WYWGmFKpRH5+Ptzc3MatjZm014rFYshksmG5LJIZHltb2wF1udgCOfmp1WokJCSMKPjpdDqaem1tbQWfz6fBZTSabH1BuqPa29tNahBshtTQSPeZTqdDeno6tm/fjtraWnR3d+Ohhx5CamoqnnrqqVG3O3OMjnsyoBjDMAxEIhEOHjyI9PR05OTkYMaMGUhNTUVaWhrCw8PB4/Fw+PBh2NnZwdfXF2q1mkqaEIdFts2cEHVjoVCIqVOnWkwenRRcyWxQf4KMxrpcI3UtHE90Op2JEKE5Nhf9abKRSf3RXGPGrbaJiYlWEUxIA0lSUhINFE1NTXjkkUewcOFCfPnll6iqqkJmZiaOHTuGw4cPs76WebdzzwcUYxiGQVtbGzIzM3HgwAGcO3cOERERCA8Px/Hjx7F7926sXLkSDMPQgqtIJDKpIfj5+Vl8Z03cK0NDQ82mbjxayGwQ8XUhsxsCgQB8Ph8FBQXjajQ1GrRaLQoKCgZVOB4NxnUqiUQCpVJpkkoczmmIYRgq+mktwaSurg51dXUm8i8tLS1YsmQJZs+eja+//trim7jz589j69atyMvLQ3NzMw4ePIi0tDSLrsnScAGlHxiGgVQqxdq1a5GVlYWgoCDY2toiNTUVq1atortoUkMQiUQmellCodAipk5kJmbatGkICAgY1+ceKqR9m9ws9Xo93NzcEBERQVu82YpGozHRPRuvmxqpU0kkEnR2dtJUIumA6g+iIN3V1TXkGo+lIXNHiYmJcHd3B9AziLts2TLExsbim2++sfimDQCOHz+OH374AYmJiXj00Ue5gAIuoPSLSqXC2rVrcf36dRw7dgwCgQBHjhxBRkYGTpw4AYFAQNNiiYmJNEVDPvgikcikQC0QCMa0wEyaDsgAINulPoCf5NyFQiF4PB4kEgkYhjFpR2ZT6ouk5VxdXTFjxgyLrY1o2EkkEkil0n5dPI275awlmBCbYWMzr7a2Nixfvhzh4eH43//9X1bWLnk8HhdQwAWUftHr9fjjH/+Il1566Y6bs0KhwPHjx5Geno6jR4/Cy8sLKSkpSE1NRXJyMt21Etl9YzFGcnIxZ9qBdMK0tLQgPj6e7urYTF+6XMZNEETt1zi4WHJXqlQqkZeXBy8vL0ybNo01p6jeLp5E2cDX1xe3b9+mKrxs75YDehQcKisrTeRq2tvbsXLlSkyYMAEHDhxg7evgAkoPXEAZJUqlEqdOnUJ6ejqOHDkCR0dHpKSkIC0tDXPmzKE3QZVKRU8uw+1+Ggi9Xk9TGpaUyh8OTU1NuHHjBqKjo/vV5TKuU4nFYiiVShPp/fHcpSoUCuTl5VH7ZrYEk94YKxs0NzeDYRgIhUL4+/uPyfCpOWlsbER5eTni4+OpgoNMJkNqaip8fHxw6NAhVp+wuIDSAxdQzIhGo8GZM2eQnp6OzMxM2NjYYMWKFVi1ahXmzZtHb4JqtZreKNvb2+Hm5kaVkYcTELRaLYqKiqDX661iZgMYuS6XXC6nRf2urq4RScmPhK6uLuTn5yMwMNCs3itjhcFgwPXr16FUKhEeHk4DjFqtNpGBYdO1QoQ/4+Li6DXR1dWFVatWwdnZGd9//z3r52W4gNIDF1DGCK1Wi5ycHOrpotVqsXz5cqSlpWHBggV0t0UmgonSr4uLCw0uA/XUq1QqkwFLNu8+gZ9UCxobG5GQkDCqtBzxhyenPWPpfXOe0GQyGQoKCjBx4kSEhoayPpiQuZjezoV9DZ8SYzo/Pz+L3qxbWlpQVlZmUvdTKBRYvXo1eDwejh07xnrZHYALKAQuoIwDer0eFy5coLL7crncxNOFfKBJPpzI7hNlZKFQaFJslcvlKCgoGLX97XgxlrpcvX1w+itQD5f29nYUFhYiLCwMISEhZlvvWKHX61FUVAStVmsSTPqCpF8lEgna29vpfBDxdhmvwEmcLGNjY6mwqlKpxJo1a6DRaHD8+HGrmSvhAkoPXEAZZ/R6vYmnS2trKxYvXozU1FQTTxdjGXmJRAIHBwfqRFlRUYGQkBCEhYWxftdsLE0y1rpcJCCTqXMHBwcaXIhPyVBoa2tDUVERIiIirMKOgAQTIlcznPoSmQ8iRX3ynhGvkrG6vsRiMYqLixETE0OdLFUqFZ544gl0dHTg1KlTrFeXlsvlqKqqAgDEx8fj008/xYIFC+Dt7Y2JEydaeHWWgQsoFsRgMCAvL4+KVzY0NODhhx9Gamoqli5dSj9QRBn51q1b6OjogJ2dHQICAiAUCod1oxxvLKnL1VtNmkiaEDXp/k515EbH5jkeY/R6vcnE/mg64Xp7lRC7Aj8/P3h7e5strSqRSHD9+nWTpgyNRoO1a9eiubkZp0+fZp3vTV9kZ2djwYIFd3x9/fr12L179/gviAVwAYUlkGIq8XSpqamhGkUrVqzAv/71L5SWluLjjz8Gn8+naR7jGyWbhgLZpMvVl+Cnsa8LuVG2tLSgtLR0wO4zNqHX61FQUACGYUYdTHpjMBggk8lMWriNi/oj7bJrbW1FUVERZsyYQc3dtFotNmzYgJqaGpw9e5b1vkIc/cMFFBZCdJeMDcN4PB5eeOEFbNy4kXq6kBslmXUBQGsu5hYWHA5s1uUipk7kRkk02WxtbdHc3GySz2czOp0OBQUF4PF4iI+PH9OmDIZhIJfL6WlPLpePqMuOpBKjoqLo6U+n0+H5559HSUkJsrKyrCKQc/QPF1BYjFarxS9/+UucOnUKjz/+OC5evIjCwkLcf//91NOFTJkzDGOyC9fr9fTkMp4T5wqFAvn5+Vahy0VulJWVlWhrawOPxzOZdWFTa60xJJjY2NggLi5u3Dv8iKK0RCJBR0eHiRFWf52JRBUhMjISgYGBAHpOWBs3bsS1a9eQnZ1tFSlGjoHhAgqLee2113Dq1CkcO3YMgYGBYBgGdXV11NPlxx9/xH333Uc9XYKCgmhwkclk9OSi0+moMvJYDrh1dnYiPz8fQUFBVjGzAQC1tbWoq6tDQkICbG1t6awLUTYgQZktgoo6nQ75+fng8/kWCSa96W2ERUQ//fz8aH2vo6MD+fn5JqoIBoMBL774Ii5cuICsrCwEBwdb9HVwmAcuoLAYIqXRV7cLwzBoaGhARkYGMjIyqEgdCS7Gni5EtVYkEkGtVtP6gTl9zqVSKYqKihAaGopJkyaZ5XeOJcZ+9caKtoTerbVkF06k9y0BUTkmdSlLB5Pe9G6EsLGxgYeHB9ra2hAREUGDhsFgwG9/+1ucOnUKWVlZVnG9cAwN1gSUlJQUFBYWQiwWw8vLC4sWLcJHH31Ej8cc/cMwDFpaWqiny/nz5xEdHU2DC/F0Mc6Fi0Qis8mZ9KXLxWaIn7pYLEZiYuKgAYLswon0PnFYFAgEcHNzG5eTmFarRX5+Puzt7a1ikNVgMFA5FT6fD4ZhsGvXLjz44IO4ceMGjhw5guzsbEyePNnSS+UwI6wJKH/7298we/ZsBAQEoLGxEb/73e8AAJcuXbLwyqwLhmHQ2tpq4ukydepUqoxsXNcgwcXYupeIVw61fkBuGjNmzLCKgqqxa2FCQsKwJ+uN54OMxRjHsoVbq9UiLy8PDg4OiI2NZVWTQ390dnYiLy8PYWFhmDhxIiQSCT744AMcP34cjY2NmDt3Lp566imkpqbC39/f0svlMBOsCSi9OXz4MNLS0qBWq1kpV20NkEL94cOHkZGRgVOnTmHSpEnU08VYgr27u5ueXIy1sgQCQb+ifHV1daitrR22LpelIN4gnZ2dZjGaMhZjJHMbxi3c5rjxE/8VIrFjDcGkq6sLeXl5CAkJQWhoKICea/H999/Hzp07sWvXLpSVleHQoUO4cuUKzp07h3nz5ll41RzmgJUBRSqVYuPGjWhsbMTFixctvZy7BplMZuLp4u/vT08uCQkJ9GbVly880RdzdHQ0qy7XeKHX61FcXDxmcu4Gg8FEel+v15tI748kRaXRaJCXlwdnZ2fWtV/3h1wuR25uLiZOnIiwsDAAPcFk69at2L59O86dO4eYmBj6eJFIBE9PT4soCX/++efYunUrWlpaEBsbi88++wyzZs0a93XcTbAqoGzZsgXbt29Hd3c37rvvPhw5csQqjKKsEblcTj1djh07Rj1d0tLSMGvWLHoDJMrIIpEIHR0dcHd3B8MwUKlUSEpKsgrhPjJNrtPpBtW5Mgd9NUIQi+ihDgWSYOLi4mJRM6/hoFAokJubiwkTJtDaCMMw+Pvf/45PPvkEp0+fRmJiooVX2cP+/fvx9NNP45///CeSk5Oxbds2fPfddygvL7eK1C1bGdOA8vrrr+Ojjz4a8DE3btxAZGQkgJ6uJqlUilu3buG9996Dh4cHjhw5YhXtp9ZMd3e3iaeLs7MzVq5ceYenS2dnJ1WzNRgMcHV1pScXtgYWMrMBwOzT5EOhL4toY2/4vnbmarUaeXl5FneGHA7d3d3Izc1FQEAAbRlnGAZffPEFPvjgA5w4cQLJycmWXiYlOTkZM2fOxPbt2wH0nDCDg4Px4osv4vXXXx+z5zUYDCZ/T4Zh7qr725gGFNKbPhBhYWF9ph8aGhoQHByMS5cuYfbs2WO1RI5eqFQqnD17Funp6Th8+DD4fD5WrFiBRx55BB9//DEiIiKwY8cOatlLZPdJ55NQKISLiwsrPiSkM8rOzo41bbakVkVmXXobrZFg4ubmhunTp1tFMFEqlcjNzYVQKDTpKNy5cyfefvttHDt2DPfff7+ll0nRaDRwdnbGgQMHTNSB169fj46ODmRmZo7J8+p0Otja2kIul6O4uBgzZ860qCTRWDCmr4YIy40Eg8EAoGe3xjF+ODo6Yvny5Vi+fDm0Wi2ys7Pxn//8Bxs2bIBQKISdnR2ysrIwf/58BAYGIjAwEDqdjrbV1tXVwdHRkZ5cxquttjdqtRr5+fmsqz84Oztj0qRJmDRpkonRWmVlJVxcXKBWq+Hh4YEZM2awIigPBgkmfn5+JsHk3//+N9566y0cPnyYVcEE6MmE6PV6qiVGEAqFuHnz5pg8p16vh62tLaRSKRYuXIh58+bB0dER8fHxd9UphRXh8erVq7h27Rrmzp0LLy8vVFdX4+2338bkyZO504kFsbOzQ1RUFK5du4bly5fjl7/8JQ4fPowXX3wRcrkcy5YtQ1paGhYuXIiAgAAEBARAr9fTttrc3FzY29uPSEJ+NKhUKuTl5cHd3Z3Vu3wHBwcEBwcjODiYdkbx+XxIpVJcvnyZFvXd3d1ZecMh77Ovry+1RmYYBvv27cOrr76KzMxMzJ8/39LLZAV8Ph8KhQKzZ89GfHw8tmzZQtulyd+2dzrMGmFFUb64uBgvv/wyioqKoFAoEBAQgCVLluCtt96yikG5u5nNmzdDp9Phiy++oCkjvV6Py5cvU0+XtrY2LFmyhHq6kHoKaasViUQmEvJCoXDMvDa6u7uRl5dnFVpiBKVSiby8PGqYZjAYTGZdbG1tTaT32fCaSDDx8vIyeZ8PHDiATZs24bvvvsPSpUstvMq+GY+UV11dHdzc3Eyaiv785z/j5MmTuHDhAoAe6+Nz586hoaEB69evR0BAgNUHFVYEFEtTV1eHP/3pTzh37hxaWloQGBiItWvX4ve//z1rBQLHC41GAzs7u35vYgaDAbm5udTTpampycTThbQUGwwGOrMhFovHZGZDLpcjPz8fQqEQERERrLjxDgZJGfUXAHu/bwDo++bt7W2Rmw+p85ATIFlzZmYmnnvuOezbtw8pKSnjvq7hkJycjFmzZuGzzz4D0PM+T5w4ES+88MKoi/IajQaRkZF444038Pzzz9Ovb9u2Dfv378d///tfZGZm4vLlyzh37hxCQkIglUrppsKa4QIKgBMnTmD//v144oknMGXKFJSUlOD555/HunXr8Mknn1h6eVaDwWBAUVERDS41NTVYuHAhUlNTsXz5crq7JjMbpPOJ+JMIhcIR3yRJyoi0rFpTMPH19UVkZOSga2YYxmTWhYh+klmX8SjwajQa5Obmws3NzaTOc/ToUWzYsAH//ve/sXr16jFfx2jZv38/1q9fjy+//BKzZs3Ctm3b8O233+LmzZt31FZGQktLC/z9/elwsbe3NzIyMvDhhx+iqakJHh4e+PWvf41ly5bh9u3bePnll3H48GGr1zXjAko/bN26FTt27EBNTY2ll2KVMAyD0tJSGlxu3LiB+fPnIy0tDStWrICPjw/Nufe+SQ53ILCjowMFBQWYNGkSncxmOyQ15+fnR+sPw4FhGHR1ddFZF5VKZRZdtoHobzbm1KlTWLt2Lb766iv8/Oc/N/vzjhXbt2+ng41xcXH4xz/+YbbWZlJo/9nPfobr168jJycHAoEAeXl5aG5uxrx58+Di4gI+n4+DBw/i7bffRmZmptVrm3EBpR/eeustnDhxArm5uZZeitXDMAwqKiqo7H5RURHmzp2LtLQ0rFy50sTTxXggkJhfCYVC+Pr69hlciM/GlClTrMbHW6FQIC8vz6ypud66bEORzhkORE/MycnJpGsuKysLP/vZz/DFF19g3bp1VnEyHEv0er3JdVpeXo7U1FR4enoiIyPDROxWJBLhypUrWL9+Pf7whz/gN7/5jSWWbFa4gNIHVVVVSExMxCeffGKSA+UYPQzDoLa2lgaXa9euYfbs2UhNTUVKSoqJp4tcLqdpMaVSece0uUQiQXFxsdWoHAM/BRN/f3/aZmtuekvnuLu70+AyXDFMwFTp2Fic8sKFC3jsscewbds2PPPMM/d8MDHmtddew8aNGxEaGor6+nosXrwYrq6uOHDgAEJCQtDc3IwPPvgA58+fx5NPPoktW7ZYeslm4a4OKMOd1Ad61HMffPBBzJ8/H1999dVYL/GehmEY3L59m3q6XLp0CUlJSVR2PyQkxEQZ2Xja3NXVFXK5HJGRkZgwYYKFX8nQkMvlyMvLQ2Bg4LgZkKnVajojJJVK4eLiMqwBVGLoZWtri7i4OBpMLl++jFWrVuEvf/kLNm7cyAUTI3744QcsXLgQt2/fpnN4jY2NWLJkCezs7JCRkYFJkybhypUrUKvVePDBBy28YvNxVweU4U7qNzU1Yf78+bjvvvuwe/duq27fszYYhkFzczMOHjyIjIwMnD9/HjExMTS4GN+Ar127BplMBkdHR6hUKnh5edFBSrZ25ZFgEhQUZLGmAa1Wa9KO7ODgQN+3vmZd9Ho98vPz77Aazs3NRUpKCt577z289NJL93ww6Z3mUiqViIuLw7/+9S88+OCD9PstLS1YuXIlurq6cPjwYURERFhw1WPDXR1QhkNjYyMWLFiAxMRE/Oc//2GFTMe9CvF0IcHl3LlziIyMRGpqKrq7u/E///M/uHz5MkJDQ2l6RyQSsda2l3SgBQcHIywsjBU34N7uimRGiMy6MAxjooFGPg+FhYVYvnw53nzzTfzud79jxWthC7du3UJISAgMBgOmTZuGZ555Bq+99prJYyQSCWbNmoXf/va3eOGFFyy00rGDCyjoCSbz589HSEgI9uzZYxJMOPMfy0LaLjMzM/HJJ5+gsrIS8fHxWLBgAdLS0ky6jYhtr1gspsrIZAfu5ORkkfWTYGIs5842DAYD2tvb6XtHOpTs7OyQlJRET30lJSVYtmwZfvOb3+DNN9/kgokRv//97/Hpp59i2rRpcHJyAsMw8PX1xXPPPUctponNtFqttohc/3jABRQAu3fvxi9+8Ys+v8e9PZaHYRj86U9/wj/+8Q+kp6fTusvJkycREBBAPV3i4+NpcCG1A5FIZDFP+M7OTuTn55sYTbEdvV6P3NxcqFQq2NjY4Pz587h69Srmzp2L7du3Y9OmTXjvvfe4YNKLy5cvw87ODrW1tbh27RquXbuGnJwcTJ48GWq1Gmq1GgEBAfjHP/5xV5uJcQGFg/XU19fj4YcfRnp6OmbMmEG/LpfLcezYMerp4uPjQz1dZs6cSU+avT3hjQvTrq6uY7JmYoEbGhpqNcNqZDBVo9EgISEBtra2KC4uxhdffIETJ05AKpVi8eLFWL16NVJSUuDr62vpJVuEocij7Nq1Cx988AF++OEHNDQ0oKSkBFqtFs8+++w4rdIycAGFJbz//vs4evQoCgsLYW9vj46ODksviVUQ6e/+6O7uxsmTJ5Geno6jR4/CxcWFerrMnj2b/mzvwrSTk5NJcDHHzlsmkyE/P9/qgomxoyUZjKypqcGSJUuwZs0a/OpXv0JmZiYyMjLQ1NSE+vr6e+6kYlyAP3HiBFQqFXg8HlJTUwH0XF92dnaoqKjA4sWLkZOTYzXzUeaACygs4Q9/+AM8PT3R0NCAnTt3cgFlFKhUKpw5c4Z6utja2mLlypVYtWoV5s6dS2+WOp0ObW1tEIlEaG1thb29/YBdT0OBTO2HhYUhJCTE3C9tTDAYDCgpKYFCoTCxR7516xaWLFmCFStW4LPPPjPZlXd1ddGagKUZr82Yscz8Cy+8gFOnToHH48HGxgYxMTHYv38/fWxbWxvCw8PxxRdfWJV6wGhhhXw9B/Dee+8B6KnncIwOR0dHrFixAitWrIBWq0VWVhYOHDiAZ555Bnq9HitWrEBqairmz58PoVAIoVBo0vVE5i7IyWWosvskmEyePNlqdqVEIkcul5sU4BsbG7F8+XIsXrz4jmACgDXBBOhJaa5ZswazZ8/Gzp07x+x5yDXw7rvvIjMzE8ePH8eMGTPw5ptv4i9/+Qva29tx9OhR2NnZwcfHBxMmTIBcLh+z9bAR7oTCMnbv3o1XXnmFO6GMATqdDhcuXMCBAwdw6NAhKBQKLF++HKmpqVi4cCHtBDMYDCYttUQZmcju95U/b29vR0FBAcLDwxEcHDzeL21EkGDS2dmJxMRE2nnU0tKCJUuWYM6cOdi5c6fVtNCPx2enqqoKr732Gn7961/jkUcewd69e7Fp0ya8+uqr2LFjB6Kjo3Hw4EE4Ojri6NGjWL58+ZithY1wJxSOewZbW1ssWLAACxYswD/+8Q9cunQJ6enpePXVV9He3k49XR555BHqNmrcUltcXAyGYe6QjyfBJCIiwmqm9hmGwY0bNyCTyZCUlESDiVgsxvLlyzFz5kx89dVXVhNMxouwsDCkpqYiKSkJOTk52LJlCz7//HM89dRTkMlk+OSTTxATE4OSkpJ7LpgAADcKPoa8/vrr4PF4A/4bK8tRjoHh8/l44IEHsG3bNtTW1uL06dMICQnBH/7wB0yaNAlPPfUUvv32WygUCupVMm/ePKplVVZWhpycHOTn5yM/Px/h4eFWFUxu3rwJqVRqcjJpbW3FypUrMX36dOzevfuu8zsfKiRp01tlg2EY2NjYYP369fD29kZ2djYeeOABPProowCA4OBg/OpXv0JaWhprFRvGGi6gjCG//e1vcePGjQH/sXXY7V7CxsYGycnJ2Lp1KyoqKnDhwgVERUXho48+wqRJk/D444/jv//9L2QyGTw9PREZGYkHHngAAQEBkEql4PP5qKysRHFxMUQiEfR6vaVfUr8wDIPy8nK0trYiMTGRqgm0t7cjNTUVYWFh2Lt375jI3w8HS27GeDweiouLER8fj5aWFpOvG1NVVYWysjI4OTlBJpPhyJEjiIyMxMcffzwm67IGuBoKy+BqKOyBYRiUlJRQT5ebN2/SCX0+n48tW7bg5MmTmD59Orq6uqh4pUqlMpHdZ8tOn2EYVFZWoqWlBUlJSVR5WCaTISUlBX5+fjh48CArpriHq8MHmPezc+nSJepl4uXl1edjrly5gpSUFLi4uMDR0REuLi73vN0FO650DtTX10MqlaK+vh56vR6FhYUAgClTpozZ8B3HwPB4PERHRyM6OhrvvvsuysvLkZ6ejr/+9a+4desWHnzwQVy+fJkagrm7u2PKlCnUm6SmpgalpaXw9vaGUCgcM+OrocAwDKqqqtDc3IyZM2fSYNLV1YXVq1fD09MT6enprAgmAGgNy1IkJSXBxsYGZ8+exWOPPdbnYxISEnDy5EkcPHgQXl5ed4WfyWjhAgpLeOedd7Bnzx76//j4eAA9Bkbz58+30Ko4CDweD5GRkYiLi0NzczP++te/QqVSYd++ffjtb3+LOXPmICUlBampqQgMDISbmxsmT54MhUIBsViM+vp6lJWVwdvbmxb1xzPPXlNTg6amJpOTiUKhwJo1a2Bvb49Dhw5ZTO9stIx2M9Z78t1gMIBhGAQEBKC2trbfn7O3t0d8fDz9rHJwKS8OjiHT0dGB8PBw7Nixg+5aGYZBfX099XS5fPkyZs6cSSVgJk6cSHPv3d3dVICxs7PT7K6K/VFTU4P6+nokJSXRG6xSqcSaNWug0Whw/PhxVs2VDJcNGzaYbMYIw9mMVVVVoaCgALNnz4abmxs8PDzw8ccfo6CgAPv27btDop5gPOzIwQWUe5rPP/+cemrHxsbis88+w6xZsyy9LFbT1tYGHx+fPr/HMAyampqo7P6FCxcQExODtLQ0pKammvigEGVkkUgEmUwGDw8PGlzMeVKoq6tDXV0dVbwlz/3EE09AJpPh5MmT8PDwMNvzWRsMw6C7uxspKSnIz8+Hn58fZDIZZs+ejcLCQjg5OSEvLw/Ozs79BhWOn+ACyj3K/v378fTTT+Of//wnkpOTsW3bNnz33XcoLy+HQCCw9PKsHoZhIBaLcejQIWRkZCArKwuRkZE0uERGRtLgolar6cmFKCMTCZiRWPYSbt26hZqaGiQmJsLd3R1Az1T52rVr0dzcjDNnzvRbcL7XaGtrg4eHB8rKypCbmwupVIqzZ8/i1q1biI6OxldffQU3NzcuqAwCF1DuUZKTkzFz5kxs374dQE/eODg4GC+++CJef/11C6/u7oJhGEilUiqseObMGTogl5aWhunTp9McPlFGFolEkEqlcHV1NbHsHSr19fWorq5GQkICPYFotVqsX78etbW1OHfuXL8nrXuRvhSE1Wo10tPTsW3bNgQFBWHPnj1wd3cfktrwvQoXUO5BNBoNnJ2dceDAAaSlpdGvr1+/Hh0dHcjMzLTc4u4BOjo68P3331NPl6CgIBpcjH3btVqtiey+k5MTPbkMpIzc0NBAjcg8PT0B9MjOPPfccygtLUVWVhZ3Cu0HUhMhQUOj0WD//v3417/+BYZhcPTo0Xs6RTgYXJfXPUhrayv0ej2EQqHJ14VCITe5Pw54enpi3bp1WLduHbq6uqiny9KlS+Hr60uVkWfOnInAwEAEBgZCp9OhtbUVIpEIdXV1cHR0pDUXY2XkxsZGVFRUICEhgQYTvV6PTZs24fr168jOzuaCyQCQ99HGxgYMw8De3h5PPvkkNBoNjh07BoPBYOEVshvuhHIP0tTUhKCgIFy6dAmzZ8+mX3/ttdeQk5ODq1evWnB19y7d3d04ceIE9XRxdXWl3WKzZ8+muXu9Xk89XSQSCezs7CAQCMDn83Hr1i3Ex8fD29ubPvall17CxYsXkZWVZTXyMGzB+MSiVquttrV6vOBOKPcgvr6+4PP5EIlEJl8XiUTw9/e30Ko4nJ2d8eijj+LRRx+FSqXC6dOnkZGRgZ///Oewt7enJ5f777/fRHZfKpWirq4OHR0dsLOzw/Xr19Hd3Y1FixbhtddeQ3Z2NrKzs7lgMgJ4PB7V8OKCyeBwlaV7EHt7eyQmJuLs2bP0awaDAWfPnjU5sXBYDkdHR6xcuRK7du1CS0sL9uzZAx6Phw0bNmDy5MnYtGkTTp06Bb1ejyNHjmDHjh2Ii4tDdHQ0ysvL8dxzz2HSpEnYt28f3n33XQQGBlr6JVkt3JzJ0OFSXvco+/fvx/r16/Hll19i1qxZ2LZtG7799lvcvHnzjtoKB3vQ6XQ4f/489XRRqVTo7u7Gb37zG7z66qtwdHSEwWDAW2+9hRMnTiApKQnnzp2DWq1GamoqvvjiCyoIycFhdhiOe5bPPvuMmThxImNvb8/MmjWLuXLliqWXxDEMMjMzGUdHR2blypVMcHAw4+7uzqxZs4ZZtWoVIxAImNLSUoZhGEav1zMXL15k3n//fQuv+Cdqa2uZZ555hpk0aRLj6OjIhIWFMe+88w6jVqstvTSOUcCdUDg4rJDs7GysWLECu3fvxmOPPQaDwYAff/wR33zzDXbu3Ilz585hzpw5ll5mv5w4cQL79+/HE088gSlTpqCkpATPP/881q1bh08++cTSy+MYIVxA4bAY58+fx9atW5GXl4fm5mYcPHjQZC6Go3/EYjGuXr2KlStX3vE9ax2827p1K3bs2IGamhpLL4VjhFjfVXeXwzAM7pUYr1AoEBsbi88//9zSS7E6BAJBn8EEgFUGE6DHl4W0O3NYJ1zbMItQKBTDktewdpYuXYqlS5daehkcLKCqqgqfffYZl+6ycqxzK3OX8uyzz+K5556DWq2mXyOTuffKqYXDuhmJdW9jYyOWLFmCNWvW4Pnnn7fQyjnMAVdDYREXLlzA8uXLcfv2bXh4eNAp3ZaWlrt+4JDH43E1lLuA4Vr3NjU1Yf78+bjvvvuwe/duq03XcfTApbxYRFBQEIKDg3Hs2DE88cQTkMvl+Prrr/HGG2/gr3/9KzZu3GjpJXJwDMhwrHsbGxuxYMECJCYmYteuXVwwuQvgAgpLMBgMCAsLg6enJ8rKyiCTybBu3TpUVlbik08+ocGE4RziOO4CGhsbMX/+fISEhOCTTz6BRCKh37vbT+N3M1xAYQlkd7Z582b8+c9/xu7duzFx4kTs27cPcXFxAPpvByVZSy7QcFgLp0+fRlVVFaqqqu7QGOOy8NYLd8ZkAaTw3tzcjNLSUty8eRPz58/HkSNHaDAB+m8HJcVOg8EAvV4/Hks2C3K5HIWFhSgsLAQA1NbWorCwEPX19ZZdGMeYs2HDBtoi3/sfh/XCnVAsDLEUraurw5o1a+Dr6wsAmDlzJry8vKDVamFnZ9fnz+p0Opw6dQoSiQSLFi1CUFDQeC591OTm5mLBggX0///v//0/AD1GX7t377bQqjg4OEYKd0KxMHw+H+fPn8e8efPg6emJXbt24ZlnnsGZM2eg1+v7DSYAUF1djR9++AE7d+5EVFQUFi5ciGvXrt3xuIF2fZbcEc6fP7/PHSoXTDg4rBMuoFgQnU6HX/3qV1i7di2WLVuGY8eOwd/fH6tXr8b58+ehUCgG/PmIiAi88847OH/+PKqrq+Ho6Ih//vOf0Gg0AHomj5ubm2k6jGAwGEzqLnv27IFWqx27F8pyPvzwQ8ycORNubm4QCARIS0tDeXm5pZfFwWF1cAHFgvB4PERHR+Nvf/sbvvjiC9jZ2YFhGEydOhX+/v44duxYvz9rMBhQWVmJf//738jKyoKfnx+2bt2K48ePUy0ksViM5ORkfPPNNyb1FxsbG1rAF4lE+POf/4zLly+P7YtlMTk5Odi8eTOuXLmC06dPQ6vV4pFHHhk0oHNwcJjCDTayDNIWPGfOHMTGxmLHjh19tgp/+umn+PjjjxEeHg6RSIT29nZERkbixx9/hFKppAFkzZo1cHFxoWmkr7/+GmKxGBs3boSbmxt9nLUKCo4FEokEAoEAOTk5mDdvnqWXw8FhNXB3EJZBAscf//hH6iHeO5ioVCps27YNv/71r5Geno68vDwcOnQIjY2NWLx4MRiGoSmuDRs24NixY6ioqMDbb7+NzZs3o6mpCTweDzY2Nnj//ffR3NwMGxsbk7SYTqcz+f+9hEwmAwBOqJCDY5hwXV4sZdGiRVi0aFGf31OpVOjo6EBgYCAEAgGAngllhUKBBx98EHw+nwaDiIgIREVFYdWqVbCzs8PevXuxatUqAEBpaSnefvttJCUlISAgADY2NhCLxRAIBLC1vTcvDYPBgFdeeQX3338/ZsyYYenlcHBYFffmXcPKcXFxwcsvv4zf//73qK2tRUBAAD799FOoVCqq3kvSV+np6bhw4QKioqKwb98+REVF0Vbl//znP5g2bRqSk5Mhl8uxd+9e/Pvf/0Z9fT3mzZuHl156CbNmzbLkSx13Nm/ejJKSEly8eNHSS+HgsDq4lJcVYmdnh/feew9///vfUVpaCj6fD4FAgEmTJiE0NBRAj+jexo0b8emnn2Ljxo3g8XiIiooC8FMK7X//93/xyCOPwNPTE99//z22bt2KRx55BDt37oTBYMCrr76K7OxsS73MceeFF17AkSNHkJWVdcf0NgcHxxAYc5NhjnEhJyeHOXDgAMMwDLN7924mPDyceeCBB5iLFy8y5eXlzIQJE5icnBz6+OrqaobH4zGnTp1iGIZhjh8/zkyePJmpqqqijzl27BhTV1c3vi/EAhgMBmbz5s1MYGAgU1FRYenlsBriX+/g4MD4+/sza9euZRobGy29LA6WwJ1Q7hLmzZuH1atXAwCmTp2KX/ziF8jIyMD999+PiIgI+Pj44MKFC/Txu3fvRnh4OD21REdHIzg4GKtXr8bXX38NvV6PpUuXIiQkxCKvZzzZvHkz/vOf/2Dv3r1wc3NDS0sLWlpaoFQqLb001rFgwQJ8++23KC8vR3p6Oqqrq/HYY49ZelkcLIFrG75HIIKTly9fhp+fH6ZOnYrly5fj448/pgV4pVKJ7du347vvvsNjjz2G11577Z5oJ+5PVHPXrl3YsGHD+C7Gyjh8+DDS0tKgVqsHVHXguDfgAso9RH5+PhISEnD79m2EhITg0KFDSElJwa5du5CcnIxp06aBYRjs3LkTb7zxBk6cOIHExERLL5uDpUilUmzcuBGNjY1cEwMHAK4of0+RkJAAvV6P4OBgFBYWYsGCBVTxd8uWLcjIyIBYLIZGo0FbWxvnS8HRJ1u2bIGLiwt8fHxQX1+PzMxMSy+JgyVwAeUegwxLxsTEwM3NDa6urvjNb36DiIgIbN68GQkJCfj222/xu9/9DkFBQffscKOl2LFjB2JiYuDu7g53d3fMnj0bx48fH9PnHK4P/KuvvoqCggKcOnUKfD4fTz/9NCc7zwGAS3lx9KK8vBwODg6YNGmSpZdyT/L999+Dz+cjPDwcDMNgz5492Lp1KwoKCjB9+vQxec7h+sAb09DQgODgYFy6dAmzZ88ek/VxWA9cQOHgYDne3t7YunUrnn32WUsv5Q7q6+sREhKCrKwszJ8/39LL4bAw3KQ8BwdL0ev1+O6776BQKFix+7969SquXbuGuXPnwsvLC9XV1Xj77bcxefJkVqyPw/JwAYWDg2UUFxdj9uzZUKlUcHV1xcGDBzFt2jRLLwvOzs7IyMjAH/7wBygUCgQEBGDJkiV466234ODgYOnlcbAALuXFwcEyNBoN6uvrIZPJcODAAXz11VfIyclhRVDh4BgILqBwcLCcRYsWYfLkyfjyyy8tvRQOjgHh2oY5OFiOwWCAWq229DI4OAaFq6FwcLCIN954A0uXLsXEiRPR1dWFvXv3Ijs7GydPnrT00jg4BoULKBwcLEIsFuPpp59Gc3MzPDw8EBMTg5MnT+Lhhx+29NI4OAaFq6FwcHBwcJgFrobCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEWuIDCwcHBwWEW/j8se5deHg/GSAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + " plot_matrix_and_subspace(B)" + ] + }, + { + "cell_type": "markdown", + "id": "cf941efb-add8-470b-862f-14dd911cc860", + "metadata": {}, + "source": [ + "In this scenario, the vectors, despite each having three components, can reach any point on the two-dimensional green plane depicted in the image. These vectors span the green plane, which resides within a two-dimensional subspace. This subspace's dimension, also known as its 'rank', is two—corresponding to the dimensionality of the plane. If the rank were three, any point in the 3D space could be reached by some combination of the columns of $𝐵$. The rank of a matrix can be determined by using the matrix_rank function provided by NumPy.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "894b2da5-9f86-4336-ab4f-b3a3f158b978", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "matrix_rank(B)" + ] + }, + { + "cell_type": "markdown", + "id": "62c90995-d697-4bf4-96aa-0c60d0b42698", + "metadata": {}, + "source": [ + "Here, you plot a different matrix where the matrix spans a different plane, but the rank remains two.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "id": "c92466da-2d0e-485d-abda-5c7ed31cc4a2", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAGRCAYAAABCCEUTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOy9eWwk6X0e/NRdfXfzHnI49z07Ozuzc3JlWwYSWXIgxHHsz/kjsOwATpDIyWfYQWAngADHcRzDQeDECGQDcSIhHxTZkryf5JUt+ZMtSyvtodUOyeHN4QxneJPNvu+u6/uj5i1WV1f1xSLZs1sPIGhJ1rxdXV39PvW7nofSNE2DBw8ePHjwsE/QR30CHjx48ODhgwGPUDx48ODBgyvwCMWDBw8ePLgCj1A8ePDgwYMr8AjFgwcPHjy4Ao9QPHjw4MGDK/AIxYMHDx48uAKPUDx48ODBgytgj/oEPHjw4KFd/H+P/z98f+X7Nb8L8kHkq3njZx/rQ1WpQtGUmuNERkRZKRs/h4UwspWs7etEhAj+73v/N2jae/ZuBd5V8uDBwwuF6e3pOjLxs/4aMmEpFqqm1pEJAAis4PjvrLg3es8jkzbgXSkPHjy8MNjOb+Or81+t+z3L1CZbRE5ERanUHcfRHHKVnPHfsiZD1VTb1xJZETeP3XThrD888AjFgwcPLwRKUglfnPoiJFWq+b3ACAZJAHqayinqCPABqFBBgQLP8KgqVcfXe3X41ZpoxkNzeITiwYOHroeqqvjSzJeQKqfq/iayIjToGrdhIYxMJWO7Bg3aIJqwEEZBKji+HkMxuDty14Uz/3DBIxQPHjx0Pb715Ft4knpS93uWYg2SaFYPCQkhyKrckHQIXhp8CWExvL+T/hDCIxQPHjx0Naa2p/DW6lu2fwsKQSia0rQeAgBluYwAH6hJjzlhbHSs4/P9MMMjFA8ePHQtNnOb+Nr812z/RoFCoaqnrXi2cT0kLIShaRqqctVIjznhbM9ZDAYHOz/pDzE8QvHgwUNXolgt4k+m/6SuCE8QFsKQVAkRIWIQixNUVQXLsI5rmfHa6Gsdna8Hj1A8ePDQhVBVFV+a/RLS5bT9ARpQUSqICJGm9ZAAHwBN0yhKxaaveyZ2Bmd6znRwxh4Aj1A8ePDQhfirx3+F5dSy499DYgg0RTtOuJshMmJLx4WFMF4ZeqWd0/RggUcoHjx46Co83HqId9beaXgMDRqyKjeth4T4EBLFRNPX9HN+UKBwdeBqW+fqoRYeoXjw4KFrsJHdwNcW7IvwBD7WB1mVGxbhAX0+haEYgGr8mhzNQVZl3Dl+BwzNtHvKHkzwCMWDBw9dgUK1gD+Z/hPIqtzwOIEVGg4lAvpgIkuxTVNdFChwDAcKFG4N32r7nD3UwiMUDx48HDnIJHyzAntMjDkX6gk0PYXFMixUOM+lAHrdpCgVPZkVl+DJ13vw4OHI8c3H38TT9NOGxwS5IDStcc0EAKJiFNlqFjTV+HmZdIgxFIN7x++1c7oeHOBFKB48eDhSPNx6iHfX3m14DE/zkFW5aQorLISRrqQR4kMNU2chPmSsdXXgqiez4hI8QvHgwcORYT27jr958jcNj6FBg2VYiJzYMIVl1vIqy2XH40RGRFEqGh1insyKe/AIxYMHD0eCfCWP/3fu/0W6km54XFAIoiyVGwo/mrW8wkLY1gsFgNH1RYy3zsTOYCg01PF78FALj1A8ePBw6FBUBV+a/VLT1t+oGEW2kjWUgu1AgarR8nJMdT0v1pujFy86cRceoXjw4OHQ8Y2lb2Azt9mwJhLiQ0ZHV6MUVlgIG1peQT7oKLES9UWRq+4pDQ8GBnGu91wHZ+/BCR6hePDg4VAxvjmO99bfg5/zO066C4yAklwCgIYpLKuWF+UwxRgWwnXtxvdH73dw9h4awSMUDx48HBrWMmv4+uLXwVCMoy8JTdGgKdpIXTmlsAJcoCbC8bG+mgiEwM/6ka/U1l9CfAjXBq91+jY8OMAjFA8ePBwK8pW8MQkfEkJGYbwGmp62ItGJUwqLp/WaiTnC4Rm+7jijWG/pDrt7/K4ns3IA8AjFgwcPBw5FVfCnM3+KXDUHClTDOoc56rBLYZE2YrO3CUdzdfUYChR4pt54S2AET2blgOARigcPHg4cf/noL7GSWQGge7vbdXeZi/CAcworKNRHLQEuUFePCQthW82vG8duQOTETt6GhybwCMWDBw8Hivc33scPN36o/6DBlkzIsKEZdimsiBCpi0QYiqkjHifjLZqiPZmVA4RHKB6OBK1oMnl48bGaWcVfLP6F8XNICNW1AFuHDQGdTKzEEeJDtiQRFII1/zbIBx3bka/0X0HUF+3krXhoAR6heDhUaJqGarWKXC6HQqGASqUCWZY9gvkAIlfJ4U+n/7Rms6+TTrEZNgRQ11JsF8EQlKSS8d8CI6Aslx3bkb1BxoOFpzbs4dCgqiokSYKiKMb/KpUKSqUSKpUKBgcHwbIsGIYBwzCgqCbOSB66Foqq4E+n/7QmFeXn/MYAIkFUjNZJr1hbiu0iGIKQEDKOZSj9npEV+zbjU9FTGA4Pd/qWPLQAj1A8HDg0TYOiKHj8+DF4nsfQ0BBomgZN09A0DblcDjs7O4jFYqhUKqAoCjRNg2VZj2BeUHx98etYza7W/I6hatt0iTKwFUE+uJfa0gA/77ctzgPYI48mxwFedHIY8AjFw4FC0zQjKslmsxBFESsrK1hfX0c4HEYsFoOi6E+eLMtC0zTjf5VKBdWqXsD1CObFwQ/Xf4gHmw9qficyYs1m72N9tmKPFChjBgXQ24idDLWCfNBYIyLaF+EJ+v39ON97vp234aEDeITi4cCgKAokSYKqqkY0sr29DVmWcfLkSRSLRSwvL6NQKICmaSwuLiIWiyEajYLjODAMU0cw5giGHMOyLGia9gimC7CSXsFfPvrLut8LrICyotdJWJqFqqlQtXop+rAQNojBTi6lBlr9v3HC2Ikx7/44BHiE4sF1aJoGWZaNYjtN00in09jZ2YEgCLh/X9dQoigKFEVhbW0Na2tr0DQNjx8/RrFYRCgUQjQaNQiGRCZmgimXy8Y6hGDIcR7BHD6y5Sz+dOZP62odHM3V1EREVnSUoictxU4RDIGP9SEv5eHn/A2PA/RI5uXBl1t9Gx72AY9QPLgKVVUhy7KRxqIoCo8fP8by8rJBEjzPG6ksQE91cRyHixcvAgAqlQpSqRTS6TQePXqEcrlcRzAk7eVEMCRy8QjmcCArMv5k+k9sN3c/5zciiIgYQaZsH02E+BBy1Rw4mnOMYAh4RndwlFW54XGAJ7NymPAIxYMr0DTN6OLSNA0URaFSqeDhw4col8u4e/cu1tbWbDd1iqJq2oYFQcDQ0BCGhnTjo3K5jFQqhVQqhYWFBVQqFaP+Eo1GEYlE6ghGVVWPYA4RX3/0dazn1ut+T1O0Ma0eEZzJBAA0aIa3ibUbzAye4ZGr5uBjfbaT8NZjPZmVw4NHKB72DXOKC9A38Hg8jqmpKQwMDODmzZtgWbaOOAiabeyiKOLYsWM4duwYAKBUKhkEs7m5iWq1ikgk0pRgKpUKyuWy0WHmEYw7+MHaDzC+OW77t5AQQqacgZ/1O6oLAzBSV40iGAIf6wOApnUTQJdZ8XG+psd5cAceoXjYF8yzJWRDnp+fx9raGq5evYrh4dq+f6cBxnYGG30+H3w+H4aHh6FpWg3BrK+vQ5Zlg2BisRhCoRAYhjFeh7QxkzmYjY0NDA0Nwe/3G5EMqe94aIxn6Wf45tI3Hf9elsqOir9msDTbEpmQWZOGxfrnEFkR9497nieHCY9QPHQEsinLsmx0cRWLRUxOTgIAxsbGEAgEav4NRVFQVbVuo3aKXFoBRVHw+/3w+/0YGRmBpmkoFosGwayurkJV1RqCCQaDYFnWeB+PHz9GLBYzSMQuReYRTD2y5WzdJLwZYSGMXCWnDzQ2SE0JjABFVRwn4a1rtkImDMXgUt8lT2blkOERioe2YZ4tAfQZkY2NDczOzmJ0dBQXLlwATder+piJw7w5u7lRUxSFQCCAQCCA48ePQ9M0FAoFg2CePXsGTdOMAn8sFoOmaWAYBhzHGRGMLMuQJMmRYOze34cJsiLji9NfbEgUsiq31NJL0l1OcikEPM2jIBWaHkeGHG+P3G58nAfX4RGKh7agqiqq1aoRlSiKgtnZWcTjcbzyyivo7+93/LeNIpGD0vKiKArBYBDBYBCjo6PQNA35fN4gmOXlZQDAwsIC+vr6EIvFEAgEaiIYJ4IhczAfRoJ5Y/ENbOQ2HP8e5IOgKbopmbAUi4pSqfE2sQNDMQjxISTKiabnFhEjiPqiGAmPND3Wg7vwCMVDSyApLtLFRdM0stksJicnIYoiXnvtNYhiY4+JRkX5wxKHpCgKoVAIoVAIJ06cgKqq+M53voNgMIhEIoHHjx+DYRijwB+LxeD3+x0JBrCf4v8gE8y7a+9iYmui4TE8wyNVSjVdKyyEkSwnGx+kAQEhgKLcWkosU8ngJy/8ZNNjPbgPj1A8NIU1xUVRFJ49e4ZHjx7hzJkzOHPmTEtpq067vA4SpLtrZGQEfr8fqqoim80ilUohHo9jaWkJLMvWEIzP56sjGEmSGsrEfFAI5mnqacMiPACjo6tZaioiRBpqbxFExShkTa6RZHF63Xwljz5/Hy70Xmi6rgf34RGKh4YwRyUURUGSJExNTSGXy+HWrVuIxWItr9WIOLpFvp6maUSjUUSjUQAwNMhSqRS2t7exuLgInufrCIYQhnkeh0QwFEXVEAzpInvRkCln8KWZLzUcJKRAgWO4ptFEiA8BQNNUFxGQDPLBhseZO8nuj95/Ia/vBwEeoXiwBUnrzM3NIRgMYnh4GMlkEg8fPkQ0GsVrr70GjuPaWrMbUl5OcHp9kv4ixKkoCjKZDFKpFDY2NrCwsABBEIxjYrEYBEGoWZcQTLVaNWowhGDMXWTdDEmW8MWpxkV44LkcfZMuLOJtwjGN7x8/qxfryf87wSAxqYggH8T1wesN1/VwcPAIxUMdyAaoqiqKxSI4jsPS0hKePn2KixcvYnR0tKMNsJsJpVUwDIOenh709PQAAGRZNghmdXUVs7Oz8Pl8NQTD83tWtlaCAYBEIoHBwUEIgtC1SsrffPxNbOY3Gx4TESNQNbVhqot4mwT4gKOrImCKODQVLN14mwoJIWOt2yO3wTLetnZU8K68BwPmzc6sELyysgKGYXDv3j2EQqF9v4YV3bZ5tgOWZdHb24ve3l4AOsGk02mjRXlmZgaBQKBGh8xMMJIkYXZ2FtFoFLIsd6UXzNsrb2N6Z7rhMUE+iGKl2HB40exZYvVGMYMCBZ7RW4R5mm9YZ4kKewZdHM3h9rDXKnyU8AjFAwD72ZKdnR3E43GEQiHcvXvXmDbvFEfRNnzYYFkWfX196OvrA6ATBiEYItUfDAYNggkGg8a/I9eXtGZ3g9nYcmoZ76y9U2fRawax3Q3yQVvDLALiWRLgAg1TZ+bZFR/vQ7VctT0uJIRqXu+VY6/Az/sbvyEPBwqPUDzUyKfQNA1VVTE/P4+NjQ0jbbNfMgE+GCmvdsFxHPr7+435nGq1ahAMkeoHgCdPnqCnp8eQ6gf2SPaoCCZdSuNLM19qeAxN0aApGlWl2jJJNOp4iwh7RlkMxSBfsa+d+FgfitW9wj8FypNZ6QJ4hPIhhp18SqFQwOTkJGiaxtjYGJaXl13b7LuxbfiwX5/neQwMDGBgYAAAUCgU8O6770KW5RqpfpIeI1L9QC3BHLSbpaRI+OL0F0GBakgUQS6IbDWLsBB2rImYvU1EVnQUiQzywZo1QnzINuIhBl1myZfL/ZfR4+9p5a15OEB4hPIhhd1syfr6Oubm5nDixAmcP3/emNFQ1cZ+E63CKr1iJpcPaoTSDKRT7uLFi2AYpiOpfju75P26WX5t4WvYym81bNc16hfanjGWFVZ3RpIes4L8nhT0KVCOrcd2Bl2eX3x3wCOUDyGs1ryyLGNmZgbJZBI3btww8v+Au+kop03tg5zyahXk2jST6pckqSWC2Y+b5Vsrb2Fqe6qha6I5egiLztGJyIjIS/oaVudGApI2qygV43dOGmB2isQnIidwPHLc8f14ODx4hPIhgtW3hKZpZDIZTE5Owu/347XXXquZoSDHkChmv+jWlNdRohmRNpLqX1tbg6IotlL9nbpZPkk+wbeefAsAwDGc7XS6yIo1ysCKan9/mOshgN4qXDejoum/r+nkcoh4nAy6vOike+ARyocE5nZggqdPn2JpaQlnz57F6dOnW3JT3A+6ucvrqF+/VemaTqT6m7lZqqoKlmVRRhl/Mv0nUFQFAivYRhOk3ZfUL5yk6a1kQlO0bbQT9dUPQoaEUF2rcJAL2kZBvb5eXOy72OTKeTgseITyAYedNW+1WsXU1BQKhQJu375tyIzYwc0aCjkfu9c46g39qLCf992JVH8wGARN0zUEs7i4CIql8N3cd5EpZXTiov0oKaUajxjzHAmBnVd7gKsfWgzxoboUlpO3iXUwUmAElJWy7cCkJ7PSXfAI5QMMu8J7IpHAw4cP0dPTg7GxsabyKWS40Q0Q4lBVFZubmxBFEZFIxNsQ4E7az06qP5fL1czBUBRVo0NGTNDe3H4TCVqXhqdBI1PKQNEUg0xomkbMF0OmnDF+JzL1HVs8w6OqVOs2f2sh3sf6bH3jiTcKgV19hSDABXB9yJNZ6SZ4hPIBhXW2hDyJrqys4PLlyxgZGdmXQnAnoCgKiqLgvffeQ7lcNmx4Q6EQNE1DNptFKBT6UBHMQUZmFEUhHA4jHA4bUv25XA6pVKpGqn82P4vZ0ix6enrAcRyCgp5eoinaeAAIMAHEc/EagglwAWjyXtceBQoszdY5L1pbijmaq2v7JaiRWbGrr5hwe+R2Uz0wD4cLj1A+YLCbLSmVSpicnISiKLh//74xnd0K3CSUbDaLQqGAkZERXL9+HRRFoVgsYmdnB5lMBuPj48YTNPmf3+//wBPMYb0/mqYRiUQQiUQA6A8dD1cfYurdKaiaivX1dTA0g6A/CFZgIYoiOI6Dn/OjrJaNBxNVVcGAQTKXhAbNIJioL2pEF+b3JCl7isIUKPAsbxudCExt3cauvkLA0RzujNxx47J4cBEeoXyAQJ7yt7a2cOLECdA0je3tbUxPT2N4eNiYdWgHbtRQVFU1oiOe53H16lWjphMMBsFxHJ4+fYrXXnvNqAFYvUh6enoQi8Wamnh1gqMkrKOsHaUraXx769tgOZ08QqEQOJVDPBtHPp9HIpEAx3AIBoJgeAaiKBrS+wHheZ1E0z/fIBvEbm63xtGSoiiExFBNxNLIEljkRCO11cw7/vrQdU9mpQvhEcoHBGS2JJ/P4+nTpxgdHcXs7Cy2t7fx0ksvYWhoqKN191tDMUdHly9fxrNnz4y/kSYBsqGbn6BPnTpleJEkk0msr69jfn4eoig6Kvm+qDgKQqsqVXxl/isoyaWaYVOGZ/Y8bjSAVmgkcgkUsgXs7u6CYRgEfAGU/CXwAg+GYRASQihIhZoIhjyEVFBBVanqEcxzaXu798tSrCGzYq2jWEGBwv1RT2alG+ERygsO82yJpmlgGAaKouDtt98Gy7IYGxuDz+freP39pLx2dnYwNTWFwcFBXL58GalUquFa1r9ZvUjslHyDwaBxjFkH60XBUUUobzx6A/Fi3PiZoigE+WDNRk4GFsn1JZIvrMwino6jUqkgwAeQElPgRR6iKNbIvoisiIJUgKZpEGgBu7ldgNIfHMjsC/n/oBBEupzWZetVuaGJ18W+i+j19x7cxfHQMV6sb5+HGqiqClmWa7q44vE4JEnCiRMncPbs2X1bz3ZCKCTFtbq6iqtXr2J4eLjhWmQDavY6ViXfarVqtMiadbBIeiwcDrsiannQOOwI5a21tzCfmDd+NjTCTJt4mK+ffqdpGn6/HxzNgQtxoDQKsiQjk88gnU6jWq2C53ViEUURgUgAFEVBYAXImgyKpowan/meZRkWGTUDFfo8TDOrX2+QsXvhEcoLCLvZElmWMT09jWQyCZqmcf78eVdeq90aSrMGADcn5Xmex+DgIAYHB43XJgSzsbEBWZaNIb+enp6GHWQfljmYpdQSvrPynZrfaZoGH+czah1+1m/IpVgR5ING4TwoBJGjcobZGBmYLJVKyGfziO/EIQgCQv4QaJ6GKIp1si+apsFH+7C6s4qoL4qqULWNYAhGw6M4ET3h6jXx4B48QnnBYJVPoSgK6XQak5OTCIVCuHnzJt577z3XXq+dGoo1xWWNDppFO/vd1K0yJeYp8pWVFQCoGfILBAJH3kFGHggOA8lSEl9b/JrtdWagf1Zmp0Q7yIp+39mpC5MIxu/3IyyEkSqmwCosdnO7KCVLkCQJgiAYEQwhGBkyeI1HtqKrFlsjGEIwNE3j3vF7bl4SDy7DI5QXCObZElLMfvLkCZ48eYLz58/j5MmTKJfLUFXVtY2qlZSXU4qr1bVaTXm1A7spcusMBukgI7WB/bpRdjOqShVfnv+yrdIvR3EoKkUEqaDhlGgHUmMJ8SFHCXpAn54vSAX0+HuQqWQMN0tFUVAqlVAul5FMJiHLMqK+KHx+H7LlLPwBf02TBoCaFFlMjGFEGEE+nzfUlI/azdJDLTxCeQFgN1tSqVQwNTWFUqmEO3fuGLMF5s35MAilnRmXRmrDBw3zkN/JkyehqqrhBa+qKqampiAIghG99PT0HEoH2WFEKJqm4c8f/Tl2i7u2f2cpfRsI8aGGPu8aNN3YSi429I0P8kGoqlq3FsMwxiQ/oDdZ8AqP3dwuKtUKypUyisViTQRjJpi7I3cB6C6Y1Wr1yN0sPdTDI5Quh5017+7uLqamptDX14cbN27UdDaRIryb0+1ONZRmKS67tQ4y5dUOaJo2yGNtbQ0vv/wyFEVBMpnEysoKZmdnEQgEalqUX7QOMoLvr30fC4kF27/xDI+iUsQoP9qQTPycHxW5ApqiHdWFgedqCKqityM3IB1A7yJTNAU9Qg8qUgV+vx80TaNcLiMej0NVVQiCoKcxo8O41n/N1mysW+ySPXiE0tUgXxYSlWiahoWFBayuruLKlSsYGRmp+zeEUFRVPTDb3lZTXE5rWdMaB5HyagdkEC8ajRrpGUmSjPrL48ePUSqVDCfFWCxm+JDsFwf9nh8lH+HN1Tcd/y6yIgRKQF7Ow8c5t5czFAOBERq6NwJ6d1hJLjUkHUCfJWEZFsXK3tAjiWBI6lGSJD2FW1XBpTi8/f23baX6gcN1s/TgDI9QuhAkxUW6uGiaRrFYxOTkJDRNw9jYmCHqZwX5srilEGwtyh+EjMtRE4odOI6rseo1OynOzc0ZRlekRTkUCnXcon1QG1yilMCfP/pzx+vK0AyqShWSJtn+nUBgBVAUZSuXYgYFCoqmOLo3mmGVVbFL/XEcB7/gB8dw+NQrnwIlUQ2l+p28YNx2s/TgDI9Qugx2Ka7NzU3Mzs5iZGQEFy9ebLhxHWTKq90Ul91aTufV7V9qs5MiMbpKJpNGB5lZJr6np6flDrKDItGKXMGX5+yL8AQBNgBZkyGrcsO1AmwAyXKy6WvGxFhLx0WECBRVqbOAtl4viqLAMzwu9F7QrYh57Fuqf79ulh4awyOULoLVmldRFMzNzWFnZwcvv/yy8bTcCCSd5KYPvKqqmJ+fbzvFZYduqaHsB2ajK7K55fN5Y3NbXl6uqdHEYjH4fL5Da0rQNA1fe/Q1JEqJhu+BoigUq8WGjQEBLuCovWVGWAjbSszbrVeUinVKw3bnEOJ1o607x+pFIDuV6m9EMKqqIp/PY2BgwCOYDuERShfAzpo3l8thcnISPM/jtddea0sU0U1CkSQJkiQhkUi0neKyOy+zbpTT344C+zW6CoVCCIVChkx8NptFKpXC9vY2FhcXwfO8kR6LxWKG1fJBvOfvrX4Pj5KPGh4TESI1KSe7TZNn+IZtxAQBLgBZletk6+3WqypV+Dl/HUlZCYXMuZzvOd+SzEqrUv1mgiFNAIRgCoUCHj58iI985CPGmo3skj3UwyOUI4bVmpeiKKysrGBxcRGnT5/G2bNn276B3TLF2tnZwcOHD0FRFO7du7fvIvSLnPJqBzRNIxqNIhqN4vTp01AUxXhyXl1dxezsLPx+/4G0Jj9KPML31r7X8JgAG6hJhdl9JjRFg6M5Ry8SAkISItv4gYehGDA0g4pcsSUeM6EE+aDxuneH7zZc1wl2Uv2E5Hd2dmqUrAnJkEYWjuNs7ZLJcKVHMM7wCOWIQG7W9fV1xONxQ9J9enoa2WwWr776qiFp0S5omt5XhGLu4jp79qzxdLdfmIvvdpvYi5LyahcMw6C3t7emg4wQzMbGBiqVCt57772aza2T650oJvCNJ99oeB15hgfLsChUaqMO66YY5J/PikjO9RVCEqqmNlQHBvS241w1V+c1bz0HkRFRlsvQNA3DoWGciLgjs2JH8oRgNjc3sbCwAJZlDTfRaDRqiKqaU2RkyLJcLnsEYwOPUI4A5sK7JElGcfHhw4cIh8MYGxvb15PrflJepVIJExMTUFUV9+/rEuFLS0sdn4v1vBr97YNKKFZwHIf+/n7jf3NzcxgdHUUqlcL8/Dyq1WpN91I4HG7aQVaWy3hj6Y2GEQVN0WBoBpJa29Vlve4RIYJsJQuebXwPEpIIC2FUZOf6iZlEyop9k4CmaWBoBho0o0mg0+ikFViVrBVFwdraGp4+fWpYJZgHXUma0tz0YiYYMgdDUmTk/60t8h90eIRyyLBa8zIMg2KxiPfffx8XLlzAiRMn9n0DdprysuviKhaLrnaMAe4KRL7oIKmeoaEhDA0NGR1kpMC/trYGVVXrupes8iRfe/S1psXzIBeErNnXOox0ExdEtprVp+arzoOOhCQ4mmsYnYSEven7kNBAskXTRSlJYT8qRnGx92LD9+MmGIZBIBCAKIq4desWZFk2lBRImtLn89V58VgJRpZlSJJUYzRmjmA+6ATjEcohwUk+5fHjx5AkCffu3UM4HHbltdqNUBoNKpLIwQ2JkGbzJkc52HiUsA55kg6ykZERo1hMWpRJB5m5RfkH8R9gPbeOkuQs+x4RI8iUM0Yqywxy3QVWQFnR002y5pzqMpOEXYGdwMf5UKgWjIn5RvekSIsoyAVDjeD28G3Q1P6sF9oF+V4CulWCNU1JCIZ48QQCAeNziEajLROMWYdsv/YS3QaPUA4BdrMl8XgcU1NTCIfDUFXVNTIh67dKKNYUl7WLy01tsGYRyocl5WVGs/dsbo+1di/t7Ozg21PfxtuZt9ET7IHKqfD5fHX1lyAfRLachciItsOJJN1EgYKsyghwAcfOLpEVUZR0LS+aoh2P4xgOiqoYqsWN1gzxISwry+ijdJ8bH+vD9YHrDa/LQcBMKFZwHFfjxWOugy0vL6NQKCAYDNYQDMdxTQnGOsX/ohOMRygHDOtsiaZpmJ+fx/r6Oq5cuQJRFDE1NeXqa7aa8mplUNHNQUkv5WWPdt67uXspOBDEtyrfwjHxGLKFLErZEuLxODiOg8/ngyiKiAQjepEbGjiWc6xhBLiAYWzldD4szUKDZsiqBPmgrf4XRVHg6dp2Y6doI8AFjDXI694YugGeOXxrZ5KGbgXmOhiwZ/aWTqfx+PFjFItFhEKhGoIhk/nAHsEQoUvAXibmRSMYj1AOCHazJYVCAZOTk6AoCmNjY/D7/Uin067NjBA0S3m1o8V1ELIoZBBQkiREIhHXp/tfJHT6nstyGV+e/zJkVUYsFDO84M0mV9lMFundNMACYX8YZbEMXuBrNilN0xBgAshLeTAMAx/rs62JUKDqtLycCvFW1WKBFWzXFFgBFaVSM5vE0ixuHbvV0TXZL/ajf2c1e6tUKrZuooRcSCefHcGUSiUsLS3h4sWL4HkeLMsilUrVdJ51KzxCOQBYZ0sAYGNjA7OzsxgdHcWFCxeMG2m/Lb52aBShmFNcjTTBCNwkFLLW8vIynj17BoZhjIKzoigoFouIRCJHEq0cJZm1+341TcNXF7+KZClZVxQ3m1yF+BDSpTRKpRIYmUF8Nw5FUQyTK5/Ph95AL54pz9ALvVbAMZytBa+5bgLsTbFbYdcWLNACKqglH3OKzXztr/Zfta3zHAYapbzahSAIRqMFUKsFRzr5wuGwQTBEbJQoZOzs7ODSpUvGYPE//If/EP/0n/5T/MIv/IIr53dQ8AjFRZDZErM1r6IomJmZQSKRwCuvvGKEyAQHRSh2a3aixeWm2KQk6e2qm5ubuH37NkRRrHFVXFhYwJMnT4yJ8p6eHmOi3MMevrPyHTxOPQagF77t0k5kY2cYBpFwBBQo+KI+yLJsmFxldjPY2d6BBg3ZbBbRYBQ5tEYSVukU4HkKzNIZxtFcnZ0wBaomEiKEQlP0gbYKN4ObhGKFVQvOTDDErpoQDJHpMdslkBpNt8MjFJdgLbxTFIVsNovJyUn4fD6MjY3ZyqccBqF0KjdP1gL2/wSfTqcxMTEBALh16xZ8Ph8kSTIkSzY2NnDu3DkwDINkMon19XXMzc0ZniQ9PT1GHvqDhHabHeZ35/HW2lsA9MFCuxZgazRhrlOwLKunXiIx0BSNbCGLzc1NVKtVJLYTKCgFo/7i8/kQ8UXqIpEgF6wjCfNAohl2XWDW8yPX4EzsDPr8fS1fC7ehKIorA7zNQFFUnV21uVV8dXUVmqZhfHwcKysrCAaDKJVKTbMJ+8FnP/tZfPazn8XTp08BAFevXsVnPvMZfOITn2hrnQ/Wt/OIYJ0tAYCnT59iaWkJZ86cwZkzZxw3DZKecvPpyFxDaTfF5YROCUXTNENK5ty5c1hYWHD80hIxPzJsRjppksmkkYcmT3E9PT0tDfx9kBAvxPHG0hvGz0E+WLdZi6yoCz5iry5hVRw210M4jgMAHBs8BgAoVUoolUooFovIpXKI03FwImcQDMuygOVWZmgGGqUZfvMEdl1gdl70hFCOMjoBDjZCaQRrq3gqlcL09DT6+/vxhS98Af/n//wflMtl/OZv/iampqbw4z/+47h586ar5Hf8+HH8p//0n3D+/HlomobPf/7z+Pt//+9jfHwcV69ebXkdj1D2AbvZEkmSMDU1hVwuh1u3bhmboxPMhlhu3cyEpPYrNw/sqdJ2QiiyLGNmZgbJZNKQkllcXGy5y8vaSWN+ipuamjLqLyRF1qpkfDeh1etakkr48sKXDa8RClRdrYOlWP2eNKWjQlz9gKI1QgD20lWCIOgT4RQNnuaRLug1mFwuh0QiAT/vR4JPQPTpFr0swzoW8q1dYAEuYHucpmnoFXpxKnqqpWtxUFBV1SDYbjiP48eP4/d+7/fwO7/zOzh//jx+5Ed+BN///vfxH//jf8RP/uRP4gtf+IJrr/nJT36y5uff/u3fxmc/+1m88847HqEcBuxmS5LJJB4+fIhoNIrXXnutpZvTTChugaIobG9vI5fL7VtunqzX7vnl83mMj49DEASMjY0ZtRAzOZk301ZIy5omIJLxRE2WiP319PS0VX85ahJq9vqapuGrj76KVCll/M5ukl3kxLoNu6rWml1ZIwRN02zJiZAB8XYnHWScyiGejSOdTqNarSLmjyHBJYzjzA9F5i4wIiJJ5lKsuBK60vAaHAYURWlL1fsgz8P88EfTNNLpNP7ZP/tnOHfunCE2epCv/6UvfQmFQsGQX2oVHqF0ALvZkkePHuHZs2e4dOkSjh8/3vIm5TahlEol7O7uGq3JbuRd25Vy2djYwMzMDE6ePIlz587VbDLmyXsr2nkNO8n4TCZTU38hir7d7Anfynv+9rNv40nqSc3vrHpcdoXzIB+sIRg/57eNEHysr2Y9MlVvhciKkDXZEC31M37Es3HDbEyWZfA8D5/Ph75wH8paWZ8OpxgwFIOSYj/JH+JCOCWcanwRDgFHlfKywkoo5XIZiqIYRXkiNuo2pqamcP/+fZTLZQSDQbz++uu4cqU9ou++b1gXwzxbomm6NW+5XMbk5CRkWca9e/cMP+xW4SahkBQXz/Po7+93rYjXasqLGHFtbm7i+vXrjoZgBzHYaDa0AmrrL8QTvlvrL43e+2x8Fu+sv1PzOytRhPmwrfyJORrgaR6SKtVFCHUpMj6EbNlew8vH+YzXIYOQgUDAuM9kWTZmYDY3N1GUixAFEb2hXhS5Iniet32vr/S9AjZz9FtRtxJKoaDXoQ66y+vixYuYmJhAJpPBl7/8ZXzqU5/Cd77znbZI5eg/xRcEqqqiVCrh4cOHuH79Omiaxvb2NqanpzE0NNRxjQKAMY+xn3Mzd3G5HQ63Qiik+K9pGu7fvw+/39/WWm5Lr1jrL+Vy2dDDstZfVFU9sjmURq+7U9jB1x9/vf7fYO/f+FhfXccVUCt1QlM0WIa17QgLckFsaBsAamVVrGBoxliPZ3hU1fr0FcuyCAaDGIwNoiAVIEkSWJlFPKdHMQCM4r4oiuB5HiIr4krPFazl1hyvw2HhsLq82j2PfD4PmqYPfKiR53mcO3cOAPDqq6/ivffew3/9r/8Vf/RHf9TyGh6hNIF5tkSWZezs7ECWZSwtLWFzcxMvvfSSMbzUKfbTOmzXxZXNZl3dIJvVUOLxOB4+fIihoSFcunSp4ZeyEXEc5KYuiiKGh4eN+gsRXEwmk5AkCQ8fPkRvb6+RIjvMXLrdU3tJKuHL81+GpNSmtvyc39Dj4mgOiqbY1iXMUidOEinA3jyJXUHfDNJ6TCTwGwlRkvfTG+xFtpLFgE+PVCuVCsrlMorFIpLJJGhanzvJpty9XztFt0YoxWLxSBpOVFVFpdLc1tkMj1AawCqfQnLwP/jBD8CyrCGfsl90SigkxWXdyEm3mVtwqqFomoalpSU8ffoUV65cwcjISNO1GvmqH9amYhVc/N73voeTJ09CkiTDC8Pn8x1K/cXuPauaitcXX6+x6CVgKP0zpkCBZ3lbsUeR2SvO27XpEvg5P1L5FChQtgV9AnPrcYALNPRcIQKUdvUa0kEWiUT0ppaqhFtDt5BKplAqlfDWW28Z1/sg3CyboVsJJZ/PHzih/MZv/AY+8YlP4MSJE8jlcvjCF76Av/3bv8U3v/nNttbxCMUB5tkS0jq7ubkJAOjt7cWlS5dcbfN1S24ecNdTnqxn3fiq1SomJydRKpXaqh01SnkdFUiBPxqN4syZM7b1l1AoZBCMWX/Mrdc349vPvo2n6ad1x5k1sexafwl4hkdZKSPAN978SRQTYO3beQmINlcjt0XjtVkeKlTbeo0ZFEXh1ugtXDt/DRsbG9ja2sLJkyeRSqWwsrKC2dlZY6j1sJoquolQzB2ihULhQIcaAf3h9Od//uexubmJSCSCl19+Gd/85jfxd//u321rHY9QLLCbLSHzFKlUChRF4eTJk67eeJ3KzTt1cbnlKU9gJahUKoWJiQnEYjHcuHGjrS86IRQ7AumGtAdgX39JpVJIJpPY2NiAoig1fiT7eXq0vueZ+AzeXX/X9liBEVCRKw2jDo7RpU4EVj/W6ZqSSCLEhbCj7hhaXnaQFKlO8NH2tWkOxWoRPMvb1musIIOMpMHF6j9CZo7MpE7IhWhfuYluqqGYU66EUA7yoeuP//iPXVnHIxQT7GZLMpkMJicnEQgEMDY2hjfffPPQtLescEpxdbpeqzC3+j579gyPHj3C+fPncfLkybZv8sMqyrsJqw6T2fDqyZMnNfMvndRfyDXcLmzjLx7/he0xHKOLQDq1/hL4WN3UigaNiuqc/+ZZHgzNIJlLNjy3IB+EoiooyvbFejMCXAAq1KbEAwBnYmfQH9AJ2y4y4DgOAwMDRqegWftqbm4OkiTVdO2FQqF9P+R1U4Ri7fI66AjFLXiE8hyqqqJardbcVMvLy3j8+DHOnTuHU6dOGY5rhHDcQjMCaFeLy+2UF4nSSEthKwoAjc6tG1Ne7RxrNbwiTn6d1F/ItShKRdsiPIGf9aMklRqmkhiaMeoXjVJdHMOhIlegQoWqqQ3fP03RqGpVwwOl0XEAWiIToNYvnkQojWAldaKakEwm62ySO40au5VQSA3lRcCHnlBIiosoBNM0jWq1iocPH6JYLOL27duIRqPG8URe2k00IpROtLjcTnmpqoqFhQUEg0GMjY3tu1ja7SmvdmCefzlz5gxkWbZN1TSqv2jQ8PrC67bDhMBey67Vj8SKIKfPKTSrc/hZPyRVQkVq3MHj43xQVMWQe2mEEBdq+roEJ8IncDp62vhZVRuTmhV2NslENcFsk2yuvxAF30bYjx+Km7CLUF4EpWHgQ04odimuRCJhtJDa1Qf2OzNih2Zy862047ayXidYX19HPp/HwMAAbty44YoNcDemvNx6bZZlG9ZfZFmu6WRSVRXvJd7DrrjruGaQC+oS8w2e/snn0ixCYCgGGjSjxtFI7VigBaQr6YbrAXqarayUm6bEACDAB/DK0Cs1v9tvZGCnmpDNZpFKpbC9vY3FxUXwPF8TNVplecgsUjdGKMVi0SOUboedfMri4iJWVlZw+fJljIyM2H7RDipCMa+5H7l5wJ2Ul6IomJubw/b2tj6sNjjoSlrKTBzm9Y5aT+ugYFd/IQSzvLyM5eIyxjPjhhuf9QGGoijQoJGqpBxeQUeEiyAn5Zpu6mE+3HQt4LmUS7V5xMExHARWsG1xtkJkRPA0jyt9tZPXbqeaaJo2XBFPnz4NRVEMWZ7V1VXMzs7C7/fX2CKQ+68bCcVLeXUx7Kx5S6USJicnoaoq7t+/3/Bp4KAjFDfk5veb8ioWi5iYmDD0wKanp117gm9EHC9iyqsdmOsvo6Oj2Mhu4C+++xegKMpQ82VZ1hDBFEURMV+s6cbOgEFFrTStcwT4AApybcrMLkIhEVGzz4OmaHA011JKjKVYaJSGm0M36/zlDzoyYBjGEAwF9mR5SFOFOaWUyWQQi8WONPVlTb0VCoWOa5aHjQ8VoViteWmaxubmJmZmZjA8PIyLFy82vZEO0hCr0xSXm+e4s7ODhw8fYnh42Ji1cTMd1a0pr8NGUSrizxb/DBRDged5DAwM1PjBJ5NJaLKGpJgEIzDw+XwQBKFu86dAIeKLIFlq3K0lsiJYikVBqa/BmNcUWAGSKtV5m9iBdIA1quuQcxQ5EbIq45XBV+r+fti1C2tbeKVSwfb2NpaWlrC4uIhKpYJIJGKkxw5b980u5TU6Onpor78ffCgIxSyfQsJrVVUxPT2N7e1tXLt2DYODgy2tdRBdXhRFYWdnB0+ePDkyuXlVVfHo0SOsrKzgpZdewrFjx/a1XqNz67Yur8OGqqn4i6W/qKt3mP3gaYpGgA5gM7OJUqmEnZ0dqKpqaGH5fD7wPI+QEKqRibcDS+lfc7tIwvxZEJ93gRMcGwQIyCxMK/7vZAjz7shdCGy9pcBR+5AIgoDe3l48efIE9+/fr/HdIR1kkUjEqMEEg8EDu19Jk5CZwAqFgiuKHIeBDzyh2BXe8/k8JiYmwPM8xsbG2hJdcztCIXLzAI5Mbr5SqWBiYgKSJNmm/NzsGjsqLa9uwl8//Wssp5cBOL9n8vRPUmSArk5AIph0Oo0AG0DOlwM42NZfgL3oAEBDaRVAL64XpaKjnhdBiA8hV8lBZEVb6RczCPEwNIPbx27bHtOuDfJBgERJdh1kpO5FOsjMzqKxWAx+v9+18yd7i9fl1YWws+ZdXV3FwsICTp06hbNnz7YdyroZoZAUlyAIhuOgG2iH9JLJJCYnJ9HT04NXX33VflPyUl6u4eHOQywkFur8TMyICBHIqlyXSuJ5HjzPIxwOw8f6kClkgCqQyCds6y80TRvRQYCzv7fINSeyKs3kVcyKxBzD1dkLmxHkgsY8zOXeywgLYdvjumH+wxoVEFjrXqqqIpfLIZVKIR6PY2lpCRzH1RDMfoRFyd7iEUoXQdM0VCoVVCoVcBxniCXOzMwgnU7j5s2bHRvUuNHlZe3iyuVyRpOAG2glRaVpmjG4efHiRYyOjh6acKOmadjd3cXu7i5isZjRZXNUhHJYT8eb+U188/E3G26eQU634iUzJXbgaR6yJiMUCKEqVHEsdKyu/iLLMmK+GFJ8Cj3BHuS1vOPsT5Db86avKM7pM5ZmoUFXJOYYrmF0IjKi3kr8/DO9N3LP8dhuIJRWz4GmaUQiEUQiEZw6dcroILMOtpoJpp10njmTQuC1DR8hSIrr2bNn2N3dxauvvop0Oo3JyUmEQiG89tpr+xrM22+Xl10X19LSEqrV5p0yraJZior43mezWdy5cweRSKThem5P3m9tbSGZTKK3txfz8/O6bwbLIhAIIJfLHWiO+qhQqBbwlfmvwMf66qx7yXsloo4CLTimp8zeJmEhbNRPzPUXAOApHqlcCsVSEfGdOPJSvq7+Augbf7qURhBBhAQ9lWUHiqJqBiv9rN8xkmEpnXhkVX9IOh09jYGAvdkacPBdXq2g08YAaweZLMuGsOjy8jKmp6cRDAaNFuVIJNJQOYEU5Mk9QVJuHqEcAcyzJSzLQlEUPHnyBE+ePOlYe8oKMknfCRrJzbstleK0Xjabxfj4eFtT727VUCRJQj6vPynfuXPH6FoqlUqYnZ1FuVzGgwcPwDBMR97w3QpFVfD6wuvIVrLwc7XFVXJdiU1uVanCL/hRVuxTScTbhGh72YFneGjQEAgGEIvEICkSwtVwTf2FoiiE/CFo0Awpl0b3oFkckqEYx84uiqIgsLUT/XdH7toeS9DupPxBwCnl1S5YlkVfXx/6+voA6HUvMne0sLCASqViaJDZKSfYCVR6hHLIsJst0TQN2WwWlUqlpafwVtFJhNJsUPEgxByt62maZnitnzlzBmfOnGn5S+xGOooQGUVROH36NEKhEKrVqlEEDQaD4DgOp0+fRjabrfGGDwQCBrlEo9EDaTE9yHTbt55+CyvZFQS5oK27IgBDf4uneUeiMNc3nCIEmqLB0nvujCIroqpUa+ovxItEqSjYzexClmXsbuwiISRq6i92rwvopOYUnVhViQcCAzgTPdPw+rxIKa92wfM8BgcHjS5ScwcZUU4gGmSxWAyyLHuEcpSwzpZQFIV4PI6FhQVjMM9NH4V2ayitys27HaGYN0hFUTA7O4t4PN5R/Wi/hLKxsYGZmRmcOXMGqVTKlhDMk8pkypl4k5AnPJIeIy2cPT09XZ8em9yexPub7+s/OJxmiA0ZxWuzb7sZAT5gpMoaRQhmd0aWZm3JiaIo9IX6kBWyCKk6sQ/EBhDPxpFKpVCtViEIAnw+H3rDvTXnQ4FCSbZ3a7ST1TeLQDqhW1Jeh3EOJOVInEOJe2UqlcKzZ8+MaG1tTbdEHhoa8tSGDwPm2RLSdqhpGubn57G2toYTJ05ga2vLdVOediKUo5KbJ4RCbtiJiQkwDIOxsbGOOlA6raGoqor5+Xlsbm7ilVdeQX9/Px48eNBW27BZxpyozBLr3qdPn4KmaYNcui09tp5bxzef6I53ftZedj7ABpApZyBCBEMztsfwDF/jbeIUIVgjiQAXsD3OuvELjACN04wHDUVRUCqVoFQUrG6sQlIko/7SH+m3nWcJ8sE6heOwEK6TWbFDt6S8Dns6nqIoBAIBBAIBjI6OQtM0PH36FBsbG4jH4/jn//yfY3t7G0NDQ3jjjTfwD/7BP8CJEydcPYff+Z3fwZ/92Z8ZzQRjY2P43d/9XVy8eLGj9V5IQrHOllAUhWKxiMnJSQD6PIcsy1hfX3f9tVvZ/M0pLuuQYKdrtgPy5dza2sLMzAyOHz+OCxcudPwE1klnW7lcxsTEBBRFwf37941i8X7ahs0zAsePHzdEAA87PdYK8tU8Xl943ShMswwLWBr5REZEVt7b2IN8sG6g0FxbAfRrYBchWH3jaYq2jWKshXdN03RCMWmAMQyDaCgKJsogIkcgSRJKpRJKpRJWsiuoatWaAn9QCKIsl+s+v1vHboGhm1//D3LKqx1QlK6aEAgE8Morr+Db3/42vva1r+Ff/+t/jS9+8Yv4tV/7NZw6dQr/4T/8B/yjf/SPXHnN73znO/j0pz+N27dvQ5Zl/Nt/+2/xsY99zHDMbBcvHKFYZ0soijJSKsePH8fFixdB0zRyuZzrE+1A8zmUTuXmD+Jcp6ence3aNQwNDe1rnXZTXslkEhMTE+jr68PVq1drNnU3J+UbpcdIATQajTZNj7n9dGwuwgO6nIn16Z2hGWiUZgwR0hRt63Jo9Taxc04UGVHf0E2kYCUYYG9w0XwcS7EoKkX4sDfca/WY5zgOHMfhWO8x5Kt5VCoVlMtlXTI+mYLACmBF1iAZmqYhsIKtzIodPkwpr2YwR0o+nw+3b99GoVDA9773PRSLRXz3u991VYblG9/4Rs3Pn/vc5zAwMID3338fP/qjP9r2ei8ModhZ85prA9evXzfc3YC9jd/tKdxG0UQ3yM2TyABAnZdLp2iVUMyOjo1mWw5qUr5b0mN/tfxXWM2uGj8T614zfKyvJr0V4uv9ROxqElYDLkJMZu0tClTd67E0C0VT6gQkBUaoS2E5+dVTFAWKoiCKoi5cGYtBpEUkcgmje2xnZweCIODuyF2UciXwEb7pRv1hTXm1ch5EdoWmaYRCIfy9v/f3DvT1Mxn9HiRt0O3ihSAUO/mUXC6HiYkJiKKI1157ra42QD4UtwnFLkJpN8VlhVuEkkgkMDk5ib6+PqTT6X1N7JrRSg1FlmXMzMwgmUw2dHRsFKG4nfZrNT2mKIprEeL41jjGt8aNn+3ae621DgpU3cS53UxIkA/WrEWBqiMmAAgKwZp/a7TyWgYRGYpBSS7VfD+c/OpFVqx7HRItmedfFEVBpVzBxcBFzMzMGF1MjXSwuiE66FZzrXw+f2gdXqqq4ld+5Vfw2muv4aWXXupoja4nFKtvCQDjKbhR+yv5UGRZ3rfDoBnW9JRbcvP72Uw1TTPmbS5fvozh4WFsbGy41grbbA6lUChgfHwcHMdhbGys4ZP/UYlDNkqPVatVTE9Pt5Qea4S17Br+avmvan5nbe8NCbWRiKZp8LP+mgl1s7yJGdafw0LYtuhujULs0mSA3jm2o+4Y7zPABRytg3mGryE9J4kWhmFw+9Rt3Llwp66LyeykSAjG5/N1TcrrKAUqzedxVH7yn/70pzE9PY3vfe97Ha/RtYRini0hNxyZ8M7lck19zcmH4rbUvLnLqxvk5qvVKqamppDP53H37l2Ew+F9r2lFo5QXkbtvtfDfLeKQ5vRYKpUyZDQ6TY/lqjm8vvB6zWZOrHsJSA3DCmtNQ4NWRwp+zl8TYTiRiXXWxSniMEdFJIKpKBXbz8A6GxPkg3XT/maQVmFrFxPRwUomk9jc3MTCwgJEUTRskwVBOLJNvRuiJEB/ADaL1RaLRQQCgQN/4PrlX/5lvPHGG/jud7+L48ePd7xOVxKKqqqQZbkmxZVKpTA5OYlIJNLShDdFUaBp2lWNLHIusixjfn6+4xSXFZ3KuWQyGYyPjyMcDmNsbKzmy3jQkvOapmFpaQlPnz5t6xocll5YOyDdNb29vbbpsfn5efj9fsfuMVKEtz7dmzWyWJqFCrWOKHyMD0W5CD/8dcVwMxhq7/X8nH0Lsv5m9v6zUcQR4kPIVrPQNA0MpcvWk440K0RONEhJZEWUpJLjZ3UychJDQfsmELMO1unTpw2ZkocPH2JtbQ2PHj1CKBQyopdIJHJoaahuqaFYI5SDdmvUNA3/8l/+S7z++uv427/9W5w+fXpf63UVoTjNlpCNq5mIoRUH4a5I0m+JRMI1ufl2N39N0wzV5LNnz+L06dN118RtyXnz+VWrVTx8+BDFYhH37t1DKBRqa61uVxtupXuMDFf29vbize03sZZdq1nD3N5LgbKtQTz/o/HZORXDBXZP24uneUiqZMilmGGup/AM7xhxAICs6eShaRpEVnRUDWZoxoiqzOKQTmgms2IGy7JG8ffmzZsAYFzn2dnZminygx5i7ZYIxa4of5CE8ulPfxpf+MIX8NWvfhWhUAhbW1sAgEgk0patB0HXEIpVPoWiKJTLZTx8+BDVarXtjQtw3wyLpHcA4M6dO66F5yQ91UoDgbn4/eqrrzp2Y7iZ8jKTE5FQCYVCuH//ftvXoJuIo1WY02MAjLpAMpnEt2a/hR9mflgzl8GybE3dwokoRFZEUtadFp1SU8Bel5hZGNL2PBkOJbkEhmZAU7SjNW+ACxipuCAbRFEugod9xE+iLKfCvhn9/n6ci51z/LsdyL1A0zQ4jsPQ0BCGhobq6i8kDWmtv7iFbiIU83kcdFH+s5/9LADgox/9aM3v/9f/+l/4hV/4hbbX6wpCMc+WkFTV9vY2pqenMTg46OjT0QxuEYq5i+vSpUuYmZlxvXMMaN6RRozB9lP87gRkrfX1dczOzratBeZ0XuZ//yIRDelq0sIadvO7GBQHUSwWa3zho8EoaJ7GQGTAkSh4hoemabadWgQcvdclZjdbYl6LHNdoPWDvuoeFMHakHcf7yCyz4lTYN6Od6ITALJlkPcdW6i9mgtnPA163pLwURanZ60gN5aDg9nfuSAlF0zRUq1VUKhWwLGs8Vc/OzmJjYwNXr17dV33CDUKxdnGJooiZmRnXpVKAxk9Jm5ubmJ6exokTJ3D+/PmmT1NuRiiapiGfz2N+fh43btwwlFT3s575v8l8w4vkh5KtZPH6wutQNRWCIBgmaaqqgpEZ7OZ2oaQUxHfi4AXeiF7I5k0IgKM4KJpim8IC9HpJK8ZXPtbX8nH5ah4BLtCQdIC9Okuj6Ml87NW+qw2PsYM5QmkEp/oL6R6bnp7eV/2lWyOUF0kYEjhCQiGzJaurq9jY2MCdO3dQKBQwOTkJmqYxNja2bx/l/RKKXRcX+QK4mUozE4oVRA9rY2Ojbniz2ZpuEEq5XMby8jJkWcZrr72278/EXI8xR2RHPdjWDmRVxusLr9tuyDRNwx/wY8g3BAoUCuWCIRufyWSMwcCB8ABUTgVDM6io9qZWRASyUWQCwNAAC/P2nV9mcIxOYFWlClVrnGaVNMlWo8sOrcqsWOEUoTSDVSa+UqkY9Ze5ubm2RUS7iVCsRfn9PsAdJo6EUEhkQsI7RVGMYbNWn8BbQaeE0mhQkaTkDoNQSHSkaVrbBOvGEz+RUAkGg2BZdt9kQs7LCS9KyuuvnvwV1nP2OnEBLqB3bbF+FKSCIVsSCoWM+75armI7tQ1KplBSS+A4DsVisU42PsgHUVbKdbIqVgT5ICRFcpTGJ+AZHsVqUa+1KPaKwcb74ANQVAUl2bmjy7zujaEbDY9xApmS3+8DhSAIdfUXQjDm+gtJkVnrL92U8jKfR7FYbLt2fJQ4EkIhmzKZLykUClhcXDQUad1CJ4TSyqCi291j5AtlXjMej+Phw4cYHBzE5cuX277Z9xOhWCVUeJ7HkydPOlrLCqeOthelhvL+5vuY2J5w/DtFUY71BoqiIAgCBiK6NEy6pEuVkPkXWZYN2Xi/z48KW2nYzkvWlBTJsfPLDB/jg8qoNRGHU4TCUAxkyA07ugiuD16HyHamynAQkYG5/kLawEn9ZXt7G4uLixAEwYheSKqyGyOUYrHoyoPcYeHIUl4URSGbzWJ2dhaapuG1115zXVupXUJpR27ebTFHc6cXaZO+cuUKRkZG9rVeu5BlGdPT00ilUoYW2M7OjqszLe38vpuwklnBt55+y/HvIiuCoZiGaSeaoqGpGnJSDjRNg2VZo/4iSZKRHpNyEjJ0Bhqn1XSPWRHiQpA1uU7jywqGZgAKjha/1vehqEqdHpjT+7l97HbT45xwGFPydvWXTCZTY9MLAGtraxgcHDzU+RczyPf/qKRX3MCREcqzZ88wNzeH48ePY319/UCE+lollHa1uA5ivoWmaZTLZczNzaFUKnXUJm1GJ4ONREKF5/maLrKD6Bizw1FGKM1emxThrYOJZvg4X538vBUhPoSCVHD0fiHpsZgQw1Z2C6VSCfl83ugeI+RC0mM0RaNYtW8jtr5uupyu+71dhCIwQtNaDMHV/quIitGWjrXDUQhDsiyL3t5ew/+lXC7jrbfegizLHdVf3ALZq6xzKB6htIBAIIDbt2+D53msrKy4LuIItBZJdJPc/MOHDxGLxVxxmWx3sHF7extTU1O2EipuD0nafdbdnPKSVRl/tvBnji6JgO53kq/kG6adiK+JNYVlvRYRIYJUJWV0j0WjUaiqakQvJD3WF+xDmk3XdI/ZwSrb0ghRMdq0o4sgyAdx69itlo51Qjekmki78cWLF416Vjv1F7dgRygH3TbsNo6MUPr7+yHLMiqViuEu6DahsCyLSsU5bO9Ui8vNCIXUKyRJwujoKK5cueLKdWg15aVpGh49eoRnz545eqccRIRCCtQ8z7tSlD1IfGv5W9jIbTj+naZohNgQUpVUw3UiYgTJUrLmd9br6uf8xhR7zWvQdI2qr0AJKBQKSOaThuQ4iV7M6TGe4fVivMMwpPl7F+SDulpFk1oMoKfFBvwDOBbcn+xQtwhDAjD8ldqtv7g14Gz2eAL0a1MoFLyifFsn8PzGd1sVGHCOJLpFbt5crxBFEQMDA65trK2QQLVaxeTkJMrlMu7fv+8YWrutC6aqKqanp7GxsQGe59HT03NgUd9+8d7Ge5iOTzc8JsSHkJUaP9VHhIijvAkBT/NgKKZpm67IimApFqyfxYB/wCBna3rM7/cjFogBTR6oKUqXhqkq1YYpPQIiwfLqsVebHtsM3eCFYiYUK6z1F0VRkE6njeiFzL+Q6GU/9Re7TjMv5dUmyId4WO6KbsjNuzEwafZzGRsbww9/+EPXhyUbrUeEJSORCO7fv98wxeZmykuSJCQSCQQCAdy7dw/lctl4+qtUKnjvvffQ29uLnp4ehMPhQ3l6ddrQnmWe4YebP3SUMAF0ogBVLxlvRkgIQVEVxygBgCGr0mxzZSlWjyroWpUBu/QYLdFIJpLIVXNG95g1PaZpmkEQZDiyESiKgsAI8HG+tmVW7NANKS+zQkczMAxTU3+pVquGPIy1/hKLxRAKhVomTDtCKRaLHqG0AvNAG8MwrqsCA/Ubv5ty8/shFGJZfOrUKZw7d864md1uRXZab21tDXNzc47CknZruUEoyWQSz549A8/zuHPnjiHVTQqfa2trGBkZQTKZxNTUFDRNM578DjJ3bYdMJYPX519vOP8R4ALIS3nQcN6IRFZEsVps2FZLURSCfBAVpYJS1Xk+hIKupyVrcsMpd5qmMRQbQraSRW9/LzLFjOEJb02PaZoGnuFRVaotfcakJfrHTv6YK5FFt6S8Oj0Hnudr5l9IjSuVSuHZs2cA0PI9bCUUVVUP1Q/FDRx5hALAGG50G4RQ9pvislu3k81fURTMz89ja2urbubGbUKxiypUVcXc3By2trbaklDZL6FomoaVlRUsLi6iv7+/Tl7C/DrHjh3DsWPHoGmakbve2trC4uKiQT69vb11EvJuQlZl/Nn8n4GmaceNm6j5BriAYxGbRBM8wzsW9DVNQ4jTN+mwEG7YqktEJpsex+tuj35WL8azLItQKGTk4iuVipEe41Ueq1urGIgMoMAW6oYrzYiIEWTKGQT5IF7q78zRz4puSXm5QWp2LqG5XA6pVKqu/kKK/OY0v525FgCvhtIu3FYFNq8ryzLefffdfaW4rOgkQikWi5iYmABFURgbG6t7UjkIQjGvVy6XMT4+bkzdt/O0v58aiqIomJ2dRTwex61bt5DL5RCPx21fw0xaFEUhHA4jHA7j1KlThglTIpHAwsICqtUqIpGIkR5z04ToLx//JTbzmwhw9vcKTdFgaAaSJNU4Lda8H5O3SVgIGyKLVvgYH/JyHr10b8Oog+hp2VkK16zH6v4qGjSwDAvYBP4kPXZy8CSml/QagKRKSCfTNcOV5vRYiA8hW9aJ89WhV8HS7mwd3ZLyOoiHE3P9hRi4kfrLs2fPMDMzU1N/qVartoTiRSgtwPzlZ1n2QFJeuVwO+Xweo6Oj+0pxWdFuhEJSbceOHcOlS5dsv0AHMX1P1iNe8wMDAx1P3XcSoRASA2AIa+bz+Y4sgFmWRX9/P/r7+43UQiKRQDKZxJMnT8BxXI3DYqedNz/Y+AGmdqZqJN6tINpaYT7s6FxIogme4R2L7AIrYFfdBadxhgik7etxwRolYaeIiOh0KarS8HXN7wEAekO9UBgFwWgQsizXpcdigRgKQgGCT4Bf8OPm0E3HddvFi57yagfN6i/VahUsy+Lp06fI5XLgeR48zx/IjB7Bd7/7Xfze7/0e3n//fWxubuL111/HT/3UT3W83gcyQiEprpWVFXAch6tX21dBbYRWIxRVVbG0tIRnz57h6tWrGB4ebrim2xGKLMtYXl7G0tISLl26hNHR0Y7WMrf6thoFpFIpjI+Po7+/H1euXDFIzA2DLXNqgUibmztvZmdnDeXZ3t5ehMPhpuetaRqepp/ib57+DQA9CrGDWXnXrr3XeozIiqhW6ov6DK27JKqaaohA2oFneJSVsnGcU2Gfpmjw9F5qzel1yd9KUgkatLprbk2PKZKCcqmMbD6LSqKCl2IvYXV51UjZ7Hde6oOU8moX1vrL0tIS0uk0stksfuZnfgb5fB6RSAR/9Ed/hI997GMdW0Y0QqFQwPXr1/FP/sk/wU//9E/ve72uIBQ3IxRzF9f169cNWQU30UoTQaVSweTkJCqVSsOWXAK3CQUAdnd3oWmaIaHSKdq9iVdWVrCwsIALFy7gxIkTdf/e7Ul5s//7uXPnUKlUkEwmkUgksLamOymaoxdRrC+QZ6tZ/Pnyn0PVVEd3xSAfNKRLrN7tBGZ7XpZmHdNTZs8SP+e31etiKMYYhiSv7xjFmCKORq/L0RxUTTU0ugRKQFEuQmDqn4JpSp99YTgGgXAA0ICfPPWTUIsqHj9+jFKphHA4bFzXVojbig9yyqsdkOakYDCIy5cvY2ZmBp///OfxO7/zO/jSl76EX/mVX8Hw8DA+//nP40d/9Edde91PfOIT+MQnPuHael2R8nIrQrF2cZXL5QOpzTTb/JPJJCYnJ9HT04ObN2+29BTnJqHk83msr+tquB/5yEf2Pd9DPitr0dAK4mWzs7Pj6CbplD5z88lLEIS64n4ikTCMmaz+8LIq443lN1Ck9Kd/juHqZkYEVqhV/bU5XY7hIKuyMRgY4AK2BFDjWaIBVbUKEfUk5+f8RtrKbHZlhdWvxOl1KYqqaxCgKdrx2ltl8y/3X8a543utwqVSyZgoX11dBdB6RxPBhynl1Qxmcy2O43DixAn09PTgb/7mb1AoFPC9730PFy9ePOKzbIyuiFD22zbs1MVF6hJuT+E7EaCmaXj69CmWlpZw8eJFjI6Otvy6bhEKkVAJhUIQBMGVYVHyZWsUQZTLZSMyvH//fsPN5DA95c3F/dOnT9f4w8/Pz0OSJLyVegu79C4ikQgCYqBOpoSkp0gU4Wf9dREARVE1KSeaom3TWGGh1rPEz/pRVeoJxWqU5eRLYo6ayOs6pcWsKsg8w6OoFBFBpO5YO6Ouu8O1joykcD88PFxD3KSjSRTFmolyuwerD3PKywpFUWpqf2al4UAggJ/4iZ84qlNrGUdKKGQT2U/bcKNBRfI0bbXV3C/sNn9JkjA1NYVsNttRimm/hKJpmlE3unbtGsrlMlKpxlIgrcIsBWGHVCqFiYkJ9Pb24urVqw2jGKsFsNW98aBh9ofXNA3fefIdrG6tgqZprK+vI8SHoPF7Cr80TcPP+ms2c7vuKetmbWeKZUdEsirXzbHYGWXZ1WvMtZBGrwvYE4TIirYPW1bSA4AT4RMYDjnXAK3ETRwVk8lkw/RYN2zmzSLvw4KiKDXp2BdNaRjooghFkhrLb9uh2aDiQRGKNULJZrOYmJiA3+/H2NhYR1EBTdOoVp0nshvBTkLl2bNnrkvO2234q6urmJ+fx/nz53Hy5MmWhiTb+f1B4mnmKd7ZfgcMwyAajSLoD6JULqFQLCCdTiMej6PH34M0vyfAKLBCXaRgTTlRoGpSZm+9zYABix/5iFyjkxXgAthWt+Gn9vwu/Ky/rjZjV68h0+1WvxK7+RQ7fxaGtm8EMNeAzGjXL97qqOiUHpMk6cj9Ppzmoo7iPKxtwy9SyzDQJYTCsixKpcYOcma0OqhIhNYOwruErEmmzs+cObOvLoxOIxQnCRW3FYKBWkdJ85DkzZs3jVbIVtY6zJSXE1LlFL66+NWaDT4khKBChSDqBWof7UM8E0e5VDZaaPvD/VA5FT6fTy+icvWpKNIyDACSBHz3OyxQDWBmPod/8ou152F+zxzDQdbkenFGyy1FUfrEvDU1F+JDdedinksxI8gHkSlnaiIUnuFtjbp6fb37llmxS48lk0msra0Z3vDN0mMHhW6IkgB7QvEilDZANpF2ivLtanEdxNAkWXNqago7OzttTZ07oRNCIWR27tw5nDp1qobM3CzyWyOUSqWC8fFx4zNod0jyqP1QJEXCV+a/UlNroCiq5oldZEVU1SqCoSCCoecqvLKKXCGHYraI3d1d+Hk/sr4sOJGDKIrGdZLUvWj7G99ggXIEEDN49ebefUgsggloigZHc3X1Dx/nq4sYnBwhrdEKR+/NpZhBUVTde2dovaPMrvB/Z/iOqxGkOT1WLBYhCAIikYhjeiwUCh3oht/NhHLQEUo+n8fS0pLx8/LyMiYmJtDT04MTJ060vV5XRCitbvqdaHEdBKFUq1UUCrqkxWuvvWbbhtou2iEA0k21vb3tGB24rRBMiCCdTmN8fLyleonTWkcdoXx96evYKezU/C7ABoyiO5FMMW/EFEWhJ9gDhtPTY1ABpaoglU8hs5uBoijw+XzoDfUiy2XBcRzicQoz7/cAYgYUreHatb3Pg6Frr5tT7YOlar+i1vSacf6WQUy7ji4CQkjm621uY7aue23gWt3v3YKqqnXpMSIYup/usXZgLYYfFY6CUH74wx/ix3/8x42ff/VXfxUA8KlPfQqf+9zn2l6vKwil2RzKfrS43CaU7e1tzM3NgaZp3Llzx7Unm1YJhURoABpGB26mvAB9g9rY2MDy8nLL9RInHHTbcCO8s/4OZndnLScEVJQKOI6rkUwxw1pzCIkh5Ogc+kR9EzTk4wt5pAtpMAyDSu4YIOipsrNnVZBbhWf4mpqIE0lYj7NLrxFYBzGdohgAderJETHiOLfy6jH3ZFbsYNc2LIoihoeH69Jj7XSPtYNujlAGBwcP9DU/+tGPurpPHHnKC2i86e9Xbt4tQiGktra2hrNnz2J5ednVm7AVQkkkEpiYmMDg4CCuXLnS8PXdTHmR1uunT5+2VS+xw1FGKE/ST/DtZ9+u+z1p3eXA2XY4AagRgSQiiWbwPI+oP4qiXESkLwJVUvEXD2iAAvr7C7j2UgrpNAefz4dwKGxs6j7ah1w1hyBfnys3T7sLrICyUra9RiJTS4BOBAXokRA5VtM0BBj9fdndSxzDuSqzYodmbcN2mm6Nusc6SY91M6F4NZQO4BShuCE374ZxU7lcxuTkJCRJwv3796FpGh4/fryvNa1oRACapmF5eRmPHz/G5cuXcfz48abruZXyqlQqmJiYgKZpePnll/dFJuS8jqKGkiwl8dWFr9q+hqzJDcmEova6tswiiVaQdmKWYVGtcph/pF//H/9xGv39oh7BZPLY2d6B6BMRDURRVsoIoP4hiaVYIyKyzsFYwbO6PAs5P/NcihXm4ryf86OoFh039Gv91+DnDrYDq93NvFF6bG1trSPLg25qG36R7X+BLiEUaxRhjgauXr26L7n5/UrjE2HFvr4+3Lp1CwzDoFgsui6T4iQOKcsypqamkMlkcOfOHUQi9UNodnAj5UU6yKLRKDiOc0WkrlGEclCoKlV8Zf4rtgXnIB/EhrqBHrbHMe1D0kciK6Io1XdMAahpJ/azfnz7B0UoMouhYyrOnKEB6PpYIT6EeDYOqSwhnU2jWC6iKldRrVbh9/shCIKuVcb7ka1kQYFyrG8AqFEfduroIiBy9uR8K1LF8R6hKAp3hu/Y/s1N7Hfo2I30WDe0DWuaVkds3hxKmzCnvEiEYk5x3b9/f98M3WmEomkanjx5gidPnuDSpUs4fvx4zfmSG+Agayj5fB7j4+OGq2M78y37TXmtr69jdnbW6CD727/9W1dI1E7Xy1z0Pwi88egNxIv1kvnk9RmKgaIpoDX7z7KqVMHQjO3cB4HACKjIFUSECOK5DN5/X/+s7t3dO56maJTkEnyiD33hPuSreUgbEgRBgKIo2NnZgabpQ5VFfxGcwKE32OuYvgJ0kshUMo4dXWaQWghLsTURj92GfqHnAnp89dI5bsPN71Cr6bFYLIbe3l4jPdYNKS+yR1kjlBfJCwXokgiFZfWumq2tLczMzOzbUdGMTmookiTh4cOHyOfztlEBufkOklC2trYwNTWFkydP4vz5820/xXW6QauqioWFBWxsbNS0Q7u14R92yuuttbcwn5i3/Zuf86MoFcHRnFFDsSLEh5Cv5hHgAw0FF/PVvDHhPjnJoFKmEI1puHhx7zM1ZO8tNQ5RFI0Hp2q1CkZiEM/FQadobFPbxgyH1fyKqBTTFN3QxAvQC/w5KVfTdNDoet8buef4NzdxkJt5o/TY+vo6VFVFT08PyuXygdhntAPy3bfWUI566LNddAWhkM1yamrKFUdFM9ollEwmg4mJCQSDQYyNjdm2Ex7EBD4hFFVV8ejRI6yuruLll1/uuMujkwilWq1ifHzcqBWZb2a3usYOM+W1lFrCd1a+4/h3hmIQEkJYV9YhwD6dp2pqzaCiHfycH1W1iryUh6oC772n3x937yggb4tMz9tJoJjB8zwEn4Cz0bMoVAsoloqGrazZ/Mrv96M/1I9MJdOwCE9ACvzmY53STcfDxzESGmm4nls4THFIp/RYIpHAo0ePsLKygt7eXkOa/zBbiWVZrvG11zQNhULBi1DaAUVRNW2w++0gskOrxlWapmF1dRULCwtNvdbJ791sRyapuR/+8IeoVqu4d+/evvKn7RKKuV7y6quv1hHlQUQoVodGNyOUZCmJry1+zXFNkRFBU3TDzT3ABcBQTMPNmqEYVOQKVKhQNRVzczSyWQp+v4Zr1/bujyAfhKqpDU2vAD0iqigV3QmSguH7AsAwvyoWi8ikM0gwCURDUeTZvDG5bwdS4G9GZgRWEciDxFGJQ5rTYxsbG7hw4QIAXZdueXnZcFM0a48d9HCl9fPzurzaRCKRwHvvvYehoSHkcrkDcSZrJUKRZRmzs7PY3d1tidSId4GbhflCoQBJksDzfMuS943QTpfXxsYGZmZmGhKpW11jhDji8TgePnwInufR29uLQCDg2vUkRXirBL0Zfs6PdCXdcB2e4ZtuwEE+CFmVUZb013rnXX1TuHVLgfUjrCrVelkV1EZnGrQaDxQzzOZXQS6IUqmE3dwuStkS4vF4nXUvWdfP+6Fqap27pF2E0uPrwYWeCw3fs5vohvqFqqrgeR7hcLguPZZKpTA1NQVVVWu6x9xORdl5snhdXm0iGAzi6tWrGBoaws7OzoHkMRmGaSi6mM/nMTExAY7jDJvaVuBGOzLB6uoq5ubmAADXr1935YmNpKgaddGQesn6+jpeeeUV9Pf3N11vv9A0DYqiYHJyEufPnwfDMEilUnjy5AkAYHJy0rBJ7WQiWtM0fH3p645FeEAvTueqOdvNnYDMdjQ6hgIFUDBqF0+e0Ijv0OA4DTfNMiucH7Im10iymM+XIMAGGvqemMEwDMDDSM8oimJY95LiviiKCPgDECBApuSWPj+3ZVaaoRv8UOy6vKzpsXw+j0QigXg8jkePHkEQhJrusf2mx6yEQj5PL0JpA4IgYGhoCMDBSKQ0W3dzcxPT09M4ceIEzp8/39aN7UaEoigK5ubmsLOzg5deegkPHz7c13pmmHOxdhtEtVrFxMQEqtVqS910bqSkFEXB0tISNE3DrVu3jKhkcHAQpVIJb7/9NmKxmPGlFUXRIJdoNNpSk8a76+9ifte+CA/onVYhPoRU2Vnan4gvNotOIkIE6XLa+JlEJ6+8osL8XMLSbNMaB6DPkzQ6L4IAF0BJKtV0dBG3P7IBVatVFItFaGUNj5OPAWZPoNHn8xmfp/neCHABXOs/OJkVO7wIfigURRmR4alTp6AoitE95lZ6zEoo+bzeAOLVUNrAQbg2WmG3rvnJvNPC937bckulEsbHx0FRFMbGxmoEGN34gpkVgq03dzabxYMHDxCJRFpOr+2XUMrlsiEoCQDRaLTGsoCc7+joKE6cOAFZlpFKpZBIJAwjrGg0ahCMXcrhUfIR3t9633EOA9C9Rppt7lEh2pRMAlxAr3M8x+YmhZVnNChaw+3be/dbj9iDZDnZcC3gOTk1ScEBew6LdikxM3ieh8AL8LN+BKoBlMtllEolJBIJw3eD47iaz/Tm0E1wzOFqWh11yotE8e10lDIMY9yHAAzL6WQyaZseIwTeCNYoqVDQo14vQukQbvrKm2ElFKuUS6e50P0Q4O7uLiYnJzE0NITLly+Dpmljc3VryMrJZZHUS9qV299PDSWTyeDBgwfo7e3FmTNn8P3vf9+YPTGvT86XoiiwLIv+/n709/dD0zQUi0UkEgns7u5iaWnJGFgjXTnpahp/+fgvHd0KAV2SRKOcZ0kAnXAUTWmY6uIZHizN1pDOu8+jkytXVITD+jUPCSHbNJcVPtZnbGzNEBWjSJaaExQAxHwxJEtJ3STseXG/t7cXkiShVCqhUChAVVW9uynUi5GTI5Ak6VC7m4465UW+w/s5B6vldD6fRzKZbCs9ZjclLwjCocr4u4GuOdvDiFDIRj44OIjLly/va86lkwilkYSKebbFDVjXM6sPNKuXOK3XSYRCCIwMSJbL9npUzfScAoEAAoEATpw4AUVRjOhlcXERhXIBb+bfBOfjoHKq7Rc2yAVRlIp63cPhtYNcECW5BBXOnwGReTc7KKZSFOYX9OtNBhl9nA+yIhuSKE4gw4ZWAy07RISIo/yK3bF2ZluA7lrJcRx4nkc8HsexwWM4wZ3A7sYuni4+RSgUQm9v74F3NzWr8R0GyPfDzeFKkh47efJky+kxu5RXIBA48nRgu+ialNdBRiiyLGNpaQnLy8sta2G1sm47BNhMQsVtQjE/8VsdHTvpHGk35UUsiVdXV2sIrJmdcCuvwTCMMbCmqiq+OPVFyFUZ2VwWxVIRDMPA7/cbw4AiJ6KiVODn/I7pLo7m9GN4f53woxl+1g9FU2rMrX7wAwbQKJw5q6K/X9On1lVFt+htUGBnKAY0dB8UO4tfM0J8CGW5jKra3NUzxIdQVapNi/uapoGhGYQDYXzy2ifR6+utS990oo3VKtzezLvxHFpNjxFiJf9PCOVFwwc+QlFVFaVSCRsbG7h3755rRa52urxakVAhKSC3PUyy2Szm5uYQCoVqHB07Wa9VQiFKA4VCoW6exolQmhGNE76/9n08zT/FSN8IMhXdgbBcLqNYLCKZTEKRFYT9YTACg0gwAtgEpTRoQ36lWHVOmZE5jhC/dw8VChQePtyLTogPSUWpOE7WE/g5PyRVQlEpgmedZXWIRleAC6BSsY86jDWfG3cFuBY2I03vZjsePo5en77hWdM3ZPhva2sLi4uL8Pl8Rqqx1UYJx5d//lkfdcqLOLseBpzSY2tra6hUKnjrrbfwv//3/8axY8cQiUQO/Lz++3//7/i93/s9bG1t4fr16/iDP/gD3LnTuYbbkROK2bXR7QglnU5jZmYGmqbh/v37ruaGW+3yakdCxU3JeYLJyUmcPn0aZ8+e3dfN2SrZFQoFPHjwAD6fz/aaOzUfdHJuj5KP8L2174GmaKN2QlGU0ckEACIlYiezA0qisLy6DIZhjElzImVCIokYH3OMYEJCCJlKBgIr1BDF++/TUBQKx46pOHFCReh50b/Z9DohJx/ja6i/xTGcUfNpJK0CADytW/iyNNuUzAA9kkkqScdBRjttLJJqXFhYQLVaRSQSMdJj7aZoyP101CmvoyI0c3qsXC4bqV2KovClL30Jq6uruHv3Lj72sY/hYx/7GO7du+fqHvYnf/In+NVf/VX84R/+Ie7evYvf//3fx0/8xE9gYWEBAwMDHa159CYAz7FfVWAzNE3Ds2fP8N5772FkZAQURbleaGwWoaiqivn5eUxPT+Pll1/GhQsXmn5x3BqW1DQNCwsL0DQNFy5cwLlz5/b9pW2lhrK7u4t33nkH/f39uHnzZsNrvl89r0Qxga890ifhg3zQtvgdESIoa2WEw2EMDA3g5MmT6OvrA03TSCaTWFlZQS6eQ7achSIrjoOQPta3p9LLCEYXWbUKvP9Af0K/d08xSMRMcHYI8To5URSFilJx/GyILXBVqSLIBxsSD03RYBkWkipBZMWGnW7k2mSrWQyIAxgNjzY8loA0Sly6dAn379/H7du30dfXh1QqhR/+8Id46623MDc3h+3t7ZoOPid0S8rrqOdggD3XyJGREXz2s5/Fv/k3/wZ3797Fpz/9aSwvL+NnfuZnsLKy4upr/pf/8l/wS7/0S/jFX/xFXLlyBX/4h38Iv9+P//k//2fHax55hELAMAzK5cYFzFYgyzKmp6eRSqVw69YtiKKI5eVl14t/jTb/SqWCycnJlmc8CNyIUKrVKh4+fIhSqQSO43S7WhfQTNTx2bNnePToEa5cuYKREWcdKDdSXhW5gi/PfxkVuQIKVE37LoHZUjfIBZGX8jXRS09PDwRKwG5mF2pRRTlXxuPc45pZDZqmDRVfVVNrpOIBGCKQsZiGG1f9hqxKkAvWTaUTmL3kQ1wIK6rzJkHeA0VRDQkK0NuYc9VcS9EJOVbTNFyNXG14rBPMjRKjo6NQFAWZTAbJZBJPnz7F7OysUXzu7e1FOBxuqDR9VOgG6XpyHlZhyJ6eHvz8z/88fv7nf9514qtWq3j//ffxG7/xG8bvaJrG3/k7fwdvv/12x+seOaG4mfLK5XKYmJiAIAgYGxuDIAjGlLzbJjpOEQrxXI/FYm1LqOyXUHK5HB48eIBgMIh79+7h+9//vqs1GbvNXlVVzMzMYHd3F7dv325KYPtNeWmahq89+hoSpQQAfdO16mMJrICyXDae0u2e1kVGhKRKCIVDyBfy6In2QIaulZVOpw0pk95QL8Drcx1EKl5/38APnotAvnaXQVXVu9coUKio9nUOnuYha7LRktyowG5OmTWy87UeG+ACDWdoBFZARalA1VRE+AhOBU45HtsOGIYxCvfnzp1DpVJBIpEw6gMAjL/39PRAFMWuiA5eFHMtt6/T7u4uFEWpm8EbHBzE/LzzYHAzHDmhEOw35UXaU0+dOlWT4jErA7t541ibCMzikp16ru+HUEitxvz+3S7yW9eqVCrGsOL9+/dbkq1pdk2aRShvrr6JR8lHxs/W7ijSOUU2dT/nr6s9MDRTM4/iY3yoKBUEg0Gj9iLLMliZxXZmG+VEGQzNIOlLQvDpelmzsyxyWQp+kcb166rRzmtHcMBeSopEGmYrXiusrouNhhjNZNIs1cZS+tednOv13uugpYPZ0AVBqFP2TSQS2NzcxMLCAvx+P4LBoCHFc1SbejeQGmA/Kf+iDTUCXUQonUYoqqpibm4OW1tbtvMV5GZxu4PMOow4OzuLeDyOV199FT09nRkTdTrb8ujRIzx79qxu6t8t/S27tYg6cSwWw0svvdTyhtAsQml0vguJBXx/7fvGz3absp/z12zoDFV/Xo0cEAl6/D3IVrMYFAehaRp4lcd2ZhvpdBo7O3F87/unAQ24e5OGQu215zoNTZpTcNb3ab4OPs6HglQwoqpGxBPgaj1arK9hBkVREDjBqAX5OT8uxS4hGW9tSHI/MBf3T58+DUmSkEqlsLW1BVVV8eabbxoqCER48bDSYN1KKIVC4UDbhvv6+sAwDLa3t2t+v729bchhdYIjJxRy43QSoRSLRUP63urfYV7/IFqSyZrkHIiESqviknZol1AkScLk5CSKxSLu379f90TjZteYOeVFNNCayfw7rQO0X0PZLe7ijaU3av5uTWVZ5dmtHVl2x/hZPzaVTYSw1woc4ALISXukRNM0KJYyHhQePdKQTvFg1RB6jo1jZUV3WewJ9SCv5us2qLrXNEVN5vfDMfr8inlK3+l68Axfp15sV0sisKbNbgzd0J0bj6B+wXEcBgYGwPM8stksbty4gUQigUQigcePH4PjOINcDtqX5CijIzOsqbdisYjh4eEDez2e5/Hqq6/ir//6r/FTP/VTxjn89V//NX75l3+543WPnFAI2t30ify5Wb7ECW4qA5vXJIKGx44dw6VLl/b9pNPubMuDBw8QCAQcW6Ld9BihKAqKomBxcRErKyu4fv16R62FjYjDaXMry2WjCE/g5/w1w4V2dQZiy0sQFsJ19QWWqf0KkBkS8/lZ137vPR4oRXDjx5I4d+44KpUKSqUSMukMsuWsXm95PljZG6q38LWLmkhHlzllZZeuA/am9UvKXmQUEmrTZGZExEjNsCZLs7g1dAupndSRt+wyDFOngmCdLA+Hw0ZxPxQKuXrO3RKhyLJ8qBEKAPzqr/4qPvWpT+HWrVu4c+cOfv/3fx+FQgG/+Iu/2PGaXUMorU7Km1M8V69ebYnF3fYu0TQNyWQS6XQa165da9jV1A5ajSi2t7fx8OHDunpRp+u1AuJhQlHUvs2/7JRuza9j/fmri1+t068yb8oiK6IoFWsiFp7ma6ITH1ef5hJYoWYTZijG1ovE/PPGBoWVRyFQ/ixu39IHGUVRRCwYQ0kuIaJEUCzqLovlXBk72zsQfaLROeYX/LYpLLt0lR3xUKBsU3ZOLcUhPoRsuXbdawPXEOADSGrJIyUUOx0v62Q58SVJJBJYXV0FAINcenp69u2h1C2EYo1QDsNc6+d+7ucQj8fxmc98xigZfOMb3+jYJRboAkIxF8+bPZ2TdtxKpdLW1LubQ5OSJGFqagqpVAqhUMg1MgGaE4CmaVhaWsLTp09x7dq1prlOt4ryxWIRm5uboGkar732mu2kf7twilCsv//OynfwOPW45nfEqwR4XmBHveCjj/MZ0QiRQrEKPtZEMFp9/QV4XiA3/e69t30AU8XVq7IhAgk8j3Rk/V4LhULoiejpsVwxh1KphGw2i3g8jp5AD1ROhd/vNzZDuyFIu3QdAFs74gAXsI1kSIuymWgpisKdY/okdDfoaDXbzM2+JKqqGsX99fV1zM3NIRAIGOQSjUb3LRt/FLBrTDisovwv//Iv7yvFZcWREwoBiSKcbrJUKoWJiYmO2nHdilByuRzGx8fh9/tx8eJF44nJLTQiFKucSStk6kZRPpFIYGJiwujKcYNMnFJx1s1tfnceb629VXccz/KG6KLd0zpDM8bviBSKdcO1zpRYiYPATFSpFIO5WR7gSrh7d+/3PFMbDZlfUxRFPYKJxUBpFPKFPArFAra2tgDok/zb6W2EQqGaDcWargPsiQfQ02V114jhIWlSHYmei51Dr19/+u8GQmnn9WmaRiQSQSQSwZkzZyBJkqGLNTs7C1mWDd0xYtDWbP1uiFDsJPSLxaLX5bUfEIKwDhppmoanT59iaWkJFy5cwIkTJ9r+ErhRlCeFaJJmisfjB1KXsSMUUi/x+/1tScjsJ+WlaRpWVlawuLiIy5cvG/pYbqCVlFe8EMcbS2/U/d28eTt5pAf5oFEzcJrhMM+UBBi9CB8Uar/Afs5fU9MYfzsCcAWcfS4CSSCyIqqVvbSY02uGRV0+PxDUc+OUQuHp6lPIBV3ShFj4hgIh5FBLbgE+YEt4dpEMQzFgada2hdgss9INhLKfzZzjOAwODmJwUO/EKxQKSCaT2N3dxePHj8HzfI3Fgd1DaDcQCtlHDruGchA4ckKxmxchG6YkSZienkYmk2lpaM4J+yEUsxmXuRDtdl0GsCeAnZ0dPHz40HCVbPeJrpNzVFUVs7Oz2NnZwa1btxCLxfD48WNXC/xOv9c0DSWphC8vfNl2/kJkRWQrWUNbywrzLIZdegjQIxgSsfhYH7bVbQRQ/+U11zDoahTvz+QBULh3b+9eYim2JvpxiiIYiqk5jqd5XaiSAvr7+8EwjGHhm4lnkJfzRt0lFoqhIldsr79dJGOXugOA4dAwTkROGD8ftReJm4RGUZThWGku7pPOsVKphHA4bKTHSHFfUZR912H2C6snCyHHF82tEegCQiEg7b2k1mFOLzkp9LaKTgmlUqlgYmICkiTVSagchJCjec126yV26KTLi7xnRVFw//59Y9DPzZmWRikvTdPw1UdfRapUb4VLNm+ztpYVRCcrwAUcjwlyQWQqmRpZFStEdq9OE+JDeOM7eSgKg2PDKkZH947383uS+EE+6NhpFeADNQOI5iFHcu8Hg0GEQ2HQFI1gKYhSqYRivohMMgOKpYzOMUEQdH06mmvaFm2GVQTyRY9QGsFa3C+VSkZ67NmzZ6BpGj09PYZE0VGC1E/Mn4UXobgAsvGvra1hbm7OFZVc87rtgEio9PT04NVXX60Llw9itoUQiizLePjwIXK53L4k99slPWINHI1Gce3atZoQ3O0WZE3TIEkSKpVKjcfGm2tv4knqie2/8/N6CqqRo2JZKdvOaJhfuySXHGsrBDzDoyyX4WN9SBWKeDCuX4v7puiEpmiUJL11V2RFlKSSrcwLBapGeLKR1hfp9hIEAaIowt/vR66SMzrHdnZ2oGn63MtAZAAyIxv3ZiMyiYpRXOy9WPO7DzKhWOHz+TAyMoKRkRGoqopsNmt0j5H/JumxSCRyqJGbnZ6YV0PpEOYbmmEYPHr0CJlMBjdu3EBfX58rr9HO5t+qhMpBRCgk7fH2229DFEXcv39/X5FZO+dIpFucrIHdlnHJZrNYXFyEJEmGQ+DTwlM8qj6yHQ6lKRplqQyBERxJIMSH9EL48zZip2Os8vLW90rqNCSCeTAOXQSyR8P583vXIMAHkKvkwFKs3qnjMCVvTr1ZN/2aiXlQNekrcq40TRvpHECPIsvlMnZSOyiWi/ogYKgXFb4CXuBt79fbw7frivdHTShHlXKjaRrRaBTRaBS5XA7RaBSCICCZTGJmZgaKotSYinVqE94qrB1e5EHLI5R9oFgsolKpgKZpjI2NueoM1yqhKIpiCB02k1Aha7r5pSwWi9jZ2cGpU6dakrtvhlaiCnNqrdGwopsRCpHLOX/+PHp6epBOp/Fo4xHeSr4FFarhf04UfwH9yR1AQ5FEVVMbPv0D+kxJM68SH+tDtpoFz/DIlgv6ICOAu3cUmD+SqlwFBQoC60xyAAxpfaeaDkFQ2EuZNYo2BEFAf7gf2UpW74ysqMgWssin89A0TY9snl8/lmXhY324PnC9bp2jJpR2u7wOAoqigOd5DA0NYWhoyNYTXhTFGk94t33e7VqGAXg1lE6xvb2NqakpsCyL06dPu0omgL75N3O6KxaLGB8fB8MwLUmomAto+/1SaJqGJ0+eYGtrC+FwGBcvXmz+j1pAswiF2BJns9mmqTU3aijEFliSJFy8eBGjo6Mol8sI94QxvTENjdIQi8aMgipR/PX7/PBFfSjB2dLWz/nBMmxD+96QEIKmag11vEjLMYkOZmdp5HIU/AENL72091BC2oybkRORzvez+mR/I58SMpzYSJOLgEQyPMuD4RiwPhY96EG1WkWxWEQ+n0cikQDHcRg7PoZ8Jl83p3HUSrvd0GFlPQez6dXJkychy7Ixub+0tIRyuYxIJGKkx4LB4L6//3ZKwwC8GkonKJVKmJ6extWrV7G+vu7aU7AZzSIUIuPSjoSKuSttP18K86Z+6tQpFAqNXfnaQSMpl2KxiAcPHoDn+ZZSa/uNUEhdKJ/PQxAEhEIh49z+cukvkSgkDCO0UCiEWCxmaKXRVRoLzxYACkb0QtwWCURWtC3k17wHjUJJKdnWVsh7C/K6Ai7Z0N99V/+cb99SYH4wVTSlKZkAgEZpupOizUyIcV4UZRCPj/U51mIIiGAkRVF1KUCe58HzPKLRKFRVhVSRcCl0CbOzszWpnN7e3iOPUI66ywxoTmosy6Kvr89Iv5dKJUOW/9mzZzWy/T09PR2lqO2EIX0+35EPXHaCIycUv9+PH/uxHwPLstja2joQX3knQtE0DY8fP8by8nLLMi4EZhXjTrtECoUCxsfHIQgC7t+/j+3tbeRy9l1CncCJBMiwYjsEup8aSqlUwoMHD8BxHO7evYt3330X6XQagUAAeTWPxdQicrmc8eU2W8P2RHrA0Az8PX6Uy2WjW0eWZYiiaIgyZsvZhptwgAtAUiVDut0ONEUDGoyW28ePaezu0uB4DTdu7N0/AS4AULBtzTXDx+nk0KimY3w+FIyajVMthoAQUzOfFJqmcefUHbx67lWjFTWRSGBnZwePHj0CTdMIBAJIJBL79ofvBN2Q8mo3SvP5fDh+/DiOHz8OVVUNU7HV1dUaU7Genp6Wi/t2Ka927ZS7BUdOKIA+oHRQvvKAPaGQyfN8Po+7d+8iHA63tSZN0/vaZOPxOCYnJ3H8+HFcuHABNE27Xui3W29lZQULCwu4dOkSRkdbs34la3USoaTTaTx48ACDg4O4dOkSNE3D8PAwNjY28PjxY8zIM9hIb0CWZRw7dgw8zxuTwwDgZ/xIlpOGZhbJZ0uShFKphEqpgq3sFsoo10Qv1i8jQzMoVBpHf2E+jLyUN177nXf0L/mNV1SYM6AczdUc5wSO4sDxXNMoRmAEFKWi3g7dxDeeyKw0qrEQUBRltAqb5zRIKufhw4dQFAXz8/OQJAmxWKxGQv6goapqV7Tsdhol0TSNWCyGWCyGs2fPolqtGp1j09PTUFW15po6pfIPW7r+INEVhELgpq+8GVZCITMugUAAY2NjHd/UnagYk3rJkydP6qIit1WRzYRHCuHb29sdebZ0kvIipmcXLlzA6OioIa1z6tQpnD59Gms7a/jKX38FkixB0zTs7OzUFOTDfBiSKhlkZo5ciGbWiYETSJfSKJb0tlriRGeIMfr96A/2I11ONzxXBgwqSsWIYLJZCqAAmtFw+/beZyKyIspKuWGkA+idYhRNNazpEAisAJZlmxIPee/NIhOCM9Ez6PPbd0qyLAtBEBAIBHDy5EljytxciCZzHAcVvbwIKa92YFfcTyQS2N7exuLiomE9TYr7TuZ/hFC8CGWfOIjZDuu6ZJNzY8al3Wl5Ui/JZDK2UZGbw4Pm9arVKsbHxyHLcs2wYjtoh1BI59izZ8/wyiuvoLe3t0anjaIo5PN5fOXdr4DjOQyP6KRaKpVQLBYRj8chUiIyvgxogYbf7wfDMEbkQv4X5sOQZL02QQikt7cXkiShWCyiUCiglC4hy2UBAYYgY91nrukF+5K8V/RfX6ewukJjeFhFKKS/bwp6zaJZZADodY5mNR1AH9akQLVEECKj2+aWlXLD9B6BdZDRCrOnu3nKXJZ1KZhkMomFhQVUq1XDAKtVjaxW8CKmvFqFubh/6tSpmmu6uLiISqWCaDRqDFeaay9ehLJPkM2KZdmm3VidgKTS5ubmsL6+buvs2AnaSVGZi+BOk/8HkfKqVqt4++23EQ6HbQc0W0Wr6T1zk8Hdu3cRCARqpCUoisLu7i5+OPlDbNFbGBgYMDYV4ovhZ/1IF9NABYhn49jd3TU8RggphPgQilIRsqJHCubohWVZhMNhDPUOQVEVpHN6BGMeCiTRC8MwCHJBVJQKWNPXIZPRzyka29u4o0IU6Wq66TUgOmHNNn1N0yAyIrLVbEubmo/1IS/nHYv7ZhwLHsOp6Kmmr2+3obMsi/7+fvT390PTNBSLRSQSCezu7mJpaQmCIBjkYn7SbhdH3eVFVH4P4xys15TUAkmBn3xX33vvPZRKpUOfQfnt3/5tfP3rX8fExAR4nkc6ne5ona4gFIKDilBkWTZ8FcbGxlzLD7d6vqSLbHh4GBcvXnS8gd1OeeVyOaRSKZw9e3bf0Vgr0VO5XMaDBw/AMAzu3btXk8Ikr72ysoKlpSVUeisI5uu/NBzNQVIlhP1hlPkyRkIjRrdXqVTC1tYWBErArm8XA5EBaKxWF72oqgqWZiHJEkROhM/vgz+gf+bWttqoLwpJkVClq+jx7aUBs1n9fCMR/T1HhAgUKE2vAc/wEFgBxXJzIU0/60dRKaIHzdOPLMWirJYhKVLTYwHgzvCdpse00uVFUVSdAVYqlUIikah50iYE045971GnvMhnedjnQFGU8XB0/PhxY2QinU7jv/23/4bl5WVEo1F85jOfwU/8xE/g7t27rs++WFGtVvGzP/uzuH//Pv74j/+443W6ilBaNdlqB6lUCtPT0wCAe/fuuRretuJfsry8jMePH+PKlStNvVPcilBI99rGxgYCgQDOnTu37zWbpbwymQwePHiA/v5+XL582Xj6oyjKeF8LCwvY2dnBtevX8P8s/T91axiuhXJRr1VAlysh9ZJQKASW0qPYXD6Hjd0NlColfVbl+ReU4zjQFA2REVGWyshX8zXRC8dxiEQiiEajEGkRyVwSUlpCLpdDLpcz6jfpTBQAEAnrhfCiVISKxp8NTdFgabapXz2gEw/P8i1FG4CekkuVm6fQAN2h8XLf5abHdbKhMwxjtNGSJ21i3/vkyRPwPF9j39toIzzqlBe5L466PVfTNIRCIVy+fBnj4+P49//+3+O73/0ulpaW8NnPfhaKomB1dfVA02C/+Zu/CQD43Oc+t691uopQ3IxQzPLrp06dwuPHj12/cRqdryzLmJ6eRjqdxp07dxCJRJqu5wahmFNOZ8+exe7u7r7WI2hEKETa//z58zhx4kRdvYR01FWrVdy5cwczqRnbNloiZWL1FzHO4flUuqzJOHHsBLKVLGRZRrFYRLFYRDqdBk3T6I/0o8AVMBgdRK6aA0MxNdELsBcJ9YR7kM1l0RPWZwiKxSKy2SwSu0EADHimglxRQjQYbVrnINP8Ti3CBMQZkohXNttUo2K0aYuysTbN4CfP/qStR4oV+51DMT9pj46O1ij8kiFAEr309PTUFZqPOuVlVfk9Kljtf1VVxbVr1/A//sf/MJppXpSaSlcQilnC3o0IhUioJBIJ3Lp1Cz6fD48fP3b9BnZKUZGpe5Zlcf/+/ZblsfdLKGTeg7xuMpnEzs5Ox+uZYVdDIZEQkW3p7++Hoig1ZFIsFjExMQGfz4fbt2+DZmi8s/5O3fpmT3Qf67OVrifSJRQoQ5SR1EvC4TA0TQOncthKbaGcK2N3ZxecwBmbHnlapkCBYzh9RuR5oVvTNGMoMBaLoVTiAYWBT6xgbSOBLXoLvI836i/W+4i08QpM88/az+lpISc1ZOu6qqY27Soj+IkzP4HT0dMtHev2YKNV4bdYLBp1gidPnuiaY6bay1GnvMyR61HC2hhgdmukaRpXr149qlNrG11BKARutA1bN3NRFCFJet7ZbUKx6/La3d3F5ORkW0ODBPshlGQyiYmJCQwODuLy5cvGhu5W15i1hqIoitGxdufOHQSDwToySaVSxrUg2mRTO1N1XVIBPoB85bmlL8XYPo2bp9JDfMhWryvAB1CSSujr60NYCCORTxjRSzKZBMuy8Pv9GIgMoIIKRE7Eys4KZFmGIAjGta9UKFQrFCD7cfxUAD2BIcSzcSMKMiRhnqfHeoO9yFQyxgR7IxDi8bHNO+2IqRbHtNbWfm/kHl4ZfKWlY4GD1/Iy1wlI9JJMJg1/EpqmkUgkjBrNYW/sZCM/akKxk14hpLwf/Pqv/zp+93d/t+Exc3NzuHTp0r5fi6CrCGW/KS8yLDgyMlJT/Db3e7tZ3DJHKGZnycuXL+P48eNtr2cuLrdzk6+urmJ+fh4XL17EiRN7Bkpudo2ZyalcLmN8fFwfnLt7FxzH1XVyra+vG+dEroWmaXh77e2adXmG182jnndEBbhAHVn4Of+ez4gGVJT6TkAiWa9BM+TiSb0kEolAVVWUSiUwEoPljWUoioIQF0JBKWBoaAiCIBjXPpsFUI7C15MCTSsoy2UIggBBENDT02Ok2UqlEoqZIuJ0HKJPhBbSAM75iTcshFsmHpERUZErRhqwGS71XsKPn/zxpseZcZjSK+bo5fz58yiVSnj//fdRKBTw/vvv1/w9FosdysDjYXV4tXsehULBlS6vX/u1X8Mv/MIvNDzmzJkz+34dM7qCUMhN3WlRvpmECtnk3O4gIxGKoiiYnp5GMplsuV5iB3JTtdobr6oq5ufnsbm5aTus6LbkvL7ZZvH++++jt7cXV69eralLkCjm0aNHWFtbw40bN2rOaSm1hHgxbvxMUzRoijbSWzRoFOXa+oOZKIDnqTFLBEODBkMxxjpBPlh/DE1jMDqIvJTHSHAE25vbqGgVcByHjY0NcNxeakwpDgK+NMJhDWEhjFw1Z5CpeaiyL9YHVVWRK+agVTWsba9BUZQatV+yMQa4gCOJWDd1lmahQYOsyoZgZCMMh4bxyfOfbJscjrIoTpSQz549i2g0atRelpeXMTMzY7gruiXAaIejruEQWB903XJrJG3Kh4muIBQCskG38+RECr6FQqGhYu5BGWKVy2W88847YFkWY2Nj+7ITbYdQqtUqJiYmUK1Wcf/+fdtWaLddFhVFwbvvvouzZ8/i1KlTdcV30ohQKBRw586dukLiW2tv1fwc4Gp90oN8rfS8lSgA2G6wVr91u3qDj/WhKBdRrVaxubmJ/lA/fDGf8b7IUGU6nsbs4wqAfgQCMhQwtm3JRPerqlQhiiJC4RD8Eb8xVGlOs0WCEUg+CSzPws/7G0YnZsFHIrPSCGEhjJ+59DMtp8XMOGpxSEJoxD2RPHyUy2Wjc8wswEiK+25FL91EKObzKBaLh16EX1lZQTKZxMrKChRFwcTEBADg3LlzbUVLXUUohKVbTU3lcjk8ePAAwWAQ9+/fb3ijHQShVKtV7O7u4vjx423XS+xgJpRGIO87HA7j5s2bjtfKzTbk1dVVAMDLL7+MgYEBwwuGkAlJg3Echzt37tR9FqvZVaxl14yfrUq9FCiUlXLNvwkItekeP+ev66CyalrZbcIMzUDRFBSKBWxtbSEWjSHcE4as6cRD7Hd7Ij1QNRVT89Lz16MwvzRf05bM8zygPbf0Levnz1O8/t/UXpMASbNVK1VIZQlbO1tQVRW9wV5ovOY4bW6WVWm22QusgJ+7/HNGd1m76AZCsfvOiKJY466YyWSQSCTw9OlTzM7OIhwOGwRDvOE7gbV2cRQgDyhORfnDwmc+8xl8/vOfN36+ceMGAODb3/42PvrRj7a8TlcQirnLC9Db6JoRSrsSKm63JD979gzxeByxWAxXrlxxZV3yHhqRwPb2Nh4+fIhTp07h3LlzDd+3GykvczoPAPr6+uqK75lMBhMTE+jv73ckVnN0EuTq/T6s5lNhvl4a3toKS7zhG4ECBZERsZXcQjweR39/P0Z6R+rqNOZoqFDQ772To2GcOHGipi2ZoigMRAZQ4ArGEJ/IiahWqjXpP3K+veFeFHwFhGIhQAEy+QyKeX3ynNzj5XIZoigi6osa2l8iKzbsAqMpGj914afQH+g8pXHUhNJKl5dZgPHcuXPGgHIikcDKygpomjYil3bl47shQiF70lGLQ37uc5/b9wwK0CWEQkDC30Z1FHPdoB0JFbcIxdySPDIy4qr2lnkI0AqzqOS1a9cwNDTUdL39prwqlQoePHgAALh58ybefvttSJIEhmEMMtna2sLs7CzOnj2LEydO2G5Q24VtPE4+BqAr65o1s/Q3h5q0lp3roo/11aSKRFasq7f4WX9ddBLiQ3i69RSZTAZDQ0MI+AP1RX3teTT0PG2WyQCo+iCEc7ZtydvpbRRTupRL0BdE1peFz+8zCIKkxkJCyCAIiqIQESPQaM2IXsi0fjweh4/2YZffhc+vtyVzAlfjQ2/F3z39d3E2dtbx763gqAmlkw1dFEUMDw9jeHi4Rj5+ZWXFiF4IwYTD4YbvrxsJhdgMvIhujUCXEQrQuHW4XC5jYmICiqI41g2c4AahlEoljI+PGzbF6+vryGabi/q1AztCIS266XS6Lan9/aS8stksHjx4gFgshqtXr0JRFAiCgO9973vo7e1FX18fCoUCVldXce3atYbE/s7aO9CgGRGG1e8jIASMp3FbwgHAMizw/DmDoRmomlo3ZW4XwSytLaFUKmFkZAQ8z9sW7MNibTSUyVKAyhuyK8Z5cgGUlbJRLJYkCazMYiu9hUQyYbQlk9bkXDVn1F9o0MhVcsY5k6HARCKBi6cvIlvKIl/II5vNIrWbwg6/Y5CLtS53e/g2Xj32quP1bhXdQCj7lQMyy8dXKhVDG2t1dVX30zHVXqzRy1E7VgKoUZMgKBaLh2IfcBDoCkIx31ROG38qlcLExITRXdTujbBfQkkkEpicnKyZ83BbzBGoJwFCYgzDtDUkCXTusrizs4PJyUmcOXMGp0+fNjaej3zkI8jlctjZ2cHCwgJkWUYoFEI+n4coirbdOOlyGrO7s4AG54L081N0Ihye4Y05FZLCskYiAiMgL5kiGFrE4rNFqKqKkZGRvfqcpahvlYKXZaCQFgAhC3OzHpmsN5OYwAugeArHfMeMtuRisYh8Mo/trW2IPtEgmFgghnQ5DVrbixplWQZHcajIFbAci2g0ilgshiAbxFZ6C8ViEZlMBhRFGWKW14av4e+c+juOn107OEpCIVGcmxGCIAg10Us2m0UikagxvyIPA+FwuCvahu3qOF6E4iKs0/KkXvHo0SPDh7yTL0GnhGJ+fasp1UF1jpE1U6kUxsfHMTAwgCtXrrR985OUV6sbh1l77Nq1axgcHKyrl4iiiFQqhUAggEuXLiGbzWJ3d9fQcerr60N/f7+hQvvO+jtQNRUR0d4Qys89T1M1IByREY2UGJnlsEJgBSOVRas0nqw+AcVSGB4eNq6btahvlyLLZgHIPrBiBUTln4au0WWNnMze78T9sCfcA1mVUaqUDAn9ZCKJXX4XvI83DMCq1Sp247uIRCKQNdkgGYZiUJT1Lp9gULcjrlQq+iBgkUbPdg8eVB4YUeJ+BgKPmlCAg5M9oWka0WgU0WjUML8inWNra3pziCAIYBgGlUplX92Z+4GVUKrVKiRJ8gjFLZhTXrIsY2ZmBslkErdu3UIsFut43U42f6uEi/X12/VDaQUkQllbW8Pc3BwuXLjgWJtoZS2gtdBeVVVMT08jkUjgzp07hue7mUxyuRwmJiYQjUZx5coVMAyDcDhsTEKnUins7u5ibm4O1WoV/ogfb+6+id5Qr2PxnEQlToTD0qwReYR5ezJhqT1BRqkiYXd7F7xfFyk0XzcKe//N0Zytz3s+ywB8HpEIQFGoq6+Y17ISDEM/H6DVlBpv9wAb0OsuxSK2t7d1klc19IX74I/Uer1YGw0oioIgCOgL9+EXXv4F8Bpf01LLsmzNQGA7g7tHSSiHLXvC8zyOHTuGY8eOGfNUS0tLKBaL+P73v49gMFgTvRxW5GJn/wvg0Lu83EJXEIpdyov4rXMct+/5DvO6rYKkmiiKMiRcrHBbbh7Qr8WzZ8+QTqdx8+bNfUkwkOvaLO1VrVbx4MEDaJqG+/fvg+f5usn3eDyOqakpw22xzmLXpEJ78eJFFAoFvDH9Boq5IrZ3tsHxe4ODxORKZEXkq3mE+JAj4QS4ADKVjD4t7yCQGOD1Ywr5AnLJHHxhHyKRSM05mu11aYoGx3C2Io7lbAigCwgT2XoHorPKvzil4qABsiYbBlb5fB47OzvoD/cjU8lgd2XP6yXgD0Dm9A5HMo8F6IT5U+d+Cj5a1xA7duyY0VJLBgKJnEk0GkVfX19LRlhHqaVlHoY9bFAUhUgkgnA4bBhgkc6xqakpaJpmWPf29vYeaPRi59YIwKuhuAXiCzA/P18nobIftLP5E12sZqkmtyOUarWKcrncUdOBHVqZayEzLZFIBC+99BKAWhVWkvIjEvytdJdRFAVO5LBFbWH4+DBK1ZLReru5uWkUpAciA+B9vOPwHk3RKFQLddPy1mPy1TwymQzKmTJCvSHblktzUd86UGleK57Wo45IRHMmOg2oqrXilda2Z4IgHzQiLNKRdH70PBROQQghw+ulWCwivZPGprpZY4PM0Ax++tJPYyQ8YqgyGOdrSuucP3/eMMIiBGM2wrKz8f0gp7xaAYncrda9uVwOiUQCGxsbmJ+f12eUnhf3I5GIq+fsZP971LWdTtE1hEJmJgqFAgqFAl5++WUcO3bMtfVbUTI2S963Uq9xsyifz+fx4MEDUBSFs2fPuvKE0oxQdnZ2jJmWM2fO1DjYkc9jfn4e8Xgcr776aluSMuNb46BAoapUa/xMNE1DuVyGVJawsbOhzxwJrO7U+NzPhCDIB5Gv5MFRnK36MKB3cj3ZeAK1pCIyELGNJHl6r6hvHaisWYsPIpktAKDRF66vrxjHCbVaXI3W1KCnspLJJHK5HM6dOAeV2fs8zNdGZESk8ikUi0WkUins7OzgR0Z+BGJBRNVXRSAQMD4jEsGQe5qkxkZGRgwpeWKENT8/D0mSEIvF0NfXh56eHvh8vg9VysvpHKwDuBRFGW3ip0+fhiRJRufY9PQ0VFU1Zl56e3tt77d2YJfyelH95IEuIpRqtYrJyUmUy2UMDw+7SiYAjOKbExRFwezsLOLxeMv1GreK8mRjP3HihDE85wacBiXNQpYvvfQShoaG6uolZg+Tu3fvtvXFUVQFM7szNV1X5nPy+XwYjA7q9rzFtFG8TiQShqZWIBAAS7F1sirW9/Fk7Qk0SUPPsR7H+gEZPDQX0esX00UnMxkAMo9gpOpofmX+faM1/azeZBCPx1EqlXB69DTA2qcgyYQ/kcfv7e3FSz0v4WboJnZ3d/H48WOj6YGQAiF98j9r9NLT04O+vj5cuHDBuL7b29tYXFyE3+83OqFEUTz0J+JumQFpVlvkOK4uekkmk9ja2jKuI2lLjkajbb8na33zKGRX3ERXEIqqqnj33Xfh9/tdHxYkaLT5E9kQABgbG2t589xvhGLuqnrppZdw7NgxvP/++677ypuvp6qqBnE6Fd8LhQImJiYQCARw+/btthWap+JT2MpvOf6doRlQoFCQCnWKwCT9k41nkdWy0ATNSAGZv3iyLCO3q891DI0MObopMjSDfDWva3k1ML4iopPZNA+oDIJh+2P97F6nWLM1oQFbW1tQFAUnR0+C45wjLetDxJnYGXzy8idBU3RNxLG7u4uFhQVUKhXEYjH09/ejr6/PUEsm5GKNXnw+H0ZHR3Hy5EnIsoxEIoGZmRk8evQICwsLxhP3QdcMjEtzxF4oQPukZo5eTp06BUmSkEwmkUwmMTs7C0VRjNoLiQKbwWqulc/n27JR7jZ0BaHQNI1XXnkFgUAAjx8/RrHY3I+7XTjVO0hrbn9/v9G51M6anUYoRNIklUrVKBS7PdtiXq9arWJ8fByKouDevXsQBKGu+J5MJg0LgPPnz7d9Y9tJ1FsRYkNIV9K25xoMBhEMBBEVo9jObKNQLCCTydR4kPA8j934LmKhGIYHhlFS6gchCYJcEAWpAEVTGtrtKqoCTQVyiSDA5+qGGglIJxdLsQ3XZMBg8dkiaJrG8ZHjCPDOQo+kOYGg39+Pn7740zWDmlbr3WKxiHg8bswE+f1+4+/RaFR/T8/11uyiF9Lscfv2baOldmNjAwsLC3UdTwel9HvUm+Z+oySO4zA4OIjBwUFommYoH5AokESapIZl91rWCMUt6fqjQlcQCgCEw2GoquqKyZYd7GooKysrWFhY6Lg1l2zW7eaiy+UyHjx4AJqm64YV3SYUMtyYz+fx/vvvIxwO46WXXqqR86coChRFYW1tDQsLC7h06RJGRkY6er35xDySpaTj332czzYVZgaJFgRRgCDWepCQlEOAC4CjOSRyCceUDQ0aRaloqPc6gcyn0FIECpsDRWmwGwMQGAG5ag40RYNn+DrpF4JqtYpcPAeO4zAwMGAruW8GR3MoQ5dZCXAB/NzVn4PAOkcJFEUZplSnTp0yIo7d3V1MTU1BURRjTqWvrw88zxu1F9KeXK3qkZKiKMbMy+nTp1GtVo2Op8nJyZpp897e3g+U0q+b4pAURRm1MPKZkOs4Nzdn1LDIdSTRi1UIl9RQXlR0DaEQHMSwIFmXbNQk7bOzs2PrI9LOmmS9Vm/MdDqN8fFx9PX14erVq3VfqoOIUJLJJJaWlnDy5EmcPXu2TnZe0zQsLCxgc3OzzsOkXTSKTniGh0ALKGnOEQVD6crA1ml5lmVBURQqlQr6+/vR4+/BZmoTxWwRqqoak+Rmq9+goD/pNfOCpykaYT6MufUsABahEGC315HhSacuMUB/WIhvxREIBTDQM4CoGG0oYMnRnBGdcDSH/+vK/4WI0J6fDsuyNU/KuVwOu7u7WF9fx9zcHILBoDFwGg6HUS6XMT09bRCENXoZGBjA0NCQ7vXyfK2VlRXMzc255lPyIqa82gHLshgYGMDAwIChz5VIJLCzs4NHjx7B5/Ohp6cHxWKxRkrJq6G4hP2abDUDISqiB6aqKu7fv99SntMJ5GZs9UlnfX0ds7OzOH/+PE6ePGn7ZXSTUEi6Y3Fx0ajRWOslsixjamoKpVIJd+7c2Vd32XJ6GZv5Tdu/ETXfguwcKVCgEBbCSJVTde8jlUoZAo8DkQHkpTz6+/uNp20Svezu6nMdAX8AYo+IklZquOmJrAhVU1GUi3pBHrBNd7G0Pjxplcs3o1AoYGdnBycHT4L20w3nawh8nA/ZShYUKHzywicxHBpueHwzmPP8Z86cMdJZ8XjcEPrUNA2hUAhXr14Fx3E1nWPkf8BeCjIUCtVoZZl9Sgi59PQ4N0XY4YOQ8moVFEUZc0ikhkU68DKZDNLpNLa3t/Hd73730FNeT58+xW/91m/hb/7mb7C1tYXh4WH843/8j/Hv/t2/a0u5maBrCIXgICOUarWKt99+u2M9MLs1geb+JWRTX19fx40bN9DX1+d4rFvDkqqqGqH25cuXbTu5SqUSJiYmwPM8bt++ve90htVAy4wAH9Cny6vO0UlYCENSpZrfaZpmdEmNjIwg7K/N6ZN2WUEQEIvFjLkOqkJhfnkeoGB0jfl8vroNhAwjqpqKTEb/W9gmQAhwAf0YB4IgEjRDA0PgAzx41nm+hoDM2QDAR099FJf7Ljc8vhOYJ8SJ22YgEIAsy3jzzTcRiUSM1BiRejG3JZvbe1mWxdDQkKGVRYYqnzx5gpmZGUSjUYNgmhWWuyXldRTnwLKs4aZYKpUQiUSwsbGBb37zm5icnEQgEMC/+lf/Cp/4xCfw0Y9+dF8Pvc0wPz8PVVXxR3/0Rzh37hymp6fxS7/0SygUCvjP//k/t73eh4ZQdnd3Ua1WcenSJcfooF2Q2kOj85UkCZOTkyiVSrh3717TcNYNQiFujpIkGdpR1uJ7Op3G5OQkBgYGXBke3cxt4mn6qe3fwkIY2XIWIufcPRcWwqgq1Ro5E0VRsL29bQg8CpwAjuYa1iMYhkF/rB8UKAR6AyiXy0a6QZblmtSYj/eholQMEss8z4xFwrURCk3RUFTFtmaiaRrS6TTS6TSOHTuGwcggynIZklIv62IFaTl+ZfAVjB0fa3jsfkFSrSdPnjSUDsrlMnZ3d2u02EjthcjWmAnGbqiyp6fH8Ign0Yt5LSIJYzdUedSE0g1qw6qqwu/34yMf+Qi+/e1v41d+5Vewu7sLWZbxL/7Fv4Asy1hZWTmwaO7jH/84Pv7xjxs/nzlzBgsLC/jsZz/7wSAUt1Ne5El9c3MTNE3j1KlTrq0NNJ6WJ8OKgUAA9+/fbyklQNM0JElqepwTyGsGg0HcuHEDP/jBD5BKpRAOh8FxHCiKwubmppF661Rs0wqn6CTAB5CtZGsmxq3ws37kKrka50FJkrC1tQWWZTE8PAyGZiCyItDkVBmKgciISFfSRrssecIjqbFCoYDEbgIxfwwqrxqkm83oi1tTXiEuhKJcrCMITdOwu7uLYrGI4eFhiIKIqlrVraEbeJkAzx0q5TJORk7i42c/3vDY/YLI5pDPm0AURRw/ftzQYkun04jH41hcXDTakkn0QuZWnIYqeZ7H8PBwja5bIpHA4uIiqtVqXUH6w5TyagRrurxcLuPll1/Gb/3Wb0HTNGxvbx/6dcpkMh3XUbuGUMyujW5FKJVKBePj41BVFa+++ireffdd16eDnWoe8Xgck5OTGB0dxYULF1p+zf1EKLu7u5iYmMDo6CjOnz9vPNmvra3h2bNnRutiOp3G9evXG6be2kGilMBCYqHu9wIjoCzpG6vT0zqRhTcXp8vlMra2toz2VYrSaysludSQbCno+mDWtBmBWbAxyAYRz8SRK+QMwcZE4jQAGqGQCsJcNEWjqlbr1lRVFTs7O5AkCcPDw+A4DkEuCI3SGnrGE4T4EDiGw89c/hmjFfkgQB4erl692lA2x1wPAfR60O7urkEwPp/PIBcy9NvKUCVpcSZ1HFKQJi3rR7mpdyOhmN0aKYpqSerITSwtLeEP/uAPOopOgC4iFALSNrzfjZ+E+KReQm54t8NcKwGap9CvXr2K4eH2iqydFuVJC/SVK1cwPDxsfFmPHz+O0dFR5PN5TE9Po1AoQNM0LC4uIpVKob+/v05IsV0QA62a90HpqTVFU/RhQJt0EQ0aHM2hKBcRESKoVqpGYTsWixnnRSTrI0LEcTAQeF6DUSTHdl7jOD4MUIAv4IMv4DMk4vMF/b7I5daxvk4h4A/geM9x5KRczfVRFAVbW/rg5vDwsHE/MTRjO19jB5Zh8XNXf06Pug4IKysrWFpa6ujhgbQlkyJyMpnE7u4uZmZmIMuyQT7mtmSnoUoSCZ04ccIoSK+srKBQKODNN9889KFKwN7L/ShgRyhuFOV//dd/Hb/7u7/b8Ji5uTlcunTJ+Hl9fR0f//jH8bM/+7P4pV/6pY5et+sIhVxca392OyDS73bdVG72ngO1EYVZ7t48rNjueu0QCtHb2trawq1btxCNRuuK75VKBTMzM2AYBj/yIz8CmqaNJ8aJiQkAMCaue3t727ru2UoWUztTtb+0eJs4PQUGhABylRwYmkGumjPEE/v7+40vlZ/T02EMxTSsnZDuqxDf2EfCz/qRl/JgqL17QL8/REhV/TwvXhyCJBVAV2ksLC9AoZWaluTt7W1jxoS8t6gYRbqcbvjaBCE+hE+e/yRiYud2DI1A7KJXVlZw8+ZNY9CxU1hbYPN5XU7GLJ5IopdIJFIz8+I0VFkul8FxHE6fPo1EIoH/v73vjo6rurfe907v6tW2XFVsWbKKbewYYwfjCpZMSeg2CeQ9AknIRxIggSQkL+EReIGE4kAgQEggBHeMqXGj2MZWs2XJspqtrhlppJGml3u/P4ZzPF1tJI1g9lqshWeuZs60s8+v7d3Z2Ym6ujooFIoJkZGfTLVjT/juR+FqG77//vuxbdu2kNfMnj2b/n9HRwdWr16N5cuX48UXXxz180YMoXimvAC3JMFICcXTb95X+t2zxTecIDUUT/mWYHL3w8FICMXhcKCyshI2mw2XXXYZpFIpPRl6ephUVFQgLi7OSzmZzC0QX26dTofGxkacOXMGcXFxtAtlqNdxov2E38yIp6Wurxc8vcZDUFEhVKCpowlGoxGpqan0OT1VhpViZUiJ+wHbgNu1MUS6SSwQw87ZA3rWk5ZhmZyHXC6EWjIdLt4FhVVBnRi1Wi096JCaAsuyUIgVcHLDr/tdOetKTFNPG/b1IwGZKdJqtVi8eHHYW1A9B/g825JJuhWAV2Hfd6iSRDF2u52qTpMBTSLE6Ckj79mWPJo21mCIBEIh78t4RCjk9zsctLe3Y/Xq1SgqKsIrr7wypvckYgiFgFjrjnTjt9lsqKyshNPpDCj9zjDMuDksDg4OoqamJiztyMMlFJPJhPLycsjlcixdutTrPSPdZ1qtFtXV1Zg1axZmzpwZdO6F+HITEcGenh50d3fTEyP5cvrKcFgcFlR2V3o9nq+lrqdsPIGXoCIPNLY2wmJztwWT1mWWYd31C5cdLNigLbgSgQQ2pw08eC/XRr/XSR7PaYeN8b9mYODLgryap1GRQqSgTowsy8JoNEKtVkMoFNKZF5VUBYvCAlbKQiwWD5k6XDljJRYkLgh5zWjBcRzOnj0Lg8GAxYsXj2u7KYGvcZXBYEBPTw8uXryIs2fPQq1W0+iXtCX39fWhvb0dM2fO9PrOeg5VEhMsYuFbW1vrZeGrUqnGlKYlv7HJTHn5roEMQE7kHEp7eztWrVqFjIwMPPnkk9DpdPS+0dRvIo5QgJEX5g0GAyoqKhATE4Pi4uKgX5LxIBSn04nGxkZkZmYG3bRHguF4rPT29qKyshLp6enIzMz0G0YjdZympiYsWLAAycnJw35+z9y5w+GghVkiFUPIJS4uDqc6T3nVNDxNrABvL3gCqVAKi8PdGkwEHh2cA+np6Zc+Nx5eKsMqSeABQXKScvJOOngYEB6PF6zbzPBlh1eM0h0VSYVSeh01xUpMpNassbGxYHgGVrMVdosdPfoesCzr5WXie9JbmLQQl8+4PPAaxwiXy4WqqirY7XYsXrx4UixtGYah/ixz5871aktubm6GUCiESqWCXq/H7NmzkZGREbItmQxVzp49GzabjUqZtLS0QCAQ0ML/SIcqAff7RQ5ekwXyen3FISfS/vejjz5CQ0MDGhoaMG2ad9Q8GpHeiCEUzw92JK3DZPp87ty5Q27o4XRYJIVtk8mEadOmYdasWWF53KEilNbWVpw7dw45OTlIT0/3q5eQNmliW+wp6zBSiEQievokg2w6nQ51dXUwWUz4aPAjCKQCtwujWOInlujpBU9eGw8eLt4Fu92Ors4uKOVKpCanen1uni6Jgax2ye1EHh645OwYCJ6PF0yV2GAAwAkQE8PAyTkhE7pP9/39/ejr60NKSopX1MsyLKQiKYQCIZwKJ5TxSpoaCzTzMjt+NjbN3TSct33EcDgc1F20qKgobHpbY4VnWzLHcdSoTSQSobGxEXq9ntZeFAqFX1uy71BlcnIy/S4aDAb09vaiubkZZ8+ehUajodHLcPxEIqXDC/BOu0209Mq2bduGrLWMBBFDKJ4YTiTBcRzq6urQ0dEx5PQ5QbiEJ4lXiNlsplPB4QIhBV/wPI9z586ho6MDRUVFdCrck0zsdjtOnz4Np9OJJUuWjNn8xxOkFTQuLg6ZmZk42ngUqHU7PvboeqCRa8BKWaoGLBKIvCIBBgzkAndB3GKxuGUe4tMgVnmniXzlSpRiZcBiPOn8ArwnzkNdR0QgA2FggAUcMihjDBALxBi0u137BgcHveo6BAqxu6GANAOQegD5LnjOvNgMNkwXTUdTY1NYuuo8YbPZUF5eDqlUiry8vEnvWgoGMjy5YMECpKamwmw20+ilvr4eUqmUpsZIW3Ko6EWj0SA2NhZz586FxWKh0UtzczNEIhGt4QQaqgQiY6iR1E88rbqjasPjgKHcFYkMu8PhGJFVbjgiFFK7kMlkuOyyy1BbWxtWMcdAKS+n04nKyko6bS+TyQJ6mFRUVEClUqGgoGBcfyw8eJzpO0NrLwqhAl19XTCZTOjv7wfLskjWJIMTc9TXnGzsg4OD0Ol07o6gOI3XAKBc6O+SGKjY7UkSQHCTK88IBoCXHLzX6+GBQZ0akLhl6yUCCVo7WmGz2bzqOgSERIj9cCCQmZeUhBTcOv9W8Gbeq3BNTuZjUfA1m80oLy9HTExMSKvqyUZXVxfOnj2LhQsXIikpCYBbDmfGjBm0ldi3LZmks0hbMoCQQ5Wpqak0YieSMPX19bBarV5DlWSvmCzZFU/4khqx/57IlFe4ETGE4pvyCrbxe9ZLioqKRpQ7HWsNxXNwkAwrhrsu45vyIpuGVCrF0qVLvZ6PkElvby9Onz6NadOmYe7cueOeF67WVdMNXS1WY8A+4GXxa7Pa3GKNukFwHIcEVQIGxANwOBwYGBhwCzzGJHltxkJGCAfvLVdCXAw9QQrmBGTi3Bdi1t3RRR7P13PE61pejbYe92Mq5QzqL9bD6XIiLS3N7/vlGUGFdICEe2r/+pzrkaxOBtSgxWbSVdfc3Izq6mrExMRQNeDhmisNDg6ivLwcKSkpIxqcnWi0t7ejrq4u5CxMoLbknp4edHZ24ty5c1AoFF5tyUDooUpCIABoCrKnpwcNDQ2QSqW0+2yy3zNfcy3iAxWNUMKMYBFKR0cHzp49izlz5lA9opE+7mg2f57ncfHiRdTX12P+/PleXiHhrMuQxyOEotfrUVFRgbS0NGRmZnrllcnpqrW1FefPn0dOTs6IhyhHA57ncbztOIAvJVN80lEMwyApJgkDsgHExsVCyAnRO9CLfn0/XC4XxGIxbDYbrFYreMY9vDqUvwiBmL3URkygkqj8NnWWcXcK2p2X6jdigTgg8SjECjS3DYL8FBhnP3ghj7S0NL8TrGcExYChzQXBsGHuBmRoMvzeH1K4JhpYJPXT2NgIiUTilfoJdIru6+tDZWUlZs6cGZZGkPECGaxctGjRsKU8PNuSPT3de3p6UFVVRVuJExMTA7Yl+0YvEokE6enpmD59upfKb2trK01dk+glnCni4cA3QjEajVQuaKoiogiFiNH5Riie9ZJFixYNu7/aF6MhFNKK2dPTg8WLF/sNibEsG1btMUIoZDgzKyuLWsD6Ft/Pnz+Prq4uFBYW0rzzeKNeXw+dWUclU3wn5D03WrFADBfjgtlihlAoRGpqKmw2Gzgbh/MXztN5juSYZJg4k9fG6Jv+CkQSAPwn532GKgG3vItnVEMgFUphdVih1V76W4nSibiEFL9N2jfiUYlVfrMsnlg+bTnyk/OD3k9ArHnJZ6zX66HT6bwm0snmKZFIqC5XZmamX1dOJKG5uRkXLlxAUVHRqAZ8CXw93T3bkqurq73UkkmEPNRQJaljtba2QqPRUIdF4g8fHx8PjUYz7imxYDMokXpAGA4iilAIPDd+opxrt9uHpdY71OOOpN5BtMB4ng86rCgQCGCzBZ59GA2IenFdXR0KCwsRFxdHfyCETBwOB86cOQOr1TpmD5OR4ljbMbDMJckUX5AiOsuwAAdcbL1IBR5ZloVEIoFSrITaqobFYgFrZ9HU3kRVV6l/vNijBhSAJAC3va9vC7BnRxcB8RzxhIAVgOM5uHgXWlo4AELApkZsvN3vB+1HZjxg54JLwOQk5GBVxqqg9weDQCCgbdmeRlmtra2oqamBVCqF1WrFnDlzRu2oOd7geR6NjY1oa2tDcXFxWOsBgdqSSfRy4cIFL5tkovgQzOuFDE5Pnz6dtsiTwn51dTV4nveShAnnUCWBbx2HuDVGCSXMIG3DAwMDKC8vh0ajQWFh4ailWAiGKvZ7gtRq4uLiQg4rjpSkQsHpdKKmpgYAsGTJEigUioAeJhUVFZBKpWHxMBkJWgwtaBtsg0ocxNKWvxQxCFwC1LfUewk8ApdqGSzLIjkmGUaHETNiZsBud+t4GQwG9PX0oVvcDbnCTS4JqoSAtQrfFmC1WO1HJizr3wHGgKE+KCaTCe3tYoAHGJENLMv4XSsXeZNZKOXkdFU6NmduHvOm4GuU1dTUhKamJmg0GjQ3N6O1tZXWXeLi4ia9Ywnwn9If7/ZXqVSK9PR0pKen07Z2kjo8c+aMn1oyIRen00mN2JxOJx2qTExM9HK97O3tpa6XZKiSRELh2PR95aXMZvOEHg7HAxFFKCTlJRAI0N/fjxMnTmD27NmYPXt2WD7A4W7+nZ2dqK6untDZFlJ8JwRB1FjJcxAPk8rKSlqInegulc/bPveSTPEF8U5n7AzOt5/3EngkIP7pEoGEzpd4mmTFxcVBzsrd9r5mM6wGt/KwQqGgMvMMw/i1AMtF/vUcAFAK/WVWSBRFJrEFgtmAXQmxOnR7MoFvmo8gRhqDG3JugJAN38+K6HK1trZSrTaO49DX14eenh7U1dXBZrN5dUVNRg6e53nU1tZCr9dP2JS+J3zb2j3bkhsaGiCRSOgQZFtbG1wuF3Jzc2mKOdBQpVKpxKxZs6i8DBkoZhjGSxJmtIe6aIQyAeA4Dr29vejv70dhYeGo6yWBMFR6iud51NfXo6WlZdi1mnBEKH19faioqEBKSgrmzJmDQ4cO0YlZ8oXr6OhAbW0tMjMzvTwtJgrdpm50mbpgtgcvnLt4FxwmB9q0bV4CjwRigRhG+5fCjAz8NMAAd7eXlbdCo9EgNSEVJruJznN0d3cDcNcdkmKSwAndRc1AxXog8FCkWqKGwWrwshT+znd4yEU2DFic/tf6kEmg1mbA7Tf/rfnfgkIcvlO554m/uLiYvp+kFhAfH083T51O5yeXQ7qixnuDInXGgYEBFBcXT3hxOxA825I9a1NnzpwBx3GIj4+n7evEnyVYW7JAIPAaqiQHkYsXL6KmpgZqtZp+HiOpgYyXjtdkIqIIhcyXmM1mxMTEhJVMgNBFeafTiaqqKphMJlx22WXD/mDHGqGQSf/MzEz65U9MTMTJkyfd6Z6EBNjtdmi1WixatMhL8HIicbLjJKwOa9DTuUKkQKe2E/0D/QEHAQG3NIvD5XALRgZJGSnE7ol3ESuCw+UAwzBUDobIzDutTrRr22F32KGQKaBRaSCQCPxOir5DkUqREgargZpipaenQywW02jHM6OqFCkDFvIDeZewDIvrsq9Dojx831fPTTrUid/z/fEUWPRUkg7HzEuodZ4+fRoWiwWLFy8el1rDWEFkWlpaWqBUKpGVlYX+/n50dXVRAh5JW7JarUZMTAzmzJlD6ziEYIRCoZdTZag0fZRQxhlnzpyBQCDA3Llz0dHREfbHD0YoZChQIpFg2bJlI/rRjTZC8YyGCgoKEB8fT09I+fn5cLlc0Ol0aGhooFLfXV1d4DhuwnPmfZY+nO89HzCiANw/vs6OTphtZqSmpQZ8/4j8fKjuKDIoyIKFkBX6RRfEW0OtVkMZo3SbbdkBnUFH3yOiXiuRSALqjHV1dcHpdCI9PZ3+2H0HHqVCKSxOix95kgjLF+vnrMesmPBI7wDeulzFxcUj0uXy7Ioip2nfmRcSvYy1xkHW6XA4UFxcHDGSL75wuVyorKykRntCoRAxMTFeBOzbluw5VBnK64XIE3nWcXp7e9HY2AiLxYKYmBivoUpfKw3Pz9ZkMkVrKOFEXl4eWJaFTqcLaysuQSBC8RVaHGldYjSGWE6nE6dPn4bRaMRll10GuVzuV3x3Op24ePEipFIplixZApPJRHW0bDYbbX9MTEwc91PhiY4TAfW0yGvp0/ZBKBAiMTUxKNEpxAqAR8hWW6VYiQHrAK3FBIKQEdLoJkGZAIPNgDSF21CMaGl1dXVBykrBSt1KwWqFGjaHDW0dbQC8TbF8Bx6FjJB2f/lCJpT5tSkvTV+KgpSCoK9ppAinLhfxffedefF0TiSFfeLmOVw4nU5q1zDSAeOJBFGZAICCggK/dfq2JQ8MDNDOOqKWTMhFrVZ7kUug6CUmJgZxcXGYN28eHars7e1FU1MTxGIxjRSJb1E0QhlHiMVi+iaHWxUY8CYUnufR0tJChwJH29M/0rVaLBZafF+6dClEIpFf8X1gYACVlZWIj49HTk4ObbclBUfiakhmVTQaDSWXcHfWmOwmnO4+HfA+u92Ozs5OJCoTIYuTBc0ds3ArIIfyKiH1DrVEHZJ0SErMt74hEAhoIZXnebAuFj2GHvTp+9Cv64eNs0EkEiE5OdnrR0yaBACEHLAkJmCeyIrPwpUzrwy61pGCeOrIZDIsXLgw7FGo58yLp9wJqSsQHxxPuZNAcDgc9Ducn58fER1mgUBIj2GYYckRMQwDjUYDjUaDOXPmwGaz0fRhS0sLWJal5EL8WYY7VOlyuehQZV1dHex2O12PxWKBTCaD0WiMEko4MR6+8p4gj8txHGpqamixcyxDgSOJUPr7+1FeXo6kpCTk5OTQLyN5HADo7u7G2bNnqby37ybNMAzdOGfPnk0lwrVaLRoaGiCXyym5hKMg+0XHFwE92onAY3p8OiRqSdDaCuAubhsdxpDXqCTuobRQZEJSYr7yK75QiBQwM2bEx8VDHCfG+YvnaWqhra3NXTeRy6FRaWDElyTnI5nvC6XI2+ArVZmKksySsBW8J1qXy1fuhGiskZkXcjInzRXkdRIxSrlcjoULF066HlYwEDJhWRaLFi0aFelJJBKkpaUhLS0tYFsykcwh6cOhhipJJx7P8zCbzaiqqoLRaMSrr76K5557DrNmzUJSUhLsdvuE1aI2b96MyspKaru9Zs0aPP7446NW3WD40YjejxNIj7jRaMSxY8dw1VVXhfXxydCSRCIBx3EoLCwcc0fK4OAgTpw4gTVr1oS8jsjGzJs3DzNmzKBfPhKVEA+T5uZm5ObmUhG9kcDpdNITVU9PDxiGQUJCApKSkkZVd7E5bXjm1DOwOb0748jmMz1lOuJj40PqWbEM69UiHBD8JdLx1PLyhVqihtVpBcdzIR0SlWIljHYjhE4h6lvrvdqXXS4XzGYzzGYzGDsDs8stF54SkwKHwBFwg2ThHmwkz6mWqLEtbxtUkvAM7UWaLpfNZqMtt729vRAKhfSAQmZhFixYELFkQtKGQqFw3CIoT8kcvV5P01kkeiG/ac/OMQLymy8vL0dGRgZEIhHeeecdvPrqqzh37hwYhsFVV12FLVu24NZbbw372j3x1FNPYdmyZUhNTUV7ezt+8pOfAAA+//zzUT1eRBKKxWLBkSNHsG7durD+uNrb23HmzBmkpKSELaVgMpnw6aefYt26dQHv53keDQ0NuHjxIhXICySjUlNTA71ej0WLFo3Jw4TA079Ep9ONqu5yrO0YDl446PVaSLvttNRp7toEZwtOAjwQJ4uD3qoP+TwqkQo2zuYvo+IBBgwkQneUEUiTi0AicLs28lYezR3NXqZYnmAZFuABo9kIxsFAZ3DX7Tyn9Um+3XP2RiwQ4/a825GsGL5pWShEui4XmXnp7OxEV1cXANDvUUJCQkS0CHtiMtJxpC2ZHObsdrvfXJBvW7LL5UJZWRnmzp2LhIQEMAyD2267DcuXL8fatWtx4MABGAwGPPHEE+O+fk/s27cPpaWlsNlso6rfRVTKi4D8kH0nSccCIqHNsizy8/PD9sMVCAQ0f+r7mC6XC6dPn8bAwACWLl0acPLdbrejqqoKHMdh6dKlYXPa8x30IkX94dZdnJwTX3R8Qf/N8zx0Oh0sFgumTZsGjUwDkUAEiy145KGRakJHJnBv7By4kGQCuLWzOHAh6zCAW3PL0G9AV0+XnymWJ4ivfIImAVanFVK1FA6HAyaTiUqeiMViKOQKQA3wAh4CVoAtWVvCRiZTQZeL1O/0ej2mTZuG9PR09Pb2eikBT+TMSyg4HA6UlZVBIpEgPz9/wiIoT8mcrKwsPxtt0v6fkJBAB1Pr6uogFAqhVqtpaqyxsRHFxcUoLCxEYWHhhKzdE3q9Hv/85z+xfPnyUTeDRBSheNZQgPAQimeEkJOTg5qamrB+6cmX1netVqsV5eXlEAgEuOyyywIW341GIyorK6FWq8fsRR8KnnWXWbNm0bqLTqdDY2MjpFIpkpKSvOouZ7Rn6ObtcrnQ3d0NjuOQnp6OWEUsTHZTUP92wE0ALs41JKFoJBr0WftCvwAeYFgGRmtoMhExInRoO2AYMCA1LTUoOTNgYHFZIBa4ByI5nqO+GmKxmJqXmc1mwAY0tzaDZVmsnb0WMVxMWNz+Ojs7UVNTg9zc3BFZNE80BgcHUVZWhmnTpmHOnDlUDZi03JK0Dyl+k7oL0dKaKNjtdi+jsclKx3n+1sh75Nn84HK5IBKJaMpdqVSC4zi8/vrraGho8BOfnQg88MADePbZZ2E2m3HZZZdh//79o36siEp5cRznni0A8MEHH2DFihVj6lrybM8lWmCHDx/G2rVrw/aFc7lc+Oijj/DNb36TppEMBgPKy8uRkJCA+fPn0wgGAPWxJl+wGTNmhE1aZjQIVHeJj4/HgZ4DsLE2uFwudHV1UQvWGFkMBmwD1AclEGRCGWwum5/HvC/UYvWwoo5Qci8EPM/DordgwDKA+OTQA3ykXiNmA0vae4KYdGWrspErzUVPTw8cDodX2mekBVQi656fnz9pg6rDQX9/PyoqKjBr1izMnDkz5LXElpccVMxmM9XSIj4v4wW73Y6ysrKIbxTgOA5VVVUwGAyQSqXYs2cPPvzwQ8yfPx/vvfce9uzZEzR1PhI8+OCDePzxx0NeU1tbi+zsbACgdaCLFy/i0UcfhUajwf79+0e1J0VUhOKJsdr1kq4ZiUSCyy67jArBAeF1ayOPQzq9PHXAMjIyaL86iUoA94ZCvFVSU1PDso7RghBFcnIyrbscbzyOxo5G+n4Re1aVRIVBm1tJOFjkIWSFcPEuSIXSkGSiFCth5+xDb+giecgCPOD+PLXdWkgYSchZGAK7y07tiId6brPDjMyETFyXcx1YhqUGUL4dUZ7pw2A/RE8l3rHKuo839Ho9KisrMW/evGFJ/RBjq9jYWDqD4WnxO5aZl1CYKmRCLLyJEodUKkVGRgZcLhfeeecdCAQC3HbbbdiwYQNuvvnmMRHL/fffP6RP/OzZs+n/k3RcZmYmcnJyMH36dBw/fhzLli0b8XNHFKF4/hDH0jpMhhXT0tKQlZVFv2Se6alwTfUSpVKn04mGhgY0NzcjPz8fiYmJAYvvRJupqKhoUsLbUCB1l66WLsTFxUGr1VK74c7WTvRIeiCVS5ESmwIr508EDBj3HIfDDIkgeC1IKpTC4rBALpaHJBSxQEwn7IPB6XSis7MTsdJYyOPlQ24oSpESAlbgp9EVCCzDIlmRjC1ZW+g0vacB1OzZs2Gz2WjjQ1NTEzXI8t04yYai0+m8dLkiEaS2k52dPer20UAWv75aWr4WvyOFzWZDWVkZlEolFXqMRJDPXq/Xe2mdnTx5Eq+++ir++c9/4uqrr8axY8fw7rvv4uzZs2MiFPL9Gw3IwXi0lhwRRSieIBL2I0VLSwvq6uoCDiuSjT3cMy4Mw+DcuXN08j1Q8Z14mNhsNixZsiRiXdma+ppwrt395U9KSoJSqQTLsGA4Bn2DfTCZTKjvrwfP8lRDSiKRuDfaL90TJQJJUBIgPiQCVgCjLXiEwDKsOyIIMbtCBytVidAkaALOy/hCJBANXbOBm/QYMPjW/G9BLAi+4UkkEkybNg3Tpk0LKkKYkJAArVYLk8k0KUq8I0F3dzeqq6uxYMECpKSkhOUxfWdeyDR6S0uLV4SXkJAwbHFFQiYqlSqiW5h5nsf58+fR09PjRSYHDhzAd7/7Xbz66qsoLS0FAFx++eW4/PLLJ2xtJ06cwMmTJ7FixQrExsaisbERjzzyCObMmTOq6ASIYEIZaYTCcRxqa2vR3d0dclgxnP4lgLv4znEcrFYrTa35Ft/NZjMqKyshk8mwePHiiJWp4Hkeu07tQl9f3yWBRw9zK7VajfSEdBisBjrLQVpJkzXJMIlNkMlkkIglAQv2nj4kGokmeGfXl8/p5JxB6ytksDIlLgVJiUkhoxgCpUiJfmv/sN4LmVCGLdlboJYMv4Xb1yBrYGAAWq0W586dg8vlgkajgVarDYuO1nigo6MD586dQ15eXtiFWQl8p9FJg0hPT4+XPAmZ5wiUvrRarSgrK6PzMJHWak1A9PrInkQOEh9//DG2bduGv/71r7jhhhsmbX1yuRy7du3Cr371K5hMJqSmpmL9+vV4+OGHR91tGlE7m+cXYyQRClEpdjqdWLZsWcgT4EhMtoYCMQBjWRZZWVlenVyk+N7X14eqqiqkpqZGxNBaMLhcLhw8eRD1unqkp6fTlKCvA6KLc3n5RfA8D4FLAJ1BB1OvCXABOpkOMrkMCoXCa0MgwpBDpbHIc5JBRl8Q6ZnUxFQoVIphRSYyoWzIiIdAxIpw5awrkaocfX2LYRjI5XL09fVBrVYjOzsbfX19XjpanooGk33Cbm1tRX19/YQ3CkilUq8Ij/i8nDt3js5zeM68EDIhigKR+nsi9bLOzk4UFxfTpoQjR47g5ptvxnPPPYebbrppUte4cOFCHDx4cOgLR4CIIhRPDDdCIZt6TEzMsETqwhWhdHV14cyZM5gzZw417HG5XLSmAlzyMMnKyorYOQPAnT6orKxEeU850tPTKQmoxCovMvE1tQLcqSEn66RmQzJGhs6+TjrLIZFIoFAokBqbSrvCFGJF0K4t4roYzAfeYDBAr9cjNTkVGpUGQoHQz5HRF0JGCJZhhxXFAMAVGVcgKz5rWNcGQyBdLqVSSXW0PBVuAUxauy0AqtBQWFg4qXU9TwtfMs+h0+nozItcLofNZkNMTAxycnIilkwAoKmpCe3t7SguLqbR6KeffopvfetbeOqpp3D77bdH9PpHi4gjFE/XxqEIhWzqI3F1HKtOGHHQa2pqQl5eHpKSktDV1YWLFy/C5XLRqdeGhga0tbWhoKAAcXFxo36+8YbRaHSrxsoBp8bpNr9CYCMpP6910uTwpTKvQCCAg3fQbh+n0wmz2QzeyqOmsQYCoXtTdSqdEIgEfo/n6booF8m9yIxM6Q8MDCAtNQ1x6jiY7KaA/iReawQLsdBd3B9OdFKYUoil6UuHvC4UzGYzysrKEBcXR8U9PeHZWcfzPAwGA50JIta1JHoZz3qLpxtkUVFRWBQawgXf2SlycBSJROjv78cnn3zi5x8fKSAWzZ5kcuLECdxwww147LHHcOedd34lyQSIsDkUwJ2+4nkeZ8+ehUgkQmZmpt81JJxsbm5GXl7eiAbDjh8/joyMjFG167pcLlRXV6Ovrw+FhYVQqVRwuVwYHByEVquFVquF2Wym6aK8vLwxCU+ON/R6PaqqqjB9+nTUcrU4rXWrCgsZIRiWgcN1KZUkF8q9VHgZMFCIFV41jkDzIhKBBE7OCYfLQfWzdAM692N+6V0ik8kgFUmpRhfLsmB4hhKV55R+amoq9Zj3XZMfeLfopNlpdgv2hdAJA4BZMbNw44Ib/fxRRgKiy5Wamop58+aNeOMg7bY6nQ59fX3jNolO8vudnZ0oKiqK6K4zi8WCU6dOISEhAdnZ2eB5ngo1ErM0QsLEP36ycOHCBVy4cAFFRUVU8qesrAybN2/GL3/5S9x3331fWTIBIphQzp07B57nkZOT43W/0+nEmTNnMDAwQDf1keDkyZNITU0dcQrKZrOhoqICPM+joKAAYrHYb8aETMdzHAeRSITBwUGo1Wo6hR5JhViSjsvOzoYqQYXnTz0PF+8Cy7CQCqV+qS2V2NujRCPxrq2wDAsBI/CqZwgYAYQCIRWXZMBAJBDB5rTBarXCZHLb+3IuDmqFGkKpEAqFArHyWEpMHMehu7sbTqcTqampiJPH0dSZUqQMOUtC1ui71kBIlCdia95Wqhc2GoRbl8vTfbGnp4fKp5PU2GiVFcjvq6enB4WFhRH1vfQFifaIrEmg99SXhH2lTiaqPnXx4kU0NTV5RXtVVVXYuHEjHnzwQfzsZz/7SpMJEIGE4nA4wHEc6uvrYbVasXDhQnqf2WxGRUUFRCIRFi1aNKr+9fLycsTHxyMjI2PYf0PkJ2JjY2lXCanDkOK7wWBAZWUlEhMTkZ2dDZZlvWYUent7IZfLKbmo1epJ+XKRNEdLSwvy8vIQHx+Pj5o+orpdwaIMz66tQNf43saAgVws96pv+JKSe0GACCJo+93Rnd1mh1wqh0QugVQqpRtpcnIy1DI1zHYzePB+a/IFWY+vUnAgKEQKbMvfhhhpTNBrhsJ463J5yqfrdG6HSs/U2HBFGokQaX9/P4qKiiK6hdlsNuPUqVNITk4edkOLZ32qp6eHtm4TEh4vWfjW1lY0NDSgsLCQDqyePXsWGzZswI9+9CM8/PDDX3kyASKYUJqbm2EwGLBo0SIA7vRMRUUFUlNT6YY9GlRVVdGhtOFAq9WiqqoKs2fPxqxZs7ykqH09TObMmYMZM2YE/OI4nU66GfT09EAgEFByiY2NnZBTFGmtJqrGKpUKFocFz556FnaXPajEiafMilwo97PHJQONnht8oKhAJpT5Tdj7Xidn5ejq74LRaITFYgHLslCpVIjXxEMgFtC0VSjpF7lIDovDvcahZFtErAi35N6CdHV60GuGAon2JlKXixSsdTodDAYDlEolJReVShXwO8hxHM6cOQOTyYSioqKwCZGOB0wmE8rKypCSkjKq1CEA2rpNfnNGoxEajYZGeaFUDUaCtrY2nD9/3qup4dy5c9iwYQPuuusu/Pa3v/1akAkQgUV5As/ieWtrK86dO4fs7OxhyUAM93FDged5NDc3o7GxEQsXLkRycrLfsCK55sKFC1i4cGHI3n2hUOjl9d3X1wetVovq6mpwHEc3g4SEhHERiXQ4HDh9+jQcDgcWL15MT7SnOk/B7rJDIVIE7KoSsSIaVYgFYjg4h19xWylWekUevm6KgDsK8C3y+13HAy7WBbFYDLvdDrVaDZlMBpvFhvaOdjh5t7y8RqnBAD8ABPiNSgRuL3kePBgwISfxGTC4et7VYyITosu1aNGiCW23JUOlM2fOhN1u93IWJN1SiYmJdJaDKF/bbDYUFxdPmIHTaGAymXDq1CmkpaVh7ty5o96MPWde5s6d6yWKSmZeyG8uNjZ2VL+79vZ2nD9/HgUFBZRM6uvrcfXVV2Pr1q34zW9+87UhEyACIxSn0wmXy4WOjg60tLRArVajs7MzbN1StbW1YBiGCqMFAsdxOHv2LM0xE4lpIlFPpu1J6oCc9kcDzwE4rVYLq9WKuLg4Gr2E44dvsVi8WlhJR4zD5cAzp55xezVwLjh5/7QQiSBYsJAIAxtleUYeCpECZofZj3R8CcUziiBQiVXo7u9Gd3c3YmNjERMT406didyDlaTuAhswYBuATOaedSHeJSzLQsSIaKREjLaCYVXGKnxj+jdG8E5egqcuV0FBQcTocpHDCjmV22w2xMXFwWw2QyAQjNmnfrxhNBpRVlaG9PR0qm48HiAzL+R9stvtXnIww0khdnZ2ora2FosWLaJ7U3NzM9avX49rr70WTz311KTPF000IpZQ2tvbUVNTA7lcjoKCgrB1bpw/fx4OhwMLFiwIeD8ZknS5XCgoKKDujp6Ric1mo/MD+fn5YU0dkKE9nU6HgYEBaDQaKlsxmnw3qe0kJSV56ZoBwMmOk/i4+WOIBYFVdz27rQLWP+BNFBKBJCAx+XZjSQQSuHiXX13DaXaipavFyxSLCFLSNTGsW37eZqFFfZvNBolYggRNAhgJQ0k4VBdYfnI+rp53dcj3Lhg8dbmIBHkkgnRDEdl0l8sFlUpFo+HhypxMFAiZTJs2bUIVuIngJ4leBgYGaAoxISEhYL2zq6sLNTU1XoOgLS0tWLduHTZu3Ijnnnvua0cmQIQSSn9/P06dOgWn04nVq1eHtce8sbERJpMJeXl5fveRlk+NRoPc3NyAxffBwUFUVlbSSd3xdISzWq00T67X66FQKCi5DGczIIXiQP70HM/Rzq5gp3hSfwjVJaUQK2Cym8AybFBi8iQjlmX9ZON5nofNaENnTyeSk5Pp4SFg8T9A7cTpdIJ1sOju74bFYoFQKESC2k0uUqnU733K0GTgpgU3DTnDEggcx6G6uhqDg4MoLCyM6KI28QiRSCTIy8uDy+WiJ/Le3l6IRCJKLhNVxwsG0vgyffp0zJkzZ9LWAbjfN08LZM/uuri4OGolThxYAXcdbd26dVi9ejVeeOGFCXGKjEREHKF0dHTQPn6tVotvfvObYX38CxcuoK+vDwUFBV63a7VanD59GhkZGZgzZ07A4rtOp0N1dTUyMjIwa9asCT3dETMjsiGIRCKaFgvUGkly+wsWLAhYKD6tPY2jF48GJQoizigTyoJOmNMIgAeUksDpJc9urECzKzzPo7e3Fy6LCzFJMTTaCxQRMWAgFAi95mMAb+LhOA4WiwW8lUfPYI97nR7zLokKd3uwTDRyInC5XKiqqoLdbkdhYWFE1yFIC3swJV7PlI9O57Y/HovHy1hAyIR4A0USfLvrLBYLeJ5Heno64uLikJKSgq6uLmzYsAFLly7FK6+88rUlEyACi/IMw2DhwoVQKBTo7OwM++OzLOtVlOd5HhcuXEBDQwNyc3Np0dy3+N7S0oLGxkbMnz8/bCqsI4FIJEJqaipSU1P9VG15nqcnzbi4OKohFExKg+d5nO4+HXI2QyVWwe6yh/Q0IQOAvnpfnpAILxGKbxGe4zhotVowTgZxKXE0ty8TBTbm8i3+k9s8oxiWZZGgSYBFYYEiTgGr1Qqz2Yze3l4IOAFWZq6EXqtHQkLCiFKVDocDFRUVYFkWxcXFETWZ7QuLxULb3IPpXXnKnGRnZ8NoNEKr1VKPF2IRTYQsx+vwNDAwgLKyMsycOROzZs0al+cYCzyttGNjY1FVVYW0tDRYLBZcccUVYBgGEokE06dPx4svvvi1JhMgAiMUl8sFp9MJi8WCI0eOYN26dWH9Mnd0dKC1tRVLly6lPfk6nY4WVgN5mJB8+aJFiyKm+EpApDtIUd9isUAgEGD27NleIo+eaNA34O3at4NOjjNgIBPJ4OScQRWBSeQRqn1XyAjBgQPHc37pK+IECQDzZsyD2eWudYhYEcDALwoBD0hFUq9UGXGG9H0dgaIbASNA6exSSKwSmicnsulJSUkhh/vIaZ+YOEXypkHabUnNbDS/Hc9uKL1eH9TjZawgzqbDcYScbPT29qKqqsor4m9sbMTdd9+N7u5uGAwGOJ1OrF+/Hr/85S9DNv18lRFxxyxfX3mO48L6AyYRimfxfdmyZVR23tfD5PTp07Db7Vi6dOmwh8cmEgzDICYmBjKZDHq93j2zER+Prq4u6lFNUmMk3/9Z62chZUiUIiU4cMHl5eGOPARsaNVghVgBg83g1ujyKKwTUyyRSIRpqdNgcbk7xFiwELGigIV035SakHE7Q/q+DrFAHDD1tmneJmQnuX/ks2bN8jPGIq6UZNMk30OTyYTy8vKgulyRBFIDHGu7ra8CMBkU9PR4IdHLaDvGCJmQ+l4kg0gU5eTkUDLp6+vD1q1bkZ6ejoMHD0IgEODkyZN49913I3q+Z7wRcREK8ZXnOA4ffvghVq9eHdYPSKfToaamhjrv5ebmes2mkOK7yWRCZWUlFAoFcnNzIzrFQQQeY2JivMyGLBYL3TT7+vrcwowKJz7SfQSRSBR0wxlKqkTICCFgBQE7tQhIN5aQFXp1fhFTLCKPoZFq3JHLl7pbwQjKs5uMBQupyF8eJtjaL59xOVbOWBn09ZBNU6vVoqfHXXchXVDNzc1IS0sb9XDdRMFgMKCiogIzZswYt/qe56CgTqeDyWRCTEwMJeLhdmISr3oyCBzJ6OvrQ0VFhZd75cDAADZv3oy4uDjs2bNn0g6a27dvx/bt23HhwgUAwIIFC/DLX/4SGzZsmJT1ABFMKADwwQcf4PLLLw+r2FtTUxPOnz+P2bNnY+7cuQGL7+REkp6eHvEbiafAY6i+fYfDAZ1Oh39U/QN1ujoIhUI6w+HZCRUjiUG/rT/kc2okGthctpBDg2qJGkaH0WsuhJhixcTEICYmBkJWCB5u0cZQJOYlmx+CeFiWBXh4RS25ibkoySoJ+Xo8QVptW1tb0d3dDYZh6Ik8MTExIk+fRENsok/7FovFKzUml8tp5OIZ5QVa69y5c8c8pDze6O/vR3l5ObKyspCe7h5+NRqNKC0thUwmw/79+ye1y4940c+bNw88z+O1117DE088gYqKiqBjEeONyD12wz1dHi67XlJYb2hooB+Cb/EdcMso1NXVITs7m36JIhWdnZ2oqakZ1lpFIhEEGgEcKgdmKmbCYnHPcXR3dwNwT14nqhNhE4b2kmbAgOO5ISfQLQ4L5EI5TT8ZjUbodDrEx8dT4TylWOk20hL7T9Z7gkjqA6EbAJQi7wL9NPU0bJq3KeTr8Vs7w1CZnOzsbMTGxkKn01E3Q1J3Cad0x1hAcvvjpSEWCjKZDNOnT/fyeNHpdEE9XshpfzLWOlKQiG/evHn0t2UymXD99ddDJBJh7969k94yfs0113j9+3e/+x22b9+O48ePRwmFwPMHGi53RU974NzcXJw9ezagjEp9fT06Ojoi3sOESL5cvHhxRJIfx9qOAXCf5Il0B8/zsFqtcFgc0Oq0MHWYIJfL6X++9SuNRDNkBKOSqMCAoRs/McXyLH6zDAuTw+QmnRCKwZ7+9KGIh2VYWByXpvhjpbG4IecGCNmRfcUD6XIRT47h1l0mClqtFmfOnMH8+fNHZccQTgzl8aJUKmE0GjFnzpwpQSbl5eWYM2cOjaIsFgtuvPFGuFwuvPfeexE3zOpyufD222/DZDKN2g8+HIg4QvHEWM2wAHfOvrKyEg6Hg77RLpcL9fX1bgVbtZpK4pvNZixZsmRS/RSGgqfAY3Fx8bAlX/qsfajtqfW7nWEYKOQKiFVuEyq9UQ+TyYT+/n7odDpIpVJKPrHy2ND+IwDAuyOKPmsfeJ6HXq/H4ODgJY/6L6EUKWF1WmHn7CEbBEjbsaf5ViAoRUrabSYVSvGt+d+CXDSyz/HixYtobGwMStISicSvWO15IifkMhZp+eGiq6sLZ8+excKFC5GUlDSuzzVSkEaRmJgYzJs3Dx0dHaipqYFMJkNjYyO6urroezVZqtvBQBobZs+eTes7NpsNt9xyCwYHB/Hhhx9GlBHZmTNnsGzZMlitViiVSuzevRvz58+ftPVEXA2F53nY7e7uomPHjmHWrFmjnvsgLZRKpZK2e7pcLvT09KCrqws9PT0QCoXgOA5SqTTih9U8BR4XLVo0omLg+43vo6yzzP+OL4cSHS6Hnxy8w+GAyWRy62fZAYVMAV7CQ6FQBH2fVGIVjHYjOJ7zMsXyup53z5rw4EOmzkjbsYgVhWwAAH+JeASMADcuuBEzY2YO9ZZc+vMx6nKRuguJXoge23jVXdrb21FXV4e8vDw6qR2pICk5UtQeL4+XcIAMWJLBZcB9IL3tttvQ3t6Ojz/+OOIyF3a7HS0tLTAYDNixYwdeeuklHDlyZNJIJeIIBXCfCADgiy++QHp6+qhqGb29vaisrMS0adNovYS8VNLJRQraYrEYDocDDMPQuYS4uLiIahENJvA4HJjsJjx76tmAGzIphoeSeRexIrhcLljMFvQN9sFsNtOivkKhgEQicQ94CdytxEabEd3d3XC5XEhJSfFbq0qsAg8+pHAjWZvRboSQFYb0PvE02rp63tXIT84f6i2hCLcuF8/zXtLynvMu4ai7eKobR9rm5ouenh6cPn0aOTk5AVNyZAqdvFdEyJIU9ieye8poNOLUqVNe0/oOhwN33HEHGhoacPDgwYgnbwBYs2YN5syZgxdeeGFSnj8iU16evvKjqaG0tLSgrq4OOTk5SE9PD1h8J+JupNuEnDK1Wi1qamqoP3xSUtK4ScoPF6EEHoeDLzq+CEgmZHJdxIoCStcDl2ZDpEIpOIZDisKtJGA2m2E2m9HZ2QmGYaBWqpEYkwgj7yYTlmWRlpYWcK1CVog+a1/INZMai1QoDTmtDwAc3Cmz5dOWj4hMPHW5Fi9eHJYiq68XejjrLsQqoaioKOIGbH1BVBxCKUt4TqFnZmbCbDZDp9Ohs7MT586dG5bHSzhAMhlElBJwz0p973vfw7lz53Do0KEpQSaA+ztNDuSTgYiMUIgN8EjNsMhUu6fsSKDiO3EsXLhwYcAvSiBJeRKWJyYmTqj8dyiBx+HA5rThmVPPUBteArlIDqvTOmTLLpk692rd9QDP87BZbXDZXDAOGt0RhVCIuLg4KBQKP0LRSDUwWEPb8QJusgMQ0hwLuKQnlpOQgy1ZW4b9/kyGLpdn3UWn0wEYXt3FMyXn6VUeqdDpdDh9+vSYDMc8BRpJappELsTjJRwgrpCpqal0GNTlcuH73/8+Tpw4gcOHD9P5k0jDQw89hA0bNmDGjBkYHBzEG2+8gccffxwffPABrrrqqklZU0QTSnV1NSQSCebNmzfk3zgcDlRWVsJms6GwsBBSqdQvMnG5XDh79iwMBgMKCgqGld4gKQxCLkaj0cuvZDznEoYSeBwOjrUdw8ELB71uE7FuQnRwDggYAZ0F8QVJgwUyx/K7zjiAC20XIFe4vUlMJhMcDoeXZ4lG5j5VDxVxMGCgkWrQb+0f8vWpxCqoJWrcuvDWYXd0eepyLVq0aFKGVodbd+F5HnV1ddBqtSgqKopo/3fgUudZON0rPT1edDod9S4hBDPa36DFYsGpU6eQlJRELYY5jsMPf/hDHDlyBIcOHYrowcvvfve7+M9//oPOzk5oNBrk5eXhgQcemDQyASKcUM6dOwee55GTkxPyeiKPIZfLkZeXF3Dy3WazobKyEizLIj8/f9QnUovFQsnFYDCM2a8kEHiex/nz59HZ2YlFixYFFHgcDpycE8+des6rVsGChVQopd1awaITTz0szxqFLzQSDTr0HTD2GiHVSL1SMXa7HWazGSaTCZydg0KqACtjIZfLQ77/ofTBPCERSCAVSnFH/h1QiIe30UaiLhfP8zCbzV4+OKTuYjAYMDg4iOLi4kmfexgK3d3dqK6uHtfOs0DeJeS9SkhIGLbHCyGTxMREqnnGcRx+8pOf4P3338ehQ4ciUqwy0hGRhEKkV+rr62Gz2ZCbmxv0WlJ8T0tLQ1ZWFnieD+phQtRXw1VsJ/lxrVYLvV4PpVJJyWW0xVeXy4Xq6moYjcYxG4tVdFXgQMMBr9u8vEnAgmVZv/qKp+iiTCTzmu/whEKsQFdvF/p7+hGXGAeFMvCmLmAEYHgGdrMd3Qa3Z4lIJKKRCynqA+5U3FCDkwQJsgRcm3MtEuXBrZc9MVV0ucj3qqmpCTabDVKpFMnJyZM27zIcEDLJy8sLaYUdbthsNkouvb291NY3lMeL1WrFqVOnEB8fj+zsbEomDz30EPbs2YNDhw5h7ty5E/YavkqIyKI8AUmdBIOn1zyZDeA4DgzD0C8S8W0niqbh/DF6ziUQvxKtVovm5mZIpVJKLsPttfeMohYvXjymvD7P83SQkcC3k0spUfrVKHxFF4VM4K+IWCBGp7YTvX29yJyRCYfAEfA6BgykQikcLgcEcgFS5am0qG8ymdDZ2UkHLTVKDQQiwdCzLnCn7a6afdWwyWRgYAAVFRVITU2NeDkdkUhEPW8WL16MwcFBvwn0pKSkSW+zJSANLhNNJoD7N0g6QYmtQ09PD86ePRvQ48Vms1Fpf08y+fWvf40dO3bg8OHDUTIZAyI6QmlpaYFOp0NRUZHX/SQd1tHRQdsnAxXfL168iKampjHVIEYDT7FBnU4HgUBAySWY/LfJZEJFRQU0Gk1YnCBrdDXYXbeb/tvXX50BA5FA5KUozDJfpsO+LL6LBWI4XA4/f3iWYdHf048+Yx/S09IhFouDDicSEgvWlszzPCwWC8xmMxwWh3swUiqgk/rBooj1c9ajKLUo4H2+IPpRU0EmnTQLOBwOFBYWejWATPS8y3BAfNUjbSaG53lKxDqdDkajESqVChaLBTExMcjLywPLsuB5Hr///e/x0ksv4eDBg5MmWfJVQUQSiqevfFtbG5YuXep1X2VlJSwWCwoLCyGXywN6mNTW1qK3txeLFi2a1MlWjuOg1+spuRAzLHLCZFl22AKPI8HLlS+jy+j2G5EKpbC7vCfSA3mG+Pq3B6qv8ByPgd4BDFgHkJKSggRlQtAOMUIiAlbgTkUGm4j/UvDRyTnRZ+yjw5ROp5MW9RUKBSXZpelLsWbWmmG9D6RLbiroRzmdTlRUVAAACgoKhmwWIM0i4zHvMhwQfTNPX/VIxeDgIH1vHQ4HXnvtNaoCsWPHDhw8eBD5+cNvOY8iMCI+5eUpvWI2m6lH9tKlSyEUCumcCiETu92O06dPw+l0YsmSJZPuYUImgRMSErxmXc6dOweHwwGlUomBgQFkZmaGraOkqa+JkomAFYDjOb/N3MEFt9EF3LMivoTjcrkw2DMIk9OEtLQ0iASioB1bnkKNSpEypPgjEXxUiVWQSqWQSqWIj4+H3W6HyWTC4OAgenp6IJFIsDB1IZYnLx/W+xBIlytS4XA4UF5eDpFIhPz8/GFFqAqFArNmzaLzLiTl2tTUBIlEQjsRNRpN2OtFZFp/KgxYOhwOVFdXIyYmBrm5udR2+m9/+xu++OILiMViPP7449i8eTPWr18/6iaYKCKcUDy7tfR6Pc2BZ2VlAUBAD5OKigqoVCoUFBRERH7ZEwzDIDY2FrGxsZg3bx7q6urQ3t4OiUSC+vp69Pb2Ijk5ecye3p+3fe5+PjCQCvwHA327tnxtdAG3/4gnCTgcDgz2DMLO2JGamgqWZaGSqAIShVQohcV5yTRrqJZjg83gJQJJIBaLIRaLERsbC6fTCTknR5G8CJ9//jkUCgVNIwbq7BlKlyuSYLPZvDrPRrP5+9YSAin/hqvu0tbWhvPnz08ZMikrK4NMJkNubi5Nc5Ep/kOHDoFlWbzzzjt47LHHMDAwgO9973uTvewpi4hMeREbYGLEM2/ePNTW1iIrKwvTp0+Hy+Xy8zDp7e3F6dOnMW3atDG51U0EPFNyBQUFUKlUXrMug4ODiI2NpSfMkURZHYMdeKXqFQD+UQeB51wJKZi7+EuRIDHHIrfZbDb0a/shkAkQHx/vJvAANRjAHdmwDEtvDyXpIhfJYXFYwIMPeR3gTtHdkX8HVBIVbYAgelAikcjrNN7c3DxqXa6JhtVqRVlZGdRqtZc5WrjgaREdjrpLa2sr6uvrUVBQgNjY2LCuNdwgUZ9YLEZ+fj4lk5dffhmPPPII3n33XaxYscLrb3ien7C947HHHsOuXbtw7tw5yGQyLF++HI8//jg9ME9FRDShDAwM4NixYxAKhUGL74D7S37+/Hnk5ORE7FQrwXAEHonTolarRX9/P9RqNT2ND9VGvKN2B+p664LOl5DJcsCdDhMwAj9S8NzcLRYL9N16qGJVUKkvyV8EqsH4FvWDkQ7gniEhTo5CVhgwLUcgFohx28LbkKL0l/AgnT2k+EpSoJmZmUhLS4u4KNUTZrMZZWVliI+PR05OzoRsZERnTKvVjrju0traioaGBhQUFER8WsjpdKK8vJzuHYRMXn/9dfz0pz/FO++8g1WrVk3qGtevX48bb7wRixcvhtPpxM9//nNUV1ejpqYm4gdYgyFiCYUMoOn1eqxYsQIKhSJg8f38+fPo6upCfn5+xJ+YRiPwaLfb6QbQ29sbMtXTY+7Bi+UvUpkU3+4s4FK3FwMGcpHcLx1FbHsdnANGoxG9ul6kJKdAIvc+ycqEMprWIvAlmUCkA7jnUoQCIZWDCSX9wjIsrs+5HvPiQqslcByHM2fOwGAwICEhAXq9HjabbdIkc4aC0WhEeXk5kpOT6ZT2RMN3hkMikdCGEd+6S0tLCxobG6cEmbhcLpSXl1MlBIHA3RTy5ptv4r777sOePXuwZs3wmjomEjqdDklJSThy5AhWrgxuWR3JiMgaisViwYkTJ+gGIJFI/MjE6XTi9OnTsFqtEe9hAlyagxipwKNYLKa5cc9Uz8mTJyEWiym5aDQaHG8/DpHAbbkbiEykQiltHVZJVAFTTOT2/v5+9On7MHvabPBi78cKJMUSiBQCCVKSuRTqD8+wMNmD11jWzFozJJmQ74LD4cBll10GsVjsJZnT0tKCmpoaxMTE0NTYZE6dE5n0adOmha2rbzQYTt0lMTERZrMZFy5cQGFhYcSnEF0uFyoqKsAwDCUTANi5cyfuu+8+/Pvf/45IMgHcIrAAIr4uFQoRGaHodDq0tbVh7ty5OHjwIJYvXw6ZTEaL7+SkL5VKsXDhwog6eQbCWAUeA8F31sXKWXFw8KB7iFLEBHwOEjEElTb50lekQ9uBwcFBZM7IhJ31T1f5EkqgSCSY/pfvc4eqnRSlFmH9nPVB3wPgkoGaQCBAfn5+0KiPpBF1Oh36+vrComowGpC64MyZMyNW2sOz7tLR0QGHwwGNRoO0tLRJm3cZDlwuFyorK8FxnFfb9d69e3HnnXfizTffxObNmyd5lYHBcRw2b96M/v5+fPrpp5O9nFEjIgmF4zjY7XaaB+3v70d8fDySkpIgkUhQXV2NlJQUZGZmRqx8BgEpYo7ncCXHcdhzeg/KWsugNWjB8zyd3ZDJZGBZlg4phkqHKYQKNLW75T4yMzJh5f3lTzxrMOTfVpfVr/4RiFB8ySNUjWVO7Bx8a/63wDLBP1+SFlUoFMjNzR12vcThcFByIe3IJHIZT2kTvV6PyspKzJs3j1rLRjKIzXROTg7VsJuMeZfhgOM4VFZWwul0orCwkJLJu+++i23btuG1117D9ddfP8mrDI67774b7733Hj799NOIn5cKhYgklO7ubkilUrCsW2vKYrGgu7sb7e3tsFgskMvlyMjIiOjTUrgEHocDi8OC106/hl5Lr1tO3majw4EulwtyuRwpsSkQSURwweXV0UXAcRz6df0wO8yYO2MubFzgtJlny7GnarEnfEnH9+8IgkVKSYok3L7wdkiEwT/bcOly+UrKE5O1xMTEsMqkkyiVOBdGOojFg69c/kjqLhMFjuO8rAhIxuKjjz7CLbfcgr/+9a+46aabJnxdw8W9996LvXv34ujRoxEbtQ4XEUkoN998Mz7++GNs2rQJW7ZswTe+8Q089NBDiI+Px3e/+104HA50d3djYGCA5sWTkpImfYiRIJwCj8PBZ62f4fDFw363Eztlm9lNME6HE0Kp0G/y3Ol0ol/bDwfjQEZaRlDSkQgk1DmRBQuJUOJXmAf8U2AykQw2p80vipEKpX4ikEqxEtvyt0EjCZ6rHxgYQHl5OdLT08PaIs5xHE31aLVaOBwOWkdISEgYdWqVCCcuWLBg1HbWE4nGxka0trYO6b0SyN/F0853ImwBSDOGxWJBUVER/YwOHz6Mb33rW3j++edx2223RUQU5Que5/GDH/wAu3fvxuHDh4dl0xHpiEhCcTqdOHz4MHbs2IE9e/bA6XRCIBDg5z//OW677TZKHFarFTqdDt3d3V7ttcnJyZNWdLXb7dRrYywy+cOFw+XAM6eeCaoIDLgjARdc6Df108iFqNhKpVIMDg4iThkHTbwGIqHIz4yLPg5JWX0plRKog0sikMDustPoRsgKwYDxi2IUYoVfMV7EinDbwtuQqvK3iyUgMjXjrctFZNIJuZhMplHNBhF5koULF064cOJIQYy82tvbUVRUNCI7ZFJ3IR2JE6EzRhw3TSYTioqK6G/t008/xXXXXYennnoK3/3udyOSTADg+9//Pt544w3s3bvXa/ZEo9FEvFVBMEQkoRB0dHRg8+bNtHvnwIEDGBgYwIYNG1BaWoo1a9bQ07/dbqc/fiIln5ycTIuuEwEyqU+G1CZiBuJkx0l82PRh0PtZBJ5oJ4OjAwMDEDNisCIW8Zp4MBImIAkKGSE4cEM6PHrWSViGhUQQOIrxrbEwYHBdznXIig8+1EWUo7OyspCenh70uvEAqSHodDr09/dDpVJRcgm28ZL62VTQuhoLmQQCmXfR6XQwGAxe71c46i7EgI94xZDv7PHjx7Flyxb8/ve/x/e///2IJRMAQdf2yiuvYNu2bRO7mDAhYgmF53kUFhZi0aJF+Mtf/gKJRAKO43D8+HHs3LkTu3fvhlarxbp161BaWop169bRHwEpunZ3d3vNbiQnJ49bEXE8BB6HAsdzeO7UcyEnzINt/qSlNi4uDqlxqTCbzOjs6/TyKlEoFBCLxWAYhj5OqK4srwHFEFFMoBrLlTOvxGXTLgv6OshJPzc3d9zMm4YLMhtE6gjEqoBM6jMMgwsXLqC5uXlKzG3wPI+GhgZ0dHSguLg47Acw3/eL1F1IE8RI6y48z6Ompgb9/f0oLi6m0U9ZWRmuueYa/PrXv8aPfvSjiCaTryoillAA9wlv2rRpAb8YHMehvLwcO3bswK5du9DW1oY1a9agpKQEGzdupB4kTqeThuE9PT2QyWS05qJSqcLypevs7ERNTQ2ysrImtEPjtPY03jn/TtD75SI5HC6HX7ppYGAAvb297sKzOg4ykYySjqdXidlsBsuyUCvVUCgUiFHGwOK0BCzWA97kNRyfeoKClAJsnLsx6OsgNgT5+fkR16Pv277NsiwkEgnMZjMKCwunBJnU19ejq6trQiyGx1p34XketbW10Ov1KC4upqnHqqoqbNq0CQ8++CB++tOfRslkkhDRhDJckMIcIZeGhgZceeWV2Lx5M66++mrExsZST/menh50d3ejp6eHDgYmJycP2wTLEzzP09bKhQsXTqgfBM/z+GvFX6Ez6wLeLxaIIRPKvDZ1nufR19cHg8GAlJQUyGQyxEnj0GftC0gSHMfBYrGAsTPoG3BfI5FLaDuy5/vFMixYsHDyzpAWvr41llkxs3DjghsDtgeTNMxU0eUizRg9PT0QCoXgOI62uyckJEyKd30okE7E7u5uFBcXT/hw8EjrLsQHqbe314tMqqursXHjRtx33334xS9+ESWTScRXglA8QU4wO3bswO7du3H27FmsXLkSpaWluOaaa5CQkEDJxfNkKRQKvUywhvpSBhJ4nEic7z2Pt2vfDngfy7AQs2KAAe2iIgqrZrMZqampEIvFkAgkcHLOgB1dBAwYSIQSt/S+8VJRn+M4r1kXjUyDQdsgFCJF0DkXwDtySZAnYGveVkiF/gVu8jn29vaisLAw4rWNyGbX09NDfXoGBwfp98tkMiEuLo6mxia73Z3nedTV1VEDu0hQmghVd5HL5aivr4dWq0VxcTEtWtfW1mLjxo343ve+h9/85jdRMplkfOUIxRMkN0zIpaKiAsuXL0dpaSk2b96MlJQUqgmm1+vR3d1NZxFI5BIoxzscgcfxxmtVr6FtsM3/Dt5t7cuDp11UHMehu7sbTqcTqampEAqFEDACqCQq9Fv7Qz6PSqwCD97L7dFv1sXpglqphkqpgkQmAc8E/kp51ljkIjnuyL8DMdIYv+tI947RaERhYWHEtIMHA8dxNKdfVFQUsEPHd7MkHYmkSD2R8CQ/z805kuBbdyHijjk5ObSFu76+HuvXr8ftt9+Oxx57LOKHnL8O+EoTiieIJfDOnTuxa9cunDhxAkuXLkVJSQlKSkporYbjOPT19dGOMZ7naeQSFxcHm81GZV/y8vImJY3RYmjB62deD3gfKZqTLiqXy4Wuri4wDIPk5GQIBAIwYKAQK2B1WOHk/fW2KHggRhYTknR4noeUkULXr4PFbIHJaqIui3K53Ov9IdGJkBXi1txbka7279RyOp2oqqqC0+lEQUHBuLddjxUk3UpaV4cTedhsNq/NUi6XU3IZTep1JPCsQQQjv0gCSct1dHQgLi4ODQ0NuPfee7FkyRLU1dXh6quvxrPPPhslkwjB14ZQPMHzPNrb27Fr1y7s3LkTn3/+OQoKClBaWoqSkhLMnDmT+tITh0Vywud5HrGxsZOqIfavs/9CY1+j3+2kdkHkVRwOBzo7O2lXDfnRqSVq8OC97H4DIVRh3RMyocwt8ugwweFweM26SCTumotaqYZYJIaLd6E0qxTzE+f7PQ6Z4REKhSF1uSIFLpcLp0+fhs1mQ2Fh4ajIz+l00tRrT08PBAIBJZfY2NiwbpSkO6qvr8+rBhHJIDU00n3mdDqxc+dOPPvss2hubobZbMY3v/lNlJSU4JZbbhlzu3MUY8PXklA8wfM8uru7sXv3buzcuRNHjhxBbm4uSkpKUFpainnz5oFhGOzbtw8ikQgJCQmw2Wyw2+1ISEigDosT5bvRberGSxUv+d3uqamlFCvRO9iLzs5OKJVKaooFuFNYRrsRQoEQDpfD73EIFGIFwCOk2yLgllRhGCZge7DT6aQdY4ydgVPgxOpZq7FxwUa/9m1PXa7RuhZOJJxOp5cQYTgOF77RMcdxtEA91u+YZ6ttUVHRlCATIv9SXFxMiaKjowNr167FlVdeiRdeeAENDQ3Yu3cvDhw4gH379k14LTMKb3ztCcUTxGt679692LFjBw4ePIjMzEzMmzcP7733Hl599VVcc8014HmeFly7u7thtVqpxWpiYuK4nqz31O3BWd1Zr9vEArdcu4NzQCqUom+gD13dXYiNjaVzEYCbJMx2M1RiVdAuLMDdiSUWiAOShC9ipbHos/aFvIYBAxYsUoWpyJfme81uJCUlQSAQoKKiYkKNpsYCh8OBioqKIRWOxwKe5zEwMECL+haLxauoP5JoiOd5nD17FgaDYcqQyYULF3DhwgUv+Zeuri6sX78ey5Ytw9/+9rdJN087evQonnjiCZSVlaGzsxO7d+9GaWnppK5pshEllCDgeR56vR633norDh06hPT0dAiFQpSUlGDLli30FE18N7q7u6lER3x8PJKTk8Nu6tRn7cNfyv7ipYnlO43OW3k0dzQjMTHR67RGrX45F6Qifw0tAgEjgJAVQiQQeRXiA0ElUmHQMTTpqMQqxMpicfOCmyFgBbR9m2yWLpcLKpUKmZmZtMU7UmG321FeXg6JRIK8vLwJ29TIIKpOp8PAwAA0Go1XB1QwcByHs2fPYnBwcNg1nskGmTsqKiqCWq0G4FZJ2LhxI/Lz8/H6669HRDr0vffew2effYaioiJce+21UUJBlFCCwmq14tZbb8Xp06dx4MABJCUlYf/+/di1axfef/99JCUl0bRYUVERTdGQH353dzeMRiM9VSYlJY25wPx+4/so6yy7dIPHNDrP8zAPmKHVa5GcnOy1yQhYAYSMEDaXjTo2BgJxcXTxrqCEQyAXyiFgBcOKYlKVqbhpwU2QibwLwETOPTk5GQzDQKfTged5ql4bHx8fUakvkpZTKpXIzc2dtLURDTudTge9Xh/UxdOzW26qkAmxGfY08+rt7cWmTZswb948/Otf/4pI/yOGYaKEgiihBIXL5cJvfvMb/PCHP/TTYTKZTHjvvfewc+dOvPvuu4iNjcXmzZtRUlKCpUuX0lMrkd0nPhIxMTE0chlp2sFkN+HZU896uSCSojlJ1fEWHuoktdfG4Wv1Swr2gUCK+iqJKmTBXsSKIGbFMDlD11cAd0rsxgU3Ik7mPeEeSJfLswmCqP16kstknkotFgvKysoQGxuL+fPnR0wU5eni2dPTA5FIRAcpW1tbqQpvpHfLAUBbWxvq6+u95Gr6+vpwzTXXYNq0adixY0fEvo4oobgRJZQxwmKx4MMPP8TOnTuxf/9+SKVSbN68GaWlpVi+fDndBK1WK41cDAYDTVkkJSUNq3Xz0IVD+Lztc/pvIl/CcZw7bWR3ITklGQKRdwrGc2p9OGRCjLiCDSaSFJuIFYWswwDu9NnNuTdjhmaG1+0dHR2ora3FwoULg+pyedaptFotLBYLnTqfaH94k8mEsrIyat8cKWTiC5fLBb1eD61Wi87OTvA8j+TkZKSkpCA+Pn7Saw6h0N7ejrq6OhQUFCA2NhaA2xK3pKQE8fHx2LNnT0RHWFFCcSNKKGGE3W7Hxx9/jJ07d2Lv3r1gWRZXX301tmzZgpUrV9JN0Gaz0Y2yr68PKpWKKiMHyofbnDY8c+oZKitPOrocTrcvDMdxyJ6ZDaPTx8DKR8gxWLrL8/aQrcJfDk1aHVaqPBwKmzM3Y2HSQq/bRqvLZTQaqUTH4ODgqKTkR4PBwUGUl5cjLS0trN4r4wWO43D69GlYLBbMmzePEozNZvOSgYmkkz4R/ly0aBH9TgwODmLLli2Qy+V45513In5eJkoobkQJZZzgcDhw5MgR6unicDiwadMmlJaWYvXq1fS0RSaCu7u7aT6ckAtplTzWdgwHLxwEcMkl0WK3oLOzE0KhEGkpaRCwAq8hRaVICZPDRCONQGZWgHuGxOZym18JGAF48EGJgpDNcOZTVkxfgSsyrqD/JqoF7e3tKCwspMXW0YD4w5Noj0ydByPk0cJgMKCiogIzZszArFmzIp5MyFyMr3MhaRwhhxij0UiN6RITEyd1s+7q6kJNTY2XxL/JZMJ1110HhmFw4MCBiJfdAaKEQhAllAmAy+XCJ598QmX3jUajl6cL+UGTfDiR3ZfJZIhLiMO+7n2www4BI4BUJEW/qR+dnZ2QyWTu6Wqp2qvmQTu6PDS6AtVFhIwQLMtST/fh+JywDAsGTEj9rwWJC1CaVUr/PZ66XL4+OMEK1CNFX18fKisrMXv2bGRkZIRtveMFl8uFqqoqOBwOLzIJBJJ+1el06Ovrg1Kp9PJ2mSjiJE6W+fn5VFjVYrHghhtugN1ux3vvvTdl5kqihOJGlFAmGC6Xy8vTpaenB+vWrUNJSYmXp4vT6URPTw+O1h/FB80fQCgUIlmTDBtjQ29vLzQajXuSmmG9hhQFrAACRkBJAkDAugjLsJAKpNSXhAULlmW9iv4EZH6FBx/SDwUApqmn4ZbcWyBk3bUjT2mS8dblIoRMps4lEgklF895nKHQ29uLqqoqZGZmTqgdwWhByITI1YykvkS8g0hRn7xnxKtkvMhFq9XizJkzyMvLo06WVqsVN910E/r7+/Hhhx9GvLq00WhEQ0MDAKCgoAB//OMfsXr1asTFxWHGjBlD/PVXE1FCmURwHIeysjIqXtnW1oarrroKJSUl2LBhA9RqNbaXbUevuRdCpxCtulZYrVYIBAIolUoolUokqhNp665vRxdBoMjDN2IJRhSeisQMmJAT9jHSGNyRfwfkInfaaTJ1uXzVpImkCVGTDtbySza6+fPnIzU1uBVxpMDlcnlN7I+lE87Xq4RhGDqpHxcXF7aivk6nw+nTp72aMux2O2699VZ0dnbio48+ijjfm0A4fPgwVq9e7Xf71q1b8eqrr078giIAUUKJEJBiKvF0aWpqwuKNi6FZpkFhbiGOHD8CnVaHq666CgzDUL0sqVAKkcztsJisSfbrvBKwAvC8d13El2AYMBAJRF5RDflbMr8CBCcdAJAIJdiWtw0JcnfqIpJ0uQIJfpJ2ZM+NsqurC2fPng3ZfRZJcLlcqKioAM/zYyYTX3AcB4PB4NXC7VnUH22XXU9PD6qqqpCbm4vk5GQA7ihp27ZtaGpqwn/+858J9RWKIryIEkoEguguPfj2gzh15hR6+nvAsAyWLFlCjZAYhoFSpITW4J7O5608zC6zl0eJp3Uvga9bYrDb/KIdHkEn7AWMADcuuBEzY2YCiGxdLmLqRDZKoskmFArR2dnplc+PZDidTlRUVIBhGBQUFIxrSzDP8zAajTTaMxqNo+qyI6nEnJwcGv05nU7cddddqK6uxqFDh6YEkUcRHFFCiVA06hvxz6p/Yu/uvTjffB4LFixAS0sLurq6MGP6DGRnZ2PRgkVgJSwVfLRYLV4GWEql0q3yK3X7wnsKSHpCJpRR6RYC35RYqAn7jXM3oiClAIC7Q6e8vHxK6HKRjbK+vh69vb1gGMZr1iWSWms9QciEZVksWrRowudLLBYLJZf+/n4vI6xgar9EFSE7OxtpaWkA3BHW3XffjZMnT+Lw4cNTIsUYRWhECSVC8ffKv+Nv//4bzjWewy233AKVSkUnyWtra1FXXYfWrlbMSJ2BrPlZyMzOhEqlorL7NpsNjI1BV38XOI6DWqGGUqWESCLyihiIb4onAqW2gg1FLpu2DN+c+U0Abq/68vJypKenT4mZDQBobm7GhQsXUFhYCKFQSGddiLIBqbtEiqCi0+lEeXk5BALBpJCJL3yNsIjoZ2JiIm2E6O/vR3l5uZcqAsdx+MEPfoBPPvkEhw4dwvTp0yf1dUQRHkQJJQLRPtCOf1f/G519nWBZNuBmphQr0d3TjdraWpypPYPWllakpqUiOysbWVlZiI2JhVgohs1pg9PuhN1iR+9AL5xOJ02LyeVyqCQqL0IJlP6SC+W0G8wT2fHZuDb7WjAMA71ej6qqKsyaNQszZ84M+3sSbnj61Xsq2hL4ttaSU3hSUtKkzUUQlWNSl5psMvGFbyMEy7LQaDTo7e1FZmYmJQ2O43D//ffjww8/xKFDh6bE9yWK4SFiCGXz5s2orKyEVqtFbGws1qxZg8cff5yGx18n7K7djaquqqD3y4QyWJ1WyMVymOwmmro5d+4camprcPHiRaTFpmFu7lxkZ2UjIzUDRocRPM/DbrfTtBjrYiGQCrzIJVBKLFC6K02ZhlsX3gqRQBRQlyuSQfzUtVotioqKhiQIcgrXarVeDotJSUk0KhxvOBwOlJeXQywWT6jK8WjBcRyVUxEI3I0hr7zyCq644grU1tZi//79OHz4MObMmTPZS40ijIgYQnnqqaewbNkypKamor29HT/5yU8AAJ9//vkQf/nVQo+pB8998VxQLS3AnZJiwAQcQuR5HmazGc3nm1F5thLNtc2InxaP7KxsZGdnIyEhgW6AYojR3dftrrk4OEhlUkjlUigUCrphBZqw10g02Ja/DUqxkm4aubm5U6Kg6ulaWFhYOOLJejIfRGZdiBhjcnLyiGZdRgKHw4GysjJIJBLk5+dHVJNDMAwMDKCsrAyzZ8/GjBkzoNPp8Pvf/x7vvfce2tvbsWLFCtxyyy0oKSlBSkrKZC83ijAhYgjFF/v27UNpaSlsNltEylWPF/ae24uKzoqg90sEEkiF0pDSJyRtpRar0W3oRl1dHWpra9HY2IiYmBhkZWchPycfMYnuwTWWYcFwDHoNvTAajbDb7ZBK3cSSGpcKs+tSukssEGNr3lYkKZJw4cIFNDc3j1iXa7JAvEEGBgbCYjTlKcZI5jZI5BIu+17ivyKVSpGXlzclyGRwcBBlZWXIyMjArFmzALiJ/He/+x1efvllvPLKK6ipqcGePXtw/PhxHDx4ECtXrpzkVUcRDkQkoej1etx9991ob2/Hp59+OtnLmTAMWAfwp+N/CilrEieLg96iD/k4CrE7hUOm2wmsVivOnz+P2nO1aDzbCEW8AllZWShaUARN0qXTNfGFt1vsMFqMkEgkUCqVUClVuDnvZsyOmR02Xa6JgsvlwpkzZ8ZNzp3jOC/pfZfL5SW9P5oUld1uR1lZGeRyecS1XweD0WjEqVOnMGPGDMyePRuAm0yeeOIJPPvsszh48CDy8vLo9d3d3YiJiZkUJeHnnnsOTzzxBLq6upCfn49nnnkGS5YsmfB1fJUQUYTywAMP4Nlnn4XZbMZll12G/fv3+3mRfJXxQcMHONZ6LOj9UoEUDs4RknAUIgWdbA8kowK4BSYtNgvqztehoboBNU01kMqkyMrKQnZWNtLT090FVYkGvaZeWnPJleSiKLUIPM/DarWiuLh4Sgj3kWlyp9M5pM5VOOBp39vd3Q2bzUYtooc7FEjIRKFQTKqZ10hgMplw6tQpTJs2jdZGeJ7Hn/70Jzz55JP46KOPUFRUNMmrdOOtt97C7bffjr/85S9YunQpnn76abz99tuoq6ubEqnbSMW4EsqDDz6Ixx9/POQ1tbW1yM7OBuCeotXr9bh48SIeffRRaDQa7N+/f0q0n44VFocFTx17ym9anUDEiqASq6C3ho5O1GI17Jw9pONijCQG/bZ+OvTocDjQ2NiImtoanD9/HiKRCNnzspGTk4P06W5yWZy2GJclXEbVbMmcC1FGjlRiITMbAMI+TT4cBLKI9vSGD3Qyt9lsKCsrm3RnyJHAbDbj1KlTSE1NpS3jPM/j+eefx+9//3u8//77WLp06WQvk2Lp0qVYvHgxnn32WQDuCHP69On4wQ9+gAcffHDcnpfjOK/Pk+f5r9T+Nq6EQnrTQ2H27NkB0w9tbW2YPn06Pv/8cyxbtmy8lhgxOHLhCA41Hwp4HwMGSrESZoc5ZHQiEUggFohD2vIKGHfqRSaSuTvEfIr/TqcTTU1NaDzbiDP1Z8CwDBbPWoz/XvHfeOKJJ5CZmYnt27dTy14iu086n5KTk6FQKCLiR0I6o0QiUcS02ZrNZpoW8/SGJ0ZrhExUKhUWLFgwJcjEYrHg1KlTSE5Oxrx58yiZvPzyy3jkkUdw4MABfOMb35jsZVLY7XbI5XLs2LHDSx1469at6O/vx969e8fleZ1OJ4RCIYxGI86cOYPFixdPqiTReGBcXw0RlhsNOM7dumqz2cK5pIiEw+XAibYTQe9XS9w1ilBkAriHD/usfSGvUYqVsDltsDgsATvJhEIhsjKzsCB7AdY61mKgYwC9/+nFHXfcgeTkZIhEIhw6dAirVq1CWloa0tLS4HQ6aVvthQsXIJVKaeQyUW21vrDZbCgvL4+4+oNcLsfMmTMxc+ZML6O1+vp6KBQK2Gw2aDQa5ObmRgQpDwVCJomJiV5k8ve//x0PP/ww9u3bF1FkArgzIS6Xi2qJESQnJ+PcuXPj8pwulwtCoRB6vR5XXnklVq5cCalUioKCgq9UlBIR9HjixAmcPHkSK1asQGxsLBobG/HII49gzpw5X4vopLyzPKg1r0aiwYBtACJB6Lx7jDQG/db+kNcwYODg3DL2ochJJVFhwDaAWHksbl5xM6773+uwadMmfO9738O+ffvwgx/8AEajERs3bkRpaSmuvPJKpKamIjU1FS6Xi7bVnjp1CmKxeFQS8mOB1WpFWVkZ1Gp1RJ/yJRIJpk+fjunTp9POKIFAAL1ej2PHjtGivlqtjsgNh7zPCQkJ1BqZ53m8+eab+OlPf4q9e/di1apVk73MiIBAIIDJZMKyZctQUFCABx54gLZLk8/WNx02FRERhCKXy7Fr1y786le/gslkQmpqKtavX4+HH344on2kwwGO4/B5S+BZG5VEBYPNMKQHiVKsBM/zIWdXAHek4+ScVD04IHjA7rJDLBDjpoU34X8e/B9cfvnleP755yEQCLB27Vr86U9/wrFjx7Bz50787Gc/Q29vL9avX089XZKTk5GcnEzbaru7u1FRUUEl5JOTk8fNa8NsNqOsrGxKaIkRWCwWVFVVISkpCTk5OeA4jpJyeXk5hEKhl/R+JLwmQiZxcXHIzs6ma9q5cyfuu+8+vP3227jyyisneZWBkZCQAIFAgO7ubq/bu7u7wzYTc+HCBahUKq+moqeeegpJSUn417/+BcBtfXzw4EG0tbVh69atSE1NnfKkElFdXpOFCxcu4Le//S0OHjyIrq4upKWl4dZbb8UvfvGLcRcIrOqqwu7a3X63y4Qy2F12uDhXUJVfwN35xYOnnV1BwQOx0lj02UKnxNQSt/vjjQtvRFZCFux2O0QiUdBNjOM4nDp1inq6dHR0+Hm6kOvIzIZWqx2XmQ2j0Yjy8nIkJycjMzMzIjbeoUBSRsEI0Pd9A0Dft7i4uEnZfEidh0SAZM179+7FnXfeiTfffBObN2+e8HWNBEuXLsWSJUvwzDPPAHC/zzNmzMC999475qK83W5HdnY2HnroIdx111309qeffhpvvfUW/vnPf2Lv3r04duwYDh48iIyMDOj1ekrQUxlRQgHw/vvv46233sJNN92EuXPnorq6GnfddRduu+02PPnkk+P2vDzPY/vJ7dCatF63i1gRWIaFzWULGZ0IWSEEjAAyoQz9tv6QzxUnjRuyQwxwtx2vyFiBZdNHnmrkOA5VVVWUXJqamnDllVeipKQEmzZtoqdrMrNBOp+IP0lycvKoN0mSMiItq1OJTBISErxO+cFAxEEJuTidTtqOHB8fPyEFXrvdjlOnTkGlUnnVed59911s27YNf//733HdddeN+zrGirfeegtbt27FCy+8gCVLluDpp5/Gv//9b5w7d86vtjIadHV1ISUlBTzPkfQ2HgAAJORJREFUo6+vD3Fxcdi1axcee+wxdHR0QKPR4L//+7+xceNGtLa24kc/+hH27ds35XXNooQSBE888QS2b9+OpqamcXuOup46vHnmTa/bfH1Igqn8MmAgF8thsVsgYAVwcIFdFIEvU10uZ0CBR08oRArMT5qPTZmbRvFqvMHzPM6ePUvJpba2FqtWrUJpaSmuvvpqxMfH05y77yY50oHA/v5+VFRUYObMmXQyO9JBUnOJiYm0/jAS8DyPwcFBOutitVq9pPfHY9Ym2GzMhx9+iFtvvRUvvfQSbrzxxrA/73jh2WefpYONixYtwp///OewtTaTQvu3v/1tnD59GkeOHEFSUhLKysrQ2dmJlStXUomj3bt345FHHsHevXunvLZZlFCC4OGHH8b777+PU6dOjdtzvFz2MloHWr1u8zTECuVBQq7TSDUwWIPLsMiFcrAsG/RxPJGblItrc64NexqF53mcP38eO3fuxK5du1BVVYUVK1agtLQU11xzDZKTkym5eA4EEvOr5ORkmvf2BfHZmDt37pTx8TaZTCgrKwtrao4YYGm1Wi8DrKSkpLDUIYmemEwm8+qaO3ToEL797W/j+eefx2233TYlIsPxhMvl8vqe1tXVoaSkBDExMdi1a5eX2G13dzeOHz+OrVu34le/+hV+/OMfT8aSw4oooQRAQ0MDioqK8OSTT3rlQMOJi/0X8UrFK163+XZqBSMUz+skAknQIruYFYMHD6lQGnI2BQBSlCm4o+AOSITj2wTB8zyam5spuZw8eRLLli1DSUkJNm/ejPT0dEouRqORpsUsFovftLlOp8OZM2emjMoxcIlMUlJSaJttuEEMsLRaLQwGA9RqNSWXkYphAt5Kx57ilJ988gmuv/56PP300/jOd77ztScTT/zsZz/D3XffjVmzZqGlpQXr1q2DUqnEjh07kJGRgc7OTvz+97/H0aNHcfPNN+OBBx6Y7CWHBV9pQhnppD4AtLe344orrsCqVavw0ksvjdva/nn6n6jvraf/9nVIDOZB4ulXEqq+ImAEEAvcDQW+boy+UIqVuKvoLmikmhG/jrGA53m0trZi165d2LVrFz7//HMUFxejpKQEJSUlyMjIoJuUJ7mYTCYolUoYjUZkZ2dj2rRpE7ru0cJoNKKsrAxpaWkTZkBms9nojJBer4dCoRjRACox9BIKhVi0aBElk2PHjmHLli343//9X9x9991RMvHAZ599hiuvvBKtra10Dq+9vR3r16+HSCTCrl27MHPmTBw/fhw2mw1XXHHFJK84fPhKE8pIJ/U7OjqwatUqXHbZZXj11VfHrYOm29iN7Se303/Tji6PLq1ARle+1wWy7vX9e19PeV+IWBG2FWxDunpyT/g8z6OzsxO7d+/Grl27cPToUeTl5VFy8dyAT548CYPBAKlUCqvVitjYWDpIGam2vYRM0tPTJ61pwOFweEnvSyQS+r4FmnVxuVwoLy/3sxo+deoUNm/ejEcffRQ//OEPv/Zk4pvmslgsWLRoEV588UVcccUV9P6uri5cc801GBwcxL59+5CZmTmJqx4ffKUJZSRob2/H6tWrUVRUhH/84x/jKtOxs2YnznSfAeDd0UUgEUhgd9m95kpErLt1l2h9BSIcAkIiYlZMBxkDgQGDGxbcgPlJ88P10sICnufR09NDyeXgwYPIzs5GSUkJzGYz/vrXv+LYsWOYNWsWTe90d3dHrG0v6UCbPn06Zs+eHREbsK+7IpkRIrMuPM97aaCR30NlZSU2bdqEn//85/jJT34SEa8lUnDx4kVkZGSA4zjMnz8f3/nOd/Czn/3M6xqdToclS5bg/vvvx7333jtJKx0/RAkFbjJZtWoVMjIy8Nprr3mRSbjNf/osfXjmxDPgeM6vo4vAN6pgwEAmknl1ewXyggfgVaQfKjpZM3sNVmSsGOtLGleQtsu9e/fiySefRH19PQoKCrB69WqUlpZ6dRsR216tVov+/n6o1Wp6ApfJZJOyfkImnnLukQaO49DX10ffO9KhJBKJUFxcTKO+6upqbNy4ET/+8Y/x85//PEomHvjFL36BP/7xj5g/fz5kMhl4nkdCQgLuvPNOajFNbKZtNttXdmA7SigAXn31Vdxxxx0B7wv32/Pu+Xdxsv0kgMAbvogVwcW5wOFLG14eUEu9ayXB6itKsZIKPgoZYUiJlcLUQmzOjuzhMwKe5/Hb3/4Wf/7zn7Fz505ad/nggw+QmpqKkpISlJaWoqCggJILqR10d3dPmif8wMAAysvLvYymIh0ulwunTp2C1WoFy7I4evQoTpw4gRUrVuDZZ5/F97//fTz66KNRMvHBsWPHIBKJ0NzcjJMnT+LkyZM4cuQI5syZA5vNBpvNhtTUVPz5z3/+SpuJRQllAmG0GfH08afh5JxUQt4XvrcHui5QMd7XKyWUttes2Fm4Ne9WCNjJV98dDlpaWnDVVVdh586dyM3NpbcbjUYcOHAAO3fuxIEDBxAfH4/NmzejtLQUixcvppGmrye8Z2FaqVSOy5qJBe6sWbOmzLAaGUy12+0oLCyEUCjEmTNn8Pzzz+P999+HXq/HunXrcN1112Hz5s1ISEiY7CVPCoYjj/LKK6/g97//PT777DO0tbWhuroaDocD3/3udydolZODKKFMID5u/BiftnwasP5x9OhRNJxvQGdnJwQiAR588MHAxBHA4923DsOCBcuyAQ22EuQJuLPwTkhFkVFfGC6I9HcwmM1mfPDBB9i5cyfeffddKBQKXHPNNSgtLcWyZcvo3/oWpmUymRe5hOPkbTAYUF5ePuXIxNPRkgxGNjU1Yf369bjhhhvwX//1X9i7dy927dqFjo4OtLS0fO0iFc8C/Pvvvw+r1QqGYVBSUgLA/f0SiUQ4f/481q1bhyNHjkyZ+ahwIEooEwSb04anjj0FBoxfRxcAHDp8CDHSGGgHtKgor8BvHvkNrA7rpdTXlwhUX/GtwwSrnShECtxZdCdiZbFhfnWRBavVio8//hg7d+7Evn37IBQKcc0112DLli1YsWIF3SydTid6e3vR3d2Nnp4eiMXikF1PwwGZ2p89ezYyMjLC/dLGBRzHobq6GiaTycse+eLFi1i/fj2uvvpqPPPMM16n8sHBQVoTmGz87ne/w7vvvovKykqIxWL09/ePy/N4yszfe++9+PDDD8EwDFiWRV5eHt566y16bW9vL+bNm4fnn39+SqkHjBVTV9ZyiuFk+0m4OBdcvCtgXeObq76JJcuXIDkpGXC5PVJ8yUTEivwiFpVE5VecDyQkKWSFuHHhjV95MgEAqVSKq6++Gq+88gq6urrw+uuvg2VZfOc738GcOXPw/e9/Hx988AH1xMjLy8MVV1yBzMxM6qPy6aefoq6uDv39/cOuoxEymTNnzpQhEyKRYzQavcikvb0dmzZtwrp16/zIBEDEkAngTmnecMMNuPvuu8f1eQiZ/PrXv6aRWl1dHbZs2YK3334ba9euhcPhlkCKj4/HtGnTYDQOrVDxVUKUUCYATpcTJ9pOQCwQB7X4VUvVcHAOsGABHgG1uRRihVcLcIw0xo9g1BK13+Q8Awal2aWYrpkehlcztSASibB27Vq8+OKLaG9vx9tvvw2FQoF7770Xs2bNwl133YX9+/fDbrcjKSkJubm5uOKKK5CdnQ2n04nKykocPXoUtbW10Ov11PjNF319fSgvL59SEjCETAYHB73IpKurC5s2bcLKlSuxffv2iJdTf/TRR/HjH/8YCxcuHPfnamhowOnTp/Hyyy8jNzcXb7zxBp5//nn89re/RU1NDTZv3gyr1X2ge+yxx3DnnXeO+5oiCRHhh/JVR1V3FViGDS5/wgNWhxXgATEjDvipCBiB1yS9WqIOWHR3uPyJaNWsVchNzvW7/esGoVCI1atXY/Xq1fjzn/+Mzz//HDt37sRPf/pT9PX1UU+XtWvXUrdRz5baM2fOgOd5P/n4vr4+VFRUIDMzc8pM7fM8j9raWhgMBhQXF9M2Vq1Wi02bNmHx4sV46aWXIsI2OZIwe/ZslJSUoLi4GEeOHMEDDzyA5557DrfccgsMBgOefPJJ5OXlobq6Gps2jV1kdaohSijjCCr9sghAEFWTe+65B7PTZ2PANgCNVAMrH9j3RCVRUQKRi+Qw2vxD6UDF/vyUfFwx86sj7RAuCAQCXH755bj88svxxz/+ESdPnsSOHTvwq1/9Ct/73vewdu1a6ukSHx+P+Ph4ZGdnU9n9mpoauFwuaDQa9PX1TTkyOXfuHPR6vReZ9PT04JprrsGCBQvw6quvfuX8zocLUivp7e31MsjieR4sy2Lr1q0AgMOHD+Pyyy/HtddeCwCYPn06/uu//gsqlSpiFRvGG1/Pb8wE4f7778eK0hV4v/X9oNfExsbCyTlDDiEyYGCyu+skEoEkYH0FgN9EfEZMBjZnTY1Zk8kEy7JYunQpli5discffxyVlZXYsWMHHn/8cdx9991+ni6xsbHIyspCXV0d2traIBQKUV9fj/7+fipeGakne57nUVdXh56eHhQXF1M1gb6+PpSUlGD27Nl44403xkX+fiQYjQ5fuMAwDM6cOYNNmzbhiy++8LPqJWhoaEBNTQ1kMhkMBgP279+PjRs34kc/+lHY1zRVECWUcURiYiLaL7SH7NdXiBRgwIS0+FVL1DDYDBAwAj+ZFgK5UO6lTBwvi8eNuTdOmVmTSAHLsigsLERhYSF+97vfobq6Gjt27MAzzzyDe+65h07oCwQCPPDAA/jggw+wYMECDA4Ooru7Gw0NDaiurvaS3Y+Ukz7P86ivr4dWq0VxcTFVDzAYDCgtLUVqair+/e9/R8Tp+v7778e2bdtCXjOeygODg4PgeT7kRPs999yDzZs3Y9asWZBKpVAoFF9rMgGihDKuaOhtQKexM+Q1YlaMQccg+g39sFgsMBgM4HgOXV1dAID4uHhYBe40mFwkD1qHEbKXPkqZUIab826GTDQ5ciNfFTAMg4ULF2LhwoX49a9/jbq6OuzcuRP/93//h4sXL+KKK67AsWPHqCGYWq3G3LlzqTdJU1MTzp49i7i4OCQnJ4+b8dVwwPM8Ghrcc06LFy+mMvaDg4O47rrrEBMTg507d0aMJAipYU0WiouLwbIs/vOf/+D6668PeE1hYSE++OAD7N69G7GxsV8JP5OxIkoo44hPWz4Neb9cKIfFZYGTc+LQoUOoqqqi973wwgsAgLu/czdE00UhjbSkAikG7O4IR8AIcOPCGxEvjw94bRSjA8MwyM7OxqJFi9DZ2Yn/+7//g9VqxZtvvon7778fy5cvx+bNm1FSUoK0tDSoVCrMmTMHJpMJWq0WLS0tqKmpQVxcHC3qT2Qk0NTUhI6ODhQXF1MyMZlMuOGGGyAWi7Fnz55J0zsbK1paWqDX69HS0gKXy4XKykoAwNy5c4elhOA7+c5xHHieR2pqKpqbm4P+nVgsRkFBAQoKCsb8Gr4qiA42jhPaDG14qTy0n8pQ4o2AOyoRsaKQ13k+zpacLchPyR/5gqMYEv39/Zg3bx62b99OT608z6OlpYV6uhw7dgyLFy+mEjAzZsyguXez2UwFGAcGBsLuqhgMTU1NaGlpQXFxMd1gLRYLbrjhBtjtdrz33nsRNVcyUmzbtg2vvfaa3+2HDh3CqlWrhvUYDQ0NqKiowLJly6BSqaDRaPCHP/wBFRUVePPNN/0k6gk8hx2jiBLKuOFfZ/6Fcz3ngt4fJ4uD3qIP+RgKkQIs47bvDSZB7ykmuTJjJb45+5vDXuNzzz1HPbXz8/PxzDPPYMmSJcP++68jfDt/PMHzPDo6Oqjs/ieffIK8vDyUlpaipKTEyweFKCN3d3fDYDBAo9FQcglnpHDhwgVcuHCBKt6S577ppptgMBjwwQcfQKOZWGO1SALP8zCbzdi8eTPKy8uRmJgIg8GAZcuWobKyEjKZDGVlZZDL5UFJJYpLiBLKOKDH1IPnvnguKAlopBq3f3qIQjzgHlwctA0GVQwGLolH5ibl4rr51w37tPTWW2/h9ttvx1/+8hcsXboUTz/9NN5++23U1dUhKSlpWI8RRXDwPA+tVos9e/Zg165dOHToELKzsym5ZGdn08/KZrPRyIUoIxMJmNFY9hJcvHgRTU1NKCoqglqtBuCeKr/11lvR2dmJjz/+GLGxX33lhOGgt7cXGo0GNTU1OHXqFPR6Pf7zn//g4sWLWLhwIV566SWoVKooqQyBKKGMA/bU7kFlV2XA+1RiFaxOq7dEfQDIhXK4eFdQv3jAXS8BgDRVGrYu2gqhYPglsaVLl2Lx4sV49tlnAbjzxtOnT8cPfvADPPjgg8N+nCiGBs/z0Ov1VK7j448/pgNypaWlWLBgAc3hE2Xk7u5u6PV6KJVKL8ve4aKlpQWNjY0oLCykEYjD4cDWrVvR3NyMgwcPBo20vo4IpCBss9mwc+dOPP3000hPT8drr70GtVo9LLXhryuihBJmDFgH8KfjfwoYVRALX88hxUBgwNBW4VCIkcaAAYM7i+6EQjz8zcZut0Mul2PHjh0oLS2lt2/duhX9/f3Yu3fvsB8ripGjv78f77zzDvV0SU9Pp+Ti6dvucDi8ZPdlMhmNXEIpI7e1tVEjspiYGABuIcw777wTZ8+exaFDh6JRaBCQmgghDbvdjrfeegsvvvgieJ7Hu++++7VOEQ6FaJdXmPF56+cByUTEisDx7ojEU0IlEGJkMeiz9IW8hgEDF+fC7YtuHxGZAO6JaCKM6Ink5GScOxe87hNFeBATE4PbbrsNt912GwYHB6mny4YNG5CQkECVkRcvXoy0tDSkpaXB6XSip6cH3d3duHDhAqRSKa25eCojt7e34/z58ygsLKRk4nK58P3vfx+nT5/G4cOHo2QSAuR9ZFkWPM9DLBbj5ptvht1ux4EDB4JquUXhRpRQwgiLw4LyznK/2xkwEAvEMDlMIY2vAHdNZDhf2hhpDK7JugaJisnr1Y9i7FCpVPj2t7+Nb3/72zCbzXj//fexc+dObNmyBUqlknaLLVu2DCkpKUhJSYHL5aKeLmVlZRCJREhKSoJAIMDFixe9IhOXy4Uf/vCHOHHiBA4dOhR2S+uvMhiGAc/zEAgEuOOOO3DzzTdP2dbqiUKUUMKIE20nAqoJk/SVp4RKIKglapgd5oDGWF7ggZUZKzE7bnSTwkQapLu72+v27u7u6IYziZDL5bj22mtx7bXXwmq14qOPPsKuXbtw4403QiwW08jlG9/4BpKTk5GcnAyXywW9Xo8LFy6gv78fIpEIp0+fhtlsxpo1a/Czn/0Mhw8fxuHDh6eM1lgkgZAKy7JRMhkGopWlMMHhcuCL9i/8bo+RxNBaiEqiCihLD7hbhI02I+QiechiPQB8I+MbKEgb/TCVWCxGUVER/vOf/9DbOI7Df/7zHyxbtmzUjxtF+CCVSnHNNddQT5fXXnsNDMNg27Zt1NPlww8/hMvlwv79+7F9+3YsWrQICxcuRF1dHe68807MnDkTb775Jn79618jLS1tsl/SlEV0zmT4iBblw4TjrcfxfoO3CKSvha9EIAnYtSURSMDxHK2xhGoTzknMwbcWfGvMX/K33noLW7duxQsvvIAlS5bg6aefxr///W+cO3fOr7YSReTA6XTi6NGj2LFjB/bs2QOr1Qqz2Ywf//jH+OlPfwqpVAqO4/Dwww/j/fffR3FxMQ4ePAibzYaSkhI8//zzVBAyiijCjSihhAEuzoU/H/+zV1eWXCiH1WWlJBHIHx5wt/6KBWJYnJYh6ytpqjTcUXAHRILw6EE9++yzdLBx0aJF+POf/4ylS5eG5bGjGH/s27cP3/72t3HVVVehsrISBoMB69atg9PpxGeffYZDhw5h/vz54DgOx44dw5EjR/Dzn/98spcNwD1w+dvf/hYHDx5EV1cX0tLScOutt+IXv/hFRIhTRjE6RAklDKjsrMSec3vov0WsCAzDeNVTZEIZLE6L398SDxMGDEQCUVBHR41Eg7uK7oJSMrQ2URRffRw+fBhXX301Xn31VVx//fXgOA5ffPEFXn/9dbz88ss4ePAgli9fPtnLDIr3338fb731Fm666SbMnTsX1dXVuOuuu3DbbbfhySefnOzlRTFKRAlljOB5Hs9/8Tx0Zh0AgAULqUgKs8NMr1GKlV7S8gSeGlyhdL0kAgm+U/gdJCu/Wqmoo0eP4oknnkBZWRk6Ozuxe/dur7mYKIJDq9XixIkTuOaaa/zum6qDd0888QS2b9+OpqamyV5KFKPE1PvWRRjqeuoomYB3k4cnmQBu0vGFRupBIDxobYXnea/rWYbF9Quu/8qRCeBWu83Pz8dzzz032UuZckhKSgpIJgCmJJkAbl+WuLi4yV5GFGNAtG14jPCUqI+RunW1PCEXymFyeLcKq8QqDFgv1VPUUnd9xW63++WP189dj3nx88K/8AjAhg0bsGHDhsleRhQRgIaGBjzzzDPRdNcUx9Q8ykQILvRdQNtAGwB30d2XTAD46WvJhDKYHWYv4Ugyd7Jv3z7s27cPTqf730unLUVxWjGAwFFOFFFEGh588EEwDBPyP181hvb2dqxfvx433HAD7rrrrklaeRThQDRCGQNIdEJmSHwhFUq9OruI/IpnW7BnfWXx4sV44403sHbtWsxPno91c9aBZVl0dXVFBw6jmBIYqXVvR0cHVq9ejeXLl+PFF18c59VFMd6IEsoo0TXYhQZ9AyQCCewue8BhRIlAAqvTbd/rKb/iBY/AQ61WQ61RQ9esw/VXXQ+TyYS//e1veOihh/B///d/uPvuu8fzJUURxZgxEuve9vZ2rF69GkVFRXjllVembO0nikuIEsoo8WnLpxAwArAMG3BYUSwQX4pOeEAlVfnNociFchgd7uiE53nExsZCIVQgvjMeFpMFt912G+rr6/Hkk09SMok6xEXxVUB7eztWrVqFjIwMPPnkk9DpdPS+aDQ+dREllFGgz9KHGm0NFGIFBu2BlYOJVD3gVg8ONLAoFAiBL2W7GMYdwfx0w0/x3P8+hzdffRMzZszAm2++iUWLFgEI3g5K6itRooliquCjjz5CQ0MDGhoa/DTGovXCqYtojDkKfNbyGVRSVVAyETJCWhdRS9QByUQikFAZe57nwYDBioQV6KjrwLlz57Bq1Srs37+fkgkQvB2UFDs5joPLFVy2JdJgNBpRWVmJyspKAEBzczMqKyvR0tIyuQuLYtyxbds22iLv+18UUxfRCGWEMNqMaOprgsEa3PxKKVai39YftFgPAFKRFDaXjUYdCxULcf/W+5GQkADAXaCPjY2Fw+GASBRYasXpdOLDDz+ETqfDmjVrkJ6ePvYXOIE4deoUVq9eTf/9//7f/wPgNvp69dVXJ2lVUUQRxWgRjVBGiFMdp0KaX7FgYXKYQhbrRawIg1Z3dMKyLKR9Utx3/X2IiYnBK6+8gu985zv4+OOP4XK5gpIJADQ2NuKzzz7Dyy+/jJycHFx55ZU4efKk33WhTn2TeSJctWpVwBNqlEyiiGJqIkooI4DNacOJthNeMyS+UEqU4HgODMOElKrnwIHjOBzZcwTP3fccNm7ciAMHDiAlJQXXXXcdjh49CpMpuHcKAGRmZuKXv/wljh49isbGRkilUvzlL3+B3e6u3RgMBnR2dtJ0GAHHcV51l9deew0OR+C1fh3w2GOPYfHixVCpVEhKSkJpaSnq6uome1lRRDHlECWUEeBk+8mAAo+esDltkIlktF3YFyzD0s6uJEUSNs/djKefehrPP/88RCIReJ5HVlYWUlJScODAgaDPw3Ec6uvr8fe//x2HDh1CYmIinnjiCbz33ntUC0mr1WLp0qV4/fXXveovLMvSAn53dzf+53/+B8eOHRvRe/FVwpEjR3DPPffg+PHj+Oijj+BwOLB27dohCT2KKKLwRlQccphwupx4+vjTAUUeCdQSt7d3qPoKEYFUipW4s/BOxMhivO4nbcHLly9Hfn4+tm/fHrBV+I9//CP+8Ic/YN68eeju7kZfXx+ys7PxxRdfwGKxUAK54YYboFAoaBrpb3/7G7RaLe6++26oVCp63VQVFBwP6HQ6JCUl4ciRI1i5cuVkLyeKKKYMojvIMFHZVRmSTABAyApDkgkDBhanBSJWhBtzb/QjE+BS6+9vfvMbCAQCr9sIrFYrnn76afz3f/83du7cibKyMuzZswft7e1Yt24deJ6nKa5t27bhwIEDOH/+PB555BHcc8896OjoAMMwYFkWv/vd79DZ2QmWZb3SYk6nc1je9l9FGAzuzzAqVBhFFCNDtMtrGOA4Dp+3fh7yGpVYBb1FH/IaYrJ1/YLrMU0T2t97zZo1WLNmTcD7rFYr+vv7kZaWhqSkJADuCWWTyYQrrrgCAoGAkkFmZiZycnKwZcsWiEQivPHGG9iyZQsA4OzZs3jkkUdQXFyM1NRUsCwLrVaLpKQkCIVfz68Gx3G477778I1vfAO5ubmTvZwoophS+HruGiNEja4mJFnIhDLqzBgUX0rUf3P2N7EgacGY1qNQKPCjH/0Iv/jFL9Dc3IzU1FT88Y9/hNVqpeq9JH21c+dOfPLJJ8jJycGbb76JnJwcuFwuCAQC/OMf/8D8+fOxdOlSGI1GvPHGG/j73/+OlpYWrFy5Ej/84Q+xZMmSMa11quGee+5BdXU1Pv3006EvjiKKKLwQTXkNA5+1fBb0PhErCqzR5QOVRIWcxBxcnnH5mNcjEonw6KOP4k9/+hPOnj0LgUCApKQkzJw5E7NmzQLgFt27++678cc//hF33303GIZBTk4OgEsptH/9619Yu3YtYmJi8M477+CJJ57A2rVr8fLLL4PjOPz0pz/F4cOHx7zeqYJ7770X+/fvx6FDh/ymt6OIIoqhEY1QhkBDbwM6jZ0B72PBQiQQDR2dAEhXp+PqzKvDti6WZXHzzTfj5ptvBgAsXLgQOp0OMpkMr732Gn73u98hJSUFu3fvRmJiIvbt24ejR49i5cqVYFkWTU1NuHjxIo1oYmNjwfM8brnlFsyZMwdXXXUV3nvvPUpQX2XwPI8f/OAH2L17Nw4fPvy1eM2jxebNm1FZWQmtVovY2FisWbMGjz/+ONLS0iZ7aVFEAKIRyhDwNNDyAv/lzAnHBZVgIUhWJqMkuwQCVjAOK3Rj5cqVuO666wAAWVlZuOOOO7Br1y584xvfQGZmJuLj4/HJJ5/Q61999VXMmzePRi0LFy7E9OnTcd111+Fvf/sbXC4XNmzYgIyMjHFbc6TgnnvuwT/+8Q+88cYbUKlU6OrqQldXFyyW0C3iX0esXr0a//73v1FXV4edO3eisbER119//WQvK4oIQbRtOATaDG14qfylgPfFSNzujKG84AFALpLjzsI7ESef3I6h//mf/8Grr76KY8eOITExEVlZWdi0aRP+8Ic/0AK8xWLBs88+i7fffhvXX389fvazn30t2omDiWq+8sorQ3p7fN2xb98+lJaWwmazhVR1iOLrgWjKKwSCRSfEnVHMiv0k6T0hZIW4MffGSScTAHj44YexceNGJCYmorW1FfX19Vi1ahWEQiFeeeUVLF26FPPnz8dPfvITxMbG4qGHHsKVV16JoqKiyV76uCN6phod9Ho9/vnPf2L58uVRMokCQDTlFRQ9ph7U9fjLbyhECqoSLBPJQsqwlGSXYEbMjHFb40hRWFgIl8uF6dOno7KyEqtXr6aKvw888AB27doFrVYLu92O3t7eqC9FFAHxwAMPQKFQID4+Hi0tLdi7d+9kLymKCEE05RUFLly4gGeeeQZvvPEGWJbFvHnzsGTJEvzhD3/4WqS8Ignbt2/H9u3bceHCBQDAggUL8Mtf/pI2T4wHHnzwQTz++OMhr6mtrUV2djYAoKenB3q9HhcvXsSjjz4KjUaD/fv3R/14oogSShTeqKurg0QiwcyZMyd7KV9LvPPOOxAIBJg3bx54nsdrr72GJ554AhUVFViwYGzzS8Gg0+nQ29sb8prZs2dDLBb73d7W1obp06fj888/x7Jly8ZlfVFMHUQJJYooIhxxcXF44okn8N3vfneyl+KHlpYWZGRk4NChQ1i1atVkLyeKSUa0KB9FFBEKl8uFt99+GyaTKSJO/ydOnMDJkyexYsUKxMbGorGxEY888gjmzJkTEeuLYvIRJZQooogwnDlzBsuWLYPVaoVSqcTu3bsxf/78yV4W5HI5du3ahV/96lcwmUxITU3F+vXr8fDDD0MikUz28qKIAERTXlFEEWGw2+1oaWmBwWDAjh078NJLL+HIkSMRQSpRRBEKUUKJIooIx5o1azBnzhy88MILk72UKKIIiWg/aBRRRDg4joPNZpvsZUQRxZCI1lCiiCKC8NBDD2HDhg2YMWMGBgcH8cYbb+Dw4cP44IMPJntpUUQxJKKEEkUUEQStVovbb78dnZ2d0Gg0yMvLwwcffICrrrpqspcWRRRDIlpDiSKKKKKIIiyI1lCiiCKKKKIIC6KEEkUUUUQRRVgQJZQooogiiijCgiihRBFFFFFEERZECSWKKKKIIoqwIEooUUQRRRRRhAVRQokiiiiiiCIsiBJKFFFEEUUUYUGUUKKIIooooggLooQSRRRRRBFFWBAllCiiiCKKKMKC/w/x/idOUc2JwwAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "rank of B 2\n" + ] + } + ], + "source": [ + "B_=torch.tensor([[1,0],[-2,1],[0,1]]).numpy()\n", + "plot_matrix_and_subspace(B_)\n", + "print(\"rank of B\",matrix_rank(B_))" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "id": "17a1bce9-b00c-4516-a17c-94c91b5fa9e6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "matrix_rank" + ] + }, + { + "cell_type": "markdown", + "id": "e7205fe3-ee86-4bdf-8ff8-a80974009792", + "metadata": {}, + "source": [ + "Here, you present the matrix ```A```. The rank of this matrix is also two.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "id": "16fcbaeb-94b8-4aee-af84-f4457cb31df1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/latex": [ + "$\\displaystyle \\left[\\begin{matrix}1 & 1 & -1 & 1 & 0\\\\-2 & 2 & 2 & 0 & 1\\end{matrix}\\right]$" + ], + "text/plain": [ + "⎡1 1 -1 1 0⎤\n", + "⎢ ⎥\n", + "⎣-2 2 2 0 1⎦" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "A=torch.tensor([[1,1,-1,1,0],[-2,2,2,0,1]]).numpy()\n", + "Matrix(A)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "b092ae8f-57aa-48ba-b68a-a9f271fe57ce", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "matrix_rank(A)" + ] + }, + { + "cell_type": "markdown", + "id": "6bea05a6-cb00-45e9-b99f-9e304da883a2", + "metadata": {}, + "source": [ + "For the matrices $ C = BA $, if $B $ and $ A $ both have a rank of $ r $:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "id": "b9afc203-763c-4809-b3b7-03890312fe3d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/latex": [ + "$\\displaystyle \\left[\\begin{matrix}1 & 1 & -1 & 1 & 0\\\\-2 & 2 & 2 & 0 & 1\\\\0 & 0 & 0 & 0 & 0\\end{matrix}\\right]$" + ], + "text/plain": [ + "⎡1 1 -1 1 0⎤\n", + "⎢ ⎥\n", + "⎢-2 2 2 0 1⎥\n", + "⎢ ⎥\n", + "⎣0 0 0 0 0⎦" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "C=B@A\n", + "Matrix(C)\n" + ] + }, + { + "cell_type": "markdown", + "id": "5874ff64-dd94-45fe-b54e-585ff8d386cf", + "metadata": {}, + "source": [ + " The columns of $ C $ will have the same rank as $ B $. Furthermore, the span of the columns of $ C $ will be the same as the span of the columns of $ B $.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "id": "ed566ee6-7f64-42db-8356-70e675c8e5e8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "rank of C 2\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAGRCAYAAABCCEUTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAD7vUlEQVR4nOz9eZBk2VkejD93z32tfenu6r2nZ+uZnp7uHgESBiGB+dgkQIQDZEdgfxjsHwH+A+wIRWCMMQHhwCYcAn/GSLZDtkMDg7AEEiAkjaSRxqOZrrWrl6rurr2ylsys3POuvz9undM3b93cb1Vlz9wnokOa7qqTNzPvPc953/d5n5cxDMOABw8ePHjw0CXY474ADx48ePDw7oBHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHjx48ODBFXiE4sGDBw8eXIFHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHjx48ODBFXiE4sGDBw8eXIFHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHjx48ODBFXiE4sGDBw8eXIFHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHjx48ODBFXiE4sGDBw8eXIFHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHjx48ODBFXiE4sGDBw8eXIFHKB48ePDgwRV4hOLBgwcPHlyBRygePHjw4MEVeITiwYMHDx5cgUcoHo4FhmEc9yV48ODBZfDHfQEe3lswDAOKoqBSqYDjOPA8D47jwHEcGIY57svz4MFDF2AM76jo4Yig6zoURYGmaahWqwBMgimXy6hWqxgcHPQIxoOHJxhehOLh0GEYBjRNw+LiIkRRxNDQEFiWBcuyMAwD+XweW1tbiMfjqFarYBgGLMuC53mPYDx4eILgEYqHQwVJcWmahlwuB5/Ph+XlZaytrSESiSAej0PTNAAAz/MwDIP+qVarkGUZADyC8eDhCYCX8vJwaNA0DYqiQNd1sCyL6elp5HI5qKqKkydPolQqIZPJoFgsgmVZjIyMIB6PIxaLQRAEAKghGF3XAYBGMIIg0DoMy7IewXjwcMzwIhQPrsMwDKiqClVVYRgGWJZFNpvF1tYWJEnCjRs3AJjEwDAMVldXsbq6CsMwsLi4iFKphHA4jFgsRgmGRCZWgqlUKnQdQjDk5zyC8eDh6OERigdXoes6VFWlaSyGYbC4uIiHDx9SkhBFkaayADPVJQgCLly4AACoVqvIZDLIZrO4f/8+KpXKAYIhaa96BEMiF49gPHg4OniE4sEVkJSUoigwDAMMw6BarWJ6ehqVSgUvv/wyVldXHTd1hmFq+lIkScLQ0BCGhoYAAJVKBZlMBplMBnfv3kW1WqX1l1gshmg0eoBgdF33CMaDhyOGRygeuoY1xQWYG/j29jZmZmYwMDCAF154ATzPHyAOgmYbu8/nw/DwMIaHhwEA5XKZEszGxgZkWUY0Gm1KMNVqFZVKhSrMPILx4MFdeITioStYe0vIhnznzh2srq7i8uXLGBkZqfn5ehqQdrQhfr8ffr8fIyMjtI+FEMza2hpUVaUEE4/HEQ6HwXEcfR0iYyb9MOvr6xgaGkIgEKCRDKnvePDgoXV4hOKhI5BNWVVVquIqlUqYmpoCANy8eRPBYLDmdxiGga7rBzbqepFLK2AYBoFAAIFAAKOjozAMg6rHMpkMVlZWoOt6DcGEQiHwPE/fx+LiIuLxOCURpxSZRzAePDSHRyge2oa1twQwe0TW19dx+/ZtjI+P4/z582DZgzZxVuKwbs5ubtQMwyAYDCIYDGJsbAyGYaBYLFKCWVpagmEYtMAfj8dhGAY4joMgCDSCUVUViqLUJRin9+fBw3sdHqF4aAu6rkOWZRqVaJqG27dvY3t7G88//zz6+/vr/m6jSOSw2qEYhkEoFEIoFML4+DgMw0ChUKAE8/DhQwDA3bt30dfXh3g8jmAwWBPB1CMY0gfjEYwHDyY8QvHQEkiKi6i4WJZFLpfD1NQUfD4fXnnlFfh8voZrNCrKH1V/LcMwCIfDCIfDOHHiBHRdx9e+9jWEQiHs7u5icXERHMfRAn88HkcgEKhLMIBzF79HMB7ei/AIxUNT2FNcDMNgaWkJ9+/fx+nTp3H69OmW0ladqrwOE0TdNTo6ikAgAF3XkcvlkMlksL29jYWFBfA8X0Mwfr//AMEoitLQJsYjGA/vBXiE4qEhrFEJwzBQFAUzMzPI5/O4evUq4vF4y2s1Io5ecQBiWRaxWAyxWAwAqAdZJpNBKpXCvXv3IIriAYIhhGHtxyERDMMwNQRDVGQePLzb4BGKB0eQtM78/DxCoRBGRkaQTqcxPT2NWCyGV155hfpttYpeSHnVQ73XJ+kvQpyapmFvbw+ZTAbr6+u4e/cuJEmiPxOPxyFJUs26hGBkWaY1GEIwVhWZBw9POjxC8XAAZAPUdR2lUgmCIGBhYQGPHj3ChQsXMD4+3tEG2MuE0io4jkMikUAikQAAqKpKCWZlZQW3b9+G3++vIRhRFOnv2wkGAHZ3dzE4OAhJkjwnZQ9PNDxC8UBh3eyIisswDCwvL4PjOFy/fh3hcLjr17DjSd48eZ5HMplEMpkEYBJMNpulEuW5uTkEg8EaHzIrwSiKgtu3byMWi0FVVW8WjIcnGh6heADg3FuytbWF7e1thMNhvPzyy7TbvFMch2z4qMHzPPr6+tDX1wfAJAxCMA8fPkSxWEQoFKIEEwqF6O+Rz5dIs71hYx6eNHiE4qHGPoVlWei6jjt37mB9fZ2mbbolE+DdkfJqF4IgoL+/n/bnyLJMCYZY9QPAgwcPkEgkqFU/8JhkPYLx8KTAI5T3MJzsU4rFIqampsCyLG7evImHDx+6ttn3omz4qF9fFEUMDAxgYGAAAFAsFvHmm29CVdUaq36SHiNW/UAtwXjTLD30IjxCeY/CqbdkbW0N8/PzOHHiBM6dO0d7NMikxG5ht16xksu7NUJpBqKUu3DhAjiO68iq32lcsjfN0sNxwCOU9yDso3lVVcXc3BzS6TSuXLlC8/+Au+moepvauznl1SrIZ9PMql9RlJYIxptm6eE44BHKewj2uSUsy2Jvbw9TU1MIBAJ45ZVXanooyM+QKKZb9GrK6zjRjEgbWfWvrq5C0zRHq35vmqWH44BHKO8RWOXABI8ePcLCwgLOnDmDiYmJlqYpdoNeVnkd9+u3al3TiVV/s2mWuq6D53n4/X6PYDx0BY9Q3uVwGs0ryzJmZmZQLBbx0ksvUZsRJ7hZQyHX4/Qax72hHxe6ed+dWPWHQiGwLFtDMMRO5uTJk940Sw9dwSOUdzGcCu+7u7uYnp5GIpHAzZs3m9qnkOZGN0CIQ9d1bGxswOfzIRqNepsV3En7OVn15/P5mj4YhmFqfMjIEDSSBrNPs3SSKXsE46EePEJ5l8LeW0JOosvLy7h06RJGR0e7cgjuBAzDQNM0vPXWW6hUKnTjCofDMAwDuVwO4XD4PbVZHWZkxjAMIpEIIpEIterP5/PIZDI1Vv1EmFEqlRyt+om03Jtm6aEZPEJ5l8Gpt6RcLmNqagqapuHGjRu0O7sVuEkouVwOxWIRo6OjeO6558AwDEqlEra2trC3t4dbt27REzT5EwgE3vWb1VG9P5ZlEY1GEY1GAYBa9d+9exfFYhFvvfVWS1b9zaZZegTz3oVHKO8ikFP+5uYmTpw4AZZlkUqlMDs7i5GREdrr0A7cqKHouk6jI1EUcfnyZVrTCYVCEAQBjx49wiuvvEJrAPZZJIlEAvF4vOkQr05wnJvfcdaOiFV/IBBALBbDyMgIterf3NysseonJGO36vemWXqwwiOUdwlIb0mhUMCjR48wPj6O27dvI5VK4emnn8bQ0FBH63ZbQ7FGR5cuXcLS0hL9NyISIBu69QR96tQpOosknU5jbW0Nd+7cgc/nq+vk+6TiuE/z5HtoZNVPPn8nq/5mBOMNG3vvwCOUJxzW3hLDMMBxHDRNw7e+9S3wPI+bN2/C7/d3vH43Ka+trS3MzMxgcHAQly5dQiaTabiW/d/sG5yTk28oFKo5QZP0zJOCXlC3EUKxo1OrfjvBeNMs3zt4sp4+DzXQdR2qqtaouLa3t6EoCk6cOIEzZ850/bB2QigkxbWysoLLly9jZGSk4VpkM2v2OnYnX1mWqUTW6oNF0mORSMQVU8vDxnFHKLqut3QNnVr1O02zdBo25hHMkw+PUJ5AOPWWqKqK2dlZpNNpsCyLc+fOufJa7dZQmgkA3OyUF0URg4ODGBwcpK9NCGZ9fR2qqtImv0Qi0VBB1guRwnGhXoTSDO1a9cdisRqZuvU+XlhYgM/nw9DQkDcu+QmGRyhPGOz2KQzDIJvNYmpqCuFwGC+88ALeeust116vnRqKPcVljw6aRTvdbup2mxJrF/ny8jIA1DT5BYPBY9+sOt3Me/EaWrHqD4fDNQRDUpTlcpn+f2sEQ5owrUX+4/68PNSHRyhPEKy9JSRd8ODBAzx48ADnzp3DyZMnUalUoOu6a5tEKymveimuVtdqNeXVDpy6yO09GERBRuzgu51G+aTisEjNbtVfrVYdU5TxeJz+f+shxClF5hFMb8MjlCcATr0l1WoVMzMzKJfLuHbtGu0tsG7OR0Eo7fS4NHIbPmxYm/xOnjwJXddpgVnXdczMzNQomBKJxJEoyN5NEUozSJKEoaEhqji0WvUXCgXkcjns7OwccFIm1wjAkWC8WTC9A49QehxOo3l3dnYwMzODvr4+XLlypUbZZC2AuoFGNZRmKS6ntQ4z5dUOWJal5LG6uopnn30WmqYhnU5jeXkZt2/fRjAYrFEwPWkKslZxXKRmteovl8vo6+uDIAg1NTBi1W8XWXjTLHsT784n5F0C8rCQqMQwDNy9excrKyt46qmnMDo6euB3CKHoun5oY3tbTXHVW8veSX0YKa92QHowYrEYVTApikJPz4uLiyiXyzQ9E4/Ha07P3aAXxAC9ECUBjyOYdq36AW+aZa/AI5QeBElxERUXy7IolUqYmpqCYRi4efMmNfWzgzwsbjkE24vyh2HjctyE4gRBEGry/9b0zPz8PB10RSTK4XC4Y7nrcW9wvUAo5NBE0IlVf71ZMN40y6ODRyg9BqcU18bGBm7fvo3R0VFcuHCh4cZ1mCmvdlNcTmvVu65ef6it6Rlyek6n01RBZrWJTyQSLSvIeoFEe4FQyMGpHtyy6vemWR4uPELpIdhH82qahvn5eWxtbeHZZ5+lp+VGIOkkN+fA67qOO3futJ3ickKv1FC6gfX0TDa3QqFAN7eHDx/W1GiIyeJxihIaoRcIpdXmSoJOrfobEYyu6ygUChgYGPAIpkN4hNIDsPeWsCyLfD6PqakpiKKIV155pS1TRDcJRVEUKIqC3d3dtlNcTtdFSMP+kB73kK1uB12Fw2GEw2FqE09MFlOpFDVZJOkx4oHV7eu6hWbRwVHAnvJqF61a9VsJJhAI1BBMsVjE9PQ03ve+99E1vXHJ7cEjlGOGfTQvwzBYXl7GvXv3MDExgTNnzrR9A7s1FGtrawvT09NgGAbXr1/vugj9JKe82gFx8Y3FYpiYmICmafTkTDywAoHAkUmTm6FXIhQ3Sa2eVX8mk8HW1laNkzUhGSJkEQSBRi/WccneNMvm8AjlmEBu1rW1NWxvb1NL99nZWeRyObz44ovUlK9dsCzbVYRiVXGdOXOGnu66hbX47kQsvXBaPwxwHFfjgWW1KFlfX0e1WsVbb71Vs7kdpQdZLxDKYUdJTiRPCGZjYwN3794Fz/N0miix6gdQkyIjQ+EqlYpHMA7wCOUYYC28K4pCi4vT09OIRCK4efNmVyfXblJe5XIZk5OT0HUdN27cAAAsLCx0fC3262r0b+9WQrHDalHS39+P+fl5jI+PI5PJ4M6dO5BluUa9FIlEDnWz7QVCabeG0i2crPpXV1fx6NGjtqz67eOSSYrM6kN23J/tUcIjlCOGfTQvx3EolUp4++23cf78eZw4caLrG7DTlJeTiqtUKrmqGAPcNYh80kE2c9JB7tR/oev6AfWSm59XLxDKcddxOI5DMBiEz+fD1atXO7bqf69Ps/QI5YhQzz5lcXERiqLg+vXriEQirrxWuxFKo0ZFEjm4sek06zc5zsbG44S9ydPef1EsFqlEmSjIrBLlRgqyVnDchELSv8f9PVjrOHarfkVRKMG0atVfj2DezdMsPUI5Ajj1lmxvb2NmZgaRSAS6rrtGJmT9VgnFnuKyq7jc9AZrFqG8V1JeVjR7z1Z5rF29RIrLgiDUeJARBVmrOO7NnHwGx725NhIGCILQkVX/e22apUcohwx7b4lhGLhz5w7W1tbw1FNPwefzYWZmxtXXbDXl1UqjopuNkl7KyxntvHenMcnWMb3z8/MIBAI1qRnrDBI76sm4jxLk8HPcmylJQ7cCJ6v+TCaDbDbb0Kr/3T7N0iOUQ4JTb0mxWMTU1BQYhsHNmzcRCASQzWZd6xkhaJbyaseL6zBsUUgjoKIoiEajrnf3P0no9j07jem1NljOzs4iFArRHphoNFpjctkLhNJLEUqn6jr7sLdGVv1EbWYlDCvBlMtlLCws4MKFCxBFETzPI5PJ1CjPehUeoRwC7L0lALC+vo7bt29jfHwc58+fpzdStxJfJzSKUKwprkaeYARuEgpZ6+HDh1haWgLHcbTgrGkaSqUSotHosWxux0lmbr5fnucPnJxJ/eXu3buoVqs1Dr4kxdkLEcpxR6lu9sI0suonSj7yPVit+olDxtbWFi5evEgbi3/iJ34C//gf/2N8/OMfd+X6DgseobgI60AgUnPQNA1zc3PY3d3F888/Tx90gsMiFKc1O/HictNsUlEUAMDGxgZeeukl+Hy+GsO/u3fv4sGDB/Q03Uk9wEMtRFGs2dicxiQDwOrqKvr6+hqOST4svBsJxQ67F5yVYOxW/URkYY0kSY2m1+ERikuwF94ZhkEul8PU1BT8fj9u3rzpaJ9yFITSqd08WQvo/gSfzWYxOTkJALh69Sr8fj8URaGWJevr6zh79iw4jkM6nab1ADKTJJFI1IyMfbfgqBVW9jHJe3t7eOedd5DP57G6ugoANfWXQCBw6NdHJMPHTSiaph1JQynDMAe+ByvRr6yswDAM3Lp1C8vLywiFQiiXy02zCd3gk5/8JD75yU/i0aNHAIDLly/jE5/4BD784Q+3tc676+k8Jth7SwDg0aNHWFhYwOnTp3H69Om6DwtJT7l5OrLWUNpNcdVDp4RiGAa1kjl79izu3r1b96ElZn6k2YwoadLpNM1Dk1NcIpE49Ia/dzuIRBkAnn76aTAMg0KhgHQ6je3t7QP2JIlEoi1PuVZx3Coz63Ucx/1kl4pnMhnMzs6iv78fn/nMZ/A//+f/RKVSwW/8xm9gZmYGH/jAB/DCCy+4Sn5jY2P4d//u3+HcuXMwDAOf/vSn8SM/8iO4desWLl++3PI6HqF0AafeEkVRMDMzg3w+j6tXr9LNsR6sA7HcupkJSXVrNw88di/uhFBUVcXc3BzS6TS1krl3717LKi+7ksZ6ipuZmaH1F5Iia9Uyvpdw3EIEa1GeZVlqsEgUZMSeZH19HXfv3oXP5zvQ3Nctjmsjd7qORoq4o76OsbEx/O7v/i5++7d/G+fOncN3fdd34Zvf/Cb+7b/9t/jBH/xBfOYzn3HtNX/4h3+45r9/67d+C5/85Cfx7W9/2yOUo4BTb0k6ncb09DRisRheeeWVlm5OK6G4BYZhkEqlkM/nu7abJ+u1e32FQgG3bt2CJEm4efMmrYVYycm6mbZCWvY0AbGMJ26y5DRNVE+t1l+Om4R6QWHldA1We5LTp09DVVXae0Ga+0KhUI0HWScpyePukifQNO1QIrBOrsN6+GNZFtlsFv/kn/wTnD17lpqNHubrf/azn0WxWKT2S63CI5QO4NRbcv/+fSwtLeHixYsYGxtreZNwm1DK5TJ2dnaoNNmNvGu7Vi7r6+uYm5vDyZMncfbs2QOT+Nwwh3SyjN/b26upvxBH316eCd8LEUqrdiA8z9c095HeCydpLElJthIVv9dTXnbYCaVSqUDTNFqUJ2ajbmNmZgY3btxApVJBKBTCa6+9hqeeeqqtNXrvCethWHtLyKmqUqlgamoKqqri+vXrCIfDba3pJqGQFJcoiujv73etiNdqyosM4trY2MBzzz1XdyDYYTQ2WgdaAbX1FzITvlfrL8cdoXT6+vbeC6JcSqfTmJubg6qq1OQykUjQCYp29MpG3ivXYSeUYrEIAIeu8rpw4QImJyext7eHV199FT/3cz+Hr33ta22RikcoLULXdZTLZUxPT+O5554Dy7JIpVKYnZ3F0NBQxzUKALQfo5trs6q43A6HWyEUUvw3DAM3btygxd5W13LbesVef6lUKrQfw15/0XX92CKFXolQ3IBdGmuVhC8vLwNAjcklqXn1UsrrKMcGtHodhUIBLMseelOjKIo4e/YsAODFF1/EW2+9hf/wH/4D/uiP/qjlNTxCaQJrb4mqqtja2oKqqlhYWMDGxgaefvppqvHvFN1Ih51UXLlcztWNqlkNZXt7G9PT0xgaGsLFixcbPpSNiOMwN1efz4eRkRFafyGGi+l0GoqiYHp6GslkkqbIjjKX/qRGKI3AMAdnwNsnKJKaF5k3ctzo1QilVCodi+BE13VUq9W2fscjlAaw26eQHPz//b//FzzPU/uUbtEpoZAUl30jJ2ozt1CvhmIYBhYWFvDo0SM89dRTGB0dbbpWo7nqR7Wp2A0Xv/GNb+DkyZNQFIXOwvD7/UdSfznujfSo6hfWEb0nT56kNS8yJrlcLuONN96okSgf9TTLXiWUQqFw6ITy67/+6/jwhz+MEydOIJ/P4zOf+Qy++tWv4ktf+lJb63iEUgfW3hJStNzY2AAAJJNJXLx40VWZr1t284C7M+XJevaNT5ZlTE1NoVwut1U7apTyOi6QAn8sFsPp06cd6y/hcLjGD8vNjefdGKE0g7Xm5fP5sLm5iZMnT9L02O3bt2lT61GJKnqJUKwK0WKxeKhNjYB5OP3Zn/1ZbGxsIBqN4tlnn8WXvvQlfP/3f39b63iEYoNTbwnpp8hkMmAYBidPnnR9/nUndvP1VFxuzZQnsBNUJpPB5OQk4vE4rly50taDTgjFaRM77tM6gVP9hRSb19fXoWlazTySbk6Px/2ej3sWCrkGlmUPzB8h9RcrqRNyId5XbqKXaijWlCshlMP8nv74j//YlXU8QrHAqbdkb28PU1NTCAaDuHnzJr7+9a8fmfeWHfVSXJ2u1yqsUt+lpSXcv38f586dw8mTJ9u+yY+qKO8m7MVm68CrBw8e1PS/dFJ/eS9GKFY4RQaCIGBgYIAqBa3eV/Pz81AUpUa1Fw6Huz7k9VKEYld5HXaE4hY8QtmHruuQZbnmpnr48CEWFxdx9uxZnDp1ik5cI4TjFpoRQLteXG6nvEiURiSFrTgANLq2Xkx5tfOz9oFX1nkk7dZfjptEe4FQWlF52UmduCak0+kDY5I7jRp7lVBIDeVJwHueUEiKizgEsywLWZYxPT2NUqmEl156CbFYjP48sZd2E40IpRMvLrdTXrqu4+7duwiFQrh582bXxdJeT3m1A2stgHSTO6VqGtVfvAilPWGA05hk4ppgHZNsrb+0Mia5m3kobsIpQnkSnIaB9zihOKW4dnd3qYTUqT7Qbc+IE5rZzbcix21lvU6wtraGQqGAgYEBXLlyxZUxwL2Y8nLrte3zSOz1F1VVa5RMbt9L7aJXCKWbyMDJNYF4kKVSKdy7dw+iKNZEjXZbHtKL1IsRSqlU8gil1+Fkn3Lv3j0sLy/j0qVLGB0ddXzQDitCsa7Zjd084E7KS9M0zM/PI5VKIRQKYXBw0JWNx0oc1vWOe1M7LDjVXwjBPHz4EIBJQuvr64fm5tsI7wZCsYNlWToVcWJigo5JTqfTWFlZwe3bt+mYZDIWgXwGvUgoXsqrh+E0mrdcLmNqagq6ruPGjRsNTwOHHaG4YTffbcqrVCphcnKS+oHNzs66doJvtHk9iSmvdmCtv4yPj0PXdSwsLGB3d5e6+fr9/pqN7rDdb3vhVH7Y12Afk0xk4URUYU0p7e3t0WbL44I99VYsFjuuWR413lOEYh/Ny7IsNjY2MDc3h5GREVy4cKHpjXSYA7E6TXG5eY1bW1uYnp7GyMgI7bVxMx3Vqymv4wCx0wiFQnjmmWc6qr90i16JUI5yA7fLwqvVKlKpFBYWFnDv3j1Uq1XqQRaPx4/c980p5TU+Pn5kr98N3hOEYrVPIeG1ruuYnZ1FKpXCM888Qw3umuEwVF4Mw2BrawsPHjw4Nrt5Xddx//59LC8v4+mnn8bw8HBX6zW6tl5TefUKWqm/EP+xbvtfCHqFUI5zDokkSUgmk3jw4AFu3LhRM3eHKMii0Sgl9lAodGifGREJWQmsWCy64shxFHjXE4pT4b1QKGBychKiKOLmzZttma65HaEQu3kAx2Y3X61WMTk5CUVRHFN+bqrGjsvLq1fRaENvVn9hWZZucp3WX3qBUHrhGkiU5KQgI587UZBZJ4u6PSaZ7C2eyqsH4TSad2VlBXfv3sWpU6dw5syZtkNZNyMUkuKSJIm6r7qBdkgvnU5jamoKiUQCL774omPPhJfyOn441V9yuRzS6TQ2NjY6rr/0ymZ+3HUce1RA4PS5E5NLMiZZEIQagulGWEH2Fo9QegiGYaBaraJarUIQBGqWODc3h2w2ixdeeKHjATVuqLzsKq58Pk9FAm6glRSVYRi0cfPChQsYHx8/MuNGwzCws7ODnZ0dOunvOAnluPtAOoFVyQSATlO0+48RgqlXf+mF4Va9QCitXgPLsohGo4hGo3RMslNjq5Vg2knnWTMpBJ5s+BhBUlxLS0vY2dnBiy++iGw2i6mpKYTDYbzyyitdNeZ1q/JyUnEtLCxAluWO17SjWYqKzL3P5XK4du0aotFow/Xc7rzf3NxEOp1GMpnEnTt3oCgKeJ5HMBhEPp8/1Bx1L8KN92qfplitVqk9DBl2Reov1jpAL0QovaA061QYYFeQWYn94cOHmJ2dpWOSCbE3ck4gBXnynZCUm0coxwBrbwnP89A0DQ8ePMCDBw869p6yg3TSd4JGdvNuW6XUWy+Xy+HWrVttdb27VUNRFAWFQgEMw+DatWuQJAkMw6BcLuP27duoVCp455136CzzdmfDP4k4rKhMkqSW6i9EsHKc6IUoqV7Kq13UG5OcTqdx9+5dVKtV6kHmpNxzMqj0COWI4dRbYhgGcrkcqtVqS6fwVtFJhNKsUfEwzBzt6xmGQWetnz59GqdPn275IXYjHUWIjGEYTExMIBwOQ5ZlFOUifJIPoVAIgiBgYmKC1gbI9QaDQUousVjsUCSmx1m/OezNtFH9ZXNzE5VKBd/61rdqPuOjVF09SSmvdmEfk2xVkFmVe4RgVFX1COU4Ye8tYRgG29vbuHv3Lm3Mc3OOQrs1lFbt5t2OUKwbpKZpuH37Nra3tzuqH3VLKOvr65ibm8Pp06eRyWToA3M3dRd/OfuX0A0dfIXHicQJhAfCGIgN1MwmISc8kh4jEk4yp/y4T7fd4DiIzFp/YVkW+Xwew8PDSKfTtNHP6uR7GP0vVvRKyusorsHv98Pv99PJoaVSiaYml5aWaLS2uroKABgaGvLcho8C1t4Skgc2DAN37tzB6uoqTpw4gc3NTdeH8rQToRyX3TwhFHLDTk5OguM43Lx5syMFSqc1FF3XcefOHWxsbOD5559Hf38/3nnnHWi6hi/PfxnfuP8NSgabuU2s5ddwN38XYV8Yp5KnMNE3gYnkBLUxJy6zZHTvo0ePaOrmSU6PHbcogOO4juovbqFXUl5H3R1vHZM8Pj4OwzDw6NEjrK+vY3t7G7/wC7+AVCqFoaEhfP7zn8eP/diP4cSJE65ew2//9m/jz/7sz6iY4ObNm/id3/kdXLhwoaP1nkhCsfeWMAyDUqmEqakpAGY/h6qqWFtbc/21W9n8rSkue5Ngp2u2A7pJb25ibm4OY2NjOH/+fMcnsE6UbZVKBZOTk9A0DTdu3KCNWVWtis/Nfg45I1f3d/OVPGbWZjCzNgOGYTAQHqDkMh4fx9jYGMbGxmpSN0eZHnMTxy2VdirK2+sv1lO01cmXEEw7fVxOeDenvNoBwzAQRRHBYBDPP/88vvKVr+Av/uIv8C/+xb/A//pf/wu/+qu/ilOnTuHf/Jt/g5/+6Z925TW/9rWv4Rd/8Rfx0ksvQVVV/Mt/+S/xwQ9+kE7MbBdPHKHYe0sYhqEplbGxMVy4cIGG8W53tAPN+1A6tZs/jGudnZ3FM888g6Ghoa7WaTfllU6nMTk5ib6+Ply+fJlu6uvZdfz5/J9DZdUDNa16J1TDMJDKpZDKpfDtB9+GwAkYT4xjIjmB032n0R/rd0yPkQKotbO83sn6uE/Hxx2hNHp9+yna6uRL+l98Pl+Nk2+79Zf3UsqrGayRkt/vx0svvYRisYhvfOMbKJVKeP311121YfniF79Y89+f+tSnMDAwgLfffhvf/d3f3fZ6TwyhOI3mtdYGnnvuOTrdDXi88bsti2wUTfSC3TyJDAAcmOXSKVolFOtER3tvy+TKJP5q9q9QkAt101KtvIaiKXiw/QAPth/gy/jyE58e64UIpZ2N1O7k6ySTJf5jrdZf3qspr1aug9iusCyLcDiMH/qhHzrU19/b2wMAKoNuF08EoTjZp+TzeUxOTsLn8+GVV145UBsgX4rbhOIUobSb4rLDLULZ3d3F1NQU+vr6kM1mXbNCb6WGoqoq5ubmkE6nayY6arqGL819CW8vvU3XchP29NhYbAxj8TGcSp5qOT2madqhRIitopcjlGZw6n8hUWKr9ZdeiA56dbhWoVA4MoWXruv45V/+Zbzyyit4+umnO1qj5wnFPrcEAD0FN5K/ki9FVdWuJwxaYU9PuWU33w2hGIZB+20uXbqEkZERrK+vu3b6bdaHUiwWcevWLQiCgJs3b9KTf66cw6vvvIq1TG0t67CsV0JiCOt761jJrOBbD77VcnpMlmXMzs62lB5zG09ahNIMkiRhaGgIQ0NDLddfeiXldZwGldbrOK558r/4i7+I2dlZfOMb3+h4jZ4lFGtvCbnhSId3Pp9vOtecfCluN21ZVV69YDcvyzJmZmZQKBTw8ssvIxKJdL2mHY02e2J3by/8P9p9hNfeeQ2FauHAWvXQzeYa9UeRK+dg4PEaTumxieQEJvomcCp5iqbHMpkMtdE4jvTYkxyhNIJT/SWfz9f4j/l8PmrbL0nSsW3qvRAlAeYB2CpyKJVKrrhKN8Mv/dIv4fOf/zxef/11jI2NdbxOTxKKrutQVbUmxZXJZDA1NYVoNNpShzfDMGBZ1lWPLHItqqrizp07Hae47OjUzmVvbw+3bt1CJBLBzZs3ax7Gw7acNwwDCwsLePTo0YHP4NsPvo0v3/nykXRgcywHv+jHXnmv6c/mK3lMr01jem0aDMNgMDyIib4JZItZnOZPY3h42DE9dufOHQQCgUNRj/VChHJUhGb1wbLWX6anp7G6uor79+8fmP9yVGmoXqmh2COUw57WaBgG/tk/+2d47bXX8NWvfhUTExNdrddThFKvt4RsXM1MDO04jOmKJP22u7vrmt18u5u/YRjUNfnMmTOYmJg48Jm4bTlvvT5ZljE9PY1SqYTr168jHA4DMCOCL819CXc279R9P266DfsEHwwYKFQKzX/YBsMwsJnbxGZuE6urq5guTOPC2IWW1WOkuTKZTHadHjvOCOU4C+I8z9Pi7wsvvAAA9HO+fft2TRf5YacheyVCcSrKHyah/OIv/iI+85nP4HOf+xzC4TA2NzcBANFotCM5eM8Qit0+hWEYVCoVTE9PQ5blmo2rVbg9DIukdwDg2rVrroXnJD3VymnRWvx+8cUX66ox3Ex5WcmJWKiEw2HcuHGDfgbpYhqvvv0qUrkUAEDkRfgFP3RDR1kuQ9XdjRTDUhglpQRNd+f7VXW1Jj0W8UWoesyaHgNA6wLpdBpLS0s1dYHjmAvfDY7bHJLcVyzLQhCEuvUXkoZ0s//Fil4iFOt1HHZR/pOf/CQA4P3vf3/N3//Jn/wJPv7xj7e9Xk8QirW3hKSqUqkUZmdnMTg4WHdORzO4RShWFdfFixcxNzfnunIMaP5wk8Fg9uK3Ew5jhsna2hpu3759QAxxP3Uffz7556goFfo7sipDVk0TTQYMAmIAAiegKBRR0Sp03U4Q8UeQL+dr6iVuI1fJYXptGnPrcwhIAQTFoClN7jvYXJnP5w/MhSfk0mw++XEXpI+bUKyWSVa0Wn+xEkw3B7xeSXlpmlaz15EaymHB7ZTrsRKKYRiQZRnVahU8z9NT9e3bt7G+vo7Lly93VZ9wg1DsKi6fz4e5uTnXrVKAxqekjY0NzM7O4sSJEzh37lzTTcjNCMUwDBQKBdy5cwdXrlyhElHDMPD6/dfx9ftfb3hjGjBQkksAgLJaBsuwCPvCMHQDFaUCWZNbIkCO4RCQAsiV63fZd4J6G6rEm27I+Uoe+Uoem7lNfOvBtyDyIsbj45joM9NjfdE+RKNRmh4jfRnW+eSEYMLh8LH3XFhx3IRijVAaoV79hajHrP0vndRfejVCeZKMIYFjJBTSW7KysoL19XVcu3YNxWIRU1NTYFkWN2/e7HqOcreE4qTiIg+Am6k0K6HYQfyw1tfXDzRvNlvTDUKpVCp4+PAhVFXFK6+8Qr+TilLBn0/+Oe6n7re9pqZryFfyNCKVeAkhKQTO4MAxHDTj4Gdr3dyPAkEpiKpSdUzVyaqMxe1FLG4vAgAivgiNXk4lT9XMhS+VSrQu4JQeey8V5Z1QL0Jphkb9L/Pz822biPYSodiL8uQ9Pgk4FkIhkQkJ7zRNo81mrZ7AW0GnhNKoUZGk5I6CUEh0ZBhG2wTrRsqLWKiEQiHwPE9fP5VL4dW3X0W6mG57TafrqqpVFKoFVOUqIoggKAXBszwUTUFJLiEkhVBRKq7XYeqh3ZRarpLD1OoUplanatRjJD02OjqK0dFRx7QNx3Hw+/3Y2dlBLBZz3cy0GXqBUBiG6foanPpfCMFY6y+EzO31l15KeVmvo1QqtV07Pk4cC6GQTZnkj4vFIu7du0cdad1CJ4TSSqOi2+ox8kBZ19ze3sb09DQGBwdx6dKltm/2biIUu4WKKIp48OABAGB2bRafn/48FE3paG2yvh2EaHRDR7FapH8fD8Sh6RoCYoCmxw4LDMMgLIW7SqlZ1WON0mMTExNQFAWzs7NQFAX3799HpVI58vRYLxCK25GBtf5irXOl02mkUincu3cPkiTV1Ll6NUIplUpdZ2qOEseW8mIYBrlcDrdv34ZhGHjllVdcbx5rl1DasZt326rDqvQiMumnnnoKo6OjXa3XLlRVxezsLDKZDPUC29ragqqp+Ou5v8abD9/s6HoIWpZ8sxyCYhCZUqbm7/2iHyInQtVVlOUydMMdYudYDgExgFzF3frMgfSYP2JKk/tP42TiJPx+P62/WE/Vy8vLYBjm0NVjx00oRyFKcKq/7O3t1fiPAcDq6ioGBwePtP/FCvL8H5f1ihs4NkJZWlrC/Pw8xsbGsLa2diidyK0SSrteXIfR38KyLCqVCubn51EulzuSSVvRSWMjsVARRbFGRVaSS/jSgy/Bt3c0cliRF8GxnOPmXpbLKKMMwPzMgqKZHquq1RqVWVuvx4lgwNRERoeFXDmHxa1FzK3PQTM0GEUDE30T4KIcxuJjDdNjVvWYW+mx4yaU4+iD4XkeyWSSDpqrVCp44403oKpqR/UXt0D2KnsfikcoLSAYDOKll16CKIpYXl4+lBu7lUiil+zmp6enEY/HXZky2W5jYyqVwszMzAELldXMKj7zzmeQKqRwMnmyq2sCHqe2nGSihmHQegmRHDeCrtemx0RehI83mx1b7X0J+8LY0DagaAoEHL7tR9QfRb6Sp5HVTmkH2fUsHpYeQuRFnEicoM2VyWiy5lRNohc302O9QCjHnWoicuMLFy5AEIS26y9uwYlQDls27DaOjVD6+/uhqiqq1SqdLuj2jc3zPKrVat1/79SLy80IhdQrFEXB+Pg4nnrqKVc+h1ZTXoZh4P79+1haWjowO+WdpXfwxuIbLW3u7YKMI+A4jr7fkBhCsVrsuL/EqfeFZ3louoaSXDqwbsQXMTd3HL5FDGCmu+wWMVbSl1UZC1sLWNhaoD9/uu80JpIH1WNWa/7l5WUAqPEeazU91gt9MMdNKOQ5IfOV2q2/uNXgbJ3xBJifTbFY9IrybV3A/kncbVdgoH4k0St289Z6hc/nw8DAgGuk2orKS5ZlTE1NoVKp4MaNGzS0VjUVfzX7V5hcmQQAKLoCgRUQ9Ufp5txp7YJc19bWFvL5PDiOQ8AfQEgKIVPNIAR3wntr7wsA8CwPv+gHy7CoKBWIvOh6vaQeOIaDX/K3XezPlXOYXJnE5MokGIbBcHSYmluOxkZdSY/1QoRy3H05VkKxw15/0TSN9hk9evSI9r+Q6KWb+ouT0sxLebUJ8iUe1XRFN+zm3WiYtM5zuXnzJr7zne+43izZaD1iLBmNRnHjxg264WRLWbz6zqvYyG7Qn2UYBlW1Sk/XHMshLIXBMmzbtQtN01AqlSCKIsbGxsAYDOSKjO29baiqivX1dfj9fvj9fkiS5Npmo+oq8pU8RE4Ex3FQNAURXwS6oYNlDu+ELHJmPagTvzErDMPAenYd69l1fHPxm66lx3qBUI47QrE6dDQDx3E19RdZlqk9jL3+Eo/H20pFOhFKqVTyCKUVkA+ZYRhwHOe6KzBwcON3026+G0IhI4tPnTqFs2fP0pvZbSlyvfVWV1cxPz9/wFjywc4DvPbOazUneyeQxkQCiZfgE3zQDR2lasmxMREwyXxvbw8cx2F0dBR+3o+qWgUv8tBZ0+E3EomgXC5ja2sLhmFQcvH7/V3XlYJSELIqoyybRX1ChIZhICgGEfFHIGuP/71bBMWguZ7SeL1ONnR7eizqj+L84HmMxcbaSo+RzfS40Cspr06vQRTFmv4X8llnMhksLS0BQI1Sr1H9xU4ouq4f6TwUN3DsEQoA2tzoNgihdJviclq3k81f0zTcuXMHm5ubB3pu3CYUp6K8ruuYn5/H5uZmjYUKAHxz4Zv46r2vOl5Ds/RZVa2iqpq1KpZhEZACEFihZnPe29vD7u4uAoEADMNA1B9FtpitqWswDINQKERPZNVqFeVyGYVCAbu7u+B5Hn6/H4FAAD6fr62N0F4Mt8KAgaJchFE2r0XgBPgF88EvK+WOem4ivggK1ULT1KBbnfK6oePt5bfx1qO3DqTHxmKP1WOGYVBr/o2NDZTLZdy5cwf9/f2uqsdavu4eSXm5QWoMwyAQCCAQCNTUXzKZzIH6CynyW9P8TsO1AHg1lHbhtiuwdV1VVfHmm292leKyo5MIpVQqYXLSzIXfvHnzwEnlMAjFul6lUsGtW7do1z15/apSxV9M/QXubN5puF6rGx+JUgh4lsfe7h4qhQpOjJ1AuVKGIRvIVXIHyMT+GpIkQZIkxGIx6LqOSqWCUqmEnZ0daJoGn89Ho5d69TcGDML+cEvzUggUTaEkwoBBQAhA4AVomoaiUmz6WUT90bZer1vYO/vt6TGJl8z02H73vjU99s1vfhOjo6OoVqs16bF4PI5kMnnozZW9kvI6jL4Ta/2FDHAj9ZelpSXMzc3V1F9kWXYkFC9CaQHWm5Tn+UNJeeXzeRQKBYyPj3eV4rKj3QiFpNqGh4dx8eJFxwfoMLrvyXpk1vzAwEBN1/1OYQef/c5nsVPYabpWJ1BVFaubqwCAoeEhBHwBsDqLglxAUAy2ZW/Csiw9/QHmXJpyuYxyuYxMJkMtTMgflmXBszwkQequ8x0GSkoJ2A9SSAMkwzKoKo8jM8CMzoJS8MjIhAGDsK95Z39VreL+1n3c3zJ916L+qKke65uArMmIxWKIxWIAatNjKysrAEBP1IchmX3SU17toFn9RZZl8DyPR48eIZ/PQxRFiKJ4qNNCX3/9dfzu7/4u3n77bWxsbOC1117Dj/7oj3a83rsyQiEpruXlZQiCgMuXL7u2NtB6hKLrOhYWFrC0tITLly9jZGSk4ZpuRyiqquLhw4dYWFjAxYsXMT4+Tv/9zsYd/MXUX9RsiPXQiS9YpVLBxsYGgsEg+vv7EfKFIKsyCnIBBbmAolwEy7LwC35wLAel2l5aSRAECIKASCQCwzBQqVRQLpeRzWaxvb2NkD8Ef8APXuRdfSA1XUO+Wls/kgQJ0M3orBPzyk4Im2M5+AV/R0q1vfIebq3cwq2VW1hdWcVd9S6eGn/KVI9FRx3TY5ubm7h37x5Vj5GUTbfpsXdTyqtd2OsvCwsLyGazyOVy+MhHPoJCoYBoNIo/+qM/wgc/+MGakRFuoVgs4rnnnsM/+kf/CD/+4z/e9Xo9QShuRihWFddzzz1HbRXcRCsigmq1iqmpKVSr1RpJbj24TSgAsLOzA8MwqIUKYJ4I/+7u3+GNhTdcfS0rSL0kmTRTK/b6BXVs1jU6c17RTGlyxBeBZmht2aowDEMjEwDwc36kc2nki3mU02YNx83ivhVVtWoSoqZA1dQDxpaHAZETwbIs/ey6gW7o2MhtIL2QxjcWvgGJl3AyeZI6JzupxzKZDBYXF1EulxGJRGjBORKJtL3hvZtTXu2AiJNCoRAuXbqEubk5fPrTn8Zv//Zv47Of/Sx++Zd/GSMjI/j0pz+N7/7u73btdT/84Q/jwx/+sGvr9UTKy60Ixa7iqlQqh1Kbabb5p9NpTE1NIZFI4IUXXmhpA3OTUAqFAtbW1gAA73vf+2h9oSSX8Nqt1/Bg+0Fb67UqMTUMA9vb2ygWixgeHkYgEEDEV9vM1+j3ZU2mJ26WZRESQ+BYDhW1gqrSPJICHjcr+oN++IP7dSJbcV8QBEou3fpjhX1hFOUi/e6snfs8xyMgPLb7dzK2bDfyC4gBKJrSsc2M0+tbv5OqWsW91D3cS90DAPSH+zEeH8ep5ClH9RiRJ5P0WKuKJuvrHzeh9AKpAbXDtQRBwIkTJ5BIJPB3f/d3KBaL+MY3voELFy4c81U2Rk9EKN3KhuupuEhdwm2tfT0CNAwDjx49wsLCAi5cuIDx8fGWX9ctQiEWKuFwGJIkUTLZ2NvAX838FbbyW22v2QqhqKqKzc1NGIaB8fFxU+nFCY71hHpuw1boul5zAhd50Zwhr5sNi3ZpMsuwCEkhxxSQvbhPai+kuM8wDEqlEk2jtYqoP4pcOVe3DqRqKnLa4+vxC34IvABd1ztqDrWT12HDL/iRLWWxnd/GO8vvgGVYUz22X9wfjY5iZGQEIyMjMAyDTq4kiiafz1fTUe50sHovp7zs0DSt5v6zOg0Hg0H8wA/8wHFdWss4VkIhufluZMONGhVJGGsfq9ktnDZ/RVEwMzODXC5Xk2LqZs12YBgGrRs988wzqFQqyGRMp96plSn85exfQtXUmnG8qq6aliRdSlcrlQo2Nzfh9/sxMDCAoC8IVVNRlA+aLTaLcOrBbqsSlILgWA6qpkJRFQi80FI9gWVZaq0BmN/b5uYmqtUq1tbWHIv7B9bYJ692i+9lpUx7UoixZVAMQuSbO0Q0I69OUe+QEJJCKCtlaPrj51I3dKxl17CWXTuQHptITiARMdNe1omK6XS6YXqsFzZzu1z3uECUiwRPmtMw0EMRiqK0r/Vv1qh4WIRij1ByuRwmJycRCARw8+bNjixkWJaFLHfmmeVkobK0tARFVfCXM3+Jt5fepj/rZEkSkAJgGAZlueyYlrFGKHbkcjns7OzQHoZG/R5uwYBBU0sBMQCe48FzPCK+SNt9I4IggOM4xGIx+P3+A8V9SZIouWxs+LC9JeLyRQ56pDvbFmJsWZSLMHgDIT5kRmBGrbElUXIdpQy5VfKyp8digZg59yV5GieTJ2smKtZLjymKcuzzPuxjd4/zOuyy4SdJMgz0CKHwPI9yufXu5FYbFYnR2mHMLiFrkq7z06dPd6XC6DRCqWehUlJK+Mv7fwku0vjkpepqzcneJ/gg8ZJj9GL//zs7OygUCrReEpaab3z1FGOdfG7W5kFy8mfAwC/4IfLmzJRS9aApZKNrsxb3VVWl6bG9vT188fMXkN0T8NWvVfErv+JOIZd8Fk7GlgIngGVYZEvZrl+n0etbP/tuemiypSxuLd/CreVbLaXH0uk0VldX6Wz4Zumxw0IvREmAM6F4EUobIJtLO0X5dr24DqNpkqw5MzODra2tA13nnaATQiFkdvbsWZw6dYpuDEu7S/jfU/8bm/lNjEbaG9BVUSq04GvtuRDYx7ldTdOwsbFB6yV+ye+K2WI7qbd6G58Boya1xLEcAoIZgbU78ZHneYTDYYTDYYTEEF5TVICTwQsaVlZWDhT33aoFGDCgaip0Q0dFqVBjS4ZhUJHdn1pJJoa2ciBoFfb0mE/wPW6utKTHSqUSJElCNBqtmx4Lh8OHuuH3MqEcdoRSKBSwsLBA//vhw4eYnJxEIpHAiRMn2l6vJyKUVjf9Try4DoNQZFlGsVgEz/N45ZVXXJmk1w6h6LqO27dvI5VK4YUXXqCNUgDw5sM38bfzf+tKbcTac6EaKiROAgcOy+vLECUR/f39CEpBqLra8nCqbmfdcywHv+hveeOz9434BB9EXqQd/a2k5kgKSPTxqFRZJBMsTpw4caC4b629dGNpbldyEWNLAkmQIPGSGZnJtXWOdkC+B57lD2VapRUVpVKTHhuNjWIgMgAtreH8yPma9FilUjnQXNmueqwd2Ivhx4XjIJTvfOc7+MAHPkD/+1d+5VcAAD/3cz+HT33qU22v1xOE0qwPpRsvLrcJJZVKYX5+HizL4tq1a66dbFolFBKhAaixUFE0BZ+f/jxm18y+m8NQzmxntpHNZs0HO56g89473dCsaOV6Jd50H+7GudcagZHielAIQuQO1r3sp3a5al5jss84UNyXZRnlchmlUgnpdJoW94nvWKP7xPrew74witViQ6KrKlUqoybd+TzLm8aXTYwo7eBZHgInHMm0SuCxFQ6JXra2tvDNjW/i0vYl6j02Eh1xTI+1ox5rB70coQwODh7qa77//e93zU8O6IGUF9B40+/Wbt4tQiGktrq6ijNnzuDhw4eu3oStEMru7i4mJycxODiIp556ir5+upjGq2+/ilQuRX+220jACrJONpt9XC/xhbFb3KU/Q0/+LUhi611Xo+slkxxbmcLYKnTDlCYXlSJ4lUcgGKATH6tK9YByjGgm4rGD10lsMqLRKO3cJ+SiqmpNcd/auW99z6SHph0ll27UTq0UOAF+0Q8YzY0t/bx5GKmolSPZUAn5Wa1iyHC91cwqVjOr+PrC12vSY6f7TiMeiSMSieDUqVNN1WOdpMd6mVC8GkoHqBehuGE378a43kqlgqmpKSiKghs3bsAwDCwuLna1ph2NCMUwDDx8+BCLi4u4dOkSxsbG6L8tbC3gtVuvHWh0c4tQNE2j/SVDQ0MIB8OQBOlAysleewmL5oNdVaqoqI+vrZPIKeqLHjCTPAyQwrhP8NEZKVF/1LSMKZeh6+a1X77cmPibFfet/06K4hFfxJWUk6IpUMqPjS39oh8iJx4QWYSkEAqVAlRdPZI+EJ7lIfHSAWsaJ9myPT1mVY+d6jtVNz22uroKwzDaTo/1kmz4SR7/C/QIodijCGs0cPny5a7s5ru1xifGin19fbh69So4jkOpVHK9uayeOaSqqpiZmcHe3h6uXbuGaDQKwHwQX7//Or5+/+t1VVPdEkq1WsXGxgZ8Ph84jkPYHwbHck3TIwf8rgQJPt4HzdAgV+SWVV6036NydJLZkC/0uC5hOeOUigIgR8ALQDKRhdqGNNla3DcMg3bu53I5qLKKcq6MR9VHCAQCrg4VM2BKkMuw9L5IQYiseKDH5DAh8iJYhnXsS2ql6diuHhuJjbiaHusF2bBhGAeIzetDaRPWlBeJUKwprhs3bnTN0J1GKIZh4MGDB3jw4AEuXrxoThe0XC+5AQ6zhlIoFHDr1i061ZH0t1SUCv7P9P/BnY36lvPdEko+n8f29jbtL9la3UJFqUBn2ydSa87fgAE/70fUH0WxUkRFqTj2uQicAJE7ujG9QOOU025GA8QcIgkdmqYiIJj9L5quoaS0LoBgGAY+nw8+nw+D/YNYX1sHw5vSdjJUzGrL72axWNd1cAyHdCkNwPQECwhm+vKwCMYv+KHqak2UakW7Lha6oR9Ij51MnKTy5FbSY1ZrfvLcHTehkD3KHqE8SbNQgB6JUHieh2EY2NzcxNzcXNcTFa3opIaiKAqmp6dRKBRqogICcvMdJqFsbm5iZmYGJ0+exLlz5+hDl8ql8OrbryJTzNCmPlVTXTMiNAwDu7u7yOfzGBoaQjAYRNgXxiP1EcJ69ze3AQNltYy98h40TYPIi/CLflTYCliYn2VADJjKMYcT7WGA2sA3IK/c/j9FonUs7febQ1uV9RIll6zJiIaidYv7ZKgYkSZ3er8xzMEGyapaRUkpoVAtmO4DYhA8ZxpbluVy1ynGkBhCRW1c9+rWFqmiVHA3dRd3U3dp78tgZBCn+w42V1rTY2tra9B1HYlEApVK5VDGZ7QD8uzbayjH3fTZLnqCUMgNNTMz48pERSvaJZS9vT1MTk4iFArh5s2bjifEw+jAJ4Si6zru37+PlZUVPPvsszUqj9m1WXx++vO00GolEVqMBVCWy6gy1bYjFFIv0TQNY2Nj8Pv88Ak+5Mo519Iw9shJVmUz768o0KEjEUhAhw5DPdx6CQHP8vDxPuQr+YbR8N6e+f6j0YPX5TQSmch6nQQKjZRc9uI+qb3Yi/uBQKBlRwaO5ej3aIV1MydTKwl4lodfMK1nykqZNl22ilbUagRu3Fscw8En+qh6jHiPjcZGafQyHBl2TI/t7u7i/v37WF5eRjKZpNb8RyklVlW1Zq69YRgoFotehNIOGIapkcHaeyrcQKuDqwzDwMrKCu7evXtg1rod5O/dlCOT1Nx3vvMdyLKM69ev0/ypruv42/m/xZsP36z7+/ZibFAKws/5EZACNRMU68FaLxkeHkbQFzRVUPsSXTdVY/Xg5/00HQM8JknD2DeEdDklI/ESwABltQwRjTfnXG6fUCLNPwP7SOSQFALLsJA1GRIvtezJZR0pCzwu7pdKpQPFfb/f7xjRi7wIjmle97JD1VXn/p0WVHz2CZKN4MY9JXACeI4/8B51Q8dKZgUrmRW8fv91+AV/jfdYLBJDJBLB+vo6zp8/DwDIZDJ4+PAhnaZo9R477OZK+/fnqbzaxO7uLt566y0MDQ0hn88fymSyViIUVVVx+/Zt7OzstERqZHaBm4X5YrEIRVEgimKN5X2xWsSfvvOnWNpdankt0i1eVIooVUs1M9JLculACoLUS8jJLOKPoFgtHtjA3Xj4CTEVi0WkUilzBkQghGgweiDF5USSbs0aCUpBVJVqyzLkvX1CibRAKFYQaTJJOZXkEsK+cI2nWqsn9EbFfbvvmCRJNHVYVp17U9pJNzn177AMa6bHLL0v7Vq3dJvyEnkRDEwfumYoK2Xc2bxDx10nggmcSp5CJpvBOfYc+hP9B9JjmUwGMzMz0HW9Rj3mdirKaSaLp/JqE6FQCJcvX8bQ0BC2trYOJY/JcVxD08VCoYDJyUkIgoCbN2+23PXuhhyZYGVlBfPz8wCA5557jj5gq5lV/Ok7f9rRCFtrRHFgRvp+7UVRFaysryCXz2FoaAiBQKDuhuCa8mi/7yCVSiGRSEASJFQrVaykzI7oVCpFUzrWdKLVEBJAU5JshE6ce3M05dXyr1DYU07W70LiJISkEHyiry2StBb34/E4NE2j6bGtrS1IrARWZOHz+1wfKkZIkoB8FwLXmuOzFd0QCin4d2pFky6mkS6msbS8hPvqfUwMTND0mF09RmbpbG9v4/79+5AkqUY91m16zE4o5Pv0IpQ2IEkShoaGAByORUqzdTc2NjA7O4sTJ07g3LlzbYW0bkQomqZhfn4eW1tbePrppzE9PU3/7e2lt/GluS91nOapN8OEnIxJvYQxGDx17imIgpnyqXe6dCPlZRgG0uk0DMPA6Ogo4sG4mW7yifCH/FhZWYHP5ztQkCbd5tb3YSdJ0m/RKHohXdrt+lUZBpDbz/60G6FQyaxDysmAgYpaMQ0uOZ0O5DJgoKJU2nJNJtP+QqEQIr4IdnO7KJaKdCO0Fvf9fj/9Pt04KOiGDt3QsVvcrR2PoKlNFXCdXkNQDKKqth5hNoJhGDBg1E2Pne47jWg4inA4jFOnTkHTNKoecys9ZieUQsEkbK+G0gYOY2qjHU7r6rqOu3fvYm1t7UDhu1V0O7+kXC7j1q1bYBgGN2/epJ+Foir40u0vYXJlsuO1rXB6YK31koGBATAsg6pWhazKCIpBc5zAvtLHLVgHcAHAQHwA2UIWulE7YCkajSIajULXddptTryyfD4fAoHAATmtvd/COimRdItzDAe/5O8o2isWGegaAzAGwuHWCSUoBiFrcl3JrB3WgVxWktQ0DUWl2BKhkwhTEAXExBgdKkZs+Xd3d+lnKQhC14cEgRMgcAKNWOzjEawGo1XlcW2JoBNCCflCZh3HpZSz09RIa3qMAYPR+KipHtu35k8mkzQ1Xq1WqXrMKT1GCLwR7L0wxaJ5APEilA7h5lx5K+yEYrdy6TQX2g0B7uzsYGpqCkNDQ7h06RJYloWiKCgqRfzJG3/S0VRFO+rdwPZ6Ccnrk0jIWscQOAEBMQDd0MGxXMebDyEwv9+PRCKBnfUdc2NnAJJ1skdULMvWFKQVRUGpVHKU09ofWPvGHPaFqdcVA6ZtOezefkATDgOtHjyt1vqdwE6SxBSTZVnTNdmmumrkFmz9LJPJJBRFQblcRrFoTn9cXl6mkWC9oWJOkHiz5tkoVXegyZWXIAmSKbSotldDAjqzp2kEQkqNBDghKUR7X95eehscyz1WjyUnMBwdxvCw+Yekx9LpdFvpMacueUmSjtTG3w30zNUeRYRCNvLBwUFcunSpqz6XTiKURhYqS+klfOnRlzByYsSV/ht7syDpL8nlausljWoJiqbQDYo0JEb8kbaiF2uDZDKRBA8eS0rrAgMCQRBo9GKV01pP3GRTtD6wATGAslKGqpmHFSKHtdrZN9vQ2lF4Ae2pnIDWNlRN12rqFpIgQeIk031AldsaH0BGHYuiiO3tbfT396NUKiGbzZr1F1txn1yfrgOZDINAwEAyGqhJO7YKqwKOYcwaUsQXAdjGxAQcztRK8nzUc2oIiIEDdjGarmE5vYzl9DK+hq/BL/hxqu8U7d4n6bGTJ0+2nB5zSnkFg8FjH4/cLnom5XWYEYqqqlhYWMDDhw8PbOTdrNsOAdazUAGAby58E1+9+1VUtfZ7R+rBSiiapiGVSkFVVYyPj8Mn+hCQAm3XEkpKCXzZvGVEXoRf8NNeC6daz+7uLvb29jA0NIRkLGkaGZa7b1a0ymmtJ+5SqYRMJgOO4xAIBNAf6zdrF5Zn0i6H9Yt+BIUgJRmnz5/0oEQcelDs1xWWwm2l1TqO+vbdB0RehMAKYMAg6o+iqlYP+Lo1em1rcR94XAze2pLx+tc1lIoyymUJuZyASoUBwOD5p/z4oR/Ndy3j1nUdFbWCfDUPjuNqUpX22TXdDP5qhHqEwrEcfLyvhsTroayUMb8xj/kNU1iTDCUpuZxMtJYeI98F+V9CKE8a3vURiq7rKJfLWF9fx/Xr110rcrWj8qpnoVJVqvjCzBewmlmlA5Tc7PVgGAbVahU7OzsQRRFjY2MISOYDaz91tbKWvSGRThhk9iW9HA9FUVCoFpBKpSDLMsbGxpCMJCnp1BsnXE9E0ArIiTsSiZhOv+UKOJ3D8sYyVFWtsTKxNwOWZVNezSkcIr6I2fEOBhX1cVqJNjU2iFB4lockSEdqFeMX/WbhW6092Yu86DhO2AkHNtL94n42y2D+tkNvTjWK+cUMvqdY6HqomH0zt6YqAdAaEsdwyJazHb9Os2uwvwfa19KhU8NuYRe7hV18Z+k7LafHVldXUa1W8cYbb+C///f/juHhYUSj0UOPUP7Tf/pP+N3f/V1sbm7iueeewx/8wR/g2rVrHa937IRindrodoSSzWYxNzcHwzBw48YNVztfW1V51bNQ2Sns4NW3X8V2fvvxmiyHoBiET/K1PRu9HlKpFGKxmGntbauXtItGtvNExaQoCrZT2wj5Qjh74ix4jj8Sp2ACnuMxkBhAvpLHWGiMRi9kpnm9OSWGYRzoeCfNfMU9FtAriEScX7ORkuuwUGNiaYN9nDARWtgteuzfp6IAi4ss7txhsbBwsI7CyFFcvLKFl67uYmenVJNqdCLrZmiUbgLMAxfP8tgr74FlWYTEEDiWaysKa+UarK8v8qLZcO2SIMWeHguIAVM9ZkuPVSqmpx1Jc332s5/FysoKXn75ZXzwgx/EBz/4QVy/ft3VPex//+//jV/5lV/BH/7hH+Lll1/G7//+7+MHfuAHcPfuXQwMDHS05rETCgHPm6dbN2AYBpaXl3Hv3j2cOHECS0tLrtsoNItQrI7JdiXZ3c27+Nzk5w4qXmAgX8lDNuQa+aWiKy11u1uxu7sLwzDQ19dn1h4CUeRKnW/srZyUSqUSUqkUwuEwon1RcByHfDWPgBQwi+KajEJ5Xw10CF33Ii+CY7kaUjgQveyrnaxWJqqqOh5mSDNfJi8AEDCUCCLiR01RPCgGqUKuU7R7Cm2nMH3AUsWSVpIrMlSVxe3bLO7eZbG4yEJVmZrfBkx128vPh3HtfdswszBJAMm6ZE3+NCvuNyIUYqVCvktdr+19EXmRzq5pFoU1uwby+hIv0Vk4h4WSXKpJj107dQ3fd+n76MTP0dFRfPKTn8R/+2//Df/jf/wP/MIv/AL++q//Gh/5yEfwxhtv4MyZM65dy7//9/8eP//zP49/+A//IQDgD//wD/GFL3wB//W//lf82q/9Wkdr9gyhcByHSqX7U4eqqpidnUUmk8HVq1fh8/nw8OFD1zT3BI0ilGq1iqmpKciyXOOYbBgGvnL3K3hj8Y2mlvN2+aV1cFJJKdEisx3WegnHcQj4A4j4ItgrdZd/bpaOI4XH/v5+JONJ8CxP0z/Wk7vImw63IWlf+olalU2nREPMFhudLK1WJYlEAqqq0rpLNptFPp933BD3cgzAahBDZeTK5vVJgoSQGIKsyl3JV9t9vxF/pCPpM4Gqqdgu5bCwwGJuTsKjxQvQZT/AqgBXRDSh4+IFDX19Br7wBR4XL+r4we/zQwztHlirHllns1lsb29DFEUaDdZzwXB6JutZqVhhj8Jqel/kUssHJ103ZeskfehGVqBVXBi8gA9cMMfvOg3XSiQS+Nmf/Vn87M/+rOuOyLIs4+2338av//qv079jWRbf933fh29961sdr3vshOJmyiufz2NychKSJOHmzZuQJIl2ybs9RKdehJLNZnHr1i3E4/EaC5WSXMJrt17Dg+0HdddstGnX2JAwlm53yyYqyzI2NjZovWRjdQMC2373cjswDAPb29solUoYGRlBMpqErMp1FTuKpqCklJCv5Ol8DoETUJE7P0x0KtHleR6RSIR6JvE8X7MhSpIEjgtArprhv7Wp0cf7kC6mYcCgViRup2Os6KTgb0W1Cty/z+LOXRYPH7DQtP2NnKkiNlTBpYsaLl1kcPqEjw5GOzGuYmTQh2K1+Wvah4ppmoZSqYRyuYzNzU0AqCFr4jBuJ5R2rFQI7IcvnuVNiTXDoiyXG3bSG4aBgBCAoiquTgNthssjl/HDz/wwJYlmw7Xc9hEjvV32HrzBwUHcuVN/LEYzHDuhEHQ7CGt9fR1zc3M4deoUzp49WzO7BHD2yukGdhGB1Vzy3LlzOHnyJL2Gjb0NvPr2q8iWsg3XbLUoT8wSCURehFpWsbOxg2gkilg8hpAUwjrWUVbKdNRrN3C6Nk3TsLGxAcMwzOJ7OIl8Jd9wY6/p2t+vvZC1BVZAxGfKKEtKa7Uet9Q/9aYsrq6aJC5KGrLZbQQCAQzEBmpe025FQnstdNPmPrWlIxQy0KKrzwFwLAe/4G/7YFCpEBLh8PDhfnPmPuIJA6cnyhgZyeLy5cfedeR9iJyI0UE/Jct256VwHEd9xwBQ3zHSuU9ky8SKh2GYrq1UCFRddXZ/hn6g7uQX/MjpuSMlk+fGnsMPPv2DNc+Ck2z4SWtqBHqIUDqNUHRdx/z8PDY3N/H888+jv7+/5t+tJwA3QZoRydq3b9/G9vY2XnzxRSQSCfpz06vT+MLMF+qmqKzoVOW1kdrA3t4eBgcHEQ6FEQvEoOkaeI53VYZsXcvarDg4MIhoMNpVWo1hGCi62ffC8zxtKOMYDhWtciCvTac5HoKUFHhsxCgI+6OAo+bGVClUMLc9V3dGPFDba/HNb/L4+t/FEYtw+P/9ah5Vh675RqlYkTPrQq3IVwGgXN4nkTscHj1i6NhiAEgmdVy8aP7p7zeQz5dQLB5M8fgE3wGSJPPgSYOo1RCyFUiSBEmSaOc+IRfDMLC0tIR4OI6iWIToE12vd9rdn8n7AID1wjoM5mgEIwDw4skX8cFLHzzwnTulvA5TNtzX1weO45BKpWr+PpVKUTusTnDshEI+2E4ilFKpRK3vb9y44dj1TpyB3SYUsia5BmKhQvX8uoa/nvtrfGfpOy2v2S6h6LqOzc1NKIqCsbEx+CQfQr4Q0kXTAl7RFQicgIg/0pLteKsoFArY2tpCPB5HX7IPftHfMpm0Kg82DONgEXZ/o1MUBRzHHYlEl0iGE1EJAwMDqCgVRNVo3RnxgUCAHmLW1xl88w0WEIrIlg0srykYH90vJu9HmY2+b7/oh6ZrTTfvcpnBvXumOuvREgPDQiJ9/TouXjBJpK+v+b1Vz4VZN3TXzDlZlkUwaCrPqtUqzp08h529HeSLeVR2K20X99sBeR9E2MAypnos4ou4pqysh+unr+N7L3yv83XZUvIkhXxYEEURL774Ir785S/jR3/0R+k1fPnLX8Yv/dIvdbzusRMKQbub/vb2Nqanp2vsS+rBTWdg65rlchnf+ta3MDw8jIsXL9JryFfyePXtV7GaWW1rTYZhWi7wWusl4+Pj1JbDnmOXVZn+HcdyCEthM7fc5tAkcm2kWXFwcBDJmJkqITNTukUjgiFFWL/gBxjzpBn1R1FRKgfUcm4il2MAOYhItERrI/Vs5Pf29mjPz+ZmHF/+cnQ/zWTgwx9WMThoHCgm+3ifmeYT2Jq6QbM0U6n0mESWlhgYxuPPrn/gMYkkk60bM4alMIpya0OxHB2sWd60y29x2qNhGAiLYciGjHAkjHAk7FjcbxQNdgKrk4GiKSirZeQqOfN9CAEIvABVV5sSfjv4rrPfhe869111/52IaAgOO0IBgF/5lV/Bz/3cz+Hq1au4du0afv/3fx/FYpGqvjpBzxBKq53yhmHg/v37WFpawuXLl1ticbdnlxDX3Gw2i2eeeQajo6P035Z2l/Bn7/xZyykKK1qNUMgsEdJfEpJCZlgv126s9vXsfRakcayVh8cwDJRKZt1mbGwMiUiibh9EMzR6n42uISyFaW3FSiKkZuFmFEZQykYAoYxwxPnkareRV1UN3/qWgW9+MwCAwfBIEVdfLODkSR6aVjsEi7gN5yo5SIZET/08yyNfOdiJXiwyuHvXlPguLTOAhUQGB3Vc2CeRRKL9TbBduxgrnIriAdG5492KsBRGWksjjjj9u3q1LELYAGp8x9qti9pVcvaplfbRzn7JrCM5GVu2iu+98L24fvp6w5+xRyhHMVzrp37qp7C9vY1PfOITtGTwxS9+sSOzXIJjJxRr8bxZFEHkuNVqta2udzebJhVFwczMDDKZDMLhcA2ZvPXwLfzN/N90ZTnfjFAIkQ0MDCAUCiHmj2GvstdUhuyEGodelqdd9PbQX1EUFArmkKjx8XEkQgnXPZWaoZGPk+OERLa7TYC85k6uADCM4+hfOwwD+Lu/k/DOO+bGcOWKiu/6LhXVqnFgCJZVSkudpjUFATGAdClNGxLLJQFTMyqmb1exsmJanxAMDZkEcuGCjni8AyLY30zdtjVRdbUmHekTfKZrsqGhVDUlvVF/FBu7G02l/PZoUJZllEol+nkSabLf72/aue8kuW4kx9V0rSb6pr0vhtGSaIRhGHz/pe/H1ZNXG/4csUc6jqL8L/3SL3WV4rLj2AmFgEQR9b7gTCaDycnJA3LcdtbuFvl8Hrdu3UIgEMCFCxewsmIOhVI0BV+Y/gLm1ucQkAJmIbmDVEwjAtB1vcbOxCf5EJJCDS0p2qnJWDcBBgwCUgACK2Avv4fVlVUIogBJkhAPxV3ZfOoRoNPf1XPRdYKj4mpf4VOqtha9WF8zlze7v5sRiiwDn/sLAYv7Hebf+70qrl3TAPgQCNQOwSqVSlRKSyI/nucRC8SwV95DPm9GInfuylhdUQAwgC4Bqh+jYzouXs7j3AUFsVj3hB4Ug4cmbCCwTnvkGA4RfwSaoYFn2tt+GIahxf14PE6L++VyGdvb29B1vab2Qor7ZA6Ok+S6nf60Aw4EDSaIMgyDD1/+MJ4ff77pukTpZq+heCqvLkAIwj4XwDAMPHr0CAsLCzh//jxOnDjRdoOiG0V5MoyLyJK3t7ehaRoypQxe/c6r2MyZG4T1ROMTfJB4qeV8bD0CIPUSQRCouaO1cbDd9ZrBgGktvre3h93dXQwNDEFkRKiq6oq9CPn+nL5H6/UKnNCWi64TaqIX9vHoWicLeKDWk0vTgGKBjP6t/xqFAoPPvsojtcmC4w388N9XcfHiQeKyDsECHivlysUydjZUPForYWUljFSq1sJkZETHxYtlXLhQRDRq0FkpAid0PA6ZYzkEhACyxSyCOBoTQpZhEZACyJQyAABZkyHxEiL+CJVYtxPdk+I+qTXIskwJO51OPzYJjfYjW8w6HlQ7bXi2TxAlaT7iYv0Dl38AT4883dJaZG866hrKYeDYCcWpX4ScLBRFwezsLPb29vDSSy8hFot19BrdEIp1GNdzzz1HPW44jsPK3gq+/Y1v123Csp7MaEqJMVNNTmoSJwIg9ZJoNIpkMomgFGzYONhsvVZAmhWLxSJGRkaQiCSwtb2FYqUIv+GnRpBV5XCa+IDH0lU3/bF0Xa8hfFJ72eNNpZbEm1bt5DWJbT3PGwgEnD/H7W0Gn/2sgFyOgT9g4CM/oWB0tLXPvFLx4f7dAWyuD2ItVUtuff1lnDldwcWLBgYGfAdqL041i1YnPZKhWLvy7pHZo5MxyNYanmEYUHSFRg410mRNbttPSxRFiKJIRxxUK1UYqoHlzWVqs2Mv7jsN1+oEJMLnWA4/+tyP4sLQhZZ/l+xNVl+5YrH4xE1rBHqAUAiIvJfUOqzpJatDbyfolFCq1SomJyehKMoBC5VvL30bX370ZZw8ebKltQ6klERTTSKrjx8cOwHY6yVRf9Q0WjwEHywCMhpY13VaLylUC1ANlYbmdisVYmNflIstpRbrER3Z3BoZH7oJEr2U1BIG+UFqDKjrOmRNbmpb/+gRiz97jYdcZRBPGPjJjypNaxnZrJnOmr/DYnMlCDAhgJMBGBgbN3Dxgo4LFzSIooJy2awXLC9vQRRFWoi2zigBHO6tfaUScVGw1p3Ie3RTwdQM9axU7NFBPWlyJ35dPMcjFAuhWC1iLDx2oLhPiv+KolCpf7fgOR4/9vyP4dzAubZ+j9RPrJ+FF6G4ALLxr66uYn5+HhMTEzhz5kzXp6hOCIVYqCQSCbz44os0JVdRKvjc5OcwszzTcV2GnjD3D6VkUy5LZTra1F4vCfvan4XeboRiHQ08MjyCaDDa1OrDmldmGRYhn9mM2IoFiaZp0DStph4WlsIoVopHWvAPCkGU1TLYyuOTqk/wQSn6AYVFNHLQ6n96msVffZGHoTMYG9fxEz+uwF/HkCCT2a+J3GGxubn/GnIIEMoYGMzj6ac5PPUUg1DI+p5rGwGJjcnW1hYMwzhgY0JgVypZbUhUzexCt6b6DjtCEXnzIOgUbTRLNzlJk1vx6xI4AQIr1Kak6ki9S6US/f9WsUS7n4vIi/iJKz+Bib6Jtn4POJjmB7waSsewfnEcx+H+/fvY29vDlStX0NfX58prtEMojSxUtvJbePU7r2K3uOvq7BKyKZfVMlRVxe7mLoJiEGMjY3TwUCeF03ausaZZMdGHgBSoIZNW1tIN/WANSZDMmejy45noZE4LmbYoSZI5MIs3h365dWJsBVF/FJvqJkJG7cNbUSrYTCuAwCMaMYmOYRlU5Ar+9isa3njDfHSeekrDD/2QCrt6NZ1mcOeOKfFNpSybBWPg9GgY5y/v4dx5DdvbKxgdHW3YHc6y7IHai93GpN5mSGxIiLRc4M2xzqquIoNMl59eY7RiuNjqxu3o1yWYvVdWvy6RE6l1T6PXJFJvWZbh85kpRSthW235m3XuS7yEn7z6kxiPj7f0XuywK7wURUG1WvUIpRuQkwLLsrh58ybVobuBVglF0zTMzc1hZ2fngIXK3PocPj/9+cen8Q5GADeDoigoFAuIxWKIJCPwiT7AMB+eoBRsO03RKqGQ1Jq1WdFpAFe7BFqj7mE5OryKA4ft7W0kk0n4/X7IVRm6rCNTzaCYKtJpjG53SVvRinqM1FAiUXMuuqYBX/gCj9vTIUCT8D0fqODm9+RBPpbdXZNE7txlsb31+LoZxsDJkwYuXtTx4jNhaNxj597tbbQNJxuTUqlUsxmSz4/n+RrzTFVWqVScSJMj/khdkUKnqNdxb0U3B7IDkzcFP1XztdNoS/o/rIRtL+7zPF8jTbbek37Bj5+6+lMYiXXe1e4kGQbg1VA6RSqVwszMDHiex8TEhKtkAoBaPDRCqVTCrVu3wHFcjYWKruv48p0v49sPvl3z891MGHRCJpNBoVCAJElIJpOI+qMHjBathf1StbndRTNC0XUdW1tbqFarZrNiOIGKUnFct9v3SJoqd3d3UVWrGB0axUByAJVqBbpk1izS+TRisRidw23v2+imjmZFq9MVKaFEDJTLwJ/9mYCVFRaMWMaHP5THs8/qSO/yuD8fwtwMj9Rudb8eAjCsgVP7JHL+vI5AwIxycpWsK++BwEnpVCqVaPQSC8SQ5tOOfRqarqGklGgkSgaKddsg2mrHvdsjJYpyEYqm1Kj5mvUiOToeW4r7uq47ztDx+/3oi/XhYy99DEPRzr2vAGenYQBeDaUTlMtlzM7O4vLly1hbWzuUQmGzCIXYuNgtVIrVIv7s1p/h0c6jA7/jFqFYN/VYLAZFURDxRxxPzvUK+4qiOIb4jaxcFEXBxsYGOI4zySSUaFjw7zbFZ60L8TwPgzVtLyqqOakuIATotEpBFGjfRqlUQqlUooVUEr3YT4qtQuIlgIFjgdgOUpTnOOC//3cB6TQLUTLwvR9QsbfH4P/7/wTs7rLA/omfFQycPinhuad5nL+oAJxZC+IYDn6xvluwm5sq2QxjsRhCYgipTIr2aVhTOYFA4IDCyRpRtrMpW9HW8C+XCCUgBiCrMj0I1VPzGYYph9eMWpfwRtfAsiy95wDQGTqMyuAidxELMwtIJ9JIJBJIJBIdHXqcjCE7cQHoBRw7oQQCAXzP93wPeJ7H5ubmocyVr0cohmFgcXERDx8+PGDjspZdw6tvv1q3KE1uwm4G35BNned5jI2NoVqugtGYlmZe1Cvst6K2IjMqQqEQ+vv7W+6U7pRQVFXFxsYGWJbF2NgYVldXIRgCipUiGNYkvfXtdRSqBYQRhsSZNiQqr4JlWVpItZ8UrZtjKw61ZLpiaxY/QG4/o/Klv+ZRKTMQRQM+n4EvfvHxa7GcgYkJAxcvaDh3TofPZ34hBmq90+pZ8RzGAYq4BeQqOcc+jWKxiHQ6DYZhIAgCrSfUKK5smzLpEteN+tFLux33bhBKUAqiolQaKgKd3IY5hoOsyW1fA8/zGB8cx8de+pjpUrG3h3Q6jZWVFdy+fRvhcJiSSzQabWlvcEp5kVHATxqOnVAAc/KbW0O2nOBEKIqiYHp6GoVCAS+//DIils61W8u38MXZLzZMKTEM09WpnXRLRyIRs7/EF4Rcad8WnMCutgpLYSg+5UA+mTQr9vX1IRlPQuTFljaBTm/uSqWCjY0NhEIhU2RhACN9I1jfWaekoOs6VFXF8PAwRFFEVauiolbMGSkWV1uO4eDz+ZBIJBzHz1qjF/v1tjuEq1hkqLFjpWyuJcsMZJkBxxmYOG0aMJ47p6OeX6HIiSgpJfq91DS6VlufKtgOSL+HUzRkT+WkUikYhkGHLTl1mRPY7y/SL0I263bcDAi67QEJ+8IoVlszsySwS5NZhkXEH4Hf52/JNTkeiONnrv0Mov6o+d/xOOLxOM6cOQNZlpFOp7G7u4vZ2Vnouo54PI5kMolEIlE3lX/U1vWHiZ4gFIJuh2zVg51QSI9LMBjEzZs36cOjaiq+OPdF3Fq+RW3fDd2MBKxhMkGnhJLJZJDJZNDf32/OX9+vl2iG5sqJVTd05Kt5FOQCZFWmXkrrG+vIZDMYHh5GMpKEarTe+d7Je83n87T4HolEwDIsfIIPRRRx4sQJVCoVbG1tQdPM9721tVVTkAf2/ZRkc24GDDyeVMkpdOKiYRiUXOybYyAQQDKcbHuzy+UAQTSgyKSr38D4uIFnn9UakgiB08nZ3ujqF/wI8KYU1g1yETkRHMe19J2yLAue5yEIAmKxWN1CtBNBWzdlBow5f8fQasw7W0WnB5V2UmuNoGoqCtUCFCimA4Hgh8ALZrrVJk3uC5k1k7DPuVguiiKGhoYwNDRkjl/Yr2OlUincu3ePjp5OJBKIx+N1h/8RQvEilC5xGHNL7OuSyY72Hpe98h5efftVrGfXAdSO3LX2V1g9ujqZX7K1tYVKpYLR0VH4fKZ1Odns3JQiW9crVopY3FyErus4efIk+sJ9UHQFctU9VY8dRDk2NDQEv99v9iMYZuGUYRjIsoytrS1IkkSHopENzerLRAiG4ziTONQyDMVssJR4iY6MJY1qyWTSrCnt114quQo2NzfpOq32GOztMZRMAMAwGCwvM9jeYfDokY4zZ3RMTOiOUxhbiYaISqmkmqfikC9kTt5scyY6AXEWaGt0riXdU68QTQjaqhwjBzCWYREUg9RKhf6dJXpp1IvU6b3ejTOy0zXQDnUYKCtlmiUgPTxk2NtHX/goglJrkQPDMLTv5dSpU1BVFZlMBul0Gvfu3aM100QigXK5XFN78SKULkE2Pp7nm6qxOgFJpc3Pz2Ntbe3AZMeHOw/x2q3X6p7snPorRF40R+u2eE9bi+DEj8uebjoMQiGNopIkYXh4GNFglM5CJ7JRnufrFvbbvTa7ckwQzJQVKZoyDINSqUTtZOLxON3USL7f6iqbz+fpjBErKQCoIUUiUmAZFgqrwCf64BvwIV/O0+jF3hRIiMoJpCB//ryGs2d1LC6yePiQRbnEYHaWw+wsBzAGxkYNnDljEszAgOHoaFsP1s/TvpER+/dWhj4FxSCqamOJbjuwFqLtBE08skLBEOLhOHJ6rmG3e41Dr9xeQdwJbjsj67pe9xpID89IbAQ/9eJPwS92rj7leR79/f3o7++nETVJj6XTabAsC1mW8dZbb6FcLh95D8pv/dZv4Qtf+AImJychiiKy2WxH6/QEoRAcVoSiqioqlQrS6TRu3rxZM9nxjcU38JW7X2mrp4SkLmTD3MwiPrP+UpSLjuE+2UDD4TCSySQCUgCarh0gMLcJpVqtolKp0MmKISlUOwsCBopy0bmwb8tNt3Jt9uI7UQoVqgX6u6SI2dfXV1dnb3eVJWovIiYAUBO9sCxLLcUNw4DIiZA4CYzBUCv7erJaQRCg6zod50xAJMPJpIFnn9Xx7LM6NA1YW2OxuMhicZHBzg6L1VUGq6ssvvZVIOwPY+JCFmdOszh1qnlarO7naFPz+QU/jV6Kcu09E5bCKCqtWd7Y0eqGLggCotEo9chSqyqq1SpWNldo9FJPHOHk0MsxHBRdaZtQ3CYTa6NtPZxInMBHX/yoqQ50CVa14tjYGG2ZyGaz+I//8T/i4cOHiMVi+MQnPoEf+IEfwMsvv9yWu3onkGUZH/3oR3Hjxg388R//ccfr9BShtDpkqx1kMhnMzs4CAK5fv05PpLIq4y+m/gLzG/Mdr80wDFTN8vAzjy2tSWrMXi+J+CMoVJzTIW4SSjqdRj6fhyAIGB4YNqc5Num7cCrss6zpzEua4eqB2LYEAgF6CguJIeQrefrA7uzsoFgsYnh4uK1ueI7jDthmECkx6VUhD2gsFIOsyigqjzvzWYalXdU8+7huQCxNyGeVz+dpWie7FwNgzpJ/fB3AiRM6TpzQ8YEPmFHMgwcsFhcELC0Gka/kMT3FYXqKA8OaNZczp83opZXxu05wSsOQ6IUBY0q9u0j9tBsh+AQfIACCT0AoGqLiiFKpVCOOcGoCdHLoDUthhH3hpl5d9aT03aAZoZxKnsJHX/woBM7dGfdO1xEOh3Hp0iXcunUL//pf/2u8/vrrWFhYwCc/+UlomoaVlZVDTYP9xm/8BgDgU5/6VFfr9BShuBmhGIaB5eVl3Lt3D6dOncLi4iIlk93CLj779mexne+gTdkCOwFYjRN1Xcfe7h5UWcX5ifPQWA1hn/NMhnrrdQJryimRSECv6tAN/cA0x6br7Bf2CXyCz+wVcejYJ7YtVCq5v4Hnq3madkulUtA0ranNSDNYbTMSiQTtCyiVSpALck29xO/308+URC+GYdBJlYqumJLefZmmKIp0eNPuTggAB5bNQ5Z5x/6CaNTAy1c53HyZQb60i5WV/ejlAYtMmsHyEoPlJRZf+YppMEnI5eRJHdaPoJ1NnUQvZOAYSb92YmPf7r3mZKUiCAIEQaDiiEqlQknaKu32+/0HPkNC/EyFeZx+5XjThkh5bJgalho/N52iEaGcHTiLH3/+x8Fzh79F2sf/6rqOZ555Bv/lv/wX6LqO+fn5J6am0hOEYrWwdyNCIRYqu7u7uHr1Kvx+PxYXzaL0/a37+Nzk51yZQ16vcVBRFGxuboJlWQwNDQEcEBJDgGGmx+rJE7slFHvKidd5rBZXm+bgW0FVraJQLaBYLZqnZJ/ZGLe2uYad9A4GBwcRDAYhsAIYhkFBNic8ks+C53mMjo66bqVClF7jg+PIlrKoVCooFovY3d2lGxohGJI2sBaLOZaDn/MjKATB+0ziiMfjKJfNzc/nq2BjI0/TFGRzZFm2phDO88DEhFmo/z6YhpCEXJaXGOT2GNy6xeHWLQ4cZ2D8hIHTExp8vvYa4RgwNUahNdELxyMgmNFLKxJYoHUya8VKhQgjiELPSdptVY5ZX5+mX/chcKbnGMdwXc3DaYR6hHJx6CJ+5LkfAcceTWOhffyvdVojy7K4fPnykVyHG+gJQiFwQzZMLFR4nseNGzfg8/mgKGa+9svzX8abj950La1Ecvf210+lUrTvIigGoRm1s9xpaozjISu1p7FOr82pWXF9Z921+ep2w8G90h5SqRSq1SrOTZxD0B80JxIqZVp8L5fL9LNIJpOHIoMkCpy98p7jhmYtJvM8X9OrAgA7uzsolovwRXyAun8Kl3nIRRHgS5iYSIDn4zTNRixhYqEYeJGH6BMdo5d43MDVqxquXtWgKMDSEmumxxZZ7O0xePSQwaOHLIDTeONbOs6cNov7J07o4HlA1wE795IBVfU2WFVTkdMstRcSidWJXlq91zrp9wCcoxdrYyoRaAiCcOAz1HXdvM+qezVOw50OFHOCUx/M0yNP4+8/8/cPzUPOCU7WK8lksut1f+3Xfg2/8zu/0/Bn5ufncfHixa5fi6CnCKXblNf29jampqYwOjqKCxcu0JtC1mR8bfVr8Mm+uqqeTmAngGw2SwvOkUgEEV/EsVBvnykiCeaYWpmTOyKUXC6HnZ0dJJNJJBNJ+Hgf3WDdIk/rWqqq0uL46OgoVKjQNbNDn+d4RH1RZLNZpDZTSCQTNU2jbsJpaJMV1mKy1USRTNsk98fQ0BAkybTmqKgVpLYBCD74JR6xgPmIsAwLSZLMBjXej63MFoqlInbSOzV1A5Jmq70O4OxZHWfPmhvy7i5DC/vLKyyyGRZvvw28/TYHnjcwNmZgfYPBM0/reN/7VPj9Zr1B5MWWjQ/JDBFS+xI4wVQpGbXRSzOSd6vfw0r2JFW5vr5O1Y/WCDAcDEMSJPqM1BsoBrSmgqsHu8LryvgVfOjyh468/8NuX18sFl1Ref3qr/4qPv7xjzf8mdOnT3f9Olb0BKGQL7DTonwjC5WNvQ386dt/is3SJk7oJw6FUEhTXrlcxujoKCRJaks+WlWqqCpV88bat83gWA4ludTwYSFdzoVCAcPDw0hEEqbbqsXmw21rD1J89/v9GBgYMAuKUhi5iikflVUZm1ubyOVyGB4aRl+s7/HIXc29vheiumm1MdNqokhSg8SdYX19HYIg0OilUDAjnEhMRn5fxk5O/D7eh5JizqqwWsKUSiVqx+/Us2FFMmkgmdRw9aqOxcUV6PopPHjIYXGRRSHP4NEj83l4+20Ob7/N4gc/xOLaS+jqZG7tqyIn/gAfoC4ETnBbVWUFz/NgWRbxeBw+n49GL/m9PPZ298AITF1T0HoDxdp1ILCqzK6evIrvv/T9x9JMaJ8J5Na0RiJTPkr0BKEQcBwHXdfbkhMSC5VisYjr16/XfBHTq9P4y5m/hKIprktygX2Vl6pidXUVLMtifHwckijBz/s7KiIyDAPd0JEr58Cy7OO0BS8e6BMhkxU1TaNOwSW5tkvZzYeD1IvW1tYQj8cRi8WoDJSQCREEyLKM0dFRiKJYQ27EesRpkmA7sJsBtgNZlikh9vf3U9EAiV5SqRQePowCGEAwaKZlWJYFDPOUny6laVOlxEswYJgOAD6fY88GSbM5uf0CAM/rOHVKx4UL5mextcXgO2+bSjEAgBLA+fNFVJTu62AE5MRfkAsQVAFRfxQ+wQcGDI1eDpNM6HXsP+ckeomFYwCAYqVIvw/rdEWnkQaNBopZ56Q0ev0bp2/gAxc+cJhvtSHsEUqpVDryIvzy8jLS6TSWl5ehaRomJycBAGfPnm0rWuopQiEsbWfsesjn83jnnXcQCoVw48YNehrUdA1/c/tv8Najt0x1kmiexNyeX0L6IyKRCPr6+hAQA9BRq45qB1YHY8CSttjvfiZ9IqVyCcuryxAlEcPDw4gFY46dw26S6N6eubmQ4jvP8OA4DoVqgRIradwcHR11jAQPWI+IfjBg6vbvOCHiiyBfzXf0vkidyd5QaZ2HYRgG7t0zf16Sylha2oRf8iMRSyCtpOlpuaapct8tmWM5yKzppmztOC+VStjZ2YGu6zXRixPhDwwY+L6/p+K5ZzWU82EsPirBF3CPTJxgnYzIMizigTh0QzeniHboLdcKrAdHn+CDpmtQNMVxuqK1fuU0G56ANCMS1Pin2dSJhmHgytCVYyUTwzAaFuWPCp/4xCfw6U9/mv73lStXAABf+cpX8P73v7/ldXqCUKwPNmDm6JsRSj0LlXwlj1fffhWrmVUAliZEXQbHcIj6o23NP6+HbDaLUqkEn89n9pj4wgcihHZhJxQ7ZFVGZi+DVCqFeDyOE8MnIHJi3Ry3G4RiTecBpju0xEtQNRVluQyGYVCpmPYmwWAQfX19LUVG9gefeHTJioyK6mzXEfFFOlb8EF8x0g9UDwzDoFg078PR0RDOTJxBtVJFei+Ncrlc05QWCAQcZckketENnXadAwebKulY6UqlJnoRReDSmTAK1QLOXnT3EGSFPRPgZKUicILZewJ0fX87gWGYhhGnVSYOoO5sePLHfpCxD3nzi2aEU5EruHLqCvqNo00J2UFqxsdtDvmpT32q6x4UoEcIhYCkFxrVUXRdx507d7CxsXHAQmU5vYw/fftPHa3CWZaFrMo0jLc27pWVcsvT6qwbbDgcBgzQngA3vIUakQBpkhwYGEAiloCma9itmNP/iApG1mQa0XRLKJqmYWNjAwAwPDyMlZUVBIQAykoZumEWNAuFAra3txGPxxGNRjtOs1nrAyInwieaJ9ayXAYMIOgLdkQmhmEgk8lgb28PQ0NDNS4J9bAfjKEv5oPAqzD8Bob8QzX1kkwmg62tLUdZsjV6YRkWAcEkHntTJSEWMquEpHWG4kOuFMLbARU42KLrenPdrfdZpzAMw5Qjq9WWiarebHh7k6tT9KLpGo2oP/jUBzEijND7+7hgJxTDMFyroRwHeopQgMbS4UqlgsnJSWiahhs3btRsDv/34f/F387/bd0b07652hv3iMSy0YNC0joMw2B8fBzFQhGMyriaa3YiAcMwkEqlqKlkPBKnJoIE1v9PNPwVtr4xXzNYi++k8z3qj+L2vdvwB8yNT5Zl5HI5DAwMuHqikjUZctnckAVWQNAfhK7r9PtpFYZhYHt7m4olWh1+tJdjACWEUCQPWXt8uLGehhvVS6wOvYZhPO7aN8wUjMAL0HQNul/HLnYxPj5OoxetrGFme6bhxugWSIQicAJ4jm8qcLCrrWqil6qzI3cj+DgfKkoFHN+ZUMYavdgteupFLwzD4Aef/kE8N/Yc1tbWjn2IlaZpj+t0+yiVSi0dfHoRPUEo1lNtPelwJpPB5OQkkskkLl++TG8ERVPwhekvYGZtpuFrNJsBb5VYklqFZmgoVc1hQtY+j76+PvhFP+SyjGwpizDcO03YCcXarDg+Po54KF7XuoVA0RTslfdMfyrD9HtiGAYVtbW54cVikabVYrEYWLAI+AJghhmEqiEUi0VaE5AkiU5hFEXRVSEAUXJlS1n6d8SKX9XVhoV90p2v6zpGR0db9kJSVaCYjgJCHqFI46J/M1my1W+M53nzRL0/6wWAacfPB2iHeDKSRKFaQCgRarko3S1ETqQF7HZhr70EpSA4loOiKk1rL2FfGI/UR65GYU4WPeVyGblcDtvb2/D5fPjhZ34Yp8KnYBjGgWL4ccDegwK4p/I6DvQEoVhh75Y3DANLS0u4f/8+Lly4gPHxcbppZUoZvPqdV7GZ22y6bjvpH7unlVJSsLezh8H+QQRCAVovIQZ3bsLafU+GUwWDQfT39yMWiLUVDRHV2IFIjBehqM4NYta0WihkWvYLrEA9uXieR6VSgSiK6OvrowVTq49TMBjseEQvQb28ur2w7xN9YMHWFPat3fkjIyNtXYdRjQJiDjxvoM48JEdYZcmGYdDohXTtW2XJPp/PVJttmt5nJbkEv+CHoipmI+z+RkdEAmRjtBelnSS17UBgzTksbrhGtOMyHPFHkCvlOnIbbhX26AUG8N0nvhsxxDA1NQUAkCQJHMehWq0eWhTYDHZCkWUZiqJ4hOIWrCkvVVUxNzeHdDqNq1evmjfGPha3FvHa5Gstn6yaRShOMAwDG6kNlMtlDA0NwSf5kAiatQuJkw5NimwYRk2zYjweR1AMtp1asxb5yf+3qsbINEQD5swU8l5Jekji9tUxSgkMw6BarWJzc5MKEVjWbPaLRCI1iiZyQrcWr9txS21VyaXqKm30I/0IqqJiY3ODpqVa3bCIpcnUwxwAHtEo0OlexzBMzWx3uyyZFO8DgQAG+gbgE31UgURmvQicAB/vA9jHTZXxeJwWpUn0wrJsjZ1Jq+QZEkNQdRWqroKD+2kfpwmPHMOBZ3mkS2kaYR9F34fACfjxKz+OM/1nAIA+XwsLCyiVSvjmN79J3RzoMLgjilycxv8COHKVl1voCUJxSnkVi0XcunULgiDg5s2b9ARhGAa+fv/reP3+621t5u1u/tZ6ydjYGCRBQkAKIF1M05+ReAl+zo+QFEJRLrpCLgzDYG9vD5VKBcPDw4iH4wCDut3gzdYC6tuUk5SFpmnY3NiEj/fh8tnLUKGCYzg6O9xUPhWxtbWFWCxm9qDY1nOaoVEsFuvOM6m3kUR9UexV2q9JGTCwnd3G1tYW4vE4+pP98IumVLzeDHR67QyLgGhamhDb+kjU3VQMkSUTI02/3w/GYLC8vAyWZ2s+G+DxpErgcVMlz/KoclWa2nGyM2nWVAk8tlLRdO1INnQSvUT9UaRLaYi8CIExDzMcyx2q+EDkRXzkhY/gVPIU/TuGYRCNRhGJROgALDKbZGZmBoZh0NG9yWTyUKMXp2mNALwailsgcwHu3LlzwEKlolTwucnP4V7qXtvr1jNydAKpl5BUk1/wO27qqqGiIBdQqBbAsRwCUoCmXzpputM0DaqqQtd1jI2NIR6Ko6JUOh6c1EyGDDxu9JMkCfGBOPJyHlF/FBWlgrA/DFVTsbG1gXQmjf7+/pZOTtYTurVYWiqVamw2rPNMWLAI+oIdkQnweM4KEQgcmLgpmem7slLb7CawZkGaKAPJYK2oi4Riv8bBwUH0x/shqzKqSrXuZ0PqJYZhWthbZck+3gcdOm2qTCQSNVby9cb42q1Ujqoz3Go/L6syytq+qSVjRkssw0LW5IYTHtuFT/DhJ1/8SYzFxxz/nfR/2Ef35vN57O7uYn19HXfu3EEoFEIikUAymTTdtF2MXuqN/z3u2k6n6BlCIRt+sVhEsVjEs88+i+HhYfrvW/ktvPqdV7Fb3O1o/VZTXnt7e9jd3aU3T8gXQkV23tStUY+mPzaApPJKXqgxf2wEsrEDQCKRQH+kH3uVva6inmaEQorvsViMNvqR1BrDMKiWqtjZ2UGlXMH5U+fh8/s66kWwF0vt0ttgIIh4JI6qUm3b2t4wDOzu7lL7Gac5K7qhO3bsG7qBilap+X729iOUqIv2Y4Zh0Jkrw8PD6Iv20aip2WdjlSWTz8beVOkXHkcvxH3ZMAzar0HG+PZH+5Hn8/D7/VQkcNhgwCDsP2g/T5t3DaPmuxF50XRxbiGybAS/4MdPv/TTGI4O1/0ZXdcP3G8Mw5g+fJEIJiYmoCgKnao4OzsLXdfpXPhkMtnWXB8nOKW8ntR58kAPEYosy5iamkKlUsHIyEgNmdxev43/M/1/Wu4VcUKzCIVITIvFIkZGRsx5701mV9dLo1F55f7lSoIEn+B7PC/c9jtkY49Go6hWqwhLYWTL2Y7fq9N7s4MYWZLiOwsWIi/S4rt1hsnwyDAURoFSUShZkpnh7RZ07dJb1mBN1Vh2B5VK5UDxutGDRaxeFEVpa85KRamAYzlUlAqtnQCm9Jr0oEQi7my2VunyyMgI+qJ9de+pZrJkInqwz3qxRi8+wWfWvgyVRjvEMDSVTqFUKFGRACn48zx/KBsYaZR0siGql4a1115CUvvRS1AK4mMvfQwD4YGGP+eksLJDEIQD0Us6ncbm5ibu3btHU7yJRMJURLYZWdi75I/DdsVN9ASh6LqON998E4FAAKOjowc2wKAUxAsnXsC91L2aGkY7cLKaJ7C6546Pj0MURYTEUFM/rlbrMsT8EQCdJQKYxoY7uztUVRWPxrG1uWUOUJKijZZsGfaHlmxwpVKJFt9FTqQnQoZhIMvmoCqSCjjgnWRRh5H0i6IrdGNrFUTJ5Q/54Q/56QRFa/HamhqzPnjkO2NZFiMjI231E9iL/tbIspCOALKAgUQRaDKlshl0Xa8ZLJYMJ9sSVthlyVYTSlVVHWXJsiZTkie1IZETka/mEYlEaiTOW1tbtHZAiMz+OXeKeo2SBK0ovOyRpcRLkASJNrw6RS8RXwQfe+ljSIaa27/rut4WAVijl1OnTkFRFKTTaaTTady+fRuaptHaSyKRoGMUGsE+XKtQKFAHhicRPUEoLMvi+eefRzAYxOLiIkqlWjnryeRJnEyexPc/9f3Yzm/jbuou7qXuYT273vIGVm/zt0tzfYKvpXG55LrbVY6pump21e933BuKgYunL0KSJFS1KqpaFT6juzDaipq03L6hJKnRcBxnNkAqFVqgJfWjSCSCRCLR9Ma2Rikcw5n9Ki34czkpuYhU1iqXdRr1K4oidnZ2agweW0Uj+xbdMJAtFQGRgS+kmB37+0O02k2/WAlvdGQU0WB3ZotW0UMrsmQAgGF+/+lS+nFTJSfUSHhHRkaoCo3UDkRRpOTSSUGaWO03apTsRDJsvdes0Qv5+6g/ip+59jOIB+JNVjLRLqHYIQgCBgcHMTg4aKbu9p0PUqkU7t27RyPNZDJZN3qxRyhuWdcfF3qCUABQ6WmzIVv94X70h/vxvrPvQ6FSwP2t+7iXuoeHOw8bWr07bf4H6iVSiM6CbwWdyoatCjKS9y/KRQicgLAUBsdzrkmSyTrW4jtJJ4alMC3QMgxDpcpknku7sA8SI6kxWa3152pFyWXtI7CO+iUpB5ZlKQG2Ipclqa1GB4VCAdA1BgxjIBze79jXatMvToV9O0iEJ0kShgaHEJKaR7vtwC5LdorsgsEgEuEE8noeHMc1bKpUDOWAxJkcLKxpOCKgaASRE+nohUbotgfFHr0MRYbwE1d+AtFA65F9KymvVsEwDK2FnTp1Cqqq0uhvfn4eiqLUKMdI9GI3wiU1lCcVPUMoBO0M2Qr5Qrhy4gqunLgCRVPwYPsB7qXu4f7W/QOnI+sGba2XDA8Pw+/3m35clVzbUmSyXqsPBzFSDAQC5mTFQJRuNpquoagUwRs8wmzYHIgE0867XVsL6zUSWSkpvsMwPztiOw8DNXNVWgnVW4Hdn8svmPYX2WK27bVInr9araK/vx8cx9GeF13XD6R/rLDKghuBSIbD4YPTEusV9u1W/OT7DYfDGOgbgCRIHbtPtwp7ZGdoBirlClJpc6KmXbJNoide5M0eI0tdTOGUmqZKYgmzt7eHnZ2dhk2VIi+CAdOyQ7FbaZ3+cD9+8upPIiS1d7LvNkJpBJ7nMTAwQGcGkUhya2sL9+/fp4PGiFs5gVdDcQnk5up0yJbACbgwdAEXhi7AMAysZlZxb+se7m3ew05hh0Yo5GEyDMOslwgiQr5QR+kIcs32sLUeiONtIpFAIp4wZ4nYTq6E+A7IXn3m6bidCAp4rIIaGBhAOBwGA4ZOOSRChVQqBVVV2ypstwvN0MyRrpU905hTDAOMOXFP1Rp/3/UMHklXOtn0nHpewoFwjSy4EUhBvhXJsJMVf7FYRGrTtKzpT/SDZVjXxtW2Cp/ggyEYZm9LOHBAsk0giiIlZsMwUFYfF/ZFToQkmKkulmEPyL+tXllEIBCPxGkNpxW4pTAbigzhp1/6aTrBsR0cJqFYwTAMJfyTJ09CVVVkMhns7u5ib2/PnG6aSuH1118/8pTXo0eP8Ju/+Zv4u7/7O2xubmJkZAT/4B/8A/yrf/WvOnJh6BlCIeh2DDBgfoHjiXGMJ8bx9y7+PewWdvGdhe/gK4WvYG1tDT6fDwMDA/AJPnAs13E6gtyMzR4Osqnn83kMDQ0hHomDZVnHZkWnVJdu6DWjX/2iHwL3eEJdvdekctH9/hGBEwADKMpFMAxDLUo4jmu7sN0O6HRFuUjfj/XUTh1s1YMS62YGjwzDQJIk2klu3UB3t3axi10IPgHBYLCpDxbpQYm0qYdQdRVrW2vY2dnBwMAAxvrHzMiwSzfeduEX/FB1tSb1a5UlV6tVOp1S13UsLy/TupQ14lB1FUr1scMwaaqUNRksyx5oqizlSkjvpCFIAo1emh1M3LBdGY2N4qeu/hQ1qGwXx+XlxfM8naZYLpcRjUaxvr6OL33pS5iamkIwGMQ//+f/HB/+8Ifx/ve/37WMgRPu3LkDXdfxR3/0Rzh79ixmZ2fx8z//8ygWi/i93/u9ttd7VxKKHclQEhOBCRgjBk6eOYmqVMXDnYd4sPOg6xNks1qHruvY3NyEqqoYGxtDLBQzG9pk5yijlQZM+6zwgBig3ci6odcU3wVBMF1whQAq6uPieyczTDpBK9MV7Q62xJizUC5gY3OjLYNHsoEOJgdRUSooloo03WBXRtk3PRqhtCEZNgwD2WwW2WwWw8PDSEaTyFfzVJBAUn3UaBSHM98kIAbMMdJ1UqNEfGJ1OiB1KbsXWyNZssRLVKjAMiz6o/0oq2VU5SqNXqxrkfqLk9qwm3vuZOIkPvriRyHynXuZtZpZOEzouo5AIID3ve99+MpXvoJf/uVfxs7ODlRVxT/9p/8UqqpieXn50J7PD33oQ/jQhz5E//v06dO4e/cuPvnJT747CKXTlFc96LqO+fl56th76dwlAMDzJ56Hqql4tPsI91L3cC91r2N7k3qEQgrhoiiane/BeNM6TbvFeOIsDOynJxgR66l1SLyERH/CtOjWORQqBTCsOW6VpN6Ib9Fh3aydTFckdjCKoiCVSiHsC2N0ZBRVrdpQdGFF2GcOpyIzRsgJj6TG6imjSA2l1S55EgWWSiXaY0JIneBAYV8M0QFP7VjxN0JICjVUoRHbnEQigWj0cfhFmiDtXmz1yNcwDLOpsmJed1gKm1MoOR9YkQXHcQeaKnd3d6nzMlmPSJw7ve8m+ibwkRc+YkbcXeCoUl6NYBcGVCoVPPvss/jN3/xNOrbiqCXEe3t7SCQSHf1uzxAK+dDcjFCq1Spu3boFXdfx4osv4s0336y5kXmOx9mBszg7cBYffvrDWN9bN4v6qftI5VItX7fThlkqlei42b6+PkR8kZbqNN2ouwrFAn3NRCIBv+DHiYETSO2kkN/K00bBSqXS8rCpjmAA0UDnMlkSPYVCIUSSEepp5RfMVB8phMPhOYv4I3VTmM0MG3d3JwCwCId1OC5ugbWpcmRkBMlwsmnR37Gwz0lmD08DK/5GaNZ8Sw4PpIG1HqyyZOAg+Vpnvfj9foR9pkJQ1/XapkpegmZoZqHf4utmnxvD8zz93XY2zHMD5/Bjz/8YeK77rasXCcU6rZFhGAwNDR3p9SwsLOAP/uAPOopOgB4iFAIiG+42JM5ms7h16xadn0JIql6YyzAMRmOjGI2N4gMXPoBMKUMjl+X0ct00lJMcmXSh9/f3Ix6NQxKkljfXTgmFSKDJiFvGMKMRTdDQP9KPPq0P2Z0sKtUKYAC7u7sol8sIBoMNzRrbBQsWQX/7zsgE5DTtNAGyrJRpjUXg952SDYMaczpZfNSDfY58tVpFYX/0bz6/hrU1BsFgkJ7OrddBUoqA2ceRCCU6er9OhX2g9VG7EV998gQee4d1cniwy5Kts178rB9bwhYdtEYK+/amSj+/P26XqYDn+ZoGzb09c17P8vJyy02Vl4Yv4f959v8Bx3afpjKMg7PcjwNOhOJGUf7Xfu3X8Du/8zsNf2Z+fh4XL16k/722toYPfehD+OhHP4qf//mf7+h1e45QyIdr12e3g9XVVczPz+PcuXM4efLkgc2glZsoHojj5YmX8fLEy6goFSxsLeBe6h4WthZqVFZ2OTIZDzw6OopYOHZgTkQztEsoJO1SKBSoZYzACGA4BsVqkebKSU/B0OiQ+UCq5s27ndqGZtRazXd6auNYDhIvdZQ6BB5vgK2YUFqHO3Esh7AvTBVK7aaSzPvDB0U23/eFC0NQlKLjJEae55FKpWhTm9X0sBuoulrTsU/s3u09PACaRoBWVVw9f7N2YJ31EvFFsL23XVdVRxohdV1HSdm3GbI0Vaq6Su9JlmURj8dRKpVoUyBJQ5JJleTZfWb0GfzQ0z/kWkRBDoG9FqG4JRv+1V/9VXz84x9v+DOnT5+m/399fR0f+MAHcPPmTfzn//yfO37dniEUa8oLMJv/2iUU67z5F154AcnkY/sFcuN0kk7zCT48Pfo0nh59Gpqu4dHuI9xPmQ2VhACs9i3EKbgTI8V2CMVe8Od5Hj7OB1mToSpqzQyTA13lAhCMBRGIBsBoDKrlKrK5LLVWJ4OyWv0OiJKrE5FDKwaP9cBzPEROrNlc6/WINAIpyPsDBgIBHsDBSYxbW1v0oBMKhuDn/R2TZyMYMGqHVdkK+40iQPJZFovFtsYetwLSkFpPVUfuf2vtxd5UaRgGeM68TzmRg0/yQRCEmgZNYgtDrHeunb6GD174oKubfy8QimEYhxahEBVZK1hbW8MHPvABvPjii/iTP/mTrj6TniEUApZlwbJs2xt/tVrF5OQkVFU9MG8eMDdqN+ozHMvhTP8ZnOk/gw89/SF8gf0C9rCHW4vm7JaBgQHEgjFzIl0HOfFWCUVRFGxsbEAQBIyNmTLVkBSiReFWZpiQ1wMPSGEJg+FBQAe0qoa9wh6Wd5chCAJN/dRLjQWlIKpKtSObfVKLkGW57T4YqyutFQdSSYIfBmPQcc5OoAV5m8KLnM5ZlkWhUEAkEoFP9KFSrOD29m16Og8Gg66PQCYghX0yqMowDET8EZTlco1QgUTI1WoVIyMj7vUUGUDU7+xu4DR218kuxypLLpfLSO2YLteqqlJZMjG0JBtqtVrF2chZnJXO4o033kA4HKad5uFwuLtO+31COc6Ul/0aSAPkUfahrK2t4f3vfz9OnjyJ3/u938P29jb9t07qNz1HKED7hfm9vT3cunULsVgMV69erXuTHIYkOSpEwRd5/NO/90+RGExgcWcRdzbuoFAptB2dAK0RCrHFIA+YYRimrUg5Rx+ybDaLTCbT8gwTChbg/BwS/gQSfQkYioFioYitzS3o0Cm5kJ6OTpRcBDV+V6OjbT3crciRgf1U0n7PC7Hn5zjONOy0pC4b9aCQoVj9/f3oi/cBMH2l4lqcFq7J9ET7LBO3wDIs/IL/QNc9KezLqowHyw+g6RpGRkY6ThfbwYBByB9qaU5NPbucUqmEbDYLljWbJMvlMuLxOGKxmNnPsh+5kJSlT/DBgIEbEzfwPee/ByzLolqtUiuT5eVlcByHRCKBvr4+JBKJtt+vppny+eM0YSR7kd0c8ijH//7N3/wNFhYWsLCwgLGx2rkxnTzTPUMo1i+2Henw2toabt++jbNnz+LUqVMNb5BOIp96MAwD9+7dQ7FYxNjYGCYmJgAAL5x4AS+ceAFVpYrF7UWz7rK90HKTWzNCOeC3ZYB23JPftdqldzVtjgEYkUEoEUIoHgKjM6iUKkhn00ilUuiP9iMn5Noe8Qs8llSTJtN2HuywL4xCpdB2BEgK+AQib6aSFE3BXtYkF3uEQoh5aGgIfdE+M524T2L20zlJjbXS89IOBFYAx3E1105QUSooVoqmlQrL4+zEWbAc21G61Q4WZkTUaVrPKksm/TqZTAYsyyKTyaBcLtdEL4Zh0APAd539LtyYuAFVNaMWnucxODiI4eFh6LpORSgPHz7E3NwcotEojV5amSfSKwovoDbtdtTWKx//+Meb1lraQc8QihWtRBK6ruPu3btYX1/HlStX0NfX13TdZsaTrUJRFExPT6NUKiGZTDoqaCRBwlMjT+GpkafMruTMMu5tmqqxTClTd+1GhLKzs0OHNPn9fvAMD47lUKgWamaYtNMI2DIYwOAMSGEJI+ER+Fgf8oU8cvkclneWIYoijV6apX5IhBWNRulgr1bRyC24XVhnb+RzEiAHMRDXwLEFqJpaMxQrGU3WtUwHQG1I6sluSbG5XVWdxEswYNSdBUKMRnmeR/9gP4pKEVAshX32YDTWCliGhV88GBF1CtI8SVSITlJi8vn9/St/Hy9PvExrDLqu1zy3LMvSe+fs2bPUq44QjCAI6OvrQzKZRDwed4x8e0nhZfUE9NyGDwEcxzWMUGRZxq1bt6AoimO9pB7ciFCKxSLeeecd+P1+XL9+HfPz800721mWxankKZxKnsIHL38QW/ktKkm2W/A7Wq84Fd95n7khanLTGSZugliTl+QS+ACPRCCBhGGmxgqFAjbXNwEWdPO0d0mTvoh2HY0ZmONrOx0R3AiGAWT2VEDMwxdWoWkG8pk8IAOnT55GyB+qGZvbCur1vJDCtTU1Vm9jc7JSsYLU0Xw+3wEb/3qFfdVQUa6WG3bsE7VeO+rERiApw8HBQXr6ts96KZfLKJfKmOAnUF4qYzo/jb6+PvT19dHaCyEXIoIBHjsvDw8PY3R0FJqmIZvNYnd3F/fv30elUqlx+SV7xXHZrlhhJ7VKpQJN04405eU2eoZQ7Cmvehu/tV7y4osvtnUK77aGsrOzg8nJSYyPj+P8+fMdF/oHwgMYCA9QC/57W/eoBb+dUMimwfM8RkdHTSdYMYSi/Lj4TprzWp1h0inIafmAkms/NRZOhBFOhMGo5jVld7NIqSma+lEUBblcru2+COIWfBhkAgDVKrCxYW4ufr+O9Q3T7mVoaAihQAhVpYqwLwxVV9seIgY497wUi8W6Y34ZhmlqpVKtVrGxsYFQKIRkMtn0O3fs2GdYlJXawj7P8eBZ3jVTy1wuh93d3YbfOcuyCIfC+OkbP41nRp9BoVDAzs4ONjY2cOfOHWoP1NfXRzv9dV2nf+zRCyEQADQFubOzg4WFBfh8PiSTyUMTULQD+3AtMgfKi1BcRr0IZX19HXNzczhz5gwmJibaviE6JRTDMLC0tIT79+/jqaeewujoKP23bqOekC9E6y6KpmD64TT+6lt/hYAUwE5250DxnXTck4KivaHxsBCUgnQQVzMYvAF/xA9f2AfWYFEtV7Gb3kVJLkEURTp21t4w6AQiC27FLbhT5CwZNFnegCAwGBkeQTwYp/LcmiFikjlRrySX2la2WQvX9brI+2P9qAgVSD7n1BhJGTZS7zWCvWPfL/gh8iI0TYOiKS2P2m0Ga2NlI4NDlmHxI8//CJ4afgoAaF3KOtN9Z2cHU1NTMAwDyWQS/f39lBhIaowU9q3RiyRJGB0dxfj4eI3L78rKCk1dk+il236ddmGPUAqFAp0/86SipwiFnM7tEYq1XvL888+3rK+2oxNC0XUdc3Nz2NnZwUsvvYRYLFbz7yzLuuY9Riz400NpXLx4Ea+/8zqeefYZZPQMdgu7CEkhSib23o3DvAk7VXIxDANVV7GztwOd0XHqxCkwGoNCsWB6jDmYEVoh8RIMwyEichlbW+R1DYTDPIYGBs2RBg4RkWZoB5ySnYaItQp76ofXeaQyKZQyJTrnhaQOeZ6nUnDiw+YGyopZG9INnSoGARzwJWsHmUyGGmY22qh5lsePXfkxnB887/jv9pnuZC7L0tISZmdnqbVRX18fFUcQcnGKXggZRaNRrKysIBqN0gmLxCqGDNw77JRYvR6U446cukFPEQqBdeOXZRmTk5OQZRnXr1/vSgHBcVxbI3uJF5hhGLhx44bjg8FxHKrV9gqejUCK6/fu3cOH3vchJBIJaJqGncKO2a2/dQ8ruyvYTG0e+gyTRv0HrYDY4/M8j5GREfMBFUyCiiQigAqUS2Wkd9NIqamabv2wP9ySLNgNLC9rII/C0MAQAr7Wi9FOQ8Q6dRaOBWLIVXLo7+8/MNxqe3ubqh/j8bir0ShRupHPuqZjf19mLatyS5EL6dLP5XJNVYYCJ+AjL3wEp/tP1/0ZKxiGoVHZ2bNnUalUaPTy6NEjcBxHySWZTFITSlJ7IX+Ax43T4+PjOHnyJJ0Pv7u7i9nZWRiGgUQiQQnGzQZRAnsdh0xr9AjFZZAHJ5fL4Z133kE0GsULL7zQtWqpWbHfClKrSSQSuHz5csPelnbnyteDqqq4ffs2AODatWsIBoP0YegL9aE/3I/nR57Ht976FralbSAKLGeWXXOttYJ6cnVIJlaDR8ccPwNAAPxRP0YiI2B1FnJFRjaXRT6dR0pI1UhKDwvFYhFra/vr6yIksT2rHCvq1inkMhS9gVOyg5WKfc5LJpNBJpOBJEnIZrPI5XKu9LwExACqatUxlWnAQWbN1y/sW7v0R0ZGGn5vIifiJ6/+JE4mT3Z03QDg8/kwOjqK0dFR6LqObDaLnZ0dLC4uYmZmBvF4nBJMIBCg5KKqKrWMIbJklmXR399P58Pn83ns7u5ibW0N8/PzNO1MIiE3Nn27vVSpVDo8w9YjQk8RCknlcByHbDaLN998E6dPn8bp06dd+QJb3fw3NjYwOzt7pL0tpVIJ77zzDo02JEmq0akzDINsNovJyUmMDI3g/effb6bbNBUPdx/ifuo+7m/dd6XWQJRcnfYfNDJ4dALDMDA4A0JQwJm+M8gX89BkDfl8Hutr62A5lkqSiWOyGyAFY447DagSJF99eW67sNcpyBAxRVPMoWj7b4E2Djbx5SInfp/P51rPS1AKNpRC22GVWdsL+7IqY2dnh/Y/NboGiZfwsZc+htH4aN2faRcsy5qTUBMJnD9/HqVSCTs7O7QYL0kSbYJcXV2Fpml4+umnqbmrPTVGRBQTExOQZRm7u7vY3d3F5OQkGIahkUsikeg4Q+BFKEcAXdexu7uLbDaLF154oeN6iROapacMw8D9+/exvLzccq3GjQglk8ng1q1bGBoawpkzZ/CVr3yFdsySG259fR3z8/M4f/48xsfH6e/yHI9zA+dwbuAcDMPAxt4GlSRvF7brvWRdkOFJndYt2jF4tILKgst7AAuwPhZRXxTRviigAqViCTtbO9AMraau0EkvgX2k8C/9v37IagUV+fDSazVDxPadknVdB8MydYm73onf2vNiGAYt7Ft7XprZ5VhnxnQCK2EahoFcOgdWZXH25FnIev2I2S/68bGXPobh6HBHr9sqAoEATpw4gRMnTkDTNKTTaWxvb2NmZga6riOZTFL5ut/vp6TiJEvmOK6mqZIcRJaWlnD79m1EIhFKMO3UQA7Lx+s40VOEQvpLSqUSYrGYq2QCNC7Kq6qKqakpFItFXL9+veUvttsIhXT6nz9/nt78/f39eOuttxAIBNDX1wdZlrG1tYXnn3++xvDSDoZhMBIbwUhsBO+/8H5kShnTxHLrHlbSK01PokExSKc6totuDB4byoL3U2OBWMA0stT3jSzzWSq5tdrMt3KdZCjW6OgokuEklWC72QfaCIqmQDd0+HgfStUSQlIIHMOZJ/39lFmrvlykD8Pe81IsFuv2vDSaGdMuyBAoMhdGNmTwnGnFb6DWPy0oBfEz134GA+EBV167VRCbluXlZYRCIVy4cAHZbBabm5u4e/du27LkSCSCWCyGM2fO0DoOIRie5ym5xOPxhml6j1AOGTMzM+A4DmfPnsX6+rrr69cjlGKxiFu3bkGSJNy4caOtELbTCMUaDV25cgXJZJKekJ577jlomobt7W0sLCygUqlAEAQ61jeRSLRswX9t4hquTVxDWS5jYXsB97fuY3F7kaYuCLrx5OrG4LEtWfB+t74YEjEQGsCgPghN1pDL57Cyu9K0G13XdaRSKSpmSIaSh9bb0gh2KxX70C2e5bGyugJFVtr25WrW85IMJ5EVsq7UpsjnqWmmfxh1Crdb8YtBRPwR/MjzP4JksP6B6LCgaRomJyfpoD2e5xGLxXDq1Km6smRrUyVRjDlFL4Ig0KZKUsfZ3d3F4uIiyuUyYrFYTVOlfZSGVbRQLBa9GoqbePbZZ8GyLLa3t10dA0zgRCgkLzo6Oorz58+3Xdx0GrDVDKqqYnp6GoVCAdevX0cgEKBkQuolqqpiaWkJPp8P165dM2eXbG/j7t27qFarVP7Y39/f0sbgF/14ZvQZPDP6DDRdw9LuktlQuWla8HdqZ2KdtdKuwWPdRskWYbAGWB+LmC+GWF8MUMzUmHXGC0mNGYbhylCsbkGk0PVqNcSXCwBOjJ9A0GeqGjuR8Np7XvycH5vpTVNZZ7E7CQaDbdemdF3HxsYGADxW8DnAgAGBF/BjV34M8UC8ret3A6qqYnJyEgBw5cqVA+RslyUTr7yVlRXMzc0hEolQciGeZI2il1gshkQigXPnztEa1+7uLh48eABRFKkCjUSTXoRyiBBFkX7IbrsCA7WEYhgGlpeXce/ePVy6dOmA02Yna7aCcrlMi+8vv/wyBEE4UHzP5XKYnJxEMpnEpUuXwLIsJEmiBUdS9CaDxKLRKCWXVmTVHMvhdP/p/397Zx7X1Jm2/yuEJex7wiIiKAgqO0qx1mq17gLW2hlbrU6XmbH7/Kb7tJ12ZtpOa6ftTG2dzlurnW5vW3Gre62IWpfKKoKyI7IlgUAgIfs5vz94n6cJspOQg57v5+MfQiBPwsm5n+derguRgZFYMn0JmpXNqJRVolJaiZbOliG/ltEIPA5VLXjICAA49/xeVx9XODAO0HZr0dHRAam0x87ZyckJQZIgi4HFsaR3e25viC4XsUFgwPyy0/8/pWQHBwfojLrrTpgDYtZF1lvuxNwCmdSmBnNONJlMaG5uprWFgTZh/u7+uHfWvfBytc7MzHAwGo0oLCyEQCBAUlLSoJsdgUBA35/JkydDp9Ohra0Ncrkc9fX1cHBwoMHFz89vWEOVJpOJDlWWl5dDr9fT9Wg0Gri6ukKlUo37gCJgR1qVswGkpY/Y986fP9+qv18mk6GyshLp6ekoKyuDTCZDUlISfH1HvnMazlo7OjpQUFAAsViM2NhYehECvyiOSqVSlJaWIjIy8jq3yb7QarVobW2FTCaDQqGAm5sbDS5D6bDqTaemE5WySpRLy1GvqO+3njIagUdPkeeohuaGg06nQ6u0Fe6iHh8RrVYLk/AXh0pr2h8PxGBSKgPpcvWFyFEEF6fBTcQEEMBD5DFgxx6ZeVGre1wq9Xr9dT4mZD3mQU8ikQy4zkCPQNybdi88XMb+JkmCiYODAxITE0ctBGnelkxqcD4+PjTAuLu7XzdUaX5rJT5PDg4OPcO63d0oLi6Gg4MDCgsL8eGHHyIiIgJisRhffPGFTVvlzcnIyEBRURHtyly4cCHeeusthISEjOj3cTKgqFQqnD17FnfeeadVfz8ZWnJxcQHDMEhOTh613EJXVxfOnz+PhQsXDvg4IhsTFRWFiRMn0ouPnEpYlkVdXR1qa2sxY8YMiMXDL1wajUa6o2ptbYVAIEBAQADEYvGQ6y7m6Aw6VLdWo1JaiSp5FU3TjFTgEbCuWvBgkN23r68v/H394eLkAo1O0yNkqVZBoVT06fFibTxdPGnhvy+Gq8vVG0cHR4icelJW5oHaAQ5wFw1fft7cx0Sj0VCfF5FIRGdhBjuRBnkFYe2stXBzHvuagMFgQGFhIRwdHZGQkGATVWGNRkODi0KhoOkscnohn2nzzjEC+cwXFBQgPDwcTk5O+P7777Fjxw5cuXIFAoEAd955J1atWoV169ZZfe3mvPfee0hPT0dwcDAaGxvx9NNPAwDOnDkzot/HyYCi0WiQm5uLxYsXW3X32NjYiJKSEgQFBSEuLs4qF5parcbp06exePHiPr/Psiyqqqpw9epVJCQkICAg4Lp6CcMwKCsrg0KhQGJiolUkNciOSi6XQy6Xj6ju0vv31SvqkVuci8KaQrj7ug+rgGhLteC+IEEvMDAQ/r7+EEBwvYQ7C8AEaLu1aO9sh0b/iz/HSDxe+sLL1Qtdmv6Vikery9UbAXqEJYVCIQQQjDq1R2ZeVCoVVKqe5gHz1Fhf79EE3wn4VeqvIHIaW20soCeYkJSyrYJJb0hbMtnM6fV6av7VX1uyyWRCfn4+pkyZgoCAAAgEAqxfvx6zZ8/GokWLcPDgQSiVSmzevNnm6zdn3759yMrKgk6nG9F8DadqKARykfaeJB0NLS0tKC0thYODAxISEqwWqIhnNsuy1/1Ok8mEixcvorOzE2lpaRaT7ySY6PV6FBcXg2EYpKWljc4Qy4zeg16kqD/SugsAqFpUiHSOxOq1q9HNdtNhyqaOpgGl3W2tFmwO0Xsiplj+3v4wMsa+1QQEABwBkZcIQZ5BcGAdYNAaoOxSDtvjpS8Ga8+1hS4XCxYagwYiiNCt74aLowtETiIYGSO69d0j0mNzdHSERqOBl5cXPD09odFo0NXVhdbW1utmXsL9w/Gr1F/B2XFsUjbmGAwG5Ofnw8XFBQkJCWMmTy8UCulnaerUqVCr1WhtbYVUKkV5eTlt/w8ICICPjw/VJiQGZKSGWl1djdTUVCQnJyM5OXlM1m6OQqHAl19+idmzZ494WJNTAYV8YMmuwhoBxfyEEBsbi7KyMqueeshF23utWq0WBQUFEAqFuOWWW/osvqtUKhQVFcHLy2tAeZfRQny6yeQvqbvI5XJUV1fTonp/dReiymowGDBz5kyIRCJ4wANiTzFunXIrVDoVnXepa62zKDqTqXtbqgUTzGdhQkJC4Ofp16OQ3E/dwhyBQABWwMLRzRH+bv4IYAPA6Bmo1L94vJinxga8hvqQUukNOUGJxWKrFmIdhY5wcnCinXM64y/mWkKHHqVksEC3YWiOjiQd5+XlRWtlLi4utEuJpMaam5sR4hmCuAlxaG9rp1paY4Ver0dBQQFEIhHtFrUH5p810pasUCjQ2tqKkpISmEwmODk50ZS7h4cHGIbB559/jqqqquvEZ8eC5557Dlu2bEF3dzduueUW7N+/f8S/i1MpL4ZhYDD0aB4dOXIEc+bMGZUYpHl7LtECO3HiBBYtWmS1C85kMuGHH37AHXfcQdNISqUSBQUFCAgIwLRp0yyK70R2nlxgEydOtJq0zEjor+5C5MHJsKmrqyvi4uIGvUkYTAbUyGtQKatEvaIear16eB1JI8R8EDA4OBj+nv4jnqu5/pcDAlOPx0u7sh06o67fjqihFMGJmoBEIrHq3IGzsOcUNRR3RpoacxD2q5Ss1WrR3NxM/d8HIkochTsi7kC7oh1yuRzd3d1USyswMNCm8xV6vR75+flwc3NDXFyc3Y2z+oNhGBQXF0OpVEIkEmHPnj04evQopk2bhkOHDmHPnj39ps6Hw/PPP4+33nprwMdcvnwZMTExAEDrQFevXsVrr70Gb29v7N+/f0T3JM4GlB9//BEzZ84ccSqAaGOR4y8Rgjt27BgWLFhgNYVelmVx5MgRzJs3DyKRyEIHLDw8nPark1MJANTX11NvleBg20pQDIe+6i4sy8LHxwczZswYdgMDy7JoaG9AhawCldJKtKnbbLJuc+vj4OBg+Lr72qzwz7IshKwQOq0OHcoOdHV3wUXU0xHl4eEBXw/ffk9j5rpcQUFBVvXfIPMtIxUKdXZ0hshR1KOUrO+m7pJ+fn50erw/pgVPQ2ZC5nXe6OZFa1dXVxpcfHx8rHbTHy/BhGVZXL58GQqFAqmpqfRe8e677+L777+nHZpLly7FvffeO6rAIpfL0dY28GctMjKyzzpqQ0MDwsLCcObMGaSnpw/7uTmZ8gJG565IhhVDQkIwdepUepGZp6esFVCIUqnRaERVVRVqa2uRkJCAwMDAPovv5eXlkMlkSElJscvxdiDM6y4+Pj64dOkS/Pz8YDAYcPr06WHXXQQCAcL8whDmF4YFMQvQpm6jOmON7Y3DstTtD3NP9ZCQEPi6+dq0ViMQCMAIGDi5OSHQLRBiVgyTrkfIsrW5Fc1o7nNY0FzyZTAl3uEy2HzLUDAXftRoNFC2KhERGgFHkeOAvzc+NB4r4ldct5s119IyGo19amn1tvgdLjqdDvn5+fDw8KBCj1yEZVlcuXLFIpgAwIULF7Bjxw58+eWXWLFiBc6ePYsDBw6gtLR0VAGFfEZHAhnSHqklB6dOKKQfHgBOnz6NqVOnDvuNqa+vR3l5eb/DitZIpfXmhx9+gK+vL1QqFVJSUuDu7n7dycRgMKCkpAQ6nQ6JiYmcdmWrr69HVVUVpk+fDolEAgAWdReFQjFo3WUwuvXddJiytrV2RDtrMljp6uoKsVgMT5HniBWSRwORUtHqtRCYBFCr1VAoFTCYDLRjTK1WQ6/XIzg42Kr+NQPJz48E4v9uXttxdXKFs9AZepMeGoOGPjZlYgoWTx9eJ6b5NLpcLodKpYKXlxcCAwMREBAwZHFFEkw8PT0xffp0TgeTiooKyGQypKam0s/9wYMHsWHDBuzYsQNr1qyxy9rOnz+PCxcuYM6cOfD19UV1dTVefvllOgs3kgYhzgaUs2fPIiIiAkFBQUP6WYZhcPnyZUil0gGHFY8dO4a0tDSrGRRptVrk5ubC3d0dqampdNof+KVe0t3djaKioiHXIewFufibm5uRmJjY7wlqsLrLcJsLRiLBbz5YGeAXAFdnVwvvjrGi31QTix4hy24d5O1yaA1aCyFLa5xQhis/Pxikc0ssFve74XISOsHV2RXTQ6bjjql3jPo5yUaltbUVbW1t181z9HUtabVa5Ofnw9vbG9OnT+es3DvR62tpaUFqaiqtIx07dgz33nsv/vOf/+Dee++12/pKSkrw5JNPUlHc4OBgLFmyBC+99JKFzflw4FRAAX45al24cAHBwcFDkkQhhWOj0Yjk5OQBd/85OTlITEwc1XQ8gRiAGQwGJCYmws/Pjx4ZSTBpb29HcXExgoODER0dzdmL32QyoaSkBGq1GklJSUMuolp73oVI8JdLy1EprexTgt+83dbfpyeAWcvHZDgMlmoymUxU5yxYEgzWwKJT1Yn2znYIHYWj8ngZrfx8b4bTKHDr5Fsxb+o8qzyvOUSehJxeyDwHOb2IRCIaTHx8fDBt2jTOfp5YlkV1dTUaGxuRmppKA3Rubi7WrFmDDz/8EPfffz9n1z9SOBtQCgoK4O/vj/DwgR3dyE2dFI4H2/2fPHkS06dPH1AGfii0tLSgpKQEkydPRkNDA6Kjo+mEbG8Pk6lTp45YK2ws0Ol0KCoqojIVI03JsCxL513kcjk6Ozvh5eVFU2MjSTP2luBv72iHQqHomf739htVIXo0uLu4Q6vvvyW5ty6XRUqGBVgjC41agzZlGxWyJP8GS994ibzQpe1/WHK4dHR0oL29fUi2A/Oi5+HWKbda5XkHwvxaam1thVKphJubG3Q6HXx8fMZ0zmQkVFdXo6GhwSKYnD59GqtXr8a7776Lhx566IYLJgAHA4perwfLsiguLoanpyciI/v3myY39eG4Ov7000+IiooakbQJ0HOh19TUoKamBvHx8RCLxTh79iyEQiEmTJiAgIAACIVCVFVVoaGhAQkJCfDz8xvRc40FKpUKhYWF8PHxsXouWqfT0eBC6i6BgYEQi8XDrruwLIvS8lJcKL8AR39HdBo7odKqxsRzvjeDSakYDAY0NTXB1dV1cF0uFgAD6DV6tHe2Q61VD+jxYk3pGvOus+Dg4EFz5nfG3olZEbOs8tzDhWwcHR0dYTAYLIQax3rmZTBqa2tx9epVpKam0jrU+fPnkZWVhddffx2PPvroDRlMAA4HlNLSUjg5OSE6Ovq6x5DjZG1tLeLj42nheCicO3cO4eHhI2rXNZlMuHTpEtrb25GcnAxPT0+YTD0dPjKZDDKZDN3d3fQmEB8fb5XUmq1QKBQoLi5GWFgYJk+ebNOLvHfdBfilG2WwugupjykUCiQlJcHDw8NCgr9SWjlm+mCDSamQQUBPT096Yh0OAkYAo87YkxrrareYRBf7iK0aTBQKBbq6ugbtOhNAgKUzliJpYpJVnnu4aDQa5OXlISAgADExMWBZ9jqhRl9fX5oas6enSF1dHerq6pCSkkLrtPn5+cjIyMArr7yCp5566oYNJgCHA8qVK1fAsixiY2Mtvm80GlFSUoLOzk56Ux8Ow6nNmKPT6VBYWAiWZZGUlARnZ+frOrnIdDzDMHByckJXV9eoUz62gqTjYmJiRlyAGykMw0CpVEImkw1adyHDqTqdDklJSf2mZEYqwT8cBpNSsbYuF1gAxp5iuaZbg25jt4XHy0hPk+YtzMHBwQMGEweBA5bHLUf8hPgRvojR0d3djfz8fCpr0td7SmZe5HI52tvbr5M6GavU2NWrV1FTU4OUlBQ6P1dcXIxly5bh+eefx7PPPntDBxOAgwHFYDCAYRhUVlZCq9UiLi6Ofq+7uxuFhYVwcnJCYmLiiDplhlqbMaerqwv5+fnw9fWlXSW9i+9KpRJFRUUIDAxETExMj2+FWcqnra2tZ5f5f8HFy8vLLhcXSdnV19cjPj5+1LUka6ynv7qLj48PysvLIRQKkZCQMOTajlKjpHWXgST4h75IwNvVe8D5FlvocgE9pwNPkSc6NZ0QMAJoNVoolApodD0eGiTADDXlw7Is5HI5tFrtoC3MQgchMhMyERsc2+9jbEl3dzfy8vIgkUiG3NBCTsLk9EJmXshJ2Fay8NeuXUNVVRWSk5PpIGhpaSmWLl2KJ598Ei+99NINH0wADgeU2tpaKJVKJCYmAuhJzxQWFiI4OJjesEfCUGoz5shkMhQXFyMyMhIREREWUtS9PUwmT56MiRMn9nnhGI1GuotqbW2FUCikwcXX13dMdlHmqaPExESrtU5bExKEW1pa0N7eDqFQiNDQUEgkkhHNu/QnwT9UhiKlYitdLoFAAA9nD3Tprn9uASOAUWuEskuJDnUHnJ2daXDpT8jS3P89ODh4wCDk6OCIVUmrEC25PuU8FqjVauTn5yMoKAhRUVEjuhmTmRfymVOpVD1t5v/X4u7u7m6Vm3xDQwMqKiqQnJxMW+2vXLmCpUuX4uGHH8Zf//rXmyKYABwOKPX19ZDL5UhJScG1a9dw5coVxMTEICwsbFS/n/ihREVFDfg4lmVRW1uL6upqxMXFQSKRXDf5Th5TV1eHuLi4IQ9hMgxDPb5lMhkYhqHpHlLUtzbmAo+JiYlWlf2wNuS0R3xcRlJ36QsiwV8hq0ClrBId3R0DPp6oJA80F2MrXa6hPDeFBWAAulRdUHQqAAFoxxhJjZn7vwcHBw/43jkJnXB38t2IDBzapsvaqNVq5OXlISQkBFOmTLHazbj3cK6zszP9zPn6+o7oc9fY2Ijy8nKL2bfKykosXboU69atw9///ndOd6NZG84FFKPRCJPJhKamJtTX18PLywvNzc1ISkqySrfU5cuXIRAIqDBaXzAMg9LSUrS2tiI5OZlKTBOJeoFAAJPJhLKyMnR0dIxqt092USS4aLVa+Pn50dOLNY7oGo1mWAKP9qS1tRUXL15EZGQkJk2aRL/eV92FvE8BAQEjmuqVdclQIe0p6jcrmy0K7UIHIUSOon6HJW2py+UgcICr08gHNQXGnmHajs4OaPQ9qTHSGTVYMHF2dMavUn+FiX4TR7r8UaFSqZCfn4/Q0FCbNoqQmReyWdHr9RZyMEP5ezY3N+Py5ct0Bg3o6fBasmQJ7rrrLrz33ns3VTABOBxQGhsbUVZWBjc3t2EN2g1GRUUFDAYDpk+f3uf3yZCkyWRCUlISdXc0P5nodDoUFxcDABISEqzmYQL8kosn9QRvb2+IxWKIxeIRybWY7/bNdc24CGkUGEw0s7+6C2lJHknzQ5e2C1WyKlTIKtDQ3gAA/abHhlPUHi6ODo5wEjpZSJyMFJZlARPQJmujVsHOLv17vIicRPh16q8R6ju2TRoEEkwmTJgwpgrcLMtCpVLR00tnZyc8PDzo6aWvemdLSwvKysqQkJBA65D19fVYvHgxli1bhg8//JDTnzVbwcmA0tHRgby8PBiNRsyfP9+qO+rq6mqo1WrEx1/ftdLV1YWCggJ4e3tjxowZfRbfu7q6UFRURCd1bekIp9VqLeY43N3daXAZiuYREeMbqj+9vSD2x3V1dSNqFOhv3oUo2w73deuNetS01vRIwcgrodH/cnPvLZNvTV0uJ6ETHAQOQ5KfHwomkwnNzc0QCoWQSCQQCoQw6UxQq9Vo62qDQCCgwcXP2w/3pd2HIK+hSR1ZG9L4QlrY7Yler7eQgyEzL4GBgfDz86NW4sSBFejZDC1evBjz58/Hxx9/PCZOkVyEcwGlqakJBQUFCA4Ohkwmwx13jF4vyJy6ujq0t7cjKcmyp14mk+HixYsIDw/H5MmT+yy+y+VyXLp0CeHh4YiIiBjTG7TBYLAo6pMJ7P7kwPsSeOQiLMuivLwcUql0RG3gvTGZTGhra4NMJrNK3cVcgv9K8xVcrrk8pDrEcBmt/HxvyKS+s7Nz3/7v/+fxolFroFarcVvobYgMiaS7clt1Q/UFCSbEG4hLEGkh8tnTaDRgWRahoaHw8/NDUFAQWlpasHTpUqSlpWH79u03bTABOBhQmpubodfr4e7ujnPnzuHOO++06u+vr6+nyp/AL7vjqqoqzJgxA0FBQdT/2bz4Xl9fj+rqakybNm3IgpW2gnhYk105y7L0punn54fq6upBBR65ABkUJQZo1lZgJnUXuVxOTxUjrbsYDAYUFhaiU98JkViE6rZqq0nwuzq5wsgYYTAZRv27gJ61Njc305PaQBsfL5EX1s5aCxfWhQbhrq4ualUQEBBgtW6ovujs7ER+fj4mTZqEiIgImzyHtZDL5VSXT6vV4t5776UOlmFhYTh48CCnm13GAs4FFJPJBKPRCI1Gg9zcXCxePDx57MFoamrCtWvXkJaWBoZhUFZWBrlcjqSkJHh7e/fpYXLlyhXI5XIkJiYOajY01hAPdVLU12g0EAqFiIyMRGhoqFVTMtbEYDCgqKgILMuOeKZoOIym7kIGVomJE9mBqnXqnmFKWY8E/0gCgpuzG/RGvdUkZIikv7u7O/z9/Qf87Pi4+eC+WffBx83H4uu9u6FcXFwsUojWqg0QZ9OIiAiLBgwu0tbWhuLiYosTf3V1NTZt2gSpVAqlUgmj0YglS5bglVdeGbDp50aGc+0+vX3lGYax6hHSwcEBJpPJovienp5OZed7e5hcvHgRer0eaWlpnNx9CAQC+Pj4wNXVFQqFAp6envD390dLSwv1qCapMa54sPR3g7Yl5l7fERERFnWXmpqafusuarUaBQUF8PPzQ2xsrMXN1N3FHYlhiUgMS7SQ4K+QVUCtG7xDy93ZHRqD9eTnhyP74u/uj/vS7oOn6PoUo0gkwoQJEzBhwgSaQiSW1eaDggEBASPesJBgQup7XIZIFMXGxtJg0t7ejg0bNiA0NBTHjx+HUCjEhQsXcODAAas26Yw3OHdCITbADMPg6NGjmD9/vlX/QHK5HGVlZRAIBPD09MSMGTMs3CFJ8V2tVqOoqAju7u5DUjG2J/0JPGo0GnrTbG9vh4eHBy3q2zKNMdhaCwoKqC4TFzph+qu7eHh4oLa2FiEhIcMarmNZFk0dTVRnrC8Jfg8XD3Tru60WTLRaLfWHGawRQewpxr2z7oW7y/C64cwHBeVyOdRqNXx8fGggHmonZkdHBwoLC+kgMJdpb29HYWEhYmJiEBISAqAnTZeRkQE/Pz/s2bPHbhvNrVu3YuvWrairqwMATJ8+Ha+88gqWLl1ql/UAHA4oQI+74m233WbVgbGamhpUVFQgMjISU6ZM6bP4TnYkoaGhI57SHSuGKvBoMBjojaC1tRUuLi4WRf2xeI1krfZoahgqRHjw2rVrkEqlEAgEFjpjI9nctHe3U+vjhvYGeLh4WFV+nmiI+fr6DlozC/YOxtqZa+HqPPrTqkajsUiNubm50ZNLf9dUe3s7ioqKMGXKlFEPKduajo4OFBQUYOrUqVTvTqVSISsrC66urti/f79dT/3ff/89hEIhoqKiwLIsPvvsM2zevBmFhYX9jkXYGk4HlB9//BGzZs2yikQIKayXl5dDIBBg4cKF1xXfgR4ZhfLycruIJg6X5uZmlJWVDXutZEdOAoxAILAo6tsiBUXkaaKjozntDQP80m4dFRUFX1/fPusuI5Xu0Og1qJJXoVJWiWp5NfVxHynd3d2QSqVD0hCb4DsBv079NVycrJ+S6UtN2tzF09HRke72x8M1QFJy5oFPrVZj9erVEAgEOHDggFVldqyFn58fNm/ejAcffNAuz8+5gGJuA3zixAkkJCSMWgLe3B44JiYGpaWlmD9//nUyKpWVlWhqakJ8fDynPUyI5MvVq1dHLfBo7rgok8lgMBjojWA0OXJzSAvzcORp7AUZrpwxY8Z17dbWnncxMSbUtdXRustAemF9oVarIZVKERgYOOimK9w/HPek3ANnR9u3A5NGEfJedXd3w8PDAyqVCpMnT+Z8NxcJJuYpOY1Gg3vuuQdarRaHDh2yqgCoNTCZTPjuu++wYcMGFBYWYtq0aXZZB6cDyqlTpxAbG0uHh0aCXq9HUVERDAYDkpOTAfTYcIaHh0MikVBZlZKSEnR3d1t1Kt8W2FLgkUwMk44xtVoNX19fmhobbq6YZVlUVVWhsbGRdtFxmatXr6K6utpi+rk/ep/ygNHNuwA9EvwV0h6dMWmndMDHkr+TRCIZVBlgcuBk3J18NxyF9qkDNjU1oaysDK6urtBoNHB3d6fvlb1Ut/ujq6sLeXl5Fs0COp0Oa9euhUKhwNGjRznVil9SUoL09HRotVp4eHjgq6++wrJly+y2Hk4HlLNnzyIiImLEcx9EsdTDw4N2E5lMJrS2tqKlpQWtra1wdHQEwzAQiURITk4e04Gu4TLWAo8ajYYGF6VSCU9PT4ui/kCQlmxiRsYlL5jeEMO2hoaGEQU+UnchwYXosY2m7jKQBH9nZyfa2tqGJEg5VTIVq5JWQehgn2E70m5LitoGg8EiNWY+hT7SQGwtyIAlqfEBPRvS9evXo7GxEceOHeNc5kKv16O+vh5KpRI7d+7EJ598gtzcXP6EYg7xlf/5558RGho6olpGW1sbioqKMGHCBERFRVkU30knFykSOzs7w2Aw0FoCUbnlQgcSwd4Cj3q93sLbRSQS0ZNLb1l5Yoql1+upHhpXIWZucrkcycnJo86LDzTvMtK6i86go3WX/Mp8NMuaERQUNGhBeHrIdGTEZ9jtOiZCn7GxsX1qs5mnW80FP0m6dSy7p1QqFfLy8iym9Q0GA37zm9+gqqoKx48fH1WmZKxYuHAhJk+ejI8//tguz8/JgEJcG/Pz8xEQEDDsPnVSfI+NjUVoaGifxXci7kaKbmSXSXbkJpMJAQEBdKranjsnrgk8klMeuRE4ODhYuFIWFxfDyckJCQkJnG63ZhgGly5dQldXl00m9QHr1l1qa2tRU1sDcYQYzZqe9JhS07fpV2JYIpbNWGa3dBJpbBiqsgTLsuju7qbvlVKppAKNpEZkq9dC5PJDQ0MxZcoUAD2boocffhglJSXIycnhtHyROXfccQcmTpyIHTt22OX5OR1QhmuGRabam5ubqdlNXx4mxLEwLi6uz11HX5Ly5FgeGBg4ptPnXBd4JLtMmUwGqVQKvV4PkUiEKVOmIDAwkLMBxWQyobi4GHq9fsxSnSOtu5in5My9ygFA2iml1sdEgj8lPAWLp1lXYWI4yOVyXLx4sc/GhqFiLtBIUtPk5GLNTkTiChkcHEy9V0wmEx555BGcP38eJ06coPMnXOOFF17A0qVLMXHiRHR1deGrr77CW2+9hSNHjlhdsmqocDqgDNUMC/hFykOn0yE5ORkikeg62XmTyYTS0lIolUokJSUNKb1BUhgkuKhUKgu/Elumc8aLwCPwS2cMKd7LZDJ0d3eP2Xs1HIgul4ODAxITE+0S9IZadyHimTKZDCkpKQPWorq0XWhWNtvNZRHoEVktKSkZVTDpDTGkI+8V8S4hAWak15VGo0FeXh7EYjG1GGYYBk888QRyc3ORk5PD6cHLBx98ED/++COam5vh7e2N+Ph4PPfcc3YLJgDHA8qVK1fAsixiYwf2tCbyGG5uboiPj+9z8l2n06GoqAgODg5ISEgY8Y60d6F6tH4lfcGyLCoqKsaFwCPwyylqypQpFh9A81qCUqmkXvEkNWYP7CH7Mhgk3WPug0PqLkqlEl1dXUhNTeWMdE5/SKVSXLp0CXFxcRCLxTZ5jr68S8h7FRAQMCRbB+CXYBIYGIipU6fSYPL000/j8OHDyMnJ4Xx7MxfhZEAh0iuVlZXQ6XSYMWNGv48lxfeQkBBMnToVLMv262Hi6+uLadOmWa0GQfLjMpkMCoXCKtIm5gq8XG9hBnosUK9cuYLp06cPmCs3ryW0tbXBzc2NBpexah0dSJeLS5D3qqamBjqdDiKRCBKJZExVDYYLCSbx8fFjOmuk0+locGlra6O2voGBgfD19e3zb6zVapGXlwd/f3/ExMTQYPLCCy9gz549yMnJobUUnuHB6YBSW1sLpVKJxMTEPh9n7jVPhOwYhoFAIKAXkkwmw6VLl6iiqa0+jMSvhOhBkS4osVg85BumtU5RY4H5cGVCQsKw2inJVDV5r4RCIQ0u/d0ERktnZycKCwsRHBzMeTkdhmFw8eJFaDQaJCYmoqury6LuQppF7N1mSyANLvYeXCW2DiTAGI1Gi9SYs7MzdDod8vLyqEEeCSavvvoqvvzyS5w4cQJTp06122sY73A6oNTX10MulyMlJcXi+yQd1tTURP2c+yq+X716FTU1NWNegzAXG5TL5fSGKRaL+5X/VqvVKCwshLe3t82dIEeLeattUlLSqIYrSX6cpBEZhqE7TGt11xH9qPEgk06aBcggrnkDiC3mXUYL8VWPj4/nVFsty7IWgVilUsHT0xMajQY+Pj6Ij4+Hg4MDWJbFG2+8gU8++QTHjx+3mwbWjQInA4q5r3xDQwPS0tIsvldUVASNRoPk5GS4ubn16WFy+fJltLW1ITEx0a4yCQzDQKFQ0OBCzLDIDtPBwWHIAo9coLeqgDXz+n1115kX9UdyYiP1nfGgH2U0GlFYWAgASEpKGrRZgDSLWGveZbg0NTXhypUrQ1IWsDddXV30vTUYDPjss88gEong7u6OnTt34vjx40hISLDzKsc/nA4oUqkU1dXVmD17NoCeFr+CggK4uLjQGYfexXe9Xo+LFy/CaDSOyTT5cOg962IwGODh4YHOzk5ER0dzuqME+KU7SiAQIDEx0ebt071vmMNtgBhIl4trGAwGFBQU0Pmd4Z7MSC2B1PPM1aS9vb2tnkZsbGxEeXk5zRBwGYPBgLy8PGpFwbIssrOz8emnn+Lnn3+Gs7MzVqxYgYyMDCxZsoTzTTBchptDAv+HebeWQqGgOXCS4+zLw6SwsBCenp5ISkriXNpIIBDA19cXvr6+iIqKQnl5ORobG+Hi4oLKykoqpzHWnt5DgUzqkw/lWLy37u7uiIiIQEREBLRaLU1fVFZWwt3dnQaXvjp7iC5XYmIi53fPOp3OovNsJDd/FxcXqiphPu9SXFwMwLp1l4aGBlRUVIybYJKfnw9XV1fMmDGDprnIFH9OTg4cHBzw/fff480330RnZyd++9vf2nvZ4xZOnlCIDTAx4omKisLly5cxdepUhIWFwWQyXedh0tbWhosXL2LChAl0QImrmKfkSA3CfNalq6trVKKM1qarqwsFBQUQi8W0K8aekAYIogfl5ORksRuvra0dsS7XWKPVapGfnw8vLy8LczRrYW4RbY26y7Vr11BZWYmkpKRRq4DbGnLqc3Z2RkJCAg0m27Ztw8svv4wDBw5gzpw5Fj/DsuyYXd9vvvkmdu3ahStXrsDV1RWzZ8/GW2+9Na6bAjgdUDo7O3H27Fk4Ojr2W3wHei7yiooKxMbGcnaqlTAUgUfitCiTydDR0UHnN8Ri8Zi3EZP6zqRJk2zaJTdSSGcPOb0YjT3e7NHR0QgJCeHcKdWc7u5u5Ofnw9/fH7GxsWPWOk2ureHWXa5du4aqqiokJSVxPi1kNBpRUFBA7x0kmHz++ed45pln8P3332PevHl2XeOSJUvw61//GjNnzoTRaMSLL76IS5cuoaysjNNiqgPB2YBCBtAUCgXmzJkDd3f3PovvFRUVaGlpsYpviq0ZicAjEWWUyWRoa2sbNNVjTVpaWlBaWjouAjXDMCgpKYFSqURAQAAUCgV0Op3dJHMGg1ghSyQSOqU91vSe4XBxcaENI73rLvX19aiurh4XwcRkMqGgoIAqIQiFQrAsi6+//hpPPfUU9uzZg4ULF9p7mdchl8shFouRm5uLuXPn2ns5I4KTAUWlUuH8+fNwcnJCe3s7FixYQAMICSZE0Var1SIxMZHzA4BkDmI0Ao+9Uz3Ozs40uPRW/B0tpAbBtXbQviDXgsFgQFJSEpydnfuUzPHx8aGpMXtOnROZ9AkTJnCmq68vnTESjLu7u1FXV4fk5GTOpxBNJpNFpxw5oe7cuROPPPIIvv32W7v6hQxEVVUVoqKiqHTNeISTAUUul6OhoQFTpkzB8ePHMXv2bLi6utLiO9npi0QixMXFcWrn2Re2EHjsPetCFH/FYvGohgPNnSvHQw2CGKgJhcIB1Y1JGlEul6O9vd0qqgYjgdQFJ02axFlpD/O6S1NTEwwGA7y9vRESEsIpTbbemEwmFBUVgWEYi7brvXv34qGHHsLXX3+NjIwMO6+ybxiGQUZGBjo6OnD69Gl7L2fEcDKgMAwDvV5P86AdHR3w9/eHWCyGi4sLLl26hKCgIERHR3NWPoNAipi2HK7sbzhwuF09DMNQ8Uwy48NlSFp0uJ1nBoOBBpfW1laLFltbSpsoFAoUFRUhKiqK+pRzGaKEEBsbSzXs7DHvMhQYhkFRURGMRiOSk5NpMDlw4AA2btyIzz77DHfffbedV9k/mzZtwqFDh3D69GnOz0sNBCcDilQqhUgkgoODAxwcHKDRaCCVStHY2AiNRgM3NzeEh4dzerdkL4FH892lTCaDXq+38HXpbwdvNBpRXFwMo9FI00Zcxlq6XL1TPcRkLTAw0Koy6eSUSpwLuQ6xeOgtlz+custYwTCMhRUByVj88MMPuO+++/A///M/WLt27Ziva6g89thj2Lt3L06ePMnZU+tQ4WRAuffee3Hs2DEsX74cq1atwq233ooXXngB/v7+ePDBB2EwGCCVStHZ2Unz4mKx2O7ttQSuCDz25RFPTnrmk+dkDsLFxQXx8fGc9TAhdHZ2oqCggBoiWWuHzDCMRTA2GAy0jhAQEDDi1CoRThxMQJMrVFdX49q1a9cFk94MVHfx9/cfk+uINGNoNBqkpKTQv9GJEydwzz334KOPPsL69es5cYrqDcuyePzxx7F7926cOHFiSDYdXIeTAcVoNOLEiRPYuXMn9uzZA6PRCKFQiBdffBHr16+ngYMMu0mlUov2WolEYreiq16vp14bXBN4JBLpJHXh4+MDHx8fNDU1wc/Pz6pKzLaCtDHbWperr2A8ktkgIk9ib+HEoUCMvBobG5GSkjIsO2RyMiYdiWOhM0YcN9VqNVJSUuhn7fTp01i9ejXee+89PPjgg5wMJgDwyCOP4KuvvsLevXstZk+8vb05b1XQH5wMKISmpiZkZGTAYDDglltuwcGDB9HZ2YmlS5ciKysLCxcupLt/vV5PP/xESl4ikdCi61hAJvXJkBqXZyC0Wi2uXr2Ka9eugWVZi1kXrvbAE+XoqVOnIjQ0dEyfm9QQ5HI5Ojo64OnpSYNLfzdeUj8bD1pXowkmfdHbC8f8/bJG3YUY8BGvGBJMzp07h1WrVuGNN97AI488wtlgAqDftW3fvh0bN24c28VYCc4GFJZlkZycjMTERPz73/+Gi4sLGIbBuXPnkJ2djd27d0Mmk2Hx4sXIysrC4sWL6YeAFF2lUqnF7IZEIrFZEXE8CTwCv9yco6KiIJFI0NraCqlUCoVCAVdXVxpcbOnlPRzITn/GjBk2M28aKmQ2iNQRiFUBmdQXCASoq6tDbW3tuJjbYFkWVVVVaGpqQmpqqtU3FL3fL1J3IU0Qwz0VsyyLsrIydHR0IDU1lZ5+8vPzsXLlSrz66qt48sknOXHd3mxwNqAAPTu8CRMm9HlhMAyDgoIC7Ny5E7t27UJDQwMWLlyIzMxMLFu2jHqQGI1GegxvbW21yc2yubkZZWVlmDp16rjo0CBaTH11nhmNRgtfFyJrQqT37fEhJTYEw/VdGQv6at92cXFBd3c3kpOTx0UwqaysREtLy6AWw9ZgtHUXlmVx+fJlKBQKpKam0tRjcXExli9fjueffx7PPPMMH0zsBKcDylAhhTkSXKqqqrBgwQJkZGRgxYoV8PX1pZ7yZCduPhgokUhG5BpobjIVFxfH+QFAlmVp905iYuKgygJE1oTcLEkHlFgshp+fn83rLSQNM150uUgzRmtrKxwdHcEwDG2CGKjDzl6QTkSpVIrU1NQxbx4Zbt2F+PC0tbVZBJNLly5h2bJleOqpp/CnP/2JDyZ25IYIKOaQHczOnTuxe/dulJaWYu7cucjKysLKlSsREBBAg4v5ztLR0XFYO/G+BB65DMMwuHLlClpbW5GcnDzsHDnDMBbS+yaTaUSzLkOF/B3b2tqQnJzM2boOgdzsyPvr5uaGrq4uen2p1WoLbxd7t7uzLIvy8nJqYMeFmaOB6i5ubm6orKyETCZDamoqLVpfvnwZy5Ytw29/+1v85S9/4YOJnbnhAoo5JDdMgkthYSFmz56NrKwsZGRkICgoiEq6KBQKSKVSuhMnJ5e+crxDEXjkEiaTicrUJCUljXq9fRlhmc+6jFa5gHTvqFQqJCcnc/79ZRiG5vRTUlL67NDpfbMkTRCkSD2WmAc/85szl+hddyHijrGxsfQaq6ysxJIlS3D//ffjzTff5HyH4s3ADR1QzCGWwNnZ2di1axfOnz+PtLQ0ZGZmIjMzk9Zqek+dsyxLTy5+fn7Q6XRU9mU8zGwQaRLSxmxtmZq+NLNGsxMfbwOWJN1KWleH8np1Op3FzdLNzY2+XyNJvQ4H8xpEf8GPS5C0HGltr6qqwmOPPYZZs2ahvLwcK1aswJYtW/hgwhFumoBiDsuyaGxsxK5du5CdnY0zZ84gKSkJWVlZyMzMpDLt5g6LUqkURqMRLMvC19d3XGiIaTQaFBQUwNPTc8zamEl7rUwmg1KpHJbLIpnhcXR0HFCXiyuQk59Op0NycvKIgp/RaKSp19bWVgiFQhpcRqPJ1hekO6q9vd2iBsFlSA2NdJ8ZjUZkZ2djy5YtqK2tRXd3N+644w5kZmbivvvuG3W7M8/ouCkDijksy0IqlWL37t3Izs5Gbm4uZsyYgczMTGRlZSEqKgoCgQD79u2Dk5MTAgICoNPpqKQJcVjk2swJUTeWSCSYOnWq3eTRScGVzAb1J8horss1UtfCscRoNFoIEVpjc9GfJhuZ1B/NNWbeapuSkjIugglpIElNTaWBoqmpCYsWLcKCBQvw8ccfo6qqCnv37sXBgwexb98+ztcyb3Ru+oBiDsuyaGtrw969e7Fz504cP34c0dHRiIqKwqFDh7Bjxw6sXLkSLMvSgqtUKrWoIQQGBtp9Z03cKyMiIqymbjxayGwQ8XUhsxtisRhCoRCFhYVjajQ1GgwGAwoLCwdVOB4N5nUquVwOjUZjkUoczmmIZVkq+jlegkldXR3q6uos5F9aWlqwZMkSpKen49NPP7X7Ju7kyZPYvHkz8vPz0dzcjN27dyMrK8uua7I3fEDpB5ZloVAosG7dOuTk5CA0NBSOjo7IzMzEqlWr6C6a1BCkUqmFXpZEIrGLqROZiZk2bRqCg4PH9LmHCmnfJjdLk8kET09PREdH0xZvrqLX6y10z8bqpkbqVHK5HJ2dnTSVSDqg+oMoSHd1dQ25xmNvyNxRSkoKvLy8APQM4i5btgwJCQn4/PPP7b5pA4BDhw7hp59+QkpKCu666y4+oIAPKP2i1Wqxbt06XLx4EQcPHoRYLMb+/fuxa9cuHD58GGKxmKbFUlJSaIqGfPClUqlFgVosFtu0wEyaDsgAINelPoBf5NwlEgkEAgHkcjlYlrVoR+ZS6ouk5Tw8PDBjxgy7rY1o2MnlcigUin5dPM275cZLMCE2w+ZmXm1tbVi+fDmioqLwv//7v5ysXQoEAj6ggA8o/WIymfCXv/wFTzzxxHU3Z7VajUOHDiE7OxsHDhyAr68vMjIykJmZibS0NLprJbL75mKM5ORizbQD6YRpaWlBUlIS3dVxmb50ucybIIjar3lwseeuVKPRID8/H76+vpg2bRpnTlG9XTyJskFAQACuXbtGVXi53i0H9Cg4VFZWWsjVtLe3Y+XKlZgwYQJ27tzJ2dfBB5Qe+IAySjQaDY4ePYrs7Gzs378fIpEIGRkZyMrKwuzZs+lNUKvV0pPLcLufBsJkMtGUhj2l8odDU1MTLl++jLi4uH51uczrVDKZDBqNxkJ6fyx3qWq1Gvn5+dS+mSvBpDfmygbNzc1gWRYSiQRBQUE2GT61Jo2NjSgvL0dSUhJVcFAqlcjMzIS/vz/27NnD6RMWH1B64AOKFdHr9Th27Biys7Oxd+9eODg4YMWKFVi1ahXmzp1Lb4I6nY7eKNvb2+Hp6UmVkYcTEAwGA4qLi2EymcbFzAYwcl0ulUpFi/pdXV0jkpIfCV1dXSgoKEBISIhVvVdsBcMwuHjxIjQaDaKiomiA0el0FjIwXLpWiPBnYmIivSa6urqwatUquLm54fvvv+f8vAwfUHrgA4qNMBgMyM3NpZ4uBoMBy5cvR1ZWFubPn093W2QimCj9uru70+AyUE+9Vqu1GLDk8u4T+EW1oLGxEcnJyaNKyxF/eHLaM5fet+YJTalUorCwEBMnTkRERATngwmZi+ntXNjX8CkxpgsMDLTrzbqlpQVlZWUWdT+1Wo3Vq1dDIBDg4MGDnJfdAfiAQuADyhhgMplw6tQpKruvUqksPF3IB5rkw4nsPlFGlkgkFsVWlUqFwsLCUdvfjhW21OXq7YPTX4F6uLS3t6OoqAiRkZEIDw+32npthclkQnFxMQwGg0Uw6QuSfpXL5Whvb6fzQcTbZawCJ3GyTEhIoMKqGo0Ga9asgV6vx6FDh8bNXAkfUHrgA8oYYzKZLDxdWltbsXjxYmRmZlp4upjLyMvlcri4uFAnyoqKCoSHhyMyMpLzu2ZzaRJb63KRgEymzl1cXGhwIT4lQ6GtrQ3FxcWIjo4eF3YEJJgQuZrh1JfIfBAp6pP3jHiV2Or6kslkKCkpQXx8PHWy1Gq1WLt2LTo6OnD06FHOq0urVCpUVVUBAJKSkvDuu+9i/vz58PPzw8SJE+28OvvABxQ7wjAM8vPzqXhlQ0MD7rzzTmRmZmLp0qX0A0WUka9evYqOjg44OTkhODgYEolkWDfKscaeuly91aSJpAlRk+7vVEdudFye4zHHZDJZTOyPphOut1cJsSsIDAyEn5+f1dKqcrkcFy9etGjK0Ov1WLduHZqbm/HDDz9wzvemL06cOIH58+df9/UNGzZgx44dY78gDsAHFI5AiqnE06WmpoZqFK1YsQL/+c9/UFpairfffhtCoZCmecxvlFwaCuSSLldfgp/mvi7kRtnS0oLS0tIBu8+4hMlkQmFhIViWHXUw6Q3DMFAqlRYt3OZF/ZF22bW2tqK4uBgzZsyg5m4GgwEbN25ETU0NfvzxR877CvH0Dx9QOAjRXTI3DBMIBHjsscewadMm6ulCbpRk1gUArblYW1hwOHBZl4uYOpEbJdFkc3R0RHNzs0U+n8sYjUYUFhZCIBAgKSnJpk0ZLMtCpVLR055KpRpRlx1JJcbGxtLTn9FoxMMPP4xLly4hJydnXARynv7hAwqHMRgM+O1vf4ujR4/innvuwenTp1FUVIRbb72VerqQKXOWZS124SaTiZ5cxnLiXK1Wo6CgYFzocpEbZWVlJdra2iAQCCxmXbjUWmsOCSYODg5ITEwc8w4/oigtl8vR0dFhYYTVX2ciUUWIiYlBSEgIgJ4T1qZNm3DhwgWcOHFiXKQYeQaGDygc5tlnn8XRo0dx8OBBhISEgGVZ1NXVUU+Xn3/+Gbfccgv1dAkNDaXBRalU0pOL0Wikysi2HHDr7OxEQUEBQkNDx8XMBgDU1tairq4OycnJcHR0pLMuRNmABGWuCCoajUYUFBRAKBTaJZj0prcRFhH9DAwMpPW9jo4OFBQUWKgiMAyDxx9/HKdOnUJOTg7CwsLs+jp4rAMfUDgMkdLoq9uFZVk0NDRg165d2LVrFxWpI8HF3NOFqNZKpVLodDpaP7Cmz7lCoUBxcTEiIiIwadIkq/xOW2LuV2+uaEvo3VpLduFEet8eEJVjUpeydzDpTe9GCAcHB3h7e6OtrQ3R0dE0aDAMgz/+8Y84evQocnJyxsX1wjM0OBNQMjIyUFRUBJlMBl9fXyxcuBBvvfUWPR7z9A/LsmhpaaGeLidPnkRcXBwNLsTTxTwXLpVKrSZn0pcuF5chfuoymQwpKSmDBgiyCyfS+8RhUSwWw9PTc0xOYgaDAQUFBXB2dh4Xg6wMw1A5FaFQCJZlsX37dtx+++24fPky9u/fjxMnTmDy5Mn2XiqPFeFMQHnvvfeQnp6O4OBgNDY24umnnwYAnDlzxs4rG1+wLIvW1lYLT5epU6dSZWTzugYJLubWvUS8cqj1A3LTmDFjxrgoqJq7FiYnJw97st58PshcjNGWLdwGgwH5+flwcXFBQkICp5oc+qOzsxP5+fmIjIzExIkTIZfL8cYbb+DQoUNobGzEnDlzcN999yEzMxNBQUH2Xi6PleBMQOnNvn37kJWVBZ1Ox0m56vEAKdTv27cPu3btwtGjRzFp0iTq6WIuwd7d3U1PLuZaWWKxuF9Rvrq6OtTW1g5bl8teEG+Qzs5OqxhNmYsxkrkN8xZua9z4if8KkdgZD8Gkq6sL+fn5CA8PR0REBICea/H111/Htm3bsH37dpSVlWHPnj04d+4cjh8/jrlz59p51TzWgJMBRaFQYNOmTWhsbMTp06ftvZwbBqVSaeHpEhQURE8uycnJ9GbVly880RcTiURW1eUaK0wmE0pKSmwm584wjIX0vslkspDeH0mKSq/XIz8/H25ubpxrv+4PlUqFvLw8TJw4EZGRkQB6gsnmzZuxZcsWHD9+HPHx8fTxUqkUPj4+dlES/vDDD7F582a0tLQgISEBH3zwAWbNmjXm67iR4FRAee6557BlyxZ0d3fjlltuwf79+8eFUdR4RKVSUU+XgwcPUk+XrKwszJo1i94AiTKyVCpFR0cHvLy8wLIstFotUlNTx4VwH5kmNxqNg+pcWYO+GiGIRfRQhwJJMHF3d7ermddwUKvVyMvLw4QJE2hthGVZ/POf/8Q777yDH374ASkpKXZeZQ/ffPMN7r//fvz73/9GWloa3n//fXz33XcoLy8fF6lbrmLTgPL888/jrbfeGvAxly9fRkxMDICeriaFQoGrV6/itddeg7e3N/bv3z8u2k/HM93d3RaeLm5ubli5cuV1ni6dnZ1UzZZhGHh4eNCTC1cDC5nZAGD1afKh0JdFtLk3fF87c51Oh/z8fLs7Qw6H7u5u5OXlITg4mLaMsyyLjz76CG+88QYOHz6MtLQ0ey+TkpaWhpkzZ2LLli0Aek6YYWFhePzxx/H888/b7HkZhrH4e7Ise0Pd32waUEhv+kBERkb2mX5oaGhAWFgYzpw5g/T0dFstkacXWq0WP/74I7Kzs7Fv3z4IhUKsWLECixYtwttvv43o6Ghs3bqVWvYS2X3S+SSRSODu7s6JDwnpjHJycuJMmy2pVZFZl95GaySYeHp6Yvr06eMimGg0GuTl5UEikVh0FG7btg0vv/wyDh48iFtvvdXey6To9Xq4ublh586dFurAGzZsQEdHB/bu3WuT5zUajXB0dIRKpUJJSQlmzpxpV0kiW2DTV0OE5UYCwzAAenZrPGOHSCTC8uXLsXz5chgMBpw4cQJffPEFNm7cCIlEAicnJ+Tk5GDevHkICQlBSEgIjEYjbautq6uDSCSiJ5exaqvtjU6nQ0FBAefqD25ubpg0aRImTZpkYbRWWVkJd3d36HQ6eHt7Y8aMGZwIyoNBgklgYKBFMPnvf/+Ll156Cfv27eNUMAF6MiEmk4lqiREkEgmuXLlik+c0mUxwdHSEQqHAggULMHfuXIhEIiQlJd1QpxROhMfz58/jwoULmDNnDnx9fVFdXY2XX34ZkydP5k8ndsTJyQmxsbG4cOECli9fjt/+9rfYt28fHn/8cahUKixbtgxZWVlYsGABgoODERwcDJPJRNtq8/Ly4OzsPCIJ+dGg1WqRn58PLy8vTu/yXVxcEBYWhrCwMNoZJRQKoVAocPbsWVrU9/Ly4uQNh7zPAQEB1BqZZVl8/fXXeOaZZ7B3717MmzfP3svkBEKhEGq1Gunp6UhKSsJzzz1H26XJ37Z3Omw8womifElJCZ588kkUFxdDrVYjODgYS5YswUsvvTQuBuVuZB599FEYjUZ89NFHNGVkMplw9uxZ6unS1taGJUuWUE8XUk8hbbVSqdRCQl4ikdjMa6O7uxv5+fnjQkuMoNFokJ+fTw3TGIaxmHVxdHS0kN7nwmsiwcTX19fifd65cyceeeQRfPfdd1i6dKmdV9k3Y5Hyqqurg6enp0VT0d/+9jccOXIEp06dAtBjfXz8+HE0NDRgw4YNCA4OHvdBhRMBxd7U1dXhr3/9K44fP46WlhaEhIRg3bp1+NOf/sRZgcCxQq/Xw8nJqd+bGMMwyMvLo54uTU1NFp4upKWYYRg6syGTyWwys6FSqVBQUACJRILo6GhO3HgHg6SM+guAvd83APR98/Pzs8vNh9R5yAmQrHnv3r146KGH8PXXXyMjI2PM1zUc0tLSMGvWLHzwwQcAet7niRMn4rHHHht1UV6v1yMmJgYvvPACHn74Yfr1999/H9988w2+/PJL7N27F2fPnsXx48cRHh4OhUJBNxXjGT6gADh8+DC++eYbrF27FlOmTMGlS5fw8MMPY/369XjnnXfsvbxxA8MwKC4upsGlpqYGCxYsQGZmJpYvX05312Rmg3Q+EX8SiUQy4pskSRmRltXxFEwCAgIQExMz6JpZlrWYdSGin2TWZSwKvHq9Hnl5efD09LSo8xw4cAAbN27Ef//7X6xevdrm6xgt33zzDTZs2ICPP/4Ys2bNwvvvv49vv/0WV65cua62MhJaWloQFBREh4v9/Pywa9cuvPnmm2hqaoK3tzd+//vfY9myZbh27RqefPJJ7Nu3b9zrmvEBpR82b96MrVu3oqamxt5LGZewLIvS0lIaXC5fvox58+YhKysLK1asgL+/P825975JDncgsKOjA4WFhZg0aRKdzOY6JDUXGBhI6w/DgWVZdHV10VkXrVZrFV22gehvNubo0aNYt24dPvnkE/z617+2+vPaii1bttDBxsTERPzrX/+yWmszKbT/6le/wsWLF5GbmwuxWIz8/Hw0Nzdj7ty5cHd3h1AoxO7du/Hyyy9j7969417bjA8o/fDSSy/h8OHDyMvLs/dSxj0sy6KiooLK7hcXF2POnDnIysrCypUrLTxdzAcCifmVRCJBQEBAn8GF+GxMmTJl3Ph4q9Vq5OfnWzU111uXbSjSOcOB6Im5urpadM3l5OTgV7/6FT766COsX79+XJwMbYnJZLK4TsvLy5GZmQkfHx/s2rXLQuxWKpXi3Llz2LBhA/785z/jD3/4gz2WbFX4gNIHVVVVSElJwTvvvGORA+UZPSzLora2lgaXCxcuID09HZmZmcjIyLDwdFGpVDQtptForps2l8vlKCkpGTcqx8AvwSQoKIi22Vqb3tI5Xl5eNLgMVwwTsFQ6NhenPHXqFO6++268//77eOCBB276YGLOs88+i02bNiEiIgL19fVYvHgxPDw8sHPnToSHh6O5uRlvvPEGTp48iXvvvRfPPfecvZdsFW7ogDLcSX2gRz339ttvx7x58/DJJ5/Yeok3NSzL4tq1a9TT5cyZM0hNTaWy++Hh4RbKyObT5h4eHlCpVIiJicGECRPs/EqGhkqlQn5+PkJCQsbMgEyn09EZIYVCAXd392ENoBJDL0dHRyQmJtJgcvbsWaxatQp///vfsWnTJj6YmPHTTz9hwYIFuHbtGp3Da2xsxJIlS+Dk5IRdu3Zh0qRJOHfuHHQ6HW6//XY7r9h63NABZbiT+k1NTZg3bx5uueUW7NixY1y37403WJZFc3Mzdu/ejV27duHkyZOIj4+nwcX8BnzhwgUolUqIRCJotVr4+vrSQUquduWRYBIaGmq3pgGDwWDRjuzi4kLft75mXUwmEwoKCq6zGs7Ly0NGRgZee+01PPHEEzd9MOmd5tJoNEhMTMR//vMf3H777fT7LS0tWLlyJbq6urBv3z5ER0fbcdW24YYOKMOhsbER8+fPR0pKCr744gtOyHTcrBBPFxJcjh8/jpiYGGRmZqK7uxv/8z//g7NnzyIiIoKmd6RSKWdte0kHWlhYGCIjIzlxA+7trkhmhMisC8uyFhpo5PNQVFSE5cuX48UXX8TTTz/NidfCFa5evYrw8HAwDINp06bhgQcewLPPPmvxGLlcjlmzZuGPf/wjHnvsMTut1HbwAQU9wWTevHkIDw/HZ599ZhFMePMf+0LaLvfu3Yt33nkHlZWVSEpKwvz585GVlWXRbURse2UyGVVGJjtwV1dXu6yfBBNzOXeuwTAM2tvb6XtHOpScnJyQmppKT32XLl3CsmXL8Ic//AEvvvgiH0zM+NOf/oR3330X06ZNg6urK1iWRUBAAB566CFqMU1spnU6nV3k+scCPqAA2LFjB37zm9/0+T3+7bE/LMvir3/9K/71r38hOzub1l2OHDmC4OBg6umSlJREgwupHUilUrt5wnd2dqKgoMDCaIrrmEwm5OXlQavVwsHBASdPnsT58+cxZ84cbNmyBY888ghee+01Ppj04uzZs3ByckJtbS0uXLiACxcuIDc3F5MnT4ZOp4NOp0NwcDD+9a9/3dBmYnxA4eE89fX1uPPOO5GdnY0ZM2bQr6tUKhw8eJB6uvj7+1NPl5kzZ9KTZm9PePPCtIeHh03WTCxwIyIixs2wGhlM1ev1SE5OhqOjI0pKSvDRRx/h8OHDUCgUWLx4MVavXo2MjAwEBATYe8l2YSjyKNu3b8cbb7yBn376CQ0NDbh06RIMBgMefPDBMVqlfeADCkd4/fXXceDAARQVFcHZ2RkdHR32XhKnINLf/dHd3Y0jR44gOzsbBw4cgLu7O/V0SU9Ppz/buzDt6upqEVyssfNWKpUoKCgYd8HE3NGSDEbW1NRgyZIlWLNmDX73u99h79692LVrF5qamlBfX3/TnVTMC/CHDx+GVquFQCBAZmYmgJ7ry8nJCRUVFVi8eDFyc3PHzXyUNeADCkf485//DB8fHzQ0NGDbtm18QBkFWq0Wx44do54ujo6OWLlyJVatWoU5c+bQm6XRaERbWxukUilaW1vh7Ow8YNfTUCBT+5GRkQgPD7f2S7MJDMPg0qVLUKvVFvbIV69exZIlS7BixQp88MEHFrvyrq4uWhOwN2O1GTOXmX/sscdw9OhRCAQCODg4ID4+Ht988w19bFtbG6KiovDRRx+NK/WA0cIJ+Xoe4LXXXgPQU8/hGR0ikQgrVqzAihUrYDAYkJOTg507d+KBBx6AyWTCihUrkJmZiXnz5kEikUAikVh0PZG5C3JyGarsPgkmkydPHje7UiKRo1KpLArwjY2NWL58ORYvXnxdMAHAmWAC9KQ016xZg/T0dGzbts1mz0OugVdffRV79+7FoUOHMGPGDLz44ov4+9//jvb2dhw4cABOTk7w9/fHhAkToFKpbLYeLsKfUDjGjh078NRTT/EnFBtgNBpx6tQp7Ny5E3v27IFarcby5cuRmZmJBQsW0E4whmEsWmqJMjKR3e8rf97e3o7CwkJERUUhLCxsrF/aiCDBpLOzEykpKbTzqKWlBUuWLMHs2bOxbdu2cdNCPxafnaqqKjz77LP4/e9/j0WLFuGrr77CI488gmeeeQZbt25FXFwcdu/eDZFIhAMHDmD58uU2WwsX4U8oPDcNjo6OmD9/PubPn49//etfOHPmDLKzs/HMM8+gvb2derosWrSIuo2at9SWlJSAZdnr5ONJMImOjh43U/ssy+Ly5ctQKpVITU2lwUQmk2H58uWYOXMmPvnkk3ETTMaKyMhIZGZmIjU1Fbm5uXjuuefw4Ycf4r777oNSqcQ777yD+Ph4XLp06aYLJgDAj4LbkOeffx4CgWDAf7ayHOUZGKFQiNtuuw3vv/8+amtr8cMPPyA8PBx//vOfMWnSJNx333349ttvoVarqVfJ3LlzqZZVWVkZcnNzUVBQgIKCAkRFRY2rYHLlyhUoFAqLk0lraytWrlyJ6dOnY8eOHTec3/lQIUmb3iobLMvCwcEBGzZsgJ+fH06cOIHbbrsNd911FwAgLCwMv/vd75CVlcVZxQZbwwcUG/LHP/4Rly9fHvAfV4fdbiYcHByQlpaGzZs3o6KiAqdOnUJsbCzeeustTJo0Cffccw++/PJLKJVK+Pj4ICYmBrfddhuCg4OhUCggFApRWVmJkpISSKVSmEwme7+kfmFZFuXl5WhtbUVKSgpVE2hvb0dmZiYiIyPx1Vdf2UT+fjjYczMmEAhQUlKCpKQktLS0WHzdnKqqKpSVlcHV1RVKpRL79+9HTEwM3n77bZusazzA11A4Bl9D4Q4sy+LSpUvU0+XKlSt0Ql8oFOK5557DkSNHMH36dHR1dVHxSq1WayG7z5WdPsuyqKysREtLC1JTU6nysFKpREZGBgIDA7F7925OTHEPV4cPsO5n58yZM9TLxNfXt8/HnDt3DhkZGXB3d4dIJIK7u/tNb3fBjSudB/X19VAoFKivr4fJZEJRUREAYMqUKTYbvuMZGIFAgLi4OMTFxeHVV19FeXk5srOz8Y9//ANXr17F7bffjrNnz1JDMC8vL0yZMoV6k9TU1KC0tBR+fn6QSCQ2M74aCizLoqqqCs3NzZg5cyYNJl1dXVi9ejV8fHyQnZ3NiWACgNaw7EVqaiocHBzw448/4u677+7zMcnJyThy5Ah2794NX1/fG8LPZLTwAYUjvPLKK/jss8/o/5OSkgD0GBjNmzfPTqviIQgEAsTExCAxMRHNzc34xz/+Aa1Wi6+//hp//OMfMXv2bGRkZCAzMxMhISHw9PTE5MmToVarIZPJUF9fj7KyMvj5+dGi/ljm2WtqatDU1GRxMlGr1VizZg2cnZ2xZ88eu+mdjZbRbsZ6T74zDAOWZREcHIza2tp+f87Z2RlJSUn0s8rDp7x4eIZMR0cHoqKisHXrVrprZVkW9fX11NPl7NmzmDlzJpWAmThxIs29d3d3UwHGzs5Oq7sq9kdNTQ3q6+uRmppKb7AajQZr1qyBXq/HoUOHODVXMlw2btxosRkjDGczVlVVhcLCQqSnp8PT0xPe3t54++23UVhYiK+//vo6iXqC+bAjDx9Qbmo+/PBD6qmdkJCADz74ALNmzbL3sjhNW1sb/P39+/wey7JoamqisvunTp1CfHw8srKykJmZaeGDQpSRpVIplEolvL29aXCx5kmhrq4OdXV1VPGWPPfatWuhVCpx5MgReHt7W+35xhssy6K7uxsZGRkoKChAYGAglEol0tPTUVRUBFdXV+Tn58PNza3foMLzC3xAuUn55ptvcP/99+Pf//430tLS8P777+O7775DeXk5xGKxvZc37mFZFjKZDHv27MGuXbuQk5ODmJgYGlxiYmJocNHpdPTkQpSRiQTMSCx7CVevXkVNTQ1SUlLg5eUFoGeqfN26dWhubsaxY8f6LTjfbLS1tcHb2xtlZWXIy8uDQqHAjz/+iKtXryIuLg6ffPIJPD09+aAyCHxAuUlJS0vDzJkzsWXLFgA9eeOwsDA8/vjjeP755+28uhsLlmWhUCiosOKxY8fogFxWVhamT59Oc/hEGVkqlUKhUMDDw8PCsneo1NfXo7q6GsnJyfQEYjAYsGHDBtTW1uL48eP9nrRuRvpSENbpdMjOzsb777+P0NBQfPbZZ/Dy8hqS2vDNCh9QbkL0ej3c3Nywc+dOZGVl0a9v2LABHR0d2Lt3r/0WdxPQ0dGB77//nnq6hIaG0uBi7ttuMBgsZPddXV3pyWUgZeSGhgZqRObj4wOgR3bmoYceQmlpKXJycvhTaD+QmggJGnq9Ht988w3+85//gGVZHDhw4KZOEQ4G3+V1E9La2gqTyQSJRGLxdYlEwk/ujwE+Pj5Yv3491q9fj66uLurpsnTpUgQEBFBl5JkzZyIkJAQhISEwGo1obW2FVCpFXV0dRCIRrbmYKyM3NjaioqICycnJNJiYTCY88sgjuHjxIk6cOMEHkwEg76ODgwNYloWzszPuvfde6PV6HDx4EAzD2HmF3IY/odyENDU1ITQ0FGfOnEF6ejr9+rPPPovc3FycP3/ejqu7eenu7sbhw4epp4uHhwftFktPT6e5e5PJRD1d5HI5nJycIBaLIRQKcfXqVSQlJcHPz48+9oknnsDp06eRk5MzbuRhuIL5iUWn043b1uqxgj+h3IQEBARAKBRCKpVafF0qlSIoKMhOq+Jxc3PDXXfdhbvuugtarRY//PADdu3ahV//+tdwdnamJ5dbb73VQnZfoVCgrq4OHR0dcHJywsWLF9Hd3Y2FCxfi2WefxYkTJ3DixAk+mIwAgUBANbz4YDI4fGXpJsTZ2RkpKSn48ccf6dcYhsGPP/5ocWLhsR8ikQgrV67E9u3b0dLSgs8++wwCgQAbN27E5MmT8cgjj+Do0aMwmUzYv38/tm7disTERMTFxaG8vBwPPfQQJk2ahK+//hqvvvoqQkJC7P2Sxi38nMnQ4VNeNynffPMNNmzYgI8//hizZs3C+++/j2+//RZXrly5rrbCwx2MRiNOnjxJPV20Wi26u7vxhz/8Ac888wxEIhEYhsFLL72Ew4cPIzU1FcePH4dOp0NmZiY++ugjKgjJw2N1WJ6blg8++ICdOHEi6+zszM6aNYs9d+6cvZfEMwz27t3LikQiduXKlWxYWBjr5eXFrlmzhl21ahUrFovZ0tJSlmVZ1mQysadPn2Zff/11O6/4F2pra9kHHniAnTRpEisSidjIyEj2lVdeYXU6nb2XxjMK+BMKD8845MSJE1ixYgV27NiBu+++GwzD4Oeff8bnn3+Obdu24fjx45g9e7a9l9kvhw8fxjfffIO1a9diypQpuHTpEh5++GGsX78e77zzjr2XxzNC+IDCYzdOnjyJzZs3Iz8/H83Nzdi9e7fFXAxP/8hkMpw/fx4rV6687nvjdfBu8+bN2Lp1K2pqauy9FJ4RMv6uuhsclmVxs8R4tVqNhIQEfPjhh/ZeyrhDLBb3GUwAjMtgAvT4spB2Z57xCd82zCHUavWw5DXGO0uXLsXSpUvtvQweDlBVVYUPPviAT3eNc8bnVuYG5cEHH8RDDz0EnU5Hv0Ymc2+WUwvP+GYk1r2NjY1YsmQJ1qxZg4cffthOK+exBnwNhUOcOnUKy5cvx7Vr1+Dt7U2ndFtaWm74gUOBQMDXUG4Ahmvd29TUhHnz5uGWW27Bjh07xm26jqcHPuXFIUJDQxEWFoaDBw9i7dq1UKlU+PTTT/HCCy/gH//4BzZt2mTvJfLwDMhwrHsbGxsxf/58pKSkYPv27XwwuQHgAwpHYBgGkZGR8PHxQVlZGZRKJdavX4/Kykq88847NJiwvEMczw1AY2Mj5s2bh/DwcLzzzjuQy+X0ezf6afxGhg8oHIHszh599FH87W9/w44dOzBx4kR8/fXXSExMBNB/OyjJWvKBhme88MMPP6CqqgpVVVXXaYzxWfjxC3/G5ACk8N7c3IzS0lJcuXIF8+bNw/79+2kwAfpvByXFToZhYDKZxmLJVkGlUqGoqAhFRUUAgNraWhQVFaG+vt6+C+OxORs3bqQt8r3/8Yxf+BOKnSGWonV1dVizZg0CAgIAADNnzoSvry8MBgOcnJz6/Fmj0YijR49CLpdj4cKFCA0NHculj5q8vDzMnz+f/v///b//B6DH6GvHjh12WhUPD89I4U8odkYoFOLkyZOYO3cufHx8sH37djzwwAM4duwYTCZTv8EEAKqrq/HTTz9h27ZtiI2NxYIFC3DhwoXrHjfQrs+eO8J58+b1uUPlgwkPz/iEDyh2xGg04ne/+x3WrVuHZcuW4eDBgwgKCsLq1atx8uRJqNXqAX8+Ojoar7zyCk6ePInq6mqIRCL8+9//hl6vB9Azedzc3EzTYQSGYSzqLp999hkMBoPtXijHefPNNzFz5kx4enpCLBYjKysL5eXl9l4WD8+4gw8odkQgECAuLg7vvfcePvroIzg5OYFlWUydOhVBQUE4ePBgvz/LMAwqKyvx3//+Fzk5OQgMDMTmzZtx6NAhqoUkk8mQlpaGzz//3KL+4uDgQAv4UqkUf/vb33D27FnbvlgOk5ubi0cffRTnzp3DDz/8AIPBgEWLFg0a0Hl4eCzhBxs5BmkLnj17NhISErB169Y+W4XfffddvP3224iKioJUKkV7eztiYmLw888/Q6PR0ACyZs0auLu70zTSp59+CplMhk2bNsHT05M+brwKCtoCuVwOsViM3NxczJ07197L4eEZN/B3EI5BAsdf/vIX6iHeO5hotVq8//77+P3vf4/s7Gzk5+djz549aGxsxOLFi8GyLE1xbdy4EQcPHkRFRQVefvllPProo2hqaoJAIICDgwNef/11NDc3w8HBwSItZjQaLf5/M6FUKgGAFyrk4RkmfJcXR1m4cCEWLlzY5/e0Wi06OjoQEhICsVgMoGdCWa1W4/bbb4dQKKTBIDo6GrGxsVi1ahWcnJzw1VdfYdWqVQCA0tJSvPzyy0hNTUVwcDAcHBwgk8kgFovh6HhzXhoMw+Cpp57CrbfeihkzZth7OTw844qb864xznF3d8eTTz6JP/3pT6itrUVwcDDeffddaLVaqt5L0lfZ2dk4deoUYmNj8fXXXyM2Npa2Kn/xxReYNm0a0tLSoFKp8NVXX+G///0v6uvrMXfuXDzxxBOYNWuWPV/qmPPoo4/i0qVLOH36tL2XwsMz7uBTXuMQJycnvPbaa/jnP/+J0tJSCIVCiMViTJo0CREREQB6RPc2bdqEd999F5s2bYJAIEBsbCyAX1Jo//u//4tFixbBx8cH33//PTZv3oxFixZh27ZtYBgGzzzzDE6cOGGvlznmPPbYY9i/fz9ycnKum97m4eEZAjY3GeYZE3Jzc9mdO3eyLMuyO3bsYKOiotjbbruNPX36NFteXs5OmDCBzc3NpY+vrq5mBQIBe/ToUZZlWfbQoUPs5MmT2aqqKvqYgwcPsnV1dWP7QuwAwzDso48+yoaEhLAVFRX2Xg6nIf71Li4ubFBQELtu3Tq2sbHR3svi4Qj8CeUGYe7cuVi9ejUAYOrUqfjNb36DXbt24dZbb0V0dDT8/f1x6tQp+vgdO3YgKiqKnlri4uIQFhaG1atX49NPP4XJZMLSpUsRHh5ul9czljz66KP44osv8NVXX8HT0xMtLS1oaWmBRqOx99I4x/z58/Htt9+ivLwc2dnZqK6uxt13323vZfFwBL5t+CaBCE6ePXsWgYGBmDp1KpYvX463336bFuA1Gg22bNmC7777DnfffTeeffbZm6KduD9Rze3bt2Pjxo1ju5hxxr59+5CVlQWdTjegqgPPzQEfUG4iCgoKkJycjGvXriE8PBx79uxBRkYGtm/fjrS0NEybNg0sy2Lbtm144YUXcPjwYaSkpNh72TwcRaFQYNOmTWhsbOSbGHgA8EX5m4rk5GSYTCaEhYWhqKgI8+fPp4q/zz33HHbt2gWZTAa9Xo+2tjbel4KnT5577jm4u7vD398f9fX12Lt3r72XxMMR+IByk0GGJePj4+Hp6QkPDw/84Q9/QHR0NB599FEkJyfj22+/xdNPP43Q0NCbdrjRXmzduhXx8fHw8vKCl5cX0tPTcejQIZs+53B94J955hkUFhbi6NGjEAqFuP/++3nZeR4AfMqLpxfl5eVwcXHBpEmT7L2Um5Lvv/8eQqEQUVFRYFkWn332GTZv3ozCwkJMnz7dJs85XB94cxoaGhAWFoYzZ84gPT3dJuvjGT/wAYWHh+P4+flh8+bNePDBB+29lOuor69HeHg4cnJyMG/ePHsvh8fO8JPyPDwcxWQy4bvvvoNarebE7v/8+fO4cOEC5syZA19fX1RXV+Pll1/G5MmTObE+HvvDBxQeHo5RUlKC9PR0aLVaeHh4YPfu3Zg2bZq9lwU3Nzfs2rULf/7zn6FWqxEcHIwlS5bgpZdegouLi72Xx8MB+JQXDw/H0Ov1qK+vh1KpxM6dO/HJJ58gNzeXE0GFh2cg+IDCw8NxFi5ciMmTJ+Pjjz+291J4eAaEbxvm4eE4DMNAp9PZexk8PIPC11B4eDjECy+8gKVLl2LixIno6urCV199hRMnTuDIkSP2XhoPz6DwAYWHh0PIZDLcf//9aG5uhre3N+Lj43HkyBHceeed9l4aD8+g8DUUHh4eHh6rwNdQeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHisAh9QeHh4eHiswv8HGj6E8fc9pwwAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print(\"rank of C\",matrix_rank(C))\n", + "plot_matrix_and_subspace(C)" + ] + }, + { + "cell_type": "markdown", + "id": "32c5878b-07e6-4b9e-9048-f9a3f7d7e062", + "metadata": {}, + "source": [ + "### Understanding LoRA in PyTorch\n", + "\n", + "LoRA (Low-Rank Adaptation) is relatively simple to initialize in PyTorch. You initialize LoRA with the dimensions of the input (`in_dim`), $ m $, output (`out_dim`), $n $, a rank (`rank`), $ r $, and a scaling factor `alpha`. The parameters are initialized as follows:\n", + "\n", + "```\n", + "self.A = torch.nn.Parameter(torch.randn(in_dim, rank) * std_dev)\n", + "self.B = torch.nn.Parameter(torch.zeros(rank, out_dim))\n", + "```\n", + "\n", + "The use of ```nn.Parameter``` makes these values learnable parameters.\n", + "\n", + "In the forward function, LoRA uses the notation $BAx$ PyTorch, the input vector is a row, so the output becomes $x^TA^TB^T$ will drop the trapose from now on. The forward pass is implemented as:\n", + "```\n", + "x = self.alpha * (x @ self.A @ self.B)\n", + "```\n", + "The use of ```nn.Parameter``` makes these values learnable parameters.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "id": "32e2c20a-b272-4bf7-8985-296d2facc08c", + "metadata": {}, + "outputs": [], + "source": [ + "class LoRALayer(torch.nn.Module):\n", + " def __init__(self, in_dim, out_dim, rank, alpha):\n", + " super().__init__()\n", + " std_dev = 1 / torch.sqrt(torch.tensor(rank).float())\n", + " self.A = torch.nn.Parameter(torch.randn(in_dim, rank) * std_dev)\n", + " \n", + " self.B = torch.nn.Parameter(torch.zeros(rank, out_dim))\n", + " self.alpha = alpha\n", + "\n", + " def forward(self, x):\n", + " x = self.alpha * (x @ self.A @ self.B)\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "id": "cccba2e5-91d6-41a4-af68-2b8404b587e9", + "metadata": {}, + "source": [ + "This class ```LinearWithLoRA``` copies the original linear model and creates a ```LoRALayer``` object. \n", + "\n", + "```python\n", + "self.linear = linear.to(device)\n", + " self.lora = LoRALayer(\n", + " linear.in_features, linear.out_features, rank, alpha\n", + " ).to(device)\n", + "```\n", + "\n", + "Then, in the forward method apply both the original linear model and the output Lora model to the input x and add them together ```self.linear(x) + self.lora(x)```. This corresponds to:\n", + "\n", + " $xW_0 + xAB $\n" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "id": "19dd8b64-7bb2-44eb-bfd2-7c788b78605f", + "metadata": {}, + "outputs": [], + "source": [ + "class LinearWithLoRA(torch.nn.Module):\n", + " def __init__(self, linear, rank, alpha):\n", + " super().__init__()\n", + " self.linear = linear.to(device)\n", + " self.lora = LoRALayer(\n", + " linear.in_features, linear.out_features, rank, alpha\n", + " ).to(device)\n", + "\n", + " def forward(self, x):\n", + " \n", + " return self.linear(x) + self.lora(x)" + ] + }, + { + "cell_type": "markdown", + "id": "6a5003e5-c77e-475a-8c75-11fbfea325ee", + "metadata": {}, + "source": [ + "### Applying LoRA\n", + "To fine-tune with LoRA, first, load a pretrained TextClassifier model with LoRA (while freezing its layers), load its pretrained state from a file, and then disable gradient updates for all of its parameters to prevent further training. Here, you will load a model that was pretrained on the AG NEWS dataset, which is a dataset that has 4 classes. Note that when you initialize this model, you set `num_classes` to 4. Moreover, the pretrained AG_News model was trained with the embedding layer unfrozen. Hence you will initialize the model with `freeze=False`. Although you are initializing the model with layers unfrozen and the wrong number of classes for your task, you will make modifications to the model later on that correct this:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "id": "5908508a-a521-4a39-8b41-ac8157ea8bcd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=4, bias=True)\n", + ")" + ] + }, + "execution_count": 69, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from urllib.request import urlopen\n", + "import io\n", + "\n", + "model_lora=TextClassifier(num_classes=4,freeze=False)\n", + "model_lora.to(device)\n", + "\n", + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/uGC04Pom651hQs1XrZ0NsQ/my-model-freeze-false.pth')\n", + "\n", + "stream = io.BytesIO(urlopened.read())\n", + "state_dict = torch.load(stream, map_location=device)\n", + "model_lora.load_state_dict(state_dict)\n", + "\n", + "# Here, you freeze all layers:\n", + "for parm in model_lora.parameters():\n", + " parm.requires_grad=False\n", + "model_lora" + ] + }, + { + "cell_type": "markdown", + "id": "08738e26-8def-49e8-9d79-4925651a3162", + "metadata": {}, + "source": [ + "Note that the `for` loop in the above code froze all of the layers in the neural network, including the embedding layer.\n", + "\n", + "Additionally, note that the original model was on a classification problem that had four classes, while the IMDB dataset has just 2 classes. To account for this, you will replace the final layer with a new linear layer where the number of outputs equals 2:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "id": "8d6e2366-2c81-46c4-ba57-e27f446c4ebf", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 70, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.fc2=nn.Linear(in_features=128, out_features=2, bias=True).to(device)\n", + "model_lora" + ] + }, + { + "cell_type": "markdown", + "id": "afd1344f-e3dc-4968-b45b-4c255f854224", + "metadata": {}, + "source": [ + "Let's view all of the modules in the object.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "id": "202eebcf-d0dc-4d90-829b-3754d7658099", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 71, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora" + ] + }, + { + "cell_type": "markdown", + "id": "0229bf1d-195f-4d18-a99d-3080266db335", + "metadata": {}, + "source": [ + "Your task now is to replace the hidden layer with a LoRA layer. You can access the hidden layer as follows:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "id": "9e1a22b0-cca2-49c5-9bb7-5f7349db0cec", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Linear(in_features=100, out_features=128, bias=True)" + ] + }, + "execution_count": 72, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.fc1" + ] + }, + { + "cell_type": "markdown", + "id": "c7b71d41-bb47-4674-b989-727b273e48c8", + "metadata": {}, + "source": [ + "The following replaces this layer with a LoRA layer:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "3c7137ce-82c0-45c8-b337-8c0070e2e0ac", + "metadata": {}, + "outputs": [], + "source": [ + "model_lora.fc1=LinearWithLoRA(model_lora.fc1,rank=2, alpha=0.1).to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "62ad6238-94b8-4980-90e5-a5c2487c2e92", + "metadata": {}, + "source": [ + "Let's look at the hidden layer again to ensure that it is indeed converted to a LoRA layer.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "id": "16da330f-38bc-4118-9fcb-f9b88abe4b59", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + ")" + ] + }, + "execution_count": 74, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.fc1" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "85e9a340-c580-4512-b345-b9710d6a5eca", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + " )\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 75, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora" + ] + }, + { + "cell_type": "markdown", + "id": "1172a78a-3d2f-4cd9-8ff0-af92088a6ba2", + "metadata": {}, + "source": [ + "At this point, training the model is similar, with the only difference being that, except for the output layer, only the learnable parameters \n", + "```A``` and ```B``` will be updated. The code to select the values for `r` and `alpha`, which is not run, is nonetheless provided herein for your convenience.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "aa919697-b5f0-4c5a-9e26-de86ee43bd2e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + " )\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 76, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "14beb4cc-f276-4955-ab67-b5902123729f", + "metadata": {}, + "source": [ + "
\n", + "Click here to see code to select r and alpha\n", + " \n", + "```python \n", + "ranks = [1, 2, 5, 10]\n", + "alphas = [0.1, 0.5, 1.0, 2.0, 5.0]\n", + "\n", + "results=[]\n", + "accuracy_old=0\n", + "# Loop over each combination of 'r' and 'alpha'\n", + "for r in ranks:\n", + " for alpha in alphas:\n", + " print(f\"Testing with rank = {r} and alpha = {alpha}\")\n", + " model_name=f\"model_lora_rank{r}_alpha{alpha}_AGtoIBDM_final_adam_\"\n", + " \n", + " model_lora=TextClassifier(num_classes=4,freeze=False)\n", + " model_lora.to(device)\n", + " \n", + " urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/uGC04Pom651hQs1XrZ0NsQ/my-model-freeze-false.pth')\n", + " \n", + " stream = io.BytesIO(urlopened.read())\n", + " state_dict = torch.load(stream, map_location=device)\n", + " model_lora.load_state_dict(state_dict)\n", + " \n", + " for parm in model_lora.parameters():\n", + " parm.requires_grad=False\n", + " \n", + " model_lora.fc2=nn.Linear(in_features=128, out_features=2, bias=True)\n", + " model_lora.fc1=LinearWithLoRA(model_lora.fc1,rank=r, alpha=alpha )\n", + " optimizer = torch.optim.Adam(model_lora.parameters(), lr=LR)\n", + "\n", + " scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.1)\n", + " \n", + " model_lora.to(device)\n", + " \n", + " train_model(model_lora, optimizer, criterion, train_dataloader, valid_dataloader, epochs=300, model_name=model_name)\n", + " \n", + " accuracy=evaluate(valid_dataloader , model_lora, device)\n", + " result = {\n", + " 'rank': r,\n", + " 'alpha': alpha,\n", + " 'accuracy':accuracy\n", + " }\n", + "\n", + " # Append the dictionary to the results list\n", + " results.append(result)\n", + "\n", + " if accuracy>accuracy_old:\n", + " print(f\"Testing with rank = {r} and alpha = {alpha}\")\n", + " print(f\"accuracy: {accuracy} accuracy_old: {accuracy_old}\" )\n", + " accuracy_old=accuracy\n", + " torch.save(model.state_dict(), f\"{model_name}.pth\")\n", + " save_list_to_file(cum_loss_list, f\"{model_name}_loss.pkl\")\n", + " save_list_to_file(acc_epoch, f\"{model_name}_acc.pkl\")\n", + " \n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "dbc3dc3e-70c2-4ae3-b3b7-7359cf42dd6b", + "metadata": {}, + "source": [ + "\n", + "Let's set up the training components for the `model_lora` model, defining a learning rate of 1, using cross-entropy loss as the criterion, optimizing with stochastic gradient descent (SGD), and scheduling the learning rate to decay by a factor of 0.1 at each epoch:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "a6588ce7-93ff-4658-8fde-b15e1a9dea06", + "metadata": {}, + "outputs": [], + "source": [ + "LR=1\n", + "criterion = torch.nn.CrossEntropyLoss()\n", + "optimizer = torch.optim.SGD(model_lora.parameters(), lr=LR)\n", + "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "4e604097-52b3-4ac5-ace0-b749af99b5c6", + "metadata": {}, + "source": [ + "You have pretrained a model using an identical procedure for 300 epochs and saved it for your convenience. However, to give you a taste of how training works in practice, run the following code to train the model for just 2 epochs.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "1285bd07-be04-4502-b844-141fae08d4b4", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/10 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "cum_loss_list=load_list_from_file(\"model_lora_final2_loss.pkl\")\n", + "acc_epoch=load_list_from_file(\"model_lora_final2_acc.pkl\")\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "54c3cb54-ff64-46ee-be40-dbad7383eed1", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEYAAAAQCAYAAACr+QluAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAABJ0AAASdAHeZh94AAADhklEQVR4nO3Xa6hVVRAH8N/NqxlFTyIjMs0gJKPHh4qil2YRYWAhRWgWFPRBRDIoDBsnqPyQlBBUYigZ1IeUIrQiUrIgCIKosIdUJj3tZWqZmt4+rH1k3+M55557kwjqD5t11qxZ/z0zZ2bW2j19fX3+x4HorU8ycxNOaaP7fUSM6kSWmdOxopreHhFLh2pYN1yZ2YPbqucM9OAjLMWSiNhX0z0OU3ENzsRJ2I0PsAzL6vr9AlPhVzzaQr5jAEdOxmOV3hGddAfCILiewU3YgmfxOybjcVyIm2u60yr5t1iHzTgB1ymBvDozp0VEH60DszUiFgzSkR4l6j9hFe4azP6hcGXmVCUoX+C8iPixko/ASszIzBciYlW15VNci9VNmTQP7+B6JUgr4ZChOtCE2ZiIW/HbP8Q1tRoXNYICEbEb86vprJp8bUS8VA9KJf8OT1TTyxryVhlzaFXfoyvD3sf6iNjbyrrMHI+FWBwR6zNzYgdnOmKQXI1+93mLtYbs4swcUQWrE/ZU458NQauMGaU0vQeUXrMWGzPz0mbFzOytdDdj3gAv74ghcDWyZGyLtVOrsbf2u9N7G73olYa8OTDLMEkJzuFK934SY/ByZp7VpH8fzsEtEbGzkwFdYLBcq6vxzsw8tiHMzOHImt4xA/AsxASsiYhXG8J+pRQR2bTpQ9yRmTswFwtUtZ2Z5yv/7KKIeLsLR9piiFzPYQauwobMfBF/4AqcqGTeaOxrR5CZsxW/Pq649qPb5ttoTpdUhL14Wun089tt6gZD5ap63hTcgx8ws3o2Kkf19kp1S5v3zsJibMDlEfFzfb2nm5tvZh6FrdgVESMz82j80qUPiyNiTgfug8ZV4xyp3Me2RcTxLdbn4BGlIiZFxAHBa3UqtcIF1djo9rvwVBvdc5Ve8RY+wUClcTC5GrgRI5RLXz9k5t1KX3kPk+tHfR37M6Y6KjdHRL+7Q2aOwWs4DfdGxIOdLMrMBQjtr/HjMByfRcSe5vVBch0ZEduaZGdX9g7DhIj4prY2H/fjXVzZXD511DPmBszNzPX4UqnRccq3xUiswcOdHOkSryvfY2Ox6W9yvZaZO5WS2I7xir07MaUpKDOVoOzFm5id2XzW2BQRy+kfmHU4XUndi5TjequSxiuwovEd8S/C80rZTMdh+BpL8FBEfNWk27jvDMOcNnxvYDldNt//Iv4CzKFk7EtLSAoAAAAASUVORK5CYII=", + "text/latex": [ + "$\\displaystyle 54.492$" + ], + "text/plain": [ + "54.492" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader , model_lora, device)" + ] + }, + { + "cell_type": "markdown", + "id": "bb5c0973-b6cc-4024-9a2b-f9c406fb2503", + "metadata": {}, + "source": [ + "Instead of evaluating the model you just trained for 2 epochs, lets have a look at the LoRA model pretrained on 300 epochs:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "ccd15d85-9762-4bec-b5d5-665fe235b832", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "%%capture \n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/JWPRb1RMhKLRMUWOKw9pxA/model-lora-final2.pth\n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/_dm02rLyTrwsXEQh2r32sQ/model-lora-final2-acc.pkl\n", + "!wget https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/OZbVqKjoqOSIwnET8AB1KA/model-lora-final2-loss.pkl\n" + ] + }, + { + "cell_type": "markdown", + "id": "69dca0d8-c197-4d15-a9b2-a45265066a20", + "metadata": {}, + "source": [ + "The following shows the progression of the training of this model for 300 epochs:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "f7b34b56-6f20-44cf-b0fa-6b1bd88d8985", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHVCAYAAAB8NLYkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOy9d7glVZk1vqpOuKEjHW4HYhOkQRppSSKKIEpQRlFGPxVFTBgaHTCMozOODHwDo47hGwOGEXB+inFEERWJoig5SG6aDnSgc+6bTqj6/VFn7/3ud7+7qs7tmxpqPc997jmnqnbtinvt9aYgjuMYBQoUKFCgQIECBfZ4hGPdgQIFChQoUKBAgQLDg4LYFShQoECBAgUKPE9QELsCBQoUKFCgQIHnCQpiV6BAgQIFChQo8DxBQewKFChQoECBAgWeJyiIXYECBQoUKFCgwPMEBbErUKBAgQIFChR4nqAgdgUKFChQoECBAs8TFMSuQIECBQoUKFDgeYKC2BUoUKBAgQIFCjxPMKbE7oorrsCxxx6LSZMmoaenB2effTYWL17srHfXXXfh1a9+NSZMmIDJkyfjpJNOQn9/v16+ZcsWnHvuuZg8eTKmTp2K973vfdi1a9doHkqBAgUKFChQoMCYY0yJ3R133IFFixbh7rvvxs0334x6vY7TTjsNvb29ep277roLZ5xxBk477TTce++9uO+++3DhhRciDE3Xzz33XDz++OO4+eabccMNN+BPf/oTLrjggrE4pAIFChQoUKBAgTFDEMdxPNadUNi4cSN6enpwxx134KSTTgIAvOxlL8NrX/taXHbZZeI2Tz75JA4//HDcd999OOaYYwAAN954I173utdh9erVmDt3buZ+G40GHnroIcyaNcsijAUKFChQoECB8YcoirB+/XosXLgQ5XJ5rLszrjCuzsb27dsBANOmTQMAbNiwAffccw/OPfdcvPzlL8fSpUsxf/58/Pu//zte8YpXAEgUvalTp2pSBwCvec1rEIYh7rnnHrzpTW9y9hPVaohrNf39/gcewAknnzyCR1agQIECBQoUGG7ce++9OPbYY8e6G+MK44bYRVGEiy66CCeeeCKOOOIIAMCyZcsAAJdccgn+8z//E0cddRT+53/+B6eeeioee+wxHHLIIVi3bh16enqstsrlMqZNm4Z169aJ+9r8ne9i0ze/qb/3Dg4CSG6QOXPmjMThFShQoECBAgWGCWvXrsVxxx2HWbNmjXVXxh3GDbFbtGgRHnvsMdx55536tyiKAAAf/OAH8Z73vAcAsHDhQtx666246qqrcMUVVwxpX9M/eAGmved8/b17zRpg/nzMmTMH++yzz9APokCBAgUKFCgwaijcp1yMC2J34YUX6qAHSqyUenb44Ydb6x922GFYuXIlAGD27NnYsGGDtbzRaGDLli2YPXu2uL+wWgWqVf29NGHCsBxHgQIFChQoUKDAWGJMqW4cx7jwwgtx3XXX4bbbbsO8efOs5QcccADmzp3rpEB5+umnsf/++wMATjjhBGzbtg0PPPCAXn7bbbchiiIcf/zxI38QBQoUKFCgQIEC4wRjSuwWLVqEH/7wh7j22msxadIkrFu3DuvWrdM56oIgwKc+9Sn813/9F37xi1/gmWeewec+9zk89dRTeN/73gcgUe/OOOMMfOADH8C9996Lv/zlL7jwwgvxtre9LVdEbIECBQoUKFDg+Y0DDjgAQRA4f4sWLQIADAwMYNGiRZg+fTomTpyIc845B+vXr09tM45j/Ou//ivmzJmDrq4uvOY1r8GSJUtG43DSEY8hAIh/V199tbXeFVdcEe+zzz5xd3d3fMIJJ8R//vOfreWbN2+O3/72t8cTJ06MJ0+eHL/nPe+Jd+7cmbsfq1atigHEq1atGo7DKlCgQIECBQqMINodtzds2BCvXbtW/918880xgPj222+P4ziOP/ShD8X77rtvfOutt8b3339//LKXvSx++ctfntrmf/zHf8RTpkyJf/WrX8V/+9vf4je84Q3xvHnz4v7+/t09vN3CuMpjN1ZYvXo19t13X6xataoInihQoECBAgXGOdS4/dTS5Zg7d2/9e7UcoqNcytz+oosuwg033IAlS5Zgx44dmDlzJq699lr8/d//PQDgqaeewmGHHYa77roLL3vZy5zt4zjG3Llz8YlPfAKf/OQnASQp22bNmoVrrrkGb3vb24bpSNtHEU5SoECBAgUKFNgj8dqv34cFl9yk/751+9LMbWq1Gn74wx/ive99L4IgwAMPPIB6vY7XvOY1ep358+djv/32w1133SW2sXz5cqxbt87aZsqUKTj++OO924wWxkVUbIECBQoUKFCgQLu4+aPHOopdFn71q19h27ZtOP/88wEA69atQ7VaxdSpU631Zs2a5c2Hq37nefTSthktFMSuQIECBQoUKLBHYkK1jEmdlba2+f73v48zzzzzeRtgWZhiCxQoUKBAgQIvCDz77LO45ZZb8P73v1//Nnv2bNRqNWzbts1ad/369d58uOp3Hjmbts1ooSB2BQoUKFCgQIEXBK6++mr09PTg9a9/vf7t6KOPRqVSwa233qp/W7x4MVauXIkTTjhBbGfevHmYPXu2tc2OHTtwzz33eLcZLRTErkCBAgUKFCjwvEcURbj66qvx7ne/G+Wy8USbMmUK3ve+9+HjH/84br/9djzwwAN4z3vegxNOOMGKiJ0/fz6uu+46AEme3Ysuugj/9//+X1x//fV49NFHcd5552Hu3Lk4++yzR/vQLBQ+dgUKFChQoECB5z1uueUWrFy5Eu9973udZV/96lcRhiHOOeccDA4O4vTTT8e3vvUta53Fixdj+/bt+vs//uM/ore3FxdccAG2bduGV7ziFbjxxhvR2dk54seShiKPHYo8dgUKFChQoMCehGLc9qMwxRYoUKBAgQIFCjxPUBC7EUIcRdj+mxuw6TvfRTQwMNbdKVCgQIECz3PsGKjjxsfWYaDeHOuuFBhDFMRupBAEWHfJJdj41a+i/txzY92bAgUKFCjwPMeF1z6ED/3wAVz+uyfHuisFxhAFsRshBEGASiv5YX3NmjHuTYECBQoUeL7jT09vBAD8z13PjnFPCowlCmI3gjDErlDsChQoUKBAgQIjj4LYjSAqeyf16wpTbIECBQoUKFBgNFAQuxFEZe+WYlcQuwIFChQoMMKoloohvUBB7EYUhY9dgQIFChQYLUzuKmoOFCiI3YiiMMUWKFCgQIHRwuTOylh3ocA4QEHsRhBKsWts2IC4Vhvj3hQoUKBAgeczJnUZYlcUlXrhoiB2I4jS9OkIOjqAOEZ9/fqx7k6BAgUKFHgeYwohdn21IknxCxUFsRtBFLnsChQoUKAARxTFI6KodZTNkL61r7ASvVBRELsRRpHLrkCBAgUKKAzUmzj1K3fgwh8/NOxtU664ra8+7O0X2DNQELsRhiZ2RQBFgQIvSERRjIdWbi3qdwrYOVDHY2u2j3U3RhW3PbUByzf14rePrB32tqkKWBC7Fy4KYjfC0JGxhSm2QIEXJP7w+Dq86Vt/xX/8/qmx7sq4wz/+4hGc9fU78bdV28a6K6OG/hH0fYsosesvTLEvVBTEboRR3X8/AEBtxYqx7UiBAi8QbO2t4b9uXYLVW/vGuisAgIdbpGXd9oGx7cg4xKrWNVqxuXeMe5Jgyfqd+Obtz4wo+eofQeU2IqbYrYVi94JFkc1whFE96CAAwODSpYjjGEEQjHGPChR4fuOjP34Idz6zCTc/sR6/+egrxro7WL4pIS1RkX7CwWA9AjB+Ijhf+9U/AQB2DjTwT2fOH5F9jKRJnt5j24vgiRcsCsVuhFE94ACgVEK0cycaGzaOdXcKvADx16Wb8NZv34XF63aOdVdGBXc+swkA8Og48d1SalRB61wMNhJi1zvYGNX9/u7RtXj7d+/G+h2yivromm0jtu+RVAPjQrErgILYjTjCahXV/Vrm2KXPjHFvCrwQ8Y7v3YN7V2zBZ375yFh3ZVQxuXPsDRJRFGPF5sTcWCSMdTHYSEhO7+DoKnYf+dGDuGvZZvznHxaLy8vh8A6NfTVDXAcao+RjVxC7FywKYjcK6Di4ZY59ZukY96TACxmjPXiOBSh52n/6hDHsSYLntvej1lKlooLXOVCKHSU+o4mB1v45yuHwucx864/P4PB//QP+9HRisemvyfscDtjErjDFvlBRELtRAPWzK1BgNEGJzpypnWPYk9HBll4zmO2zV9cY9iTBik0mgKPwsXOhfOx2jbIpVsGn6lZKwzc0fvHGRBX8x18kijkNnhiKivvQyq1YunGXuIxOHrb1F4rdCxVjb6t4AaDjoIMBAIOFKbbAKGPjrkH9edak5z+xo9GV1fLYz1uXk/4UvM5GHMfaFDtWwROTSQkuinJp+IPc1KSDBk80o7itfa3fMYA3feuvAIAV//F6ZzklikXliRcuxv7N9wKAMsXWljxT+NkUGFVQxSh+AbjvL7cUsjHsSAsrNhliVyh2NhpRrK/RaAZPUGI1xUPshlOxU6g1I2f/zTbviTXb+vXnetM16dJ7fnvhY/eCRUHsRgHVefOAMERz+3Y0N20a6+4UeAFh+SZjshn0+BM9n0CPNxoHzI4Su4LX2aD3Y+8o+tjRoIIJ1ZK4znD62HHYptj2tu0m/d0umFrp5GHHQEHsXqgoiN0oIOzsRMehhwIAdtx00xj3psALCVTBUv5Mz2eMN582yxT7AlBM20GNErtRCOz5yb0r8dqv3GGVMPNx//IwKnbcJYCmO2m2Ofmg68vEznyuN+PCQvQCRUHsRglT3/QmAMC2n/+ieNjGGFEUe/NX7WmI4zi1ogFVjAZJmoXtffXnZe3S5SNs+uyvNa0AjTQ0mhFWbSFE8/nPq9sCvR9HIyr2n375KJZs2IWP/OhB/ZvvHqkMo4/d9AlV6/vumGLrzXRix8cWuv5wI45jrN7aV4xn4xAFsRslTHnD3yGoVjH41FMYeOyxse7OCxqX3vAEjr/8Vjy4cutYd2W3cfnvnsTLrrgVP7l3pbicBhMo09fGnYN45Rdvwxu/8ZdR6eNooRnF1vGOhCX2xC/chpdednMuM9dz2wasgXU8KIjjCVRBHs1UPDXim+ZV7IYxj91e3YbYNaPYNsW2SfYbpO9ZplgAaIzgbOKHdz+LV3zhdnz7jmUjto8CQ0NB7EYJpalTMen00wEAaz7+CWz/zW/GuEcvXKhUAcs3jl19yg07BvCz+1fttmr2vT8vB5CQVY6IER1F7L7352XYMdDA4vU7LXPYno6HV22zoiuH28du50Bdq3XLctw7y1n904LW2RgrHzsKqjbRz8Op2O01wQRobOurYYAQ2nYVO0pKd0jEjj3OI6nY/edNTwMAvnDjUyO2jwJDQ0HsRhEzPngBStOmob5qFZ771D9icFkx0xkLKD+Vdl+qw4mv3rIE//iLR3DDI2uHpT2JIG7qHbQGkcFGE/VmhP99YLX+bSTLG402bn9qg/V9uBWyZzcbs2q3x+meQpnBlR9+YbKyYZlixyh5NvVZoyRoONOdhKQ++Obemj35aPOeaJA+SpUleHtS5Oxw4aX7TdWfiwjc8YWC2I0iOg4+GAfffBM6DjsMAAqT7BhBvfzGcqDd0pvklxsuXz9JnBpgGe4H6xF+9+habCY+Yn31sVFKRgK3L06I3YkHTwcwfKbYa/6yHJfd8ASWEf+9PAOm8vfbd1r3sPbn+QKq2NWa0Ziox/SaUDWsHVNsM4rxL796FNfeI7tD0NfMpp2Dlj9hu6oyNa3KPnZs/RFU7GZPMXkx/7QkqarxX7cuwddvXTJi+yyQDwWxG2WEEyage+FRAICBxXKdwgIjC/VuHMuBVr1wdw6MHLGqMfIx2Ihw97LN1m/Ut2lPVpTW7xjA48/tQBAApxzaA2D4FLtLfvMEvn/nclx153L9W55oRkXsDpo5cVj7A+xZ18rXVx6lPdIBFFLOOnpNKLFsxxT78Kqt+OHdK3HF754Uj5VGQ2/cNWgpdm2bYhtZUbGuYjdS9woljbcv3oANOwbwlZufxpdvfvp5GZi1J6EgdmOAjkPnAwAGnyqI3VhAvfzaTTUwnFCka9fgyJkwuKo02Gg6TupqMP3tI2tx1KU363qWexruaPX7yH2mYuakDgDDb4p9eNU2/bmR495R/o0Hzkhq1g5Xd/75ukdx0pdux849IE/Z2u39OP7yW/Hlm9x3HTXFAkDvCLsFqPuCIvYQuyDIT+yeXLsTALBzsIFNu9yIaer3RqOkgfYnl1mKHSeKA/Um3vCNv+D9P7i/vR3lAH1/3rF4I9bvMFVuikChsUVB7MYAnYe+CAAw8HRB7MYCzXFgilWz3V0jqNi5xC5ySjcpovcPP3kI2/vrOO+qe9vax9rt/bjtqfW4Z9nmMSXKa7YmGfmPmDtZD8pDCQiM4xiPP7c909TKj3Vrbw23PbUedy7ZpH0ZV7f6NG+mInbDc35uf2oDVm3p14RiPOOyG57Ahp2D+PptbjlFnjB7pKtPSNUk7LxvJFrWcy8v27jLSXfz9HpzHWi6HbMP0xb100zbjw/1jKhYfout3tqPR9dsx61PrW9rP3lAJzebe2tYRpKDj+W7oMAYE7srrrgCxx57LCZNmoSenh6cffbZWMzMkyeffDKCILD+PvShD1nr8OVBEOAnP/nJaB5KW+h40YuAIEBz4yY0Nm/O3qDAsEK9c8by3VPXit3IEzvlvD9YjxwTSX/Lx26ipxh6GmqNCGf+vz/jvdfcj//z3btx7T3P7l6HdwNKyaiWQ328Q1ENbnhkLV7/X3fi/92S7ifEfZfe8d/34L3X3I93fv8efOnGxVi1pQ/NKEZXpYQ5LV+k4brf1IC6Jyh2T6WQT0exG2FiJ5EoSkAo0ZSu1drt/Xj1l+/A8ZffYv3+1DpzjCsEYkdvw2eZYtcuAcrKY8fveTWRi+Phn8jyVCr0PBS8bmwxpsTujjvuwKJFi3D33Xfj5ptvRr1ex2mnnYbeXvvh+MAHPoC1a9fqvy9+8YtOW1dffbW1ztlnnz1KR9E+wu5uVPfbDwAw8FQRKj7aUC/4sTQX1PXgvHuDWTUlQ77yx5nUmfgWDTaajh+TUuzmz56kf9uWs3h4f61pReYphWosoIhWOQx0FOJQrq86htVb+1LX4wMyXf/pDbu0GXb/6d26P8NVeUId10j6Zw4XlglER4H72I10LjvJn81nipXunafXJ4pUvRlrEhrHsa3YbRaIHbnuril26IqdmO6EtUfTyKh7dt32AVz5x6W5n3Mf+ORmMSF2e5IP6PMR7U/ThxE33nij9f2aa65BT08PHnjgAZx00kn69+7ubsyePTu1ralTp2auoxDVaohr5qZu9o5+PrOO+fNRe/ZZDC5+GhNPPHHU9/9CxnjwsWsMk2LXUQmdIAkF9fvEjjK299cRxYYMVMshao1IE72uiknf8fCqbTi5FYCQBj6IjOX5VEpGuRQSYtd+O+qYqJkpDNy2uFpB1aCtvTWsbJncDpg+YbdMwxLUed45wgrXSMMxxY5w8ISk2PmiYqVbZzJRtZ9evxML99sLG3YOWpMbSbGj+1jLqsTsTroTUbFj9xhNZ9SIYpRLwH//eRn++87lKIXABScd1Nb+KfjzvrhQ7MYNxpWP3fbtSQ2/adOmWb//6Ec/wowZM3DEEUfgM5/5DPr63Nn0okWLMGPGDBx33HG46qqrUmcMm7/zXTx9zLH6b9nrXje8B5IDHS0/u8HFhWI32lAvpOGeVDaaERb96EF8+46lmesOlym2s+LPp1ZvDZyTyIC0tTUIzWiVOVIqCc13R4ME0uBmuR9Dotwa0SphsFumWDVwmnskFgcpPqjRr1t6a9qJvmdyx271R0JzGE2xW3preNf378Fv/vbcbrcFAD+/fxXec/W92DXYwKZdxpl+v1bKFwpuih3pqFhJsaPXhKph0vhBr7lS6SiZAbJ97Nxlyf87l2zCO//7Hj0h8IH2Ucpjx/st5cxTVVMkYijhwZVb8Y7v3Y0nntth/c6f9zXbjGJfBE+MLcZUsaOIoggXXXQRTjzxRBxxxBH693e84x3Yf//9MXfuXDzyyCP49Kc/jcWLF+OXv/ylXufSSy/Fq1/9anR3d+Omm27CRz7yEezatQsf+9jHxH1N/+AFmPae8/X37jVrgPnzR+zYJHQcfDAAYHDZ8ow1C/gw2Giio5ydKHaw0US1FGrlRL1zhvvl8/vH1uG3j67Fbx9diw+9Kn0mPJTgCel4Oyv+uZkaBCZ0mMd8e39COKZP7MBz2wd0eaMBMsg+tHJbrv5wwvN8UOzU4K/a890ifFCjpGFLbw2bW3kKZ0zsMKbYYTo9atfDYYr9998+iT8v2YQ/L9mEv3vJXGtZ72ADOwcamNxVRnc131DxqV88AgD4zh1L8bIDp+vfO8rufcpNsbtG2hQr+dhlmGLpM0evufInU8TusDmT8eTaHXh2c1I7lUbVpl131ad3fv8eAMCnfvE3/PSDJzjr1ZsRAtg+dv31JmqNCFVybvkhUrKs+q/ePfz8+/Dmb/0VAPCJn/8Nv/+HVzp9nzulE89xJbKQ7MYU44bYLVq0CI899hjuvPNO6/cLLrhAf16wYAHmzJmDU089FUuXLsVBByWD5+c+9zm9zsKFC9Hb24svfelLXmIXVqtA1dTvK02YMJyHkgsd8+YBAGorVjgvggLZuOT6x3HNX1fgDxedhEOJfxjHxp2DOOGKW3HqYT34zruOAWBe5sNdeeK5bfl9zJTZJ6857Qd/XYFLfvM4rjr/WJ2nDQA6U4it2kdHOUS1lJhs1cAwfaJS7JL9c8Uuzz3J1YExVexax1ouBVC5ZYcyuKhtmi0FkJL/Gz76Cvzrrx/Dgyu3Of5FdF/99ab21Zs+sQp1FofLx244FbtnNsjBDcs27sLr/+tO9Neb6KqUcMPHXqHz8eXBxp2DzDTnHjs3xfaNQfAE7RYldnEMfP3WJfh/ty7BTz94Ao7efy9ZsWv9f81hPXh6/U7015tYv2PQSt6bZj3iZJNH3AKJsnbaV+/AvBkTcOJBM5xlNI2LOs/qeZdK7Kn3gs+Fwwce3KImjnOmdrnEruB1Y4pxYYq98MILccMNN+D222/HPvvsk7ru8ccfDwB45hk3fJ6us3r1agwODnrXGWtU9tsPCAJEO3eiWUTGto1r/roCAPBft6VHL/7qoTVoRDH+8LgJ9zeVJ4a3T1vacEZWxKDWyJdx//PXP444Bj7ywwet36kptsFe1IrEVUqho5hMn5AMBurFP0iiZZU/HsdAvYk7nt6oTWiuYjd2dWcVqayE4W4FTyiyr9qjx7jf9G59vrmPHZ8kLGk52k+f0GF87IbpflP7Gg7FbrNAJIAkN5tSc/vrTfz3n9srfzjYiGxnenGd0c1jJ5piU6Jiv3zz02hEMT7xs4cB2BMXdWzKHDpnShf22asLAPCje561gmnUZuXQnSjxLkmuFT+5dyXW7xjE3cu2OGl4uDlV7Us972mKXbuVPvafbpvTFSmdQ0is6UfB7MYSY0rs4jjGhRdeiOuuuw633XYb5rVUrDQ8/PDDAIA5c+akrrPXXnuho8NNSDleEHZ0oLL33gAS1a7A0JAWFQoAofAy1VGxwzyt3OoZJCVQYtBOmod+lq6EmmL72DI1CFTLITqYyXbGpESxUy9+ngZFMlv995+X4d1X3Ysft0onjScfuzpV7HYneIINfvQYwyBAuXW/0fMTx7EeoFXFgnWtUnEzJlaH3ccuioaP2EkKEWD6Wmp1/pcPrmnr/q41Ijyz0eQ1kw59tPPYqUv2u4+9Eh85+SDrN8DvY7ei5fdGJy6bdtWwadcgOU8mEfXXb3sG77vGJARWSu1L99vL6RMnm5LJ+i9LzcS/xpRiTuxUv5V5lkYa63u7dRx5iN0Oogrvy/wk1fM+d2qXs11B7MYWY0rsFi1ahB/+8Ie49tprMWnSJKxbtw7r1q1Df39ixli6dCkuu+wyPPDAA1ixYgWuv/56nHfeeTjppJNw5JFHAgB+85vf4L//+7/x2GOP4ZlnnsGVV16Jyy+/HB/96EfH8tByoXrAAQAKYrc7kGbBWcvVy3y4TbFb2yiETV+qvgCKJet34vLfPekdfAG7pmU/UzzUPqql0PHNmzkxmfQolWSAveSlF/OGnYkCvrHlFD+eomIbgo/dUFIuqGOQAmzCwNxPlMTSw1ZKqMKMiR1mcpHSnR0DdVzx+yfx+HPbs/sYt++f6QNPWK2gru3x86bhxXMnY7AR4dp75VqoEgYbkZVOQzTF1o2rQNKX0THFljwpcSxTrLA9N78/vW6nvk/CIMBHTz0ErzwkMZVu2GlMk4oPvnR/gdixZ4YrdlEU425C7PrZOeIpTyJG7CTFTin5gzlMsTTKt5v1TfV99mRBsYuAH9+7Ej9p454pMHwYU2J35ZVXYvv27Tj55JMxZ84c/ffTn/4UAFCtVnHLLbfgtNNOw/z58/GJT3wC55xzDn7zm9/oNiqVCr75zW/ihBNOwFFHHYXvfOc7+MpXvoLPf/7zY3VYuVFtKZSDy5ePcU/2XJQzajpKip16IQ03D2knLxQlBj7l5b//vBzf/dMy3PDIc1Y6Ego6MPFBWikQlVJgKQGlMMDkVt3MvsH8ih0/b07B8fESFds61KEQTWOKdX3swiDQChZtm35WSqgC9bFLUzFuenw9vnPHMnzrj+kR1VQd3DGCCYrVPsIgwDuOT3Ju3tFGublaM7LVItHHLlk+nUVojxSaRF1T7wWL2LHKE1O77dqy/H56dkufpWy+dL+9cOkbk8A/m/gnn4/ad4rTJz754Irdo2u2W/3i5upsUyypS8sUuzzBEzTK1035oxQ7l9jtGmzgM798FP/0y0fbUnoLDA/G3BQr/Z1//vkAgH333Rd33HEHNm/ejIGBASxZsgRf/OIXMXnyZN3GGWecgYceegg7d+7Erl278PDDD+ODH/wgwnBcuA+monrA/gCA2rLl2PnHP6KxdesY92jPg1QmiKIkBACol+lwJ9FsR7GjZh+fYqdMqwP1pg524KCHwBWPmiZ2oRU5110pYUIryrG31kQcxy6xE1NDtP570sU0m/7zed+KLXjlF2/DSy+7GR/98UPDfu7lqNj29xExxY62EXgVO0LsJhrFrloOMbGjnMvHTikxgxnF05s5JgQc37htCd78rb+kKmIlNgFSxxQERt3NKrNGMVhvWqZVyf1SmWL3mmAH8owUqLommcetkmIxsD8xPe4YqDsTl0YU6zbV+SsLxF9hYkfF+S1Lsbt98QbrOz9HW9lkkit2vQKxq7cSl+cJnlixifoKsuNvbT+5q4Luqt1v6mqilP7xgDVr1uCd73wnpk+fjq6uLixYsAD332/M5lIVqyAI8KUvfcnb5iWXXOKsP3+Us2xwjH/28zyGMsXuuv12rP7Qh7HhC25FjQLpKGcQeIn36ajYMfKxi+PYSluwa1AmhBFRyChhaHpIhaPYtV7glbIdPNFZLaG7I3kR99eaaERurjYxmSsjO+342N3yxHqs2tKPLb01/OZvzw17KTWt2BEfO9q9gXozVxQpN1fRQ7IUO4sEyMRuxoQqAg+J4OAm4Kz+AfmjYn/+wGo8uHIbHlltm3npNZg2wZ44qN34VMosDDYiK+GwROQVsVP7Hq0ExbYp1izn6U5oipcVm3rd3IWE2Kn2SinEPwyAD550oN0GOy3cF5bXluWK3e2LbRVVnWblekFNtzqVj/axy1ZIV5BKGuoaqvyE6tgrpRA9k2wXBPp+S3MlGU1s3boVJ554IiqVCn7/+9/jiSeewJe//GXstZcxkdPqVWvXrsVVV12FIAhwzjnnpLb94he/2NqOZ/cYbRTEbgzRwYJFtv/qV2PTkT0YlQxTbEkgflp5GmbLYd6oWD5A+JQXqhrRgZe+KPOYYrmPXXe1pP1lemsNR62T+giYgcGcP3udtKhYd7Y/QopdaGrFUtXx7G/+BSd/6Y/isUr9pAmKFcIgEBU7yxRLiV1rsOP5EyVIUbhp/QMSYpZH+VTnmh/7RqKkcBMgJSPhEIjdjgE7slrassaJ3SiZYqliR88fD+ag53q5QOyaUewEmUiKnf4YAP905nw89LnX4tBZk/Q+6LrcF5bvUyl273zZfggC4E9Pb7RS1rg+doIpto2oWG6K/dE9z+KY/3sLrntotb5nS2FgpVxJ9mHa5qricKO31sDOgbr+49HWCl/4whew77774uqrr8Zxxx2HefPm4bTTTtNp0wBg9uzZ1t+vf/1rnHLKKTjwwAPFNhXK5bK13YwZM1LXH2kUxG4MUeYl0CoVxI09u0zQaCPLx44qduolHgmD9u4iimJr4E6LuK0zUuNTr9RAFMcAtZRZjtmkKe5Y7YuK7aqUdNLivsGmzmEXBMkf3TeFq9jx/oqHIa473P54dh471xT79Pqd2Nxb0+k9BupN/OqhNY6awP2Q6MAaBmaiYA3cZHycQUzmyndMIhEcPhWUgxOGPClC1DacuGzYMeCsoxATEqTcGdohdvy8pvnY7d2Kqly/Y8BZJw9ufGwdlpEI3Oe29ePXD68RFLbkfxgGmmzTdbhiR7u8YlOfUG3Er9g1o9hx+QhbZrq9JlQtskxN5DzhuFP7tfWuOHDGRLzmsFkAgKv/ssJZP83Hrt5GHjuq2DXjWKfxWbxul26vHAbomWT72dFzNdKK3Wu/fh8WXHKT/vvW7bKf6vXXX49jjjkGb3nLW9DT04OFCxfie9/7nrfd9evX47e//S3e9773ZfZhyZIlmDt3Lg488ECce+65WLlybINGCmI3hgjCEDM+9lFMOKmVzbteR23VqrHt1B4AOkBmm2LN8gYjJsNpiuVOzGkRt3WmbPmiG2laFtpV6rNCzwVX7NRAzoMnuqol7RNDFbuOcqgHcUl84/n/OFFJU+z4ueZ54HYXOo8dTXdCdsH9A3/54Bpc9NOH8a3b7XyYaqyTFLTAo9jRwZcqF9Nb6l2e9Cv83vSBn7Y8kbGqba7Y0fuIE231NSCm2HZ8Fnm5K+nY1f2pEh9v2DmYqahyPLVuBz70wwfwyZ//Tf/2hm/8Bf/wk4fxP3etsNbVwRPWMZnldWZep8e7YnOv6GOnfjKKnXnOeGS1ug+Sz2Y/NJqd+wzzU67M1ZVSgPNffgAA4IZH1pJ+J/9NuhNiitWTlnyK3Y6BunUdY3JO6s1IP8OlMHACKOqjSOxu/uixePSS0/TfR06Rq/4sW7YMV155JQ455BD84Q9/wIc//GF87GMfww9+8ANx/R/84AeYNGkS3vzmN6fu//jjj8c111yDG2+8EVdeeSWWL1+OV77yldi5U07+PRooiN0YY+ZHPoL9vvtddBx+GACgVkTIZoIqD5mmWPIybTC/qeEUjVQJKYW0QbDOXqg+xY4qY7Q9akKjx+CPihVMsS3/of5aUysnHeWSURKE/jf1eZMVuzTzqlOlYhRMsVyhBczgpsxDm3treG5bPz7+s4fx2Jrt+tgauqSYbWorte432n96rmi6Ex7wklZ5wpD49OPk1yWPn13TEwVJiZ2r2CX/w8AQkqyJUJoiKfrYtfoze0qnnmi0U70FADbtTK4jDVxSPmC/JYQHIMETIUS/R155wjXFsqTUlik2+a1E3kecrFPVnZJlqTqE/u4odsm6lVKIea28eZQY6jx2rQ7Rd6UxxUbOMgl9zDQeRXYbDfLMnX/iPHz4ZEOo7Jx/Ixs8MaFaxqTOiv7zlZmMoggvfelLcfnll2PhwoW44IIL8IEPfADf/va3xfWvuuoqnHvuuejsdKN+Kc4880y85S1vwZFHHonTTz8dv/vd77Bt2zb87Gc/2+1jGyoKYjdO0DEvseEPLs0uIP9CBzUhlLOiYslipZRxk+JwYONOZnpKeWfymb/fFNtqK7YVO5vYmQU8j51N7GxTrAqeaEQxtvcn+++sUMUujymWK3Zppkb7+4iaYgObnNJ+NrSKYpSH3z6yFr98cA3+v7uedYIYTBBB8r+iTWh2agy1DvWFnDkExS4rtyI/xzvaUewafh87XrXEkJFAuztk+/+1t0xNKDoroTbHrmmT2NUFk7kC9e2i93MpkANsaiwqlja5bvuAq2pKwRNkIqkVu9Z3GqBvyDIzl7Lr7zPF0uhvuo3qYoeQHonXQc5S7Lh6St9D9SjW56NcCrD31C58+oz5mNuqQkHdTTbvGh/BE3PmzMHhhx9u/XbYYYeJZtM///nPWLx4Md7//ve3vZ+pU6fiRS96UWp1rJFGQezGCaoHJcSutqxQ7LJAlYesBMWhqNgNP7Hjil2qKZYNotSc9rP7V+G4f78Fj6zeZvkCUsXD5xvlKnZm9m752FXLVrJRFc3bWSmlRkBypa6dqNh2Ai2GAhqhx4mUPfDZx9CMYl3NY7DRFEqKJf8D7UMVWstp+6UwsIidUuyUdS7Vxy6n3yc/jzsH6li+qRcvu/xWXPMX+d2h2nZNsTl87ML8ip2UDmViy5czrVZsR7mky3GpGrt5IVUIUaDuEfQeKOXwsePPHE1tQtvk6U5o2hjfPQQwU2ydRg/bx8BPKXWv0DWRhftbqsjj+NhlEbsGJ3ZkQtSILB87BXWMVNEeacUuL0488UQsXrzY+u3pp5/G/vvv76z7/e9/H0cffTRe8pKXtL2fXbt2YenSpanVsUYaBbEbJ+hoRd0MLisUuyzQmXUWNwssYtea3Y8EsdslO+FLSAue+MdfPIINOwfxhRufMuoR8/fZuIv62Jl2+uq2elOjwRPUFFspoUxUPOUD01kuiRGl/Ji8eezaUOz4OeCI4xiLSWb/LCjlpkwSFBtTrNtH4ysU6/uiEcU6Fx9PUKzOi1KvpKjHIAispLbKLJtHscubgkeKqP7n6x7Fuh0DuOQ3T4jbGB87eyDf2KaPXbt9Awyxkx41Q+xC7LNXkjOO1ljNA3XtJIWZ+odZQTBhgBIhVgo15mNHW2xEkajYaVNsYEfF0n3qoA3yLtKm2CgWAxwUfES/UjLqehzTnLDJcp42hbad18eO3y/Ux64RxZaPHT8u6kPrq0c82rj44otx99134/LLL8czzzyDa6+9Ft/97nexaNEia70dO3bg5z//uVetO/XUU/GNb3xDf//kJz+JO+64AytWrMBf//pXvOlNb0KpVMLb3/72ET2eNBTEbpygeqBR7IY7eevzDTSJa5bpykpAysxrw+pjx2aladeQm70kU+xe3VUrvQglJxt25DTF6tl96ARPANB+TZu1YhdaAw4HVzrbUezcQIv0k/+T+1bh9K/9Cd+/M1/x+YblY2cTKdpPbmJtRBG5L2JDsLSPXbJeap4ypdoEATorJU1mVOoTNeyl3hOsXz5IxC4rJ2BTEzu/KVaK+ATai4qV/CYndaYodnXj2zlUxU5dO+k94AtwKQUmctoyxab42DWa6Yqdai+JuFX7t59zalugSatpmpcsU6wCVaZVO3TVPIpdVkkxyRSrNqk3qWJn9qU43nhU7I499lhcd911+PGPf4wjjjgCl112Gb72ta/h3HPPtdb7yU9+gjiOvcRs6dKl2LRpk/6+evVqvP3tb8ehhx6Kt771rZg+fTruvvtuzJw5c0SPJw3l7FUKjAaqBxwAhCGinTvR2LgRlZ6ese7SuAWfWaeBLuaz+7SUJO1iYw7FbtWWPtz0xHq8ZJ8p1u8qjx0daA+cMQHrtiemMh6ht8HjY8dNsabyhBsVCwDd1TK29tWxpWVG7qCmWOG8ckLMD7GdPHbcbHfbU+tRb8Y4/cVJCqBVWxLlZsXmfAqOqTxhcpQ1hUGf56drUMWuGZNC6TZ5DZkiI5UUU+fuY6cejCfX7sT82UmusjyVJ/L6ffLluwbrDqGniOPYq9hZjvUOoUj+J3ns5HU4pEhnVbouVbGrUMXOELs4jvGje1biqH2n4oi9p7gNwExesjJ30OtFTbG+4Ak+mWpEkUNcm5E5T1S1KocB6oQI8nsIMApfM45Rq7vBD7QfEmhaH3V81CslVbEjPnZxHFtWDQC48bG1TgWMpK9ASHxT1TNHj10y22/rq6PejDKrBI0GzjrrLJx11lmp61xwwQW44IILvMtXsNruP/nJT4aja8OKgtiNE4TVKqr774/a8uUYfPLJgtilgM+s00BflPUR9LHjdWKlQfAN37gTW/vqDrFTisvT6014fKUUWnnsaHObfKbY1Dx25kWt6s5OaAVQbOlNTFadlVKqPxUfqNrzsZPbAhLC/ZEfPYhmFONvnz8N3dWyyb2Wo55lsm+38oROz0JNsazvdICiip0xxSbbqbFPMjVxc+0FJ9npFvJUnsib7oSf450DDe0jKIGuzn2m6DWI44RcGiWLKHYpKm5a3wC/YtdoGtNmRznE3lqxM0T+b6u3419+9RgW7jcV133kRM8+bZN50mdz3IONJjrKJYuk+UqK2S4e9mQqUezcABMePAEk90hi4revKeVPeU2xvvuhWgotIsdNxx05FDsgOWbqprFzoI5F1z6EainEV976Eqcv5rzKmQnUMfKUTlt7a+iZnB5dWmD4MPYUuoBGV8tRs+/hh8e2I+McUgi/D3RpI4paL+zk+3CaYl3/G3cdlZLhb7y0U0uxe2qdIXZN2s/IHmQG6k3jP5ai2OmSYswUq0ywXa2UJyp6sLMc5gqeULtsx7yaRgIbUYyBekKwlFnKJNXNl9dMMsWqXVrBE44p1vgKUQf5dhQ7vY4nkEcrInkUuwwey8nVzoGGo8RRUALq1AOO/NeEkhGqLqVBJnYVqz0FSqKoKXbDzkF9zXe0gh92sByRA/UmPvzDB/Cz+1ZpUk6PhZYC29RS0nnwBI1KVaCEJwZT+6PYOb5m5AZPAMY0yfPYUWIXEGJJFVd+Cn2nvBwG1j55QuVqWSZ2VMEFXD+7/lpTBxRtY+edvjvpvSQpdlzd3DROImNfKCiI3ThC11FHAQD6C2KXCp4hPg181k1XH05TLG8raxAEjJKjko4+TYhdFMUWCbEj3+Rapn5TrOxjN8HxsUtX7LhSxw8xjdjxda08cHSgIYEMgGs+9EENymVBsZPMpiZfnTGx0bxkSsGiqUwAExVbt/rfWhbIxG44FTt+X+0YqKcm9aXHnlYyi69rTLH5gye47yhAFTv7d6rEVsshpk+oorMSIo6BtdsGrP1xQvW3Vdvw+8fW4bt/XiYGT1CCtmHHgOUPlhwTSOCBTHK4+wNfrvrHgycA1w+Tnku+TjNKz2MnEUfATnei9kH7K+Vyk8gpP6a0xMJRZPbRb2UmMO8WyQcVGD9+di8UFMRuHKFr4VEAgIG/PYK4ObJ1E/dkUBUni5zRd3O9GYmpL4YDvK08pFEpC+rl/dR6W7GjJIQ3p8xqefLY8ahYZYpV+1c+dlbwhHBueOBBmnmVw1XsZHKuBhr1W27FLjLqpPYJY/509m/J93oztlQfy0Qcxc6gnOZj51Ps8kTF5nUPkIIn0kyxdIAdzFTsZPPybpliW4EkXK1UJFMpT0EQOH52qj2u/tA8bDz/XxzHlhr4yZ//DUdechOeWLtDH08QBJZipuD42LFr4ab/oMET5vcye4aoWVuBqsoWsXN87JLvnUyBqzJi14zsd4Sk2EVx7JxLXlasSZZz9xJKdgc9ip1Jd2K3y9NBFRhZFMRuHKHj4IMRTpiAqK8Pg2OY3HC8g7+A08CT03Lla7jAAwLzkEbloKyc9pdQYhfZAwM3eyqVJo+PXaUUWM7UyhSrfOy2Eh87oyS4/VXj/nD42NEBhpofa9oZXhG7bMUuJgNsOXSTz1rBE6rvWg2KLFMsd5jnZlaV7kQyW/pSKkokgqMhmBQlcFPtzoF66jZ0oObqp+v3FuPe5Vuwva9uRQNLiXDTjoHC52Nnqp2Y+3JKK9BCVdNo6uvCyIfgH0kTS9NdLd3Yi/5WXWDAkBCJbA8yxY4fLj9/duUJQbFL8bGj55TWeOanUPVBqewKNEgoWc9+t3UIxK4RxY7vm6vYme9bhbJwajGdTJQtU6zZF8V4SVL8QkFB7MYRglIJnUcuAAD0P/Tw2HZmHIO/gPOi3ozsnGbDqNi1m84DMASrHkVYu2PAmblThcxnyqL74YqdemlXmSm2kyl2Knijo2wcslNNsR6z4VB97CTVpB1iR82iZStBsa0w0na1j10z1gQ4Iv52qo9OHjup8oRgjqNQKkbaHUEDZdLA79nMVCfMN9Naxq7X3cs2463fuQv//KtH9TVuJ4+dFBXr87EzEbGGsJQY2WoQskah+lZvRloZUs37CturWqbq3pCS+1qR2pJi1zp/VNXWplJy7bmqq1qxS4pBr5OnpBiPUq2UQotMNqPYChISFbvIVez480XPtUpcrvZDyaN61yRR04WP3XhDQezGGQo/u2ykpWng4D529Ptw5gvMG81GoUyicewWc29GsRXB6RtkrOCJOjfFJsvcWrEJoZs2oWKtTxU70RTr8bFTObMkHysFp1aszxTbZMETOYrC07YqNN0JI6J0XzoqNjKqTyOKLcWkSXwyTR47daxEBcw0xSb/0+437vuXtZ7CzoySYnT9tKhYAFjVikjdsGOQmKCRm9hJy7N87OiEg6dVMVGc8gSi1og0GdMqXkPuo9q/q9jRe4/72LE+k4mS6od07XU9YVbCMBBNsXE+U6xD7AKrvbw+djzNkKPYkeVbWqZYVaWGBnSp9w/1rwPMeaDP5IkHT8e+07qc/hQYORTpTsYZOlu17ApTrB/tpTsxnxsR87HL55efC656lb0NNa9wMyo1KUmKnTIL0d8dxc4TPKGUwp5JdvqBzOAJx8dOEccAtWaGYsfOh+WjRs7d4O4qdmFoJZ+lZtqkXXtbmseOm9/q1BSbGhWb/OcO7gq5Kk8wXzEf+H2WReyoMuqaYu111XJa6YSaYqPW+eR5zxSkaiJKsQPsbSVTLPflU+TZV8e23oxNgmJ1v3h8k9U7Qylr+pgENwC1j5hprIrMVMsh+utNO4+dpdh5omJJWyG5j+iz704Qk/+dLC+dyglXCgNtEs70sROIXZpipyp3dFVL2DnYsMijulf4Pc9NsftN68aP3v8ypy8FRhaFYjfO0HFQkgOrtmxZUYHCA8sUm+WTRAfqpm2uyFJHBupNvP27d+MrNz+d2SdOXNpR7AA4DvCWKTYSfOwaysfOY0oCDZ6wfezU7L9nUoe1fgdNd5Ki2JlUK8nvlbJbP5WDnw9KAugibYoVnLR9oAM/zWOn2qb7ViZUSg6okz4PilBf3Tx2rgoY+kyxui85FLsMHqvWU+Rc+aN517d87NIVO6oCG4d/sLQa2X2jUIod35bWiVXgkwpet9e0o65dZF37hLjIHVT3VZim2PHKE+xaKJVRkSbLFCuW1Wo9L+z4rGONefCEvU8TPOH62CXtmPXotjS3nEJDMMVyxY6eaxUVO0GXhTOEX00aea1uY4qNrP4VGF0UxG6cobrvvkC5jKivD43168e6O+MStTZMsVYh72bcVlTsfSu24K5lm/Ffty4RC5xTtONvptBNFDtp0LWjYv2DsOmDTXTr2nRUYqbY5PNMRuwsU6yo2Jn90H1XSrZCIYEvoj5qVrqThp3uJI9i1yCDaxDYTuX83DUj83uyLTHnRfa61HSvFTuhVqxJUiv3L0/libymWHVdprYCDSiR4aoOwPPYyUEIfDlViANSUoz2U4Jkilfl1ZJ23etcKQspQJgp1s0fB/27lXYmjq13A32+tGIXuoRIgbal1EkKpTJaplh1f7DKE7T/EvFXvCuOY2tSl9sU21IFKRmmZFwsKRbHjh8k90mkxG97K4+dmoBK5ulSiRO75L86lz73hAIji4LYjTMElUpC7pCodgVc7I4ptp2oWDooPbV2Z8qa7iCZR7GjKlp/zTU3Uef/PKZYwI5q0z525cAyzXRVhmiKdXzsWu0ThcKnSuUOnmB5ydJytCnUmXpATYVJ2hhY35N9tvpBKgTQFDOAO1gCnsoTgmpDwaMXJeT2sWstn9RZARcIu6QyUOR4uPrpIy5UmQmDwBq88+Tio1CRrkm75vc6cRNQ0METjNA12X1Fn7V+VrVBtTttQhX3/vNrcNaRcwCY+0rd3xLZtt8rQooh5hfYjE2ew/Q8dnFrn6Yt2xSbUnmi1SWH2JVtYheTd0QpDFCWiB2JIlbg94QUAKMIchS594zjY8feHz4Vu8DIoiB24xDVgw4EAAwuHZ/EbtWWPtz8xPoxMxVbeeza8Emqk1qgebalSx9atTVjP/b3PIpdpRRqkwk3xVLFzh5ok+WDQh47wJA5ms/Ll6C4ZzJX7EInB5d0TFqxa40BFdK277Cl1Bp62W6mO1FtUb8jBW6Kjcg5BVopMwiBoKQhmQgkn/P42PkGMW4almBUKt9Rqj4l65VLASZWbRdpidhZPnYZwRMDJNLaHBPyK3ZiVKys2Om8g7SAPFfsyMmwJgLkcx9Tu7QSWAowsaOsSYlR7ND6b9/nPP+dpJJrxU6ZYi3FzqzHI6e1jx0ldoRY9ucInnDSnYQqmMfcj4ZABqIpthm70appip2C2jc1PfN+8ONS94IvUrzAyKIgduMQHfMSYldbPj6J3T/+4hF84H/ux+PP7RiT/Vum2AwCRZc2yUCdZ1s6gDy0clvudYFsNRBIZrtqxusSO1shU+2riFY1CHOioMxhdGZeKYVafewoG5LXWSlZA29nuUSUBOEYHR87m1AB8uCerMv66VPsGLFL/ILS2Y7aJ/c7Um3zpMOAIZNN0n4zsol/g3w3PnZC5YksHzvWHwlSMmUJqj/lMLCuHQB0VtMVO1qYnipSigSYpNckqW4YWKQlzY+Sk4JqObSUYlGxo6ZYj2LHj8NO8WMHHtS0X6ki+cn/QSd4AvpYAZfgxHDv2cGGmSip/qljEhU7PclCa5/CscZ28AS/P9RXnqBY9YHmSKSTv4qg2EVR7BynVE2DQ5FjPkkChOCJ1m7VsRe8bmxQRMWOQ1QPbCl2y5aPcU9kqNqim3vHJjeRnccufV2u2FFFJktwpG0/vGpb7v1I3yVUy0FCRur2AKW2l/LYdVVL2DXYEH3sAEM4qE9gtRRiSlcFn/+7w7FXd9UyVc6c1KEjKzsrpdS6oNy3zhADoth5OJgiCiqKr8HUEQU18DTZb+VSiB/8dQV2DTaw6JSDxWMuM78j1Ud6KDwHH3Uob0YxIjIQUZ/MNB+7LFMsT0shod10J2EYJBGn2wfMfsh6v390Le5dsQVnHTnX2n6w0UR3tWz1o1oKUW82tVmOKsS0Viwg+14qcNI3saPMroX9LAK2Kc/klrOPNVk/0uZIem+o2sJJ34jvXuueVIqSNsWy4Amj3LoTM19ScEUa6T2cVitW8rGjptjemn0MFOpYqSmWBrRQ5ZESSInYSZOktHQnCrpCThwDsX2Plx0fO9sMXZhixwYFsRuH6DhwHgCgtnTpGPdEhnqBDGet1XZQayMq1vKxa0ZtmWLp8uWberG1t4a9JlTFdflMV5r5lsPAGvzKYahfwNzHjprD6GxczZ7VIMP3U9eKHSF2rYHoPSfOc/rUM6kDyzb2AmAlxYT+0+z+gBn4quTlnqhnrnJEU6M0ozi3Ypcca4SOcoxLb3gCzSjGucfvh6nd5joYU6w9aKt2xMoT5PCUUtWIIsfsyNUWRRQkNSkrjx0/VgpfMl4OmgyZK3a06Q//6EG9HsVAPUJ31d5PtRyit9Ykfpt2KTUrEW4bPnYTOkpetbKh3QSESFF1LmL3HAN+UywNnqgys3ytYScXVv2iOfEoeCUHwKjkqm1KBq08dp6oWNsUm/xvNKPUYDATPGGIGiVt1KRLCSQnXKotp1YsLykm3H/UFBuyxXwyoyYx6v3jm+wUGFkUpthxCKXYNTZuRHNnutP+WEC9QPL4kY0E2vGxs6JiIx4hmZ/YAemqHe+GRIz4LLpSMj5tjimWmF9jojopPyqlWrqm2NZARdINpL1caQBFZ6VkKQkcXKmTFDvfOVVKhElmTImdWa/WdIndYKOJWsMUcufRnWmmWDePnU1KAePj1Iwg+NjZpM34NLmKY1ZUbBrotU6DGofDMMBERuykZ2Fbv50ORVJ6FfE3y+xjsmqrpjwzTSY3TaiWERAdkW4qBk8wvzeaqkUyfQO20h3FJnhCuRtoxY6ZYs2+YC1XSMyOyWd17APMx46SIiuPHVN1peAJtf9dg/Zzz88vf+4Bmdjx1DxSVCytsqLglhRzr2+3jop1n+8KC55QfDIrUrzAyKIgduMQpUmTUJ6bRHNt/dG1Y9wbF+rRzmNuHAnYTs7p69LF9aYd6ebr/nPb+tFXazjL1+8YkDeAO9OWlA3u0FwpBXrg4dGf1DE7TbFzTLGR7WMnmWQoaMqTjnJoXsySKZaRIq0ekbe3zwdLrasGxbzpToAkfxi95nxw0gEFXlOsqw7S89av1c/I8i1sNO0gAroPSXH0OYq3pdi1lq/Y1Cv6FjbJvmjy36Tt1vEQ014387uTlF5N7JSPnaBUppnoFdQ9d0jPRHRXSzh+3jR7YI/ddctCVKyU5sSn2FmpQqLYMcWqiN4sUywnOPSZUyRJHbo6X/Q+tPznuGIXu+uoz7sGeT1Wj2JHrmPZUjnd/voUu2acp6SYYIq18tjZy9wExUqxsydEBUYXBbEbp5h54UcBABu//vVxV16MD+yjDZ4hPg2WM3yTpztxt92wYwAnffF2vOfq+1xH5rT9OC9kd52qkGS0rE2x+fLYmeAJmdhpxY6ZpHygSYqz8thxM6ZaoxQGYrQoheqmGnDrAjGi/ab7H2zY5ipuPnLTnZhlTvBE0z4GwCY7XNHl/lFSeS2qokngRFMCJZx/eWYTTv7PP+Lff/ekux7x5+OmWNWG8oGl/VWg1SUU1DWxTbFKZfIfN4dadsCMCXjwc6/Fv73xCK+PnVJZK6R/TlQs87Hj+wFYupPYDZ7wKXY06ABw76kojvUNzp8hkdh5omLppIJeCUPsWMUZdno1sSvLip15XknAS+CmIVH9yao8IUXFdhNTrJPuhBHIIDDHnvSlIHZjgYLYjVNMedPZmPy61wHNJjZ//6qx7o4F9WznKZs1Ehhsg9hZil1kRz1K6sOqrf1oRDFWbelzBuG0XTlRscIAWHUUu1C/GKXKE1IeO+VEzfPYqcGE+9hVhNJCFJZiVwlJRnxBsYtN3wAzkASBq1K423LFjiowZj1eeQJIiFc9TbFj6g9NUhxFtsogKXZ1EjxhkYnIzWOnfewsU7K9Dgcd23xRr5Rwrtic+Dw+u7nPWc8OnpB97LaQoCZutn5w5Vb87P5V1v1ZLaWbYgGbQPjQIARb3ac+Hzuj2BGlK29ULDmFPAeco9i1CI6v8oTfx474hbJnqENNTkhdWl8eO3rv2ape8p9fH35/SHnsLDJMomupX6SYoDhyTa1plScUdILiyJ2YZJUUKwS7sUERPDFOEQQBJp91Fnb87neoP/fcWHfHgnq2s6o+jBTai4o1nxPFTl5G11HL2ol05W1JygYfICqlUPuo9KUodjGZKVNTLB0EOsohGjVDgGg+rzRM7TbmvOzKE7baZZl+wgCDsP2irG3VIFly1Q4xKjZFseOF3rX6wxzx1WAn+VVKl7IZxwgtMhHpgTVNuZJM0hQ+PzMKTTjJ5EO6h4wpFpjsmGKTZZTY8QS0//KrxwAAHW87Sv9mfOzk4Ilkf9mmWJNjz9znyj+P+qwBso8dT7VDzYJSQmiAleOKDGH0KnYsj506HD5ZoP3No9j5omLpvSeZYvl+nQTFSrGjwRNles5a2zGVVTTFRlFmVKxI7IhiF7DFPI9diU0wC8VubFAoduMY5ZkzASRBFOMJilCMXYLidqJiqfnHVmSk/lNfJ748bU98cJcGQD7wV0rmBeyUFCMDC3WMVsRusGGTVJ2CofVjnZmkfKDkwM5j5ycV2hRPSI9UkYGCD5K2AkOInRAVO9iwFTvXFNsiFB4lQ9qXRNJ5tK5dUiz5TV0vMSo2h4+dV7EjZve01CftmmJ5UmKF1Vv79WdFrhQJpLWJVd/NfeGX7BrCdUjasP3Z6Lq2WTH5LyVrbnjuF4okKtYu+6WjYlWUppPHzlUH1e9cZVbgwRNBYAfI0Dx2VlfJaVHn0/Vx8xE74mMn3OdxHOvn0ZfHrhlJCYpZ5Qkx3UlaguKi8sR4REHsxjHKPT0AgMamTYib8gt6LEAJx1igHR87urjOfOyk/isCQV+UYmMMNHEs/W5vb38th6HxsavbA5Kd580cZxdR7OixqFqwqj5s3uCJQ2ZN0p8rpYAoM/5j5OZMGnnr97Fr9YcRULoM8BC7emQNgFmmWMBOZ2Fd86bdd6udiKVGiVzlygRPuD5f/pJiORQ7ooaqc59GrsMgsEre0bbTTLEKe7XSxZRCQ8ppgmK/Uin3H6BBLMzvivUPMEE+dF1uirUUu6Z9XSQkJcVkxW6QmWJ5STGJUBm/UPt41DPK/fYUqL+prdjB+SyZgO1+JP99UbEm4MRW0MXKE1FklRwEkmfLXsc9t4bYuf1zfeyS/+o6FOlOxgaFKXYcozx9WvKkRBGaW7ZoBW+sEcNvKhoN1NpJd0LYFFVgkm3d9amvU56ACL6sUgodNU2BKw2Vcqj9ZZQTeEclRK0ZOT5cpvJEi9g1bJKqzUNcscsgdlO6Krjz06egWgot5U1SZswgmPxXu09ynbmETdq2w0NcFQYFU+xAI8PHTjDFUlObRey06uj2MY6Zw34UQ1nAnJJi5PrQ8yAhj4+dlWal6UYG6/Usxc42xaq2t1rETp4Q6koaQUD8Mw3p5UEjaTWEeb/lpLWx9Swqczo1KwZEZQXs4/dNBCiimJbRswkpJ2Eh2xc/rpgs40FP3BTLg2aojx3tqlR5wqn8wK0EgmJXESYwlIiqFDU8b6YUFetTvym6KmW9D36H+6Ji1TNZCHZjg0KxG8cIymWUpk8HML7MseoFMkaWWEu9yQrgsHzsWEkx2RTrRgamra/7oX3I/KZY3l4ldE2xSnmjL9yYzJRpVCxtjmfD507kadhnr270TE7y2YVEAXCOkfj80eNJovDSB37jiO43ZdJ+c8WulqLYiVUMLFMs6QfzE/QdY/I50tdREyEyaK/bPoDF63bmKCmWrdjRAdeUPfMTOyl4Qit2fdnEzpASd3BOJjXJZxM8oZalELvIvQ4AJR90XTcqlrsx2NHM2c88DZ5wfOycdCetPnnM3nRi5zPFalWKK3Y6j11kkVkrQTHrl9mvPJmkPnZlYQJD0yOp+40T7ETRTPex4xO6Kgnw4pMkQDK7J//V/VyYYscGBbEb5xiPfnbq2R6r4An6MsqsrUmW01qZgM8UawZVvjhNsVP9UOZAOfjA/l4ukcoTmtjZZh51DNoUqxIU15tW/zuYisCVi7xIG8C5f5rava30pSt2coJigdiR3wYbkaUk1HjwhHCsVMmQFbvse9fOY2cPls0oxju/fw/O+vqfsXnXIABz7iSoAS9NbVJQ11B6vpqaPPl97PKYYqmCxQdnGqzjBE+kPAQ+k7Qm2VQNlfLYMTeGhrA+4H/vUMWuWkqekxLLQad9BpkplqujNGcbj2bnPnb8ePNExYYexY6/N9Q59Sl2YuWJ1mKePJiaqhXckmL2chopT4NqFNxasfbErTDFjg0KYjfOUZ45A8B4I3ay+WK0YCl2bfjYNZp2XinZz8ofFZu2J3UuKikEhw/qNEFxHzHFAm7UqGrORMV6TLGsVmwexY4ijaDFjNBZip0QVCBtWxGCJ8R0J1SxazQtJ29HsROUIlrFwMpl6FFoJCSmNHuAokEiK7f0od6MsWFnQuzS1Ant5J6yL308KjJbmhyo/gSBGxUbCcTOEzxBE8iWOAGIaYRl8lspRYnWbQomcYCSWrp/12zL1WLfJMwXMBVFxsdUKcO+qE1uinXSFQnPlv7OIrs5d/FFxUqVJ7JMsZJqKJZhE8g4j8KntWKVAphVUqyzUhKTIOtjZe8XN3gCBcYABbEb5xiXil3r//hIUJy+LiVT9YhHxbrr235G/rY41LpasUtRvBSSPHZcsWsFQWQFTzSadlSsUsJ05Yl8UbEcab5UPFqTDiRZip0mvkKqiKx0JwMZpljJt8tWMsy6WaZYq93IJThq0I5icx/6fK0oeEJcDjuRdmviJCp20Pvy5bHb2muqGXDneAVay5OTH6pWc8UutaSYdpiXB3srQbHgA8rVYtvfMXsyRxMUK19On3oY6n21+sOOiyrK3E9VqeOqG2mKXUxOv6jYcVMsu1x6H0Ggn2U7RYw6DinQx72uahI0oeXSwaNyeXBFJ1XsIvf+9Zlii3QnY4uC2I1zqMjY+oYNo7bPTbsG8dZv34Wf379KXO6b5Y4WbP+z9hQ72mVpgGgQYuekO0nZlY6KLbmDmF6H/VQphXr2rUhCB1Pekv0ax2jbxy5FsWuZK7OCJzh4vU7aB9V/tYgOJGWiZElQ7XUIil2edCc1q26oHBVrm6jMfu1IV7s/aWg2TR47XnmCYtATHUnBozA5KLEw5NZdjyp2blRsS7HL4WNHTbH8mKgvFU934lNk6TI+2EMgtXVhXW7ute4RarpPmTxwpTqvYudLM0LbUuCTJX7+qL+p5WNH1uFRsapNX+7MUhiQ+rduVCzNf6huQ95vqthN6JCJHc9D2Vku2aZYdk96gycKH7sxRUHsxjnGQrH7/p3Lce+KLfjULx4Rlxsfu+y2dg7UhzXfXaMZeU00Sd9i7BwwioXlsO1ExQrELtUUm6bY2URKGpSlcjzc0VwykViKXYWaYs12XOmrsQEuL3yKHe06T3eSVJ5wCZu0PY/eTZZlETteUszeh5Q6I6AqQ+Re8zy3JFXsdB47gdiZ5LdpptjW/r1mRFexywqeKJdCKw1GjORc5omKpSqjGzxBIiyVCTqHYqeeHTkqllWBaah1hQTFrZ3TSYKUXoYjimNNVNR95qhpOaNiKYHlJk1O7Dh5oXnsfD523BTb4VH66TOmiF21LNznRJnWptiSe13VdVcuHVkJijsrJa+/KiAodmwCIFQ2KzAKKE77OMdYEDtp8KJQj3aWYrdk/U4suOQmfPp/ZYI4FGRFkX3qF49gwSU34dHV21t9tVUCi9gJ5MsKnmDL0w5XDUblFF8kPnjQiDMFRdAazFSp+t3d4ctjZwcl6KjYNk2xvgTL9DsPnrAVO/+gC9CSYnKUo1aryP4G63aNyzofjFIKyjvBE2372CWf0xQ71bc0cSJwEkW4++Ltpd1D6lCpOTaKY+wYaFht8VJ1CoMpwRO8ikGyv2wfO3+C4uQ/TxYOMFOsk8fOXT+tD1JJMSeBricq1vFtI/tzKk+U7PQnfsXO9oOl94c6r9p0XJEnhPQZqwrHRF0geFJpPqmjwRNK7a0xH0yuuFumWGGy65rd7XYKxW5sMKbE7oorrsCxxx6LSZMmoaenB2effTYWL15srXPyySe3ytKYvw996EPWOitXrsTrX/96dHd3o6enB5/61KfQaNjFlfdUVMaA2M2Z0qU/U/VLgae78OEbtz8DAPjZ/auHrW/cZ4jziF88kOzrO39a6iznap+o2OmSYlK6E3+/tI+d8sESHd/t7+UwcCLXJFMsnY1blSeIg7IiiE6t2LajYu3BdddgAzc+tg59g3bpJsAuOq4HGG9JMbT6kzMqliwfaDRz+djZTuWmbYkg5DLFRi7BkRRQX6JaCtofcV/UTJmi2FFTLAC89Zh9db3fKLZz2KnfJFAfOzndiU0ScuWx0+4IPh87d/+WXyTz0+RVQHT/UlRP7luapdhptwLlAyoEAflKivHj0/sgOR3p5baqU7BbpepT7PQzbnzsysJ9Tl0lTAS3S+wU4epWxC4jeKKjXNKqG92HghsoY78HCmI3NhjTBMV33HEHFi1ahGOPPRaNRgOf/exncdppp+GJJ57AhAkT9Hof+MAHcOmll+rv3d3d+nOz2cTrX/96zJ49G3/961+xdu1anHfeeahUKrj88stH9XhGAkqxa27chDiOrZfDSEGRBwBYs60f82fzRKjJ/6yI1C1skBkOuM7Gch908lamEkgmRQodkRhLWeD9x8sHBtnHjpkxJMWu4uaxo8fYXTGPrEplkWSal6NiO4YcPJF8/+i1D+L2xRtxzkv3cY6DqlntKna+hLNKSUpV7HJExVITlXXN2wmesNKd2P8pfGkvKCRyQ2GTz2zFTpGgT55+KM592X444YrbEMex5V+XBkVGw9A1nUokwed7SaGDWNh54ImHk3UFlTW0z1Hbil1M8tjl9LEzZl9zD9WbdiohXx473qYCTYnDVTTfNpUMH7swMPutWr6k5jjcSQhTYiOToHiCxxTL0510VkKifgMhc0fx+djVtWKHAmOAMSV2N954o/X9mmuuQU9PDx544AGcdNJJ+vfu7m7Mnj1bbOOmm27CE088gVtuuQWzZs3CUUcdhcsuuwyf/vSncckll6BarY7oMYw0Si1iF9fraG7bhvJee434PunLZc3WfsyfPVlcnmWKHQli5yp2ch8md6ps6eY3V7Fzt8uTL0uCTsArBAco8OaqJI+dAs9Hxz93EdLdV09UaVpCSCcozllSjINHJt6+OFGK//dBo7q6PnZBao1ZwBy7VuyiGP9z1wrUmzFmt5IjA8Y0xIMnBlN87ETFjhyHFTyhSWn2tW0QU5oasKSM/mpwTJt0qUXSfmOmKmpTrORjxxQ72jdJsfOBRi1Kih0nJLtTUsyYPd390wTFXNUcSoJiXlKsxBWlUF1He19W8FPd3rcTPFFKJ3ZSHjuuXPF7RaqhnPTPrC8rdq6Z1GeKbZDz0+2JinUUu0rJIub89vXVBdbBEwWzGxOMKx+77dsTv6hp06ZZv//oRz/CjBkzcMQRR+Azn/kM+vr69LK77roLCxYswKxZs/Rvp59+Onbs2IHHH39c3E9Uq6G5a5f56+0dgaMZHoTVKkpTpwIAGhtGxxxLH15aLFwvb/3PqvowMoqd7RNC30N00FS+R1ZJMcfHzh04bYdte7mPDFCFIzXdCWuvXHL9m1TwBN2cqwdq4FC574LAqFW8pFj7xE4eYKjypxZRNSt3VGyrnZ39dfzrrx/HZTc8YalMauDhwROpil2TDMrqOIi/lpQDLZdiR+4XajHnA7kimmmnOi0qlv+WZoo1KUVMHyhJ2SG4TgDA1G5bddcqY+AG8NAauW6tWP+J0/5/7ERIqV7UfSol2xVNscI15IhiyceOm2JhHY9qqsH6k67YycRVQYqK5Zyf30PVsvvc0/dNEhVbsvpoHUfk+oOWBRKrns+JHT7FjvnYlUkeu8g1xfp97OwJUYHRxbipFRtFES666CKceOKJOOKII/Tv73jHO7D//vtj7ty5eOSRR/DpT38aixcvxi9/+UsAwLp16yxSB0B/X7dunbivzd/5LjZ985v6+7r68BOQ4UR55kw0t21L/OwOfdGI74++gFdv7XOWa7+UDNVj80godjxTO+lDb82QvsldldZys249T1QsLe3kBGrIfaLNVLUZxl2P74/msVPoYHUpk7bMdmEAdJZD9Naa2u/NNsW2fOzaKClGoVNOsL5O6ixjcFdyPbmPZRBkm+qMopmst2PA+MDWLTXONUMO1JmPnUdloIMMVTIkp/3cPnaRbd4CkgFzkKzXjo+dNDngZEldQ7FWrCaarmoTx/b9S/GmhXvjVS+aie/fuRx/XrJJV++QomJpn9qpPKEIacWj4tBDl3IP5i4p5rvHInP/+HzsTPCE6lNstSm5FLgJitODJyTFjit0/JSboKIYv31kLb70h6fw9be/1FpfqXq+WrFcsVNtdlVK6Ks10WgaH8QJOngiXbHjwRMBM8X6omL1uSiI3Zhg3BC7RYsW4bHHHsOdd95p/X7BBRfozwsWLMCcOXNw6qmnYunSpTjooIOGtK/pH7wA095zvv7evWYNMH/+kNoaDZRnzsTgkiWjFkCRpdhxU5wP/KWxu6g3Izz+3A6xLwCwvd+oFTyJKODmscsyxfJB0ne4dKAxCWzdlflPFUGxk3ziGhaxC9BZKSXErqZMsRBMscrXqL0XqzbFspMzsaOMTS1ipxbRBMV6QMwZPEHTcPCo1Xozss5VlmIn1R2lA56UM699HzuXgCgo83Ga2SnNx85JtdH0P1/qukimWKkthWopxMmH9uD6vz2X9NlS7NKIXWsfrfsizT1BRTr7/K6sCHVBUU4rKZZHsbN87IScbwANnmi11WqqyfpjKXaekmL8+BSsqFh2Hn3baFNsHOOmJ9ZhxeY+/GmJed9TU2xFIsMCiVT96GwRuyTdSbKSSpOjqqts7aujXAoEHzs7j13Ioru5fybncUW6k7HBuCB2F154IW644Qb86U9/wj777JO67vHHHw8AeOaZZ3DQQQdh9uzZuPfee6111q9fDwBev7ywWgWI712JBGqMR4x2yhNbsXOJnXo/p83eh5vUAcBnf/kofv6AHWFLx/jtfYbYxYx8AMlLLDMqNsUUm6VGASRBsXBucgVPZCp2ga4bqVJZhEFAomLj1v8h5rET0icAwESWViP5n3xPasVm5bFLfleDU2/NKHb8PPMUHZl57DLqjlpkvvWlXR87epm4yqrLc+XxsRNyIXKylMvHTvBNS/osH5cuiaac23XwhKzY8XQVnHRJ4OZMhYCRKMCXVNpWBZueZ9Gv2NEExXa/FdT3gO+r1SQNfFBoN3hCR8WS+5Snu/GbYmN9rHR7GjxRFpTpOPb72FESpyZ+Kr0KkARhnfrlP2JSZwUHzkzGwo5yiMFGhC6Wxy6vj50+7kKxGxOMKZ+O4xgXXnghrrvuOtx2222YN29e5jYPP/wwAGDOnDkAgBNOOAGPPvooNpDKDDfffDMmT56Mww8/fET6PdpQ1SdGj9iZz2u2pfjYpQyO63cM2G3mkUgysHxT4gs5a3IHznhxQtrpAE0VO60qke3rTTuvlNQnK3iC+Yv5joCuViEzbw63pFjgBk9UJMXO7IAmKpV87HREpfZ3alOxI6ZYSrBolQPjp2YGkuyo2OS/Ol56KrhpdaDGiF09q/KElO7EKGTDmccOcAdlmjrEB5owmYOniEkldpHbBzp4cvcBBXV9eFqcUij3W+1bNZ0r3Yng/0e3tX3s/KZYtV6j6V63ZLm8f6ukmCZBsqmQ9jGOYxLV7j6/XPXLJnamz5xs6X4wwtNBTLHqfqDPfehR7KRasao/6ljUOyWKDGmkE8gtfTVs7atj5ZY+HZx2ztH74Lh503Dmgtn6nPFnKdmXrIj6vhcYHYypYrdo0SJce+21+PWvf41JkyZpn7gpU6agq6sLS5cuxbXXXovXve51mD59Oh555BFcfPHFOOmkk3DkkUcCAE477TQcfvjheNe73oUvfvGLWLduHf7lX/4FixYtQkdHx1ge3rBBK3ajVFaMqgpbemvoHWxonwzATXchYR0jdrVmhM7QVaPagXrZXvbGIzCxo4wbH1/nNcVKkbtu5Ql3H3RgdEhKDsWu4lHsYmG2WwnDXKZYrtiplCiK2IWheemrUmK6pmibL1aqzNDgF0pA3eCJwBSJT3FsB+Tjq7NtshQ7f7oTl+xwU2w7lSeaTTeFBN8PQFKHDDGPHSf8xg/QbUebYttW7EKrj3ViipWTLtsqZK4ExUIFENo/q25z6z6leRzTSopZtYV9ptgodnxL/XnszG9R7Eb0UlM0V9V9udv0PkgeO8mUD7hme1NSzPSl3rSf+yktv2Gdygn2feWYYlv9pIqdOo+dZAJJJ1bKveNlB07H5W9aAABYuz2Z3CfXTzY78/74vhcYHYypYnfllVdi+/btOPnkkzFnzhz999Of/hQAUK1Wccstt+C0007D/Pnz8YlPfALnnHMOfvOb3+g2SqUSbrjhBpRKJZxwwgl45zvfifPOO8/Ke7eno9wz2qZY+ztX7dT7OW32vna7Tex29NfxDz95CL97dO3Q+0UGNSm9xo5+aop1B8dGFFmKiWyKtYmgtf8UpUCh7FHspPGwUnaDJzorLvm1fezMS7m/ZtKdqBesUkJ8+bOyYBQAVkyevPwNOWpDsfOY6QCXqPWJil12gmKrPJUw4NH+5RGQ61FEBmbzOx/oDbHzt5VWeYJHEkuRwQo6eKJNHzvVZ3WdBokpNt3Hzjbh5jHF5klQrP0iSYQpVYYA+xlKKyOoEBHFTkoNArimWLWdL11RyKKGy2HgqlKhTHYsd4YMwkNTqOjgGXKfhyHw4VcdhE+e9iKcfdTezr4jQR3kpliax44qdvTZUgFoUg1fWbFLN70W6U7GBmOq2GX5uey777644447MtvZf//98bvf/W64ujXuMNo+dvy6bNgxiBfNmmSWwz/wKKzbbpPBPy3ZhF8//ByWb+rF6xbMGVK/aESgFGm3rd8oTLH+T2f92VGx9RTFzlcrNqam2NAdxGjfKcph4Mz+RcWOzdw7y5KPne3Xw/Ov5QUdKDb3mtjPmkDsqJplnLhlU6A6fJ4DLOmzvY2k2FmRsw2mcGl/LdmsJ5li8/jY2aY0qtjZx5AvQTF0fzj4KVPnQ7pnzOSGtk1MsWTyw5VewAy0tmLnD9hR/fYpdlEUm/qgnpJiitRQUqj9Iq3yWK31hKjYejPW+/KXFENmupNQUOwo4ZHq3HJzsU8FpOsAykfT3q9vG2reVcfAg6b2ndaNC199iHg8Cemyf9tnr6SC0H7Tu3H/s1t1YBJgK3b02e4dbFjHANjqN7/Ds9TLIt3J2GBcBE8USAcldqNRfYLPygcbcu64tMGRK3bbWrnKeILhdqD9i4JAHCgtU6wewM32De5jJ3S/keJj145ix8+hNKBXSqFDErIUu4AodtoUG5jIPVo5I1nW3r1CldCtVn45Suzs/+1UnpBq1/JIvH6u2DWiVMUutfJExP2zbFKaBhUxCGTlscs2xablsXMVu9Y1zKnY0d2q+7daCtEfmfOoro8OniAJZGXFLrL6LQVPrN7ah7/7+p14x/H74VOnzyeKnTy406NJ84vUeezIfbFh5wCOv+JWnHXknNSSYiYaXJliuT+c+m/2G8fmvPISf0Fgk8MysRbofrNbWlLsHBOlxxQLuGUBk+3le0sKdlLrfuTkg3HyoT3oqzXwywfXoEmiYqliR/ejiJ1Uni+OXT9j3/nl2xYYXRTByHsAFLGLBwYQ7do14vvj702eO05HxbYRPKHMpHxQbgc0nUIoKAhi8ITlsM0iJNuMivUdrhQVy9VMvm0YJMfhRsX6feyCIBlodVRszVXstBlviKZYNfA3ohhbiCmWzup5Hjt1LIAbCKCPQUXF5jDFcmI3UG+mpzuR6o5aplhXsctjim025TxknAip5lMVO6ZGUXDFjl9Dq0+Cjx0d89Uz4nPwV76QNPeeGBXbtO+fUCDu9y7fgq19ddzx9MbWNr50J63jpM+iEMnsRsWa9f+2ahs27hzEn5dsSi0pxitP+IInKFGKSPCErNjZ0dZZpliax071NEvJornxaiKxgwjpPldNV8shjtp3qlXVoiHcH3RipSaLUk5ICb7ycXrbgtmNCQpitwcg7OpCOCkxhY6GOZYPPgPMNGZMsf421nEfu1ZC2rrHVJcHdFALiSKjsL3fpNDgKTkAt6RYHLuqY90TiQf4TbE0V5Uvwa+U6gRwTRlSupM6Sz2hiF0vUex45QceIZcX1BxGy1NJip2OGCXmqayoWJ61H3B9GfvY/daIYovsucROmWLlBMVU4WkvKjayyKuC75zm8bGTdssVO3MN3fszEq6rZIrlEwRFWIxiZ0gYJzO0nbQExRt2Jqb6Xa1nWy3jfpSS24S6p33mc9oHANg1aPbhcwGhz3c1I3iC8o+kIoNLNJO+M38zwRTr5LEjkztOtkw79nfqa6hNsTRdiodcGf+3WL8L3WAO0x/6rKjV6KRNp6zx3F/O/jPOxXgzxa5ZswbvfOc7MX36dHR1dWHBggW4//779fLzzz8fQRBYf2eccUZmu9/85jdxwAEHoLOzE8cff7yTgm20URC7PQSjHRlL4VZ7aP1PkT24A7xW7BrZA6oPancJsUs++9KdKFAyFsUuWeOHQAdZbiL0K3bJf1p3k58bvl9TpJybYt1HUhOo1jGrgUYlxqXJS40ZD3pZO6CqiVTqC3Ajjmm6lcw8diWBuDKixtOdAGZgB4Q8dkI0pjXgkdWNj53YTQuNKCak3a/YKaQnKEZrv4Ji18bkwihp8sCr1udpc9R9ae6d9Dx2OniAmS5pXzcqYte6NtS/j4L6aKm2VTMVQRmS8tgp8kjJNscAcfOoeBQ71TdbsSOuAgJRsYld6FXo6DpAKwrf8xw66U4I09OKned8UlATvy+1Cn0n0TQz+l5ous+bpQinsARX4WTtjCNit3XrVpx44omoVCr4/e9/jyeeeAJf/vKXsRerv37GGWdg7dq1+u/HP/5xars//elP8fGPfxyf//zn8eCDD+IlL3kJTj/9dCsF22ij8LHbQ1CeORO1ZcvGRLEb5IpdDj8lrtyoGpa7Z4o1gxrN36QgpjthXeSJk6M4Rom4BFPiyZ36fT6F1MHe50vFv6vanXkUOwXu72QKbZNasY4ptl3FzgzCW3YZYjdomWLt+riU0HqTx2rFzh0latwU27rfuqslPUGg15bnvZMUO3XYURzb0ZWx+T0LNJt/Wh47/XtquhP5vgDcZ4U+I804tl7SsmLntsVN3moQD/VgbiJ5JaLaYD52hnSZdZRit1ORLuU356kfqg6THp9cUgytfZnzotTpZtOv2FHLgk+xC9nxAAnhke4hILmP6G9iVCxX7MgzpCaWjo9djuCJZmSukQ80Z14s3KvJOkZN189hSZUKi51gJEA2kUvgE1NXsfP3fbjQW2tgJ6mRXC2H4nv0C1/4Avbdd19cffXV+jcpd25HR4e3uIGEr3zlK/jABz6A97znPQCAb3/72/jtb3+Lq666Cv/0T//UzqEMGwrFbg+BUeyGn9j9z10r8O6r7tUvxiwfO7U4NacVG6y3D4uPnWCKJV3YIfrY2W3wQBCnTidRCbLUPb5eGPpNsZQUfuMdC/GNd7wUgPtilBIUK/AIxTohumqAVOfdmGK9zYmgZhuq2NWEABo6aPH8Xxy8ViyFL91JpRTqVA2+QA76vSw44nNTLE+unAaa0DotKpbvU0SKYuerFSst08+AFTxBTLE6QS+vZ2orWLSkmBQV66Q7EXwEN7T8aFWeQb9iZ29Lj88uKdZaT/CxU6hHseOTqECjqSs6vYusXNIuUv80R4Fiiib176XrSPugCa6zKk9IwQxqkpamutOat+qdw9enk7UGUez0e0R4J0vqtwRfLV6F0ag88dqv34cFl9yk/751+1Jxveuvvx7HHHMM3vKWt6CnpwcLFy7E9773PWe9P/7xj+jp6cGhhx6KD3/4w9i8ebN337VaDQ888ABe85rX6N/CMMRrXvMa3HXXXbt/cENEodjtIRjJlCf/c9ezeGbDLvxt1TYcf+B0V7HjxC6HKZabMXe0/N/47+3AlHcKjDM66YOk2PGBlCt2fJylfi15gyfoTNlXa5V+ff2COU4SUQUpeEKBJ4utk8S4SqHgedp2xxRr+9jx44nF45ZqxdLkzJJix7dRA3QpDDCxs4z+etMys7m1YluDspU6wwx4YvBEjvmFrdiZ370+dqmmWL9i56Y78Zti1SKHXARJ21qx4z52TLGj/pGSYqdJReu7VDJOmWKBJJoyKypWzQjp8aWVFJP8NZtR7J1QqvumWgqNup2Sxy4IVCkuklePK3bseLSvYhiIJBuwJzk+8yh/LOmER72j6s1sxc5ExfpNsWWBaJbD0PG3tLYh/Ul7hWQlKG7Xx3couPmjx2LuXJPbT3rHAMCyZctw5ZVX4uMf/zg++9nP4r777sPHPvYxVKtVvPvd7waQmGHf/OY3Y968eVi6dCk++9nP4swzz8Rdd92FkuBGsmnTJjSbTcyaNcv6fdasWXjqqaeG8SjbQ0Hs9hDosmIjYLfXM2RNhuzl1MRBiVJaZKF6Uaqag8NqiiXKGCVw2/ulWrF2G66/YH61xKfyNC3CKStX9DslW05JsRRTLDcjGXOZeREbM87um2KpSsbNpXYUXoCyNgm515eeNikq1jHF1kwurUkdZYtAAC7JlFJnqMNOnO3Nuu3ksbPTnbiDO0da9bY0Hzs3eIIqjPa6Uh67pP0AtMqGLyrW8TkLXAUK8Ct2PmK3a7BhopMzfOzUMxYEcNQwup6k2PEAKApF/umxp9UyDYNAl+JS94i0vq0qts5HEKAJWaGkeey0Ypdhvi2VQk3O1f0t+VNyGDN3SpULQphNonBz3WXFLp8plhPnsTDFTqiWrWocPkRRhGOOOQaXX345AGDhwoV47LHH8O1vf1sTu7e97W16/QULFuDII4/EQQcdhD/+8Y849dRTR+YARgCFKXYPQXXffQAAg0uWDHvbvMySm8fO9q9SSDfFJstUKTJlJqWD5VD7WQpdX7Zdgw0W8Sqb3Dix85VzAvKTUBr5VmL94v3xZalXSFPs1DvTmNPMi9/UirWPu90Xqyk5FWNrnyHKUvoWfdwgDuPCoGuXXGtDsQsSxY6DE8G01BnUqRww1zttUqLXjeSoRl/wRHqC4pZSJixLMwvz+9O4I8i+TSZPmUexE8x0aT522vzP1LT+WhM7SUDLzoGGqJzSNrSPnSIXnjqjUYts+RQ7f/CEMuHb5M3ah5XYudVmTNOdcB872xRrlE/SjvNMt9TNZlpUrPseUL+5il0KsWLnjPfN6k8Um5yPJRMEwoORkuX0HHp3n11SbDSYXU7MmTPHqR9/2GGHYeXKld5tDjzwQMyYMQPPPPOMuHzGjBkolUpYv3699fv69evb8tMbbhTEbg9B18KFABJi19y+fVjbVo+1JnhsOfVLsxL8pppik5fShA47NUeybGjEjpo+ePQpj4jVPnasDU4KYsbdGimKnTd4QvT942pf8t8xk+QoKabATbEN4idVcXzsYK2bF2r9Hf311MoiXCHw5e9L1iXtlwLnHHDFqr9mUnFM7HCJnWuKdZUiW8lw79l8Pna+4An5tZnH7C3mTkx5HlxTrGz+Mwql7GOnBlgpr1taVKwOnmDBMRt22umMduUwxeo0JoLCSvdFTeAc9Sg7eKIdxS7pF0nzIUR5WvVsW59LgXuvmXWM6dOnovF7pUwmq+odxat/SKBqaMSume4fUVtNgIt5V3H3FNUfc3z+DjgkPuV8jzVOPPFELF682Prt6aefxv777+/dZvXq1di8eTPmzJGrJVWrVRx99NG49dZb9W9RFOHWW2/FCSecMDwdHwIKYreHoDx9OqqtG7DvoYd2u73bF2/AF298ypoB+3J80WoRdEnawK9eShOq2QNzXhhTrFt5wiV28rFIUbFRFOPLNy3GLU+st0gnVwx8h0tf3l5TrMexmQ8kPv+QpP3kP3d6ppF7qv9S8fo8UDyzt9ZIXS+K7VqxaXns6DXgCV8BVzHorxtTrEjsckTFUuJv1RltS7GT89jtjmIn7TdN+XYmCCmmWMCoYTwIR1eecEyx8vHoyhRMsVP738DM470pplhuhjbBLrK5OIr975a0PHb9dRN0o/cdBpZaJuX/o3VWpShPy8eO+SoCgo8dmeT4Kk84il3JjbZV5zNN9TLBTvCSSBMxb0qcVUqhvockYudLgM3h9afU3/3bjjYuvvhi3H333bj88svxzDPP4Nprr8V3v/tdLFq0CACwa9cufOpTn8Ldd9+NFStW4NZbb8Ub3/hGHHzwwTj99NN1O6eeeiq+8Y1v6O8f//jH8b3vfQ8/+MEP8OSTT+LDH/4went7dZTsWKDwsduD0HX00ag9+yz6H3gQk04+ebfa+sLvn8JT63bitBfP1n482hTL3psDHlNsqo9da2DormbnLcsLtb+kpJg9UKo8V7yfWT52zTjGk+t24Ou3PYMDpndb6lHeBMWUcKr3nFRTE5Acm+0UHSpHn3RuneAJaoplJcWanv1lQe1jIKP0m5UQlZjzpEGXnoqwRSRqZLlTK5YUIpdMsVzxpcl2Fez8XnRfLVU6hUxVSgHqzdhyNrcUO6+PXQqxS6k8kTZB8k0QfAOoqvzR4SFNjgnXo9jR5XR71R/u97hjoK7Ps2POhP2s1jURZySTKEtpxC7LFCulelH7lNLExFSxK7vnlScodtrx+tiZN4abxw5sGzc/Xr0NHzsaJOSaQ5P/9JQmeezs3JcUduogE2jC4asyYr6PH2Z37LHH4rrrrsNnPvMZXHrppZg3bx6+9rWv4dxzzwUAlEolPPLII/jBD36Abdu2Ye7cuTjttNNw2WWXoaOjQ7ezdOlSbNq0SX//P//n/2Djxo3413/9V6xbtw5HHXUUbrzxRiegYjRRELs9CN1HvxTbf/lL9D344G63pWa3A/Wm449mksmGqDUjK4+dXWtVfsHGsanZOEE0peWQSwQo0iWVFHOTCcsDuJu2w1Q12MnIIX/h+RU7V7niZmptGk1R7JTJpxQGiIRzxAdZ7YMTmhexGqB8+8uCaptXG+GII6oK0oCOHIqdM4DJeezCMAme4Kg1I6tmsql44A7a1ERF+5J2B1ZLIerNJvOxc1UbjrRTrdNdSIpdG8SOmv0p+Pl3o2LVvWW3XyL+mRLUbhxTLCsZSBVzX7oTdRv4csZRJ39unqfwvT/UpI23WyLETjbFkqACp1asW1IMsJ+r1KjYyDwjFJzwVJiyCLQbFevPYydd30op1IRPInZSxY6G8L7PLCk2jogdAJx11lk466yzxGVdXV34wx/+kNnGihUrnN8uvPBCXHjhhbvbvWFDYYrdg9D10iT32cAjjyAaHMxYOx00A78aO8x/25zj1IplbaT9PqyKXWszqfIEL1XGj0lBSt2iXvq8JmlWvVezr9ZgGwTgEYB8Hbf8kHkEFTHxFvxmZjHqXK1Nsa3jG7IptrW+75or+H3s3O3ouUgieO3Xjhs8YXwHJcUOYEEuguO7lN8r6Z89iZHQ0fJzbJBkuHT88iYoTjXFwrvfLF9Ga10hShcwxIFGo0t9c4u2pyt2Om1Ihil2K6kr7Pqp2c8ErX4g9ZFG90qQTIf0d96ulAYHIH6JsZ3fzep7KCt29Pz7FbvIekakdeh3/lsjyn6GpSAhJ1BD2J4mWuZ+x4CUZDrfPZ91nAVGBwWx24NQPeAAlKZNQ1yvY3A3c+REmtiZ7Og8OlYlh6XBE7YpVn750kFXVuyG6GNHIku5KZb7XfmUGT4oULPPAEmyCtD8aOrlJB+v2oRG0PFDbHpeunTQUMTEX4vUHlQUIQqICqYGTWOyE5vyIu+LWKWJUPvg1TAo6BhNq1QopKU7ke4fwL6HqEO43g9VTQSVOS3wx+QElAfmofjY+SqSqD764Cspxgdr7tvZwYJwTP41ONv50rcArvnfBE/YxG5bvzGuu0lrk/+a2CllzUMeEsXOf058kw6lMvt89wCmtIWE/LealFREycfObsfuB41CVe9Wfoal+rL8N+1ikDMq1ksihetLrR5SmUe+ja8LrjpqLy943digIHZ7EIIgQMeLXgQAGHxGzq6dF9SJXL0QYvIbAHS11Dbqb5XHFEsHXTl4YmimWLGkWOs3X6BDVroTmo29GcVW9nqd8kGbV9P7VSKzYL5fb7oTaootuYMGhTpmTeKIqUabYpv2NRxqVGwWaLQpNa9K9wQ1h4dB4NTj9JliVR47CWowoqXNRMf42CZx3FQtQZkxfc7vvqjYvL5QHO0odlJJMbpvdf25n5lUIxWwI8zT+s0nLMrHbuakxPdoG0mN4yNs2hSrFdahKXa8eoz53SX4gE3E6eFLapcURSspfj6ySJf5fDSTdey+l8PQWafRdO89DsvlwBeowdrtqpSSSahW7NzzmRZNbB9H+nqjUXmigIuC2O1h6DjoIADA4LLdJHatFzTNf8QJnlHs2ouKpapNd8fwmGLp4ExNsaq/vE3ttsy6KFWesPPfmWVcHfEFT9DB3xcVmxWxRj/7ouACNshSvyGd7qRFRmJCutpB3rxT1ISfKJVpeexI+4GrBpjSScl3K90JMcVWSsYPSal8dmAGJXZq33bqDJ8plg5QitjVSR4yel6ksmh8/xypil0Ky3SU30i+rqp7lh8qWUUN1FKJLZ8CSfdjiF3S/sotfQCAg2ZOAABsayWzDgL3HvIlKJZ84ZL10smuzxSb5mMnfab9MmST+9gxVV2bYs06bh47Q2SNOdXuq/seCByyx+v1SrB97OR7g5Mv5R5jKti45zorP51vPV85swKji4LY7WGoHnQgAKC2dNlutUPTgXB1S/1Xip1tiiWKnYefUX+3LiEv21CIHR38uMoQxVLwhFrGFTu3VqzP7MNNsb6xxoqK9UQ/+nzeKEmg5YokOHnsSBJZOiDViW9YuxPmtOAOCq4QpEXF8qobXNFR94O6V6gpdmKHyShfLYUkrYvtS5j0w1VR4ti+d0weO7uPncQnrYModhIh9/vYiT+3tkerP+0pdjyIwK/EGmKvlotO/6yPNPBFglpkFHJg1ZY+LN/Ui1IY4JWHJKUOVTJriSSqX3hULCdRVIVPJXae98egNsX6FTspKpZXZOB9p+1JeezcKhLmu3ITyAoqoO4lCnrilnpfUdUR4r4cYteabOs8djmCJ/Irdrx//r4XGDkUxG4PQ8eBSrHbPWKngyfgmmC1KVYpdpYpFuRzumJXKQViXrYhETuy4zC0X1408abum8fkJuWx8yWIpYRNasu00epX4E9Q7Es/Qgc3ZT7LDJ5gil1AFDsg3Wk7C/xF7UuYHMW8Rm6LWHhqxUr91/1tbaP2ZVWeIKbYajnU58gQO9NOQG41HTHLomKbbPIiHacido0289gNWbFLM8V6FDufLxNNq0NNkiYq1q32kM/HrtV+HOP2xUlZw6P33wtzp3YCMFGxUgQmVU8B2Scy2UegjyHNx873HHoVO08lihJ5rn21YnlJMTGPXYrp1xfZ6rpkSKbYHJUnyDnzpjth35V7jC5DyN6dNFmygq8LjgKcQWALjA4KYreHoaOl2NVXr96tyFgaPOH62HFTLFG5qFkrg9iVw1CsfToUHzs6EHMzUxSbUjnmN3c7QPKxc1URBV7RwGeKpVGx3PdPQXUjbRBQL9os4iClIqAv2HrTmGXaNYXwF7EU1Qwk9486H1mKHSeZXKnhih1VpSZZpthQE9gsxU7yPUr6nfznty6NIrV97Ny2h+Jjp5ZI91AaifGVFPPVAzUqD1fs7P/m93ymWHpf3/5UQuxOObRHK6rKFCu1pfuqFDutjnES1TrGOBajq31Q94R6tnkffFGxdroTiNvyYJ+S4Afri4oF7LJ/FFIQFb+t6p5rLbWTFjzBy6KpZ1rfM+ydKL0zfC4afFIw3tOdvFBQELs9DKUZMxBOngxEEWpCPp280MoFyUfGo2I7q65iRwcmX2QhTWcwIoqdY4p1FTvVT95DSbHzpm0hgySQne4kCOgMWl7HyV9lqQHpUbHax46/PEPbhJRkmB+iKZbtWzKlAy0fu0j1iyh2KelOePCHgiI2vFoCrzxRLRtTbK1h36tJ+/SzuWb01lD9S1Psqq3JiG3SpupXPrMURVrlibQIXddX00ya7Pbt9bnSVPIodon7QHYeO3VsfbUG/rp0MwDglPkz9fXZphQ7Qf1zfOw8aUnyRsVy8Mljmo8dff4MKTLvQ76tz8cuLXiCXptGbsUupfJEyjOsa/jG/ioXfH/dLcWO58NUkOo5+wK6smvF+vteYORQnPY9DEEQ6ACK2tKhB1BQJ3Ltj6bVDK7Y+Uyxcts0ASnPgA8MNXjCfOb+KImPnRsUkfzPUuzy+9j5qhXQqNiQvGildfj7sRSagACdx87zVPJ0J/R3mvLEzr/WHrPjl8tvimVRsam1Ym1fIZ/pr5MN0Dx4gvrY1bIUO2KisvxCYztLv25b8LGjx0NPudcHMo3YtZqU7qE0EuM16Tu+cjaxDgPmv6kICfe/CtIVu4CR8XuXb8FgI8KcKZ04dNYkragaRTrNFAurj/7gCb97hASesy+/j51LJCWySZ8tKbrYfR7NZ/1eEs673UfXFJvHnYJOYNJyV9L96eAJj4+ddH/7AjiyfPEKxW5sUBC7PRAqgGJ3Up4Ylc4MNryOpiJ2tWZkmW4VfEqXiXrzKXbtm2J58AR9X0Rx7LTJ1UcFR7GLUqJ7GTnyjb+Sr5kbPGHW4VDBBDqPndfHTla89OBLzJQ+028W8ppim5bpx2wnkRRuzvQRic4cip2JWHWJHe26ZdYTroUTPGEpdqYfNcHPyavY5RyAOdKSJfsUO1+uOHrP2k7/rtKUbJcv3YkiLzta1Vn2n96NIHBr+UrBNk6CYu2qwcgOMeen+R0qqO05sUuPiqX9Sv7HxA/TLWpvt5FHsaNEsOZJWSIFOPgmBqnEjpwzf2CNfa5VbkhdeYJFxcrX0HyuClYGhbxBFwVGFgWx2wMxHAEU5r2Z4mNHBnWldNFXQFaC4nIYjlDwhG22iCJ3hs/Nywp8diqZcRXU79rJOqNvIfH9c0uKyQMy4PrW5U13oqC+0ohRnyN1FhxTrIfY8ZQqNCkrBzfF+ogEVwdLYYCOsvGrq5LPpsKGWd/2sTNk3EnyK5iLKamkKrOaCFiKnTfdifizBemZSVOn+H2k1WGPMtKgKXCsVDryuc9S7PQ1Y/tT5k9eGUT2z0r+q/tFB0+wdwO9Zr40IXYfku15MmZOTOjxiaou2Z+vaoZ6tqTKE1I0tAkmkgMg+HmqCKZYhbRzIPmSSpeT9reLpTvJo9jR/k/oKKFaCtFVKblRxIF/uwKjh4LY7YHoOOQQAMDg4sVDbsOYYm31jv6nA60KoKADkz+PnVHs+GwakInddQ+txnlX3WvVnJT6q8lPXlOs2JpBmimW79NniqUvVF5Tk/YRkF/Sqn1FgrMUO9+smNaLNZUudk+x8/nY0fsGAU3BIhG71mqMgHLwfakSbUoVqgimWJ782HyGXs4vm6QYUz8t6uuniJ3kY8eTAKeaYtN87NIUO7asQSYRUvt2Cpxsxa4U5vMN5PtTz7Wr2LnXVtXJ5elOfFGxgHmHcPO81YfW/eKYYlNUJH/whDLFch+71qSLpSKifEa67jyJOF9D8rnzPaq5omJjfzJkuh4ATODBE05UrN+cDiQ+et9+10vxnXcd7U1ZY/br7XqBEURx2vdAdB42HwBQW7ECUV+fszyOY292dsBWASwfO22+TP5XS8ZMo33TyDjjrzxhXpKiYickxLz4p3/Dn57eiK/e/LTYJq+R6Zhic9aK5ZAUHQWaEwzIDp4IAzPrzptYFjCDoU9VUdBmIUetgbV9vRlpn8SRS3fiqxU7vIodYExHdh47ezKStE8+p5j1JIXWUuwImagLissB05OkvIfMmij2VwJVVjhSo2K5Kdaj2Kmv1N9TSn4tmWKlgZy36yh2hFRVBAIptcHrOjulv8g+FHHnATUUnR7FLrVWrED+afUZV7FTbQRWW2mmWLq87omKdZMah5nPvQQdmBKZe0P0sSNtq+AJbS5m7ilpATCqP6+ePwsnvWim21fBLF1g9FEQuz0Q5RkzUJo5A4hjDC5Z4iy/6KcP46h/uxkbdg6I21sJW2OQWrHJb1qICYzipiJjbVOs3D+aJoSrGgAcEkaxfFOv+Dsf0ILANntyc1bMSKrP3JQneMKYYn2KHXTfaF4pqT9iQW5lis3MYycTI/XypGXF0vaXhrxRsTSaOiSKnUTseLSej0j4iJ1W7MQ8dmZ/1uBDFDJOpqTrTckcTaej63WSLp/24tm4/ZMn45OnH2q1kWaK1X0TbqF2SorResn2vm3FNAgCy9RZYuZ+/XsQeE3LgCEgfH/qvcD97NLSnejgCRJcZe/LfFZkQ1L8FdT9wgO0UqNiJcUuMn2T8tgBLqHzqYAKZXaf8sfQCZ4QasX61pWWcZ/XtDYmdLDKE0IeOw56bdJ9SfP3vcDIoSB2eyg6D01Uu4Enn3KW/frh59Bfb+In964St7VLaPl97EJC7AbaMsWaF7es2PmJXZYpVjal0AHYmCaSY0rW9Q0QUYqjNt9nVq3YILDzSln7STXFJn2rlORBVCHwEDvjY2de1MYUK/fZh9x57Jjpp608dl7FTh6QVeRltRSiUrYHo6ykrFIErOTTRvdN1a5BwRQLAPNmTHAIRXr0Iqz+UqQROy4uaiXWcw/YPnZmHcmFAVDHmt1vnykWMIoqIKs9/NhpcBXvi4Iidj7FGABef+Qc7D+9G8fO28v63clj51EUqSnWp6irb9wP1pdChe/HR+z4Nkkeu/TnXoJ0n2eZYrtaip2uPMEVO2HiZSud2RMB8927aoERRHHa91Aoc+zAU09619naShrKQQcXGibPyy2FgVEytGKXyxSblcfOP5B5iZ0wgBvnZxMVqxQdbl6W+pEszw7myFbsDAGkM2hpHdkUa6sC/ug4iMt54t+kxqm9LC8cZSYtjx0hj3ny2HF/JQ7Hx44pdtWyyc2mBiOfXxG9Nzhxkq43VexodLNkiuX9832nGGrlCb5MK3a+4InIKIwSoZHSeeRJp+ELngCAqd2m7JvkE8cjguskgbm0HkBMsSmK3ftfcSDu+NQp2Hevbuv3vHns1O7pPVIO7eTnfDIipTtJi0L1m2Lt9ZN3h3CQSFeC7ahYeZLD+zjBCZ6w7zGZnJvf0oM5GDEuFLsxQUHs9lB0zE+I3eBT/gCKbX3p6hfAfeyS/zTaUSkZymfPMsVGMe5auhn/+YfFlt+ScY4O5eCJFFPsNh8ZFRU7czxq/4rAcbOyVAEjWS87tYJRAeXlYroTz4AszWAVIdORd94XvKx4qZd7mZCroVae4AN4tRSKio6Txy4tKlb7+0Hsv4Jjim31ZWJnRffF9bHzDJzE15F3SSJ2tmJHU8f4B0tfgl0JWlkRJgdtEbtIvo90CTVqiqUJilUQRJuKnfaxcwi/afsTrz0Up87vwWsO68FHTz1EaMOQbMAOrqKwiF0OxU6dAz7RSYuKlUheHNvqfCCQmHKKH6w0EcuaGEiJyrNcMNKWUZcDiUzRc6B87ELWR2ldBdpknvtcoTDFjg3K2asUGI/obBG7gaefRtxsIii5L8AtvT6SZD5zkkf/B1SxU+klyPrNOMZ/3PgU/rZqG048eAZOOGg6AOJjVwrkkmIseIJGNm7LUOx8L2blW6SIHTcrexW7KDvLPS0oL/aNpjtRM+hYPkbRx661TZYpVv3sU2tojrc0f5s0cMJQLSdO3fwc2T49tPKEZIq1yRd3mlfgqVWU7xeNiq2WbHOez8RNzX9pPnZ7dVewc6CBaRM6zH6JadmkO5EG7/ymWB4ZSuErzceX0WfPF0CjiCg9hiBwyT/dLgyTiEypGz6/TjphO2V+D06Z3+M9BuOeoPqYnqAYyKfYmQhVRuTS8thZpM2QYepPGwaACj1zFTs3cj3tmc7rYzfUPHa0hm+aSh9axM5OUMyJXVa6kzzKdJ6+Fxg5FMRuD0X1gAMQdHYi7utDbeVKdMyb56zjU7/oYGH72yX/zWAZ6Jn5QN2Nsm1GQN9gw1meGRXLXiTUNBvHyYyev5wlHxhq9uQDgfaxa62bZorNq9hxz/fr//Ycvv3HpTj1sJ5W30yfnJJipPwWh46KVZUnPC/DLB87av6RSmHlgZtfK1HseFViO48dqRUrmNmNomn3k4MP4upcah+7cqjvHU3sPL5RhvSnm2J/8N7j0DvYxKqtJrq8FIZ6APcNzNJxpEbFslxuFGklxfhEyrcvydTHVSZxO0L4JBcJQ+zs330KuASuVqoaqK7iaT4rQl0Kk0mD9Iz6SGdarVg7CCD5T8m/UexsZdQEOLWeUeucOl0zil1Dfg7dqNjAOwlL81OjfoJpJcXsBMUZwRPCAYUCIZbgS5xdYHRRnPY9FEGphI4DDwQA1DyJirfmMMU2BMVO/ZL42NlO5LZ/npnp0t+1qSXMFxXbX7NJ48ZdnEYYYkRfHDRQgZtiVW9M6hYfscsuX+QzxV7/8HN4Yu0O/Onpjcl6xF/JF82YZtLLqhWrFbsMH7tG0x6o2oFrIpIrEySmn+SzXSvWr9j5iKmCLyp2wd5TAACHz50s5LFT/ZaPg/ZTgV7vF8+dghMOmm4rMCFJBdGWj514WABoWgr3/ORNd0I/u5HR9nZhYJ69NLNhKeOaqJ/58acpaW4bimQn3xsexY5Guitil5YGREoWLLWbncfOTkZOm/NGxWYEE5TZxEA6BPo802fIXS9lwkBUR19gDW+jmwVPcEKfbYr1dsf7HBYYXRSK3R6M6gEHYOCJJ1BbsUJcnid4wv5s/5b42ClTbMvHjrwDmiT/Ex106Iw8Tx67fqYGrts+gDlTuqzfmqRPCuoFFhNypgYzx8fOkw+LHoMPJbIfa9vWdjRyUs1QucIQC/1XyG+KlZerJmkqEB/hyYJkchOJHXPWTkt3wv0LcycobrX5dy+Zi1ceMgNTu6t4/LkdAMw9lOljF8cOmaIKhVYRS/Z9RcuzSe0D/tJuEtQSicKl+th5nlVfyhsFmlvQzmfnU+xCAO5z4CPjafnlfG2o62Ci5mWyHDVjK8q9EgaQ3mSalAr+ahSWYikQMpoZoBQE2mxO+85r7WalO9FqmFLOpf63jlUKyKBo38fOXY/e3xNYHjt33XTFLk9/8qxbYORQKHZ7MKot8+vg8uXi8p2tuo4clmLXdAcOKumn5bFrkvxxkmKX1xQrETtfn61IPz1bNcpKlZliowzFjqqOPvgUO3XqanoQouvaK2v/F+FlqgbeoeaxM4qdISNpCZHTIA2SUmH3KLaDRmjgBgcntT7iWimHFhGl603trgKAN4+dz8dOSndCFQo1cHMTv+tj5/bXUexyDsAcqXnsPIqdO4C6+5MUYCd4IqdilxYVmwXVRsSeF7nCQeu8k3qy/sS9qu/279zE61Msqe8fDZ6wFTu7TUkllPPYtZ7FFB9N9Vslg9ilPcLaxy5K97GjxF/5svp8+rLy2KVVWHF8CQuGMSYoTvseDEXsastXeNeRfHqkgAmApDsh/mDqBa586HiqFGOKNe3rGXkYiL4jDrFjpti1KcROcn6m5tQOFjyhuuVL2xFFsl8YhS4p5vSppdjVzcvbkE1O7LJNsbweJYc6dN+grgbKBlHT2p0w85d2tSxHxVo+dqFNfvk9xwecCR2yoSAMbF9I6TxUmJLmI8w63UkkleVyyRr3Q+NkXo409Jv8OCjR5MgbPJFmipXuCZNGxyzz+QX61BvVbloeuyxw94QsxQ6wfewkhTeNrDpRscJkkG7XJH6YCbFz1+F+sFZJsRQzvfHR9BM7bd4dgimWvgPTfOxo7lDjYye3Kd0LUsWOPH0t0p2MDQpT7B6M6gEHAIDXFAsAuwYbmNRZsX6z1DWL5NnLg8D1sbNMsSTNiG2KNYpdECTm2IF6hM5K8p/7dDiK3Q4/sZOclmlJIK3YRfYx+RS7ZhuKnWuKTb5rP6wwIC9aT/+FF50aNLJm7mm5yIAkzxuQKKaqq+362KltVH8Txc5tg/vYUZLTjGKrj6bUUfL9Hcfth0YzwuZdNfz0fpNEu9TyCxtoEWVJAfPVim3HFNsQcotxouBWIHC64lRsyFMrVuJwaRMLn2Lni4qlfeEqk9RHX2UJ3m8+2PuCkSSorqpr1SDvBw51XNrHzuPjWfJcu6Sv+Xzs6HNtRd0Lih0nX5mKHTPFpim+Rql310n7HaABZOm5MgdIiUmVa9CnMGeWFEsha+2o2AVGDoVitwdDEbvm5s1o7tihf6fP0uZdrndKxMiZ+d1WucIg0EqXrhVLdKtm7DPF2jNyRaomdiQEkyt2POJWUux0MIDwgokiY7pRA7LOF9bazucTlERN5vWxs3/XxI6YW3x57NKIVpm94H0zaV+yWOMHpEhPLPok5gXtY6UUii96J48dWYcTd+q/BACzp3TiH8+Yj332sv0ok0kASRQsnCtFKGoNpthxYhPQftrLpIAIPuC7gRH+68b3KYL0hyNVsfNExfJzI/rYKRN/mmLnIW6mXXl/QwqeaH3nz6q0v1ozeSeUwlBczzINZil2VlSsWaY2o/WEuWKnLpzy+Z09uTNZL4PomKhYf1S1+s1XFYS3JYFGW6epy8qqkGzjklMK2UQuf05bL/leELuxQEHs9mCUJk5AuSdJtaFUu8REZtbZJESY+iLteH3V0FLslCnWtENzwIk+dq0XhBqsJ7fSVmSZYtdt79efr/zjUpz4H7dhdSsdhVjrkUTFdrAExeo/L/+kjyE2s2ofzMDkUeyU2SigM2jZFJuW7iR3HjsPoVDkqtGMhmyKBexByxc8wWtT0ohWTtS9qppAnihhkMiGcg2osShtX/6sOHbN4mriQTfhxM4lbf7BW3/fTR876bI3redN3q+0bSk0JJgqi75I3pLn3vNNJtrxsTO+bMoUKycopvu3ninxeMl7oB3FTiBkDaaESiTmn19/GH56wctwyqE9bpsi6edRsf5jzSJ26SXFWu+bKL3yxKBQxtGr2GWc79QgIWfS6V21wAiiIHZ7KJ7ZsBODjaYxx7YCKPggtqml2C3duEv0k7MVO/u/ncdOLilmomLN7zxPlRqsJ2lil26KpUEftz65Hmu29eP+FVsB2GoWLQmko2JbA07eBMXNKM7tY8eFPR48EQb+qFifsgQAx82bhs5KiKP2nQoAYrAC4I9Q5ANEIzLkfiimEEexEzpt57FLFC6lzPLrae4n/36SdrJ97HzpdyRTpFrOTejGxy6/YiedRl+EqQS1SE0OtvfVsWpLMlmhZm8OpfzuHKhj+abepH8p/lqmv4acppktOXHjLgu+yUQ7UbGc1DZ0YITfd8742MmKcVqC4Ap71ul1spW+5L8VJR3KPnYTO8o4/sDpotolzRl1VYcUFwwduJJzQicvM+eWBjNxDDbcPKRZaWSk/dB+p/Unax8FRhaFj90eiN8/uhYf/tGDOOHA6fjKvHnou/deDLYUO+4vtrl3EN+5Yymu+P1T+D/H7Isv/P2RXsWOR5KGQUAqT6iSYjYRjJQp1oq0tfNUqcF4ok+xaxGB7moJfbWmtVz1RUeeSqbYONZ+fd50J94Exdk+djqtiqPY2f6FYUid7hmxS3nBv/vlB+Adx++Xo6RY8t87ODOVwLe/LNDmkzx2EuEgalnrt85KiFozEoidfOyOn1ho5z0UiR0vcecZzNSmzSh21FOpTBhXdbj/Vx7FLtXJHUZBBIBXfPE27Bxo4N5/PlXfP9VS6Cgrqu/vveY+3Nea3Ei8X/IxNKZ9/6Cslql7r7MSWtdPB084il07xC75z5/ltJqkdRIVK0dpymQNML6qCr7j5+ZStT96qL4Eu9l57Fjbwq2hJmqVzBrR/vuKun6kKnZ1QbHzmWJTTORJP73d8VbFKTC6GFPF7oorrsCxxx6LSZMmoaenB2effTYWL5Zrn8ZxjDPPPBNBEOBXv/qVtSwIAufvJz/5ySgcwdjg2ntXAgDuWraZKHYrALiE4uf3r8YVv38KAPSM35+g2P5vmWLrKtO/3Ce6X/pSBowKMynDx04pelKfdOSpaIo1bbZbUiwx1aX72OmoWMfHzv4eBn6TW5pjM2CrNVkveJ+apJqgKWx218fOFxVLfdfUAKXSKHBTrD52dgkcR/7AznsoKQOK+LmKnTygJNfXbqMhmMe4H1YeU6xj8ktTMkhaio07B7UqvXjdTqPYCfeoWrZ8E6mMIezHUUODQBMcfmz2IJ18+cArD8QZL56Nl+63l9ium9+wnXQn9vNjFEo/WdYTuVKQquwB7vngxMRXeUPdt5YpNuS1YrNVrdQ8dqmmWHtdn7iVHhWb/Kc+r1Kfa/wh8PQbyDbF5umP2Yd31QIjiDE97XfccQcWLVqEu+++GzfffDPq9TpOO+009Pb2Out+7WtfS7XtX3311Vi7dq3+O/vss0ew52MLmsi1uv9+AID66tUAXBPgw6u26c/9gimWV5Kg/4MAJEGxymMnq1tUFVGmLkVWpnYnhK5nclKP0+djp6J3KTFRfVEKjRR2T/PpcR87Bd9AROtE+uAPnnBrLFKfF4o0U6yzP8997vN34r/T8xsM4QnnpliJaNKEqGqxuldcHzu7n3o/rNkwsAd7WbFr3Y9qouEx81KViN7PAPGptBQ7+7PPj9Hqv2OKdddRoNHS9JksBaYOr0R01H1ETWlZlQVUfyXFTu2Tf37rsfvi2+862qrXGwTU/G/vrx3FTgeOtI6lznxwpeMwlSdMYE4nMf/6zOiAlMdOnjSpj5T0uHnsslW0tFqxjch+RqQ2dMStb18pLw3q05uWx05s1+djl0K4gQyfP2fSWSh2Y4ExNcXeeOON1vdrrrkGPT09eOCBB3DSSSfp3x9++GF8+ctfxv333485c+aIbU2dOhWzZ8/Otd+oVkNcM9GiTYFIjmd0k5dvZe5cAEB9zRoAsqK2//RuPLu5D321RCXIMsXSgVi9wNVg7QvgE/PYtV4Qn/+7F+O+FVswa3In/ueuZ7Gjv4EP/M/9eN2C2XjTwn3QV+OKXeS0S1OKKFimWO1jF+rf6H9/rdj8JcV8wRMKvCxQFMWWrxftc579cfjUE/VV+/WQ4xmKjx3tY3oeO3u/asLRX7NvQp+SIBGO/D529iTFF5gRkVQWlTAxFUuKnTX4B65iJ+exSz8ea/vW/xgxHlq5Vf/eV2vqY5B87Ayxs8kHB/8pCAKTx05I2GsIhzxJSPsM7KaPHfPB5X0D7Dx26jxP7ChjoF5r/U7aZwdfYYQxK48drYTDK0/4Lindv3RvOHnshDZ4jjwfCUr1sSOTzrQ8dhLaUexo13yuItK+C1Ps2GBcCaXbt28HAEybNk3/1tfXh3e84x345je/mUrcFi1ahBkzZuC4447DVVddJSYCVdj8ne/i6WOO1X/LXve64TuIUUA3SfCqiF1z61ZEfX2OP9FL9p2K/3jzkQBMAER28IQZiNPy2FFQHzttim29sA6bMxnnnXCAbmvx+p24+Yn1uPinf2v1y6/Yqb4MCrnNqImnzqJiVT99PnZq0JOiJjlKbGBS4NuFgd0/qRxUnqLY7ZpiuWJHifFum2JT89jJip0veMJ56QvkKSvdiXs/yoOZUU7NvVnRUcOCjx1TgFwfO6crrlkzTckg99BDK7fp3/vqTU10pFyLSnGsMT8wX/v0GHwJr+28drwd+TNX14ZSeUJNjHQVmbTgCeJTu//0CQCAg2ZONOulKGZplSekxMYmiTBawRNw1nGOKacpVsqZyNcx/8Vd5bqvkqjY7PWl/XPsTkkxx4ezIHZjgnETPBFFES666CKceOKJOOKII/TvF198MV7+8pfjjW98o3fbSy+9FK9+9avR3d2Nm266CR/5yEewa9cufOxjHxPXn/7BCzDtPefr791r1gDz5w/bsYw0uokptjR5MsJJkxDt3In6c8+hOWtfvezmi0/C/tMnYMXmRJFUAy61hNrqmK1yhYExfWnFzmeKtfz2jBmFwpckWPVrUocbXKF97EjCUgX1cqWJktU+VD99il05DFFvNnPWirX99hQ4iS4FgTVQSlU9cil2GTN318fOHiAsU+wQ3qu0j5VUHztbievUEdRDC56gkbVAerqTQSePnaxi0vxe1XKI3lpTB9r4zHmSw35a+aV6yuDN+9NoRnhk9Tb9e3+tQYinrNhx/yhpP266F/OspCmLac7uVlLa3TDFch+7emrwRPJfK3alAFe8eQEufPXB+MUDq3HP8i2t/viJVVqtWClNirof1LmwfezkY7InAsJybYpNqzxh9y9LqZdAXQ7aTXEk5f+rN2PPxIH2x7+DIt3J+MC4IXaLFi3CY489hjvvvFP/dv311+O2227DQw89lLrt5z73Of154cKF6O3txZe+9CUvsQurVaBa1d9LEybsZu9HF9QUG0UxKnPnYnDxYtSfew5Rzz4AkpfEIbMmAaAmMkXsqGJn2lUvXjpYqve32sZviqWKne1jpyDNBGm/lClWyq2nTG/2zNHsW/lNKQd0xdVUSw6xKwVAvWWqy1Ds1ADE1+JpUoLATpVA+WI7s+msKgBtmWLz2mU8+/crdrE+PtWvLo9i51XVhOPoyAie4FGxvKqFAi21ZOoMt6KGG65pmCs5eXzs1HbqfKeaYlv7enr9LvSSvI19tabxSS272zfj2ImUzW2KFWrF8u++SQJv0w2eyE/s6LUASPBEjnQn5TBAZ6WEg2ZOTAmCsNvgvop+xS75r1W11g9pyY/ldgQ1jvkKSs2o/an3i98Um6KQKZeDISl29vdXz+/BQyu34eUHzUjtQ6ppmC0r0p2MDcaFKfbCCy/EDTfcgNtvvx377LOP/v22227D0qVLMXXqVJTLZZTLycB/zjnn4OSTT/a2d/zxx2P16tUYHHST8z4f0FU1fLy31kBl770BJH52yqxDHyjlEN1fbzpF0aWSRXQgVi+bJlPzOMSoWPaClZzDVb8AkudO8PsbJD43CjSooc4UO+4vyNVCnZsudlPEcPgiXR3FLrQJgWiKzfGey0pU6vOL0qbY3Ux3womdT7HjhM0fFWv3X0FSxXL72NXTFTtqPteqbesekGrF8vvKLSnmU+yob564CgAzsFO1Dmj52LUuly+PHU9VIfVFMsV2tq4HN5vaed34dnKblGRXy2FbTvF0AgYglQhz8yj1faTnOrWkWKpi59+XWi+Pj11a8EbSB6XY+Z97U/UjtL67+5L7QPsRUR87YVRXavoRe08m/bZXPOOI2bj3n1+DEw6a7mzvmwT5+uP7XmB0MKaKXRzH+OhHP4rrrrsOf/zjHzGvVdRe4Z/+6Z/w/ve/3/ptwYIF+OpXv4q/+7u/87b78MMPY6+99kJHR8eI9HusQQe/3sGmCaB47jlN1OhLgkbRDjailHQnirypX0xRe9WujwJZ5l2l2LEXhzRwAZKPnWCKrbvms4C8mLUvXYX72MWt39ngpky2OYIn9MDATbGCjx3tnxSYkhbhpuD3tWH9Yb9Lit1QJsx0m0rZn8cu1usnG6galH5TrN2G5CuYmceOmWL9VS3Mvk1wgn1+7HQnNtnJm6NOrRcE6SYqtf2uwYb1ez9V7ERTrJtcVjovfNdhALzqRTPxlqP3wZsW7u09ljRTrGS2BNqMiIVNPgAaNe8ndoMNm2wBshuGdAxuHjsVnCCbWetN2xRrmR3FsAe/eZf/luZjxxX4NFXYB+1jF9sl/jh++eET8d93LsPHX/si065zz6TtJ+968rupwOhiTIndokWLcO211+LXv/41Jk2ahHXr1gEApkyZgq6uLsyePVsMmNhvv/00CfzNb36D9evX42Uvexk6Oztx88034/LLL8cnP/nJUT2W0QT19do12MBeRLFrCoodLffUX2taSlJTiEClSoxqp8kUMA6rpJgn6s1H7JRiN7HlYxfFJqLUKHaqdqTZTh0iNVW5Pnb277ov6riibB87E2Fp/+4Qu9CNitWf2zCTtB08oXx0mAKRRTZ8yKvYcZ8epRDxqFhTMiudLPGo2PTgCbvEnUts1DUzplh1/5moWLJvRhTy5LEDiH9UxnlWzfFUP721hq5g4gueyGeKdfs7pauCL73lJc66eU2xtEm7zFz+wAmARAQrUywLrpL6poMnPH21gqhyKnY+x/5ayzSvKkDYhFY+pqzgiTI7jjRTrCK4vuc+Pb1I8j8WXCMoDp87GV9561HWb47SmRLZlWb6lvpjvhfMbiwwpsTuyiuvBADHrHr11Vfj/PPPz9VGpVLBN7/5TVx88cWI4xgHH3wwvvKVr+ADH/jAMPd2/ICSq97BBnp0ypPnSPF3s06pZeKqNSJtjlWgil3MFLswMFF/2mfNZ4q1omKVc7QcicqhfOwmd1WsflXDQB9rTTLFstk9QNOdtPoLpdjJ/n5RnD8qlgeOcBNuGNgRdVK+wDzvOX8+q+R/0NoPJ4vqdKclRc0DtV3QIvYl4brFsbt/v4+dvZ6ClGiZEjsxeEL72EWWW4GPNCYVMpLfNLHTfnke0iAodr5TqdWWjAtr1GX7numvNfXkypfHzjXFSu2DrePvT5p/mE+Z2S3FjrhMACZYIa2iBDeP8s+pih07j76i91pVY8ETVKQbaoLiPPnc1CqZeexSbi2dxy5KV+zEdlPM8G4f/PeMb712+lJgeDHmptjd3eaMM87AGWecMVxd2iNACUPvYEObYmvPrUEgKHZAMujWGhH6ak3LbCqRD+qroZ1z1TJPn6wExSqPXchfuD7FLumQ8rEDkpdtFaHjYye9YGgqiA6ex661qMMJ5FCDTWylm6g1I02C+bpc2IsYsSsFqvJJK43KUKNic8zcS2Ggy7mZAcImLkOdLKv9V0qhVXOUwvaxS5ZnRcVmkY+QK3bCuerQtYATkuStakEGPAU14JuciO4xq23z+9gplUdcrOG77PR5FE2xcZzLFOszRUtIIyVScAFgk6p2ctgBdnUEgNSKFWvAJv/FiZz1We4z4I+KdUvYweqPJoCBfA7sfqYTHdfELbTR+lFNModSUowmvm5n8ij3MW0/ZLs2fOyKdCdjg3ERPFGgPVA+sWuwgco+iSm2uXET6oMqgadL7IBk0PUnKLb/B0FgzQiBfAmKVfCDq9hl+dgZYqeUDT0YCIRVvTPUwBcExpzARTg33Yk5LnVsqpYtVyR0ugbWb1exS/4bv0SzTKtbOd663txZnpcrrw5gIvGG9lJVbSsyLL2cI0uxS/53eSpPeAMcBFKRne7ELK8R30qf8lQnF8E1xcqkoVwSfOw8b0pdwD3TFCsv76OKnVRSrOmaYrkimrTP+pVTseOPpE+Zodu0a4rlDv7Ss8z3IxE7+j7xqYmAP72Lz+xcY+pgHn+yzDx2XDVM8bEz+22f2ElRsXmfe1exy0fY0prny4ZS+abA7qM47XsgLMWu1kBp6lQEXV0AgIFnnwXgvgxoZKw/QbFNpgJQ/zJ7mdMnGpDhyVOVZYpVtWRpv/jupGg4NfBVwtAMwKy/Uh67ZLkhkX9/9D44fM5kvOLgGWxdo+5R8HPBTT5DjYrNSncCyIqBMS3FzjrtQLWjiIY33YmTx86XoFgeyF1SgdxRsQAwSO5lN39W61wQ06chdq56yp3h8/vYpastZnv7u8rZ2F9v6Htd8rGT0p3s6G846/H+pQ3uaWoT/Rp41ms/eCL5z1MLSelO+D3sNcV6ngVA8h1zCRvdjrsu2FGx8nmkrzLpOeO/Sa1wU6zvFkoldpo0x7uv2KUSu5z9yfncFBhZFMRuD8euwSaCIED3wqMAAGv/8ysA3Aesk+Sy80XFmkjS5H/iM2Zm23QZh2yKZYqdZ0BQpc66O0r6BaLIId+fVFJM+SBVSqYcEK+iwQdrRTKTwSbZ/i1H74Pf/cMrsd+0bnGfvC+Sjx1AFTtXDd0dU6xPpeADhBQc0A6MKbalKIg+djExLye/+WvFygOOaz4M7Dx2wgEExFw72Ii8VS2kZM3q/lOKMt09V2DayWMnHYvTbza0K3W6jzyP0sQnSXdin8+dA3W3/RymP95n/jnZzr2v+HpDjYqNY/uZyapJmnyX7wfLZEy6UykF3tJ1vmvKEybb5FY+Jt/+FfKUpNPPmSffIO+nBNXsUGrFOuc6p+9cWn8KU+z4QEHs9kBQwtDbSp8w54orUNl3X9Q2bAQgmWKTS92OYhcGgVZVtILm8bKz8thFHsWOvQGNT1bU6mPJJJGNZIVQioarNZut/YWOPw8lqdQUrIMnItc05Mu3Ro89imJXTWQz76FWnvCtQ3+W1AsTUdieEzWHOsfVFMWuSY5fnS8TPMFrxdrr6f0IpCJLsQPssmL+4InkPyUSVZXuRPDXtJLfBm5JMZ8CllU1gPdHQQUK0Sh1KUpUUuykWB8fsZWQFjzhW0ab46mD8iIiZlggvaSYWcd/jaTPUpvqPeQzxeq8ekz55p8pskyxvkANaf8lTSiz98VhfKDT89ilbasguT7wvkrbUfBFBa8bGxTEbg8EfbErYleZNQtzLrsUUSC/xLpbSY3z+NgZMkT9YzJ87CxTrKxA8O9dlRIazUj7uHRVSjrgQqlODrGjuaxaH6lixxVGmg6D7p8mKOa1K32DJD3vPDmx2gdATLFSHrtcip357DU/kd/1fgP73A011YAaGBQRlgZLyVlbJyiu5cxjJ8zuqx4/Kgqa8iRLDaQ5Ecs6uMRVNHmKi7ym2NyKHVtOFTvlBiiaYiOX2EloJxoxLUGxpWJaypWJ+N4txY5eD0GxSyP/PpWMHqukepb0cy2fozpLrWK3LRwQ/MRSwVXs/G1kpcxJu7VM9LfrGpEFX9okeT+0P/kIIO3feMGaNWvwzne+E9OnT0dXVxcWLFiA+++/HwBQr9fx6U9/GgsWLMCECRMwd+5cnHfeeXjuuedS27zkkktaQXPmb/4YlygdNyXFCuQHJTs04WnnYYchanmrhox4UFMsfaBTfeyoKTYjeEJKUMwJAX+RRDEwQAatrmrJmBO1YmfvR/IPMslMQzKIqA0NoaJqkCItVEVQA41PxaA+dlKKFNU3HklMjyMP17KUpFIgRrlKygo/d7trilVEg0dKKlLnS3cy0MgXPCGRCivdiccnUycprkdeNVB9pQqRMsWa8+NRg9owxRrFTl7O+6OgknH31RqO6ZsiEqJiJbSTPywtVYjtJO8+B1EzdvxVM/vWasZV7AQS5plUAXbwlU3y5HX4fnzXtMEqYeTxsQs9+/f9Jl0OHRWrTbHirlLJkZWIOyWPXZ5286u8We2aZz5LyR5NbN26FSeeeCJOOeUU/P73v8fMmTOxZMkS7LXXXgCAvr4+PPjgg/jc5z6Hl7zkJdi6dSv+4R/+AW94wxs0+fPhxS9+MW655Rb9XVXJGisUxG4PhJWgeMAQu9KUKQimJ47/QcP2w6HBE7TWrFSXlSos6sHUCYpzmWJtoqTAB4ooinXgBJAoAdzBnQcsSIRGRdBVyibvntrMVuzcgSGKY0JE1YzdQ+zIbxKx4z52UlqZfJUnyEAVhhiAEMUpmMl4fcqhzpbVdmoAt7P/J6lg4phOAJJlKg1Gf83nY2f3R4pezEp3AvhMsfY6xt+QBk+4v/H1Vb+cXGi7qdjx5VSxU9dtQof7Opby2Elox8fOVux4P5GyLAAQ71YeO0qi0vzOpO8+MzGPaPa16SMyPHjC6oLnPLar2MmmWLvP/lqxch9ou0NKd5JTleZ9y7rXpVRMI4neWsPyO62WQzFy+wtf+AL23XdfXH311fo3Wu1qypQpuPnmm61tvvGNb+C4447DypUrsd9++3n7UC6XxWIKY4XCFLsHwjLF1uwIuVKr1m5Ys+vkUh87S10TzLLUdypkJMWXy9eqPKFKimXIGFEca0f7rkopyZlWsl+2fH+S349SNCqh5GNnXjD0pa8HeZImwBedphMUZ5hi1QDGi54D7fnYWeqRr4wS+d2YkNQx7Z4pVu2/Iih2xoQdW/6LgD9BsTp2fuiSH1JWuhMAJHii6VcDA/tcAMbHsyakO6GmxsS3lKW88fSl7DHzcfDNKbHb2VLdeyZ1OtslJcXymGL59xSVJ4WUpPmXqevRduUJ8jxIiYd9+wfke4//nuVjp+5jTtaddCcCwcoi9IBHsePbCc3woA6fupWauoYs81V48W4b8u9p9wz9nN6+On9BMPSUS+3gtV+/DwsuuUn/fev2peJ6119/PY455hi85S1vQU9PDxYuXIjvfe97qW1v374dQRBg6tSpqestWbIEc+fOxYEHHohzzz0XK1euHOrhDAsKxW4PhG2KZclL990X2A4Eg5zYGVPspE7X/Jp8Tv6rX6hiFzOilNYnnYA0g1g041iTAKUoGj8oT/CE4FujTbGWjx035bLgCbUfwQcrlylWUH20csYCTmg/cpliQ3mgskyxgsnMVJ4YXlOs5MBuKwQtYqd87DzBE1kRc2GQne4EMM77g/XIqwaqr/RcaOIrJChOji1JUF0KpXQnYlcyB2XTH3u5MsUqVEshpnTZvwFtmGLb8G0qC5MCaTuf6Xw40p34Jn1uNL8nKpb2k0XFchyx92ScdvgsvOIQO40RN8WGhJDwdTjylhQz7fjJn/Ir9u0rTw1igFZU8a6e2qe093WegBKzPN96w4WbP3os5s419ZB9rgLLli3DlVdeiY9//OP47Gc/i/vuuw8f+9jHUK1W8e53v9tZf2BgAJ/+9Kfx9re/HZMnT/bu//jjj8c111yDQw89FGvXrsW//du/4ZWvfCUee+wxTJo0afcPcAjYbWLX3LULfXffjeq8eeg46KDh6FOBDEjBEwqlvfdpEbsB63dTx7MpBjoArimWqhjGFCuDqoD1lKLm/DiU2U4RTzXoqBQkvLqD9ELVptgS9bGzjykMbOd0tR9aYcLUirT7mTd4Qg/ykmLXTvAEWadSkl+oUtoX7gy+u6ZYk8dO9k3kpp/Osi9Bcbq5VO83HHpULD9UdQ4bxDFeXX7Jxw5IVLTNvTV0V8u5zVRlzz3DwTefyMyu0ydWRR8rKXhCIi++qGAJvvuIL3POqSJ27VaeICmIdPkun2LHfu+qmPNkTXI8JE9653SUS/juece4+2LPi5Qo2KvYZZhi8/hoGt/YdNU3j48dQIOC8j337ZhibcUuvV3RpD2CmFAtOxMlCVEU4ZhjjsHll18OAFi4cCEee+wxfPvb33aIXb1ex1vf+lbEcaxLn/pw5pln6s9HHnkkjj/+eOy///742c9+hve9732Z/frr0k14+UEzMtdrB20Tu9UXXYzuY47BtHeei2hgACvO+XvUnnsOiGPs/eUvY/Lppw1rBwu4oMoRJ3bBnL2BJ9Yi6O9DHMdCKoqmGDCRfLZ/oyqHMdN6FDvSZl1HxWYQu0hS7NTLVg7WKAkDjzGlGFOsDp0gZmXaH/W5Roit2rcvMo/6F0o+dmq79KhYZzMHlmJXosfrKme0TTXw7S6xM4qdGnjMMlqKjQcuUD9OCm6yNf12v2flsQN4VKynLXUfERJnEtKaiQvF1952FDbuHMTMSR2C2U7simMG94H3r6McoqMcatI2fWJVvF5RbHzs3nPiAZgzpROnHjbLWc8xc6cpdh5SlGznb0Ot237lieQ/zWPnS1bO+0P9gS2lkRIr63nJTzp5XVpTUsys468VS9uRlrN0OYItVrXhS8ei289J0qXE22lI82fksMzTGfe6lDZmPGDOnDk4/PDDrd8OO+ww/O///q/1myJ1zz77LG677bZUtU7C1KlT8aIXvQjPPPNMrvXPv+o+zJ7SibccvQ/OOXofzJ3a1db+JLTtY9d3//3oPuZoAMDOm29BjBiH3nsPZv/zZ7Hp29/e7Q4VyAalFLu4YtfTAwAIGw00SJg2JXaW2VSKiiXRVdy/zJvuhJhqdfoQz8ubbqMGtqr2g7GDJ1xTrDub1ulOQqMw8tq2YSCnO7FqwnrMasYUa36Tgyfs9UXFLgezsxQIjylWUhW0IqVrX2buSoTqY1VQ7BR5pAqtVuxa91gziq3EwL40DFKCVEq+/cETLVNsSvCEMbMZklti5ne+zSsPmYk3v3SfVt+Yj51PsWMqrQ+SWkmDJaZP6BD3kSh2CVGe0lXBBScdhINmThTal4mthNQExcKEgS8berqT2IlATesbYCYLgN/HTtVnBvyEUe5X8l+XL9OmWP85ML9nKXbyvqQ2MkuK5byW7UbDuxU7/Ou2Y4pVi8cbsTvxxBOxePFi67enn34a+++/v/6uSN2SJUtwyy23YPr06W3vZ9euXVi6dCnmzJmTa/27P3sqzjthf/zusXU46Yu3413fvwc3PPKcNTa1i7Zf/dHOnShNmQIA6L3zz5h82mkIu7ow8VWvQq1VzqrAyEJKUKyXhcmLMIwj1Fav0b8b/yeb2EnVEajfkgmekBU0BWWarNMoxAxmkeSQsx2XdaSaMsWmBE+odXXwRKtofbIdMytzHztWEJ625wzCjNwC6elOaKQaPVbefx98UX48DYoCf5H6TI15oY6jIvjYVZipnO6H+kNR1c5fHcL+HoQs3YnXx66l2NX9il3ArgE1xeZRNNvNY5fpY8cUm3IY6MkWoEyxbhsNYopNU8r4lmndSUtQnDaAG8WuTVMsmWzVPamQfPuk5ygtYIHnhMsD/rzo4ImU/kh9ka6bo9hJPnbs3vERuLTHmC5TE5Yh57FL2c72bU5vlx/XeMHFF1+Mu+++G5dffjmeeeYZXHvttfjud7+LRYsWAUhI3d///d/j/vvvx49+9CM0m02sW7cO69atQ61W0+2ceuqp+MY3vqG/f/KTn8Qdd9yBFStW4K9//Sve9KY3oVQq4e1vf3uufk2bUMX7X3kgfv8Pr8SvFp2IA2dMwOd+9RiOv/wWXHL943jiuR1tH2vbptjK7Nnof/hhlKZMwa4/34m9v/JlAEBzxw6E1WrbHSjQPmwfO9vspQhWiBjNbdv0711WSTGzvl1SzA44CAKj+jQZUXL61NqIRiFmKXaAUZfUy0iZUpo5FDv1HhKDJyJ1TGZdmzS0zJak2HjAiJneJzE/KqSlO9F+icK5bdcUW8oRPGEUu/wv6jSYvGpCVKxSVMmx6fQopRBh0MpPWGticsvvxV8dwh2cqR9kHh+7zopHDXSIHjW9ZV+L/HnsQnH/HK6/WmiZGWdM7LDWKYUBmlFSts0QOz+haqege6pil+OctFt5gk500sqnSfu3FTtyb7D1VCqWtkyxrTa4CVOKvHe21W4bvnJh9nfp9njVITNx99LNOG7etNR95Y2KbVexa+eeaS94wpyb8YRjjz0W1113HT7zmc/g0ksvxbx58/C1r30N5557LoAkefH1118PADjqqKOsbW+//XacfPLJAIClS5di06ZNetnq1avx9re/HZs3b8bMmTPxile8AnfffTdmzpzZdh+P2HsKZk7qwNTuKq68Yyl+dv8q/H93P4uX7jcV//6mBXjRrHzBGG0Tu73efR7WfOofEXZ3ozJ3LrqPOw4A0Hff/eh40Yvaba7AEEDJTq0ZodaINGnRIe9xhObWrXo96v+UVVJM+ZIFATEr6pJivj4l/6lil4vYqXJerVV15YlIrhUrRcbZwRO8X+ZY7HxsapB3nbl9JcWygif4TJWqoeo855lN+4InfGV91HiXJylqHmiipvLYCSZset+0cmIjCBIVqrfWtCJjdaqXjPQKYZA3eIKaYtW28jHQtnRARZStbPhSY3AYxc7blNifchgwYmcrdpVSQuyasakVmxa00I6PXRpxSVXshmqKJUl0622aYuk5SlUaQwDN9kyxTroTrZy56zj9VJMp732Rbcp/67H74i3H7GOi2n2m2JzXsm0fuwzXCIo85mnep/FmigWAs846C2eddZa47IADDvD6kFOsWLHC+v6Tn/xkt/tVb0a4+Yn1+Nn9q3Dnkk1YsM8UXPqGF+MNR83F5l01fPmmxfjIjx7ELR9/Va722iZ2097xDnQtOBL1dWsx8eUvR9C6gSv77oOZF/1Du80VGAL4zdc72EC1nKilWh2JIzS3EWJH6nh6S4qxXHXUxy6rpJgiOjR9iGSK/e67jsYvHliNm55YD8AlVoZw2SRTQfSxa5gXs4nAs/sbBr7gCeOfZ9q1+6xNsRnBE2o7bb4WAlPymCfaLSkWegaGoSt2yX+p8oS6PpJiByR+dr21pmiKzSo3FIY5051YeezkwcxRyAJTTaKRS7HjA7O83u742FE1ivvYVUohBuoRU+z8SlnW8Ut9DgW1iX71EfG2TbGtZzIJnkiPmOeTk05yzFaEuMcUmxWwZW3j8S/MU3kiy3yaFpRCkScoIe1a0m3az2OXn9iF1n2R714fb6bY8YrP//oxXP+35xADeNPCvfGZMw/DobONMtc9rYzPvv4wHH/5rbnbHFK6k64FR6BrwREAgLjZxODTT6N74ULte1dgZME5xa7BBvaakBA7xatKcYzm1m2I63U0d+0iptiGTewEB3/qY0d9leI4zoyKpeYA6QVw2otn48SDZ+DFn/8DAGoWaxE7Vs8ztaRY6x2ufezKQoJiUlKsIpCGwYar2A01eIL7W9k+dm2YYtkAb36X1zF57IaL2DHFjhI7HTxBiZ3ZtlNIUpw73UkQoKNUsr5LMD52JI+d46zuKm46Pc4QfOyyFKZsUyxT7EqBrt8MJD52dBVFqpskj12qKdYzGRHXTfGBSlPs1OrtJyhO/kvl+zhovzsroTdfnEOc1MSwjYghfvgmKt7tu7ttOqHn5zaPUj+UdCfJ8uRdU9dKdOauxHZT75khmGILXpcPSzbswiVveDHOOGK299ma1l3Fjz/wstxtth08se7yy7HtF78AkJC6Z991Hpa/+RwsOeXV6L3n3nabKzAEcL8zWn1CObUrxW7VhRfimZNehfL2RL1LM8XyMlxhYL+gojjNFKuCJ0zqER/oi8Gn2PmiYqX8bYqcJVGxNqmi/oLUf0spdFJ/fdGbUQax4z56VmBKG7Np2yQnR4lKqR/aUW3SoNqp6IAWN6ChIQRPAMTkT8qKKfLFBw4pQCFX8IRgis2MuA3N/vOYrDjx8EbFZqSqMNvb37liN2Nih5iPrRkZV4N0Ypef1PvS+qh+KfjOabt57ExULC0plp2gmBLfpN9yHju6XXtRsTI5zONjlxU04xK77P74a8VmbWdPPPNO6Jx7PK9il9E8t1wUSMe1H3gZ3njU3qkTpnIpxMsOzB+h2zax2/mHm9Bx6HwAwK7bb0d99Woc+LvfYtq7z8PGr32t3eYKDAGcU2zeZSJ2jGIXobFtG/ofehhxvY5w6dMAgP6abYptCGkpqF8aHWxoGSkO1Yx6cVfSXhI0qSYndqTyRCzsr2S9YGxiVy6Fum2eoDgIWLoTZYoVFDu/spRuitUES/BDaycq1p/HjqwjDD7tmFbScHBPkk7jkJ7EHGBHxbqKHe2XiowdaOQwxQo+gR3lEPtN68bMSR2Y2CkbFPKYYiUFq8T8KtMuRd6BOUu5Mdu7pHYCI3ZWMfuymRxoU2xK0EKaSZWDJ9KmSBvAD+6ZiFIY4MAZE/yNCzCTrVhPPH3vB2uSwI7XLqNnb6ejYtswxfoCbnzKuLWu8mv1nOe8UdV51smMuBZcGvKgHcWO7iOvOl0Qu3z45u3P4Gf3rXJ+/9l9q3DlH+XyaFlo2xTb3LoV5ZlJluRdd/wJk844HR3z5mHqOedg6//8f0PqRIH2wM2ha7b26886KjaO0HhuLaIdSah0+NxqAPtgoN60zK+SWdbU9gy0Y7xa1xsVq3zsovYUuxpTT0yh9kgkkRIBo8ETxp9HkdRkHV+6E6l2pc9pnHK5hkDsNMFq/ZfM3Hm4ll37Uu4X/Z2XFJPWbwfnv/wAnHnEHMyektQulYgmDZKhPklqMB6oSaZY/0ASBCYf2e//4ZVoxrHXX8quPIFW2/Y6LtEJiNqYQ7FrM49d1mDKN0+iYs3rd9qEKtZsM88xNcU2G0l/85piJd85e99+tSmtTur/e9tCbOurY+akDm/baX2LY3PfeE2x5BCpoglkJFZWil0bkxneBUnJ9AfNuP6nUn/09xzd8t1jWURqqL61zkQwRe30+femtVuYYvPh2ntW4r/efpTz+yGzJuK/fvwQPnxy+xW92lbsSjOmY/CZpYibTey6805MePnLAQBxfz9Qas/3osDQwAnP6q19+rM2+cUxBpcv17+Hq54F0DLFen3sWu2rbYLAVew8fTKm2JZil/KSsEL0WbLYEhl8JRJpmWLZm6NSshMUUwIcBrZZk1e4sPPFsX1qU5JL1KS+SVGx2kyS421nBSv40p0IBDdLtcqLIAg0qUv64BLK9nzs5P74ykJN6CjrVCkSNLFLqRUrqZcuWfPuIre/YlY5KN/2ZWKKndxZRrVsR3Tr0m0kQbGvBiZvP6svkjIlt2Mvq5TCtkkdQHMKxmbi58tjZ5li/cTOm8euHVMsv8YCIfGaYvVkKp3wK0iVJ5w2yTZVjwuGBPd9lbmrZH9tKHZt5bFT76OC2eXCxl2D6JnU6fw+fUIHNuwcFLbIRtvEbuqb3ow1F1+MZX/3BiCAJnb9jzyCjnnzhtSJAu1BkYrJLVPVaqrYkXQnaBjfu9Kzy/VyOxWFaVcNkjEZiB0fO49ip/arTLFpTsz0/aHTnWgfGVUdIHZMzgALnnAGS5qg2CbASUkxNSM3A3Kt4Q40Trsl06bud1MgdoE945d88tp1oi57IgGlqFiHuAzTizVvHjvAEDt6j5kABz/5akddVL4otaa/Vqw02OVNYQK463pNbrvhY9fdOlczJna01iEDe9kodqqySn7FLktR8atNdADPm+g2C3Sy1chS7IR7ScHOYyffS+1ExfrSGqWplgpd1WQ/Pr+oofi70stBSXwmkXLcBvJdt6z0Q7428wdPFMQuD+ZO6cT9z25xfr//2S2YNbn9iRQwBFPszI9eiI5DDkF93VpMPuMMk5Q4LGH6BR8YUicKtAc1pu43vRuPrdmB1cSEQ9OdUATLnwEOTj7TahXUCZ772IVBYD2czcjvY6d+r7NKEhKCIDCRXCxCUStpUZSp2PEXUaXMFDu6HVHswsCsJ0XF+kyGWYqdGlPU+pIpNssXi/el4gnqkPPY2e0M14s1q/IE3U1XSlSsY44M3GPIAx0V22h6fRclx3juHtCOYpeVxy7rXEskQil20ydWnX1SZTRX5Yk2zqW+TzNNsent5AWd6OioWE/jdvBEmo8dv77J/3aInU9BDlLWUTho5kT8w6mH4LA5ch3RvME3vnWq5RAYzLdtO4EzFO1E0UtR+D6oxUW6k3x423H74dLfPIF6M8bLD0oCJP76zGZc8fsn8f5XHjikNoeU7mTyGac7v01909lD6kCB9qEIxr57JcRujajY2cSj1LsL5RBoRHZ9WSslh85jZwZi+mxGUezkleP71cETGS/YMAisEkMmKtYodqKPHfPLoqgQxQ6xTb6oj12JEFbRx84zk6XdkXzseNoRqVxbLh87bx47kM/ui7adKgLtwE4FY65Psm/7Ra997ERTrDwYS8vSQE2xPt9FiUS2k4LCZwJ31wsy1wHckl+lMMDerWLf81rBCLQ7kik2b1RsXsVOWq+ddvIiNI+kdr3wKfrSvaSQZorlE8N8/ZIncHlITBAEuPi1/oT8Q1PsCLGzJnTp2w31uW8nQbHlBpKzP4Vglw8fPOlAbO2r4XO/ekyPRx3lEj70qoOw6JSDh9TmkIhd7733YstVV2Nw2bKkEwcdhOnvey+6jzlmSJ0o0B7UYLbftG4AwNrt/ag3I1RKoZkRV9xL2xkCuyI7PYrULo1ipA97M44t0y0FT1Cc9YINW5KdKVhvv5h9PnZpylrZ8bEzy2gd0iAw+5MqT/hmwLFFgtuLivX5gkmwTbGedCdCf4crQTEHHYQr7Nj4PnRUbJ48dlSBHIIpdrAReVM8SGpEXvMq3z7tPGalvfC1US4FeO3hs3D1+cdi4X5TnTbUvWrVik1JM2IPvFnEzu67r53hI3bq+Wkvjx0PnkjLYzeUqFgfsRnqhENqq512pOsvtcXhuh3k63M7AR4W4c95rxem2HwIggCfOfMwfOzVh+CZDbvQWSnhgBndbeeLpGjbx2779ddj5Xvfh6CrE9Pe+U5Me+c7EXR24Nn3vBfbf3PDkDtSID8UX5g5qQPVcogoBtZtH0iWpRE7JAPELlZfVkETF6IuqUjFZL/+4AlFXOr6xZ2l2CX/VbJY7vzcaMqm2LSXe1JSrKWWOYqdMSNSU2yt4RJRbvLUIiBpT46KVQQr+W5FH3vMkRJshYy2Tz4LSp5PxdhdSJUnmh6yplJyKDIC+PNr2TnT8vfHSnfi8V0UTbE5I10BVuUgVdnLN4i5KnCIcinEKfN7MLW76rShFLuBelOfv7yVJ7IVHr9iN9Rrkgb6/jA+uB5iR85TO8ETOiq2rZJifN9KaXKfrXbhpjtprz+2j117k4a8vrVWlHEYpCrYdFHedCftTNYKJEFjL9l3Kg6dPWm3SB0wBMVu07e/g55PfgLTzz9f/zbtvHdh89XXYNOVV2LK38l12AoMHxRhKYcB9pnahWWberFqax/2ndatB9yS8n0EUJ03D7Xly9HRrAPosHzspHZ5eoowCLRalx08oVKPZKgGgSJwnuCJyBM84TFNqn3SQYQiUWyM07g6toZAREvsxU7JooJYUkzntjIKhUI7JcXMeeemZ1mx89WtbMdvLQ1WMmRWecKXxFaqQSylIAmChPi1E+hhfOwiL8HkY0riY5mf+OYlOFlpL3QbzBgrERub2CWfqa9imimW9jHrXKZFdLbjJJ8X+vmJqA9udlQsN8UqC0Izir2KXXs+dvI1CVLWGWrbeViyLyo2a9OhRsP7otKz9pHVvlpe8Lr8eGT1Nvz2kbVYs61fW5EUvvOu9i2hbb/666tWYdIppzi/T3r1KaivXt12Bwq0D5o6Y++9Ej8d5WenFbsOQ+xU5HKprxeAXRWAQkpQDNjBAN4Exa3feZSrD9zHTe3LpCGJRBKZ5v9Co2JjMMUuMC/9IHBfZD7CSPeRXSu2Rcg0uTHL2jHF0khL30ArVQgYrpJiHPaAYxNyfpmldChpOfyyiqlL0KbYeuT33xOuL692kLZLKwIz5V7Oq9j51CEK+pO6V2m+wHRil3+QTovktU2xqc3khpkYxWhmpEOyTbGu7mBM32wfypWjrXQnchvWtRziOXCDJ7K3ocfejmI31Oe+neClNEsJR+h5HxWQcf3fnsM5V/4Vz2zYhZseX49GM8aS9bvw16WbMSkl7VMa2iZ25Tlz0HvX3c7vvXfdhfKc2UPqRIH2oIkXgH32SvzsVMoTrdgRYrfXueeiPHMmyrVkncFGuimWD5bqoU8LntAJgRWxyzRNKQJnr0+JQZZi50TFEh87XrUiDAKrqD3vns8UGwamr7S9pkA6TfUMdQyG2RmFyz0mpx2ilPpmymIeuxEidlYeO62oyvVWTR5CGm3t749k/soCNcXGHtIo+Ru2E6lYzmmKVdUxJnSkm0586pBvnSpjLtVymHqOfJORtL5I4+5IBE8YVwbjqpGnFBdX7AB/OTR1uipt1YqVr8lw+BkOJVKVHlM7xG4I4iCAdska/Zzv3V742OXDt25/Bp8763B8//xjUSkF+PzfvRi3fuJVOOvIOZjbCrBqF22bYqe/53ys//d/x8BTT6J74UIAQN+DD2H7dddh1mc/O6ROFGgPilMEQYB9WoqdJnat8VQrdpUKqgfsj9mXXYryj58EAPQP1MV23aS+tpqSXlLMDrzIdrBN/isCwKNi681YVOzSEmWm+dglbcuEie6ftxsGxohmETshikRtpyoK9ArVF3KZYrUqEXhfqPZsW1a9hltxAezgFr6MLpcDR9y2fQpMGuxasTl97AJ30E87PzbR96931pFzsKW3htcvmJPa56zgDv4bT0acptbx9rOue5ncX2475vNw5bHTxA6mpJg3QTHZJ/exA/zBKkNKUOy5Z0bCxy5PM/SYOqzgifzbAfkJVTsBEXY1jqx27f8F0vHs5j6ccmgPAKBSDtFXbyAIArzvFfPw9u/dg4+nRF/70Dax2+vtb0dpxgxsufoa7Pz9jQCA6kEHYe+vfgWTTj217Q4UaB/UB66nlQl+465Ba1m5M8lkXZk5E0EYYtLJJ6PzxrUAgIF6Awjcl2ZChsx39VyqF4BPRVPLAL9TPYdWdnhUrBU84W6XaoothZaPXcQUO5rHjr8MfaW7KAm0TbH+vk1qqTi7Bty0MnleujMmdKBaCjF3apfX10sOnpD7s7ugg6UOnvAokCG7rva6Apkh6mReUB8733kN2LlITLH5B8C8UbGTOiu5UhK4CrE7WtN1uK9YljO1T80V101RVNohiHlBJ1uNLFMsVewEYkdTFln7YBPDdvrF9209Z0N8hoZCtugmto9d1iR5aMROypuYZx95gzmKyhP5MKWrojNVzJ7cicXrdmL+7MnY3t+wSjO2g6HlsXvtazH5ta8d0g4L7D5oTjSdFqHFNHQwQmdC+MqzZuntuvffF3h2O5oCqQNUAmJXsVMPaBIV6zPFtvqW08dOvax4VKxSVRqRT7ELxM+AMsUGpj8WsaMJioWBlgZPMDJlyKJZX1bskhUndrSIHQlSSVOtOKZ0V3DjRa/EpM4Kvn/ncrFfYq1Yz2C3u7BLitl57PIodmmkdihmG6VmNKMY9YZdkk63Kwx27VSeyJvHLi8cU7FAbHyJqYFsxa6doAe162xiNzz3D/Wx064XOUqKSaZYk5JEVuzaqRXrXBPhXhzqKXBzJuboj+f6ZxOp9O95tmsnpUredQtTbD4cN28a7lyyCfNnT8brFszBpb95Anct3Yw/L9mElx88fUhtDonYFRhbUEf8qjZdJgOcUuy69tsHnYcfjql///d6uyrxu/O1ayl2LN8VL9NFoZQ6X34zDhMVa5tijY9WnpJi9rKuSskyeVA/uCAIUC2bfaT5PPEZu/pKiWaaYqf8rnYSxa6dkmIAcODMia023fYBrti11Io2TI3tgA7CFabYuYOjW3IsNXhCD9T5+0PVKxU16hvoFRIfuzZMsTl97PLCV77K7g8Z2Mv28rQcdnzbTEd44mvqtBPKn3cHajdxbCZEPsWOdkkyxWozsofItxUVy++ZYfSxG1IeO6+PXfp2jlqds880yridlCrZplh5ollAxqVvfLFOD3XhKQejXArw4LNbceYRs/HRVx8ypDZzEbvFxx2fe+py6D1uYEWB4QWNWi2z6Dk14FYmT8a8X/6vtV01w/+E+6VxJ2uu6Fnbah+7vFGxsPptXsyG8GWVFOMvoyldFes3Xqg+zRTrM70l6l7ynfZGVOxa72Kj2BlfRp3upM2Xnc80JlWkGI2SYoa4ycETso+d2tZPZto5L3TQU8SOb86/l8JAyC3m36cvzcxQwVtIOxeAGzzRjik276RKUnSDYT5u2k4Ux9nBE2SfaYod317dE1kEmMJ379LUNMNF7HIpdl5il3E9nfs6RwfVtkGAJuIcFha5nxKMr2L+frxQ0WhGuPXJDTjpRTMBJM/kR04eWrUJilzEbtZnPrPbOyowfKCmLe2T1hpodVSs8FRlzWZ5cASX1JuRP0GxLweeD7zyg9oXJartVp6Y0m0Tu4gpdprYhe7LL83HTn21gyf86U60j51kim1TBfE5s5cCt7+jke6EK3a7k8cuWb/9vtL+8ImBgqtiCCkoUq5FGJp6xsNjimXnSTSDms/tmmLbMaH6VC/er+GOik0SFCvFLocpNsXHjpPS971iHvaaUMUrD56Zu18+871deSJ3cxaGUgWG7pcSu6xNswKHMvfZbI88Zr/b2+/HCxXlUoh//tWjuOXjrxredvOsVNSBHV+ICElQPmn1Bks3Irw3s4mdrNjpWqm5TLH2tj6o5TzvnYm6jMR9pTl3T+mqWE7z3MF/r1aG/yldFYGQyKaPILDNtnEcIwgCNIW+qWPQip0QPNGuCuLPY+e++B1yMyI+dvZ144fDlwPpdXLVANjuGFAOAzRSfOwkRZabqrOuRTkMUWtGwzJAOZUnBFZlmWLb9rGj7WT1RZ4I8D4Mlymf+r3qhOA5FLvutDx27JqcMr8Hp8zvGVK/eNvDoVry69uOYhcEtq9glprm5PRrZ5Kk3DgyrDn2uye9Td9Es4CMl+wzFU88t0OnLhsOFD52eyB0uhMYh3CV0V37PgkPVRaxS3zs/EpU07McgK4hqxXDnCH6agbPo2KbnlqxvAwOxZSuiqUUaWLX+n7o7En4f287Cof0TMJz2/qtba0SUqH9EqN7ieJE6UhLd6KI3c5B18euXZLgG2jp+fWZp4bPx44QO5XHzpOg2Ch2NI+d/9jTSEYadK1hj0nY6VfQXq1YILkXa83hOY+5Kk8IyqhCh2CWtLYdgmInB0/Ibe4OqI8drzTDkTeP3XBMWiRzPcArTwytbU488xBEOrlNcznhkNTpvPClSuKgi/OWFCsEu3x41wn74//+9kms3T6AI/ae4viWHjZnctttFsRuD4TkY6demGmm2Go5/UmLmk3L1MpNE4mPXXqf8kfFJv9rToJiEwySlaCYvmAqpQBdlRL6SHi4RKbeeNTeAIB1O2xi5/exs5MZJybVIFfwhK3Y+a9LGnwDrfTiH4r5Jw8mdVYwY2IVnZWSVZxe2odxDaA+djmCJ9rsqzpWZYqVypXx7zx4gpMtb99GkERQ0J94HruJWQmQLfNhen/nzZhg/bf7SZ+v1GZyg0bFah9gz8yP7lMyxfoUu6HA52drBwoMUbEbwiSLKl2lNq7DUNOdqH0B2fd4OxOHtKjrAi4++uOHAACX/OZx/VuAxKc7ALDsite33WZB7PZA0KLqNNgAMGRGGjiyFLtmvYGYEBbVhJWg2LctC55oNyqWF/FueHzsfC8YZV61gidYaTSKtChFTqbssmKttlPSnUzqSMrA2D52btt5EIbyC14OnhgZYlcth7j54lehVApw8+PrAZjjd81Zdi1ZwKi5so/dEIkdMdkn27vrKB+5ZP32C7PnLReWB66jvvsspqU7yTLT+Ei/hJfsOxV/+adXY1YrB6bdjr/PQwX1sVM+tT7TH81/KBG7tOTK7cLn92ifg6G1vTt57EpB4J1opm3n+56GvCX92jHRF6bY9vDnfzxl2NssiN0eCEqe1ABQY1Gx0stgqD52Oo9dWlRs6+e8Jke1vM7y2NF0Gdl57MzvU7oSMkV3G0WymkP3Z9qSE4IGAWuT+RJKfVPlpfpqzaRgeRgQktney872nZJf9lZQRRh4U5HsDvaakPgnqtPUno+d/54wEZrt9Ucdmy94AgDmTOnCmpbJXYqKzfSxI3kPdxdZpmK+Do+KVRVmfGi3WsLenlJFvgosuwOj2GX72Km0D4DHFOsJnhgK+OXfb3p36/f8pMoHxxSbY5v9pnXj2AP2wotmTWrLFLs7QVP5FTvyOdMa4xLkAn4Mp2+dwjBlKhoarrjiChx77LGYNGkSenp6cPbZZ2Px4sXiunEc48wzz0QQBPjVr35lLVu5ciVe//rXo7u7Gz09PfjUpz6FRqMhtvN8AFV/KiwqVleeEGbE7frYqfcD9bHLNMXmTXcSKmJnkxAa5SubYs1n+gKWiF2D+dhZ+/eYEGlf1Hp28ETyX1Ls1GrKFAtAZxRvp6SYr5/0sy+KdySiGqX+RB4CTyOoFVKDJ4ZoWjMuCH7F7qh9p+rPkil2d3zR2gVXWyVSaa3Dnt8sYjdcSttwkBoOEzwRE2Inv4toHWvpWVFkrzMjmKSdfgFJJPvR++8FgPuTDbHtMGDtZDdULoX4+Ydejn9/04LUfJ0crttB/n7mLenXzn2hLQiFKTYX/veB1al/Q0EuxW71Rz+au8F9vv713OvecccdWLRoEY499lg0Gg189rOfxWmnnYYnnngCEybY/h9f+9rXxIej2Wzi9a9/PWbPno2//vWvWLt2Lc477zxUKhVcfvnlufuyJ4Gmj+CVANIUs+w8dixBMZPpowjeyhO6pFjuqNjkf52bYsnxZJli6ctPETvLFJtyLnxO//xzGNjEUBO7FDWxo1xCtZREVO4aaGByZ2XopthA/uytbdtKX5B8Hv4Xq7onGh5VUFLs4hTFzkQCDo3wNlJU2YX7TcVvH03K6A0leEIqMTVUBJ5rZ+2PDZ5Ufd176vCZYtPbMZ+Hu1ZsUlIs3RRLFTsJF776YBzcMxGvOGTGbveLXoeTDplp5blU2B1yUg4DZ+KaF5Zil7Ex98dr57oZs2lWOh35s9ifYZwQvRDwb8S3Dkjeaf31JiqlEF2VEs45ep+228xF7MKJk9puOA9uvPFG6/s111yDnp4ePPDAAzjppJP07w8//DC+/OUv4/7778ecOXax7ZtuuglPPPEEbrnlFsyaNQtHHXUULrvsMnz605/GJZdcgmo1vdrCngiayV+9IGvaxy5ZZyg+dlFsiJskvXPiJ/XJKFPpx1BiBEBHhKko32YEQRSzCY1gipWInfR+4S/LSii/zJ3gidb5aQgngm43sbOMLb017WeXlsstDX6Tq5zAtJ2Z/lCg2vSR5pKOanajYqVD90X0ZkHdX7WG7OsHJMSO9tsxWWXsc6R87HxmSDulRGCpnnu3pdgNsZPg9/7Q27HbTP7HcawnoF7Frp5O7F524HS87MChlVny9QsATj50pvj77lz7ZNuhPfftKO/tBM44+9GKeb71cvVniO4VL1Q8csnpzm/LN/XiX371KC446aAhtZmL2M29YnSUr+3btwMApk2bpn/r6+vDO97xDnzzm9/E7NmznW3uuusuLFiwALNITdTTTz8dH/7wh/H4449j4cKFzjZRrYa4VtPfm729w3kYIw6aoLjKTFJp0ZeVDPNFBDswQ0ENoqmmWBU80WatWO5jRysXSOpgWvBE8ptZt5lDKVKQ8sKpz3Rdde6jLGLXkRA7VVZsqH5vedKd+NSgkTTFZvrYESf4fLVi2+tHifVD2v7Fc6foz2u3D2g1WCHr9JQFBWeooC34no00vyqVQifPHnYnYnQk7h9TecK4jPgUu1pzaEXPh4LeQbOvVxFi166/og/lMMDgENtph6jvDhnPO7FqR8XkLjwF2se8GRPw6TPm46KfPozbPnFy29uPm+CJKIpw0UUX4cQTT8QRRxyhf7/44ovx8pe/HG984xvF7datW2eROgD6+7p168RtNn/nu9j0zW+aNuo1cb3xClrFQA0+UZyQjd3JYxdBdnSnflVeU6wKKsipTKn3us6HphU7U3lCIpG+gcf42BESlqbYcULiqQ3KX07q3MuKnflsyoo1Wtu5/c8Dn9Oyr48j4fzu6w/fN90/NaPThNoc6rS3rWRqAulX7DqJ8/0jq7db1SR821DoElPDcB7pteO+ftZ6rf7R6zi5M/s1PXw+dnKbuwOa7kQ9N75asf/nmP3wzduX4tVtJhseCg6fOxkTqiUcsfcU9Ezq1L+36xvnQzsBELuzLV0+IXMCwLfNt4927gut2BXEbrdQCgNs2DGYvaKAIRG7HTf+ATtuvBH1tc8hrtetZQf+8pdD6siiRYvw2GOP4c4779S/XX/99bjtttvw0EMPDalNH6Z/8AJMe8/5+nv3mjXA/P+/vfOOj6LO//9rdrYm2fQOgYQWepEmosjR7fVsWE/BAvZydhFP8dSvp97PeqegZ0FPDwtWLGChiEgRpAYwQHrv22Z+f0yfnd2d7Zvk83w88sjuzmc+85nJZue17zo0oseIJmIXA1AKQeJiGFm5E+/9AsXYsaBkHRKk1+VZbb6TJ/jfwvF1fhhJFjvudXnniWBaiqXywk5Yu7zKvdZKfDUA1zqGInmC/61lsZPvp65lp7cMjL91ypfsS4gqXdXRs9j5em6UZTULaFmBBUJNnhD2k+rYaY+zmWh0ujzi9TLSBpn7Vt8xIuOK9Z7X1zEZD6sYX+gjg1U5v7RDOMv15foPB8kVKy9QrC1u+2UlYefDc5GsUeok0mSnWLDpvlle2beRckcrXO4hWtKAwH9P+djiLO/ahHr2DcpiFyjmL8Twit7Kmt+rFc9ZlkVNqwNvbDgsJvQES9DCruGN/6D2mWeQds45aPvmG6Sdey5cR8rR+dtOZFxySUiLWLx4MVavXo3vv/8efftKgYLffvstysrKkJ6erhh/3nnn4aSTTsLatWuRn5+Pn3/+WbG9upq7UFquWwAwmM2ALPaOTg7unyHeyGOW5GURXB42vHInlEF0S2uJBY+/cidC8kSQWbHqGDuT6FrWjufzZZFKkwk7A8WVF2H8WC+Ls5JFAQj4qWNnUMXY8SFA8rINWgH8dtFix33x8Rdn5g9f7Y2UH7TQfj2KrlhpTcrtWr1iffWVlc8XbDyOdx077XP94PoTcM+q33DnnFIA3N9LsM/rzoqNQKyQ/Nx9xdhJ41jFe7YgzepzvIB8jeHcUKPx/lFa7PhesX7WGNjtHDm0juXLSh4s4bi1aY3PX1/IN2sVnfa/bwjCLsCpCEOJwU4fC//zi+I5BSAz2YITBmbh/tOGhTRn0P9Bje+8g/ylS5F2+mloXrUKWddcDXNREWqfew6epuag5mJZFjfeeCNWrVqFtWvXoqSkRLH97rvvxjXXXKN4bdSoUfjHP/6BM844AwAwZcoUPProo6ipqUFuLme+X7NmDVJTUzF8+PBgT69bILeAyG8Sbg8jCiut4ORAwg4AnA2NALRN74yfGDvhuHrdXOoPB0r1AeP20VJMEVQsm0Qp7LjEUNEtrHH8zGQzxvRNx7YjTQCUN1t1jI18f8EVLazNYjTAzXe70LLYSTF2wnxBWuw0LKfcen0kT0TbFRugL6V2jJ3vOLhwO0/4am0mMLwwFR8tmuq1PiCYOnbhX0j5DP5uosIm+fs8Py2wxS5SZUpoxZeEkKfRhJElTySyNUfekSScVYZj+QvVFRussBMs2cGIR93JE0TZ6eJQCJ0lAhG0sHNVViJp3FgAAGW1guEtPGlnnonDF16E/Acf0D3XokWL8Pbbb+Ojjz6C3W4XY+LS0tJgs9mQn5+vaXXr16+fKALnzJmD4cOH47LLLsMTTzyBqqoq3H///Vi0aBEsFu/K6j0BuVtP/g/p8rCy5AXv/QIlTwCAs6FJnFtAXpvMV1Ysy/J18HQmCfhq5yPvpKFlHfT1gZeeJFlgBatHoGLJM4bmisKOprWFktoVK5y/cIOymGi088LOf4xdaDc0PckTvqws0Sx3Ih1DuV3LYqfHFRtqNp+6XE4gTIq/s75jxCorFoCsUHeQFrsIuWKpKLx/hHnk4RH+4gzjjcKDGsbFVHyJCFIiyqNm9Ma0AUBxkMJOb+eJUOrYRaLtGyE0gv7vMmZnw8Nnr5oKCtC5bTsAwHn0mM92U7548cUX0dzcjOnTp6OgoED8effdd3XPQdM0Vq9eDZqmMWXKFFx66aW4/PLLsXTp0iBX032Q10SjKKk+l0tmsdP65wsUYwcAjoYGANqZllxLMd9/ZYb139JMjlfQPf9cTJ7wISIDdZ4ApG/Z/sqdAMCfSqUAbbmINKg+VBWuWFWSiMWoLQh9x9hpr8UXvr4p+xZ8kbHc+F4P5fe5uowNEKDzRIjxON7Zufr2D+b6SOVOglqaJr6sqr7GGShKbAY+Z0Sez/HSftLjSLliI/X2UcbY8a5YHZ9F8SJSWbG+4mP14KsndqCxJUHG2IlZ6Trj5oAgLHaJq90Tiuv+swUvri3zev2ldWW44a0tIc0ZtMUu6fjJaP32O1iHD0faueeg+vHH0frVl+jcuQv22bOCmstXvFaw+/Tv3x+fffZZ0HN1V9Q10Uy0AS6Ph4tLC6NXLAA4G5sApGjeiPy5YoXtorAMUthJBYoli51mgoL8xiP7FqyOsQP8x3YBwIjCVPFxeX2H5tooylevWP/Czq6y2AVaiy98NWWXJ0+oW4pJ6wnqULrwcqGrtkvlarzr2Gm6YkUhE9w6vCx2OvcPxmIXUVes/G/n545nkAnddXf+CTWtXRian+pzvLhfhFyxwbjc9M8pfX64dH7xiydUhK5BMG5/NcG4YoXuNgBQnB1ceyr9vWKlx4FORSphlLh/40Ti58MNuGX2YK/Xp5fm4N8/HAxpzqCFXcHSpWJX78z580Gnp6Nz6zak/GkGMi68IKRFEIJDfaMUPkBcDONXWOmLsWsCJ+yk1yhRKPkX41xyBfc40AeFtytW+brc+idHfk/sckl1qNQxdtwcvmPsuLko3DxzMF798ZCiurfWN22K4iwOUq9YQdjRXmMBqeyAV7mToGPsfFnptAWcwnITA4ud+oalbbHT3hcI/SagzorVu78vQaw5NoI3KC0LuBaiG8sA5NgtyLHrCyeJlBiJVEaoHEr2/yj83+j5LIoXERPJ4Vjsgvg7lDdIX0rtVpOfkd6IXyQCWFB9fXnUnJMSfhNhp4d2h1vz/8FoMIgx2sES9H+Xu6oKoKWbWdpppyH//vuQcel8uOvqQloEIThEV6wYlybUfmOkzhNarlgdMXaO5hb+gQOO/fv5ubiXOFcsh3DTM8oOI//gDqapNPecn0/2Bhe6aciRn5f8TW81eVvO9CQs3Dp7CLY/NAfDCiSriJbVQpyDvwCisJMdV5E8oRJ2IZc78XHD1pM8EalyFXLUU6r/jsK6lDF2/urYhSfsfK3DF7567Po7RiQuo+Jv5+cmGmrgeaQEWTi113zOKVrsJAurvzjDeBOMdcofsbLYHZEJu2AJxWKnP3ki5GX1Kobm27F6e6XX659sr8DgvJSQ5gxa2B2YNRsePg5LjqepCQdmzQ5pEYTgUFvs5CVCBBdYyK5YPrOZbW9H9bJlirkYuUWOVzgmj1THkGHl3R78H8dXbJY89sap0TdSfl4tXdKxFR+c/EOhtEKgz1RvkeD9oSq8wqiFnUwsK8qdeMXYCWP8r0WNT8ucj+QJX63GIkXAGDsh+UXDYqd1cxMTFIItdxLAcugLo48OI9pjI2ex0x1jF2bMofpxsEQqcUCOfJZALcUSgUhlGIcjtn1l/2vh8nh7NnQfR+f7LZhzkWLsiLLTw40zBuOf3+7Hbe9tw/tbjuL9LUdx27vb8Px3B3DjDG8XrR6C/+9iWc1PRLajA1QPzUJNNKR7pmDl4n5zFjvfMSxmXa7Yen5mFs5jx7jHggWMlUqQiMKOkaxmHoYNukCx+rn8A19tsaMo5YfuwBztbzPymB6tYwVCoREp5W/BZilcZ6G7gfp6p1g4l4gYY6cz9lCNr2bgvgRKNMtVcPP7F3bylnAC/v4OobpiA5Vd8YWysHOgsQbNY4WCMsbO94GFLhOpwbrUFO+B+IgRPXOKFruETp6QHofztw/G7a8mmFhZoYjtaaML/A/0c5xgSqoEOhdS7iQ4Zg3PwyuXj8cf9R144MOdePTT31HZ3IU3r5mMuSO0a/EGQneMXfWyx7kHFIXaZ5+DwSql4LMMg84d22HtRt0bujOCuPCy2MkySTUtdsbA/2iOqhqgCDCwLDy1nGtdEGlcSzF+fpYBQMPsdgFGm7hdryvWl0CQ3/RcKmGnFovTS3PwxPmjFUkQ3Fzcbw1Pri60GnALJVQEi6U6K1Z9uuo6dmK5kwi5Yn2VOwnmm34oeCVPqJ6ri1lTFCWLsfOeT7j3xcwV66OsjebYiFrspMf+hNf/XTAWh+vagy5b4SvJJlgiFasnRz6PVO4kcW/6kbJ+0mH8TYL5O7ww/zh8/lslzp9QFNxBILfY6V+P3hi7RE6QSTRmDM3DjKGBs9/1olvYde3ezT1gWTj27QNlkpWXMJlgLR2KrL9cFbGFEXzDqGLHxHInbsZv7TY9rlgXn2FFsSyYjg4w7e2KhAbRFSsUQla4Yv0LSzm+BIG8n6fLrXQxqIUKRVG4QOPDzCATotx+fpfic3/5vsIrvpIn1NdbaFPUySd4+BM3utfiU+Rpi7lofGP2qmPnw2IH8D1PKVmMnR+LXTiuR19zayHveKC3jEQkXJLKzhO+35Bji9Ixtig96Pkjlc0aTNC+XiiN001kV6yvZKRgCavzhOJ96n9sXqoVV04tCWp+AWFdkXTF9s/iMnOLMoPL0O2tbD/SBIZlMa5fhuL1reWNoA0URvdND3pO3cKu/xuvAwAq7rkXeffdCzoltKA+Qviob5TCh6RLZjEL1RXr4ecy8LY5d12dePOVtxSjWQ9AGWFkPOBsWUpXbaB/fl9ZsYDUz9OhMrnpDbgWPoOkXrGhiwa1W0G02DFqi532+QjXQ12iRv9atNelTJ6QHTcKN2blelSCykeBYoCLcaQNtN92asJ6g9Ui6veC3v2DcXFFso6dfIpoWDL0xvAFM0/kesV6z5PIFjtF54lwRHKEsmKjWehXXY3A9zj9IvXS4/tj6qDsoLtg9FYe/Ggnrj15IMapXq9u6cKL6w4qOufoJeivTYXLHhNFnauqCi6+WwQhdqgD8eXdGhg/Lj89FjtG/HrNC7vaWvFDSZ4Va2A4S5SR8UjbGQTs9iDgz+IiWFVcbv+uWF9I5VmCq3Em7q/h5hRj7ITOE6qsWF+uZYYXw2yIFjtfLja5oPLVLSA6AkL9XG2xkxYmvBf8lTsRXUFhu2J1WuyCccXSlK5xelCK8mhYUrUfB0uk3JDKOb1fS+SsWMW1DGMehdUt2C+XUba8C+j9/wsmU5iiKAzISYlKVn64HDt2DJdeeimysrJgs9kwatQo/PKL1KuVZVk8+OCDKCgogM1mw6xZs7Cfrw7hj+effx7FxcWwWq2YPHmyV/96f+yvacPIwjSv10cUpuFAdavueeQELexYhkHt889j74SJODBjJg7MmIm9Eyeh9oUXwDIhBjURgkIdjG5UlDvx7X7UU+3dw6saAytZ7MQYO1ZmsfPwbbRYRiH8hNizoDtPaAgSdfKE3pgxKcYu1BIj3t+0hVfEXrEqV6z6EHILn7wcX6SyHQUBpZ5OGdcTXcuQ1jHk7zs3o7RWRjR5IkRXbFDJEwZta2woRMqipmf+cNar/FITzopkc2qImsR2xUbmWgbjTvXaNwoCWwu9GazB1LFLVBobGzF16lSYTCZ8/vnn+P333/F///d/yMjIEMc88cQTeO655/DSSy9h06ZNSE5Oxty5c9HV1eVz3nfffRe33XYbHnroIfz6668YM2YM5s6di5qaGl3rMhsNqG1zeL1e09oV8rUOukBx7T+eQdMHHyD39ttgO+44AEDHli2o+3/Pg3U4kXvrLSEthKAf0QLCfzZKLcX8u2L19IoVLHaUIOxqamGwcdlWHkZ2bA8Xi2dkPaABuKBy1eoMsBWQf5AJVhV1uRO9b3Kxjp0gqCLwoerlimX1Wezk7mkgcq5YwfDkTyBH4/PXW8AqnyssdvwfwJ+1UnR3BrnY0OvYaZen8XeMiLhiZXNEww2pCG4PQwhEw2KnNU0iW+wiFq8YRukhhSiMogYOxWKXaNmu7U43WmWlr8xGg6JwvMDf//53FBUVYfny5eJrQs95gLPWPfPMM7j//vtx1llnAQDeeOMN5OXl4cMPP8RFF12kefynn34aCxYswFVXXQUAeOmll/Dpp5/itddew9133x1w/ScNzsETX+zBv66YIGbDN3e68MQXe3HS4BwdV8CboIVd84cfouBvj8A+Y4b4mrW0FKa8PFQ9vJQIuxigjrGTsmKlzhOaBYr1xNgJFjt5jF0/bpu8pVh6ZzMqbBlId7ThCFgAlLJAsc6bpoAiuJy/8amFnd4PFK/kiSA/iJRuLaXJTm/yhPCZLr8m3LigluIzaJn2YU2KtgsnkKVMvka1xU5LSIVazFQrkUYPwRQojqQrltL420USrYSfUAgnk9MXWu+RRK5xFqkMY1rxORLcvjF3xQbhYUkwXYfZ/9wMg2WX+PzmmYNx6+whXuM+/vhjzJ07F3/+85+xbt069OnTBzfccAMWLFgAADh06BCqqqowa5bUGjUtLQ2TJ0/Ghg0bNIWd0+nEli1bcM8994ivGQwGzJo1Cxs2bNC1/vtOHYYLXt6AqY9/K1Z4+L2iBdl2C/5x4Vhdc6gJWth5mpthlqlcAXPJAHiam0NaBCE4fLYUC2SxCyJ5Qvhwc9fWgi7mHztdoityZF0Zzt2/FkOajuD6wvvACTt93R7k8wvI1ysmg6jLnQR5w5KSJ4JDS0yJFjv+9UDJE1ItPSj660ZEZMJ30oH8ZhKNm2cgSxlFUTAaKLhliTyS2Pc9X7DXRW3x0W2xC8oVK1gTg1qaJtGOsYuYKzYKlhkvq24CtxMDVNapMP5WYVnsfHyhizRi54mgsmITS9mtuXEiCgv7iM99dVg6ePAgXnzxRdx222249957sXnzZtx0000wm8244oorUMXnC+TlKcuO5OXlidvU1NXVwePxaO6zZ88eXevPT7Pii1tOwodbK7C7sgVWkwF/Hl+EM8cWhtx6L2hhZxk6FI1vvY38++9TvN741luwDC0NaRGE4FAXKJa3FGP8CDvaIJUS8QWbygVx0lau2LS7rk78R67717+B8y4EwMXgTa3cyY2VWbHE5I0A70f1dmVWLHc8R4jJE3JrGRBK/Jb8sVJAsSqLXXFWMlIsRgwrsKvmkKyGcldseDF20uu5qRZkJZvRV1VSINquWC0hp4bmhZ3Q+UNyxWqPlf/Wi/q9EI3kCVr1JSccoh9jp32s4OeJ/PtHvR5TNJVKBKAidA3CKRbuq7NMpNH7/xcpV380SDYbdfXIZRgGEyZMwGOPPQYAGDduHHbu3ImXXnoJV1xxRbSX6ZcksxETizNQmG4VO4ms3VsLAJg9PPj6dkELu9w7bseR665H+4YNsI0dAwDo3LYd7spKFL3yctALIAQPq7LYKVqKBRAzJtrgJZjkWKeeBNQARjsnVNx1dVJjd7cHqOXebBQksULxxYqDccX6c+kJFo3Qkye4ccJagnaDaLi1hFfU5U4ykk3YcM8MsW6dej/GK8YujLXIHltNNL6/609e3+jCqZ2lB+86dt5jjAYKDsizYn2/J0K12KnfC3qtasr+nfrGRtoVG52s2MiIkUhZq+R4WZUTXthJj8P52wfTvk5NrJIn9Aq7aH+uxIKCggIMHz5c8dqwYcPwwQcfAADy87kuD9XV1SgokLp4VFdXY+zYsZpzZmdng6ZpVFdXK16vrq4W5wtEeX0HFv7nF+ytbhVLh8mv8MFlp+maR07Qdr7kSZMw8PPPYZ81C0xLK5iWVthnz8KAzz9H0oQJQS+AEDxqC4i8pZiQmOzrHzVQnJ3thBO4uc3cNyB3bS0oPlGCoSi429r5RUj7GFjuoHIRE05WbMSSJ0RhF7poEPaVu1blcxsNBtitJi/3kny8PFk8EtZDgWSL0cvtoMgujkOMnXwNXlmxGm+9OcPzMLpvGuaNDK51Tsi9Ymn9Nygpxi6opWmijLGLssUuLPeh93s/XCiKUpx/qO6lWBGpeDKtzxG9RNvyLnDaqAKM7puGWcP8W4UU1ySx/3w+mTp1Kvbu3at4bd++fejfvz8ALpEiPz8f33zzjbi9paUFmzZtwpQpUzTnNJvNGD9+vGIfhmHwzTff+NxHzcOf7EJRZhK23D8bNhONr26ZhnevnYJRfdOxcqG+OdQEbbFzVVTAWFCgmSThqqiAqbAwpIUQ9ONV7kSMSQtcbsRkNADemdUiYoyeiXtreBoagLY2AABLUfC0tQNIFpMrAMDAMACtdsWGbrET9g3UUswXwihPiDF23Ho4UeblihV6xQZwOcvPxy1TdsGKLSrID9RoBzl7u2K9xwgilxGFnffaBMb1y8DHi08Mfh0+spADoSjsHOB6Jpu5/wG1NTYUuk+MXXQEhWCJABK7ODEQObd2MIk6XmuIkYXshEHZuv7/EjkrVi+33norTjjhBDz22GO44IIL8PPPP+OVV17BK6+8AoB7799yyy3429/+hsGDB6OkpAQPPPAACgsLcfbZZ4vzzJw5E+eccw4WL14MALjttttwxRVXYMKECZg0aRKeeeYZtLe3i1mygfi1vBFvLzgemclmGCgKBgOFicWZ+OvcUiz5eBc+u/mkoM81aGF3YNZsDP7hexizshSvuxsbcWDWbAz7fZePPQmRQl2g2Mz3gHV7/LcUAwLXshOsLAajkbvzMQyY2loAZjCUAZ72NsCWC4plYUhLA9PcDIoXdiyr30rmHWMnu/GpLHYUxc2t1xIhHNrjx1IUCAPFZfkKl0s4H7FAsUcQdtqTywWcPCs2aJdMkB/w0XaZBKpjJ1+D8F7y11IsVNTvn9CSJ/zvdOaYQtS0duHc4/oGuzwvlF9copwVG+ZlVn+piQTC/xOQ2DXsALW4Df0ahPM3SbQsVK3Ere7GxIkTsWrVKtxzzz1YunQpSkpK8Mwzz2D+/PnimLvuugvt7e1YuHAhmpqacOKJJ+KLL76A1WoVx5SVlaGurk58fuGFF6K2thYPPvggqqqqMHbsWHzxxRdeCRW+8DAsUiycFMtINqO6pQsDc1LQJ8OGg3VtIZ1r0MIOLKv5TmM7OkBZLCEtghAckmtLabGTx6T5tNgFcIO4+TkMBgp0ViY8tXVgaqoBqggMZYC7vQOwcWMtxcXo3L5d7ELhYVjJFRxGjJ0QXC0Iu8I0G441dSI/1Qo9qMudBFv1XZqDlSx2/OvqFmG+zlNuXXPJYv3Cq2MXeLyiFVFUYrmUz7VufIKlwrvzROTWoRYH+i12+t1jGclm3Dl3aPCL00B+pGjXsQs3Nk4QYZH8ewn/T0DiW+woH4+DRfFeC3ImedZ7InRwkL8XEmA5IXP66afj9NNP97mdoigsXboUS5cu9Tnm8OHDXq8tXrxYtOAFS2m+Hb9XtqAoMwlji9Lx8rqDMNMGvP1zOfqF2G9Xt7CrXvY494CiUPvsczDIFCzLMOjcsR3WoZH5ECT4R13wVfig7OIbzgOhx9i5ZRY/c2EfdNbWwX3kCNCvCAwoMB0d3HaWgbmkhBN2fAyeh9WfFau++SgyyARXLL+Wofl2/POScbrf5OoYu1BuUMKHFyX7gAW8W4r5us4KV6wglkP4RAzWeqCM6wn6cIHn15E8IYzRU8cuUuvQO7U8FjKW96eoZ8VG0FJr4E12kfx7RTvGMJJEyq0dzv+i8L0lUaxjiVzupLuzeMZgdDq5e+hts4fgL69vxp9f3oCMJDP+38XjQppTt7Dr2r2be8CycOzbB8okpRdTJhOspUOR9Rd9PmVCeKhj7ASxJs92DdViJ49Ls00Yj87t20F1cmKOoShF2zjzgAHcWLGsReCsXAH18hQ3PlGYCa5YCsf1y/A7nxxhKrfaZx0E6q4DYh07fkp/ZWUA5QeyKAJDWEew35SjnU2nJ3lC+KIh/P2YMAS2L7xdsfomN0VQAAWD3IIbnRg76XG4YkD9no8ESot8YrtiFX2Yw1iqIsYu2HI+EczIjgSpNhOG5tthNdG6WlMS9HPyEKm7RHF2Mr69fTqaOpxIs5lC/nKlW9j1f+N1AEDFPfci7757QaekhHRAQvhIFhDuuVj3zRU4SN9k9P9GkcfIJU+ahIZXX5OyXikDGH5eA8vC3K8fYDCI2z2M/m4P6vUpY+x4i50nNEFAicIwdEEhdURQumKF5IlAFjv56QnxeKH8jwb7TTnqnSdUNzqtQ4gxdl4txSIoFEJNnlDUsYvYcgIiP1T0e8VGZq7IumKlx4nvio3M/5AiwzjYfVWegnhDGyh8etNJoJAYruGeTnqSOaz9g46xK1z2WFgHJISPaIjiPy6EeKMut+SK9fWlWK/FzkABtvHjAZqGQbgzp9jBdnbxx2ZhHlACOjVV3O5hAmflCnjXQ/P+MHWF6MIUDq3Xeqi9Pn4uoY5dkBY7+TFDPQ/1PvqSJ7QfRwpdFjuvGLsoJE/ocAlr7heFch56iHZWrPxUwj0vdSZ4JJCvKdkSfGh3LFHEK4ZxDcJxXxoSzGIHJL4LnSCR2DZxghdChiEgfeiYtGLsfFns+Lu9r5uLPMaOTkmBdcQI0SJnyMmB2O0iPx/WIUNAp6WJpU9YlhVbilEeN9yyzCE16g8JRf01lcUnWM+Nd/JE8Eg3N+VNThApwnXydR3l5xOJWD+9+yt7fcbCFauxBoPQu1iZPBHJ5aivu95zNQXRUiySxDIrNtwbsPilJkoxdmm2wF0C4kmk4smCaV+nxs6LX7s1sUUwITEhwq6bodV31BREjJ0Qj+fLHSLERQlzJ0+aKFnsklNEV2wS33WEzsyUXLEsKwrP+mefxf6Tp8Nx6JDmcdTL0yrTIVi6Qs0k1dsFw98cXjF2/HaPKjPZ1/6AVMculGxFQ5BCLequWEr9PAiLXQSVlLcrVt9+xjD6d4aD/FBRsdjJHoc7vfC/GK1kl+4k7MK5BIr3V5Dz5KZa8exFY/HsReNCXwCh10KEXTeDUVjseFesIOxcUt03Xx/KgsXCVy0pt6w0BwDYZ8+WMkLNkt/fxmdAG3NyQMldsfz+rt93AR4POrdu0zyO+sasdeML2YWpirELxWSnDl4Wr4GqV6yvm7T8+ouWx1AEJv9n0i9c5JaboA8XEKuqWK++Onbc6xFNntDhEtZCYbGL4aeffHnRjrGLlCs2WjF2iS7slFbyMCx2YX7JOmtsH0wZmBV4IIGgggi7boas1q2YsSW6YvkYO38WAVNAi51S2NnGjEHutQsAAHRRPxgLuR56Bj4r2piTA5qVmr0LlixB7LFO7TYX6g864UbduWMH2MYGAHK3sM/T8TE3fy5sOC5QlbDjX2dZweUcWKypBU4kkjgCjo9yDJnVRCvqCWqdUyxi7EJNnqAV/TtjZ7FTxNhFIXkgkq7YaGfFJrqwU9aODP0axKp7BIGghgi7boaWxc6kstj5+xAxGYUYO+0/vZZgMaWmAgBYmoZ1wkQAsozcnGxF8oS7ixNyonu2qVnzOFpZsV179+HwBRei6+dNAABXlzPg+WihrmMXWoFi7rc63oiFUlz7E9HCJsHyGMoNV7zJ6tw3Fs3Di7OleoJax6B9CLtILkd9LfXOHUzniUgS7Tp28rIc4U4vfakJbx6tOQEgPSmxhV2k2qopCxQTCLGDCLtuhjLGjvstfIA4eIudvxuHEGPnqxaRlrCT36iF4wsffsacHFCQYuzcra3cPoLYa9YWdup7qsFAwfnHYe4xLwpdLrfm2EAIp++v+XzgObhJRKEkzClzNwvr9oVwjaRyJ8F/vAd7k41F8/CS7GS/x5AslQxYlvXbKzZUIuKKjeHdVn6oRO4VC8hLbURund0peUJ+1uFcg2i39yMQfEGEXTfDn8Wui7fY+StQKtzYfIk/dYyd/DgMy4rJA8JmY06OZLFra4e7gyuHkjz+OO41H8LOKyuWosC0tXOPBWEXYmyaYKETBVXILcXgVXaAhbL3q7+btPB3EJInQikcS1PK4wdCq9BzpCnOkoSdvxg7+RcB9drCRf3+0SveEyF5IjpZsfLHieiKlR6nJriwi1S5F0VpHXKnJcQQ8nbrZsiFnfDBI2XFchY7f1akgOVONBIWhPkYRh4vxW3jhB23T+vPP4PhRVTyceMA+BZ2XjF2BgpMG2ftE+ZzhxgjJ0wdjgtQql/H/4Y0p0f2N/BnHZVcsfpq+2lhtxphoIAMnQUrY1GnrVhhsfM+hlG02LGqLyKRW4OXsNN5rspesZFbTyAoipLCF6JusQtvrmi4YrtVjJ2YsBQZyydAXLGE2EKK5HQzFMkTQoFiVecJfwJCEHYGAwUDpZwP0LbYiS2+WBY0q3TTGLOlGLu2zb+AyRjPHcdu5/ZpbtJch1bwu6etjXvMz+dSFWLWi1eMXSjZqOrkCWEKFvB49Ao7pcUulPtEVooFK66ahMzk4IVdLFyxWuckWKQ8DKt4fyVEuRM6PhY7gLu5s4hSjJ3CIhimxS5CwkYxZzcSdpEStsQVS4gXRNh1NzRi7AT3qlDHzt+HiJlPnqApCgaKUlhUAO0MRuFeyDAsWFop/OjMTLFAsaOiEkwmH8OXyrWcY3xa7Lyfi65YRrDYCe5Qn6ejPTc/XtBf4WWjKp+zgNJi5+daC+IjHIsdAEyT9RIMRLSD9AGgX6aUPNHU4fLa7ttiF0FXrGouveI9XskTwvEYlo26xS4RO0/IC6snvLATfodrsSPCjhAniCu2m6FZx45XMk4x+9L3/vIYO63PGrdGf1bhA45hWTCM8jWKpkEbue8HDEWB5dciWuyCyIpl+MQLg5CMAaV1UC+ChU8othzKR6o6G1aAYVmFBc6fFUrYJFzTaMW8yZH/7aPlipXXsvujvt17Dfx7zONhFDF2kVyN+j2uO3lCEWMXwQXpQEzI6SbJE5EUI60Ot/g40YVdpOr4KcMiwpuLQAgGIuy6Gf5i7AR8lTKRjzVQlOaNX8t9KbliASF9Qr4nbeaFHQxgeJFnShVcsc2Kb+sC6uruFEWBaedcsUarVTXW5+loQqkEVTg9WtX17FgWorgNJNQi4YoNlkjGWunhcH2H12tyi12nrM2dxRi5jxt1AkIoWcOxbmYuWrmjIuykx+EWpvb1pSYcWrskYReoX3W8CbZ2pC+IsCPEi8T+DyN4Ie+7KdyY1KVL/LkuhQ9Vmo+xU6NVTFf4gGIU5U5k2/lixQxFgTFwws7ICzvW6QTb1eV1HLmlSxSOfIyd0aYWdsF9KsqzeNVr1YtktYBiDnnyRKAbdKRcscGg1XM3GuTaLQCAYQV2n2vwMCzaeUuN1WRQxLeFS8gWOzp+N9toCjtK8UUpvPltZs4iazX1zttDpIQtccUS4gWJsetmsBoxcOobpj9Lklm02Gl/2GjVsRMeyoPhFbFcFjPAAgxlAGMwAAxgTLLBZTIBLhc8zc0w2GyK4yjKMwjCkY+xMyWpxwYr7JTnElr9OOWxhecspOSJgMJOtByG2BotBMJtY6SXD64/Aa+vP4y/nFjicw1uhkUbL+xSLJH9qFGfm95TjWfyhDp0IvLzc1/8wj2ve04Zhk0H6zGuX0aEVta9EC5fuG+PWBQLJxC0IMKum8FoJAR4W+x8f4gIY7msWO9xWm5D0WLHsoCmK9YMOACWosBSBgAsaIMBdFoaPHV18DQ1wZSfL51Dezua3nwTMI/i9hcsbHyMnTHJBkgevKA/YAUhF5XkiWAsdpQkcOTPo4khRu6foswk3H/6cM1tgpuUkVnskiMs7CJR7qQnxdgJ8zMsG7YrduqgbEwdlB2ZRXVDIuWKlSfqEFlHiCW909bejZHci9JHhTpmxW+BYllWrNYodXKEfD6G1XbFGi1cKQ7GaOLTHribF52WBsA7gaLtp5/g2L5dml9w3QkxdslJivGhW+yE5IkQYuyEkg/8ZMIMLCvNq1fYhdNSLFgSwUogvB3lFrtkc4SFHRWisItjVqxwtGj0igUiJ0iiid2a+LYE4eqFexm1vB4EQiwgwq6bIQo72Wvq8gl669hpZsUyvgsUe2TlK+TCz8i7WancXNFKphB2fAJF2w8/wlVZCdexCrG7BDcXf26CKzZZqpPGrcXn6WgiWuwYYf3B7c8dU3mTFObkhB03JlDZCuG4YhJHDISdInYx1iYpHqOsjl27gzO9RtwVG2odO9mboSfF2MnnT2Rhl2pN7IxYAF7dZkJF+HyQx0MTCLGACLtuBqsR46a22Pn7QBIyE000pSk0PBrJE1IyArxaigGAOZdz29jnzpXFtUEm7JrQsOJ1HFmwABV33wNXRQUoWUE+2kBxPUX55AmzXSnsWN5Fqxdh7YywlnBaiqnibeTlTgJ98IsWO3F80MsIGqXFLvrH01yDLMZOcsXS/nYJGrWo1nvjNMWzjp1BiLHrvcKuO1jspPCLMOdRWfsJhFhBhF03g9WMsVOVO/Hj6jlxUDZOHpKD+ZP7+4ix81OgWOGKlW2nuZu2oaCP9BolWeycBw+h5u9/BwB0bNoE17FjYtswYSzb0SH6gY0pKYo1Nb/3Hjp/2+nznNSInSfCaSmmukmKrlgAHU7OChUobkzsFRvTOnZy90+8LHaCxZSRXLEJYrGLZ6ZiLGLsuN9RmT4iJHqfWEBWozNCFrtEFtqEnklchd2yZcswceJE2O125Obm4uyzz8bevXsVY6699loMHDgQNpsNOTk5OOuss7Bnzx7FGK4Po/Jn5cqVsTyVmKHVGUIt5Px9kGSlWPD6Xybh1FEF2uVOPN5iSN6iS90rFpCV9WBkYk3mim1YsUJxjM4dO8S2YcL+Ht4NC5r2jrFjWTj27/d5TmrkWbzc89Atdt517Fi0dHLdFlIDWB+Ew4pZsbF2xcYtxs7bYhdpS02oMXYmRVZsRJcUEMFqGS13pCjsEljZBfqfSQSEqxcpix0RdoRYE1dht27dOixatAgbN27EmjVr4HK5u6lp3gAAcYxJREFUMGfOHLS3S9Xsx48fj+XLl2P37t348ssvwbIs5syZA4/Ho5hr+fLlqKysFH/OPvvsGJ9NbNCqzWZSBZHptwhoWey8Ew7krlhxTw3h53IrxRqdnqZ5VE9dnULY0RQFpo3vOpGSApPFololC4+P1mRaUDIhyq1P964ivlyxLAu0dPHCLoD1QXLFhr6OYEmE5AnJYseizRml5IkIZMXG2qL55Plj8PfzRqEoMynw4BDoHq7YxLfYRSwrVvzwCHdFBEJwxPXr0xdffKF4vmLFCuTm5mLLli2YNm0aAGDhwoXi9uLiYvztb3/DmDFjcPjwYQwcOFDclp6ejnxZSQ1/ME4nWKdTfO5p926LlKiI5U5kNyiTUfnJoddSoy5CrKxTp9wGqAoUyz6tRJejzGJnoCgY0iRhl3zSSQAFtH//A7e/3GIHKb6OTk6G0aL88KdYFp6mJl3nJBwbkFnsdO8pO6baFStY7AC0dHJiJZDlRbRcxTIrVtFSLOqH87EGbhHuKJY78apjp/MrqjGOFrvjB2Th+AFZUZs/kV2xpXl27K1uxZ/H9433UgKSn2aFgQIK022BB/tBXeScQIgVCWUXb+atMpmZmZrb29vbsXz5cpSUlKCoqEixbdGiRbjmmmswYMAAXHfddbjqqqt8fiOvf/kV1D3/vPi8yuXUHJeIaBYoVrdX0nmTUzeM9zDaiQHymDUti6FwPKFXLcB9qBnMZvF59vXXofXLL0VhZ0yRrBYGt0vsOmGw28WYPXE7WHiam/SdFGTJExrXSi+0QfmhLMzAyFyxgdyLwvsvnNZmwaL+m8YDo9grlkWHOzpZscGEHyj268HdAMS40ARUEqsWnYAjDZ0ozffuVJJo5KVa8fVtJyMrxRJ4sB+IK5YQLxJG2DEMg1tuuQVTp07FyJEjFdteeOEF3HXXXWhvb0dpaSnWrFkDs0w0LF26FDNmzEBSUhK++uor3HDDDWhra8NNN92keaysaxci86orxedJx44BQ4dG5bwijZ4CxXpv6PJhRgMFJ6SuCnJxqN1SzPsGKQgYYX/b+PGAwYDkE6ci6bjj4Ni3T9xulol3tr0NTBufgJGS4nVjotggXbH8byERJBSTnb9esfpdsdxvVwwLFCdCGyN5jF2bQ1+iSbCoz013uZM4ZsVGm0SuY5dkNnYLUScwICcl8KAAiOVOwp6JQAiOhBF2ixYtws6dO/Hjjz96bZs/fz5mz56NyspKPPXUU7jgggvw008/wco3i3/ggQfEsePGjUN7ezuefPJJn8LOYDYDMmFIq+qmJTJadeQoihItboB3g3RfUBrWHbdGwoHcAsZqdZ5QuRwBzmJnLCnB4B9/AG3nPtDNxSXidnOWzCXV3g7noQbuWCnJ3qUsEJorlglDUHm7YrnnnMVOcMUGyIqNiyvW++8Wa4S/H8NGr9xJqDF2pjjWsYs2wv9svJJmCEqIxY4QLxKi3MnixYuxevVqfPfdd+jb1zsGIy0tDYMHD8a0adPw/vvvY8+ePVi1apXP+SZPnoyjR4/C4XBEc9lxQatAMaC02mk1ZtdC/nkjb9yunl8sUMxq94oVHjtlFjthPmNmJigTZ9kyl0jCzpQjCTsDw4iZs3SK3evGJLfYMV1dOHbb7Wj6n++/v9RSLPQYO1/JEwDQ6tBnsRPW4YqhKzYRyp3IW6m1O6PTK9brPaLzVLlWetLjnkSk6q8RIoO8QDGBEEviKuxYlsXixYuxatUqfPvttyiR3fj97cOyrF/Rtm3bNmRkZMBiCS9GIhHRKlAMKG90M0pzdc2ljNNTWuwUsVqiBQxihWKtrFi5xU5LVBhzc2BI4mLrzNlSL0qaZUSLnJYr1sCyYPi2ZG3ffouWzz5D5b33gpUlayjPi/vtCUNQZSRxoi2d/61wxepMnhDW4Y5TgeK4x9hFtY6d6nkQf+PMZDNoA4WUCGfqxht1+AAhvnSH8jOEnklcP9kWLVqEt99+Gx999BHsdjuqqqoAcBY6m82GgwcP4t1338WcOXOQk5ODo0eP4vHHH4fNZsOpp54KAPjkk09QXV2N448/HlarFWvWrMFjjz2GO+64I56nFjW0ChQDQLtTKv9yXP8MXXP5ynxVb5PKnfhyxXK/A/VEpSgKliFD0LltG8wFBcBhLhvZIO9CYU/xamKucMXKblqOAwdgHTJE4zjcb8FiF0pLsb+eMhQnDc7BjKF5itcZlpXF2AVZoDjGdezi3nnCI7liI26xC9EVCwD/unwCmjpdSEtK/NIbwSBckngJeoIS4QsO+WsQYk1chd2LL74IAJg+fbri9eXLl+PKK6+E1WrFDz/8gGeeeQaNjY3Iy8vDtGnTsH79euTmclYpk8mE559/HrfeeitYlsWgQYPw9NNPY8GCBbE+nZigFWMnpyQ72asThS+0Mms1Y+z46eTlULSSJ4QkAX8xPvlLH0bnli3oHDsa2LCBO3aSVFbAkJLiFSNoYFkwHR1gnU6wLpf4esfPmzWFndwVyK/W53p8UZBmw3my0gxKi51QoFhnHbsYFihOBFesvI5de5SSJ9RxmMFc2nH99H3x6W5IiT5xXggBAPcZkmSmI5KIQSAEQ1yFHSurZaZFYWEhPvvsM79j5s2bh3nz5kVyWQkNE8AKNXOoPjcsoF2yxKPlipUFw7Oa5U6Urlh/FjLrkCGwDhmC8iNN4mtGuxQTaEhO8Y6f4i16nuZmMB0d4usdP/+MzEvna5yXb+tjqCiSJ7qEbgqBYuy431ru7WiRCOVOBGHuksXYRTp5wquOHXE/iv935FokBmk2E3786wwkmSP73icQApEQyRME/WglLwDAoj8NxJi+abhp1mDdc1GaFjvveDBBaHkYVnSaasXguYLoiSoXHaa0VGkDy3i5YoW+sp7mZjDtMmG3ebPmlwMxxi6MXrFqhCm4AsU6XbHqrNhYxNglgCtWsKa1drnE0IFoumKJhYrjnHF9MaYoHcMLUgMPJsSEzGQzrCYi7AixpWdFD/cCtAoUA8Cdc4fizrnBzaUZY6eRHCFawFjZdo15RJejDiUlH2K0SkkuxpwcL1es0KVCbbHzNDbCeeAALIOVYlbuNtW7nkAIc3Q43KIFTr8rVrCyxsAVq+EijzXCe6mZF8AGCrBF+OamlZXd27lt9hDcNts7NIFAIPQuiLDrZmgJr1DRyorV2qZIrPDjig0mlkxucaEpCiUffYSOTRthnzsX9L5axVixfElTk0LYAUDnbzu9hJ362kTiti/M2cxnxNIGKqCLRXLF6he84WJIgJZiwnupuYMTdskWY8Tdg4lQiJlAIBASESLsuhnhtMlSo1WgWHObrC6clis4lOxPhcXFAFhLh8BaOsRrGwCY0rmes56mZi9h5/zjD79zq88lVIQ5msXEicBiRZ4dCsSmcKxaMMcDtcUu0m5Y+TEAUieMQCAQ5JAYu26GrwLFoSCfw1+WoTxxQIyy82ex03Gn9Rfkr35u4rtUyF2xxjyuDImWsFMfnm1vg6e1NeCa/CFM2dTJ9RUOVJwYkGcLxy4rNhF6oaq7mEQ6I1Z+DIBY7AgEAkEOEXbdgE6nBzUtXdyTSMaNafSDFbdpumKluDVKY6zUYUHHseXCUF1sWS3sMtIBKF2x1mHDAOiz2LWs+hBHF98YeFF+EKbUW+pEvg7JkhnWEnQhP/d46R313y8qwk4RYxfx6QkEAqHbQoRdN+DK5T/jxL9/h9pWRxRj7FS14zQSK3y6YvldhVgyPa5YfxYXtQuRtnF17jiLHVfU2DpcEnbqzFj10SnGg47Nm8F0dgZcly+ENTZ16MuI5fbhfsejpZiBimcdO+V7KSXCpU4AdSFmouwIBAJBgAi7bsD+mjY4PQyONHZELcbOq/acRtahh/FRx05lmYq0K1YoYCx3xVqGlAIGA9iODrhruWQLR1kZ3I2NXi5PimUBhkHX7j1w19Z6xenpQUqe4ISd3RKExS6WyRNikdr4iR0vi10UWneRGDsCgUDQhgi7boDQlsnpZgIWKA4G+f3Xn8VO/lhyxXoLP2eAlmKK+f1YXNTCjBaEXVMTmHbOYkenp8FUWAgAcP3xBxyHDuHgmWfh6HXXa2TFcotuWb0aB2bPwdHFiwOuT41X8oQOi51X8kQMfIaC2zOeRVGNqoJ9UUmeUHwpIcqOQCAQBIiwS3A8DAuHmxNMTjcT0dps8hn0xNgBPgoYq5In9GXFyvdXblMnctBJSQCUMXaGpCSY+/cHwLlju3buAjwedG7fDlboK8sjzNa4ciXYri60b9go9Z7ViTBHcxAxdsIllJJKgjpkSGQmm/Hk+aPxjwvHRv9gPohJjB1xxRIIhCBYsmQJKIpS/AwdOhQAcPjwYa9tws9///tfn3NeeeWVXuMToRMWKXeS4HTwLZkATtgJ97BIxE/5r2Mneyx74tGoUCxsFixTepZG+xCO6nUBAJ0sCTuWj5MThF37Tz/B+ccfMPDiDwCcZWUAMsTn1iGDgV3gsj8AgGXRsWUL7DNnBl6oak2CyA4mK1ZsKRYjy9KfJxTF5Di+UL+X0nRcq2AhnScIBEKwjBgxAl9//bX43GjkJFBRUREqKysVY1955RU8+eSTOOWUU/zOOW/ePCxfvlx8brFY/IyODUTYJTgdTo/42OlhYOLNW5G4mRn8uLMU8Xeyx4JIUbhiDSpXrK7OE36SJ9QxdilcE21PQ4PoC+aEXT/uuH+Uw5AiNdp2lZUBeRPE58mjRwOrlMfv+HlzUMJOfUqpVv2uWEEMx6uuXKxR//36ZyX5GBk6ynI8veO6EggEb9qdbrR2ucTnZqMBFqN2KIrRaER+fr7X6zRNe72+atUqXHDBBUiR3Vu0sFgsmnPGE+KKTXCE+DpAFWMXkeQJ6bHfzhOyxx7Gu6RJKAWKFYV01ULOh7BjnU6wLr5NVVISTIIr9tAhuI4eFcczjY2K/U1ZmWLdO0Mq10ezY/PmgGuUoz4jm44YNvWfqLe4DNXxmiXZyRE/BkVR4nuQWOwIhN7L7H9uxqglX4k/L3xX5nPs/v37UVhYiAEDBmD+/PkoLy/XHLdlyxZs27YNV199dcDjr127Frm5uSgtLcX111+P+vr6kM8lUhCLXYKjsNi5GVhN3E0zEvcyf5mpypZh0mPRYqdhcQsm+9NfHTuv5AmrBZTVCrarSxqTlAQrHx/hOHgQdAbnejUVFsIA7/InKSefjKb//hd5996DyrvvQdeePfC0toK22wOuVWuNavGiZ5/eEuSvfi8VR0HYCcdhPGyvEcwEAsGbNTdORGFhH/G52aj92Tx58mSsWLECpaWlqKysxMMPP4yTTjoJO3fuhF11H3j11VcxbNgwnHDCCX6PPW/ePJx77rkoKSlBWVkZ7r33XpxyyinYsGEDaDqOCWxxOzJBF3Jh5/BEOHnCr8VOeqxlsdPuFas/Y9dfVqzaZWmgADozA+4KPgbCZAJlNsOYlwc6Kwue+np46uoAAHn33QvquY8U+1MUhbz77kXWwoUw9+2DuhdehKu8HJ2//oqUk08OvFjAS0mrMz+18KrH10sEiFzY2S1GZCWbo3Ic7n1DhB2B0JtJNhth15HMJo+VGz16NCZPnoz+/fvjvffeU1jmOjs78fbbb+OBBx4IOOdFF10kPh41ahRGjx6NgQMHYu3atZgZRKhPpCGu2ASnXZY84XIzUStQTHuVO/ERY6eR4ak2ROkRMFoFjqXn3m5hY7qUDCEkSlAUBevIEeLrlM2GlBkzYO7bR7E/RQEGi0V83TZmDACga/fugOsU51AVQTbpaCOhFri9xGCn+JJQnJ0ctRg44ThE1xEIhGBJT0/HkCFDcODAAcXr77//Pjo6OnD55ZcHPeeAAQOQnZ3tNWesIcIuwelwKJMnYhVjpyxerJEVC7kwC97l6C8r1lvYAXRmpvRclgFrGzFSfGzu2wcURSFl0iTV/sr5LEMGAwAc+/aD9XjE2ngsw2h2p6h9/nm0fvyx3zVqoRY0vdEVG434OgHhehKLHYFACJa2tjaUlZWhoKBA8fqrr76KM888Ezk5OUHPefToUdTX13vNGWuIsEtw2p0+kiciUqBYJq5obzElR6xVp+WKDcHl6Hd/DZFJZ3pb7ADAOlISdqa+XJkPSz9luQ/1aiyDeWG3fz9qnn4aeydNRuf27aj9xzPYO2EiOn/bqRjf/uNPAONRvGYKxRXbS4Sd2mIXLeTt0wgEAsEfd9xxB9atW4fDhw9j/fr1OOecc0DTNC6++GJxzIEDB/D999/jmmuu0Zxj6NChWLWKK7HQ1taGO++8Exs3bsThw4fxzTff4KyzzsKgQYMwd+7cmJyTL0iMXYLTqUqeiGSMnbLzhEpMaYz1QJ4V6ztGTpfFzk9WrNd8FAVjhrbFzjpCcsWa+vbVtR7rkCEAAMehQ3DX1QEeD1o++xwtn38OeDxoW7cOtlGSYHRVVsKQP1Axh77kCf/Peyryv2dxFEqdiMdJgPZpBAKhe3D06FFcfPHFqK+vR05ODk488URs3LhRYZl77bXX0LdvX8yZM0dzjr1796K5uRkAVyJlx44deP3119HU1ITCwkLMmTMHjzzySNxr2RFhl+AoLHYyV2wk4pb81ZJTiyEhUF1MnpBtU4eb6REw/oShViKHL1esKS8XxpwcuGtrYS7ihF2gS2MsKIAhJQVMWxs8fGmU5k8/FRMwHPv2iWNZlwvumhpAVaZIvUYtvFyxvUSAyEVvUWb0hJ3wHu0ll5VAIITBypUrA4557LHH8Nhjj/nczspirW02G7788suIrC3SEFdsgqOIsZMlT0SmQLH02F+MHRCgHEoILsdgyp0Y/LhiASD55GkAANu4cV5za81PUZTojhUQRB2gFHbumhqAYWBQJU8YdSRPBIod7KmYjHKLXfRcsUbRFds7riuBQCDogVjsEhxFuZOIFygOIsbOj/XJYqJ9bvN3bIriGkkEyoqlKMCYIRN2yUqxUPDww8i9+WYYeZO6t5DzPr5l8GB0bt2quTZneTmYri4YrFa4xDYzamEXnHjVWldPJclsxAOnDwdNATn26LkkDMQVSyAQCF4Qi12Co+4VK5iCI3Er82ex0yOOBKyqgpB6LVNijJTasqUhIn25YgGAomlR1HFr9S9SAcDCx9kBgHXUKOVGhoHjAFe9XBB2lFLXwUQKFPvl6hNLcOXUkqgegyauWAKBQPCCCLsEp13VK5YV69hFInlCZrELIIb8uWKtKoud3kK8wvEDZY8aKErsLAF4Czs16qNTGjLYOnwYd6zMTGReOp970WiEdfRoAJI71sUXRVZ3s9Dlbg4gWEOh9dtv0eHD0tjboIkrlkAgELwgrtgEp0PRK9YT0Rg7+f1QXaBYLYb8Za6q+6bqFZ0GAwBP4Fg0igKMfix2XvOqDq+1HNu4cci79x5YhpTCOmwoLEOHImn8eICi0LVjh0zYVfB7KIVd53ffoGNYfyQdd5zudYT7N3PX1uLoosWgs7Iw5McfwpusByD2iiVfTwkEAkGECLsER90rNloxduqYMfX0arGmsNgZVRY7nTdaXzFSWsLMkJoKGI2A2w1DcgBhF8CtzM1JIVNWWXzAh1xtosb33gMAtK5ZA7AsHPv3c3Ookican34aR60sBv/wPSiZsnDV1IBOToYhOTnirlhXVTXAsvDU1YH1eEDFsRdhIkAsdgQCgeAN+a6b4HSoyp2w0SpQHEAMqd2Icoue1RRejJ1WQWJ11ixFUaAz0rnngVyxQcQHqhHq4rmOHUPD66+j89df+S2q5AnWA099PRz7pdYx7ro6lM2Zi4NnngV3Y6PXNfQcPaJ/IRp4+PpJAMB0dIQ1V09AsDJHq2UZgUAgdEeIsEtw2r0sdtzjyMTYSY+9YuwC9m+VHoeSFQtIgktLB8proQnzCf1ig42xC8aiYxsxAgXLliH9z39WzmFWNrI3MFzP3I5fNouvde3eA7arC65jx1Bxx51ecXmNy5eDdbsRKp6mJvGx0AatNyNYhntRTgqBQCAEhAi7BEfdeSKSrthgLHZqoSffbAtR2ImuNI07s/x4wmbL4EEAAFNRP7/zBpPRq0X6OWcjf8lDioQNU1aWYkxSMbeGjs2/iK+5ZBa59p9+gpN344rraGtD5/btwS1Ghqe5SXzMtLV5bWdZFq7q6pDn726QzhMEAoHgDRF2CY6884QjwgWK5VOoe58G7pogPTfRStepXlesr6xY9WvCWvKXPoKS/30A27ixAeZVrTSEGz9F08i4+CJpPSkpiu1511wNAOjYvFl0jzuPHlWM8Rw7plwXy6Lt+9CTHgJZ7GqeegoHTp6Ols8+C/kY3QmDGGMX54UQCARCAkGEXYKj6Dwhj7GLdIFir6xYJf7KoVAUpSh5ondtwo1ZSwjKXxMe0inJsA4fHlCoeVnsdK3Gm6xrr0X6hRei4G+PgLZZFdvSTj4JlMUCT309nIcOAwBcRzkhZ+MzZd3lf6jWwaLt++8Vr7X98CO69uzRtR55jJ1Hw2LX8OprAIDqJ57UNV93R3hPkhg7AoFAkCDCLoFxeRg4PYz4XO6KjcStTGllU2/zn9GpvpnK3bH6s2K1j8XNIRd2wZ2tenioIthgsaDg4SVIP/980DabYpvZaoFtzBgAQOPKdwAAriOcKzZ13jwAAMv3oRXnAwvH7t2iu9R17BiOLFyIIwuvVfQg9IXeGDt5seaeDE0sdgQCgeAFEXYJjLzUCcAJvagVKFZZ7ALVYFM/tyqEXXhZserXghd24cXYaWFUWeyMBgqZf7kKAND4xn/Qtm4dnLzrNen4yaDT02FgGcU+1n5FAICDZ5yJ+ldfRde+fQDLwl1TA3dlJY7dfgeqn3jSp8hTZMW2K7NiWY/0XjFmZ4d4lt0LUu6EQCAQvCHCLoGRlzoBlFmxkS9QrN6mPIAxQAFji6zkiV7RKYzTTJ5QxNjpmk62r//noUAZlSUfaQMF+/TpyLj0UgBAxX33g+GFl7lPH1hHjgSlyopNP+UU0NnZYFpaUPvsc3CWlYnbGv/7X7R8+ikaXnsNTStXaq5BYbFTuWLdVVXS2lJTgz/BbggRdgQCgeANEXYJTLtDabGLVoFiAxW4SLBJ1Q9WfXh5kWK9rbP8udKMYVjsvGPswr9WpoJ86TFNidcu947bYUhOhqeuDgBAZ2XBkJwM68gRXkWNk8eOxuC134FKSgLrdKJtnRRv17zqQ/Fx9bLHUffKv+CurVXszzTJLXZKV6zziJSRyzgdIZ5l98IgxtjFeSEEAoGQQBBhl8AIFjtB40S+QDH3m6IoDfel8rnJT69YQNlWTH9WLD9e484st+IFe67eXTOC218Lo1VyxSrcxFYrUk4+WXxu6tsHAGAbNcrLFUtTFCijEZZBXNmWji1bxG1yixvrdKL26adx+JL5CresMsZOabGTCzu2syuoc+uuEIsdgUAgeEOEXQIjxNil2UwAAJeHhZv3xUYyxs5AAXaLUbVNOda75Zjyubz7RLBZsVqu2MjG2EXAuil7bFIpTfucOeJjc18uji5p4kQYU+2KccJ5WIYM5l5glMIPAIpefgn5jywFZTLBdeQIXOXlAACWYeBpaRHHqS12rnKZxa6rdwg7+fuXQCAQCBxE2CUwgsUuI0nqeuBwc2IgEjczucVuVN801TaVxY72Xw5F7orVuza9deyCPdeoxNj56aubMu0kULxFz9S3LwAuzi3vtluV6+AvoXXIEJ/HsR13HDL+/GextVnHr1sBAExLCyC33rX5ttgxnd7txuTJFf5oXLkSzZ98omtsvDESix2BQCB4QYRdHDjS0IGt5Y0BxwkxdulJJvG1Lhf3WqRj7LJTLOiXmSTbphyrFnbq40czKzZYi1u4nSe0kM9hVF+LpCTYZ88GAFhHjhBfVwtWWrTYScLOkJoKymIBAJgHDABt56x8Qi08oVetPCMW8M6KdflxxTZ98AH2jp+Ath9/8neKcDc2omrJw6i4514wnZ1+xyYCwnuE1LEjEAgECSLs4sCCN37BeS+uR3WLf5dZVTO3XW6x63JxFrtI3MqE+6GQXDC2KF3c5m2x8y+WFAWKdQo7sVdsIGGnazYJb4tdJFyxMoudxnoLljyEfitWwD5rls/jCucpF3bmkmKYBw4AwMXlCSQdNw4A0LGVF3ay+DrAOytWabFTirLWNV+D7epCx+bN8IdHqLvndsNRdtDv2ESAdJ4gEAgEb+Iq7JYtW4aJEyfCbrcjNzcXZ599Nvbu3asYc+2112LgwIGw2WzIycnBWWedhT2qSv3l5eU47bTTkJSUhNzcXNx5551wh9FsPdpUtXSBYYHaVu/sxTaHG499thtbyxvxzmYuvuqkwdkw81Yih5uz2EU6xg4AxvVL99omoLZSebliZTF2wWbFaiZPKFyxwZ6rcnyO3RLk/t7IxYPaFQsAhuRkJB8/WfF38VUo2ZiZCZqvNWcpLkbyxIkAgOQTp4pjbeM4Yec8UAZPU5OGxU6KsfM0NXGuWmFbl1LYOQ5xIs3TopxDjXwOZ9kBv2MTAeHPQFyxBAKBIBFXYbdu3TosWrQIGzduxJo1a+ByuTBnzhy0y25a48ePx/Lly7F79258+eWXYFkWc+bMgYePGfJ4PDjttNPgdDqxfv16vP7661ixYgUefPDBeJ1WQNweLlZKSISQs2rrMbzy/UGc88J6HKxth91ixPkTimDmy404XEKMXSSEnfCbezCuX4a4TT29WS3s/Lhi9Vrs/AW/y8VT8OVOlM+LMpK0BwaBfAnq5AlfqF3M8udWPoHC1L8/cm6+Gf3ffgupp58ubjdmZcFcXAyAi7MTLHaUmbPeyi12jkOHFMcRXLGs0wnG6YTrCNfDlmlugT/kyRmOA4kv7KTkmzgvhEAgEBKIuH4kfvHFF7jyyisxYsQIjBkzBitWrEB5eTm2yMpALFy4ENOmTUNxcTGOO+44/O1vf8ORI0dw+PBhAMBXX32F33//HW+++SbGjh2LU045BY888gief/55OJ3OOJ2ZN8s+341L/rURLg8DF98mzO3xzorcV9WqeP7nCUVIsRhFYSfF2IW/JnUdsGEFUhZnRZPS6qN2P3qVOzGFkDzhJys2vALFysSL/DSrn9H6UPbVDU64ivvJnmdcdhlsE8Yj7dRTYUhKQtJxx3mJ5aTjJwMAav/5T7GmnamwEIDSYicUOrYM5sqoMJ2daHz3Pew5bjwa33hDzL6VCzctPC3Se8+xP/GFnZHE2BEIBIIXCfVdt5l3N2VmZmpub29vx/Lly1FSUoKiIq6sxIYNGzBq1Cjk5eWJ4+bOnYuWlhbs2rVLcx7G6YSnrU368dN3M1K8u/kI1pfVo6y2TbTUuTzeFrvyBikoPsVixFVTiwFIFrMu3hWr1yrmF1WMm8VIY8qALJhpA6YOUral8ipQrJoqFFfs8AI7jAYKg3JTvLYpyp0Eea5yC05+qtUr8SMU/CVP6NlH/dz+pz+h+M03RaucFtk33AA6IwOO3btR8+RTAABTH65Onqe5GUeuvQ6VS5bAcYATdtaRfIyex4Oqhx4C3G7UPPV/4nyBhZ3kqu0OFjtSx45AIBC8MQYeEhsYhsEtt9yCqVOnYuTIkYptL7zwAu666y60t7ejtLQUa9asgZl3SVVVVSlEHQDxeZWs6Kuc+pdfQd3zz4vPq1zRt+y5+DIlLjcLjyjsvC12h+s5kfnGXyZhTFG6WMNOstjxyRORtNjJXnvzmsloc7jF4wqoCxT7y4rVK8QePXsU7p43DGlJJq9t4ZQ7kVtw+kbADQsokyfUiSS+8OeK1YMpNxcFj/4NR29YJL3GCzuuc8U6AICltBQAl5HbvGqVz/kYVZye13aZxc517BiYjg4YkiJz/aIBqWNHIBAI3iSMxW7RokXYuXMnVmr0yZw/fz62bt2KdevWYciQIbjgggvQFUYR1qxrF2LIL5vFnwGffRbO0nXh4sWcYHEDALeqQK3Lw+BoI+cCHZJnV4grQUxEstyJOsYO4MSHWtQBGskTqsNbQih3YjBQmqJOPUfQBYplj/tm2ILa1xeK5IlQXbEhKBD7jBkofPIJ8bllaKnXGAefcGQdNgygub+DMTfXa5zcYte48l00f/opAKBz1y44ysqUFj2WRceWX70KIScSxGJHIBAI3iSExW7x4sVYvXo1vv/+e/TlC7zKSUtLQ1paGgYPHozjjz8eGRkZWLVqFS6++GLk5+fj559/Voyvrq4GAOTn53vNBQAGsxkwSyVE6OTkCJ6NNoJ1Tugmwb2mdMUeaeiAh2FhM9HIS1Vmcpr5AsDRibELPJl3gWLlPvIYO72uWH8o69gFt69BYbGLjLBTuGJ1RutHqlBy2hlnIHnqVHRu24aUE09E9WPLAI2sb8vAgTDYbGDa2uBu9K6T6GlpAcuycNfUoGrJElBmM1JOOgl/XDIfhqQkpMz4k2L8kQULYB4wAANWfwIqATMUSK9YAoFA8Caun9Ysy2Lx4sVYtWoVvv32W5SUlOjah2VZOBxcqZApU6bgt99+Q01NjThmzZo1SE1NxfDhw6O29mDwMKzYNKBTJuzcKmEnuGH7ZyV5iS21KzbSBYoD4VXHTvXOkcfYRcRNHIbFTj6+T8QsdtKcWuVOAu2j9TwYjJmZsM+YAcps1vwiYszJAZ2WBsrGJ4q4XN6TeDxg2jvg5LNoWacTjr17wToc8DQ2wvUHV16HTpO6kDgPHvSqoZcoEIsdgUAgeBNXYbdo0SK8+eabePvtt2G321FVVYWqqip08gVWDx48iGXLlmHLli0oLy/H+vXr8ec//xk2mw2nnnoqAGDOnDkYPnw4LrvsMmzfvh1ffvkl7r//fixatAgWS/j1yyKBPJZOsLgB3q7YQ3Vc4kRJtveN26KuYxeBdQlz6LkvBtNSLBSXoxq5dgqn3EmkYuzk6E2eiIQrVnNeDWEnZMQabN7nS5nNgJEzzjMtzXDyAg4AHHxGrfxxzi03K9y/QkZuoiG1FIvzQggEAiGBiKuwe/HFF9Hc3Izp06ejoKBA/Hn33XcBAFarFT/88ANOPfVUDBo0CBdeeCHsdjvWr1+PXD6GiKZprF69GjRNY8qUKbj00ktx+eWXY+nSpfE8NQVyYdfp8u2KPVzHWeyKNYSduo5dZAsUB55LbaVSH99mjrCwk7n+wpkucq5YWfKE7hhC1fNIWZZo6Vqb+Oxw80Be2FmVpV1y77wDhU89CTo1FQDnjnWW/yFul3eYEDpPGAsKkHbGGWJShltmDdfCXV+Pmmefhbu+PtQzCo0O7v+FbW0NMJBAIBB6D3GNsWNZ73IfcgoLC/GZjsSG/v376xoXL+QCTumKVVrsBFeslsVOFHbu6BUo9oe6QLFa28hdsZFYm/xwwYrYxg7JDVmQFvnkiVDr2EWkRA0ANx9DCgDZ1y5E1dJHkDp3DgBIrliezCuvBEXTqH36H/A0NMDT3AJXuWSx0+owQadyrlhjTg4ce/fCXePfYnf4wovgOnoUrj/K0efp//M7NpI4f/sNQC6c+/cDODFmxyUQCIREJiGSJ3o6bl8WO1XniT/qOVdscZaGsOOVjtMjCLvw1yUIDT26yatAscoZazHKCxRHNnki2HOVu7jNxsgYpeVL0FsXz1+B4nBgZYW3088/H+nnny8dU+aKNaSkgOKte4Y0wWKncsUekFyx4jpTuULVQmatu9a3xY5lGLiOcp0tOrdt879ulo1sMeHqSiA9F1RH4mbuEggEQqxJvFS3HohTLuz8WOzaHVymY7pGCRC1QImE9Ue4x+oSdgHKnVgV5U7CXZnaFRvcuc4bmY9TRuZj2bmjwl+IsAbZ9Q49eSJiy/F9TJkr1pAqdRIRrHBMczOcMoudlpvVwLttjTk5/BjOYte5fTvKr74GXbJ+zl2yIuDmkhJ4mpvR+N57XsWQW7/5BnvHjsP+aSejetmykM9PgHW5MGXHd5hQtRuzK7eHPR+BQCD0FIiwiwEKV6zLd1asIAC1LEJqYRcJw4dgdQvFFevVUswcYYtdGMkTFiONFy8dj4sn9Qt7HQLyFegud6KOsYuQsstevBgAUPDYY17b5K5Y2p4qPebFmmP/AbABakAKY425vLDjLXYNb/wH7T/9hOaPPxbHtq1dJz72tLaifsUKVD34EBpWvI7Onbtw+KKL0blzF1o+/QyswwF3TQ0aXn9DsxxLMHTt3YeihqN4ZOOrGPzHzrDmIhAIhJ4EccXGAN+uWO8CxYB2ZwMvi12MY+y8kidUrlirbH0RSewIo45dNJCfU8gFiiN0ItmLbkD6uefAWFDgfUyZK5a2yyx2vCu2c5d/EUSZzaLVT3TFCha7334DAHjqG8TxQvcLAPA0NMBVfgQA4Dh4EO7aWnRu24aGN15H1549iuO4q6rQ/OFHaFu7Fqzbjby77oRtzJgAZy7RuUOy0nmamsA4nVx9SgKBQOjlEItdDJC7Yrv81LETLHtq65jWa5EtUBx4rFe5Ez+uWK1WacEiF0GJIeykx/GoY6dcCwVTYaGmgFa6YlO9Hnf95l/Yyfcx8a5YV20N3I2NYtKFu5ETds6jR9G1U5rP09gounZdR4/CeZQTeR0/b4bz8GEAknvXcaAMNX//Ozo2bULnli2oe+VfOs5comv7DsXzQJm7BAKB0Fsgwi4GuH26YiUB5GGkHrJarlhLFCx2YoydjrFqK6K/XrGREHZy8ZQIBWiVvWL1/duoh8WieYMhScoCVljs+Bg7li/sLZRJUSPfR0qeqEMXb60DAE8D50ZtfOcdAIB1FBfLyLS3w3XsGABO2AnWO3dVFcAwoLOzYeX7QHf8ukVx3PaffgLD16+U0/jue2h44w2v1zt3qISdLFM4ENVPPoljt90Glgn/fUogEAiJBhF2MUAudBQtxWRZsfIxJo1Mzkhld8oJpo5doALF8ixWpzv8G6Z8TYkg7ELpFau2qEWqQLHfY1olYSe3vgmuWIHkKVM096dl+xizs7kHLhfavv9BfN3T0ACmqwvN738AAMi+7lqxALKrooIb09QkPhawlpbCmMeJxc4tv3KvjRwJU2Eh2K4utK9frxjvaWlB1ZIlqH5sGVzHjqHuxRdR/cSTYJ1OsXuGuX9/APqFHet0ouG15Wj57HM4DniXeiEQCITuDhF2McB38gQjGyMTdloxdl6u2MhZ7PTF2OlP3nBGwhUbRrmTaCA/XzrOrli/x1QkT0jWN7nIA0Uh/fzzlPvx3Szk4yizGXRGBgCg9dtvxNfdDQ1o+fwLeJqbYSosRMr06aAz0r0Xo6pTaR02FCa+f7Nj/34AgKlvX6TMnMkd45tvFeMd+/aJc7StX4/aZ59Dw2uvoWv3bm6AyQTL8GEAAJdOV6yrpkacU3APEwgEQk+CCLsY4KulmFzwyR+bNHx23skT4a8rqBg7dR07PztF2mIX0dpnIaLsPKHTFRsHYUfZ5BY7b1csACRNmADL4MHSuORkmAoL+XFKy55Y8qSiUnyN7exE+48/AgBSzzwDFE3DmJEZcG2WocNgzM3jJ+HDDvoUwj5zBgCg7bvvwHqk/w95WZWm/74vPu7k4/qM2dkw5XFC0V2tU9jJrIjOw3/4GUkgEAjdEyLsYoDLRx07l4bFzmigNMtiRKOOnTCFHuFkUmS9+h8bkRi7BOsDqih3otNip75OsXDFGqzyGDttV6x97lwYbDZQfKIFnZkpxtPJxSAgc8eCE4CUiaux2LmTi7mzlJSIc2ghCEZAsNjlKbf36YOk8eNBmc3wNDaKMXoA4Ni7T3zcJYup69r1u7g2MQ5QpyvWXVUlPhYsdp6mJrR+8w08bW265iAQCIREhgi7GKCnjp1g5fIVmO9dxy4Srlj94smocI3630HdAzcUaFHYJYayi0S5k5gUKLb5KFDMu1QBwD57FvdaZob4WxBIcsseABhk7tyCZY+BzsoCALj47hWmPn0AAMbMDGiRMmMGbGPGwDpmNMzFxTDmKYWduW9fUCaTGCsnxM4BvCtWg67feWGXkyPG7OnNinXJLI/Ow4fRuHIl9p0wFUcXLUbtc8/pmoNAIBASGSLsYoBPVyzjbbHTiq8DAJss6xSIrCs22OQJX6Mnl3BWm/OO6xv+2hJM2CmSJ3RnxSpLtsTCpSx3xcotdqa8POTeeQfyH34YJl5cCe5TY0Ym0s45G7Zx45A6b65ivvTzz4O5pAR9/vE0UufMgVFlmRNduD5csebiYhS/uxIl777LuWx516m4Py8Mzbzlz8ELO5ZhfAo7IemBc8Vy5+Ivxk6e/eqqlISdY+9eVD3yN4Df3rmNdLAgEAjdHyLsYoC8d2mHjzp2Yg07H9mvKRZlLenIJk8EHqsQdj7Gv73gePz6wGyU5tu1BwSBYBVLEF2nrGOn22InPY5UceKAx5S7YlVu1ayrr0bGhRdI23mRRmdmInnSJBS/8zasw4Yp9kk56SQM/PwzpJ5yimIfAIDRKFn6ZBY7QaQBgLlIKfLplGQxUQOQhKF5ALeP8yAn7FzHjoHp6ABlMnlZ+eDmWu8Zc3IUrliW9bYUt/30E/aMHIXGle9y81ZJwo5pbwc8HoDvp+ssK4OnrR21z/0TziNHvOYiEAjdlyVLloCiKMXP0KFDxe3Tp0/32n7dddf5nZNlWTz44IMoKCiAzWbDrFmzsJ9PDIsnRNjFAJfbhytW02LnQ9hZ1cIu/HWJc+iy2AVOZqANFDKTI1P9PxhrYiyQr0NvHTv5dYrVecjr2BlUiRBqBPepLzeqFnIBZ8rLA8WXOZFb8mzjxkljirzbuhn5zFg6KwsG3sIoxOoJrlgHnzhhHjxIrH3nNU+OFGPHOhxgmpu9xrSuWQMwDOr//W+wLKtIAhHIuupKwGgE096Omr8/jroXXkDdCy+K29s3bkTzJ59oroFAIHQfRowYgcrKSvHnRz4JTGDBggWK7U888YTf+Z544gk899xzeOmll7Bp0yYkJydj7ty56ArQtjHaEGEXA+QuV7lRQR6L5q9PLOBtsYt1jJ0eV2wkoRMseUKO3iQIRcmWGP2nUVbtcidapPxpBuisLCSfNE33/PLsV8GNCihdsbZxY2FITYUhLQ2mvn2gxsTHxcm3ia7Yw5yw6+LdsNbBQ5A0fjz3ePhw5Vqys2GwWkGncXGBWu5Y54EybtvRo+jcuhUuPnmCliWF2OfMEWP8mj9Zze13hIshZFkWx26+BRV33gXn0WOIF7UvvIDyBQvBdHTEbQ0EQiLS7nSjtcsl/jjcHp9jjUYj8vPzxZ9s2ecAACQlJSm2p/r5csyyLJ555hncf//9OOusszB69Gi88cYbqKiowIcffhip0wsJIuxigMtH+Q9FHTu3/xg7L2EXgXUFYxUzKix2ETh4ABI5ecLX30hNXFyx8nInKSl+x6bOm4vBP/6A5MmTdM8vd8XKM14Vrtg+ffi4upWa/VuFODtzH29h56mtg6etDe5KToCZioqQedml6PvC88hf8pByHr4Ui+SOrQHr8YDhvy2zLKsoQtz41ttgWlsBAMnHH8+tO4frhmEZOJDbh99XsOwxra3w8JZAeWJHLGEZBvX/fhXtP/yAtp9+issaCIREZfY/N2PUkq/Enxe+K/M5dv/+/SgsLMSAAQMwf/58lPNtEgXeeustZGdnY+TIkbjnnnvQ4eeL1KFDh1BVVYVZs2aJr6WlpWHy5MnYsGFD+CcWBsbAQwjh4itL1M14x9jpd8VGwGInzhV4rNJiF32RQidYjJ2y84S+70Px6J5hKiyEbfx4rpcsTQccH6zl15ilbbGTu2KNOTmia1ULS+kQ7rcsno+220FnZ8NTVwfnoUNw19aKc1EmE+wzZsDd2KhcC/9t25iXB8f+/XDXVOPwBRfC3diAAf/7H1iPB56mJnF8y6efAgAMaWlInjoVLatXI+3U00AZDLAMGoTWr74Sx7pqOJEoL6PiOhqfuDt3ZSVY/gbTueVXpM6eHZd1EAiJyJobJ6KwUPYl0Uec+uTJk7FixQqUlpaisrISDz/8ME466STs3LkTdrsdl1xyCfr374/CwkLs2LEDf/3rX7F3717873//05yvirf+56ligPPy8sRt8YIIuxjg8tGTUquOna83pd1iUjyPhGtPmEOPUJMX5WU0gtQjjWDhikS9vkggv0Z669gZFK7Y2JwHRdMofuvNqM3v02LHl0EBJEuaLzLnz4dt9GjYVLFzluJidAjCjnerGnOluej0dFAmE1iXi3suWOx4127X3r3o2rULANDwxn+QNImzRJr69AFoGi7+27mpoABpZ50Jc78i2Pg+t5ZBA5WLdLvhrquDS1b42Hn0qN/zihaOMskC0fHrr4ptrd98g87tO5Bzy82gYuXvJxASiGSzEXarKeC4U/gEMAAYPXo0Jk+ejP79++O9997D1VdfjYULF4rbR40ahYKCAsycORNlZWUYOHCg1pQJC/kkiAHy5Ak5bg+LNocblc2dAWPsrCaDqs1WJAoUB9F5wigNklsao0VClzvRbbGTHseiOHEskNfDU1jsMjKQ/ufzkX7xRaDT0/3OQZlMSDruOFAqN615wAAAXMkTV60g7HKl/ShKKqSclia6eYWSJ52/bBHHNr73nljvzjJkCAofXyZuY51OUAaDWBgZAMwDB3mt011ZqbTYHYm+sPO0tHgJSMcBSdh1/f67Is6u+rFlqH/lFUUBZwKBEJj09HQMGTIEB3z0jJ48eTIA+NyezyeBVauKo1dXV4vb4gURdjHA7ctix7C47NVNOPmJtahq5mJ7fMVvURSliLOLRPKEIDb0iA65mPHEQNiJ5U6ifiR9KMqdhNArtofoOoXLVZ0YUfDIIyh46CH1LroRixQfPARPfQN3PJX1T4yrkwU9C2Kva88e8TVPXR1q+Iw2y6BBSDruOOTcfBMAIO3MM72PXVIslj0RYhNdlZVw18hdsd7CzrF/P8pOPQ0tn3+ueN1VVYXKh5ag87ffFK+zLIuu3bvBOJ1ec7WvX4+yufNw8JRTFYkajjLZjcXtRueO38S5hIQRVxxdP4zTqagPSCB0B9ra2lBWVoaCggLN7du2bQMAn9tLSkqQn5+Pb76R+mi3tLRg06ZNmDJlSsTXGwxE2MUAp48WW24Pg4O17XB6GByqawfgv5SGXNhFQiicMDAbJw/JwaXH9w84Vm/CQKQwiDF2iaGIQkueiH2MXbQx5uaCSkqCITVVtJRFCnN/rjRK59atXNFggwFGmYtXOD6gFHxi/1kfX6AEN2v29ddj4NdfI2vBNV5jDGYz8u65B1nXXI2UaVyWsKuyCi7Zt3EtV2zT++/DefAgmj6Q4nCYzk4cmDUbTe++i9pnld0s2r77DofOOReHzztfETPYsXUryq9ZAE9jI1iXS+zFC0iZvUL9v45fOcsk09wM8G5pd02t5rnHgorb78CBGTMViSoEQqJxxx13YN26dTh8+DDWr1+Pc845BzRN4+KLL0ZZWRkeeeQRbNmyBYcPH8bHH3+Myy+/HNOmTcPo0aPFOYYOHYpVq1YB4O4Jt9xyC/72t7/h448/xm+//YbLL78chYWFOPvss+N0lhxE2MUAt6/kCQ8rpma3dnFFVy0+YuwAtbALXyhkJpvx+l8m4dRR2t9I5MRaYNFBlGKJBfJl9GZXrMFmQ/Gb/0H/N/8j9o2NFKZ+nLAT4+uys70SQERhJ7fY5eUqxmTfcANSZswQn5sHSW5Wc98+PpNKMi+dj9w77oCpkPt/cFVWwi2LsWNaWsQMWYGOX7cCkMqjAEDVo4+KRZQ7t25VjO/auRMAZ+krv+JKMV6w5ZNPFMK04+dNAPjMXj7GLvXMM7g5+WO66+rE8UKyiS9YlwsNb/wHjoMH/Y4Lhc5dOwGWRed20rmDkLgcPXoUF198MUpLS3HBBRcgKysLGzduRE5ODsxmM77++mvMmTMHQ4cOxe23347zzjsPn6jqV+7duxfNss+Au+66CzfeeCMWLlyIiRMnoq2tDV988QWssrJT8YAkT8QAlw+LncvDoMvFbWtzcB/wfi121sha7BIZ4TIkiqUr3F6xiXIekUBdTy5SmIuKFM+1kjCSJkxA41tvIWnCBPE1U65S2JmL+yPr6r/gyHXXg+nqgnXw4KDWYeRdL+4qZYwdwFntbHzdPKajA127dwMAXMcqwLrdAEWh5WPpZsC63WAZRkxscFVJ8zn27UPXrl2wjR2L9p/WAwAyr7gCDa+/jvafN3MFlWtqwLS1ATSNtDPORNM7K9G1cye3LQhh17pmDaofewzJU6ei36v/Dup6qPG0tMBgt4OiKLAsC08ttw5nOenWQUhcVq5c6XNbUVER1q1bF3AOdXcbiqKwdOlSLF26NOz1RRJisYsBvsqdtDvd4mPBYqfXFZsoLspoQfM3wkQRsKH0ijXEoUBxd8ZgsymSJYwqwQZwtfeGbN6MjIsuFF+js7IAo/S/YepbBENyMvq98TpK/vueV5JGIEy8sHNVVIoxbFRSEgCg/fvv0cnH3nTu+E20zMHthquqGq7KSrBOJ/cHpyiwDgfctbVwHDgAxuGAu0oZi+aqqobr2DE4//gDoGlkLVwAymLhyr4cPCi6N839+sE6cgRgMsHT1ATXsWNw19WL88iFnbuujluDjK49XCePrn17g7oW4jorK8F0dcGxfz/2nTAVVQ9ysZSepibR6uiSWS0JBEL8ILebGODLYtfukCpk6xJ2MotdguidqCFchkQRsKElT0iPY1WguLtj6idZ7bSEHcD1m5VDGQwK657QnzbU946Jz2hzHj0KTz0nnpLGjgEA1D77HA5ffAk6tm5F51Zl6RHXkXKxiLF5QAlMfbl1NKx4HQdPPwO1zzwLF1/4WCgP466uQtt6zlpnGzUKxqwssSVbx88/o4tPvrAMGQKD2QzrEK4GYNfOnXDXKcUcwMXq7Z92MqqfekqxNsdBzp3rqa2Dp6UlqOvRuWMHDsyajaqlj6Bj61bA7UbH5s3ccWWCMpIWO+fRo2j+ZDVYH3GTBALBN0TYxQBfwq7N4fZ6bDb6vhnZIxxjl8iIXTES5B2qSJ7QuahIl6fpDZj7SYk8gerhyRHq3VFWq6JdWCgIrlimuZnrAWgywTpylDSAZVH7f0+jQ1ZeBeCEjfPQYQBc71vBtdz47rsAuJ6zQvaqbdxYAJzFrp0XdsknnAAASJo0EQDQ9tNPaP+R6zSRPIXrlGEdxdX+69q5Ex4NV2zr118DDIMOVeV7Z5kUW+cMMs6udc3XgMeDjs2bxexXV2Ul5w6WC7sjkRN2VUseRsWdd6J9fXwr+BMI3ZEEuW32bHwlT8hp7dIRYycXdj38L5dwLcVkj/UmQihi7BLFp5zgmPkECkBZnDgQJj4z1lzUN2wrL52eDoOsz64pJwf2WTNhzM1FxiWXgDKb0fHLL2jn23vZ+F62rqNH4OR73ZqLi0Xro9A1wrFvn9iyzDaGswC6qirR8csvAIDkE7gSCfaZMwEA7eu+Rwfv9k2eOpXbjy/q3PnbToUr1tPQANblEhMrnEeOivFArNMJp6x1kuNgcK3ROn7+mVtrZSVcfBkW1uGAp75ekY3LNDcrOn2Eg2BhjFe3DwKhO9PD5UFi4KvciZygXbEJIniihTHRhF0I5U4o4ooNGrMOV6wWRr70iqlvUYCRgaEoCll/uUrxmm30aAz+fh3yH3wAGZddyh/UiOxFi5A6h2vx5Sw/AufhwwAAc3EJzEX9FHPAw4Ve0FlZ4jbHnr1i8oGVb7FmLS2FZehQLnbN7YapqEi0/ll5Yde1a5eYPSzgqqwUs27Zri5RdDnLy8VjA4DzoO9emgDgaW4WxR/T0YFOvpsH3G5F5qurstIraSMSVjvW7Razkd21dQFGEwgENUTYxQC3DmHncPvvPAFEvtxJIhNMV4xYEEryBE0sdkFjCtEVa+NdlLaxYyOyjqwFC8TH5mJlncfcm29GwbJlGLj6E+TcuBgmXqQ5j5TDIQi7kmJFvKAcU34+TAV8HB8fk2csLBDr1AFA2tlniY+Tp54gPrYMHAjKYgHT1oaOLUpXcNvadWIiAyAlMzjKlK5XUbQ5HIo6fQLHbrsdB884A12//y7G1Ilzyix/rmMV3sKuXH8ChePgQXja2r1ed9fWikLUXU+EHYEQLETYxQBfWbFamP1YgyJdoDiRSTxXbLjlTiK+pB6J3GIXTAHk1DPPxMCvvtQsPhwKlNGIgWu+QtpZZyLn1luV28xmpJ9zNszFxdya+WQNx/4DcPPJEeaSEoVbWY6psABGVcshywBlL8q0008XO2EIbliAa8cm9LdlHQ7uNYsFAND61VeKOYRkBsFCJxzTyQuqw+f/GQdmzlJY2ViXi3O9ejxo+fxzdPy8WfsCAXBVyIQd/153+bDYeZqaxGxiAOjauw8HTz0NFbffrjmvuF99vdd2AoHgHyLsYoCv5Akt9Nexi71SiOUhJWEXu2P6Q2mxC8EVmygnkuDQqanIufkmZC1cqChCHAiKomDu10+sFxcJzEVFKPz730Uh5QuTUH+Pt5YZ0tJAp6fDzGfFKsYAMOYXcOcmK5RsUTUZN2ZnI+eWm2GfN0/shCGQxCdSiPvymbJCrJ5Q+sVZ/gcAyWJnnzWLe/3IEVTcdRcc+/fz7lWpz6zj4CHR6tf67Xdo/+EH7pz4ci9y5K5YC18r0FdmbMXd9+DwRRejk+9p69jD1f9Tt1wDIGYOA4nhinXV1IDp7Iz3MggE3RBhFwOCEnY6O0/Ew5Cl11IVmWMJdewSRBDJhZ1O8UBRlCgIE+Y8ugHZ11+P3NtuDTwwQTBYrbDK2g6Z+hRyf/vkZJgHDQRlMiHjkkuk7fn5oGhaWaJl4ACvebMXLEDfZ/4BA2+RE0hW9aG0lA5RPLf/aToAwMWLLKFzRfLxkzl3r8eDtm+/FcfL68859kr9dp1lZej6/XdQFgvSzjnHa31yi13SBD6BxIcrVuh4IfTzdfExdJ6GBi93rLzvrDvOFjtXTQ3KZs7CkQUL47oOAiEYiLCLAcG4Yv1Z7OxxttjF0uo0qm8aRvZJxVlj+wQeHAPkBceD6Zsrlm0huq5H0/f//RPJJ54IAEieOEl8vd/LL6P4vXeRcvLJ4mtCfJ1J5o61yNqeBUJtQTTlSfMkHX88Us/gWo85jxwB09EBx759ALjkDPu8uYDBAMuwYbDy88itbF2790BN5mWXwsbX8QMguV0rKsQEDusILsbRVwcMwaUqiDZ5Rw915qurUnLFxlvYOfbtB+tyoYu/hgRCd4C0FIsBwVjs/MfYSb0546ETTAYDuhCbgqFpNhNW33hSTI6lBw8jKbtgBC4n7Fjiiu3hmHJzUfSvV+DYtx+WASXS6336wNSnD1i3G5TJBNblgjGfq5Mnj7OzDPC22PmCMpkAk0l0/dLp6eK2gkeWguHLq7jKyznXp8cDY0EBTH36oPDRR1Hw8MOgjEY0r/4UFXfcoehzK1jsLIMHw7F/Pwx2O7KuuUbRY9YyZAgce/dygpFPcrAMLQUAuBsavNbLdHWBaeesckIMojyj11leDuvQoeJzt8wVy3Z0gGlvVySWxBJBqDKtrYrWcARCIkPepTEgKjF2cRAKdBCWqp6GR2ay8/c3UiPcB4grtudDURSspUM44aXeZjQi47LLkDRhAtcaDFJyCJ2drRBnekg77TRuXpMJqaefhpTp09HnuWdhLioSY/s8zc1o+24tACCJ72YhrAWQkj4Ely3LsqLFLu++e5F21lko/PvjoNPTYeojxQsmjT+Oe8CLOkNKinhMprUVDJ/UIeCRiT2xwHGNzGJ3RG2xU7Zdi5TVjnW7gy7HIlogWZbr2UsgdAOIxS4GuJnIuGLlMXbBiMVIEcsYu0TDI3OnB3MdJFds7712BI68u+5UPBcsdsFY6wTyH3oQdEYGUk89BcaMDBS99KK4zZCcDDonG57aOjR//DEAwCaIMRkmPmvXXVMDpqsLnuYWeBobAYMBtrFjkXy8lKRhzMkWLY6WwYNhSE0Fw7cmM+bkwJCaKloRPfX1MBQWivu6672FnbywsSC2mv63Ci2rPxFdxzAYAIaBu67eZ4axXli3G+VXX4OOTZuQe+cdyLr6ar/jPS0tMNhsCteyp6UVdGpqWOsgEGIBsdjFAJc78skTnU6Pz3HRojeLE7nFLnhXLMmKJXhjnz0bSRMnIvPyy4Le12CzIe+vd/nM2E2dPQcAOKEGIInvjiGHTk+HISUFAGc169jIte8yDyiBwWpVjKUMBrH3ramwUBETaBt/HCiKgjEzE4BSyAGAp0GyuLmqqrgCxDLBJFgMK++/X9FCTDiGr1p27oYGtPHdPwJR99LL6Ni0CQBQ8+RTaPn8c59j3fX12H/ydBy59jrFOpnW4HrsEgjxggi7GOAKwmLnL8ZOLg7a4yDserXFTvY3DKbrh5gV24uvHUEbc98+6P+fN8QyJJEk+8bFMKSlAeBcpUI5EjkURYlFlDu2bUP1358AAKTOmas5Z+4dtyP9wguRNGUKCh79G/KXLEH/d95GwdKlAABjVhYAbyHmbmiUnrhcnEVOVvTYeeQIWKcTYJRfgIUSMb5q2VXcfTeOXH0N2lV9cZnOTjgOSW3TnH/8gboXXgAA2Pjs3arHHgPLaH/hduzfD7azE+2bN8PN9/YFOIsdgdAdIMIuBkQqxk5OPGSC3o4LPRFPEOJcjiHB6vERegfGjAzk3noLAK7AMSWrmSdHaG1W9cCD8DQ0wFJaiqzrrtUca585EwUPL4HBbIalpAQZF12IpHHjxIQCOpsTdmohJrfYAZDakvGxiK6KCoUQA4CkyZMloahRy87T1iZa97p+363YVrX0ERw85VS0/fAjAHBWPYZB0oQJ6P/aazAkJ8NTW4euXb9zczU1oX3jJrG3rtjv1uVC125pbk9Ls+Z1IRASjd57p44hQbliA4in+04dhplDczF3RL7fcdGgN1vsGDZEYSe4YnuxG5sQHzIuugj933oTBQ8v8TlG3umDMptR+PfHYTCbQzqeMVOw2CldsernQgcK6+DBoMxmwOMRCyFbx4xG/zf/g8InnhALVGu5Yjs2bhStfq5jx8TXWZZF27p1AIDGlSu54/GFm5NOmALKbEbyCVyLtra1awEA1cseR/mVV6LmiScBSO5rQOruAQAMsdgRuglE2MWAYFyxgYTdgmkD8OqVE2H2E4sXLXpznJg7iFqEcsTkiV587QjxI2n8eL8Zt/Js14LHHlOUHQkWo2ixUwoxtQWvgxd2xvx80d3auuZrAIBl4CAkTZgAU16uaAF0lh2Eu045p2CNA5QtyFzHKsQs3LZ16+BuaEDHL1xP3aTxEwAAKdOni9sBoPmjjwAADcuXo3PXLslip8LDx9i5GxvR+t13ooWPQEg04irsli1bhokTJ8JutyM3Nxdnn3029u7dK25vaGjAjTfeiNLSUthsNvTr1w833XQTmpuVJnGKorx+VvLf1hKBoOrYGRNXAPRmYecJ2WKn/E0gJBL22bNgGz8eeffdh7TTTwtrLjqLt7DV1YNlGFQvexwHzzhTLO5L8xY41x9c3TxjXi5sY7jCx4J7Vt5azcjP17F5M/afNA1d/L2BZVnRwgcoLXZdv0nt0eB2o/7ll7maeSYTbGO47iAp07j6mF07d8JdWwuad/kCQPWjj/kUdkIWcOX9D+Do9TegYcXruq8NgRBL4irs1q1bh0WLFmHjxo1Ys2YNXC4X5syZg3a+mGVFRQUqKirw1FNPYefOnVixYgW++OILXK2Rqr58+XJUVlaKP2effXaMz0YbD8OKXQvMMmucL7dmMDXSYo3eHqk9kQn9M0Laj2TFEhIZY1YWit96E5mXXRqBuYSs2HrU/P0JNLz+Ohz798PBx6nZRoxQjDfl5iL1lHmK1yyDJGFnGzlCqgnIsqL4c+zbr7LSHROtZ507uN6zBr4sScPrb/BzjRQzfY05ObCO5DpltP3wo8Ki6Ni7F26ZK1aOp6UV7ro60YVb/69/icWgIwnLsj7FpWIMqatH8EFcVcQXX3yBK6+8EiNGjMCYMWOwYsUKlJeXY8sWznQ+cuRIfPDBBzjjjDMwcOBAzJgxA48++ig++eQTuGVZVQCQnp6O/Px88ceqSteXwzid8LS1ST/t7T7HhovcWmczSwHMNpN2MHMiCzu6F1ddH5CTgjW3TsOvD8wOaj9aTJ4gwo7QsxEsXx0bN6LhdW9rlqItGQBjbh6Sjz9ezN4FAPNAqYyKqU8fDF7/E+y8+BNKj9S//BIAIPkkzvLGdHTA09QE1uNBJ2+xy1m8SKzTB0i9bAWswziXcwcffyfAtLfDdawCWjCtLWj57DOxMLOnoQGN77yjOTYc6l9+BfuOnyK6irXHvIx9Eyai7fvvI358Qvcnoe7Ugos1k6+H5GtMamoqjEZlbeVFixYhOzsbkyZNwmuvveY3/qH+5Vewb8JE8efgqadG5gRksCyL9Qfq8N8tR8XXkmTCzmrufsKuNydPAMDgPDsyk4MLLKdEV2zvvnaEno+Q7CBgGazsf5t66qnI/MtfYEhKAigK1mFDQZlMsM/myr1QSUkwFRYo9qHtdlhKuALO7tpadO3ejZbPuBp0uXfcLrp3K26/A3tGjkInH0+XPHUqit9diaSJEwGDwaukjKkvF9vXyRsRTH36iG3LHD76wnqaW9D8EV/w+Tiu4HPDW29pj21tRfvGjSHF4XXwaxJiEb3X0YzaZ54FADS+HXlhSej+JEznCYZhcMstt2Dq1KkYyZvJ1dTV1eGRRx7BwoULFa8vXboUM2bMQFJSEr766ivccMMNaGtrw0033aQ5T9a1C5F51ZXi86Rjx4Awgoa1oCgKC974RVFvTm6l82WxMyewsCPuxOAhrlhCb8Go+kKevWgxjt1yi7Q9Lw95d92J7BtugKepCea+fQAAaWeeieb3P0DS2DGavViNOULsXh3q//VvAJxItJaWwlRYCE9dHdrXrxfHUyYTzCUloAwG9HvjdTBtbaDtdsWcJv7Yzj/+ENdGmc1wHjoEplWZ/WoqLISrogJdv//OxesZjSh84gmUzZ4Nd0Ul3PX1YmkWgdp//AONb7+DjEsuRv6DD+q+hoDUR1feT1dO4ztS/DgVYgYzoWeTMMJu0aJF2LlzJ3788UfN7S0tLTjttNMwfPhwLFmyRLHtgQceEB+PGzcO7e3tePLJJ30KO4PZDMj+IegoNZjOsVvQXs/FYBgoKDJZfbpiEzh5ordb7EKBuGIJvQU6QxmHmnLyNBhzc0WBIsS40SnJoFOkz9zkSZNQ/N67MMnakMkx5uQA4Cx2TDOXwJB+/nkAAFOfQnTt4NyvdHo6jPn5sM/4kygQKYryEnUAYC4qUjw35uWCMpnglNXTMyQng2lvh3nQQLgqKsTzsJaWwty3D8z9+8N5+DC6du9ByolTFfM1/fd9AODF3SUwDxiA2meehXlACdIDxH+7q6vF81XDOp1o+M9/xOfyWEOWYQCGEXsBE3ovCWEeWrx4MVavXo3vvvsOffv29dre2tqKefPmwW63Y9WqVTBpNNmWM3nyZBw9ehQOVTPqWJNrl+L8TLRB4Wbtlq7YBF5bokKRrFhCL0EuKIwFBTDYbDDr7INrGz3ay5UrziUIu+oaMQPW1K8/AMDcp484zj5nDgZ8uAo5Pr7QyzGphJ0pNw/G3FzFa/bZswGKQsqJJypeNw/kzsnCx+k59igLJAOAuaREfFy19BF0bNyI+ldeQfUjfxPds87yctS99JKybZnDISZOuGvr0PS/Vdg7+Xh0bN3KHevAAUWyh3A9GIcDZfNOweFL5nNdPKKAq7ISrmptKyIhsYjrnZplWSxevBirVq3Ct99+ixLZP4NAS0sL5syZA7PZjI8//thvUoTAtm3bkJGRAYvFEo1l6yYnVTq+iTYoskptJu1Ln9DCjqiToCGuWEJvRIhpy+a7WAhFgUNBFHZVVWBdLoCmYcrP47bJrHwpM/6ke046PV2MqQM4V6wxN0caQFEoePRvGPzTj0j5k3JeywAuc9c6dBgA784XgLLIccfPP6PhP28C4JIzGD6WvPaZZ1D7zLPYf9I0dPzKCTe5+9VdU4OWzz8H09yM5v+tAiC5joUWcZ7GRjDt7XAe/gOu8nJ07diBhje14/7CgenqwsGzz8Gh888D64l9O0tCcMTVZrto0SK8/fbb+Oijj2C321HF9+VLS0uDzWYTRV1HRwfefPNNtLS0oIWvJZSTkwOapvHJJ5+guroaxx9/PKxWK9asWYPHHnsMd9xxRzxPDQCQa5cLOwomWfyIlcTY9QpoUqCY0Ivo88wzaPvuO6md2fHHo+R/HygyVIOFVlnyTIWFonVQsNhRSUlInjJF95wURcFUVATHnj0AeFes7LOXTksDRdMwZmbCo4r7Mw/gDBDW4byw4+cQYFkWbt7qZh44EM6yMrR995243VVZCTo9XZEccfSmmzB47XeiGxbgsm4F13DHr78CAJyHD3PHHjECrpoaMM3NnJu4TrL61b3wAtLOOtMr7i8cXEeOiILUXV8Pk8q6SUgs4qoiXnzxRTQ3N2P69OkoKCgQf959910AwK+//opNmzbht99+w6BBgxRjjhw5AgAwmUx4/vnnMWXKFIwdOxYvv/wynn76aTz00EPxPDUAXIydgFFmsTMaKBh9lA4xJXCtOGKxCx7iiiX0JlLnzeXakiUlia9Zhw8HnZIS8pwGsxm0rCSKkPgAAEnHHw/73LnIveN2GIL00JiLpLAfU57SFSvv1mFQxehZePey0KXDeegQmI4ONP1vFQ7MnoOunTsBlwsAkHHJxV7HdVVWgXE64Za5NT11dejcvh0umbADy8J1lKuq4Cwrg7uxEc7DnMXOXNwfpj6ctdJ57Bg8ss4cTFsbmj/8CG0//YRDF1yo6Hcrx1fGrpbL1SkrAu0m7tiEJ64Wu0Cp4NOnTw84Zt68eZg3b57fMfFCHmNnpg1ijJqRphQCzmI0wOFmQFGJbRVL5LUlKqRXLIEQPsbcHHh4i5G5rxQfZ7BY0PfZZ0KaU95OzZiXJ30LgzIRhKJpGFJSwLS1ATQNM299NObkgM7Jhqe2Do59+1D/yitwHTmCpvf+y+2XlIS0009H9eN/F4UeALgqKzhLnMcDg92OlGnT0PLpp2hb973f9m+dW7aIrlhz//4w9+kDx++7uQLNXV2Ksa5jR9G1axe6duxA0wf/Q/7994F1ucSCz562Nvxx6WUw5uag3yuviPsxHR04dM65oCwWDFzzldg3WJ6k4a6pBqBduYKQGCSu368HkKuw2FEw82LOZDAoEhHSbNw/m4k2gEpgATAwJ/Rv3b0VMSuWiGICIWSEODvAO/EhVEwyi50xN9enxQ4ADKmc1c5cVKQoMWIdxrljmz74QHSTCq5ZY0YG6LQ0pJw8jZ+D64bhrqyEY/8BAIBl0CCkTD8ZAN/b1keJEwDo+EUm7IqLYSrkLJeuYxVw13CuWCFu0FVVDRcf2tS1axcaV67EntFj0LJmDQCg9pln4dizB+3f/wBPm1Sg33HgADxNTXBXV6Nj82Y0ffABmj/9VNG2TVij2ujCMgwq7r4HFffdJ25z19fDVVFB4vJiDMmLjiK56uQJg8xiJ7vRp9lMqGl1JHR8HQBcP30gmjtdmDMiL95L6TZIvWKJsCMQQkUu7MwyV2w4CCVP6PR0GCwWxTHUpVtoeyrcqPTK8rXPno32738Qy5sAXFsy+RwFS5eifc4cuGtqUPPU/8FVUQnwMYKWQYOQfOKJAEXBsXevIqFDxGgE3G60fvetmJRh7tcPpj6CsDsmxhxaR45Ex6ZNcFdXw93YAADo2r0bjKMLYFnUv/QyrIMHo5EPdwI4Cxydwp2X4+BB8fX6F1/iOnPQtCJ+0VVRiUMXXggKFPq/+R/RCujYuxfNH34IAMhesADm4mI0vvsu6p77J9L/fD4KHnnEx1+CEGkSW0l0c3JSlDEfYoydKkNWstgl9s3faqKx5MwROGGgdlkCgjcUyYolEMJGnkARKYudbexYmPr3QyrfechgtYrtzdQWO5q3tlkGKCs3pJ12mlcMnlBuRBB2xsxMpJ15Jkx8KS9XVRUcB3iL3eBBMGZkwDaGa7fWySdJyGMK7X+aDlAUXH+Uc/Pl5MCQnCzGGrqOHYObj7GzjuT68boqKsRYOLarCw4+c7dr1y4cuWGRwjUstxI6yyRhJ7Zb83jQvmmT9PrPP6Nr+w50bt+Oli++FF+XF4nu2LqNW8cRLkZQ7vYmRB8i7KJIRpJksm/udImlTEwGSnTFGigg2cJ920rkUieE0CAFigmE8FG4YjVqnYYCnZqKgV98gfwHpQL3Jr7kCZ2RrhgriEnryFGK1w1JSUjzUXDYa478fABccoJT5ooFIIpLAaus+1LShAmKlmjm/lwNP9Fid+SIWAvPxu/naWwUe9qqcR48CENSkri/PBPXceig5j5yIdi5c6f4uGH5crAeD1iGQftPkrDr5OvuuYQkxyIi7GIJURJRRB5X1dThFLNKjbRBdMVaTTSsfE07Iux6HpIrNr7rIBC6M4KwMyQn+00wCBZ1TLOxgOtVa8xSeiXy/noXiv71L9jnzPaaI+OSi0FZrV4lXYwZyhZrxgIui9VdWelVjy7t3HMVbli5sDMV9UOWrI0mZeY8POZ+/QCDAZ7mZjjLy8X5fLUZs44YIT7Of/hhJE2YAABwyS12B7nyKpouYQGZYOz6/XfsHT8BB08/Q7LwAejcypdn4bN61Z0+CNGFKIkY4fKwiqxY4bHFaBBr2slbjhF6BsQVSyCEj2UgVxTYMmxoVBPMchYvRsbll8E+e5bidTotDSknnajZy9ZSUoKBn65GyXvvgpKVXFHH6RlzsgFZ1yQ6K0t0MdMpyQqrnHXEcPGxuV8RbKNGiqIzeRqfjGGzSR0u3G7+GDlchq8G2YtuQM4ttyDv3nuRdsbpYrKI6LJ1uUSBmHfvPTAPHIjsG67XnAuQysCwXV1wHjwI1ukUXciO/Qfgrq0VrYGRcp8T9EGSJ2KIMitWZrEzcsIu0WPsCMEjFigmrlgCIWSsQ4ei/3/eEFuJRQvb6NGwjR4d9H6CW9OYnyfGwqmFHWUwwJiVBTefrZp56XyFSM1edANavvgC5oEDxPkAyfVc8r8P0Prll0g75xxxm3XYMDjLyrj5TSYYUlNhyssTXaC2MWPQuX07YDIhadIk2GfMEPcVBKCrshLlf/kLnEeOAm43V6bl3HORft55cDc2ou6FF7nxBQVwV1aK+xc8vAQwcKVgKu68E57GRqTMmIGOX7fA9Uc5Wj7/HGDZiFtZCYEhwi7KmGgKLg+X+i1Y6WiD1IXCaqJhIa7YHovwBZ8IOwIhPJImToz3EgJiyi+QhF1mhtd2QdQBQObllyu2mfv1w8CvvoIhyQbKZIKpb1+YS0pg4NtoGjMykHHRRYp9rMOGoWX1au54OdmgKApGPpYPAGwTxiNj/iUwpNi9ikQLLdQ6NmwA09Ehvm4pKREFpzEjA6Y+feA6dgzW0lK0NzWB7ezkjj1ihBjv1+/Vf6N+xQpkLVwAvMSg+Y9yNPEZsqaiooQu49UTIUoiyqTLEigEK52JpsTHclcsEXY9D6lXbJwXQiAQoo7QwxbgRJGarGuuBgDk3XuvZhybKS8XtN0Og9WKgV9+gaJXXvZ7PKGtGSDFIcrXYMovQNqZZ8Ku0UfXxFvs5KIOUGbkAlJsnqlvX7GVGGU2K5JYrMOHo88TT8BSUoKkSZMAQMzENZPEiZhDbjdR5vqTudiQGUNzRSudkTaIIs5iomHhY+sSvY4dIXgMpFcsgdBrMOYXiI/VrlgAyL7pJgxY/QkyL78s4FwUTQe0dAkFkgHAmM0JO2OeZLEzFRZ47SOO99HvVZ0gkn7+eTAXFyN17hxxH3NJCShau9958tQTFM9NfUl8Xawhrtgoc+UJxRjdNw0jCtPw8vdcLITRILUUs8otdkZy8+9pkALFBELvQW4tozMzvbYbzGaxxEkkoNPTYSwsgLuiEkY+EcOYJwk2uVtWjTE7m2ujxneJ6POPp2EqLFRkzwJAyrRpSOETNoS4PCGZRQtTfj7MgwbCeYC735FSJ7GHmIiijMFAYUJxJmxmWqpjJ+tCIbfYEVdsz4P0iiUQeg+ikKIosahxtLEO4zJoJVeszGJX4NtiR5lMoLOyxOe20aNhGzNG7GKhfSzOQmgbf5zfNaVMnSo+JqVOYg+x2MUQk9h5ghITJpJMNGxmzmJnIeVOehwG0iuWQOg1mPm4M2N2tk9XZaTJuupKsC4n0k4/DQBfWsRkAm23a7qD5Zhyc+GpqwOdkQFjYWHAY2VeeQWST5wa0OqYPHUqGl5/gztGhApKE/RDhF0MEXvFGgyYNSwP6/bW4rIp/TE4N4V7fHxxfBdIiDiCniOVbAiEno9l8GDk3XO3VF8uBiRNmIB+fLFhgEva6P/G6zAkJweM0TPm5gK//w7ryJG6Mlcpmoa1tFTXmrgsXQPMfSLT25egH2IiiiEZyVxxyvQkE/JSrXjl8gmYOigbufzjEweTHqw9jVQr9ze3W00BRhIIhJ5A5hVXiDFp8SJp3DhYhwwJOM7cn+uWIfSqjRSGpCSUfPABit//r89OGLFmyZIloChK8TN06FAAQENDA2688UaUlpbCZrOhX79+uOmmm9Dc3Ox3ziuvvNJrznnz5sXidPxCLHYx5JSRBWhzeDBjqHY2EqHnccvsIRhdlI5TR/mOdSEQCIR4kLVgAUx9+iLtnLMjPrfJR9ZtPBkxYgS+/vpr8bmRjyesqKhARUUFnnrqKQwfPhx//PEHrrvuOlRUVOD999/3O+e8efOwfPly8blF1n0kXhBhF0OsJhqXHR/dyumExKJPuo38zQkEQkJizM7WVXqlp2A0GpGvkSk8cuRIfPDBB+LzgQMH4tFHH8Wll14Kt9stCkAtLBaL5pzxhLhiCQQCgUAgdEvanW60drnEH4fb43Ps/v37UVhYiAEDBmD+/Pko53vjatHc3IzU1FS/og4A1q5di9zcXJSWluL6669HfX19yOcSKSiW5YvY9GKOHj2KoqIiHDlyBH1JBg+BQCAQCAmNcN8uuuU9GCxJ4us3zxyMW2d7xxd+/vnnaGtrQ2lpKSorK/Hwww/j2LFj2LlzJ+x2u2JsXV0dxo8fj0svvRSPPvqozzWsXLkSSUlJKCkpQVlZGe69916kpKRgw4YNoGOUFa0FEXYgwo5AIBAIhO6EcN/eU3YIhYVS5q3ZaIDFGFhUNTU1oX///nj66adx9dVXi6+3tLRg9uzZyMzMxMcffwyTSX/i28GDBzFw4EB8/fXXmDlzZnAnFEGIK5ZAIBAIBEK3JNlshN1qEn/0iDoASE9Px5AhQ3DgwAHxtdbWVsybNw92ux2rVq0KStQBwIABA5Cdna2YMx4QYUcgEAgEAqFX0dbWhrKyMhTw3TlaWlowZ84cmM1mfPzxx7BarUHPefToUdTX14tzxgsi7AgEAoFAIPRo7rjjDqxbtw6HDx/G+vXrcc4554CmaVx88cWiqGtvb8err76KlpYWVFVVoaqqCh6PlIwxdOhQrFq1CgAnDO+8805s3LgRhw8fxjfffIOzzjoLgwYNwty5c+N1mgBIuRMCgUAgEAg9nKNHj+Liiy9GfX09cnJycOKJJ2Ljxo3IycnB2rVrsWnTJgDAIFW7tEOHDqG4uBgAsHfvXrFoMU3T2LFjB15//XU0NTWhsLAQc+bMwSOPPBL3WnYkeQIkeYJAIBAIhO4EuW/7hrhiCQQCgUAgEHoIRNgRCAQCgUAg9BCIsCMQCAQCgUDoIRBhRyAQCAQCgdBDIMKOQCAQCAQCoYdAhB2BQCAQCARCD4EIOwKBQCAQCIQeAilQDIBhGABAZWVlnFdCIBAIBAIhEML9Wrh/EySIsANQXV0NAJg0aVKcV0IgEAgEAkEv1dXV6NevX7yXkVCQzhMA3G43tm7diry8PBgMkfNOt7a2Yvjw4fj9999ht9sjNm9PhFwr/ZBrFRzkeumHXCv9kGuln2hcK4ZhUF1djXHjxsFoJDYqOUTYRZGWlhakpaWhubkZqamp8V5OQkOulX7ItQoOcr30Q66Vfsi10g+5VrGFJE8QCAQCgUAg9BCIsCMQCAQCgUDoIRBhF0UsFgseeughWCyWeC8l4SHXSj/kWgUHuV76IddKP+Ra6Ydcq9hCYuwIBAKBQCAQegjEYkcgEAgEAoHQQyDCjkAgEAgEAqGHQIQdgUAgEAgEQg+BCDsCgUAgEAiEHgIRdlHk+eefR3FxMaxWKyZPnoyff/453kuKO0uWLAFFUYqfoUOHitu7urqwaNEiZGVlISUlBeedd57Y8q2n8/333+OMM85AYWEhKIrChx9+qNjOsiwefPBBFBQUwGazYdasWdi/f79iTENDA+bPn4/U1FSkp6fj6quvRltbWwzPIjYEulZXXnml1/ts3rx5ijG95VotW7YMEydOhN1uR25uLs4++2zs3btXMUbP/115eTlOO+00JCUlITc3F3feeSfcbncsTyXq6LlW06dP93pvXXfddYoxveFavfjiixg9ejRSU1ORmpqKKVOm4PPPPxe3k/dU/CDCLkq8++67uO222/DQQw/h119/xZgxYzB37lzU1NTEe2lxZ8SIEaisrBR/fvzxR3Hbrbfeik8++QT//e9/sW7dOlRUVODcc8+N42pjR3t7O8aMGYPnn39ec/sTTzyB5557Di+99BI2bdqE5ORkzJ07F11dXeKY+fPnY9euXVizZg1Wr16N77//HgsXLozVKcSMQNcKAObNm6d4n73zzjuK7b3lWq1btw6LFi3Cxo0bsWbNGrhcLsyZMwft7e3imED/dx6PB6eddhqcTifWr1+P119/HStWrMCDDz4Yj1OKGnquFQAsWLBA8d564oknxG295Vr17dsXjz/+OLZs2YJffvkFM2bMwFlnnYVdu3YBIO+puMISosKkSZPYRYsWic89Hg9bWFjILlu2LI6rij8PPfQQO2bMGM1tTU1NrMlkYv/73/+Kr+3evZsFwG7YsCFGK0wMALCrVq0SnzMMw+bn57NPPvmk+FpTUxNrsVjYd955h2VZlv39999ZAOzmzZvFMZ9//jlLURR77NixmK091qivFcuy7BVXXMGeddZZPvfprdeKZVm2pqaGBcCuW7eOZVl9/3efffYZazAY2KqqKnHMiy++yKamprIOhyO2JxBD1NeKZVn25JNPZm+++Waf+/TWa8WyLJuRkcH++9//Ju+pOEMsdlHA6XRiy5YtmDVrlviawWDArFmzsGHDhjiuLDHYv38/CgsLMWDAAMyfPx/l5eUAgC1btsDlcimu29ChQ9GvX79ef90OHTqEqqoqxbVJS0vD5MmTxWuzYcMGpKenY8KECeKYWbNmwWAwYNOmTTFfc7xZu3YtcnNzUVpaiuuvvx719fXitt58rZqbmwEAmZmZAPT9323YsAGjRo1CXl6eOGbu3LloaWkRLTQ9EfW1EnjrrbeQnZ2NkSNH4p577kFHR4e4rTdeK4/Hg5UrV6K9vR1Tpkwh76k4Y4z3AnoidXV18Hg8ijcsAOTl5WHPnj1xWlViMHnyZKxYsQKlpaWorKzEww8/jJNOOgk7d+5EVVUVzGYz0tPTFfvk5eWhqqoqPgtOEITz13pPCduqqqqQm5ur2G40GpGZmdnrrt+8efNw7rnnoqSkBGVlZbj33ntxyimnYMOGDaBputdeK4ZhcMstt2Dq1KkYOXIkAOj6v6uqqtJ87wnbeiJa1woALrnkEvTv3x+FhYXYsWMH/vrXv2Lv3r343//+B6B3XavffvsNU6ZMQVdXF1JSUrBq1SoMHz4c27ZtI++pOEKEHSGmnHLKKeLj0aNHY/Lkyejfvz/ee+892Gy2OK6M0JO46KKLxMejRo3C6NGjMXDgQKxduxYzZ86M48riy6JFi7Bz505FXCtBG1/XSh6HOWrUKBQUFGDmzJkoKyvDwIEDY73MuFJaWopt27ahubkZ77//Pq644gqsW7cu3svq9RBXbBTIzs4GTdNeGUDV1dXIz8+P06oSk/T0dAwZMgQHDhxAfn4+nE4nmpqaFGPIdYN4/v7eU/n5+V7JOW63Gw0NDb3++g0YMADZ2dk4cOAAgN55rRYvXozVq1fju+++Q9++fcXX9fzf5efna773hG09DV/XSovJkycDgOK91VuuldlsxqBBgzB+/HgsW7YMY8aMwbPPPkveU3GGCLsoYDabMX78eHzzzTfiawzD4JtvvsGUKVPiuLLEo62tDWVlZSgoKMD48eNhMpkU123v3r0oLy/v9detpKQE+fn5imvT0tKCTZs2iddmypQpaGpqwpYtW8Qx3377LRiGEW8+vZWjR4+ivr4eBQUFAHrXtWJZFosXL8aqVavw7bffoqSkRLFdz//dlClT8NtvvynE8Jo1a5Camorhw4fH5kRiQKBrpcW2bdsAQPHe6g3XSguGYeBwOMh7Kt7EO3ujp7Jy5UrWYrGwK1asYH///Xd24cKFbHp6uiIDqDdy++23s2vXrmUPHTrE/vTTT+ysWbPY7OxstqamhmVZlr3uuuvYfv36sd9++y37yy+/sFOmTGGnTJkS51XHhtbWVnbr1q3s1q1bWQDs008/zW7dupX9448/WJZl2ccff5xNT09nP/roI3bHjh3sWWedxZaUlLCdnZ3iHPPmzWPHjRvHbtq0if3xxx/ZwYMHsxdffHG8Tilq+LtWra2t7B133MFu2LCBPXToEPv111+zxx13HDt48GC2q6tLnKO3XKvrr7+eTUtLY9euXctWVlaKPx0dHeKYQP93brebHTlyJDtnzhx227Zt7BdffMHm5OSw99xzTzxOKWoEulYHDhxgly5dyv7yyy/soUOH2I8++ogdMGAAO23aNHGO3nKt7r77bnbdunXsoUOH2B07drB33303S1EU+9VXX7EsS95T8YQIuyjyz3/+k+3Xrx9rNpvZSZMmsRs3boz3kuLOhRdeyBYUFLBms5nt06cPe+GFF7IHDhwQt3d2drI33HADm5GRwSYlJbHnnHMOW1lZGccVx47vvvuOBeD1c8UVV7Asy5U8eeCBB9i8vDzWYrGwM2fOZPfu3auYo76+nr344ovZlJQUNjU1lb3qqqvY1tbWOJxNdPF3rTo6Otg5c+awOTk5rMlkYvv3788uWLDA60tVb7lWWtcJALt8+XJxjJ7/u8OHD7OnnHIKa7PZ2OzsbPb2229nXS5XjM8mugS6VuXl5ey0adPYzMxM1mKxsIMGDWLvvPNOtrm5WTFPb7hWf/nLX9j+/fuzZrOZzcnJYWfOnCmKOpYl76l4QrEsy8bOPkggEAgEAoFAiBYkxo5AIBAIBAKhh0CEHYFAIBAIBEIPgQg7AoFAIBAIhB4CEXYEAoFAIBAIPQQi7AgEAoFAIBB6CETYEQgEAoFAIPQQiLAjEAgEAoFA6CEQYUcgEAgEAoHQQyDCjkAgEGS0b/oZu4cOg6elJd5LIRAIhKAhwo5AIBAIBAKhh0CEHYFAIBAIBEIPgQg7AoGQULAMg7qXX8GBmbOwZ8xYHDzrbLR88SUAyU3aunYtDp55FvaMHoNDF16Irn37FHO0fPkVyk4/HXtGjcaBGTNR/9pyxXbG6UTNU09h//Q/cWPmzEXT++8rxnTt2oVD552PPWPH4fBFF8Nx8FB0T5xAIBAigDHeCyAQCAQ59a+8guaPP0H+kiUwF/dHx+ZfUHHXXaAzM8QxNU8+hbx774ExOwe1//gHjl5/AwZ+8TkokwmdO3fh2K23InvxIqSecgo6t25D1dKloNPTkX7uOQCAir/+FZ3btiPvvnthHToUrqNH4WlsVKyj5plnkPvXu2DMzETlkiWovO8+FL/zdkyvBYFAIAQLEXYEAiFhYJxO1L38Cvq99iqSxo0DAJiLitDx6xY0vfse0i+4AACQs+gGpEydCgAofHwZ9k//E1q//hqpp5yChhUrkHz88ci54QYAgKWkBI6yA6h/7VWkn3sOHIcOofXzL9DvtVeRfMIJ4jHU5N5yC5InTQIAZC9YgCPXXgfG4YDBYon6dSAQCIRQIcKOQCAkDK4//gDb2Ynyq69RvM66XLAOGyY+t40dKz6m09NhLimBo+wgAMBxsAz2GTMV+ycddxwa3vgPWI8Hjj17AJpG0sSJftdiKS0VHxtzcgAAnvp6GAoLQzo3AoFAiAVE2BEIhISB6egAABS99CJMeXmKbZTZDGf5kbCPQVms+sYZZR+PFAUAYBk27OMTCARCNCHJEwQCIWEwDxwEymyGu7IS5v79FT+mggJxXOf27eJjT3MznIcPwzJwAADAMmAgOn/9VTFvx6+/wlLcHxRNwzJkCMAw6Ni8OTYnRSAQCDGEWOwIBELCQKckI/MvV6F62eNgGRZJ44+Dp7UVnb9uhSElBSbeDVr3wgug09NBZ2Wh9plnQWekwz6Tc79mXnUlDv/5AtS+8AKXPLFtOxrfehv5Dz4IADD37YO0s89GxX33I/++e2EZOhSuYxXwNNQj9ZRT4nbuBAKBEAmIsCMQCAlFzs03w5iZifpXXkHl0aOg7XZYhw9H9rULRVdozm23ofqxx+A8/Acsw4ah6MUXQZnNAADbiBHo849/oPafz6HuxZdgzMlGzo03ihmxAJC/5CHUPv0PVD28FJ6mJhgLC5C98Nq4nC+BQCBEEoplWRI0QiAQugXtm35G+RVXYMjPm0CnpsZ7OQQCgZBwkBg7AoFAIBAIhB4CEXYEAoFAIBAIPQTiiiUQCAQCgUDoIRCLHYFAIBAIBEIPgQg7AoFAIBAIhB4CEXYEAoFAIBAIPQQi7AgEAoFAIBB6CETYEQgEAoFAIPQQiLAjEAgEAoFA6CEQYUcgEAgEAoHQQyDCjkAgEAgEAqGH8P8B6EYLnBpLGuEAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "cum_loss_list=load_list_from_file(model_name.replace('_','-') + \"-loss.pkl\")\n", + "acc_epoch=load_list_from_file(model_name.replace('_','-') + \"-acc.pkl\")\n", + "plot(cum_loss_list,acc_epoch)" + ] + }, + { + "cell_type": "markdown", + "id": "1afaf400-babe-4c17-99d7-9ee167d19174", + "metadata": {}, + "source": [ + "Let's load actually load the model into model_lora:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "6fa85d86-2966-4d8f-b7fc-b59551f4d02e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + " )\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 83, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.load_state_dict(torch.load(model_name.replace('_','-') + \".pth\", map_location=device))\n", + "model_lora.eval()" + ] + }, + { + "cell_type": "markdown", + "id": "c85bdd49-492a-4cc2-a22f-2dccec3cc8ec", + "metadata": {}, + "source": [ + "And, let's evaluate its performance on the test data.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "id": "db4a670a-958b-41b9-a0fb-168902a1d706", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEYAAAAQCAYAAACr+QluAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAABJ0AAASdAHeZh94AAAEG0lEQVR4nO3Ye8jfcxQH8NfDY0gUcku5JnemRUa2aS4Jy7BIQwopmnsknB0l91sps9QmRDG5ZIxYLqVWmtDMJds09wcP5tLYHn98Pr/t+/ye3/P0e2T5x6lv5/s9n/P5nHPe3/M5n/P99gwMDPifhlJvJ2FmTsYlGI+t8T3ex30RMb+h14Pz67U/evAhHsLsiFjbrSOZeTomYiwOxpZ4LCKmjzBnOXYdZvibiNixTX9bTMWJOBA7Y3WNbQ7mtHweAkxm3o6rsRLPoQ/bYRwmYX5D/VGchW/xOH7DsXgAR+Cc4YLqQNcrgKyqtvfpct5PuLeDfFUH2bTq21dYiM+xA05VXuYJmTktIgYGAZOZFyigPIwLI2J12/gmjfupCijLcFhE9FX5GMzD2Zn5TEQ83WWAlyuAfKpkzsIu5/VHxMwudT/GFLzQzObMvA6LcJoC0rzexuCmuFlBcQgoEBF/Nh6nVn5XC5Sqszozb8BJynbsCpiIWAdEZnYzZdQUEa8NI/86M2cp8U/SBEbZAtspabk2M0/EAfgDiyLi7bb1Wvv3sw62WrKjMnNMJ5D/Rdo0M6djF/yK9/BGRKwZ5Tqtl/4Xg2vMoZX/gcUKKOsoM9/A6RHxXRW1smT3Dkb2qLy33i8dpZOjoR3xSJtsWWaeFxGvd7NAZvZaXw9fgo0a49tXfjUGcJRyMhyElzEBTzb0X6j8iszcpmFkEzT3wtbdOPcPaQ4mK+BsoZw0D2I3vJiZB3e5zq1KIsyPiAUMzpgWSH9hSkQsr8/v10L7ESZm5vi6rZ7A2TgeSzLzWSXbjsFOSq3aBV0f2aOliGgvRh/gosxchSsx0/pa2JEyc0bVXarEg8EZ01/54gYoLQd+w4L6eFiVrcHJuBbf4dx6faIc1b9U/W9HDm+D0KzKJ4yklJmX4D4swdER8UNrrJkxH1XeP8w6P1a+eUtQT6nb6tU0uBn2Ql9ELBsxhA1DrTq4xXAKmXkZ7lGybHJEDHqBzYx5Vakt+2VmU96iVjHuJtAzMUZp+v4LOrzyTiemzLxGAeVdJVOGZPU6ACJiBZ5X6sKlbQsdp9SSfrVqV/lWHYyOxR1Kht3aYXzPzNyn2Sz+E8rMfTNzSEZk5m64vz4+2mH8hurXO0qm9LXrMPST4GIcgrtrH7NYOY5PwRqcHxE/NfRfyczflXT8Bfsq3yG/4+SI+LKDzVeV75vdsbzh8CnVDut7pPGZObfe90XEVY11zsCVtY1YUe3vWe1vpny63Nk0nJnn4qYay5uY0aGZXB4RcwcBExErM3McblRa5wn4WcmkWyJiUdsiTynbZrpSe77A7Kq7sgMoI9FYpXg3aQ/re6IVaAKzEHsrL/JIpZ704y2lr3kkItp/HbR6ro1x2TB+vI65Pf//duhMfwOaGG062kNIrAAAAABJRU5ErkJggg==", + "text/latex": [ + "$\\displaystyle 69.152$" + ], + "text/plain": [ + "69.152" + ] + }, + "execution_count": 84, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader , model_lora, device)\n" + ] + }, + { + "cell_type": "markdown", + "id": "8eacb773-a876-4575-a595-3a7ef2a99f4c", + "metadata": {}, + "source": [ + "You get a 3% improvement over a model trained from scratch by using LoRA. Note that this occurs despite the fact that the model fine-tuned with LoRA updated less parameters than the model trained from scratch!\n", + "\n", + "The ```model_lora.fc1``` attribute represents ```LinearWithLoRA``` which contains both the standard ```Linear``` layer ``(linear)`` and an additional ```LoRA``` layer ```(lora)``` which represents the ```LoRALayer```.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "2684b92c-c477-4aac-ad75-35ba2def8dbc", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + ")" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_lora.fc1" + ] + }, + { + "cell_type": "markdown", + "id": "01f51868-4cf9-4e04-b971-c94435b9e6b0", + "metadata": {}, + "source": [ + "From ```model_lora.fc1.lora```, you can obtain the learnable parameters A and B. \n" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "id": "8fa84b7f-7a98-4d0a-9a13-53aa8de6a081", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B Parameter containing:\n", + "tensor([[-4.3192e-01, -1.1071e+00, 2.4456e-01, -3.1031e-02, -2.0012e-01,\n", + " -6.7811e-01, -1.3552e-01, -2.7458e-01, 3.2278e-02, 6.7592e-02,\n", + " 8.3020e-01, 1.1610e-05, -1.0894e-01, 7.7830e-05, -1.6789e-01,\n", + " -1.3309e-01, -5.1875e-01, 2.1928e-02, -6.5869e-02, -3.5834e-01,\n", + " -2.4473e-02, -1.1260e+00, -8.8752e-02, -7.0861e-03, -1.3263e-02,\n", + " 0.0000e+00, -6.9039e-01, -8.6471e-02, -3.9146e-01, -2.2644e-01,\n", + " -8.7611e-01, -7.9929e-01, 0.0000e+00, 3.9646e-01, 5.2164e-01,\n", + " -4.2730e-01, 2.3550e-01, 4.0447e-02, 2.3289e-01, -4.5217e-01,\n", + " 1.7721e-03, -4.7263e-01, -2.4343e-01, 6.3737e-01, 0.0000e+00,\n", + " 2.6904e-03, -7.8828e-01, 2.2559e-02, -4.3776e-02, 3.0909e-01,\n", + " -1.6914e-01, -2.0294e-01, -4.2175e-01, 7.8840e-01, -3.1771e-01,\n", + " -2.0639e-01, 1.1487e-02, -5.7238e-01, 1.4071e-01, -2.8561e-01,\n", + " 1.1753e-01, -1.6501e-04, -4.5406e-01, 0.0000e+00, 6.4464e-05,\n", + " -4.4552e-01, -1.4372e-01, 5.2899e-02, -9.7813e-01, -4.5834e-01,\n", + " -6.0968e-01, 5.5418e-02, -7.8054e-01, -1.2806e-01, -1.2291e-01,\n", + " -1.4927e-01, 1.1984e-01, -4.9344e-02, -4.0802e-01, 0.0000e+00,\n", + " 1.0085e-01, 0.0000e+00, -1.1553e+00, -1.1726e-01, 0.0000e+00,\n", + " -1.5980e-01, 1.0715e-01, -6.6047e-01, -4.0122e-01, -3.3328e-01,\n", + " 0.0000e+00, -8.4713e-05, -1.0361e+00, -2.0651e-01, 4.8110e-01,\n", + " -1.1648e+00, -1.3535e+00, 3.9261e-01, -5.9634e-04, 0.0000e+00,\n", + " -2.3484e-02, -2.9119e-02, 4.8014e-03, -9.4629e-02, -2.8612e-01,\n", + " -2.1896e-02, -3.7316e-01, 3.3309e-02, -1.2363e-01, -1.7977e-01,\n", + " 4.3949e-05, -2.0580e-01, 4.9675e-01, -3.3358e-03, 2.3369e-01,\n", + " -2.5254e-01, 8.1739e-03, 1.1209e-01, -1.7311e-01, -8.0624e-02,\n", + " -1.3280e+00, 7.7398e-01, -1.2380e+00, -4.4729e-01, -6.5231e-01,\n", + " -3.0026e-01, 8.8543e-02, -1.9287e-04],\n", + " [ 2.8719e-01, 4.5585e-01, -2.3756e-01, -4.2161e-02, 1.0720e-01,\n", + " -8.3839e-01, 2.5098e-01, -4.7875e-01, -6.7113e-02, -1.8940e-01,\n", + " -9.7494e-01, 1.5431e-05, -1.9587e-01, 2.8327e-04, 1.3258e-01,\n", + " -1.4904e-01, 7.6676e-01, -1.9980e-01, -5.4798e-02, -5.3408e-02,\n", + " -1.8818e-01, 7.9588e-01, 9.6087e-02, 2.5530e-02, -7.2462e-01,\n", + " 0.0000e+00, -1.4258e+00, -5.9680e-02, 1.7253e-03, 4.7575e-02,\n", + " 4.2872e-01, 1.7875e-02, 0.0000e+00, -1.6925e-01, 1.6071e+00,\n", + " -7.2869e-01, -2.5870e-01, 2.7839e-02, -3.0237e-01, 6.9624e-01,\n", + " -2.8908e-04, 2.9164e-01, 4.7389e-02, 2.8530e-01, 0.0000e+00,\n", + " -5.9235e-02, -6.8176e-02, -4.6494e-02, -8.1741e-02, -6.3252e-01,\n", + " 2.4469e-02, 7.2494e-02, 8.2158e-02, 6.3870e-01, 2.1799e-01,\n", + " 7.5177e-01, 3.1872e-02, 1.4100e-01, -2.3390e-01, 3.3391e-01,\n", + " -3.8685e-01, -4.5463e-03, 1.4543e-01, 0.0000e+00, 3.8082e-05,\n", + " 3.0410e-01, 4.7614e-02, -1.5299e-01, -3.4549e-01, -2.3156e-01,\n", + " 2.2711e-01, -1.0894e-01, 1.7479e+00, 5.1043e-03, 1.4021e-02,\n", + " 1.9368e-02, -1.0875e-01, 9.7929e-03, -7.4530e-02, 0.0000e+00,\n", + " 1.9233e-02, 0.0000e+00, 5.1489e-01, -2.0680e-01, 0.0000e+00,\n", + " 1.8348e-01, 5.8013e-01, 2.9426e-01, -8.2005e-01, 2.2163e-01,\n", + " 0.0000e+00, 1.8530e-04, 5.1081e-01, 1.1597e-01, -4.4384e-01,\n", + " 6.3387e-01, 9.7338e-01, -6.0469e-01, -6.5377e-02, 0.0000e+00,\n", + " -7.6416e-02, -4.1226e-02, 1.3478e-01, -2.0853e-01, 2.4558e-01,\n", + " -1.0312e-01, 4.7792e-01, 4.1285e-02, -4.4301e-02, 3.2861e-02,\n", + " 6.9632e-04, 4.0895e-02, 1.1403e-01, 4.4740e-03, -2.8703e-01,\n", + " 3.8271e-01, -2.0000e-03, -5.1493e-02, 2.4292e-01, -2.3110e-01,\n", + " 1.1481e+00, -1.6331e+00, 5.0157e-01, -1.3528e-01, 3.8709e-01,\n", + " 1.4637e-01, -1.2716e-01, -6.3597e-05]], device='cuda:0',\n", + " requires_grad=True)\n", + "\n", + " Number of elements in the tensor B 256\n" + ] + } + ], + "source": [ + "B=model_lora.fc1.lora.B\n", + "print(\"B\",B)\n", + "print(\"\\n Number of elements in the tensor B\",B.numel())\n", + "torch.save(B, 'B.pth')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "id": "e184ffb3-ccff-4637-a0f1-85b4c7c83bb0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A Parameter containing:\n", + "tensor([[ 3.2952e-01, 9.3019e-01],\n", + " [ 2.3904e+00, -5.1022e+00],\n", + " [-2.4573e-01, 2.4733e+00],\n", + " [ 5.8014e-01, 3.9014e-01],\n", + " [ 1.6970e+00, -2.4614e+00],\n", + " [-1.2420e+00, 8.3014e-01],\n", + " [-2.0468e+00, 1.1629e+00],\n", + " [-5.9361e-01, 1.8099e-01],\n", + " [ 9.2466e-02, 1.1583e-01],\n", + " [-2.0841e-02, 1.5550e+00],\n", + " [ 1.3028e+00, -9.8381e-01],\n", + " [ 1.4320e+00, -3.3497e+00],\n", + " [-1.1637e+00, 1.9436e+00],\n", + " [ 2.4898e-01, -1.1353e+00],\n", + " [ 2.4423e+00, 1.0154e+00],\n", + " [ 2.4881e+00, -4.6765e+00],\n", + " [-1.8985e-01, 1.3426e+00],\n", + " [-1.1730e-01, -2.1925e+00],\n", + " [ 2.0193e+00, -8.5886e-01],\n", + " [-3.1268e+00, 3.5134e+00],\n", + " [ 1.0935e+00, -2.9263e+00],\n", + " [-1.0435e+00, 2.5428e-01],\n", + " [-8.6704e-01, 2.3570e+00],\n", + " [-5.2725e-02, -2.3731e-01],\n", + " [-4.3896e+00, 5.0369e+00],\n", + " [-1.1174e+00, 9.9728e-01],\n", + " [ 4.7710e-01, -1.2162e+00],\n", + " [ 3.3296e+00, -5.9627e+00],\n", + " [ 1.7393e+00, -2.0707e+00],\n", + " [-4.0115e-01, 2.2691e+00],\n", + " [ 4.8368e-04, 1.1773e+00],\n", + " [ 1.9858e+00, -1.6100e+00],\n", + " [ 6.9951e-01, -1.2869e+00],\n", + " [-2.6380e-01, 2.5743e+00],\n", + " [-3.9002e+00, 2.6775e+00],\n", + " [-2.1946e+00, 3.7126e+00],\n", + " [-3.2268e-01, 1.7346e-01],\n", + " [ 4.7590e+00, -2.9553e+00],\n", + " [-6.6704e-01, 8.1369e-01],\n", + " [ 1.0073e+00, 3.6606e-01],\n", + " [ 1.3186e-01, -3.4151e+00],\n", + " [-6.6832e-01, 3.7761e-01],\n", + " [-3.0825e-01, 1.1928e+00],\n", + " [ 1.8897e+00, -1.2788e+00],\n", + " [-1.4344e+00, -2.9294e-01],\n", + " [ 2.1678e+00, -2.5469e+00],\n", + " [-6.6993e-01, 2.2932e+00],\n", + " [-3.1326e+00, 4.3424e+00],\n", + " [ 4.2197e+00, -7.1153e+00],\n", + " [ 7.1775e-01, -4.0832e-01],\n", + " [-1.6389e+00, 1.4520e+00],\n", + " [-1.6150e+00, 2.4683e+00],\n", + " [ 1.5745e+00, -4.6085e+00],\n", + " [ 6.0900e-01, -1.0851e+00],\n", + " [-1.2270e+00, 4.0171e-01],\n", + " [-2.6879e-01, 1.5706e+00],\n", + " [ 2.0390e+00, -1.8152e+00],\n", + " [-1.2317e+00, 6.0501e-01],\n", + " [ 4.1927e-02, -2.4078e+00],\n", + " [ 1.4265e+00, -9.2525e-01],\n", + " [ 5.9394e-01, -7.4587e-01],\n", + " [ 6.1463e-01, -2.6220e-01],\n", + " [ 6.7605e-01, -2.1740e-01],\n", + " [-5.9436e-01, 9.3632e-01],\n", + " [ 2.8933e-01, -1.3208e-02],\n", + " [-2.0206e+00, 3.9333e+00],\n", + " [ 2.4903e+00, -1.1771e+00],\n", + " [ 9.5112e-01, -1.7163e+00],\n", + " [ 1.7097e+00, -8.8835e-01],\n", + " [-3.4182e+00, 2.9024e+00],\n", + " [ 1.3908e+00, -1.7813e+00],\n", + " [-1.9825e+00, 2.1889e+00],\n", + " [ 3.2187e-02, -1.7187e-01],\n", + " [ 2.3456e+00, -3.3756e+00],\n", + " [-1.3550e+00, 3.8261e+00],\n", + " [-2.2111e+00, 1.7149e+00],\n", + " [-1.6745e+00, 2.1577e+00],\n", + " [-2.0270e+00, 3.8298e+00],\n", + " [ 1.6872e+00, -1.8343e+00],\n", + " [-1.3611e+00, 8.4912e-01],\n", + " [-1.5436e+00, 2.8195e+00],\n", + " [-8.4964e-01, 1.7983e+00],\n", + " [ 1.7100e+00, -6.1945e-01],\n", + " [ 1.7025e+00, -3.0842e+00],\n", + " [ 1.5122e+00, 7.2826e-01],\n", + " [ 9.1957e-01, -2.4107e+00],\n", + " [-2.5545e+00, 4.1697e+00],\n", + " [ 1.7832e-01, -2.9886e+00],\n", + " [-4.0698e+00, 5.4074e+00],\n", + " [ 1.3872e+00, -2.1134e+00],\n", + " [-3.7505e+00, 4.7124e+00],\n", + " [-1.6772e+00, 3.0610e+00],\n", + " [-7.6127e-01, 7.7868e-01],\n", + " [-5.1529e-01, 7.3124e-02],\n", + " [ 2.5028e+00, -5.3530e-01],\n", + " [ 2.1893e+00, -2.8006e+00],\n", + " [-1.7899e+00, 1.7446e+00],\n", + " [-1.1793e+00, 4.6499e+00],\n", + " [ 1.1547e+00, 1.7930e+00],\n", + " [-1.7421e-01, 2.7170e+00]], device='cuda:0', requires_grad=True)\n", + "\n", + " Number of elements in the tensor A 200\n" + ] + } + ], + "source": [ + "A=model_lora.fc1.lora.A\n", + "print(\"A\",A)\n", + "print(\"\\n Number of elements in the tensor A\",A.numel())\n", + "torch.save(A, 'A.pth')" + ] + }, + { + "cell_type": "markdown", + "id": "ca386b8c-f866-4f71-810b-324f96cb7a61", + "metadata": {}, + "source": [ + "A and B have approximately 450 parameters. If you were to store the entire linear layer, you would have 12,800 parameters, which is around 28 times more. Remember, this is possibly the simplest model that you can have.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "id": "2873e500-1b1b-4302-a782-fa3ab59b374e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + " Number of elements in the tensor A 12800\n" + ] + } + ], + "source": [ + "\n", + "print(\"\\n Number of elements in the tensor A\",model_lora.fc1.linear.weight.numel())\n" + ] + }, + { + "cell_type": "markdown", + "id": "7a88cfa5-73d4-458f-8c66-3229e4a5d49b", + "metadata": {}, + "source": [ + " alfa and the ouput layer are also saved.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "id": "6b203f60-2fdd-4c3e-9f22-55d0ba34827a", + "metadata": {}, + "outputs": [], + "source": [ + "alfa_=model_lora.fc1.lora.alpha\n", + "\n", + "torch.save(alfa_, 'alfa_.pth')\n", + "\n", + "torch.save(model_lora.fc2.state_dict(), 'out_layer.pth')" + ] + }, + { + "cell_type": "markdown", + "id": "8ec5a028-c58f-4ad6-8fcd-9fc94369c5ee", + "metadata": {}, + "source": [ + "## Loading the model\n", + "\n", + "The main advantage of LoRA is that for fine-tuning, you only need to save the learnable parameters A and B, alpha, and the output layer in your classification example.\n", + "\n", + "The saved files are converted to tensors and the linear layer, respectively.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "eff5f4e2-3b93-486d-8353-90657c44a97f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A: torch.Size([100, 2])\n" + ] + } + ], + "source": [ + "A = torch.load('A.pth')\n", + "print(\"A:\",A.shape)" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "id": "dd2b464e-5b25-4ce5-8534-e1c58b40a708", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B: torch.Size([2, 128])\n" + ] + } + ], + "source": [ + "B = torch.load('B.pth')\n", + "print(\"B:\",B.shape)" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "id": "042798fa-c836-4a1c-bda5-6df0a12c24d2", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAACEAAAAQCAYAAACYwhZnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAABJ0AAASdAHeZh94AAABlElEQVR4nM3VT4iNURjH8c+IjWkaNSUL5c/Nn51ZKCMLSQ3LWcySZCdEEzvq6VG2hMx6SvbWU5qVPylrmUEWZENIY+la3HN5O7ncidt46u33vk+/97zf85xznneo3W5b7VhbJzJzM67gKMbwDveQEfGx34EzcxoHMY49GMHdiDhWe9dUL7bwFCfxBNfxCufxKDPH+oXAZZwtEG9/Z6wrMYuNOBcRtxpw1zCDqzjVJ8QM3uCFTkUWehl/VKJUYRKvcbvyBZZxPDOH+yGIiIWIWIqIP2665nIcKjofEd+qAb/gAdZjoh+IlUQTYlfRxR7epaI7BwkxWvRzD283v2GQEKsWTYjuTEd/ZWzkPw0S4nnRXmu+o2ivPfNPILrneDIz6yY2ggP4iscDg4iIl5jHVpypfIlh3ImI5QZcKzN3Z+a6v4GoO+ZpPMTNzDyMZ9in00MWcany38cWbNNpcj+pM6cwVR43Fd2fmXPl/n1EXKQ6HaUaezFXPn4BLdzARER8WMEEx3GiXEdKbnsjN901Dv0Pv/LvvxltuHPwrH4AAAAASUVORK5CYII=", + "text/latex": [ + "$\\displaystyle 0.1$" + ], + "text/plain": [ + "0.1" + ] + }, + "execution_count": 93, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "alfa_ = torch.load('alfa_.pth')\n", + "alfa_ \n" + ] + }, + { + "cell_type": "markdown", + "id": "5f2f40f8-ebc8-4f6f-ad13-35c66f05b2e7", + "metadata": {}, + "source": [ + "The output layer:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "id": "b5ef67e4-d89d-4525-91c0-79c128af2c6b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "output_layer=nn.Linear(in_features=128, out_features=2, bias=True)\n", + "output_layer.load_state_dict(torch.load('out_layer.pth'))" + ] + }, + { + "cell_type": "markdown", + "id": "c2ce6ab9-6d2d-4274-9cbe-96f83c175159", + "metadata": {}, + "source": [ + "The model object is created and the pretrained parameters are loaded:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "id": "a1950bfb-dd37-421c-a288-502d89e8b8d6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): Linear(in_features=100, out_features=128, bias=True)\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=4, bias=True)\n", + ")" + ] + }, + "execution_count": 95, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "\n", + "model_load_lora = TextClassifier(num_classes=4,freeze=False)\n", + "model_load_lora.to(device)\n", + "\n", + "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/uGC04Pom651hQs1XrZ0NsQ/my-model-freeze-false.pth')\n", + "\n", + "stream = io.BytesIO(urlopened.read())\n", + "state_dict = torch.load(stream, map_location=device)\n", + "model_load_lora.load_state_dict(state_dict)\n", + "\n", + "model_load_lora" + ] + }, + { + "cell_type": "markdown", + "id": "6710f633-1588-47e0-9a76-da4aeb062784", + "metadata": {}, + "source": [ + "The LoRA layer object is added to the original hidden layer.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "id": "5c62b5ea-bc49-40f1-a488-9e107058547f", + "metadata": {}, + "outputs": [], + "source": [ + "model_load_lora.fc1=LinearWithLoRA(model_load_lora.fc1,rank=2, alpha=0.1)\n", + "model_load_lora.fc2=nn.Linear(in_features=128, out_features=2, bias=True).to(device)" + ] + }, + { + "cell_type": "markdown", + "id": "47867680-839c-43f2-8907-b11fce97f1aa", + "metadata": {}, + "source": [ + "The parameters from fine-tuning are added.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "id": "68cd3b53-fda5-42b4-965c-fbc98ef93999", + "metadata": {}, + "outputs": [], + "source": [ + "model_load_lora.fc1.lora.A=A\n", + "model_load_lora.fc1.lora.B=B\n", + "model_load_lora.fc1.lora.alpha=alfa_ \n", + "model_load_lora.fc2=output_layer" + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "id": "4fee5c83-2b3f-449e-bbad-1a612d600c18", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "TextClassifier(\n", + " (embedding): Embedding(400000, 100)\n", + " (fc1): LinearWithLoRA(\n", + " (linear): Linear(in_features=100, out_features=128, bias=True)\n", + " (lora): LoRALayer()\n", + " )\n", + " (relu): ReLU()\n", + " (fc2): Linear(in_features=128, out_features=2, bias=True)\n", + ")" + ] + }, + "execution_count": 98, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model_load_lora.to(device)\n", + "model_load_lora.eval()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "id": "8cb0716c-1dcd-42df-96ca-740384ef79fe", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEYAAAAQCAYAAACr+QluAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAABJ0AAASdAHeZh94AAAD0ElEQVR4nO3YaaiWRRQH8N81tYUKKooi2imwjVbFwBbbPkiSLRChWWB9SUwqKQQ7nSC0wkoIiizKCoo2KlHaDLJAEsLAsI3KoiJSy7TSzKUPMy88vj7Xe837sQMP8848//95zjlz5szM27Nt2zb/y44yuG0wMy/EZIzEAViD5ZgTEQsbuB5Mqs9J6MFneAKPR8TWvgzIzIMwDmNwCg7Hpvq9p/BUt57/wunl2+PxbO3eGBFPdN4NagHfj3dxFt7AbCzAwTi/C/4cHsfReF4JyD54FE/3ZViVqzEXI/ARHsYrOLnqe7FOwO5yuv08Ao/gj7b3g7vAN2Ia5uGmiNjU9X5I4/c4XItvMTwiVtfxodXICZn5WkS8ujMD8SXGYkFzljNzOpbiSlxRde4Op+lHj5JZa/Aqbu/GDGqA98S9+F5LUCAi/ml0x9V2dicoFbMJM2p3cpthXTrfi4j53akfET/jsdo9f3c5XTIFo3ED/mwDNDPmYmW5PIytmTlGSc2NWBoRS7q4h9b2mxa9nbFRmTm0Lcj9lM5EbB4oTmYOwyylXi7OzNFtuGZgzq7tRixTgtJUuBhXRcSqOtTJkmNa9B7b0H8sPu/dj3bJzMG4rnbfHAhOff+ssiqm70xXs/geUttp2IZR2A+n4m2ci5ca+AW1vTUzD2x8fAiygTtg5+70KrOUyVkYEW8NEOcunI7rI2LDzhQ1M6YTpM0YGxEra395LbRf4LzMHFmX1QuYgEuxIjNfV7LtIhymzMqR6HPb7JbMnILblEybMBCczByhZMnslrKwgzQzZm1tlzWCAiLiL3RmYHgd24LLcCdWYWJ9vsI5WF/xv/Tt1nYOTMYcrMAFEfHr7nLqEnpG2c1m7KhhR2lmzBe1XdsL9rfa7t0ZqLvUffVpGrIXjsfqiPi2P4ZU3lQ8hE9xYUT0GdR+cvbFCfX3xsxsgZibmXOVojy1GZhFSm05MTMHtZwcO8W4P45eg6HKoa9fkpl3KDXiE1zcPAIMAOdvPNnLuzOUuvOhkhxLaGRMRHyXmfOVg9Mtyix0DLhEqSVrNap9Zu4fEeu6jD0NDygZNqvFmeMwBF93zkWZOQP34GNc0s/l029OLbSTetFztxKYec0rQfdd6eYKerCeY5Yp2/Hl2IJJEfF7A/9OZm5Q0ng9hin3lw24LCJ+arFlEY6qeldm5sTq4BZ8gCktqb4yIp5uOLPLnF2V7QITET9k5pnKtjZW2aLXYT5mRsTSLv7LyrIZr9SeH5W708yI+KGfNnTOQXtgai+Y921/9/ovnF2Snv//dmiXfwHLqKZB8S0rTwAAAABJRU5ErkJggg==", + "text/latex": [ + "$\\displaystyle 69.224$" + ], + "text/plain": [ + "69.224" + ] + }, + "execution_count": 99, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "evaluate(test_dataloader , model_load_lora, device)" + ] + }, + { + "cell_type": "markdown", + "id": "aa14425e-297e-4fce-ab41-51686ebcbe9b", + "metadata": {}, + "source": [ + "This confirms that the model loaded correctly. You still get a 3% improvement in accuracy!\n", + "\n", + "Finally, the following shows how you can make a prediction on the following article using the function **`predict`**:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "id": "839bfe9f-fe50-4106-99f7-ede8dbdc2310", + "metadata": {}, + "outputs": [], + "source": [ + "article=\"\"\"This was a lacklustre movie with very little going for it. I was not impressed.\"\"\"" + ] + }, + { + "cell_type": "markdown", + "id": "468d8dc4-0c6c-4e9f-9433-d8b819f8afab", + "metadata": {}, + "source": [ + "This markdown content generates a styled box with light gray background and padding. It contains an `

` header displaying the content of the `article` variable, and an `

` header indicating the predicted category of the news article which is provided by the `result` variable. The placeholders `{article}` and `{result}` will be dynamically replaced with actual values when this markdown is rendered.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 101, + "id": "6264a056-7192-47fa-b02b-5f32d6e0e404", + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "\n", + "
\n", + "

This was a lacklustre movie with very little going for it. I was not impressed.

\n", + "

The category of the news article: negative review

\n", + "
\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 101, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result = predict(article, model_load_lora, text_pipeline)\n", + "\n", + "markdown_content = f'''\n", + "
\n", + "

{article}

\n", + "

The category of the news article: {result}

\n", + "
\n", + "'''\n", + "\n", + "md(markdown_content)" + ] + }, + { + "cell_type": "markdown", + "id": "d62be84d-bbc9-4021-bc17-762b265614b4", + "metadata": {}, + "source": [ + "---\n", + "## Exercise: Apply LoRA to a different network\n", + "\n", + "The following code defines a neural network called `NNet`. \n", + "\n", + "`NNet` is a neural network that was originally written to identify hand-written digits from 32x32 images. Your task is to fine-tune this network to perform letter recognition using LoRA by replacing the section labeled `### REPLACE THIS ###` in the code block below. To enhance your understanding, apply LoRA to just the second linear layer, and replace the last layer with a layer that has 26 outputs, one for each letter in the English alphabet.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "id": "b0a24c15-bcb2-4171-850b-4a47c1380244", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "This is what the model looked like before applying LoRA:\n", + "NNet(\n", + " (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))\n", + " (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n", + " (fc1): Linear(in_features=400, out_features=120, bias=True)\n", + " (fc2): Linear(in_features=120, out_features=84, bias=True)\n", + " (fc3): Linear(in_features=84, out_features=10, bias=True)\n", + ")\n", + "\n", + "###############\n", + "\n", + "This is what the model looked like after applying LoRA:\n", + "NNet(\n", + " (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))\n", + " (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n", + " (fc1): Linear(in_features=400, out_features=120, bias=True)\n", + " (fc2): LinearWithLoRA(\n", + " (linear): Linear(in_features=120, out_features=84, bias=True)\n", + " (lora): LoRALayer()\n", + " )\n", + " (fc3): Linear(in_features=84, out_features=26, bias=True)\n", + ")\n" + ] + } + ], + "source": [ + "#TODO\n", + "class NNet(nn.Module):\n", + "\n", + " def __init__(self):\n", + " super(NNet, self).__init__()\n", + " # 1 input image channel, 6 output channels, 5x5 square convolution\n", + " # kernel\n", + " self.conv1 = nn.Conv2d(1, 6, 5)\n", + " self.conv2 = nn.Conv2d(6, 16, 5)\n", + " # an affine operation: y = Wx + b\n", + " self.fc1 = nn.Linear(16 * 5 * 5, 120) # 5*5 from image dimension\n", + " self.fc2 = nn.Linear(120, 84)\n", + " self.fc3 = nn.Linear(84, 10)\n", + "\n", + " def forward(self, input):\n", + " # Convolution layer C1: 1 input image channel, 6 output channels,\n", + " # 5x5 square convolution, it uses RELU activation function, and\n", + " # outputs a Tensor with size (N, 6, 28, 28), where N is the size of the batch\n", + " c1 = F.relu(self.conv1(input))\n", + " # Subsampling layer S2: 2x2 grid, purely functional,\n", + " # this layer does not have any parameter, and outputs a (N, 6, 14, 14) Tensor\n", + " s2 = F.max_pool2d(c1, (2, 2))\n", + " # Convolution layer C3: 6 input channels, 16 output channels,\n", + " # 5x5 square convolution, it uses RELU activation function, and\n", + " # outputs a (N, 16, 10, 10) Tensor\n", + " c3 = F.relu(self.conv2(s2))\n", + " # Subsampling layer S4: 2x2 grid, purely functional,\n", + " # this layer does not have any parameter, and outputs a (N, 16, 5, 5) Tensor\n", + " s4 = F.max_pool2d(c3, 2)\n", + " # Flatten operation: purely functional, outputs a (N, 400) Tensor\n", + " s4 = torch.flatten(s4, 1)\n", + " # Fully connected layer F5: (N, 400) Tensor input,\n", + " # and outputs a (N, 120) Tensor, it uses RELU activation function\n", + " f5 = F.relu(self.fc1(s4))\n", + " # Fully connected layer F6: (N, 120) Tensor input,\n", + " # and outputs a (N, 84) Tensor, it uses RELU activation function\n", + " f6 = F.relu(self.fc2(f5))\n", + " # Gaussian layer OUTPUT: (N, 84) Tensor input, and\n", + " # outputs a (N, 10) Tensor\n", + " output = self.fc3(f6)\n", + " return output\n", + "\n", + "model_exercise = NNet()\n", + "model_exercise.to(device)\n", + "\n", + "print('This is what the model looked like before applying LoRA:')\n", + "print(model_exercise)\n", + "print(\"\\n###############\\n\")\n", + "\n", + "# Freeze all parameters:\n", + "for parm in model_exercise.parameters():\n", + " parm.requires_grad=False\n", + "\n", + "# Change final layer for one with 26 outputs:\n", + "model_exercise.fc3=nn.Linear(in_features=84, out_features=26, bias=True).to(device)\n", + "\n", + "# Apply LoRA to the second linear layer\n", + "model_exercise.fc2=LinearWithLoRA(model_exercise.fc2,rank=2, alpha=0.1).to(device)\n", + "\n", + "print('This is what the model looked like after applying LoRA:')\n", + "print(model_exercise)" + ] + }, + { + "cell_type": "markdown", + "id": "7f523e91-700c-4d28-998c-be37badf0c63", + "metadata": {}, + "source": [ + "
\n", + " Click here for the solution\n", + "\n", + "```python\n", + "class NNet(nn.Module):\n", + "\n", + " def __init__(self):\n", + " super(NNet, self).__init__()\n", + " # 1 input image channel, 6 output channels, 5x5 square convolution\n", + " # kernel\n", + " self.conv1 = nn.Conv2d(1, 6, 5)\n", + " self.conv2 = nn.Conv2d(6, 16, 5)\n", + " # an affine operation: y = Wx + b\n", + " self.fc1 = nn.Linear(16 * 5 * 5, 120) # 5*5 from image dimension\n", + " self.fc2 = nn.Linear(120, 84)\n", + " self.fc3 = nn.Linear(84, 10)\n", + "\n", + " def forward(self, input):\n", + " # Convolution layer C1: 1 input image channel, 6 output channels,\n", + " # 5x5 square convolution, it uses RELU activation function, and\n", + " # outputs a Tensor with size (N, 6, 28, 28), where N is the size of the batch\n", + " c1 = F.relu(self.conv1(input))\n", + " # Subsampling layer S2: 2x2 grid, purely functional,\n", + " # this layer does not have any parameter, and outputs a (N, 6, 14, 14) Tensor\n", + " s2 = F.max_pool2d(c1, (2, 2))\n", + " # Convolution layer C3: 6 input channels, 16 output channels,\n", + " # 5x5 square convolution, it uses RELU activation function, and\n", + " # outputs a (N, 16, 10, 10) Tensor\n", + " c3 = F.relu(self.conv2(s2))\n", + " # Subsampling layer S4: 2x2 grid, purely functional,\n", + " # this layer does not have any parameter, and outputs a (N, 16, 5, 5) Tensor\n", + " s4 = F.max_pool2d(c3, 2)\n", + " # Flatten operation: purely functional, outputs a (N, 400) Tensor\n", + " s4 = torch.flatten(s4, 1)\n", + " # Fully connected layer F5: (N, 400) Tensor input,\n", + " # and outputs a (N, 120) Tensor, it uses RELU activation function\n", + " f5 = F.relu(self.fc1(s4))\n", + " # Fully connected layer F6: (N, 120) Tensor input,\n", + " # and outputs a (N, 84) Tensor, it uses RELU activation function\n", + " f6 = F.relu(self.fc2(f5))\n", + " # Gaussian layer OUTPUT: (N, 84) Tensor input, and\n", + " # outputs a (N, 10) Tensor\n", + " output = self.fc3(f6)\n", + " return output\n", + "\n", + "model_exercise = NNet()\n", + "model_exercise.to(device)\n", + "\n", + "print('This is what the model looked like before applying LoRA:')\n", + "print(model_exercise)\n", + "print(\"\\n###############\\n\")\n", + "\n", + "# Freeze all parameters:\n", + "for parm in model_exercise.parameters():\n", + " parm.requires_grad=False\n", + "\n", + "# Change final layer for one with 26 outputs:\n", + "model_exercise.fc3=nn.Linear(in_features=84, out_features=26, bias=True).to(device)\n", + "\n", + "# Apply LoRA to the second linear layer\n", + "model_exercise.fc2=LinearWithLoRA(model_exercise.fc2,rank=2, alpha=0.1).to(device)\n", + "\n", + "print('This is what the model looked like after applying LoRA:')\n", + "print(model_exercise)\n", + "```\n", + "\n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "id": "61f9dd98-d9f8-4017-bbdd-976a69eddd24", + "metadata": {}, + "source": [ + "---" + ] + }, + { + "cell_type": "markdown", + "id": "7635b116-9be8-4a51-b687-9ddd98bc905e", + "metadata": {}, + "source": [ + "## Congratulations! You have completed the lab\n", + "\n", + "## Authors\n", + "\n", + "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his Ph.D.\n", + "\n", + "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) is a Data Scientist at IBM, and a PhD Candidate in economics at the University of Calgary.\n", + "\n", + "[Ashutosh Sagar](https://www.linkedin.com/in/ashutoshsagar/) is completing his MS in CS from Dalhousie University. He has previous experience working with Natural Language Processing and as a Data Scientist.\n", + "\n", + "## References\n", + "\n", + "[Finetuning with LoRA -- A Hands-On Example](https://lightning.ai/lightning-ai/studios/code-lora-from-scratch)\n", + "\n", + "[TEXT CLASSIFICATION WITH THE TORCHTEXT LIBRARY](https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html)\n", + "\n", + "\n", + "© Copyright IBM Corporation. All rights reserved.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.20" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/LLM_Specialization/QLoRA_with_Hugging_Face.ipynb b/notebooks/LLM_Specialization/QLoRA_with_Hugging_Face.ipynb new file mode 100644 index 0000000..f8ed8b0 --- /dev/null +++ b/notebooks/LLM_Specialization/QLoRA_with_Hugging_Face.ipynb @@ -0,0 +1,1312 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "

\n", + " \n", + " \"Skills\n", + " \n", + "

\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# QLoRA with HuggingFace\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "QLoRA is an extension of LoRA that leverages quantization. Quantization is the process of mapping continuous infinite values to a smaller set of discrete finite values. Effectively, the model's parameters are are stored in 2, 3, 4 or 8-bits as opposed to the usual 32-bits, lowering the number of bits needed to store information. Quantization offers two benefits:\n", + "\n", + "1. It reduced memory footprint. By using a finite set of discrete levels, the values can be represented with fewer bits, reducing the memory required to store them; and\n", + "2. It allows for efficient computation. Quantized values can be represented and processed more efficiently on hardware with limited numerical precision, such as low-power microcontrollers or specialized AI/ML accelerators.\n", + "\n", + "Choosing QLoRA over LoRA provides several tradeoffs. QLoRA offers the following advantages of LoRA:\n", + "\n", + "1. Substantially smaller GPU memory usage than LoRA.\n", + "2. Higher maximum sequence lengths resulting from the smaller GPU memory usage.\n", + "3. Higher batch sizes resulting from the smaller GPU memory usage.\n", + "\n", + "The main disadvantage of QLoRA is slower fine-tuning speed.\n", + "\n", + "Interestingly enough, the accuracy of QLoRA and LoRA are comparable despite the fact that QLoRA offers substantially smaller models with lower GPU memory footprints than LoRA.\n", + "\n", + "The original QLoRA paper is available [here](https://arxiv.org/pdf/2305.14314).\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Note that the following uses the popular `BitsAndBytes` library to implement QLoRA, which only supports quantization using a CUDA-enabled GPU. You will not be able to run this notebook without a compatible GPU!**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# __Table of Contents__\n", + "\n", + "
    \n", + "
  1. Objectives
    2. \n", + "
  2. \n", + " Setup\n", + "
      \n", + "
    1. Install required libraries
      2. \n", + "
    2. Import required libraries
      3. \n", + "
    3. Define helper functions
      4. \n", + "
    \n", + "
    3. \n", + "
  3. IMDB dataset
    4. \n", + "
  4. Tokenizer
    5. \n", + "
  5. Configure BitsAndBytes
    6. \n", + "
  6. Load a quantized version of a pretrained model
    7. \n", + "
  7. Train
    8. \n", + "
  8. Results
    9. \n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Objectives\n", + "\n", + "After completing this lab you will be able to:\n", + "\n", + "- Load and predict using models from HuggingFace\n", + "- Fine-tune language models using QLoRA\n", + "- Understand the advantages and disadvantages of QLoRA\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Setup\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install required libraries\n", + "\n", + "For this lab, you use the following libraries, which are __not__ preinstalled in the Skills Network Labs environment. __You must run the code in the following cell__ to install them.\n", + "\n", + "\n", + "```bash\n", + "!pip install -U datasets==2.20.0 huggingface_hub==0.23.4 transformers==4.41.2 peft==0.11.1 bitsandbytes==0.43.1 matplotlib==3.9.0 scikit-learn==1.5.0\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import required libraries\n", + "\n", + "The following code imports required libraries.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True\n", + "Tesla P40\n", + "Import Successfully!\n" + ] + } + ], + "source": [ + "import os\n", + "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"\n", + "\n", + "import torch\n", + "print(torch.cuda.is_available())\n", + "print(torch.cuda.get_device_name())\n", + "\n", + "import matplotlib.pyplot as plt\n", + "# You can also use this section to suppress warnings generated by your code:\n", + "def warn(*args, **kwargs):\n", + " pass\n", + "import warnings\n", + "warnings.warn = warn\n", + "warnings.filterwarnings('ignore')\n", + "\n", + "import json\n", + "\n", + "import numpy as np\n", + "\n", + "from datasets import load_dataset #load_metric\n", + "\n", + "from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments, BitsAndBytesConfig\n", + "\n", + "from peft import LoraConfig, get_peft_model, TaskType, replace_lora_weights_loftq, prepare_model_for_kbit_training\n", + "\n", + "print(\"Import Successfully!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's check whether a compatible GPU is available:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "device(type='cuda')" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Move the model to the appropriate device\n", + "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n", + "device" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Define helper functions\n", + "\n", + "Here are some helper functions. We will use these later to save and load from JSON:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "def save_to_json(data, file_path):\n", + " \"\"\"\n", + " Save a dictionary to a JSON file.\n", + "\n", + " Args:\n", + " data (dict): The dictionary to save.\n", + " file_path (str): The path to the JSON file.\n", + " \"\"\"\n", + " with open(file_path, 'w') as json_file:\n", + " json.dump(data, json_file, indent=4)\n", + " print(f\"Data successfully saved to {file_path}\")\n", + " \n", + " \n", + "def load_from_json(file_path):\n", + " \"\"\"\n", + " Load data from a JSON file.\n", + "\n", + " Args:\n", + " file_path (str): The path to the JSON file.\n", + "\n", + " Returns:\n", + " dict: The data loaded from the JSON file.\n", + " \"\"\"\n", + " with open(file_path, 'r') as json_file:\n", + " data = json.load(json_file)\n", + " return data " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# IMDB dataset \n", + "\n", + "The IMDB dataset is a large movie review dataset, consisting of 50,000 movie reviews for training and 25,000 movie reviews for testing. The reviews are labeled as either positive or negative, and each review is a variable-length sequence of words. The IMDB dataset is a popular benchmark for text classification tasks, and it has been used to train a variety of natural language processing models. The following line loads the IMDB dataset:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "imdb = load_dataset(\"imdb\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's display the structure of the IMDB dataset:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dataset structure:\n", + "DatasetDict({\n", + " train: Dataset({\n", + " features: ['text', 'label'],\n", + " num_rows: 25000\n", + " })\n", + " test: Dataset({\n", + " features: ['text', 'label'],\n", + " num_rows: 25000\n", + " })\n", + " unsupervised: Dataset({\n", + " features: ['text', 'label'],\n", + " num_rows: 50000\n", + " })\n", + "})\n" + ] + } + ], + "source": [ + "# Display the structure of the dataset\n", + "print(\"Dataset structure:\")\n", + "print(imdb)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following displays the available splits in the dataset (train, test, unsupervised)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "dict_keys(['train', 'test', 'unsupervised'])" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "imdb.keys()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's explore and print a sample from the training set:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Sample from the training set:\n", + "{'text': 'I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if it ever tried to enter this country, therefore being a fan of films considered \"controversial\" I really had to see this for myself.

The plot is centered around a young Swedish drama student named Lena who wants to learn everything she can about life. In particular she wants to focus her attentions to making some sort of documentary on what the average Swede thought about certain political issues such as the Vietnam War and race issues in the United States. In between asking politicians and ordinary denizens of Stockholm about their opinions on politics, she has sex with her drama teacher, classmates, and married men.

What kills me about I AM CURIOUS-YELLOW is that 40 years ago, this was considered pornographic. Really, the sex and nudity scenes are few and far between, even then it\\'s not shot like some cheaply made porno. While my countrymen mind find it shocking, in reality sex and nudity are a major staple in Swedish cinema. Even Ingmar Bergman, arguably their answer to good old boy John Ford, had sex scenes in his films.

I do commend the filmmakers for the fact that any sex shown in the film is shown for artistic purposes rather than just to shock people and make money to be shown in pornographic theaters in America. I AM CURIOUS-YELLOW is a good film for anyone wanting to study the meat and potatoes (no pun intended) of Swedish cinema. But really, this film doesn\\'t have much of a plot.', 'label': 0}\n" + ] + } + ], + "source": [ + "# Explore and print a sample from the training set\n", + "print(\"\\nSample from the training set:\")\n", + "print(imdb['train'][0])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The followiong displays the unique class labels in the dataset. For the IMDB dataset, the labels are integers representing sentiment, where 0 stands for “negative” and 1 stands for “positive”. Here’s how you can extract this information:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Unique labels in the dataset (class information):\n", + "{0, 1}\n" + ] + } + ], + "source": [ + "train_labels = imdb['train']['label']\n", + "unique_labels = set(train_labels)\n", + "print(\"\\nUnique labels in the dataset (class information):\")\n", + "print(unique_labels)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following dictionary maps class values to class names:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{0: 'negative', 1: 'positive'}" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "class_names = {0: \"negative\", 1: \"positive\"}\n", + "class_names" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since the IMDB dataset is quite large, we’ll create smaller subsets to facilitate quicker training and testing.\n", + "\n", + "In this notebook, we simulate training and testing using the `small_` datasets due to time constraints. However, it's important to recognize that these smaller datasets are insufficient for proper fine-tuning of the DistilBERT model. For more accurate results, a larger subsample, such as the `medium_train_dataset`, would be necessary.\n", + "\n", + "Consequently, all results presented here pertain to models trained with the `medium_train_dataset` and evaluated on the test set from `medium_test_dataset`. However, the notebook, as is, does NOT train models on these datasets; rather, it trains models using the `small_` datasets, as training on the `medium_` datasets would be too time-consuming.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "small_train_dataset = imdb[\"train\"].shuffle(seed=42).select([i for i in list(range(50))])\n", + "small_test_dataset = imdb[\"test\"].shuffle(seed=42).select([i for i in list(range(50))])\n", + "medium_train_dataset = imdb[\"train\"].shuffle(seed=42).select([i for i in list(range(3000))])\n", + "medium_test_dataset = imdb[\"test\"].shuffle(seed=42).select([i for i in list(range(300))])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Tokenizer\n", + "\n", + "The following loads the DistilBERT tokenizer: \n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The tokenizer provides tokenized input IDs and an attention mask for a given input text:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Input IDs: [101, 1045, 12524, 1045, 2572, 8025, 1011, 3756, 2013, 2026, 2678, 3573, 2138, 1997, 2035, 1996, 6704, 2008, 5129, 2009, 2043, 2009, 2001, 2034, 2207, 1999, 3476, 1012, 1045, 2036, 2657, 2008, 2012, 2034, 2009, 2001, 8243, 2011, 1057, 1012, 1055, 1012, 8205, 2065, 2009, 2412, 2699, 2000, 4607, 2023, 2406, 1010, 3568, 2108, 1037, 5470, 1997, 3152, 2641, 1000, 6801, 1000, 1045, 2428, 2018, 2000, 2156, 2023, 2005, 2870, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 1996, 5436, 2003, 8857, 2105, 1037, 2402, 4467, 3689, 3076, 2315, 14229, 2040, 4122, 2000, 4553, 2673, 2016, 2064, 2055, 2166, 1012, 1999, 3327, 2016, 4122, 2000, 3579, 2014, 3086, 2015, 2000, 2437, 2070, 4066, 1997, 4516, 2006, 2054, 1996, 2779, 25430, 14728, 2245, 2055, 3056, 2576, 3314, 2107, 2004, 1996, 5148, 2162, 1998, 2679, 3314, 1999, 1996, 2142, 2163, 1012, 1999, 2090, 4851, 8801, 1998, 6623, 7939, 4697, 3619, 1997, 8947, 2055, 2037, 10740, 2006, 4331, 1010, 2016, 2038, 3348, 2007, 2014, 3689, 3836, 1010, 19846, 1010, 1998, 2496, 2273, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 2054, 8563, 2033, 2055, 1045, 2572, 8025, 1011, 3756, 2003, 2008, 2871, 2086, 3283, 1010, 2023, 2001, 2641, 26932, 1012, 2428, 1010, 1996, 3348, 1998, 16371, 25469, 5019, 2024, 2261, 1998, 2521, 2090, 1010, 2130, 2059, 2009, 1005, 1055, 2025, 2915, 2066, 2070, 10036, 2135, 2081, 22555, 2080, 1012, 2096, 2026, 2406, 3549, 2568, 2424, 2009, 16880, 1010, 1999, 4507, 3348, 1998, 16371, 25469, 2024, 1037, 2350, 18785, 1999, 4467, 5988, 1012, 2130, 13749, 7849, 24544, 1010, 15835, 2037, 3437, 2000, 2204, 2214, 2879, 2198, 4811, 1010, 2018, 3348, 5019, 1999, 2010, 3152, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 1045, 2079, 4012, 3549, 2094, 1996, 16587, 2005, 1996, 2755, 2008, 2151, 3348, 3491, 1999, 1996, 2143, 2003, 3491, 2005, 6018, 5682, 2738, 2084, 2074, 2000, 5213, 2111, 1998, 2191, 2769, 2000, 2022, 3491, 1999, 26932, 12370, 1999, 2637, 1012, 1045, 2572, 8025, 1011, 3756, 2003, 1037, 2204, 2143, 2005, 3087, 5782, 2000, 2817, 1996, 6240, 1998, 14629, 1006, 2053, 26136, 3832, 1007, 1997, 4467, 5988, 1012, 2021, 2428, 1010, 2023, 2143, 2987, 1005, 1056, 2031, 2172, 1997, 1037, 5436, 1012, 102]\n", + "Attention Mask: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]\n" + ] + } + ], + "source": [ + "my_tokens=tokenizer(imdb['train'][0]['text'])\n", + "\n", + "# Print the tokenized input IDs\n", + "print(\"Input IDs:\", my_tokens['input_ids'])\n", + "\n", + "# Print the attention mask\n", + "print(\"Attention Mask:\", my_tokens['attention_mask'])\n", + "\n", + "# If token_type_ids is present, print it\n", + "if 'token_type_ids' in my_tokens:\n", + " print(\"Token Type IDs:\", my_tokens['token_type_ids'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following preprocessing function tokenizes a text input. We apply this function to all texts in our datasets using the `.map()` method:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0893c8e8a7d3452abbcabc48a4e8604d", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Map: 0%| | 0/50 [00:00\n", + " \n", + " \n", + " [1880/1880 20:38, Epoch 10/10]\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
EpochTraining LossValidation LossAccuracy
1No log0.6294510.786667
2No log0.3796230.823333
30.5182000.3683520.826667
40.5182000.3603060.826667
50.5182000.3580130.830000
60.3086000.3576410.830000
70.3086000.3535800.830000
80.2841000.3549110.830000
90.2841000.3536850.840000
100.2841000.3548080.843333

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0428b91092944c3b980428bdd8415084", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Downloading builder script: 0%| | 0.00/4.20k [00:00" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGwCAYAAAB7MGXBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9fklEQVR4nO3deXxU9b3/8ffMJJkkkAWCJAESwhKUPewFtGgNxbZi9fFTsaIgKH3IUoHUBVREoZraCuXiAsoF0SIFS/VWLxaFKFgQi0Kx6EV2CApJ2ExIQraZ+f0xyZCBBDIhyTeTeT0fj/OYkzPnzHySoPPOdzsWl8vlEgAAgCFW0wUAAIDARhgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFFBpguoCafTqWPHjikiIkIWi8V0OQAAoAZcLpfOnj2rNm3ayGqtvv3DL8LIsWPHlJCQYLoMAABQC0ePHlW7du2qfd4vwkhERIQk9zcTGRlpuBoAAFATeXl5SkhI8HyOV8cvwkhF10xkZCRhBAAAP3O5IRYMYAUAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABjlFzfKAwAAdcvpdOlcqUOFJQ4VlpQpNjJUocE2I7UQRgAAaMRcrkqhodihgpIyT4AoKHboXKn7sdBz3KGC4jKdKzl/bkHx+ecqn1fZ+1OuVc92UUa+R8IIAAB1wOVyqajUqYKS80GgoNhRKRSUeQUKz/Hi8gBR4h0Y3NeWqbDUIZer/uq2WKTwYJtKHI7Ln1xPCCMAAFyCy+XSqYISHTlVqMzTBe7HU4U6crpQp/KLVVDiUGFx/YcGSQoPsSk8JEjN7DaFBdvUzB5UfsymZiFBCre7H8Mqfe25puL4BeeEBltlsVjqt/DLIIwAAAKew+nSsR/OKfN0oY6cKtSR0wXuwHGqUJmnC5VfXObT67mDgjsEeMKCPeiiAFERLM6fV/H1xQEiNMgmq9VsaKgvhBEAQEAoKnWcDxunCjz7macL9d2ZQpU6qm/WsFikuMhQJbYMV/uYcLWPaabEluGKiwpVs4qWivKWhrDgphsa6gthBADQZPxQWFLeslGozFMF5cHD3dKRnVd8yWtDbFa1axmm9i3Phw138AhXuxbhxmaaBALCCADAbzidLmXlFXmN33AHD3drR17RpbtTIuxBSowJV1JMMyXGhKt9y3D3Y0wzxUWGykaLhhGEEQBAo1Jc5tB3Z87pSKWWDXeXSoGOnjmnkjLnJa9vHWFX+5hwJbZs5mnZSCxv7WgRHmx8sCYuRhgBADS4vKJSzwDRCweLHss9d8lZKUFWi9q2CFP7mGblXSrnw0Ziy3CFhdCd4m8II0AT4XC6Llr0qPICRxcuelTVYkmeKYqVrqvvqYoIPC65LjlYVHJPYb1wsGj7mHC1b9lMbaJDFWTjbiZNCWEEaGBOp0uFpeUf9l6LH7mDQEGJe6GjysGgwGvVxKpXWyy+TNM10Ni0ah7i1aJxvkulmVo1D6E7JYAQRlDnPB+2ng/SSn+RX+bDtqgBFg1qCE6XexphVassniut31UOrRZVWuwoyLMYUsVaBV7rGYSUT0e0ex8LL/86LNjGgD7Ui8iwYDW38xEEN/4lBLCKmyRV/qAsLKkUFqpqvi+tHCgqXVt6/q/8olL+Qq8JS0VouGDxo7CKQFDxXPlqidWtvHjhQkr2IPOrKQKALwgjTcCxH85p454Tyi8urfpmSaXnxwt4/lIvrv+/0Ct/2J7/wLx4pUHPssUhNoUG22RtAh+kFot7BcaK79M7ULh/HoQGAHAjjPixYz+c08uf7NfbXx697GCwy6ncNO/VfF/p3gbuD9OLAwQftgCAK0EY8UPHc90hZPUX50NI38RotY9p5gkCFzX9B3vf/6ByyGjK9zsAADR+hBE/cjz3nBZtPKBV246qxOEel/Gjji01LbWLftQxxnB1AADUDmHED2TlFmnRxv36S6UQMrBDS01P7aLBnQghAAD/RhhpxLLzirRo4wGt3JbpWf54YFJLTRuerCGdWhmuDgCAukEYaYSqCiEDklp4WkIYFAoAaEoII41ITl6RFm06oJX/yvSsptm/fQtNH95FQwghAIAmijDSCOScLdLijQf11r+OeEJIv/bulpChnQkhAICmjTBiUM7ZIr266aBWfH4+hPRNjNb04V10bedWhBAAQEAgjBhw4myxXt10QCv+dcSzdHqfxGhNT+2i65IJIQCAwEIYaUAn890h5M+fnw8hKQnulpAfE0IAAAGKMNIATuYX67VPD+rPW4947gfTOyFa01OTNazLVYQQAEBAI4zUo1PlIeTNyiGkXZSmDe+i6wkhAABIkqy1uejll19WUlKSQkNDNWjQIG3btu2S5y9YsEBXX321wsLClJCQoOnTp6uoqKhWBfuD0wUl+v0/vtV1f/hEr356UOdKHerVLkqv3zdA/zN5qG64ujVBBACAcj63jKxevVppaWlavHixBg0apAULFmjEiBHas2ePWrdufdH5K1eu1IwZM7Rs2TINGTJEe/fu1X333SeLxaL58+fXyTfRWJwuKNGSfx7UG58dVmGJuyWkV7soTUtNJoAAAFANi8vl8une84MGDdKAAQP00ksvSZKcTqcSEhL0m9/8RjNmzLjo/ClTpmj37t3KyMjwHPvtb3+rf/3rX9q8eXON3jMvL09RUVHKzc1VZGSkL+U2iDOVQkhBeQjp2dYdQn5yDSEEABCYavr57VPLSElJibZv366ZM2d6jlmtVqWmpmrr1q1VXjNkyBCtWLFC27Zt08CBA3Xw4EF98MEHuvfee6t9n+LiYhUXF3t9M43RmYIS/ffmg1q+5XwI6dE2UtNu7KIbuxJCAACoCZ/CyMmTJ+VwOBQbG+t1PDY2Vt9++22V19x99906efKkrr32WrlcLpWVlenBBx/U448/Xu37pKen65lnnvGltAb1Q2GJ/vufh7T8s8PKLy6TJHVvE6lpqV2USggBAMAntRrA6ouNGzfqueee0yuvvKIdO3bonXfe0dq1azV37txqr5k5c6Zyc3M929GjR+u7zBrJLSzVvI/26NrnP9FLn+xXfnGZusVH6rV7++l/f3OthneLJYgAAOAjn1pGWrVqJZvNpuzsbK/j2dnZiouLq/KaWbNm6d5779UDDzwgSerZs6cKCgr061//Wk888YSs1ovzkN1ul91u96W0epVbWKqlmw/q9S2Hdba8JaRrfKSmpSbrpwQQAACuiE9hJCQkRP369VNGRoZuvfVWSe4BrBkZGZoyZUqV1xQWFl4UOGw2myTJx7GzDS73XKmWbj6k1zcf8oSQa+IiykNInKxWQggAAFfK56m9aWlpGjt2rPr376+BAwdqwYIFKigo0Lhx4yRJY8aMUdu2bZWeni5JGjlypObPn68+ffpo0KBB2r9/v2bNmqWRI0d6Qkljk3uuVMs2H9KyLYd0tuh8CJl6Y7JGdCeEAABQl3wOI6NGjdKJEyf01FNPKSsrSykpKVq3bp1nUGtmZqZXS8iTTz4pi8WiJ598Ut9//72uuuoqjRw5Us8++2zdfRd1JK/IHUKWbj4fQq6OjdDU1GTdRAgBAKBe+LzOiAn1vc5IXlGpXt98WEs3H1ReeQjpEttcU2/sop/1IIQAAFAb9bLOSFNztqhUy7cc1n9vPqTcc6WSpOTWzTU1NVk/7xFPCAEAoAEEbBgpKXPqp3/6VMdz3ffI6dy6uabemKxf9CSEAADQkAI2jIQEWXVzr3h9sueEHioPITZCCAAADS6gx4ycK3EoJMhKCAEAoB4wZqQGwkIa59RiAAACSb0vBw8AAHAphBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFG1CiMvv/yykpKSFBoaqkGDBmnbtm3Vnnv99dfLYrFctP3iF7+oddEAAKDp8DmMrF69WmlpaZo9e7Z27Nih3r17a8SIEcrJyany/HfeeUfHjx/3bF9//bVsNpvuuOOOKy4eAAD4P5/DyPz58zVhwgSNGzdO3bp10+LFixUeHq5ly5ZVeX7Lli0VFxfn2davX6/w8PBLhpHi4mLl5eV5bQAAoGnyKYyUlJRo+/btSk1NPf8CVqtSU1O1devWGr3G0qVLddddd6lZs2bVnpOenq6oqCjPlpCQ4EuZAADAj/gURk6ePCmHw6HY2Fiv47GxscrKyrrs9du2bdPXX3+tBx544JLnzZw5U7m5uZ7t6NGjvpQJAAD8SFBDvtnSpUvVs2dPDRw48JLn2e122e32BqoKAACY5FPLSKtWrWSz2ZSdne11PDs7W3FxcZe8tqCgQKtWrdL999/ve5UAAKDJ8imMhISEqF+/fsrIyPAcczqdysjI0ODBgy957V//+lcVFxfrnnvuqV2lAACgSfK5myYtLU1jx45V//79NXDgQC1YsEAFBQUaN26cJGnMmDFq27at0tPTva5bunSpbr31VsXExNRN5QAAoEnwOYyMGjVKJ06c0FNPPaWsrCylpKRo3bp1nkGtmZmZslq9G1z27NmjzZs366OPPqqbqgEAQJNhcblcLtNFXE5eXp6ioqKUm5uryMhI0+UAAIAaqOnnN/emAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUUGmCwAANDyn06mSkhLTZcDPBQcHy2azXfHrBG4YcbmkYzukfeulwVMke3PTFQFAgygpKdGhQ4fkdDpNl4ImIDo6WnFxcbJYLLV+jcANI5L013HSD0ekuJ7SNb8wXQ0A1DuXy6Xjx4/LZrMpISFBViu99agdl8ulwsJC5eTkSJLi4+Nr/VqBG0YsFqnLCGnba9LeDwkjAAJCWVmZCgsL1aZNG4WHh5suB34uLCxMkpSTk6PWrVvXussmsCNx8gj347717m4bAGjiHA6HJCkkJMRwJWgqKkJtaWlprV8jsMNI0rVScLh09piU/bXpagCgwVxJ/z5QWV38WwrsMBIcKnUY5t7f+6HZWgAACFCBHUYkKXm4+3HfR2brAAAYtXz5ckVHR5suIyARRpJ/6n787gup8LTZWgAACECEkegEqXV3yeWU9m8wXQ0AALXiz4vYEUYkqUt56whdNQACjMvlUmFJmZHN5eMsRqfTqfT0dHXo0EFhYWHq3bu31qxZI6fTqXbt2mnRokVe5//73/+W1WrVkSNHJEnz589Xz5491axZMyUkJGjSpEnKz8+v1c/twIED+uUvf6nY2Fg1b95cAwYM0IYN3n/QFhcX67HHHlNCQoLsdrs6d+6spUuXep7/5ptvdPPNNysyMlIRERG67rrrdODAAUnS9ddfr2nTpnm93q233qr77rvP83VSUpLmzp2rMWPGKDIyUr/+9a8lSY899pi6dOmi8PBwdezYUbNmzbpopsv777+vAQMGKDQ0VK1atdJtt90mSZozZ4569Ohx0febkpKiWbNm1epnVROBu85IZckjpM1/creMOB2S9cqXtgUAf3Cu1KFuT5kZwP9/c0YoPKTmH0Pp6elasWKFFi9erOTkZH366ae655579OGHH+pXv/qVVq5cqYkTJ3rOf+uttzR06FC1b99ekmS1WrVw4UJ16NBBBw8e1KRJk/Too4/qlVde8bn2/Px8/fznP9ezzz4ru92uN998UyNHjtSePXuUmJgoSRozZoy2bt2qhQsXqnfv3jp06JBOnjwpSfr+++/14x//WNdff70+/vhjRUZGasuWLSorK/OpjhdeeEFPPfWUZs+e7TkWERGh5cuXq02bNtq1a5cmTJigiIgIPfroo5KktWvX6rbbbtMTTzyhN998UyUlJfrggw8kSePHj9czzzyjL774QgMGDJDkDnX/+c9/9M477/j8c6opi8vXaGpAXl6eoqKilJubq8jIyLp/A0eZ9MdOUtEP0vgPpcQf1f17AEAjUFRUpEOHDqlDhw4KDQ1VYUmZX4SR4uJitWzZUhs2bNDgwYM9xx944AEVFhbq0UcfVd++fXX48GElJibK6XQqMTFRTz75pB588MEqX3PNmjV68MEHPQFh+fLlmjZtmn744YdafT89evTQgw8+qClTpmjv3r26+uqrtX79eqWmpl507uOPP65Vq1Zpz549Cg4Ovuj566+/XikpKVqwYIHn2K233qro6GgtX75ckrtlpE+fPnr33XcvWdcLL7ygVatW6csvv5QkDRkyRB07dtSKFSuqPP/nP/+5kpKSPCHtoYce0q5du/TJJ59Uef6F/6Yqq+nnNy0jkmQLkjrfKH39N/cUX8IIgAARFmzT/80ZYey9a2r//v0qLCzU8OHDvY6XlJSoT58+SklJUdeuXbVy5UrNmDFDmzZtUk5Oju644w7PuRs2bFB6erq+/fZb5eXlqaysTEVFRSosLPR5Ndr8/Hw9/fTTWrt2rY4fP66ysjKdO3dOmZmZkqSdO3fKZrNp2LBhVV6/c+dOXXfddVUGEV/079//omOrV6/WwoULdeDAAeXn56usrMwrCOzcuVMTJkyo9jUnTJig8ePHa/78+bJarVq5cqX+9Kc/XVGdl0MYqZA8wh1G9n0kpc6+/PkA0ARYLBafukpMqRjbsXbtWrVt29brObvdLkkaPXq0J4ysXLlSN910k2JiYiRJhw8f1s0336yJEyfq2WefVcuWLbV582bdf//9Kikp8TmMPPzww1q/fr1eeOEFde7cWWFhYbr99ts9g0grlkmvzuWet1qtF42pqWqF02bNmnl9vXXrVo0ePVrPPPOMRowYoaioKK1atUrz5s2r8XuPHDlSdrtd7777rkJCQlRaWqrbb7/9ktdcqcb/L7ChdE6VZHGvxJr7nRTVznRFAIBy3bp1k91uV2ZmZrWtDXfffbeefPJJbd++XWvWrNHixYs9z23fvl1Op1Pz5s3z3Bzw7bffrnU9W7Zs0X333ecZ+Jmfn6/Dhw97nu/Zs6ecTqc2bdpUZTdNr1699MYbb6i0tLTK1pGrrrpKx48f93ztcDj09ddf64YbbrhkXZ999pnat2+vJ554wnOsYgBv5ffOyMjQuHHjqnyNoKAgjR07Vq+//rpCQkJ01113XTbAXClm01RoFiO1cw/W0b71ZmsBAHiJiIjQww8/rOnTp+uNN97QgQMHtGPHDr344ot64403JLnHUAwZMkT333+/HA6HbrnlFs/1nTt3VmlpqV588UUdPHhQf/7zn73Ciq+Sk5P1zjvvaOfOnfrqq6909913y+l0ep5PSkrS2LFjNX78eP3P//yPDh06pI0bN3oC0JQpU5SXl6e77rpLX375pfbt26c///nP2rNnjyTpJz/5idauXau1a9fq22+/1cSJE2s0liU5OVmZmZlatWqVDhw4oIULF140pmT27Nn6y1/+otmzZ2v37t3atWuXnn/+ea9zHnjgAX388cdat26dxo8fX+ufU00RRipjii8ANFpz587VrFmzlJ6erq5du+qmm27S2rVr1aFDB885o0eP1ldffaXbbrvN66/53r17a/78+Xr++efVo0cPvfXWW0pPT691LfPnz1eLFi00ZMgQjRw5UiNGjFDfvn29zlm0aJFuv/12TZo0Sddcc40mTJiggoICSVJMTIw+/vhj5efna9iwYerXr5+WLFniaSUZP368xo4dqzFjxmjYsGHq2LHjZVtFJOmWW27R9OnTNWXKFKWkpOizzz67aEru9ddfr7/+9a967733lJKSop/85Cfatm2b1znJyckaMmSIrrnmGg0aNKjWP6eaYjZNZcf/I716nfvmeY8ect+7BgCakEvNfAAquFwuJScna9KkSUpLS7vkuXUxm6ZWLSMvv/yykpKSFBoaqkGDBl2UqC70ww8/aPLkyYqPj5fdbleXLl08c5oblbieUkS8VFooHdlsuhoAABrciRMn9NJLLykrK6vacSV1zecwsnr1aqWlpWn27NnasWOHevfurREjRignJ6fK80tKSjR8+HAdPnxYa9as0Z49e7RkyZKLRkM3ChbL+Rvn7aWrBgACVffu3dW8efMqt7feest0efWqdevWmjNnjl577TW1aNGiQd7T59k08+fP14QJEzxpafHixVq7dq2WLVumGTNmXHT+smXLdPr0aX322WeevrCkpKQrq7o+JY+Qdrwp7ftQcj3vDigAgIDywQcfVDmVVpJiY2MbuJqGZWL0hk9hpKSkRNu3b9fMmTM9x6xWq1JTU7V169Yqr3nvvfc0ePBgTZ48WX//+9911VVX6e6779Zjjz0mm63qBW+Ki4tVXFzs+TovL8+XMq9Mx+slW4h05rB0ar/UKrnh3hsA0ChULCGPhuFTN83JkyflcDguSoWxsbHKysqq8pqDBw9qzZo1cjgc+uCDDzRr1izNmzdPv/vd76p9n/T0dEVFRXm2hIQEX8q8MvbmUvuh7v29ZpZIBgAgkNT71F6n06nWrVvrtddeU79+/TRq1Cg98cQTl5zfPXPmTOXm5nq2o0eP1neZ3pIrpvgSRgAAqG8+ddO0atVKNptN2dnZXsezs7MVFxdX5TXx8fEKDg726pLp2rWrsrKyVFJSopCQkIuusdvtnuV9jegyQvpwpnTkM6koTwqtx+nEAAAEOJ9aRkJCQtSvXz9lZGR4jjmdTmVkZHjdRbGyoUOHav/+/V4r0+3du1fx8fFVBpFGIaaT1LKT5CyTDlZ9l0IAAFA3fO6mSUtL05IlS/TGG29o9+7dmjhxogoKCjyza8aMGeM1wHXixIk6ffq0pk6dqr1792rt2rV67rnnNHny5Lr7LupDl/K7WLIaKwAA9crnMDJq1Ci98MILeuqpp5SSkqKdO3dq3bp1nkGtmZmZXjf3SUhI0IcffqgvvvhCvXr10kMPPaSpU6dWOQ24UfGMG1kvVWrVAQA0TcuXL1d0dHSNzn366aeVkpJSr/UEklrdtXfKlCmaMmVKlc9t3LjxomODBw/W559/Xpu3Mqf9ECmkuZSfLWV9JbXpY7oiAACaJG6UV50gu3vNEYnVWAEAqEeEkUthii8ANBpOp1Pp6enq0KGDwsLC1Lt3b61Zs0ZOp1Pt2rXTokWLvM7/97//LavVqiNHjkhyryDes2dPNWvWTAkJCZo0aZLy8/PrrLY5c+aoXbt2stvtSklJ0bp16zzPl5SUaMqUKYqPj1doaKjat2/vuWuwy+XS008/rcTERNntdrVp00YPPfRQndTlL2rVTRMwKsLI9zuk/BNS86vM1gMAdc3lct8c1ITgcJ9uuZGenq4VK1Zo8eLFSk5O1qeffqp77rlHH374oX71q19p5cqVmjhxouf8t956S0OHDvWspmq1WrVw4UJ16NBBBw8e1KRJk/Too4/qlVdeueJv5b/+6780b948vfrqq+rTp4+WLVumW265Rd98842Sk5O1cOFCvffee3r77beVmJioo0ePetbQ+tvf/qY//elPWrVqlbp3766srCx99dVXV1yTPyGMXEpkvBTXS8r6j7R/g5TyK9MVAUDdKi2Unmtj5r0fPyaFNKvRqcXFxXruuee0YcMGz1ISHTt21ObNm/Xqq6/q0Ucf1bx585SZmanExEQ5nU6tWrVKTz75pOc1pk2b5tlPSkrS7373Oz344IN1EkZeeOEFPfbYY7rrrrskSc8//7w++eQTLViwQC+//LIyMzOVnJysa6+9VhaLxWu5+czMTMXFxSk1NVXBwcFKTEzUwIEDr7gmf0I3zeV4pvjSVQMApuzfv1+FhYUaPny41x1033zzTR04cEApKSnq2rWrVq5cKUnatGmTcnJydMcdd3heY8OGDbrxxhvVtm1bRURE6N5779WpU6dUWHhlLUN5eXk6duyYhg4d6nV86NCh2r17tyTpvvvu086dO3X11VfroYce0kcfnR+LeMcdd+jcuXPq2LGjJkyYoHfffVdlZWVXVJO/oWXkcpJHSJ/+Udr/seQolWzBpisCgLoTHO5uoTD13jVUMbZj7dq1atu2rddzFSt2jx49WitXrtSMGTO0cuVK3XTTTYqJiZEkHT58WDfffLMmTpyoZ599Vi1bttTmzZt1//33q6SkROHhNa+lNvr27atDhw7pH//4hzZs2KA777xTqampWrNmjRISErRnzx5t2LBB69ev16RJk/THP/5RmzZt8tztvqkjjFxO275SeIxUeEo6+i8p6VrTFQFA3bFYatxVYlK3bt1kt9uVmZmpYcOGVXnO3XffrSeffFLbt2/XmjVrvO6Btn37djmdTs2bN09Wq7tT4O23366T2iIjI9WmTRtt2bLFq7YtW7Z4dbdERkZq1KhRGjVqlG6//XbddNNNOn36tFq2bKmwsDCNHDlSI0eO1OTJk3XNNddo165d6tu3b53U2NgRRi7HapM6p0r/We2+iy9hBAAaXEREhB5++GFNnz5dTqdT1157rXJzc7VlyxZFRkZq7NixSkpK0pAhQ3T//ffL4XDolltu8VzfuXNnlZaW6sUXX9TIkSO1ZcuWS96w1VePPPKIZs+erU6dOiklJUWvv/66du7cqbfeekuSeyZPfHy8+vTpI6vVqr/+9a+Ki4tTdHS0li9fLofDoUGDBik8PFwrVqxQWFiY17iSpo4xIzXhmeLLeiMAYMrcuXM1a9Yspaenq2vXrrrpppu0du1adejQwXPO6NGj9dVXX+m2225TWFiY53jv3r01f/58Pf/88+rRo4feeustz9TauvDQQw8pLS1Nv/3tb9WzZ0+tW7dO7733npKTkyW5w9Qf/vAH9e/fXwMGDNDhw4f1wQcfyGq1Kjo6WkuWLNHQoUPVq1cvbdiwQe+//76niykQWFwul8t0EZeTl5enqKgo5ebmKjLSwB10z52R/tBJcjmkqf+RWgROWgXQtBQVFenQoUPq0KGDQkNDTZeDJuBS/6Zq+vlNy0hNhLWQEga592kdAQCgThFGaqoLXTUAECi6d+/uNYW48lYxDgR1hwGsNZX8U2nD09KhT6WSQimkfqeBAQDM+eCDD1RaWlrlcxV3qUfdIYzUVOtuUmQ7Ke876fA/zy+GBgBocgJpJktjQDdNTVks57tq9rIaKwD/5gdzF+An6uLfEmHEF8kVS8Ovd99cCgD8jM1mk+S+iyxQFyqW07+S1WLppvFFhx9LNruUmymd+FZq3dV0RQDgk6CgIIWHh+vEiRMKDg72rEYK+MrlcqmwsFA5OTmKjo72BN3aIIz4IiRc6nCd+w6+ez8kjADwOxaLRfHx8Tp06JCOHDliuhw0AdHR0YqLi7ui1yCM+Cp5hDuM7PtIunaa6WoAwGchISFKTk6mqwZXLDg4+IpaRCoQRnzV5afSPx6RMj93r8wa1sJ0RQDgM6vVygqsaDToLPRViySp1dXupeEPfGy6GgAA/B5hpDY8q7GuN1sHAABNAGGkNipP8XU6zdYCAICfI4zURuKPJHukVHhSOrbDdDUAAPg1wkht2IKlTje491mNFQCAK0IYqS1PVw1hBACAK0EYqa3k4e7H419JZ7PM1gIAgB8jjNRW89ZSm77ufWbVAABQa4SRK9GFrhoAAK4UYeRKVHTVHNgolbGsMgAAtUEYuRLxfaRmraWSs1LmZ6arAQDALxFGroTVer51ZO9HZmsBAMBPEUauVHLF0vCMGwEAoDYII1eq0w2SNUg6tV86dcB0NQAA+B3CyJUKjZISB7v3meILAIDPCCN1ga4aAABqjTBSFyrWGzm8WSrON1sLAAB+hjBSF1p1kaLbS44S6dAm09UAAOBXCCN1wWI53zrCXXwBAPAJYaSueO7iu15yuczWAgCAHyGM1JWka6WgMOnsMSn7a9PVAADgNwgjdSU4VOo4zL1PVw0AADVGGKlLnim+LA0PAEBNEUbqUkUY+e4LqfC02VoAAPAThJG6FJ0gte4uuZzS/g2mqwEAwC8QRupaF7pqAADwBWGkrlV01ezfIDkdZmsBAMAPEEbqWruBUmi0dO6Me+wIAAC4JMJIXbMFSZ1vdO8zxRcAgMsijNQHz2qsjBsBAOByCCP1oXOqJIt7Jdbc70xXAwBAo0YYqQ/NYqR2A9z7+9abrQUAgEaOMFJfWI0VAIAaIYzUl4r1Rg5ulEqLjJYCAEBjRhipL3G9pIh4qbRQOrLZdDUAADRahJH6YrFIycPd+3vpqgEAoDqEkfrkmeL7oeRyma0FAIBGijBSnzpeL1mDpTOHpVP7TVcDAECjVKsw8vLLLyspKUmhoaEaNGiQtm3bVu25y5cvl8Vi8dpCQ0NrXbBfsTeXkoa691mNFQCAKvkcRlavXq20tDTNnj1bO3bsUO/evTVixAjl5ORUe01kZKSOHz/u2Y4cOXJFRfuVyl01AADgIj6Hkfnz52vChAkaN26cunXrpsWLFys8PFzLli2r9hqLxaK4uDjPFhsbe8n3KC4uVl5entfmt7qUh5Ejn0lFfvx9AABQT3wKIyUlJdq+fbtSU1PPv4DVqtTUVG3durXa6/Lz89W+fXslJCTol7/8pb755ptLvk96erqioqI8W0JCgi9lNi4xnaSWnSRnmXTwE9PVAADQ6PgURk6ePCmHw3FRy0ZsbKyysrKqvObqq6/WsmXL9Pe//10rVqyQ0+nUkCFD9N131d+zZebMmcrNzfVsR48e9aXMxqeidYQpvgAAXCSovt9g8ODBGjx4sOfrIUOGqGvXrnr11Vc1d+7cKq+x2+2y2+31XVrDSf6p9Pkr0v71ktMpWZnEBABABZ8+FVu1aiWbzabs7Gyv49nZ2YqLi6vRawQHB6tPnz7avz+Aprq2HyIFN5Pys6Wsr0xXAwBAo+JTGAkJCVG/fv2UkZHhOeZ0OpWRkeHV+nEpDodDu3btUnx8vG+V+rMgu9TpBvc+XTUAAHjxub8gLS1NS5Ys0RtvvKHdu3dr4sSJKigo0Lhx4yRJY8aM0cyZMz3nz5kzRx999JEOHjyoHTt26J577tGRI0f0wAMP1N134Q88d/Flii8AAJX5PGZk1KhROnHihJ566illZWUpJSVF69at8wxqzczMlLXSmIgzZ85owoQJysrKUosWLdSvXz999tln6tatW919F/6gIox8v0PKPyE1v8psPQAANBIWl6vx3zQlLy9PUVFRys3NVWRkpOlyam/xdVLWf6RbF0kpd5uuBgCAelXTz2+mdTQkT1cN40YAAKhAGGlIFeuN7P9YcpSarQUAgEaCMNKQ2vaTwmOk4lzp6L9MVwMAQKNAGGlIVpvUuXwpfe7iCwCAJMJIw2PcCAAAXggjDa3zjZLFKp34VjpzxHQ1AAAYRxhpaGEtpIRB7n1aRwAAIIwYQVcNAAAehBETKqb4HvpUKik0WwsAAIYRRkxo3U2KbCeVFUmH/2m6GgAAjCKMmGCxSF3Ku2qY4gsACHCEEVOSy7tq9n0kNf7bAwEAUG8II6Z0uE6y2aXco+5pvgAABCjCiCkhzdyBRKKrBgAQ0AgjJlXuqgEAIEARRkyqGMSa+bl07ozZWgAAMIQwYlKLJKnV1ZLLIR342HQ1AAAYQRgxzTPFl64aAEBgIoyYVrE0/P71ktNpthYAAAwgjJiWOFiyR0qFp6RjO0xXAwBAgyOMmGYLljrd4N5nii8AIAARRhoDzxRfwggAIPAQRhqD5OHux+NfSWezzNYCAEADI4w0Bs1bS236uPdZAA0AEGAII40Fq7ECAAIUYaSxqFhv5MBGqazEaCkAADQkwkhjEd9HatZaKjkrZX5muhoAABoMYaSxsFrPD2RlNVYAQAAhjDQmFauxMsUXABBACCONSacbJGuQdGq/dOqA6WoAAGgQhJHGJDTKvTy8JO1bb7YWAAAaCGGksaGrBgAQYAgjjU2X8vVGDm+WivPN1gIAQAMgjDQ2rbpI0e0lR4l0aJPpagAAqHeEkcbGYjnfOsJdfAEAAYAw0hh5xo2sl1wus7UAAFDPCCONUdK1UlCYdPaYlP216WoAAKhXhJHGKDhM6jjMvU9XDQCgiSOMNFaerhqWhgcANG2EkcaqIox894VUeNpsLQAA1CPCSGMVnSC17i65nNL+DaarAQCg3hBGGjPPXXwZNwIAaLoII41ZxXoj+zdITofZWgAAqCeEkcas3UApNFoq+sE9dgQAgCaIMNKY2YKkzje69+mqAQA0UYSRxi65vKuGKb4AgCaKMNLYdU6VZHGvxJr7nelqAACoc4SRxq5ZjNSuv3uf1hEAQBNEGPEHnq6a9WbrAACgHhBG/EGX8tVYD26USouMlgIAQF0jjPiDuF5SRLxUWigd2Wy6GgAA6hRhxB9YLJVWY2XcCACgaSGM+AvPXXw/lFwus7UAAFCHCCP+ouP1kjVYOnNYOrnPdDUAANQZwoi/sEdISUPd+0zxBQA0IYQRf+KZ4svS8ACApoMw4k8q7uJ75DOpKM9sLQAA1JFahZGXX35ZSUlJCg0N1aBBg7Rt27YaXbdq1SpZLBbdeuuttXlbxHSSWnaSnGXSwU9MVwMAQJ3wOYysXr1aaWlpmj17tnbs2KHevXtrxIgRysnJueR1hw8f1sMPP6zrrruu1sVC52fVMMUXANBE+BxG5s+frwkTJmjcuHHq1q2bFi9erPDwcC1btqzaaxwOh0aPHq1nnnlGHTt2vKKCA17Faqz710tOp9laAACoAz6FkZKSEm3fvl2pqannX8BqVWpqqrZu3VrtdXPmzFHr1q11//331+h9iouLlZeX57WhXPuhUnAzKT9byvrKdDUAAFwxn8LIyZMn5XA4FBsb63U8NjZWWVlZVV6zefNmLV26VEuWLKnx+6SnpysqKsqzJSQk+FJm0xZklzrd4N6nqwYA0ATU62yas2fP6t5779WSJUvUqlWrGl83c+ZM5ebmerajR4/WY5V+qPJqrAAA+LkgX05u1aqVbDabsrOzvY5nZ2crLi7uovMPHDigw4cPa+TIkZ5jzvJxDkFBQdqzZ486dep00XV2u112u92X0gJLRRj5foeUf0JqfpXZegAAuAI+tYyEhISoX79+ysjI8BxzOp3KyMjQ4MGDLzr/mmuu0a5du7Rz507Pdsstt+iGG27Qzp076X6prch4Ka6nJJd7ICsAAH7Mp5YRSUpLS9PYsWPVv39/DRw4UAsWLFBBQYHGjRsnSRozZozatm2r9PR0hYaGqkePHl7XR0dHS9JFx+Gj5BFS1i730vApd5uuBgCAWvM5jIwaNUonTpzQU089paysLKWkpGjdunWeQa2ZmZmyWlnYtd51GSH98wVp/8eSo1SyBZuuCACAWrG4XI3/fvR5eXmKiopSbm6uIiMjTZfTODgd0gvJUuEp6b61UtK1pisCAMBLTT+/acLwV1ab1Ll8vZe9zKoBAPgvwog/80zxZb0RAID/Ioz4s04/kSxW6cS30pkjpqsBAKBWCCP+LLyllDDIvU/rCADATxFG/B1dNQAAP0cY8XddRrgfD30qlRSarQUAgFrweZ0RNDKtu0mR7aS876RXr5Nad5VadZFikt2PrTpLoVGmqwQAoFqEEX9nsUh975U2pkun9ru3CzWPLQ8nFVsX92NUgnuKMAAABrHoWVNx5oh0cq90cp/78dR+92N+dvXX2OxSTGd364mnNaV8s0c0XO0AgCappp/ftIw0FS3au7fk4d7Hi3Klk/ulU/sqhZV90ukDkqNYyvnGvV0oIr48qHQ5H1BiKlpTGGoEAKg7hJGmLjRKatfPvVXmdEg/HHEHlZN7y8NK+VaQI5097t4O/9P7uqAwKabT+XBSMS4lJlmyN2+47wsA0GTQTYOLnfvhfDdP5W6fUwckZ2n110W08R6XUtGyEtmW1hQACEA1/fwmjKDmHGXlrSn7KnX7lIeWwpPVXxcUVvW4lJjOUkizhqsfANCgGDOCumcLcnfRxHSSdJP3c4Wny1tTLhhAe/qgVHZOytrl3i4U2e6C1pROUlSiFBlPUAGAAEHLCOqXo9Q90+fCAbSn9kmFpy59bWi0u4snqq0U2ca9H9mmfGvnfmScCgA0WrSMoHGwBZd30XSWrv6Z93OFpyu1pFSElANS3vdSSb5U9IN7q2q2TwV71PmAEtW26sASSoAFgMaMMAJzwltKiYPcW2Uul1ScJ+UdcweTvGNS7vfn9yuOF+dJxbnSiVzpxO7q3yckoprAUmk/NMq9gBwAoMERRtD4WCzucBAa5V7evjpFee7px5cKLEU/SCVnpZN73Ft1gptVCiztKrWuVAosYS0ILABQDwgj8F+hke7tqqurP6c43zuw5H1fHloqBZZzp6XSAndX0al91b9WUNglAkt5t1B4SwILAPiIMIKmzd5cspfP1qlO6TnvLqGqWloKT7pnBZ0+4N6qY7OfDykhzaQguxQUWumx8nbhc5d7rLRv4z9dAE0H/0cDgsMqTVmuRmmRdPaYd4vKhYGlIMe9xP6ZQ+6tPllsvgWZ4OoC0GWutdnLb6ZokSzW860+nn3LBfvlX1e5X+mci6671OvR0gQ0dYQRoCaCQ6WWHd1bdcpKzncJnT3ubnEpK5LKiuvm0VFy/r1cDnfXUmlB/X/vjUIV4aZGIabSdcFh7tlX9gjvLTSyfP/Cx/L9iueDQglGQD0hjAB1JSjk/A0L64PT6W55uTColJ67RJC50hB0TnI53TOcXC5JlR+dl9+vM+Xv63LW4Wv6yBpUKaxEVhNoqnu+UggK9FDjdJbfVsLi/plyqwiIMAL4D6tVsoa5/8L3FxeGGJez6v2LAk0Nr7voNap5PafDHbCKcqXis5W2PO/Horyqn5dLcpZJ5864tythDaq6FSb0ggBzYStOaHnIsVjc34+j1P2h7ixz36qhyv1S97nO0vL9skrHy89zlFWzf7lrL3gdr5ouUd9FgbI8lNiCy8OJTbJW7Ae5x0dV7F/yuQs2W3D5+RXHKn1d3XtZbZWeq+o1gy5+vYrfR8XPxLNdeMxRzbEafO1y+H6NZ3PW/Pzx/5Da9qvyn219I4wAqD9NYcyH0+nuDvMKKxc+lm/Vhp2zF4Sa0+4Ncv9MSi99E040DKfD2FsTRgDgUqzW860TkW1q/zpOp3tlYa+wckF4uWzYyXO3LNiCzv9FX/kv+Yv2gyv9tX/hftAVvM6F117qdaq5Vjr/V3nl1pbKm+PCY6VVXOOo9FxZpVaZyn/5l1bzmpVez6slqSbPlb+2y1lFK4qtiq+rOlaDry0236/x2r/cNZWONbuqbv6bqQXCCAA0BKv1/No4ALwwcggAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYFmS6gJlwulyQpLy/PcCUAAKCmKj63Kz7Hq+MXYeTs2bOSpISEBMOVAAAAX509e1ZRUVHVPm9xXS6uNAJOp1PHjh1TRESELBZLnb1uXl6eEhISdPToUUVGRtbZ66J2+H00PvxOGhd+H40Lv4/Lc7lcOnv2rNq0aSOrtfqRIX7RMmK1WtWuXbt6e/3IyEj+ITUi/D4aH34njQu/j8aF38elXapFpAIDWAEAgFGEEQAAYFRAhxG73a7Zs2fLbrebLgXi99EY8TtpXPh9NC78PuqOXwxgBQAATVdAt4wAAADzCCMAAMAowggAADCKMAIAAIwK6DDy8ssvKykpSaGhoRo0aJC2bdtmuqSAlJ6ergEDBigiIkKtW7fWrbfeqj179pguC+V+//vfy2KxaNq0aaZLCVjff/+97rnnHsXExCgsLEw9e/bUl19+abqsgOVwODRr1ix16NBBYWFh6tSpk+bOnXvZ+6+gegEbRlavXq20tDTNnj1bO3bsUO/evTVixAjl5OSYLi3gbNq0SZMnT9bnn3+u9evXq7S0VD/96U9VUFBgurSA98UXX+jVV19Vr169TJcSsM6cOaOhQ4cqODhY//jHP/R///d/mjdvnlq0aGG6tID1/PPPa9GiRXrppZe0e/duPf/88/rDH/6gF1980XRpfitgp/YOGjRIAwYM0EsvvSTJff+bhIQE/eY3v9GMGTMMVxfYTpw4odatW2vTpk368Y9/bLqcgJWfn6++ffvqlVde0e9+9zulpKRowYIFpssKODNmzNCWLVv0z3/+03QpKHfzzTcrNjZWS5cu9Rz7f//v/yksLEwrVqwwWJn/CsiWkZKSEm3fvl2pqameY1arVampqdq6davByiBJubm5kqSWLVsariSwTZ48Wb/4xS+8/jtBw3vvvffUv39/3XHHHWrdurX69OmjJUuWmC4roA0ZMkQZGRnau3evJOmrr77S5s2b9bOf/cxwZf7LL26UV9dOnjwph8Oh2NhYr+OxsbH69ttvDVUFyd1CNW3aNA0dOlQ9evQwXU7AWrVqlXbs2KEvvvjCdCkB7+DBg1q0aJHS0tL0+OOP64svvtBDDz2kkJAQjR071nR5AWnGjBnKy8vTNddcI5vNJofDoWeffVajR482XZrfCsgwgsZr8uTJ+vrrr7V582bTpQSso0ePaurUqVq/fr1CQ0NNlxPwnE6n+vfvr+eee06S1KdPH3399ddavHgxYcSQt99+W2+99ZZWrlyp7t27a+fOnZo2bZratGnD76SWAjKMtGrVSjabTdnZ2V7Hs7OzFRcXZ6gqTJkyRf/7v/+rTz/9VO3atTNdTsDavn27cnJy1LdvX88xh8OhTz/9VC+99JKKi4tls9kMVhhY4uPj1a1bN69jXbt21d/+9jdDFeGRRx7RjBkzdNddd0mSevbsqSNHjig9PZ0wUksBOWYkJCRE/fr1U0ZGhueY0+lURkaGBg8ebLCywORyuTRlyhS9++67+vjjj9WhQwfTJQW0G2+8Ubt27dLOnTs9W//+/TV69Gjt3LmTINLAhg4detFU971796p9+/aGKkJhYaGsVu+PT5vNJqfTaagi/xeQLSOSlJaWprFjx6p///4aOHCgFixYoIKCAo0bN850aQFn8uTJWrlypf7+978rIiJCWVlZkqSoqCiFhYUZri7wREREXDRep1mzZoqJiWEcjwHTp0/XkCFD9Nxzz+nOO+/Utm3b9Nprr+m1114zXVrAGjlypJ599lklJiaqe/fu+ve//6358+dr/PjxpkvzX64A9uKLL7oSExNdISEhroEDB7o+//xz0yUFJElVbq+//rrp0lBu2LBhrqlTp5ouI2C9//77rh49erjsdrvrmmuucb322mumSwpoeXl5rqlTp7oSExNdoaGhro4dO7qeeOIJV3FxsenS/FbArjMCAAAah4AcMwIAABoPwggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAPzOxo0bZbFY9MMPP5guBUAdIIwAAACjCCMAAMAowggAnzmdTqWnp6tDhw4KCwtT7969tWbNGknnu1DWrl2rXr16KTQ0VD/60Y/09ddfe73G3/72N3Xv3l12u11JSUmaN2+e1/PFxcV67LHHlJCQILvdrs6dO2vp0qVe52zfvl39+/dXeHi4hgwZoj179tTvNw6gXhBGAPgsPT1db775phYvXqxvvvlG06dP1z333KNNmzZ5znnkkUc0b948ffHFF7rqqqs0cuRIlZaWSnKHiDvvvFN33XWXdu3apaefflqzZs3S8uXLPdePGTNGf/nLX7Rw4ULt3r1br776qpo3b+5VxxNPPKF58+bpyy+/VFBQELdwB/yV6dsGA/AvRUVFrvDwcNdnn33mdfz+++93/epXv3J98sknLkmuVatWeZ47deqUKywszLV69WqXy+Vy3X333a7hw4d7Xf/II4+4unXr5nK5XK49e/a4JLnWr19fZQ0V77FhwwbPsbVr17okuc6dO1cn3yeAhkPLCACf7N+/X4WFhRo+fLiaN2/u2d58800dOHDAc97gwYM9+y1bttTVV1+t3bt3S5J2796toUOHer3u0KFDtW/fPjkcDu3cuVM2m03Dhg27ZC29evXy7MfHx0uScnJyrvh7BNCwgkwXAMC/5OfnS5LWrl2rtm3bej1nt9u9AklthYWF1ei84OBgz77FYpHkHs8CwL/QMgLAJ926dZPdbldmZqY6d+7stSUkJHjO+/zzzz37Z86c0d69e9W1a1dJUteuXbVlyxav192yZYu6dOkim82mnj17yul0eo1BAdB00TICwCcRERF6+OGHNX36dDmdTl177bXKzc3Vli1bFBkZqfbt20uS5syZo5iYGMXGxuqJJ55Qq1atdOutt0qSfvvb32rAgAGaO3euRo0apa1bt+qll17SK6+8IklKSkrS2LFjNX78eC1cuFC9e/fWkSNHlJOTozvvvNPUtw6gnhBGAPhs7ty5uuqqq5Senq6DBw8qOjpaffv21eOPP+7pJvn973+vqVOnat++fUpJSdH777+vkJAQSVLfvn319ttv66mnntLcuXMVHx+vOXPm6L777vO8x6JFi/T4449r0qRJOnXqlBITE/X444+b+HYB1DOLy+VymS4CQNOxceNG3XDDDTpz5oyio6NNlwPADzBmBAAAGEUYAQAARtFNAwAAjKJlBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGDU/wfAgdveDg3h/wAAAABJRU5ErkJggg==", + "text/plain": [ + "

" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "eval_accuracy_qlora=get_metric_qlora('eval_accuracy',log_history_qlora)\n", + "eval_loss_qlora=get_metric_qlora('eval_loss',log_history_qlora)\n", + "plt.plot(eval_accuracy_qlora,label='eval_accuracy')\n", + "plt.plot(eval_loss_qlora,label='eval_loss')\n", + "plt.xlabel(\"epoch\")\n", + "plt.legend()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The above code results in the following plot:\n", + "\n", + "![qlora_training_plot](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/wzMMj73IuM6fKmPZtKtQNA/qlora-training-plot.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The above code indicates that, in this particular instance, the bulk of the benefits from fine-tuning were gained within the first 3 epochs.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Congratulations! You have completed the lab\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Authors\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) is a Data Scientist and a PhD Candidate in Economics at the University of Calgary.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[Fateme Akbari](https://www.linkedin.com/in/fatemeakbari/) is a Ph.D. candidate in Information Systems at McMaster University with demonstrated research experience in Machine Learning and NLP.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## References\n", + "\n", + "[Finetuning with LoRA -- A Hands-On Example](https://lightning.ai/lightning-ai/studios/code-lora-from-scratch)\n", + "\n", + "[QLORA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/pdf/2305.14314)\n", + "\n", + "[Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA](https://huggingface.co/blog/4bit-transformers-bitsandbytes)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Change Log\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "|Date (YYYY-MM-DD)|Version|Changed By|Change Description|\n", + "|-|-|-|-|\n", + "|2024-07-09|0.99|Victor|Lab written|\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Copyright © 2024 IBM Corporation. All rights reserved.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.20" + }, + "prev_pub_hash": "856043195c4f7d96c767c084082c1b6e5e7de2222404f63f3e209d4961071444" + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/requirements.txt b/requirements.txt index 07150c1..8130d39 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,11 +8,15 @@ transformers==4.46.3 #huggingface-hub==0.27.0 datasets==3.1.0 accelerate==1.0.1 +evaluate trl -jupyter==1.1.1 +peft +bitsandbytes sentencepiece==0.2.0 nltk==3.9.1 gradio==4.44.1 spacy==3.7.2 # conda install #flash_attn==2.6.3 # /home/loc/Works/flash-attention (need gcc to build) -scikit-learn \ No newline at end of file +scikit-learn +plotly +jupyter==1.1.1