From 166d99ca21c0061e59bd45a934f6e2b74ce18f87 Mon Sep 17 00:00:00 2001 From: Chhaya Choudhary Date: Mon, 3 Jun 2019 17:12:57 -0700 Subject: [PATCH] Hire-us team submission updated Talking data V3 model submission with leaderboard score --- hire-us/models/TalkingData_v3.ipynb | 1752 +++++++++++++++++++++++++++ 1 file changed, 1752 insertions(+) create mode 100644 hire-us/models/TalkingData_v3.ipynb diff --git a/hire-us/models/TalkingData_v3.ipynb b/hire-us/models/TalkingData_v3.ipynb new file mode 100644 index 0000000..8d9eb11 --- /dev/null +++ b/hire-us/models/TalkingData_v3.ipynb @@ -0,0 +1,1752 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Problem: \n", + "Fraud risk is everywhere, but for companies that advertise online, click fraud can happen at an overwhelming volume, resulting in misleading click data and wasted money. Ad channels can drive up costs by simply clicking on the ad at a large scale. With over 1 billion smart mobile devices in active use every month, China is the largest mobile market in the world and therefore suffers from huge volumes of fradulent traffic.\n", + "\n", + "TalkingData, China’s largest independent big data service platform, covers over 70% of active mobile devices nationwide. They handle 3 billion clicks per day, of which 90% are potentially fraudulent. Their current approach to prevent click fraud for app developers is to measure the journey of a user’s click across their portfolio, and flag IP addresses who produce lots of clicks, but never end up installing apps. With this information, they've built an IP blacklist and device blacklist.\n", + "\n", + "While successful, they want to always be one step ahead of fraudsters and have turned to the Kaggle community for help in further developing their solution. In their 2nd competition with Kaggle, you’re challenged to build an algorithm that predicts whether a user will download an app after clicking a mobile app ad. To support your modeling, they have provided a generous dataset covering approximately 200 million clicks over 4 days!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Goal:\n", + "Predicting the probabilities for different click_id's in the test set.**\n", + "\n", + "For each click_id in the test set, we must predict a probability for the target is_attributed variable. The file should contain a header and have the following format:\n", + "* click_id,is_attributed\n", + "* 1,0.003\n", + "* 2,0.001\n", + "* 3,0.000\n", + "* etc." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Dataset Description\n", + "Each row of the training data contains a click record, with the following features.\n", + "\n", + "- **ip**: ip address of click.\n", + "- **app**: app id for marketing.\n", + "- **device**: device type id of user mobile phone (e.g., iphone 6 plus, iphone 7, huawei mate 7, etc.)\n", + "- **os**: os version id of user mobile phone\n", + "- **channel**: channel id of mobile ad publisher\n", + "- **click_time**: timestamp of click (UTC)\n", + "- **attributed_time**: if user download the app for after clicking an ad, this is the time of the app download\n", + "- **is_attributed**: the target that is to be predicted, indicating the app was downloaded\n", + " Note that ip, app, device, os, and channel are encoded.\n", + "\n", + "The test data is similar, with the following differences:\n", + "- **click_id**: reference for making predictions\n", + "- **is_attributed**: not included" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Evaluation:\n", + "Submissions are evaluated on area under the ROC curve between the predicted probability and the observed target." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### TLDR; Summary\n", + "- Total train samples are 184,903,890.\n", + "- Total test samples are 18,790,469.\n", + "- Total test supplement samples are 57,537,505. \n", + "- Percentage of positive data: 0.2%\n", + "- Down-sampled the data to take 10M rows using the resample function from sklearn\n", + "- Used Kaggle kernel for training and generating the output for this problem \n", + "- Used XGBoost for training and testing purposes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Leaderboard Score:\n", + "- **Public score**: 0.96273\n", + "- **Private score**: 0.96549" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", + "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['test_supplement.csv', 'train_sample.csv', 'train.csv', 'sample_submission.csv', 'test.csv']\n" + ] + } + ], + "source": [ + "# This Python 3 environment comes with many helpful analytics libraries installed\n", + "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n", + "# For example, here's several helpful packages to load in \n", + "\n", + "import numpy as np # linear algebra\n", + "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", + "\n", + "# Input data files are available in the \"../input/\" directory.\n", + "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n", + "\n", + "import os\n", + "print(os.listdir(\"../input\"))\n", + "\n", + "# Any results you write to the current directory are saved as output." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", + "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "import pandas as pd\n", + "import numpy as np\n", + "from sklearn.model_selection import train_test_split\n", + "import matplotlib.pyplot as plt\n", + "import xgboost as xgb\n", + "from xgboost import plot_importance\n", + "import gc\n", + "import time\n", + "from datetime import datetime\n", + "%config InlineBackend.figure_format = 'retina'\n", + "plt.figure(figsize=(12,5))\n", + "from sklearn.utils import resample" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Train dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ipappdeviceoschannelclick_timeattributed_timeis_attributed
08323031133792017-11-06 14:32:21NaN0
11735731193792017-11-06 14:33:34NaN0
23581031133792017-11-06 14:34:12NaN0
345745141134782017-11-06 14:34:52NaN0
416100731133792017-11-06 14:35:08NaN0
\n", + "
" + ], + "text/plain": [ + " ip app ... attributed_time is_attributed\n", + "0 83230 3 ... NaN 0\n", + "1 17357 3 ... NaN 0\n", + "2 35810 3 ... NaN 0\n", + "3 45745 14 ... NaN 0\n", + "4 161007 3 ... NaN 0\n", + "\n", + "[5 rows x 8 columns]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_path = \"../input/\" + 'train.csv'\n", + "train_df = pd.read_csv(train_path, nrows = 10000)\n", + "train_df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
click_idipappdeviceoschannelclick_time
0057449131072017-11-10 04:00:00
111199019134662017-11-10 04:00:00
2272287211191282017-11-10 04:00:00
3378477151131112017-11-10 04:00:00
44123080121133282017-11-10 04:00:00
\n", + "
" + ], + "text/plain": [ + " click_id ip app ... os channel click_time\n", + "0 0 5744 9 ... 3 107 2017-11-10 04:00:00\n", + "1 1 119901 9 ... 3 466 2017-11-10 04:00:00\n", + "2 2 72287 21 ... 19 128 2017-11-10 04:00:00\n", + "3 3 78477 15 ... 13 111 2017-11-10 04:00:00\n", + "4 4 123080 12 ... 13 328 2017-11-10 04:00:00\n", + "\n", + "[5 rows x 7 columns]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "test_path = \"../input/\" + 'test.csv'\n", + "test_df = pd.read_csv(test_path, nrows = 10000)\n", + "test_df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test supplement dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
click_idipappdeviceoschannelclick_time
004357031183792017-11-09 14:23:39
118052831133792017-11-09 14:23:51
223232331133792017-11-09 14:25:57
334288731173792017-11-09 14:26:03
44119289581301202017-11-09 14:26:41
\n", + "
" + ], + "text/plain": [ + " click_id ip app ... os channel click_time\n", + "0 0 43570 3 ... 18 379 2017-11-09 14:23:39\n", + "1 1 80528 3 ... 13 379 2017-11-09 14:23:51\n", + "2 2 32323 3 ... 13 379 2017-11-09 14:25:57\n", + "3 3 42887 3 ... 17 379 2017-11-09 14:26:03\n", + "4 4 119289 58 ... 30 120 2017-11-09 14:26:41\n", + "\n", + "[5 rows x 7 columns]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "test_sup_path = \"../input/\" + 'test_supplement.csv'\n", + "test_sup_df = pd.read_csv(test_sup_path, nrows = 10000)\n", + "test_sup_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "ip int64\n", + "app int64\n", + "device int64\n", + "os int64\n", + "channel int64\n", + "click_time object\n", + "attributed_time object\n", + "is_attributed int64\n", + "dtype: object" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_df.dtypes" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "train_cols = ['ip','app','device','os','channel','click_time','is_attributed']\n", + "test_cols = ['ip','app','device','os','channel','click_time','click_id']\n", + "\n", + "# By default, pandas sets the dtype of integers to int64. In many cases, \n", + "# this datatype takes up extra memory which is just not required.\n", + "# For example: int64 datatype takes in 8 bytes and can represent humongous integers, \n", + "# from -9223372036854775808 to 9223372036854775807. Hence, memory reduction by changing \n", + "# the datatypes is very helpful.\n", + "\n", + "dtypes = {\n", + " 'ip' : 'uint32',\n", + " 'app' : 'uint16',\n", + " 'device' : 'uint16',\n", + " 'os' : 'uint16',\n", + " 'channel' : 'uint16',\n", + " 'click_id' : 'uint32',\n", + " 'is_attributed' : 'uint8'\n", + " }" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# Reading the last 10M rows for down sampling the data\n", + "train_df = pd.read_csv( '../input/' + \"train.csv\", skiprows=range(1,123903891), nrows=61000000, usecols=train_cols, dtype=dtypes)\n", + "test_sup_df = pd.read_csv('../input/' + \"test_supplement.csv\", usecols=test_cols, dtype=dtypes)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# Separate majority and minority classes\n", + "df_train_majority = train_df[train_df.is_attributed==0]\n", + "df_train_minority = train_df[train_df.is_attributed==1]" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "60849867" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(df_train_majority)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "150133" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(df_train_minority)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "# Downsample majority class\n", + "df_majority_downsampled = resample(df_train_majority, \n", + " replace=False, # sample without replacement\n", + " n_samples=10000000, # to match minority class\n", + " random_state=123) # reproducible results" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "# Combine minority class with downsampled majority class\n", + "df_downsampled = pd.concat([df_majority_downsampled, df_train_minority])" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0 10000000\n", + "1 150133\n", + "Name: is_attributed, dtype: int64" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Display new class counts\n", + "df_downsampled.is_attributed.value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "# Feature extraction using click_time column in the datasets\n", + "def feature_extraction(df):\n", + " df['date'] = pd.to_datetime(df['click_time'])\n", + " df['dayOfWeek'] = df['date'].dt.dayofweek.astype('uint16')\n", + " df['dayOfYear'] = df['date'].dt.dayofyear.astype('uint16')\n", + " df['hour'] = df['date'].dt.hour.astype('uint8')\n", + " df['min'] = df['date'].dt.minute.astype('uint8')\n", + " df['sec'] = df['date'].dt.second.astype('uint8')\n", + " df.drop(['date','click_time'], axis= 1, inplace=True)\n", + " return df" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "130" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "del df_train_minority, train_df, df_majority_downsampled\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ipappdeviceoschannelclick_timeis_attributed
433394673022591154422017-11-09 10:41:440
440802191028402132362017-11-09 10:55:020
18550211534693192802017-11-09 03:20:400
16650992401921412362017-11-09 02:44:210
26375816903791174662017-11-08 18:45:030
\n", + "
" + ], + "text/plain": [ + " ip app ... click_time is_attributed\n", + "43339467 30225 9 ... 2017-11-09 10:41:44 0\n", + "44080219 102840 2 ... 2017-11-09 10:55:02 0\n", + "18550211 53469 3 ... 2017-11-09 03:20:40 0\n", + "16650992 4019 2 ... 2017-11-09 02:44:21 0\n", + "2637581 69037 9 ... 2017-11-08 18:45:03 0\n", + "\n", + "[5 rows x 7 columns]" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_downsampled.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "# drop the target values from train dataset \n", + "y = df_downsampled['is_attributed']\n", + "df_downsampled.drop(['is_attributed'], axis=1, inplace=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "33" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# drop the click_time from the test data\n", + "test_sup_df.drop(['click_id'], axis=1, inplace=True)\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ipappdeviceoschannelclick_time
433394673022591154422017-11-09 10:41:44
440802191028402132362017-11-09 10:55:02
18550211534693192802017-11-09 03:20:40
16650992401921412362017-11-09 02:44:21
26375816903791174662017-11-08 18:45:03
\n", + "
" + ], + "text/plain": [ + " ip app ... channel click_time\n", + "43339467 30225 9 ... 442 2017-11-09 10:41:44\n", + "44080219 102840 2 ... 236 2017-11-09 10:55:02\n", + "18550211 53469 3 ... 280 2017-11-09 03:20:40\n", + "16650992 4019 2 ... 236 2017-11-09 02:44:21\n", + "2637581 69037 9 ... 466 2017-11-08 18:45:03\n", + "\n", + "[5 rows x 6 columns]" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Merging the supplement data with test data\n", + "rows_train = df_downsampled.shape[0]\n", + "merge_df = pd.concat([df_downsampled, test_sup_df])\n", + "merge_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Length of combine dataset: 67687638\n" + ] + } + ], + "source": [ + "print('Length of combine dataset: ', len(merge_df))" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "19" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "del df_downsampled, test_sup_df\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " ip clicks_by_ip\n", + "0 0 2\n", + "1 1 62\n", + "2 2 6\n", + "3 3 291\n", + "4 4 39\n", + "5 5 99\n", + "6 6 902\n", + "7 7 1\n", + "8 8 2\n", + "9 9 1605\n", + "10 10 598\n", + "11 11 9\n", + "12 12 30\n", + "13 13 27\n", + "14 14 4\n", + "15 15 1\n", + "16 16 13\n", + "17 17 1\n", + "18 18 18\n", + "19 19 415\n", + "20 20 2330\n", + "21 21 38\n", + "22 22 15\n", + "23 23 4\n", + "24 24 4\n", + "25 25 608\n", + "26 26 21\n", + "27 27 1153\n", + "28 28 3\n", + "29 29 93\n", + "... ... ...\n", + "213604 364749 1\n", + "213605 364750 1\n", + "213606 364751 2\n", + "213607 364752 6\n", + "213608 364753 1\n", + "213609 364754 1\n", + "213610 364755 10\n", + "213611 364756 1\n", + "213612 364757 74\n", + "213613 364758 46\n", + "213614 364759 62\n", + "213615 364760 1\n", + "213616 364761 34\n", + "213617 364762 1\n", + "213618 364763 1\n", + "213619 364764 25\n", + "213620 364765 3\n", + "213621 364766 1\n", + "213622 364767 1\n", + "213623 364768 1\n", + "213624 364769 1\n", + "213625 364770 1\n", + "213626 364771 2\n", + "213627 364772 1\n", + "213628 364773 2\n", + "213629 364774 2\n", + "213630 364775 3\n", + "213631 364776 39\n", + "213632 364777 4\n", + "213633 364778 5\n", + "\n", + "[213634 rows x 2 columns]\n" + ] + } + ], + "source": [ + "# Group by ip to count the number of clicks\n", + "ip_groups = merge_df.groupby(['ip'])['channel'].count().reset_index(name = 'clicks_by_ip')\n", + "print(ip_groups)" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " ip app device ... channel click_time clicks_by_ip\n", + "0 30225 9 1 ... 442 2017-11-09 10:41:44 1232\n", + "1 102840 2 1 ... 236 2017-11-09 10:55:02 1106\n", + "2 53469 3 1 ... 280 2017-11-09 03:20:40 4071\n", + "3 4019 2 1 ... 236 2017-11-09 02:44:21 44510\n", + "4 69037 9 1 ... 466 2017-11-08 18:45:03 1175\n", + "\n", + "[5 rows x 7 columns]\n" + ] + } + ], + "source": [ + "merge_df = pd.merge(merge_df, ip_groups, on='ip', how='left', sort=False)\n", + "print(merge_df.head())" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "merge_df['clicks_by_ip'] = merge_df['clicks_by_ip'].astype('uint16')\n", + "merge_df.drop('ip', axis=1, inplace=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "train_df = merge_df[:rows_train]\n", + "test_df = merge_df[rows_train:]" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "65" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "del test_df,merge_df\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "98" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_df = feature_extraction(train_df)\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
appdeviceoschannelclicks_by_ipdayOfWeekdayOfYearhourminsec
0911544212323313104144
12132361106331310552
23192804071331332040
3214123644510331324421
491174661175231218453
\n", + "
" + ], + "text/plain": [ + " app device os channel ... dayOfYear hour min sec\n", + "0 9 1 15 442 ... 313 10 41 44\n", + "1 2 1 3 236 ... 313 10 55 2\n", + "2 3 1 9 280 ... 313 3 20 40\n", + "3 2 1 41 236 ... 313 2 44 21\n", + "4 9 1 17 466 ... 312 18 45 3\n", + "\n", + "[5 rows x 10 columns]" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "# Set the params for xgboost model\n", + "params = {'eta': 0.3,\n", + " 'tree_method': \"hist\",\n", + " 'grow_policy': \"lossguide\",\n", + " 'max_leaves': 1400, \n", + " 'max_depth': 0, \n", + " 'subsample': 0.9, \n", + " 'colsample_bytree': 0.7, \n", + " 'colsample_bylevel':0.7,\n", + " 'min_child_weight':0,\n", + " 'alpha':4,\n", + " 'objective': 'binary:logistic', \n", + " 'scale_pos_weight':9,\n", + " 'eval_metric': 'auc', \n", + " 'nthread':8,\n", + " 'random_state': 99, \n", + " 'silent': True}" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "# Change this for validation with 10% from train dataset\n", + "is_valid = False" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/opt/conda/lib/python3.6/site-packages/xgboost/core.py:587: FutureWarning: Series.base is deprecated and will be removed in a future version\n", + " if getattr(data, 'base', None) is not None and \\\n", + "/opt/conda/lib/python3.6/site-packages/xgboost/core.py:588: FutureWarning: Series.base is deprecated and will be removed in a future version\n", + " data.base is not None and isinstance(data, np.ndarray) \\\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0]\ttrain-auc:0.947969\n", + "[1]\ttrain-auc:0.965531\n", + "[2]\ttrain-auc:0.967453\n", + "[3]\ttrain-auc:0.968336\n", + "[4]\ttrain-auc:0.968627\n", + "[5]\ttrain-auc:0.969879\n", + "[6]\ttrain-auc:0.96988\n", + "[7]\ttrain-auc:0.970962\n", + "[8]\ttrain-auc:0.971825\n", + "[9]\ttrain-auc:0.973328\n", + "[10]\ttrain-auc:0.973945\n", + "[11]\ttrain-auc:0.974643\n", + "[12]\ttrain-auc:0.976179\n", + "[13]\ttrain-auc:0.977288\n", + "[14]\ttrain-auc:0.978011\n", + "[15]\ttrain-auc:0.978824\n", + "[16]\ttrain-auc:0.979868\n", + "[17]\ttrain-auc:0.980946\n", + "[18]\ttrain-auc:0.981926\n", + "[19]\ttrain-auc:0.982566\n", + "[20]\ttrain-auc:0.983156\n", + "[21]\ttrain-auc:0.98401\n", + "[22]\ttrain-auc:0.984733\n", + "[23]\ttrain-auc:0.985376\n", + "[24]\ttrain-auc:0.98603\n", + "[25]\ttrain-auc:0.986692\n", + "[26]\ttrain-auc:0.987369\n", + "[27]\ttrain-auc:0.987681\n", + "[28]\ttrain-auc:0.988089\n", + "[29]\ttrain-auc:0.988602\n" + ] + }, + { + "data": { + "text/plain": [ + "21" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Used XGBoost algorithm for model preparation for the following reasons:\n", + "# XGBoost has high predictive power and is almost 10 times faster than the other gradient boosting techniques.\n", + "# It also includes a variety of regularization which reduces overfitting and improves overall performance.\n", + "# Found the above parameters giving the best results\n", + "\n", + "if (is_valid == True):\n", + " # Get 10% of train dataset to use as validation\n", + " X_train, X_test, y_train, y_test = train_test_split(train, y, test_size=0.1, stratify=y, random_state=99)\n", + " dtrain = xgb.DMatrix(X_train, y_train)\n", + " dvalid = xgb.DMatrix(X_test, y_test)\n", + " del X_train, X_test, y_train, y_test \n", + " gc.collect()\n", + " watchlist = [(dtrain, 'train'), (dvalid, 'valid')]\n", + " model = xgb.train(params, dtrain, 200, watchlist, maximize=True, early_stopping_rounds = 25, verbose_eval=5)\n", + " del dvalid\n", + "else:\n", + " dtrain = xgb.DMatrix(train_df, y)\n", + " del train_df, y\n", + " gc.collect()\n", + " watchlist = [(dtrain, 'train')]\n", + " model = xgb.train(params, dtrain, 30, watchlist, maximize=True, verbose_eval=1)\n", + "\n", + "del dtrain\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA10AAAIqCAYAAADfKyCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xm8VfP+x/HXp9NwmgfNkySUlNMkpeE0oAEllIsoka6pJJdMJYTLlcRNKSVEioSK0nQrRdOl1I803GRI0Typ0/f3x1r7dIa9T3ufs3en03k/H4/12O3vvL57P9if813ru8w5h4iIiIiIiMRGnuwegIiIiIiIyOlMQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIYUdImIiIiIiMSQgi4REREREZEYUtAlIiIiIiISQwq6REREREREYkhBl4iIiIiISAwp6BIREREREYkhBV0iIiIiIiIxpKBLREREREQkhhR0iYiI5ABm9pSZOTMbk91jERGRyCjoEhGRoMxsvP8j/0RHv1NgrPXNbLCZ3ZzdY5GTy8xu9T/7utk9FhGRUPJm9wBEROSUdwT4M4P8/SdrIBmoDwwC5gATsnkssbId+B74NbsHcoq5FbgE+BH4NpvHIiISlIIuERE5kS+dc4nZPYjczjk3HBie3eMQEZHI6fJCERERERGRGFLQJSIiMWFmLcxskpltNbPDZvaHmc02s24Z1GlgZs+Z2SIz+ylFvXn+vTt50pTPa2YOeN1PahPknrNmftka/vujGfTf1i/zY5C8rYH2zKyKmY00s03+GJdH4/wzEmojjcAc+EdlM6tlZhPN7Fcz229mK83sxhTl85jZ381shZnt88c10cwqh+h3kd/2TWZ2hpkN98/7kH9ur5lZ+ROM/Rwzez1FvT/NbEGwzzREvyXN7Hkz+97MDprZDjO7zf/sL/GrvJXmc/8xTXstzexlM/vazH4xs7/M7Hczm2lmV2cw9rf99h41szgz629m35rZAf88Pjaz+ic4/yJm9g8zW+LXOWRmG8zsIzP7m5mlu/LI/5xuNrMv/PP9y8x+NrP3zKxRRv2JyKlHlxeKiEhUmZkBzwP3p0jeA5QE2gJtzexKoLtzzqWpPgco7v97P3AAKAUk+kcnM+vinEvyyzhgG1AQKAb8BexM0+ZfWT+rVGoB0/xxHQBSBXFZPP+sagK8ARQBduPNSz3gbTMrDYwA3gOuw5uXI/55/A1oYmb1nXNp5y+gDLAMOAs4iHfelYA7gM5m1tI5933aSmbWCZgEFPCTdvvja+EfXf3P9ECIfssBK4FqwCF/zODN/TZ//Pn8dg+lqLc9xRhKAPNT5O31z6EM0A5oZ2b/ds7dFWIM+H18hvcZ/uUfJYEr8T7TROfc10HO/wJgOlDVTzqK932oClQHOgELga0p6hQHpgKt/CTnj7ki0A24zszucs69lsF4ReQUopUuERGJtv54AcevwG1AcedccaAwcAPeD+UbgQFB6n4GXA9UcM4Vcc6VxPuBfgvwO3AVcG+gsHMuyTlXnuMBzkLnXPk0R7ofwln0IvATcLFzrrBzrrA/5oCsnH9WvY4XuJ7lnCuBFxQEVsaeAoYAl/v9F/WPlv6YqgEPZND2IKAQ0BEo7JwrArQGNuMFRu+nXbExs3OAiXgB11zgPH9cRYG/4wUul+PNaSiDAfPLFXbOFcOb+4n+Zx/4fO9O87k3SdHGMeB9vADnDOdcMf8zKYn3fdoP3JnRipdfLgEvYC2CF+QnAGvxgtuX0lbwA93P8AKsjXjf38LOuTPwvg8tgDeBpDRV38YLuJYDlwGF/PGegfc5HANeMbOLMxiviJxKnHM6dOjQoUNHugMYj/cX9r+A30Ic49LUKcXxFao6Idpt5re7A8gbwXha+fXWB8m7zc/7IoP6NfwyRzMo09Yv82OQvK1+3h9AmRD1Y3n+T/n1xqRJz+unO2AdEJcmPw7vB3+gzA1B2u7p5/0QJG+Rn3cMaBIk/3z/O+KA69PkvRloFygYpO6dfn4SUC1Ev4eBWhnMS6DcTVn4rgfOf3aQvLdTnP/FQfIbp5jbSmnyXuT4amyFMMfSzq+zFigaosyjfpmPMnvOOnToOLmHVrpERORE8uGtZAQ7SqYpex3easjnzrnVwRpzzi0CtuD91b5eBOOYj3eJVQ0zKxtBvWgb75zbHiIvlucfjufd8UsvA/0lAfP8t/8D3g1Sb47/WsPM4kO0Pc85tyRtonNuLd6lcADXBtL9e7UCK0f/cs4dDNLmKLzgPU/Kuml86pxbFyIvWj7xX5uEuscMmO+cW5o20Tn3Fd45ANQOpPuXmXb33/7TORfuVv+3+K+jnHN7Q5R5x39tk8F4ReQUonu6RETkRBa48LeMb+q/Xmpmv2VQrpT/WgXvPiEg+YfqdXiXv9XHu+emQLra3r0tv4c5pmhLF3ikkKXzj4KggR7H5+o751yw+8i2+a+Gd0/doSBl5mfQ7wKgK95nFnAO3mWEcDzoS8U5l2RmC/DuUwq1GUVG8x02/9LHnnjBXV28zyB/mmKF8S4b3BWkiYw+p5+B8qT+I8TZQGn/3zMiGGrgOzTIzAaeoGwRoAQZP0dPRE4BCrpERCSaKvivhf3jRAoF/mFm+YApePe9BBzGuwwvsHpTBm9VJJy2YyXUKhdk4fyjJNRqSlKY+eCtbAbzcwb9BvLKpEgrEyQ/mMAGEmVC5Gc032Exs2LALLxLAQMO+m0f89+X818LEzzoCrXqBMeD1JRzVy7Fv7eEPVgveIP0q8ihFEJBl8gpT0vSIiISTYH/r/zLOWdhHG+nqNsHL+DaD9wDVHHOxTvnyjh/cwSOr9jYyTuldNJuepBSVs7/dBbqksVwZDTf4RqEF3D9DtwMlHXOFXLOlfW/V2emKJud3y04/h26Mszv0NYMWxORU4KCLhERiabAZWpVMywV3HX+62Dn3Ctpf0z6K2Gl0lcLW2Br9zx+W8EUD5Eerqyc/6muYhh5KVelUv47o/kIPB8syytaGQh8t+5yzr0V5J68cmkrRMG2FP8+M2Sp9AJ/WDgdv0MiuZaCLhERiabA/TetzCzYvVgZCfz4XhUivznp78EJCFwiltEqxa4UZSqFKJPVh85m5fxPdS3DyFuZIm09xy/Ja0UQZhYXom4kwvnsA593qO9W20z2nZENeJfGAnSIoF7gO9Q+usMRkeykoEtERKLpfbzt0kvjbWsdkpmlvWdlt/9aJ0jZvMCTGTS3x38tEaqAc24Xx+8f6hSkjzJArwz6CEdWzv9U18bMGqdNNLPzOL5L4eRAunPuGMd3NexnZgWDtHkH3j1Mx/Du58uME372KcoE+24VBR7OZN8h+RuWvOW/fcDMKmRUPoXx/mtHM7s0o4I58Dskkmsp6BIRkahxzv3O8WDjUTMbZWY1AvlmVtDMWpjZa8DCNNVn+6+DzOxKfxUEMzsfmI63u92BEF1/57/WMbOGGQzx/RR9dAw8zNfMmgJf4D3TKtOyeP6nuj3AVDNr5+8yiZm1BGbirUCuBj5IU+cpvA0rqgCf+A9LxswKmFkfYJhfbrRzbnMmxxX47K8xs1CXhwa+Wy+ZWfMU42+Mt7NiRgFbVjyDt3lJWWCh/73O5/ed38xamdn7KQMy59ynwDS8lbtpZna//5Bl/HpnmNnVZvYp8M8YjVtEokxBl4iIRJVzbhgwGO/hrb2B9Wa218z+BPbhbS9+B+m3gn8O2IT3A/hj4KCZ7cb7Ud3ab2tniD7XAV/i7R63zMx2mNlm/0gZhD0NbMbbGe5TYJ+Z7QMW420V3i9rZ5+l8z/VPYG3S99Mjs/bfOAsvPuXrnPOHU1ZwTm3HrgBbxfKNsAPZrYT77LDkXjB2izg/iyMawJwBO8yxR1mttX/3OenKPMw3g5/ZwL/AQ74418K1MJ7REHU+feOtcfbvfFsvO/1fjPbgbdhzFy8+83SBvs34T07rCDwAvC7mf1pZnvwLln8EOgYizGLSGwo6BIRkahzzj2B9+DfMcCPeD8qC+P91f8zYABp7hFyzv0BXAy8xvEtxg/i/cBs4Zx7i4x18utuwns+1Jn+kbxznnPuT7znIL3uj8XwfsQOBxoAv2TmfNPKzPnnANvx7nl72f93PrzPaTSQ4Jz7Plgl59xHeM/FGov3cOZCeCuWC4HbgPbOuVArmCfknPsOuAz4HO8S1Qp4n3vlFGV+9Mf+jj/2OLx7/N4CGnL84dBR55z7Bu+hyY8BK/AC10J4czEVuJ7jD1cO1NnnnLsKbzfPqXjfm8J4j/pZD0wCehCFPxKIyMlhwZ+RKCIiIgJmtgi4BOiei7a4FxGJKq10iYiIiIiIxJCCLhERERERkRhS0CUiIiIiIhJDCrpERERERERiSBtpiIiIiIiIxJBWukRERERERGJIQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIbyZvcARCJlZpuAYsDmbB6KiIiIiJy+qgF7nHNnZbUhBV2SExUrUKBAqdq1a5fK7oHkFHv37gWgaNGi2TySnEHzFTnNWeQ0Z5HTnEVOcxY5zVnkTtc5W7duHQcPHoxKWwq6JCfaXLVq1VIrVqzI7nHkGPPnzwcgMTExW8eRU2i+Iqc5i5zmLHKas8hpziKnOYvc6TpnDRo0YOXKlZuj0Zbu6RIREREREYkhBV0iIiIiIiIxpKBLREREREQkhhR0iYiIiIiIxJCCLhERERERkRhS0CUiIiIiIhJDCrpERERERERiSEGXiIiIiIhIDCnoEhERERERiSEFXSIiIiIiIjGkoEtERERERCSGFHSJiIiIiIjEkIIuERERERGRGFLQJSIiIiIiEkMKukRERERERGJIQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIYUdImIiIiIiMSQgi4REREREZEYUtAlIiIiIiISQwq6REREREREYkhBl4iIiIiISAwp6BIREREREYkhBV0iIiIiIiIxpKBLREREREQkhhR0iYiIiIiIxJCCLhERERERkRhS0CUiIiIiImEbP348ZpZ8tGrVilatWqVKi4uLS1Vn7969PPLII9SsWZP4+HhKlizJ5Zdfzpw5c0L2k5SUxLBhw6hbty4FCxakVKlSdOjQgS+//DJkndWrV3PjjTdSo0YNChYsSKVKlWjVqhWTJk3i2LFjUZuDSOXNtp4lxzOzwcAgoJVzbn72jkZEREREToaEhAQGDRqU/H7z5s0AVKtWjYULFzJ37lzat2+fnL9z506aNWvG2rVrqV27Nn369GHfvn1MmzaNtm3bMmbMGHr16pWqD+cc119/PVOmTOG8887j7rvv5s8//2TSpEm0aNGCDz74gE6dOqWq88knn9ClSxfy5MnDVVddxbXXXsuOHTuYOnUq119/PV988QWvv/567CYmAwq6REREREQkbAkJCSQkJCS/nz9/PgCJiYk0adIEgN69eyfnDx48mLVr19KlSxcmTZpE3rxeCDJ06FAaNmzIPffcw+WXX07lypWT67z33ntMmTKFpk2bMmfOHOLj4wHo06cPzZo14/bbb6d169YULVo0uc5DDz3E0aNHmT9/Pi1btkxOf+qpp7jwwgsZM2YMjz32GFWrVo3+pJyALi+UrHgFqAV8nd0DEREREZHstXr1apYuXUqlSpXo2LFjcvrUqVMBGDJkSHLABVC2bFn69+/PwYMHeeONN1K1NXLkSMALmAIBF0CjRo3o1q0b27dvZ8qUKanqbNy4kWLFiqUKuADKly9P48aNAdi+fXsUzjRyWumSTHPO7QB2ZEffh44kUe2h6dnRdY40oM5RAHpozsKi+Yqc5ixymrPIac4ipzmLnOYstM3Pdswwf/To0QD06tUr1T1dv/32GwDVq1dPVyeQNmfOHB5//HEADh06xJdffkmhQoVo3rx5ujrt27fnrbfeYu7cufTs2TM5vXbt2qxYsYJFixbRrFmz5PTff/+dr7/+mgoVKnD++eeHe7pRpZWuXMbMqpmZM7PxZna2mU0xsz/MbK+ZzTKzC/xyZcxstJn9amaHzGyZmbVK09Zgv63ENOnOzOabWekUbRw2s+/MrCciIiIiclo5fPgwb7/9NnFxcdx2222p8kqXLg3Apk2b0tXbuHEjAN9//31y2oYNG0hKSqJ69eqpVsYCzjnnHAB++OGHVOnDhg2jWLFitG3blm7dujFw4EBuv/12ateuTdGiRZk2bRoFCxbM2olmkoKu3Ksa8BVQDhgPzALaAvPN7BxgKdAImAS8D1wIzDSzcC+CLQEsBpoAU4A3gYrAG2Z2S9TOQkRERESy3bx589i1axft2rWjSpUqqfIClxoOGjSIpKSk5PTt27czbNgwwNtsI2D37t0AFC9ePGhfgfRdu3alSm/evDlLliyhRo0avP/++zz77LOMGTOGw4cP07NnT+rUqZPFs8w8XV6Ye7UEHnXOPR1IMLPHgCF4wdj7wJ3OuWN+3mxgAnCff5zIhcBY4A7nXJLfxkvAt8CDeEFYhsxsRYismnnzQH9/+V9OrJz/R50BmrOwaL4ipzmLnOYscpqzyGnOIqc5Cy2wYUZae/fuZdq0aQA0bdo0Xbl27drx8ccfM2XKFM455xzq16/PoUOHWLx4cfIqWMr216xZA3jBV7A+t27dCsDBgwdT5S9fvpwnn3ySc889l1GjRlG1alX+/PNPpk6dyiOPPMLEiRMZPnx4uu3sQ9m7d29Y5cKhla7cazPwbJq0QCBUAHggEHD5JgJHgQTCcwDoHwi4AJxza/FWv2qZWZHMDFpERERETi1btmzh//7v/yhTpkzyhhUpnXHGGYwcOZLOnTtz4MABpk2bxtKlS2nVqhWDBw8GoGTJksnlCxcuDMD+/fuD9hdID5QD2LNnD0OGDCF//vzJgVd8fDwVK1bkrrvuolmzZnz33XfMnj07WqcdEa105V7/TRkQ+X7xX39wzqUK7Z1zSWa2DahMeNY75/YESf/Jfy0J7MuoAedcg2DpZrbi6DHqv7BaX99wBf5apzkLj+YrcpqzyGnOIqc5i5zmLHKas9A235gYNH3EiBEA3HnnnbRp0yZk/S5duqRLmzt3LgDNmjUjMdFr/+KLL+b2229n27ZtNGvWLN19Xb/++isADRs2TK7z6aefsnfvXi699FLatWuXrp/rrruORYsWsX///uQ6J5JyO/qs0kpX7rU7bYJz7mioPN9RIF+Y7e8KkR7oI7x1XRERERE5ZR06dIjZs2eTJ0+edA84DseECRMAuOGGG5LT4uPjadq0KQcOHGDhwoXp6sycOROA1q1bJ6cdPnwYCL0lfCA9f/78EY8xGhR0iYiIiIhIpkyePJm9e/fSuHHjdBtoBBw7dox9+9Jf4PTWW28xYcIEmjZtSufOnVPl/f3vfwfg0Ucf5dChQ8npy5YtY9KkSZQpU4ZrrrkmOb1JkybkzZuXxYsXM2vWrFRt/fTTT4waNQogw5W4WNK6qeRI8fni+OEEz4qQ4wI3mYa6LEBS03xFTnMWOc1Z5DRnkdOcRU5zFpnAs7muuOKKkGUOHDhAuXLluPTSSzn77LPJkycPixcvZsmSJdSqVYvJkyeTJ0/qtaDrr7+eDz/8kClTplCvXj2uvPJK/vjjDyZNmkRSUhKvv/46xYoVSy5fsWJFHnvsMQYNGkT79u254oorqFmzJr/99hsffvgh+/bt4+qrr6ZDhw6xmYgTUNAlIiIiIiIRW7duHYsWLQq5gUZAgQIFuP7661m0aFHyRhbnnHMOTz/9NP369aNQoULp6pgZ7777Lk2bNuWNN95gxIgRxMfH06JFCx599FGaNm2ars7jjz/OhRdeyGuvvcaXX37J9OnTKVSoEHXq1KF79+707t07eicfIQVdIiIiIiISsVq1auGcC7mVfEC+fPkYO3ZsxO3nzZuX++67j/vuC+dpRZ5OnTrRqVOniPuKNd3TJSIiIiIiEkNa6cplnHObAcsgP6O8amneDwYGR9hGD6DHCYYpIiIiInLa0EqXiIiIiIhIDCnoEhERERERiSEFXSIiIiIiIjGkoEtERERERCSGFHSJiIiIiIjEkIIuERERERGRGFLQJSIiIiIiEkMKukRERERERGJIQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIYUdImIiIiIiMSQgi4REREREZEYUtAlIiIiIiISQwq6REREREREYkhBl4iIiIiISAwp6BIREREREYkhBV0iIiIiIiIxpKBLREREREQkhhR0iYiIiIiIxJCCLhERERERkRhS0CUiIiIiIhJDCrpERERERLLZ+PHjMbMMj7i4uOTyP/30E3feeSeNGzemfPnyFChQgIoVK9K8eXPGjRvHkSNHIu7jtddeCzq2gwcPMmjQIM477zzi4+MpW7YsXbt2Zd26dTGbj9NN3uweQCyYWTVgE/Cmc65HivTxwC3AWc65zdFq92QyMwcscM4lZkf/KcaRCMwDnnDODc7OsYiIiIjkdAkJCQwaNCho3sKFC5k7dy7t27dPTtuwYQPvvPMOjRs3pnPnzpQqVYo//viDmTNncuutt/LWW28xa9Ys8uZN/3O/U6dOJCQkpEtv2LBhurTDhw9z6aWXsnjxYho2bEjfvn356aefmDx5MtOnT2fu3LlZOOvc47QMukREREREcpKEhISggRBAkyZNAOjdu3dyWtOmTdm5cyd58qS+cO3IkSNcdtllzJs3jw8//JCuXbuma69z58706NEjrHG9+OKLLF68mGuvvZZJkyYl99etWzc6d+7MrbfeyogRI9KNQ1LLbbMzEKgF/JzdAzkNfI03l69k90BERERETlerV69m6dKlVKpUiY4dOyan58+fP2igky9fPjp37gzA+vXrs9S3cy75ksN//vOfqfrr1KkTzZs3Z+3atXzzzTdZ6ic3yFUrXc65X4Ffs3scpwPn3AHg/7Kr/0NHkqj20PTs6j7HGVDnKAA9NGdh0XxFTnMWOc1Z5DRnkdOcRe5kz9nmZztmmD969GgAevXqleqerlCSkpKYMWMGAHXr1g1a5r///S8vvfQShw4dolKlSrRq1YrKlSunK7dhwwa2bNnCueeey1lnnZUuv3379ixcuJBVq1ZRr169E44tN8uRQZeZXQTcDzQDSgN/AquBMc659zOoN54Q93Rltk2/bh5gGHAvMBW40Tl30MyKAv2AbkBVwIDfgeXAP51zKyI68dR9VgSeAy4HigJrgX855yamKHM58Bkw3jnXM0gbBTi+6lfJOXc4gv4TCXJPl5nNB1oC8cBjwI1ARWArMAF4xjn3V7j9iIiIiORWBw8e5O233yYuLo7bbrstaJkdO3bwyiuv4Jxj+/btzJ49mx9//JEbbriBK6+8Mmid4cOHp3ofaP+ll14iPj4+Of37778H4Nxzzw3azjnnnAN4m3pIxnJc0GVmtwMjgSTgY2A9UBZoCNwJZBggRbtNM4sH3gG6AK8C9zrnjpmZ4QU8TYElwBjgKFAZaAUsBDIbdJUEvgR2AeOAEkBX4B0zq+Sce94vNwvYAHQ1s37Oud1p2rkGOAMvWAs74ArT+0AjYApwBOgEDAYamtlVzjkX5f5ERERETivvv/8+u3btomPHjlSpUiVomR07dvDEE08kvzczBgwYwNChQ9OVPeussxgxYgSXXXYZlStXZvfu3SxatIiBAwcyatQo9uzZw8SJyX+/Z/du76dj8eLFg/YdSN+3b1+mzzG3yFFBl5mdD/wb2AM0d859lyY//bpoDNs0s1J4QVpT4CHn3HMpsi/w0z9yzl2dpl4eIPi3Nzx1gcnA9c65Y36bz+IFcU+b2QfOuY3OOWdmrwHPA91Jf/9V4G7M0VkYSyi1gNrOuZ3++B7BWxm7ArgJeOtEDZhZqKC0Zt480N9f/pcTK1fQex2gOQuL5itymrPIac4ipzmLnOYscid7zubPnx8y7/nnvb+jN23aNMNy8+bNIykpiR07drBw4UJGjhzJjBkzeOaZZyhWrFiqshdccAG//PILv/zyCwBlypRh6NCh3Hbbbbz77ru0bt2aGjVqALB27VoAtm3bFrT/wL1cSUlJ7N27N8Mx5kR79+6NWls5bSONv+MFik+mDY4AnHNbT1abZnYmsBi4COieJuBK6WCQNo8FgpFMSgIeDARcfpubgJeBfHgBVsA44BBwR5rxn4d3GeA859wPWRhLKE+mPEfn3CG8jUwAbo1BfyIiIiKnjU2bNvHdd99RpkwZGjdufMLycXFxlCtXjmuvvZb+/fuzdu1axo0bF1ZfZcuWTe7j22+/TU4vXLgwAPv37w9aL5AeKCeh5aiVLuBi/3VmNrd5Ht4lg4WB9s65OUHKrAX+C/zND9CmAYuA5VG4p2mLH2SlNR8YBCTfyeic+8PM3gduNrOmzrkv/azAKlfwp+Bl3YIgaYvwAsaw7rR0zjUIlm5mK44eo/4Lq3Pa1zf7BP5apzkLj+YrcpqzyGnOIqc5i5zmLHIne84235gYNH3q1KkA3HnnnbRp0yaiNuvVq8eTTz7J+vXrSUwM3n5a06ZN44svvqBixYrJdSpXrszDDz/MH3/8EbSdJUuWAN5li0WLFg27r5yiaNGiUWsrp610lfBfo7nle2baPBeoAGwEVgYr4JxLAloDL+FtovEc3srYDjMbYWZFMj1i2BYi/Tf/Ne2li//2X++A5A00bsHb1GNqFsaRkXRjdM4dBXYAxdIXFxERERGAQ4cO8dZbbxEXF0evXr0irv/zz97P2mAPRg7lq6++AqB69erJaWeffTZVq1blhx9+YNOm9H/vnznTW7PQzoUnltOCrl3+a6VsbvMT4GEgAZhjZmcEK+Sc2+mcu885VwU4B7gNb5v1u/E27sisciHSy/uvqTbMcM59BazC21CjJMc30BjnnDuShXFENEYzy4u3M+SeGPUpIiIikuNNnjyZnTt30r59+5AbaKxcuZKkpKR06fv27aNv374AqZ7rBbB8+fJ05Y8dO8YzzzzDkiVLKF26NO3atUvOMzP69OkDwD/+8Q+OHUu+s4Vp06axcOFCzj//fC688MLITzKXyWlrzUvxdhRsT/SeEZWpNp0wp01GAAAgAElEQVRzz5jZQbyt4uebWVvnXKgVKJxzPwI/mtlEvBWmTlkYc1Uzq5Z223sg0X9dFaTOv4HXgZuBqwFHbDbQCGhJ+s0ymgFxIcYXkfh8cfxwgudayHGBG1tDXcIgqWm+Iqc5i5zmLHKas8hpziJ3KsxZ4NlcvXv3DllmyJAhLF68mKZNm1K1alUKFSrETz/9xMyZM9m1axdNmzZl4MCBqeo0atSICy64gAsvvJBKlSqxe/duFi9ezJo1ayhUqBDvvPNOuo03+vfvz6effsqUKVNo3Lgxbdq0YcuWLUyePJlChQrxxhtvcPBgui0MJI2cttI1Em/b9cf8XQdTyczuhVlp0zn3Et5GHLWBBf6zswL1zjKz6kGqlQQKEGSDjQjEAc/5uyAm94f3nLCjwNtB6kzEWwH7B15ANNs5tzELYziRx/xVtcD44oFn/Lfh3dUpIiIiksusW7eORYsWUblyZTp06BCy3O23387ll1/OunXrmDBhAi+++CJffPEFDRo0YNSoUSxYsIAiRVLfzTJgwABKlSrF3LlzGT58OBMmTODIkSPcddddrF69mssuuyxdPwUKFGD27Nk89thj7Nq1i2HDhjF79mw6d+7MsmXLwtrkQ3LYSpdzbq2Z3Ym3+cMqM5uG90ytM/CeCbUH7xlYJ61N59xrZnYIGAv8x8xaO+e2ABcCH5rZMmAd8AtQBm+FKx/ePV6Z9S3QGFhhZrM4/pyuEsA/nHMbgozzgJm9iReYAYzKQv/hWAd8Z2Ypn9N1NjCdMLaLFxEREcmNatWqRTiPM+3YsWO6ywdPJLAFfaQKFSrEkCFDGDJkSKbqSw4LugCcc6+b2RpgAN7ldJ3xNmf4Fu8BxCe9TefceDM7DEzAD7yA5cCzeKtK7fBWuLbjPUvrZedcVnZg3Il3OeQ/gZ54G1OsBV5wzk3MoN4beEHXr3jPF4ulrsBjwI1ARbyNSgYDz+rByCIiIiKSm+S4oAvAObcEbzOIUPmbAQuS3gPokZk2M2rXz3sXeDdN8sMZtZcZzrmU/d8UYfXA1jJj/Z0EszKO+YSYCz//MPCof4iIiIiI5Fo57Z4uySR/58D+ePd8xfrSQhERERER8eXIlS4Jn5k1w7vEMRGoA7zinNuarYMSEREREclFFHRlIzMrAfQLs/j4IFvEh6MtMAj4E2/L+H+EGEsC3r1sJ+ScG5yJcYiIiIiI5EoKurJXCbyAKBzzgc2RduAHSIPDKJoQwVhCtuecSwyzDRERERGRXEFBVzbKaGOOk805Nx4Yn83DEBERERE57WgjDRERERERkRhS0CUiIiIiIhJDCrpERERERERiSEGXiIiIiIhIDCnoEhERERERiSEFXSIiIiIiIjGkoEtERERERCSGFHSJiIiIiIjEkIIuERERERGRGFLQJSIiIiIiEkMKukRERERERGJIQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIYUdImIiIiIiMSQgi4REREREZEYUtAlIiIiIiISQwq6RERERCRq5syZw9VXX0358uUpUKAAFStW5PLLL2fGjBnJZdavX89zzz1H69atqVKlCvnz56dcuXJ06tSJefPmhWz7xx9/pGfPnlSuXJn8+fNToUIFunfvzoYNG0LWWbRoEZ06daJatWrEx8dTtWpVOnTowGeffRbV8xbJiIKuk8jMqpmZM7Px2T2Wky03n7uIiEhu8Y9//IO2bduyfPlyrrrqKu6//346duzI9u3bmT9/fnK5xx57jIceeoht27bRoUMH7r//fi655BKmT59O69atefnll9O1vXz5curXr8/48eOpWbMmffv2pXnz5rz33nvUr1+fVatWpaszcuRImjdvzpw5c2jevDn33XcfLVu2ZMGCBbRv356nn346ltMhkixvdg9ARERERHK+119/neeff55bbrmF0aNHkz9//lT5R44cSf53u3btePDBB6lXr16qMgsWLODSSy/lgQce4LrrrqNChQrJeb169WLv3r28+OKL3HfffcnpixYtIjExkZ49e7Jq1SrMLLm/gQMHEh8fz4oVKzjvvPOS6zz88MPUq1ePp59+mgEDBlCgQIGozoVIWuacy+4x5BpmVg3YBLzpnOuRrYM5yaJ57ma2okq16vXzdEv/VzAJbkCdowC8sFp/ZwmH5itymrPIac4ipzmLXKznbPOzHQE4fPgwVapUoWDBgqxfvz5dwBWJyy67jNmzZzNlyhSuueYaADZu3MjZZ59N2bJl+fXXX8mTJ/XFWp07d2batGksWLCAFi1aALBt2zbKly9P3bp1+eabb9L1U7duXVavXs2OHTs444wzktMDK3KJiYmZPofc5nSdswYNGrBy5cqVzrkGWW1LlxeKiIiISJbMnj2b7du306VLF/LkycP06dN57rnnGD58OEuWLImorXz58gGQN+/xQPG3334DoFq1aukCLoDq1asD3v1kAWXLlqVMmTL88MMPrF+/PlX5QFpCQkKqgEskVhR0RZGZXWRmk8zsZzM7bGa/mtksM+sapGw1M3vPzHaY2SEzW25mVwQpV9zMHjCzuWa21cz+MrPtZvaxmTUJMQ5nZvPNrLSZjfbHcdjMvjOznkHKJ/p1BptZgplNN7NdZnbAzBaYWdMQ/eQ1szvNbKmZ7fHLrzKzu81M3y0REZFcYtmyZQDEx8dTr149rrjiCh566CH69etH06ZNadmyJdu3bz9hO//73/+YM2cOhQoVSl6xAihdunRyfrCrtDZu3AjA999/n5xmZrz66qscO3aMBg0acMsttzBw4EBuvvlmGjRoQO3atZk8eXKWzlskXLq8MErM7HZgJJAEfAysB8oCDYFdzrnEFJfYzQdqAxuBJUApoBuQD2jrnJuXot2Lgf/4xwZgJ1AVuAooAFzpnEu1/Y6ZOeAboCDwl99fAeA6oATQwzn3ZoryicA8YDrQ2h/TKr+fa/w2Epxz36eokw/4BLgc+N7v4xDQCqgLvO2c656ifODcw7680MxWhMiqeVb16oX6D3kxnGYEKFfQe912MHvHkVNoviKnOYuc5ixymrPIxXrOLqhUHIBhw4bx8ccfkydPHqpVq0a/fv2oUaMGv/76KyNHjmT58uVceOGFvPTSSyHb+uuvv7j//vtZs2YNd9xxB9dff32q/O7du7N161buuusurr322uT0NWvW0LdvX44dO0bDhg15/vnnU9VbvXo1Tz31FL///ntyWsmSJenevTudOnVKt3K2d+9eAIoWLZq5ScmFTtc56927N+vXr4/K5YW6KDoKzOx84N/AHqC5c+67NPmV01RJBAY7555IUWYi8BnwAF4AFLAOqOic2xGkza+BYX69tC4ExgJ3OOeS/DovAd8CDwJvBqnTEejpnBufop87gNeAvsCdKco+ghdwvQL0S9FHHDAauNXMpjjnpgXpR0RERE4jx44dAyAuLo6nn36a8uXLA95lf08++SQ333wz33zzDd999x21a9dOVz8pKYmhQ4eyZs0aWrVqRbdu3dKVue+++3jooYd49dVXWbp0KTVq1OD3339n4cKFVK9enR9//DFdADV79mxeeOEFmjdvzs0330y5cuXYtm0bEyZM4OWXX+abb75h8ODB0Z8QkTQUdEXH3/Hm8sm0AReAc25rmqT/AU+lKfO5mW0BLkqTvjtYh865rWY2BbjHzKo657akKXIA6B8Ihvw6a81sMdDCzIo45/alqbM4ZcDlewMvsEoel3/p4D3Ab8B9afpIMrP7gZ7AjUCmg65Qf1UwsxVHj1FfN1KHTzefR0bzFTnNWeQ0Z5HTnEUu5htp3JgIwMyZM/n0009p0KBBuhUqgKuuuoqxY8dy9OjRdJstJCUlcdNNN7FgwQK6du3KO++8k+p+roDExERatWrFU089xX/+8x++/fZbqlevzj//+U8qVapEt27dqFmzZnL7P/zwA88//zx169Zl1qxZqQKym266iYsuuogFCxYktx1wum4KEUun65xFc+VO/9WKjov915lhlv9vykAlhZ+AdPdpmdkleCtNTfAuWUy7JVAlIG3Qtd45tydEHwAlgbRB1/K0hZ1zR8xsm18+4Fy8SyLXA48GtmZN4yBQK1iGiIiInF4C27GXKFEiaH7Jkt7PiIMHU1/neOTIEW688UYmT57MDTfcwIQJE4iLiwvZT7169fjggw/SpT/++OMANGrUKDlt1qxZHDlyhJYtW6ZbAcuTJw8tWrRgxYoVrFix4rQLFuTUo6ArOgL/hfk5zPK7QqQfJc3mJmZ2NTAF736p2Xj3de0HjuFdptgS736tSPoACPZftIzqpCwf2ObnHGBQiDoARTLIExERkdNEmzZtMDPWrl3LsWPH0gU5a9asAeCss85KTvvrr7/o2rUr06ZN4+abb2bcuHFBdyY8kSNHjvDuu++SL1++VPd6HT58GCDkBh6B9Kxsby8SLgVd0REIVioB/xfltp/E28iioXNuXcoMMxuFF3SdbIFLHqc657pkQ//E54vjB//ZIHJigWX/wGUgkjHNV+Q0Z5HTnEVOcxa5kzVnZ555JldeeSUff/wxw4cPT/Xw4lmzZvH5559TokQJ2rVrB3gBUZcuXZgxYwa9evVi9OjRJwy49u/fT3x8fKqVsKNHj3Lvvffy448/8uCDDybfSwbQvHlzAKZMmcKAAQOoW7duct5///tfpkyZgpnRunXrqMyBSEYUdEXHUrxdCtsT/aCrBvBdkIArD9Asyn2F6//wAs2LzSyfc+7IiSqIiIjI6e3VV19l1apV9O/fn+nTp1OvXj02bdrERx99RFxcHGPGjKF4cW+3wz59+jBjxgxKly5NpUqVGDJkSLr2EhMTU132N2/ePG677Tbatm1L5cqV2bdvH5999hkbNmzg2muv5cknn0xV/6KLLqJnz56MGzeORo0acfXVV3PmmWeyefNmPvroI/766y/69esXdGMPkWhT0BUdI4E+wGNm9rlzbm3KTDOrHGQzjXBtBs4xs4rOuV/89gwYDJyf+SFnnnPuqJmNAB4DXjaz/s65VBdpm1kFoGTauRAREZHTU+XKlVmxYgVDhgzh448/5j//+Q/FihXjyiuvZODAgVx00fG9wjZt2gTAjh07ggZcASmDrnPPPZdLLrmEBQsW8Pvvv1OoUCESEhJ44oknuOGGGwh2j/nYsWNp0aIF48eP5/PPP2fv3r0UK1aMZs2acfvttwfd9EMkFhR0RYG/K+CdeFurrzKzaXibTJwBNMLbSr5VJpsflqLdD4AjwCV4AdcnwJVZHH5mPYm3LX0f4Eozm4t3T1tZvHu9LsHbVl5Bl4iISC5RpkwZRowYwYgRIzIsF7jsMRLnnntu0E00MmJm9OjRgx49ekTcn0g0KeiKEufc62a2BhiAt8FFZ2AH3nOxxmSh3VFmdhjoB9yCtyvgQrwt2a8hm4Iuf1fDzsBNQA/gCryNM7bjPQT5MeCd7BibiIiIiMipREFXFDnnluAFQqHyNwNB91f38xNDpI8HxgfJWo13mWHa8hn10QMvSEqZNv8E46oWIt0Bb/lHhk507iIiIiIip6vI9+UUERERERGRsCnoEhERERERiSEFXSIiIiIiIjGkoEtERERERCSGFHSJiIiIiIjEkIIuERERERGRGFLQJSIiIiIiEkMKukRERERERGJIQZeIiIiIiEgMKegSERERERGJIQVdIiIiIiIiMaSgS0REREREJIYUdImIiIiIiMSQgi4REREREZEYUtAlIiIiIiISQwq6REREREREYkhBl4iIiIiISAwp6BIREREREYkhBV0iIiIiIiIxpKBLREREREQkhhR0iYiIiIiIxJCCLhERERERkRhS0CUiIiKSQ1WrVg0zC3qUL18+XfnDhw/z6quvctFFF1G6dGmKFClCrVq1uPfee/nf//53wv4OHz7MBRdcgJlRuXLloGWef/55OnToQLVq1ShSpAjFihWjTp069O/fn61bt2b5nEVyorzZPQARERERybzixYvTr1+/dOlFihRJ9f7o0aO0adOGxYsXU7NmTf72t79RoEABli1bxogRI5gwYQJffvkl559/fsi+Hn744RMGZ6NGjaJIkSK0bNmScuXKceTIEVatWsWwYcMYO3Ys8+fPp169epk7WZEcSkGXiIiISA5WokQJBg8eHDRv/vz5yf+eOnUqixcvpk2bNsyaNYs8eY5f8DRo0CCGDBnCCy+8wBtvvBGyrWHDhvHvf/+bv//97yHHs2bNGuLj49Olv/766/Tu3ZtHHnmEGTNmhHdyIqcJXV4oIiIikgts3LgRgI4dO6YKuAA6deoEwPbt24PW3bNnDz169KBNmzb06dMnw36CBVwAXbt2BWD9+vURjVvkdKCVLsmRDh1JotpD07N7GDnGgDpHAeihOQuL5itymrPIac4ipznzbH62Y6r3hw8f5u2332bLli0ULlyYunXr0qJFC+Li4lKVq127NgAzZ86kb9++qQKvTz/9FIC2bdsG7fPee+9l586djB07NtPj/uSTTwCoW7duptsQyakUdOVyZnYV0Bc4HygF/AGsByY55/6dolwp4AGgM1AN+AtYDjznnJsVou1uQG+gHlAI+A1YAvzLObc8RqckIiKSq/z222907949VdpZZ53FuHHjUqV17NiRLl268OGHH1KnTh3atm1L/vz5WbFiBYsWLeKee+7hrrvuStf+1KlTefPNNxkzZgxVq1YNe1xjxoxh69at7Nu3j9WrV/PFF19w5pln8uyzz2buREVyMAVduZiZ9QZG4QVDnwA7gLJAXaAn8G+/3JnAfLxgayHwGVAYuAL4zMzucM69nqJdA8YBt/htfghsByoDrYDv8QI2ERERyYKePXvSvHlzateuTdGiRdm4cSOvvPIKo0ePpn379rz88svUqFEDADNjypQpPPHEEzz11FOsXbs2uZ02bdpwww03kDdv6p+G27Zto3fv3rRv355evXpFNLYxY8bw1VdfJb9v1KgREydOTB6PSG5izrnsHoNkEzNbAVwAVHHO/Z4mr7Rzbof/7/lAC+AG59x7KcqUwAvGzgOqOee2+emBYG4ZcKlzbneKOnFAWefcr2GOL5iaZ1WvXqj/kBfDPdVcr1xB73XbwewdR06h+Yqc5ixymrPIac48F1QqfsIyI0eO5P333+fiiy/m4YcfpmjRovz1118MHTqUr7/+mj59+tCsWTMKFCjAmjVrGDFiBNu2bWPQoEE0a9YsuZ1HHnmEb7/9lnHjxlG6dOnk9FatWlG6dGkmT558wrHs3r2b9evXM3bsWH766Scef/xxLrroosyd/Emwd+9eAIoWLZrNI8k5Ttc56927N+vXr1/pnGuQ1ba0kYYcBY6kTUwRcF0ItAQ+SBlw+WV2AYOAeOCaFFn3+K93pAy4/DpJ4QRcIiIiknlXXXUVAN99911y2sSJE1mwYAG9evXiqquuolSpUhQuXJjGjRszePBgjh49yiuvvJJc/vPPP+fLL7/k7rvvThVwRap48eI0bNiQ559/nvz58/PMM89w+PDhzJ+cSA6kla5czMz6A//Cu7zwPWABsNg5tz1FmT7ASGA28GWQZsoAdwKvOOfuMbPCwD5gm3Mu/VMZozPuFVWqVa+fp9vLsWj+tBS4+fyF1bqiOByar8hpziKnOYuc5syTdiONYHbv3k2JEiXIly8fs2bNIjExkYYNG7JixQq+/fZb6tSpk65OqVKl2LlzJzt27OCMM86gX79+DB8+PKwx7dy5kxIlSpyw3NVXX81HH33EsmXLaNiwYVhtn2yBbfYTExOzdRw5yek6Zw0aNGDlypVRWenK3f/VyuWccy+a2Q68oOleoB/gzGwB8IC/2cUZfvFL/SOUwBMYA//F/TkGQxYREZEwLF26FICKFSsmpwVWl4JtC3/48OHkS8Ty588PQJMmTdi3b1/Q9seOHUuhQoX429/+BkCBAgXCGtfPP3s/D9LeOyZyutM3Ppdzzk0AJvj3ZzUFrgZuBT43s5pA4PLAvs65cJaWdvmvlaI+WBEREUm2bt06qlatSuHChVOlb968mbvvvhtIvQV88+bNWbNmDUOHDuWSSy5JFSgFLi9s1KhR8n053bp1o1u3bkH7Hjt2LCVLlmTMmDGp0rds2UKBAgUoV65cujqjRo1i2bJlVKlSJehKm8jpTEGXAMn3Z80AZphZHrzAqwWw1C/SHDhh0OWc229ma4ALzKyec25VLMYbny+OH8K4vEI8gWX/zTcmZus4cgrNV+Q0Z5HTnEVOc5bapEmT+Ne//kWLFi0488wzKVq0KBs2bGD69OkcOnSIDh06pAqaHnnkET755BPmzJlDzZo1adeuHQULFmTx4sV8/fXXFCxYMOzLCUNZuXIl1113HU2aNKFGjRqUK1eOP/74g6VLl7J69WqKFCnCW2+9le4ZYiKnOwVduZiZtQLmu/Q39pX1Xw8455ab2UKgi5nd6px7I0g7dfDu4QrsgPgyMBoYZWZpdy/MA5TTZhoiIiJZ06pVK77//ntWrVrF4sWL2b9/PyVKlKBZs2Z0796d7t27s2DBguTylSpVYuXKlTz33HNMnz6dcePGcezYMSpUqECPHj148MEHqVmzZpbGVL9+ffr27cvChQuZPn06f/75J/Hx8VSvXp3777+fvn37UqVKlayeukiOo6Ard5sK7DOzpcBmwPBWtBoBK4Av/HI3AHOBsWZ2L/AV3mWElfGe6XUB0AQIBF1j/Ha6A+vNbBrec7oqAq2BN4DBsT01ERGR01vLli1p2bJlRHXKlCnDCy+8wAsvvJClvkNtxFa1atUsty1yOlLQlbs9BFwO1Ac6AIeA/wEPAiOdc0cAnHNbzawB3lbw1wA3AnF4ux6uBUYAqwON+itnN5vZ50BvoCtQAPgV7+HKH5+MkxMRERERORUo6MrFnHOvAa+FWXYvMNQ/wm3/HeCdzI1OREREROT0oIcji4iIiIiIxJCCLhERERERkRhS0CUiIiIiIhJDCrpERERERERiSEGXiIiIiIhIDCnoEhERERERiSEFXSIiIiIiIjGkoEtERERERCSGFHSJiIiIiIjEkIIuERERERGRGFLQJSIiIiIiEkNRDbrM7EIz621mxVOkFTKzsWb2h5ltMbO7otmniIiIiIjIqSzaK10PAYOBPSnShgI9gXigPPCymbWNcr8iIiIiIiKnpGgHXQ2Bec45B2BmeYEewHKgDFAd+AO4N8r9ioiIiIiInJKiHXSVA7ameN8QKAaMcs4dcM5tBaYBF0a5XxERERERkVNSLDbSiEvx72aAA+anSPsdKBuDfkVERERERE450Q66/gc0TvH+KuBn59yGFGkVgJ1R7ldEREREROSUFO2gawrQ1MzeM7PxwCXAB2nK1AI2RrlfERERERGRU1LeKLf3ItAe6Oq/Xw0MCWSa2ZnARcCzUe5XRERERETklBTVoMs5txe42MwS/KTVzrmkFEXy4AVkX0WzXxERERERkVNVtFe6AHDO/TdE+iZgUyz6FBERERERORXFJOjyn8+ViHf/VhHn3DN+en6gCLAz8CwvERERERGR01nUt4w3s7Z4G2V8DgwHnkqR3QDYDnSLdr8iIiIiIiKnoqgGXWZWH/gUbwXtAeC9lPnOuSXAZuDqaPYrIiIiEo5q1aphZkGP8uXLpyp75MgRhg8fTs+ePUlISCB//vyYGWPGjMlU+4HjySefTFXn66+/ZuDAgbRv357y5ctjZlSuXDkm5y8i2SPalxc+DhwEGjrnfjGzQUHKLAPqRblfERERkbAUL16cfv36pUsvUqRIqvf79+9PLleuXDnKly/PTz/9lGHb/fr1Y9euXenSnXMMHTqUo0eP0r59+1R5EydOZPjw4eTLl4/zzz+fbdu2RXpKInKKi3bQ1QyY6pz7JYMyW4AOUe5XREREJCwlSpRg8ODBJyxXqFAhZsyYQUJCAhUqVGDw4ME88cQTGdYJFswBfP755xw9epR69erRsGHDVHk9evTglltuoXbt2smraSJyeol20FUE756tjBQkBveSSe5y6EgS1R6ant3DyDEG1DkKQA/NWVg0X5HTnEVOcxa5rMzZ5mc7Rlwnf/786ValMmv06NEA3HHHHenyEhIS0qWJyOkl2kHXz0DtE5RJQNvGi/w/e3cep2O9/3H89TG2soQUHUso5FROluokMZREmzpRKiHFqUNKdVRatG9aTnVScpJSP0qLJSenYyukxXaQbEXIkkLKOnx+f1zXPc3cc98zc3PfmPF+Ph7zuOb+Xt/t+ppHzWe+yyUiIgfIjh07GDZsGN9//z2lSpWifv36NGvWjLS0tJS0t27dOsaMGUPp0qW58sorU9KGiBzckh10jQe6m9kZ4aEZ2ZjZucCZwONJblf2EzPrAPQE/gQUB5YCbwFPu/uOLPnqA3cCZwDHAL8AK4FPgNvdfdd+7rqIiAgAa9eupVOnTtnSatasyZAhQ2jevHnS23v11VfZtWsXXbp0oUyZMkmvX0QOfskOuh4hOA7+v2b2LFALwMxaA82Am4B1wNNJblf2AzN7hCCQ2kAQaP0KtCH4d29tZue6+84w4PoccGA0wcxmWeB44EbgbiDPoMvMZsa5dULRItAnXGYieat0WHC9TWOWLxqvxGnMEqcxS9y+jNnkyZMBaNGiBSeffDI1atTg8MMPZ82aNbz//vuMHTuW1q1b88ILL3D88cfHrGP58uUALFq0KLO+vLg7L7zwAgCNGjXKd7kdO3bkO29utmzZApCUug4VGrPEFdYxizxXMiQ16HL3VWGA9TbBL+cOGDAuvC4HLnX3vPZ9yUHGzM4g+DddCZzm7mvD9DuB94ELgNsIArDOQEmgnbuPiqqnPLB1P3ZdREQkU+fOnbN9rlmzJn369OGwww7j7bffZujQoTmOdN8XM2fO5IcffqB27drUrVs3afWKSMGS7Jku3P0rM6sDXAz8Gb12V2EAACAASURBVDgS2AzMIDjZcGey25T94trw+lAk4AJw9wwzu5XgRMrrCIKuiG3Rlbj7xvw26O6NYqWb2cyMPTQcMC/pP76FVuSvwhqz/NF4JU5jljiNWeL2ZcyWX5We6/2qVavy9ttv8/XXX5OeHjtv5K/4devWjZsn2j//+U8Abr311nyXAShRokRC+eOJ9DkZdR0qNGaJK6xjlszlwEn9L72Z/QHYFc5kvRt+SeHQMLxOjL7h7ovNbBVQ08yOAEYAvYEPzGwk8F9gmrsv22+9FRERScBRRx0FBO/mSpb169czatQoHaAhIkk/un0l8ESS65SDwxHhdU2c+5H0cu7+BXAWQYB2GTAUWGpm35hZx9R2U0REJHEzZswAoFatWkmrc8iQIezatYuOHTvqAA2RQ1yy1zRsAtYnuU45OGwOr5WBWDNWx2TNF55eeYGZlQAaAecBvYC3zOxHd//vvnSmZLE0Fu/FO1cOVZFp/7yW10hA45U4jVniNGaJ29cxW7hwIdWrV6dUqVLZ0pcvX07Pnj0BuPrqq/eli5ncncGDBwOx380lIoeWZAddnwMNklynHBxmEywxTCcq6DKz44GqwHfuvinrvfAY+enAdDNbArxOsN9vn4IuERGRRI0YMYKnnnqKZs2aceyxx1KmTBmWLVvGhx9+yPbt22nbti233XZbtjKPPfYY33zzDQBz5swBghmsqVOnAtC0aVOuu+66HG1NnDiRpUuX0rBhQxo1irlFOdM333zDY489li1t48aNdOnSJfPzgAEDqFixYsLPLCIHh2QHXfcDn5hZF3d/Lcl1y4H1KtANuNvMRkdOoDSzNGAAwVLVf4VpTYDZ7h59kEal8KrTC0VEZL9r0aIFixYtYvbs2UybNo3ffvuNcuXK0bRpUzp16kSnTp0ws2xlPvroI6ZMmZItbfr06UyfPj3zc6yga9CgQQB07949z36tXbuWoUOHZkvbunVrtrT+/fsr6BIpwJIddJ1NsI/nX2b2V+BLYC3B0fFZubs/muS2JYXcfbqZPQH8HZgfHpDxG8F7uk4CpgJPhtn/DrQ0s08J3tH1K3BimHcjMGg/d19ERITmzZsn/PLjvX3v0IgRIxgxYkS+8qanp+Me/auSiBQmyQ66Hsry/WnhVywOKOgqYNy9r5nNBnoC1wDFCJYa3g08leV1AC8SBFenA00Jfs5WhelPufuK/d13EREREZEDJdlBV6sk1ycHGXcfDgzPI89/gP/snx6JiIiIiBzckhp0ufuEZNYnIiIiIiJS0CX7PV0iIiIiIiKShYIuERERERGRFErq8kIz20XOkwpjcXcvkcy2RUREREREDkapeDlyrKCrHHA8UAKYB/yS5HZFREREREQOSsk+SKNpvHtmVhZ4DmgMXJjMdkVERERERA5W+21Pl7v/AnQjmAl7eH+1KyIiIiIiciDt14M03H03MAm4ZH+2KyIiIiIicqAciNMLiwPlD0C7IiIiIiIi+91+DbrMrDbQHli2P9sVERERERE5UJJ9ZPygXNqpBjQLv++bzHZFREREREQOVsk+Mv66PO4vBZ5098FJbldEREREROSglOygq3ac9D3ARnfflOT2REREREREDmrJfk+X9mqJiIiIiIhkkdSDNMxskJldkEeetrns/RIRERERESlUkn164XVAwzzyNCB4SbKIiIiIiEihd6De07X7ALQrIiIiIiKy36Ui6PJ4N8ysGHAWsC4F7YqIiIiIiBx09vkgDTNbHJXU28w6xciaBhwNHA5oT5eIiIiIiBwSknF64eH8PrvlQDHgsBj5dgOLgQnA/UloV0RERERE5KC3z0GXu1eNfG9me4Cn3P2Bfa1XRERERESkMEj2y5FbAd8muU4REREREZECK6kHabj7BHf/Lpl1ioiISMEzbNgwzAwzY/DgwTHz7Nixg6eeeopTTz2VsmXLUqpUKerUqUPnzp358ccfs+VduHAhQ4YMoV+/flSvXj2z7oyMjLh9ePLJJ2nbti01atSgdOnSlC1blpNPPpk+ffqwatWqpD6viEhukj3TBWSeUtgIqAKUiJXH3d9KRduHMjOrAXwHDHX3Lge0MyIicshauXIlPXv2pHTp0vz6668x86xdu5Zzzz2XefPmceaZZ3L99deTlpbG999/z/jx47n99ts56qijMvOPHz+e119/nSJFilCnTh1KlizJ9u3bc+3Hyy+/TOnSpWnevDmVKlVi165dzJ49m2eeeYZ//etfTJ48mQYNGiT12UVEYkl60GVm1wBPAhXjZSE4cENBl4iISCHj7nTt2pUjjzySSy+9lAEDBuTIs2fPHjp06MCiRYsYPXo0F154YY469uzZky2tTZs2FC9enOOOO47WrVtTo0YNVqxYkWtf5s+fT8mSJXOkv/LKK3Tv3p1+/foxbty4vXhKEZHEJHV5oZmdCwwBfgLuIAiwxgD3AZPCzyOB7slsV0RERA4Ozz33HBMnTmTIkCGUKlUqZp4PPviATz/9lFtuuSVHwAVgZqSlpWVLq1u3Ln/84x8pUSLmApqYYgVcAB06dABgyZIl+a5LRGRfJHum6zZgI/Bnd//FzB4HZrn7Q8BDZtYDeB54NsntyiFm+67d1LjjwwPdjQLjtpODPQ9dNGb5ovFKnMYscYVhzJY/dn62zwsXLuSOO+6gd+/eNGvWjIkTJ8Ys99ZbwWKXjh07sm7dOsaOHcv69eupXLky5557LlWqVElpv8eMGQNA/fr1U9qOiEhEsoOuRsAod/8lS1rmbJq7v2xmVwF3A22T3LZkEe7vegw4BygNzAf6u/vYqHwlgFuAq4DjgAxgLvC8u78dlTedYMbyfnfvH6PN5QDuXiNLWheC2c+uwFqCGdAGQFl3t317ShEROVhkZGTQqVMnqlevziOPPJJr3i+//BKAL774gptvvpmtW7dm3itWrBj33nsvd999d9L6NnjwYFatWsWvv/7KvHnz+O9//8uxxx7LY489lrQ2RERyk+ygqxSwJsvnHUCZqDxfEPwCLqlzLME4fwu8AVQALgdGmdk57j4JwMyKA+OB5sA3wD8JXnZ9GTDCzE5x97uS1KfLgPOAfwMvhX0UEZFC4oEHHmD27NlMnTqVww47LNe869evB+CGG26gR48e3HbbbVSoUIEJEyZwww03cM8991C1alW6dOmSlL4NHjyYzz//PPPzqaeeyltvvcXxxx+flPpFRPKS7KBrLXBUls8/AHWj8pRNQbuSXTrBrNb9kQQzewv4CLidYLYK4FaCgOvfwEXunhHmvZ8gaLvTzMa6+/Qk9Kkt0NbdP8pvATObGefWCUWLQJ+T4x8TLNlVCn//uU1jli8ar8RpzBJXGMZs8uTJAHz99dc8/PDDtG/fnh07dmSmL1++HIBFixZlpgHs3r0bgIYNG9K+fXtWrFjBihUrqFChAr1796Zfv37ce++91KhRI1t7W7ZsyWw3cnLhlClTcuz/ihaZ0dq8eTNLlizhX//6F6eccgr33nsvp5122j6MwMEv65hJ/mjMEldYxyzyXMmQ1IM0gK/JHmRNA842szMAzKwe0CHMJ6mzAngoa4K7jwe+B7L+3+VagpMk+0QCrjDveuDB8ON1SerTqEQCLhERKRh2797NY489RrVq1bj22mvzVaZ06dIANG3aNMe9008/nWLFirFy5cq4x83vrSOOOILGjRvz5JNPUrx4cR599FF27NiR1DZERGJJ9ozTv4FnzOwYd18DPEGwrGyqma0nmAUrAjyc5HYluznuvjtG+kogEgCXAY4HVrv7NzHyRnY/J+sFJl8kWsDdG8VKN7OZGXtoOGCeJkzzK/KXdI1Z/mi8EqcxS1xhGLPlV6WzadMmVq5cCUDr1q1j5hswYAADBgygd+/ePPvss5x00kl8+umnnH766aSnp+fIf8QRR7BhwwYaN25M5cqVM9Mjf0VPT0/PPJmwefPmFC2a+Bg2b96cDz74gCOPPJLGjRsnXL6gyDpmkj8as8QV1jErUyZ6l9TeS/Z/6V8G3gN+BnD3+WbWCriH4JCG+cAz7q6XYqTWpjjpGfw+u3lEeF0TJ28kvVyS+rQ2SfWIiMhBpESJEnTr1i3mvVmzZjF79myaNm1K3bp1OeOMMwA455xz+PTTT5k/fz6XX355tjLr1q1jw4YNlC5dmooV473yc9+tXr0aYK8CNhGRRCX1vzTuvhNYHZU2jeAABTm4bA6vlePcPyYqH0DkTZXxfm7KET/g8/x3TURECorDDjuMwYMHx7zXv39/Zs+eTefOnbnuut9Xq1977bU8/vjj/POf/6Rr167UqlULCJYq3n777QC0b99+nwKi77//nhIlSlCpUqUc915++WW+/PJLqlWrxsknn7zXbYiI5Jf+vHOIcvctZrYMqGVmtd09+g2RLcLrrCxpG8Nrtej6zOx4gtmzeEFXUpUslsbiqPfDSHyZm9qvSj+g/SgoNF6J05gl7lAes6pVq/Liiy/StWtXTjnlFC655BIqVKjA5MmTmTNnDnXq1OGJJ57IVmbDhg2ZB2K89tprbNiwAYBu3bphFryB5I477uCEE04Aglm29u3bc8YZZ3D88cdTqVIlfvrpJ2bMmMG8efMoXbo0b7zxRp6HcIiIJENKgi4zOxHoCNQDSrn7eWF6daAxMNHd98sv55KrVwn21z1pZn+J7AMzs4oES0IjeSK+AX4BLjazo8MDNzCzw4Dn9l+3RUSkoOvcuXPmu7JGjx7Nb7/9RvXq1bn99tu56667KFcu++r2X3/9lfHjx+eo5/XXX8/8vkuXLplBV8OGDenduzeffvopH374IT///DMlS5akVq1a3HrrrfTu3Ztq1XL8DVFEJCWSHnSZ2b3Avfy+dyjrsrJiwDtAb+CFZLctCRsAtAEuBuaa2TiC93S1B44GnnD3qZHM7r7LzP5BEJDNNrP3CX6GWhG8HuCH/dx/ERE5iPXv35/+/fvHvZ+enp7vjfc1atRg0qRJmeXyUr16dQYMGJCvukVEUi2pR8abWQegP8HJd42Bx7Ped/dlwEzgomS2K3sn3IPXCugXJvUCOgNLgCvdvW+MYvcBdwLbge4E7996F2gN7Ep1n0VERERECppkz3T1BpYBF7r7DjO7MEaerwleyCtJ5u7LAcvlfnqMtO3AI+FXftpw4LHwK1qNGPlfA17LT90iIiIiIoVRsl+OXB/4yN1ze9PgGiDnUUIiIiIiIiKFULKDLuP3Y8XjOQrQ699FREREROSQkOygaylwRrybZlYEaEqwxFBERERERKTQS3bQ9TbQyMx6x7n/d6A28H9JbldEREREROSglOyDNJ4FOgBPhycZGoCZPQacBfwZ+BJ4OcntioiIiIiIHJSSGnS5+1YzSyd4B9cV/D6T9neC93UNB25wdx0tLiIiIiIih4SkvxzZ3TcBV5vZLcBpwJHAZuBzd1+b7PZEREREREQOZkkPuiLc/Ufgw1TVLyIiIiIiUhDs80EaZnaNmdVPRmdEREREREQKm2ScXvga0C5rgpl1NrOJSahbRERERESkQEv2kfERNYDmKapbRERERESkwEhV0CUiIiIiIiIo6BIREREREUkpBV0iIiIiIiIplKygy5NUj4iIiIiISKGSrPd09Tez/tGJZrY7Tn5395S9I0xERERERORgkazAx1KcX0REREREpEDa56DL3bUvTEREREREJA4FTCIiIiIiIimkoEtERERERCSFFHSJiIiIiIikkIIuERERERGRFFLQJSIiIiIikkIKukREpEDo27cvZ599NtWqVeOwww6jQoUKNGjQgPvvv5+ffvopW94lS5bw+OOP07JlS6pVq0bx4sWpVKkSF198MZMmTYpZf40aNTCzXL8efPDBbGVeffVV2rVrx/HHH0/ZsmUpVaoU9erV4/rrr2fRokUpGwsRESlY9IJiEREpEJ555hkaNmxIq1atOProo/ntt9+YMWMG/fv3Z9CgQcyYMYNq1aoBcM899zBixAj++Mc/0rZtWypUqMCiRYsYPXo0o0eP5h//+Ac33XRTtvpvvvlmNm3alKNdd+eRRx4hIyODNm3aZLs3bNgw1qxZw+mnn07lypUpUqQICxYsYMiQIbz++ut88MEHOcqIiMihR0GXiIgUCL/88gslS5bMkd6vXz8eeeQRHn30UV588UUAzjvvPPr27UuDBg2y5Z0yZQqtWrXi9ttvp3379hxzzDGZ926++eaY7Y4fP56MjAwaNGhA48aNs90bN25czD59/PHHnHvuudx6660KukRERMsLRUSkYIgV3AB06NABCJYURnTp0iVHwAXQvHlz0tPT2blzJ9OnT89Xu4MGDQKgR48e+e5Tq1atKFeuHEuXLs1XGyIiUrhppksKpO27dlPjjg8PdDcKjNtOzgCgi8YsXzReiUvVmC1/7Pw884wZMwaA+vXr56vOYsWKAVC0aN7/C1y3bh1jxoyhdOnSXHnllfmqH2Dq1Kls2rSJhg0b5ruMiIgUXgq6DjFm1gW4EGgAHAPsAuYBA919WFTeyUBzoCRwD3AV8AdgFfA68Ki774wq48AU4ErgcaA1UAb4GnjK3d9K0aOJyCFiwIAB/Prrr2zevJmvvvqKqVOnUr9+fe644448y65YsYIJEyZw+OGH06xZszzzv/rqq+zatYsuXbpQpkyZuPlGjhzJ/Pnz2bZtG4sXL2bcuHFUqFCBF154IaFnExGRwsnc/UD3QfYjM9sGLADmA2uAI4G2QBXgIXe/J0veyQRB12jgVGAkQZB2MXAcMBa4yLP8EIVB1/+AI4BNwHigHNAhvP7d3Z/MZ19nxrl1Qs1atQ7v88DT+XtoodJhwXXdtgPbj4JC45W4VI3ZSVWOyJF26aWXsnHjxszPp512Gn379qVChQq51rVz505uvfVW5s+fT48ePbjiiityze/uXH311fzwww+89NJL1K1bN27eBx54INupiFWrVuXuu+/OtcyWLVsAcg3mJDuNWeI0ZonTmCWusI5Z9+7dWbJkySx3b7SvdWlP16HnJHdv7O5d3P1Od+8O1AImAneYWZUYZeoBJ7r7Te5+K3ASMAO4ALg6Rv76wBdAQ3fv6+49gIbARuBhM6uVgucSkUPEe++9x6RJk3j33Xd54IEHWLNmDd27d2fx4sVxy+zevZtHHnmE+fPn06JFCy6//PI825k5cyY//PADtWvXzjV4Arj33nuZNGkSY8eO5fnnn6dy5cr06tWLjz76KOHnExGRwkczXQKAmV0KvAt0dvfXw7TJBDNd17j7G1H504FJwGR3b5El3YHdQG13/y6qTH/gPqC/u9+/D32dWa1GrYZFLn9ub6s45ET22wyYpxXF+aHxSlyqxiw/e7pWrFhBnTp1qF27NvPnz89xf/fu3Vx99dUMHz6cDh068Oabb+ZrP1f79u0ZOXIkL730UsxDNHKzc+dOGjduzJIlS1iyZAlVq1bNkWfy5MkApKenJ1T3oUxjljiNWeI0ZokrrGPWqFEjZs2apZkuSZyZVTezf5rZN2a21cw8DJTeDbPEmumaEiNtKkFwlfN4MPg+OuAKTQ6vscqIiOyVY489lj/+8Y8sWLCADRs2ZLu3a9cuOnbsyPDhw7nyyit566238hVwrV+/nlGjRiV8gEZE8eLFOfvss9m+fTszZsxIuLyIiBQu+jPuISRc1vcFUB74FPgPsJkgeKoBdAZKxCi6LjrB3TPMbANwdH7yh9aG15ybNERE9sEPP/wAQFpaWmbazp076dChA6NGjeKaa65hyJAhFCmSv781DhkyJF8HaORm9erVQP5OSRQRkcJN/yc4tPQhODijq7u/lvWGmXUkCLpiqQR8H5W/KFAR+CVO/lgqh9fN+exvXCWLpbE4H8uOJBCZ9l9+VfoB7UdBofFKXKrHbPHixVSqVIkjjsj+N5s9e/Zwzz33sH79epo0aUL58uUB2LFjB5deeinjxo2jW7duDBo0KN8Bl7szePBgIPa7uSJ++uknNm/eTK1aObepjh07lvfff5/SpUvTvHnz/D6miIgUUgq6Di3Hh9d3Y9zL7beC5sAbUWlNgTRgdoz81c2shrsvj0pPD6+xyoiIxDVu3DjuvPNOmjZtSs2aNTnyyCNZt24dU6ZM4dtvv6Vy5cq88sormfn/+te/Mm7cOCpWrEiVKlV44IEHctSZnp4ec//BxIkTWbp0KQ0bNqRRo/jL+FeuXEmjRo1o3LgxdevWpUqVKmzatIk5c+YwY8YMihUrxuDBgzMDQREROXQp6Dq0LA+v6cCYSKKZtQauy6XcPWY21t03hvlLAo+G94bEyJ8GPG5mHd19T1imJnATkAEMi1FGRCSuc845h6VLlzJ16lRmz57Npk2bKFWqFHXq1KFTp07cdNNN2Y6M/+67YFvphg0bYgZcEbGCrkGDBgHBUcG5OfbYY7nzzjuZMmUKH3/8MT/99BPFihWjevXq9OjRg969e1OvXr29eFoRESlsFHQdWl4EugLvmNlI4AeC49/PA94G4p2hvBBYEJbJ+p6uD8k5AwbBe7pOB2aa2X/I+Z6uZUl7IhE5JJx00kkJvWg4stxxb4wYMYIRI0bkma98+fI89NBDe92OiIgcOnR64SHE3f8HtACmA+cDNwBlgUuBl3Ip2gF4FbgQ6Enwc9Mf+IvHfufARqAJwUuYuxLsFfsOuCq/L0YWERERESksNNN1iHH36UDLOLctTpkdwN3hV37b+YHYL04WERERETmkaKZLREREREQkhRR0iYiIiIiIpJCCLhERERERkRTSni6Jy93T96JMzH1hIiIiIiKHKs10iYiIiIiIpJCCLhERERERkRRS0CUiIiIiIpJCCrpERERERERSSEGXiIiIiIhICinoEhERERERSSEFXSIiIiIiIimkoEtERERERCSFFHSJiIiIiIikkIIuERERERGRFFLQJSIiIiIikkIKukRERERERFJIQZeIiIiIiEgKKegSERERERFJIQVdIiIiIiIiKaSgS0REREREJIUUdImIiIiIiKSQgi4REREREZEUUtAlcpB77bXXMLNcv9LS0jLzL1++PMf9Fi1a0KJFC8yMK664Ikcbn3zyCZ06deKkk07iyCOPpGTJktSsWZOLLrqICRMm7M/HFRERESl0ih7oDkh8ZrYcwN1r7Kf2agDfAUPdvcv+aFPydsopp3DffffFvPfpp58yceJE2rRpk+Pen/70J9q1awcEgRhAjRo1OOmkk3LknThxIhMnTuT000+nZcuWlCpViu+//57Ro0czZswY7r77bh588MHkPZSIiIjIIURBl8hB7pRTTuGUU06Jee+MM84AoHv37jHL9e/fH4DJkycDkJ6eHrOeO+64IzNvVqtXr6Zhw4Y88sgj3HjjjRxzzDEJ919ERETkUKflhZLVaqAecOeB7ojkbd68ecyYMYMqVapw/vnn71NdJUuWjJlepUoVmjRpwp49e/j222/3qQ0RERGRQ5VmuiSTu+8CvjnQ/ciP7bt2U+OODw90N1Jq+WO5B1KDBg0CoFu3btn2dEX88MMPvPzyy/z0009s2LCBE088Me5MVzzr16/n888/p0SJEtStWzehsiIiIiISUNB1gJmZAX8DbgCOA34C3gf65VKmI9AdaACUJNiH9SbwpLvvCPNUAb4H/ufuDeLU82/gPOBkd5+f254uMzsc6AW0B+oCBqwEPgYedvd1UXl7A5cDtQEH5gHPufv/5W9kJDfbtm1j2LBhpKWlcd1118XM8/HHH/Pxxx9nSxs2bBhDhw6levXqMct89dVXjB07loyMDFatWsWYMWPYvHkzzz//PBUrVkz6c4iIiIgcChR0HXjPAjcBa4BBwC7gYuB0oDiwM2tmM3sV6AqsAt4FNgF/Bh4EzjazVu6e4e6rzey/wLlmdrK7z4uq5xigFTDT3efn1kEzKw9MAv4ELAJeDft1XNiX94B1Yd5ywESCgHBWmLcI0Bp4y8xOdPe7Ex0kye7tt99m06ZNnH/++VSrVi3bvcMPP5x77rmHdu3aUatWLQCGDh3K0KFDmTx5MmeffTZz5syhVKlSOer96quvuP/++zM/lylThiFDhtCpU6fUPpCIiIhIIWbufqD7cMgysybANGAZcJq7/xymlyQIcv4MrIicXmhmXYAhBDNhV7n7tix19QfuA25293+EaR2Bt4Cn3P22qLZvB54AbnL358O0GsSY6TKzt4COwEvA39x9T5Z7pYE0d98cfn4N6Az0dfcnsuQrCXwAnAs0dPc5+RifmXFunVCzVq3D+zzwdF5VFGgnVTki7r2ePXuyYMECHn74YZo0aZJnXVu2bGH37t3cddddLFy4kL/97W9cdtllcfPv3LmTNWvWMHr0aN577z0uvPBC+vTps1fPURBt2bIFCIJOyR+NWeI0ZonTmCVOY5Y4jVniCuuYde/enSVLlsxy90b7WpcO0jiwuobXhyMBF4C7byf2YRa9gQzg2qwBV+hBgqWJV2VJ+wDYDFxlZtGbfjoTzKrlutzPzI4mWCa4Brgta8AV9vXXLAHXkcDVwFdZA64sz9SXYFnilbm1Kbn77rvvWLBgAUcddRSnn356vsulpaVlHrjxv//9L9e8xYsX59hjj6VXr15ceOGFjBkzhilTpuxTv0VEREQOVVpeeGA1DK+xfpudCuyOfAj3Sf0J2ADcHGwFy2EHwemDALj7NjN7G7ieYHnfuLCuRsCJwPvuviGPPp5KEJx/4u6/5SNvGuDhzFu0YuG1Xox7OcT7q4KZzczYQ8MB8wr3j+/yq9Jjpr///vsA3HjjjZx99tn5qityZHzTpk0ZMGAAhx9+eL4P1di8eTNjxozhxx9/TPggjoIqryP2JSeNWeI0ZonTmCVOY5Y4jVniCuuYJXPmrnD/1nrwi6wfWxd9w90zzCxrQFSeYJboKIJlhPn1GkHQ1Zkw6Aq/Bxiaj/LlwuvqfOQ9MryeGn7FUzofdUkM27dvOiRA4QAAIABJREFU54033iAtLY1u3bolXH7GjBkAmXu98mP16uCfvmhR/edCREREZG9oeeGBtTm8Voq+YWZFgYox8s52d8vtK2s97j4dWAJcZGblzKwYwf6sDfwehOVmU3itksDzPJNHH1vkoy6J4Z133mHjxo20adMmxwEaEbNmzWLPnj050mfOnMkzzzwDwNVXX53t3hdffBGzrmXLlvHII48A7PO7wEREREQOVfrT9YE1i2CJYXMg+s2zTQmW6gHB3ikzWwCcaGYVsu4By4ehwEMEe7PWEQRzz4Xv5crLF8AeoJmZlcpjiWEk71kJ9G2vlCyWxuI83mNVGEXezdW9e/e4efr06cOSJUto0qQJVatWBWDKlCnMnj0bgAcffDDH4RvnnnsuRx99NA0aNKBatWpkZGSwbNkyPvroIzIyMujVqxetWrVK0VOJiIiIFG6a6TqwXguv/cysQiQxPOnv0Rj5nyY4Rv7V8Gj2bMysvJk1zFmM1wmCoWvCr6xt58rdfwSGA8cAA8ws28+MmZU2syPCvOsJ3hfW2MzuiXF4B2Z2nJnVzE/bkt3ChQuZOnUqVatWpW3btnHzderUiQYNGvDll1/yyiuv8OKLL7J69WrS09P55JNPuPvunCf2P/DAA9SpU4cZM2bw4osvMnDgQObOnUu7du346KOPeO6551L5aCIiIiKFmma6DiB3n2ZmzxO8dHi+mY3k9/d0bSQ4MTBr/lfDQzBuBJaZ2XiCFyBXAGoCzQiOlP9rVLmVZjYJOJvg9MN57j47ga72BE4K600P290ZttkauAiYnCVvbeABoJOZTSWYXfsDwQEapxIsb/wugfYFqFevHvl5xUO3bt1y7PeKbHA966zYk5A33XQTN9100z73UURERERyUtB14PUGFgN/A3oQHPv+PnAXMDc6s7v/zcz+TRAAnUNw0MXPBMHXk8CwOO28RhB0FSV/B2hkbXNj+E6xmwmWKHYnOFlxJcHLj7/OkvcXM2se5rkS+AtQkiDwWgLcAnycSPsiIiIiIgWZgq4DzIOpixfCr2g14pQZC4xNsJ1hxA/IInmWE5yQGOveb8DD4Vdebe0k/jOJiIiIiBxStKdLREREREQkhRR0iYiIiIiIpJCCLhERERERkRRS0CUiIiIiIpJCCrpERERERERSSEGXiIiIiIhICinoEhERERERSSEFXSIiIiIiIimkoEtERERERCSFFHSJiIiIiIikkIIuERERERGRFFLQJSIiIiIikkIKukRERERERFJIQZeIiIiIiEgKKegSERERERFJIQVdIiIiIiIiKaSgS0REREREJIUUdImIiIiIiKSQgi4REREREZEUUtAlIiIiIiKSQgq6REREREREUkhBl4iIiIiISAop6BIREREREUkhBV0iKVKjRg3MLOZX5cqVY5aZPn06bdu2pUKFChx22GHUr1+fZ599lt27d+fIm56eHrf+yFe3bt1S/ZgiIiIikoeiB7oDqWBmywHcvcaB7Un+mVlZ4CHgIqAqkAY0cPc5B7Rje8HMpgJ/dvdC+fOViCOOOIKbb745R3rp0qVzpI0aNYq//OUvlCxZkssvv5wKFSowZswYbrnlFqZNm8Y777yTLX+XLl1IT0+P2e7zzz/Pzz//TJs2bZLyHCIiIiKy9w75X4qTzcwaA38DmgPHALuAFcBHwLPuvjpO0SeAHsBY4A1gN3Cimc0G3nb3y2O0dSfwCLADKOfu26Pu1wS+Bb5z91pJeDxJULly5ejfv3+e+X755Reuv/560tLSmDx5Mo0bNwbgwQcfpGXLlowcOZLhw4dzxRVXZJbp0qVLzLoWLVrE/fffT6VKlbj44ouT8RgiIiIisg+0vDBJLPA48CVwNfAN8BzwL2ArcBuw2Mwui1PFBcBid7/Q3e9x9/7Av4E9QLqZWYwyZwMOlADOjHMf4L9791Syv4wcOZIff/yRK664IjPgAihZsiQPPfQQAAMHDsxXXYMGDQKga9euFCtWLPmdFREREZGEaKYree4B/g4sBy5w9wVZb5rZX4BhwHAza+Xuk6LK/wH4JGuCu/9sZnOBBsBJwLws9ZUAmgDvA+2AlsCEqDpbhtfo9AJv+67d1LjjwwPdjZiWP3Z+5vc7duxg2LBhfP/995QqVYr69evTrFkz0tLSspWZOHEiAOedd16O+po1a8bhhx/O9OnT2bFjByVKlIjb9o4dO3j99dcxM66//vokPZGIiIiI7IsCO9MVziz1NLMFZrbdzFab2QtmdkSMvEeY2e1mNtHMVpnZTjP70cxGm9kZUXnLm9lWM1sWZ3YJMxtjZh4uJcTMahAEXbuAi6IDLgB3fxe4hWCv1kAzKxKWnWxmDhjQPKzXzWxyWDQSMLWMqvIM4DDgPWBOjPsALQhmwiZG9b9oOHafm9mW8HlnmdmNuTzzGWb2rpmtC8dvpZm9ZGbHxMofp45WZvZL+G9VP7/lCrK1a9fSqVMn+vXrx80330zLli2pXbs2U6ZMyZZv0aJFANSpUydHHUWLFqVmzZpkZGTw7bff5tree++9x4YNGzjnnHOoVUsrSkVEREQOBgU26AKeBZ4HygODgOHAeQRL6YpH5a0HPEywVO9D4GngY4JA5RMzy5xecPeNYV21gHOiGzWzakAbYKa7fxUmdyWYNXzf3edFl8liMLAGqEuw5wvgNeD+8PsV4ff3h+nwe8AUWSpI1OeJwCSgsZmVydLPE4HKwDx3/zFLenGCZYvPA2UJZt8Ghf3/J/BqjGe+HpgKtCYIAp8FZgLXA1+aWZVcnjlSxzUEY7+S4JCN/+VVpqDr2rUrEyZMYO3atfz222/MmzePHj16sHz5ctq0acPcuXMz827evBkIDt6IJZK+adOmXNuMLC3s3r17Mh5BRERERJKgQC4vNLMmwE3AMuA0d/85TO9HEIAcQxDARCwE/uDuG6LqqQp8ATxDcNBFxIsEgVQPguAsq24Es1UvZ0lrGl5z3Tvl7hlmNgm4kmAP1iR3fy3sy33A8nAvV1afEMygNTOzNHePnB3eEljk7mvCOm8lCOTGZrkPOZcW3ksQTP4DuDVSn5mlEew/62JmI939wzC9HkEwthRId/c1kYrM7FyCAO5ZoH285zazuwiC3ilAO3fPPXL4vdzMOLdOKFoE+pyckZ9q9rvJkycD0Lx5EFcvXLgw894VV1zBxo0befvtt+nZsycPPvggANu2bQPg888/Z9WqVTnqjARls2bNYseOHTHbXbVqFZMnT6Z8+fKUK1cusx8AW7ZsydY3yZ3GK3Eas8RpzBKnMUucxixxGrPEFdYxizxXMhTUma6u4fXhSMAFEJ7ed2d0ZnffHB1whemrgJHACWZWPUv6V8BXwMVmlvlCpTAw6QZsAf4vS1WRJXYr89H3SJ4/5CMv7v4b8DlwBNAo7Edp4FR+nwX7lOC0w6xLDHMEXWH//wasJkvAFbazm+CwD4CrstRzI1AMuClrwBWW+Q8wDmhnZqWi+25maWY2kCDgehtond+AqzC76KKLAPjf/36f7CtVKhi+X3/9NWaZ3377DYh91HzE2LFBvN2mTRuKFi2Qf08RERERKZQK6m9mDcPrlBj3phIEINmY2ZlAb4K9UEeTcwliFeD7LJ9fJFhqdy3BsewAbQneoTXQ3WP/dpwaEwlm01oSzMydRRAITQJw91/CWaGWAOF+seZABtkP56gHlAPWAffE2b61PcwXEdnz1iJ6/1uoIsHP0fHA3CzpRrDf7CKCmbA+7u75e9yAuzeKlW5mMzP20HDAvIPzx3f5Vem53o/MWu3YsSPzPVuNGzdm0aJFlC1bNse7tzIyMli/fj1FixbliiuuiHmQxs6dO+nQoQNmxoMPPphjP1fkL0/x3usl2Wm8EqcxS5zGLHEas8RpzBKnMUtcYR2zMmXK5J0pnw7O31rzFtn4si76RriEL3oZ4SUEM1rbCZYLLgN+IzyOnSBAif5NdjjwFHC9mT3m7nuAyEaZl6PyriUIVKrlo++RPD/kI2/EBIJlgWcDj/H7UfFZT0CcDNxuZhWBGgR73aa5e9Z50SPDa13gvlzayzqdEinTN48+Rk/BGEFwuAsYk2jAVZjNmDEDIFtg1LJlS958800++ugjOnbsmC3/J598wtatW2nWrFnckwvff/99fvzxR1q1aqUDNEREREQOMgU16NocXisRvPw3k5kVJZh9ybox5kFgJ9DY3RdG5X+Z3w+1yOTu28zsNYITB881swUEB2h87u5zo7JPJTgp8BzglXidDpf3pYcfp8V/vBxmELzr68zwIIyWwPyoJZOTCI6sbwHUDNOi93NFxu0dd++Qz7YjZUq5+9YE+ryHIDj8DzDWzC5x9/EJlM9VyWJpLM5yNPvBZuHChVSvXj1z2WDE8uXL6dmzJwBXX311Zvpll11G3759GT58OL169cp8V9f27du5++67Abjhhhvithc5QKNHjx5JfQ4RERER2XcFdU/XrPCaI1giWIaXFpV2PPB1jICrCL8fghHLQIIZpR7EPkAj4jWCJY2XhKcGxnMtwV6uRcReGhmTu+8kCOwOI3iJ8p+IOgY+vJ9BEJDFO0RjAcF+tDPC4DQ/ZoTXs/Lb3wh3n00QZG4GRpnZhYnWUVCNGDGCypUrc/7553PjjTfSt29fLrvsMurVq8fSpUtp27Ytt912W2b+smXL8sorr7B7927S09O57rrr+Pvf/84pp5zCZ599xmWXXcbll18es62lS5cyadIkKlWqlLlfTEREREQOHgU16HotvPYzswqRRDMrCTwaI/9yoLaZ/SFLXgP6A3+M14i7LyEIXC4A/gpsIlh2GJ3vW4J9X8WA0WaWo04za0dwYuBu4IZwuWIiIkFWf4J/t2wvVw73mH0JnEsQSG7l94ApkmcX8ALBvrRnw/GK7ucfwhMLI54nCOb+YWbHx8hf3MziBq7hO8uaAxuAd83sstwfs3Bo0aIFF1xwAcuWLeOtt97i6aefZsqUKTRt2pShQ4cyduxYihfPvq2wXbt2TJkyhWbNmvHuu+/y/PPPU6xYMZ5++mmGDx9OnD14vPLKK7g7Xbt2pVixYvvj8UREREQkAQVyeaG7TzOz54FewHwzG0mwd+hiYCPBu7CyegZ4CZhtZu+Gec8kCLjGALnNwLxIsGywEvC8u2+Lk68/UAroA8w1s/EEM0vFgCbA6cA2oKO7T4pTR24is1YnEyzdizVTNgm4K/x+fDhDFu0+oD7BKYYXm9lEgv1llYDaYV/7Ehyzj7svMLPrCJZNfm1m/waWEOyBq04wA/YDcFK8jrv7YjNrFj7DcDO7xt3fyu+DF0TNmzfPPDI+EWeeeSbjxo1LqMzjjz/O448/nnBbIiIiIrJ/FNSZLghOIuxFsHStB9ARGE8QIGULNtz9ZYJj5tcAnQmORF9JEAjNInejCWZpIPbSwkgbe9z91rDOt4ATCd4l1p3gkImngDru/k6+nzC7WQQBJcDsOEevZw3mopcWRvq5i+BEwS4EwdOFBO/4ah1muZuo2Tx3H0pwRP3/AacQjPtVwHHACKBnXp0PZwObAd8Bb5hZ1zyKiIiIiIgUCgVypgsgPA3vhfArWo0Y+V/j92WJWc0jmKWK51iCE/ymhkvl8urXFwTHuifE3WOvHfv9/h6gQh55/ktwamBebe0BhoZf+e3fXIKANT95Yy43dPeVBLNpIiIiIiKHjII807W/3EYQyMQK7kRERERERHJVYGe6UsnMqgNXEszKdCV46e/eLgsUEREREZFDmIKu2GoRnIK4leBlyntz2qCIiIiIiIiCrljcfTL52BslIiIiIiKSF+3pEhERERERSSEFXSIiIiIiIimkoEtERERERCSFFHSJiIiIiIikkIIuERERERGRFFLQJSIiIiIikkIKukRERERERFJIQZeIiIiIiEgKKegSERERERFJIQVdIiIiIiIiKaSgS0REREREJIUUdImIiIiIiKSQgi4REREREZEUUtAlIiIiIiKSQgq6REREREREUkhBl4iIiIiISAop6BIREREREUkhBV0iIiIiIiIppKBLDlkjR46kV69enHXWWZQtWxYz4+qrr46bf8uWLfTr148TTjiBkiVLUr58eVq3bs2ECRPilpk1axbt27enUqVKFC9enOrVq3PjjTeybt26VDySiIiIiByEih7oDhxIZrYcwN1rHNie5J+ZlQUeAi4CqgJpQAN3n3NAO1YAPfTQQ8ydO5fSpUtTtWpVvvnmm7h5N27cSNOmTfn666858cQT+etf/8qvv/7KqFGjOOeccxg8eDDdunXLVmbs2LFceumlZGRkcOGFF1KnTh2++eYbXnrpJcaMGcO0adOoXr16qh9TRERERA4wzXQdIGbW2MyGmNm3ZrbNzH4xs3lm9qSZVcml6BNAL2Ae8CjwMDDCzNzMLsmlvUvCPJ+b2SEdbEc888wzLF68mF9++YWBAwfmmrd///58/fXXXHrppcyZM4dnn32WwYMHs2DBAqpVq0avXr1YtWpVZv7t27dz3XXXsWvXLkaOHMmoUaN48sknGTNmDG+++SarVq2iZ8+eqX5EERERETkIKOjazyzwOPAlcDXwDfAc8C9gK3AbsNjMLotTxQXAYne/0N3vcfe7w7TfgFfM7JgYbVYGBoV5rnL3jGQ/V0HUokULateujZnlmff9998H4IEHHqBo0d9j1qOPPpo+ffqwbds2Xn311cz06dOns27dOho3bsyll16ara6OHTvypz/9ibFjx7JixYokPY2IiIiIHKw047H/3QP8HVgOXODuC7LeNLO/AMOA4WbWyt0nRZX/A/BJ1gR3X2JmfYCXgSFm1sbdPUuWV4GKQHd3X5rUpzlAtu/aTY07PtyrsssfOz/hMmvXrgWgVq1aOe5F0iZMmMC9996bZ/5I+ty5c5k4cSJdu3ZNuD8iIiIiUnAU+pmucGapp5ktMLPtZrbazF4wsyNi5D3CzG43s4lmtsrMdprZj2Y22szOiMpb3sy2mtkyizNVYmZjwiV9jcPPNQiCrl3ARdEBF4C7vwvcQrBXa6CZFQnLTjYzBwxoHtbrZjY5LDcIGA20Jlh+GOnDjUAbYJS7vxKjjyeZ2RtZnndN+Pm4GHnrmdkTZjbTzDaY2Q4z+87MBsaZYTsv7OMdZtbEzD4ys5/DtMqxxuxgVbFiRQC+++67HPe+/fZbABYtWpSv/PHKiIiIiEjhVOiDLuBZ4HmgPMESu+HAecB/geJReesR7JHaA3wIPA18DLQEPjGz8yIZ3X1jWFct4JzoRs2sGkGwM9PdvwqTuxLMLr7v7vNy6fNgYA1QF2gepr0G3B9+vyL8/v4wPeI6YB3wuJn90czqAE+GadfH6ONFwFdAB2AGwVhNCT9/ZWYnRRW5PGxjOfAm8AKwBOgBfGlmleI8T/Ow3iIEyyjfIAg8C4zzzw9mx+677z52796dmf7jjz/yzDPPAMFhGxFnnnkm5cqV48svv2TUqFHZ6nr77beZO3dujjIiIiIiUjhZ9lVohYuZNQGmAcuA09z95zC9JDAJ+DOwInJ6YTj7VczdN0TVUxX4Atjs7vWypDcm2Jv1rrtfFlWmP3AfwZK+V8K0CQQBXPdYs05R5d8ErgTucfeHsqQ7MMXd0+OUa0sQMM4hCGxOBdq6+7+j8h0NLAW2AWe5++Is9xoA04E57n5GlvSqwHp33xlV14XAKOAf7n5LlvTzgEi7Xdx9aG7PHONZZsa5dULNWrUO7/PA04lUl+mkKjkmOZkzZw633HIL55xzDv369ctx/6effuLGG29k/fr11KxZk4YNG7J9+3amTZtGxYoVWbp0KcWLF2f8+PGZZcaPH8/jjz+OmdGkSROqVKnCypUr+eyzzzjuuONYunQpF110EbfcckuO9pJty5YtAJQpUyblbRUGGq/EacwSpzFLnMYscRqzxGnMEldYx6x79+4sWbJklrs32te6CvtMV2SzzMORgAvA3bcDd0ZndvfN0QFXmL4KGAmcYGbVs6R/RTBTdHHW5XJmlgZ0A7YA/5elqsgSvJX56Hskzx/ykTdrX8cBA4FTCAKuF6MDrlBXoAxwV9aAK6xjNsEM2p/NrFaW9FXRAVeYPoYggGsdp1szEg24DjZHHnkkAwcOpF27dmzdupVRo0YxY8YMWrRoQf/+/QEoX758tjKtW7fmqaeeolGjRsyZM4f33nuPdevW0bdvX845J5gcLVeu3P5+FBERERHZzwr7QRoNw+uUGPemArujE83sTKA3cAZwNDmXIFYBvs/y+UWCgyquBR4J09oSvENroLv/ured3we3AjeE398WJ09kBqtRuBQyWiTYqgd8C8H+OKAzcA1QHyhHsPcs4pc4bX2Rv25nF++vCmY2M2MPDQfM27sf3+VXpce9V6lSJdLT49+PPokQYOLEiQA0bdo0R9n09PSYM1nXXHMNAO3bt8+1vWSZPHlyZn8kbxqvxGnMEqcxS5zGLHEas8RpzBJXWMcsmTN3hT3oiqwjWxd9w90zzCx6GeElBDNa2wn2ci0jOGZ9D5BOsDepRFRVw4GngOvN7DF33wN0D++9HJV3LUEQEyvIiRbJ80M+8mbj7tsiZ3u4+7Y42Y4MrzfEuR9ROsv3LwJ/BVYTLBtcTTBWEOz1qhinjrV5tFGgvf766wBceeWV+cq/adMmxowZw1FHHUWrVq1S2TUREREROQgU9qBrc3itRDhbExG+ILgisCpL8oPATqCxuy+Myv8yvx9qkSkMcF4jOHHwXDNbQHCAxufuPjcq+1SgBcHBG3H3dIXLE9PDj9PiP94+iYxN3ejlhXH6VI0g4JpFsAdsa9T93M49L/AbB/fs2cPWrVspXbp0tvQ33niD119/nSZNmtCuXbts97Zs2ZLjLyRbt26lc+fObNq0iYEDB1KiRHQMLyIiIiKFTWEPumYRLDFsTlTQBTQl+9I4gOOBBTECriJh/ngGAjcTnOI3N6w3epYLgn1SdwGXmNmJsY6MD11LsJdrEbGXRibDDOB84Cwgz6ALiBwh/1GMgKsmwcxcjv1eqVKyWBqL9+J9W1l98MEHfPDBB8Dv79X67LPP6NKlCxAc+z5gwAAgCJYqVapEq1atOO644yhSpAjTpk3js88+o169erzzzjsUKZJ9i+TQoUN56qmnSE9P55hjjuGnn35izJgxrFmzht69e/9/e3cebEddJXD8ewwQdsQIgqBsg2OAkVGQXRYtGXFAGHEBBUFHR6YKNAgzOFgOcRwWAQcYhFIxGByXKkFZxLCKLBEYVIKKoAOEYNghbAECGjjzx+/34Obm3reQ1+++5fup6uqk+9d9T5/b7717bnf/fhxyyCHLFL8kSZLGhvFedM2k3Pb2hYi4sK33wuM7tJ8HbBoRr8/M+2vbAKYDm3V7kTo48c+APSnPSj1Bue2wvd3ciDiOMlbXRRGxV2be1tomIvYBTqM8b/bP9XbFJpxFGaT5PyNiTmbe3BbHJMoVravronl1vnNEvKovrohYve6r41hlo9ktt9zCOecs2b/H3LlzXxpDa4MNNnip6Jo8eTL77bcfs2fP5oorrgBg00035dhjj2XatGmsvPLKS+1/6623ZurUqVx66aUsWLCA1VdfnW222YYZM2awxx57NHx0kiRJGi3GddGVmb+IiNMpgwXfGhHnUbpR3xt4nDIWVqtTgK8DcyLiR7XtjpSC6yfAXv283JmU2wZfB5zez7NU04FVgM8Bv4mIy4DfA8sDOwDbUrpx3z8zfz6kAx6CzHwoIj4MnEsZY+tKoK8AfGONZSVKZxlk5ryIuADYB7i5tl+T0mPhE/UY/qqpeJswffr0l3oeHMjyyy/PjBkzhrT/7bbbjlmzZr2CyCRJkjSejPcu46H0RHgY5RmmTwP7A5dRCqQlbofLzG9QulJ/gNJL30cpXbdvS7lVsT8XAX0dc3S6tbDvNV7MzCPqPr8PbA58htL5xqqUTjnelJnnDvoIX6HalfyWlEGjN6F0qvEJSpF5GXBA2yYHAidSupo/lJLDH1NuvexFL42SJEnSqDeur3QBZBn9+Wt1ardhh/YzKbcltvsd5SpVNxtQegSc3c+zWq2vcxOvoCv1zBzUbXxDaHcXA/dg2Nf2aeCoOrXbrkP7SxmDtx1KkiRJw2kiXOkaKUdSCoxOxZ0kSZKkCWrcX+lqUkS8EfgIsCnltsTfUJ6RkiRJkiTAomtZbUzpBfFZymDKTfY2KEmSJGkMsuhaBrU7dZ9ZkiRJktSVz3RJkiRJUoMsuiRJkiSpQRZdkiRJktQgiy5JkiRJapBFlyRJkiQ1yKJLkiRJkhpk0SVJkiRJDbLokiRJkqQGWXRJkiRJUoMsuiRJkiSpQRZdkiRJktQgiy5JkiRJapBFlyRJkiQ1yKJLkiRJkhpk0SVJkiRJDbLokiRJkqQGWXRJkiRJUoMsuiRJkiSpQRZdkiRJktQgiy5JkiRJapBFlyRJkiQ1yKJLkiRJkhpk0SVJkiRJDbLokiRJkqQGWXRJkiRJUoMsuiRJkiSpQZGZvY5BGpKIWDB58uTXbL755r0OZcxYuHAhAKuttlqPIxkbzNfQmbOhM2dDZ86GzpwNnTkbuvGas9tvv51FixY9lplTlnVfFl0acyLieWAS8JtexzKGvLnO/9DTKMYO8zV05mzozNnQmbOhM2dDZ86GbrzmbEPgqczcaFl3tNyyxyKNuFsBMnOrXgcyVkTEr8GcDZb5GjpzNnTmbOjM2dCZs6EzZ0NnzgbmM12SJEmS1CCLLkmSJElqkEWXJEmSJDXIokuSJEmSGmTRJUmSJEkNsst4SZIkSWqQV7okSZIkqUEWXZIkSZLUIIsuSZIkSWqQRZckSZIkNciiS5IkSZIaZNElSZIkSQ2y6JIkSZKkBll0aUyIiPUj4uyIuD8ino+IeRFxakSs2evYhktEfCAiTo+I6yLiqYjIiPjuANvsEBGzIuKxiFgUEb+NiGm1U0/9AAAO4ElEQVQRMamfbfaMiKsj4smIeDoi/jciDhrgdQ6KiJtq+yfr9nu+0mMdDhExJSI+GRHnR8Sd9fifjIjZEfGPEdHx99tEzlmN6ysR8bOImF+P/7GImBMRx0TElC7bTOicdRIRB9Sf0YyIT3Zp03gOImJSRBxe35O+93NWROywrMe4LOrv6OwyPdhlG88zICLeVX+vPRjl7939EXFZRLy3Q9sJm7OIOLifc6xveqHDdhM2Z30i4u8j4vKIuLfmYG5EnBsR23dpP+FzNiwy08lpVE/AJsBDQAIXACcAV9X//wGY0usYh+k4b6nHtBC4vf77u/203xtYDDwNzABOqvlI4Nwu2xxa1z8KnAGcAsyvy07uss3Jdf382v4MYEFddmgP83VIjeF+4HvA8cDZwBN1+XnUAeDN2RKx/Rm4sebqBOB04Jc1tvuAN5izAXP4hnqeLazxfbIXOQACOJeXfxeeVN+jp+t7tncPczSv5mh6h+nIDu09z0p8J7bE903gOOAs4GbgRHO2RFx/2+X8mg78rMZ3sTlbKravtBzPtyh/B86j/G14ETjAnDWU+14H4OQ00ARcVn/oDmtb/l91+dd7HeMwHeduwKaUD1K70k/RBawOPAw8D2zdsnxF4Pq67X5t22wIPFd/iW3YsnxN4M66zfZt2+xQl98JrNm2rwV1fxsuy3EvQ77eCewFvKpt+TrAn2rc+5qzpfK2Ypflx9a4zzRn/eYvgCuBuygfPpYqukYqB8D+dZtftL6vwNvre/YwsFqP8jQPmDfItp5nJY5P1fhmAit0WL+8ORt0Lm+ocb/PnC0R2zrAC8CDwNpt63arcc81Zw3lv9cBODn1N1GuciVwN0t/uF6N8s3LM8AqvY51mI97V/ovuj5R15/TYd0767pr2pb/R13+pcHuD/hOXf7xDtt03V+vJ+DoGtvp5mzQOduyxnaFOes3T5+lfBu8M+Ub9U5F14jkALi2Lt+twzZd9zdCeZrH4IuuCX+eAZMpH27voUPBZc6GlMu/qXHdC0wyZ0u8/rb19S/ssv4pYKE5a2bymS6NdrvV+eWZ+WLrisxcSPmGd2Vgu5EOrMfeWeeXdlh3LfAssENETB7kNpe0tVmWbUaDv9T54pZl5qx/e9X5b1uWmbMWETGVcivOaZl5bT9NG89BRKxI+Xb4WeC6IbzOSJoc5dm3oyPisxGxW5dnQDzP4N3AWsCPgRfrMzdH1bx1es7GnHX3T3U+IzNbn+kyZ3AH5TbCbSLita0rImJnypfZV7YsNmfDyKJLo91f1/n/dVl/R52/aQRiGU265iUzF1OuDC4HbDzIbR6gXDFcPyJWBoiIVYD1gKfr+najMvcRsRzwsfrf1l/g5qxFRBwZEdMj4pSIuA74MqXgOqGlmTmr6nn1P5RbV48eoPlI5GATYBLlVqDFS28yKvK2DiVnxwKnUp7FvSMidmlr53lWbgmFchvVHOBiys/iqcD1EXFNRKzV0t6cdRARKwEHUG6h+1bb6gmfs8x8DDgKeB1wW0R8MyKOj4gfApcDVwCfbtlkwudsOFl0abRbo86f7LK+b/mrRyCW0eSV5GWw26zRNh9ruT8B2AKYlZmXtSw3Z0s6EjgGmAbsRClQd8/MR1ramLOX/TvwVuDgzFw0QNuRyMFoz9u3gXdRCq9VKLd8fYPyjMYlEbFlS1vPM1i7zv+FcivVOyhXHd5C+TC8M6XTlD7mrLMPUeK5NDPnt60zZ0Bmngq8n1IsfQr4PPBBSgcWMzPz4Zbm5mwYWXRJGjci4jPAEZSelQ7scTijWmauk5lB+VD8fso3lXMi4m29jWz0iYhtKVe3vpqZN/Q6nrEgM7+UmVdl5kOZ+Wxm3pqZh1A6QFqJ8jycXtb3eWwxpfOH2Zn5dGb+DvgHyvNJu3Tr0lsv6bu18Bs9jWIUi4h/pfRWOJNyxXwVYCtgLvC9iDixd9GNbxZdGu3avxFp17f8iRGIZTR5JXkZ7DZPts3HRO4j4lDgNOA2SscCj7U1MWcd1A/F5wO7A1MoDzT3mfA5q7cVfodyq8wXB7nZSORgVOetH1+v851blk3486zldedk5rzWFZn5LKUXX4Bt6tyctYmIzSnPOd4LzOrQZMLnLCJ2pXQZf1Fmfi4z59YvRW6mFPf3AUdERN/tghM+Z8PJokuj3R/rvNu9vJvWebdnvsarrnmpHxI3onxjOneQ26xL+bbr3voHnsx8hvILeNW6vt2oyX1ETKOMN3UrpeDqNPiqOetHZt5DKVg3b3nA2pzBqpRjmQo81zrwKuX2TICz6rJT6/9HIgd3UZ5b2bi+F4PZZjTou311lZZlnmcvH0+3D5aP1/lKbe0ncs7adetAo485g76Bhn/evqIew02U2uCtdbE5G0YWXRrt+n4x7B4RS5yvEbEasCOl95wbRzqwHruqzt/TYd3OlB4dr8/M5we5zR5tbZZlmxEVEUdRBlK8hVJwPdylqTkb2OvrvO8Dizkr49PM6DLNqW1m1//33XrYeA4y8znKODkrU57/Gezr9FpfT7OtH9I8z14ezHez9r911RZ1fnedm7MWtTfPAym/u2Z0aWbOytAEUHrK7KRv+Z/r3JwNp173We/kNNDEBBkcue3YdmXgwZEfYWgDFm7EOBuwkHK7VwK/Al4zQNsJnzPKN49rdFj+Kl4eHPkX5mzQ+ZxO53G6RiQHDG5w5NV7kJepdBg7sR7LHTXmoz3PlsrPhTW+w9uW704ZG+7xvp9fc7ZU7g6scf6knzYTPmeUjkaSMjjyem3r9qjn2SJgijlrIP+9DsDJaaCJ8qDnQ/UH8gLgeMo3Hkm5jD2l1zEO03HuQ3mwdSalJ7mk3ELUt+zkDu0XUwaI/hZwIqUDiaT0chUdXuOwuv5R4AzKFaL5ddnJXeL6al0/v7Y/o26fwKE9zNdBNYbFNa7pHaaDzdkScU2j/EG9Avhm/Vk6u55nCTwAbGbOBp3P6XQoukYqB0DU9yCB2+t7M6O+V4uBvXuYl4XAT4EzKc+QnFfPvazLV2jbZsKfZ8D6lCEJkjJW0kk1b4spYw/ua8665u66Gs9eA7Sb0DmjfMF2RY3jKeCc+vN5EaXgSuCz5qyh/Pc6ACenwUzAGyhdED9Auex9D2X8kjV7HdswHuP0+sul2zSvwzY7Uh4YfpzygeZ3wOHApH5eZy/gGsqHomeAXwIHDRDbwbXdM3W7a4A9R3m+ErjanC0R0xbA1yi3Yj5a/5A+WeOcTperhRM5Z4M8B5cqukYqB5Runw+v78mi+h7NAnboYV52AX5A+WD2BKVgeITyYe9jdPiQ5nn2UmxrUZ5PvYfyt+5R4HxgG3PWNbapvPxhvetxm7OX4lqe8gXcjZTCazHlqvjFlGFDzFlDU9SDlCRJkiQ1wI40JEmSJKlBFl2SJEmS1CCLLkmSJElqkEWXJEmSJDXIokuSJEmSGmTRJUmSJEkNsuiSJEmSpAZZdEmSJElSgyy6JEmSJKlBFl2SJEmS1CCLLkmSJElqkEWXJEmSJDXIokuSpAZFRA4wHdzrGCVJzVqu1wFIkjRBfKnL8ltGNApJ0oiLzOx1DJIkjVsRkQCZGb2ORZLUG95eKEnSKBfFQRFxfUQ8EhHPRcT8iLgsIj7cof36EfHfEXFHRCyKiMci4qaI+GKHtltFxI8i4uGIeD4i7omIMyNi3Q5tZ9ZbIjeOiMMi4rd1/1e3tfu7iJgVEY/Wfd4VESdFxKuHNTGSNEZ4pUuSpAYNx5WuiDgO+DfgbuAS4ElgXeDtwB8y8wMtbbcGLgNeA1wL3AisDGwG7JqZk1ra7gn8CAjgPOAeYCvg3cD9wE6ZeXdL+5nAQcDFwDuAnwLzgRcy8wu1zTHAdOCx2u5h4C3A7sBtwPaZ+dQrzYUkjUUWXZIkNaiv6KLzM13zMnPmIPaxAFgEvCkzn21b99rMfLT+ewXgj8CGwEcz8/ttbdfPzHvrv1elFFmvphRj17W0Owo4AbgiM3dvWT6TUnQtVZDV9bsBVwE3AO/NzCda1h0MfBs4NTMPH+iYJWk8seiSJKlBLUVXJ9dk5q6D2McC4CngzZn5fD/t9qVcsbooM/ceYJ8fBb4L/CAzP9K2bjngDkrxtkFm/qkun0kpuqZl5mkd9nk+sA+wRWb+vsP6OcB6mbl2f7FJ0nhj74WSJI2AZexI43vAYcBtEfFD4Brghsx8sq3ddnV+ySD2+bY6v6p9RWYujohrKUXXW4E/tTW5qcs+twf+AnwwIj7YYf0KwFoRMSUzFwwiRkkaFyy6JEka/Q4H5gIfBz5fp8URMQs4IjPvrO36Oqq4bxD7XKPOH+iyvm95p84vHuyyzRTKZ4tjBnjtVQGLLkkThr0XSpI0ymXmC5l5amZuCbwO2Bc4H3gfcGlETK5N+56hWm8Qu+27SrZOl/XrtrVbIqR+9vl4ZsYA0z2DiE+Sxg2LLkmSxpDMfDgzf5yZH6LcGrgJsEVdfWOd7zGIXc2p813bV9Rnut5R/3vzEMK7EVgzIjYfwjaSNO5ZdEmSNIpFxOSI2LHD8uUp3cID9PVo+BNgHvC+iNi/wzbrt/z3Akq37vtHxHZtTacBGwFX9nWiMUin1PlZEfH6Dq+/SofXkqRxz94LJUlq0LKO01UHFH4cuBP4NaWb9xUpY2lNpa2nwjpO1+XAmpQON26s7acC78rM5Vra7g2cS7ld8FxKhxlbUcbUehDYMTPntrSfSem9cKPMnNcl3qOA4yld3M+ijC22KrABsAswOzPf80pyIUljlUWXJEkNGoaia3lKRxq7AZsDawMLgbuAmcDZmfnntm3eSOlsYw/K810LKUXbhZl5XFvbtwNHAztROtd4kDLo8Zcz8/62tjMZoOiq7XYCPlP3+VrKs173UW6H/H5m/mpoWZCksc2iS5IkSZIa5DNdkiRJktQgiy5JkiRJapBFlyRJkiQ1yKJLkiRJkhpk0SVJkiRJDbLokiRJkqQGWXRJkiRJUoMsuiRJkiSpQRZdkiRJktQgiy5JkiRJapBFlyRJkiQ1yKJLkiRJkhpk0SVJkiRJDbLokiRJkqQGWXRJkiRJUoMsuiRJkiSpQRZdkiRJktSg/wdYAS4toLcTKwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "image/png": { + "height": 277, + "width": 430 + } + }, + "output_type": "display_data" + } + ], + "source": [ + "# Plot the feature importances from xgboost\n", + "plot_importance(model)\n", + "plt.gcf().savefig('xgb_feature_importance_v2_downsample.png')" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "3398" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Load the test dataset for prediction\n", + "test_cols = ['ip', 'app', 'device', 'os', 'channel', 'click_time', 'click_id']\n", + "test_df = pd.read_csv('../input/' +\"test.csv\", usecols=test_cols, dtype=dtypes)\n", + "test_df = pd.merge(test_df, ip_groups, on='ip', how='left', sort=False)\n", + "del ip_groups\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
click_idipappdeviceoschannelclick_timeclicks_by_ip
0057449131072017-11-10 04:00:00459
111199019134662017-11-10 04:00:006981
2272287211191282017-11-10 04:00:006018
3378477151131112017-11-10 04:00:003986
44123080121133282017-11-10 04:00:00676
\n", + "
" + ], + "text/plain": [ + " click_id ip ... click_time clicks_by_ip\n", + "0 0 5744 ... 2017-11-10 04:00:00 459\n", + "1 1 119901 ... 2017-11-10 04:00:00 6981\n", + "2 2 72287 ... 2017-11-10 04:00:00 6018\n", + "3 3 78477 ... 2017-11-10 04:00:00 3986\n", + "4 4 123080 ... 2017-11-10 04:00:00 676\n", + "\n", + "[5 rows x 8 columns]" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "test_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "166" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Creating a dataframe for submission\n", + "submission_df = pd.DataFrame()\n", + "submission_df['click_id'] = test_df['click_id'].astype('int')\n", + "\n", + "test_df['clicks_by_ip'] = test_df['clicks_by_ip'].astype('uint16')\n", + "test_df = feature_extraction(test_df)\n", + "test_df.drop(['click_id', 'ip'], axis=1, inplace=True)\n", + "dtest = xgb.DMatrix(test_df)\n", + "del test_df\n", + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
click_id
00
11
22
33
44
\n", + "
" + ], + "text/plain": [ + " click_id\n", + "0 0\n", + "1 1\n", + "2 2\n", + "3 3\n", + "4 4" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "submission_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "11" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "gc.collect()" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [], + "source": [ + "# Get predictions from the best iteration with model.best_ntree_limit.\n", + "submission_df['is_attributed'] = model.predict(dtest, ntree_limit=model.best_ntree_limit)\n", + "submission_df.to_csv('xgb_submission.csv', float_format='%.8f', index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.8" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +}